Benchmarking Java regular expressions with Australian inputs

Regular expressions often appear small enough to treat as free CPU work. In Java, however, a concise pattern can trigger repeated backtracking, while a larger pattern can add allocation and scanning overhead. A result that completes in microseconds on a clean fixture says little about a service handling long form submissions, mixed valid and invalid data, or bursts of concurrent traffic.

Performance testing Java regular expressions with input patterns should therefore compare both the expression and the data reaching it. The useful benchmark uses representative success cases, near misses, hostile strings, and realistic payload sizes. It also separates one-off compilation from matching cost, records enough evidence to expose regressions, and verifies that any faster expression still accepts and rejects exactly the same inputs.

Why regex cost depends on input shape

Java compiles a java.util.regex.Pattern into an immutable expression tree, while each Matcher holds mutable matching state. Compiling once and creating a matcher per operation is normally the right production arrangement. A matcher can be reset and reused, but it is not thread-safe, so sharing one across request threads can corrupt state or produce misleading benchmark results.

The dominant risk is pathological backtracking. A pattern such as (a+)+$ appears harmless until it receives aaaa...!; nested quantifiers give the engine multiple ways to partition the same text, and the failed final character causes exponential exploration. Overlapping alternatives, repeated optional groups, and several lookaheads can create similar effects without looking obviously unsafe.

Input shape changes the cost even when the grammar does not. matches() requires the entire input region to match, whereas find() scans for the next occurrence. A valid Australian mobile number may stop early, while rejecting an almost-correct value can force the engine through every branch. Anchors, case-insensitive matching, Unicode character classes, and line-oriented data can also alter the amount of work. Benchmarks must preserve those semantics rather than replacing them with easier strings.

Designing Australian representative test data

Australian test data adds useful variation without requiring production records. Use synthetic fixtures containing Sydney, Melbourne, Brisbane, Adelaide, and Perth for city fields, alongside relevant suburbs and Australian English labels. These values should appear in ordinary search, billing, and delivery workflows so the corpus exercises successful extraction as well as validation.

Local formats matter as much as city names. Mobile numbers commonly appear as 04xx xxx xxx nationally or +61 4xx xxx xxx internationally; a pattern may need to accept only one presentation. A baseline expression could start with (?:\\+61|0)4\\d{2}[ -]?\\d{3}[ -]?\\d{3}, then be tested against formatting variations accepted by the application. Dates are usually written as DD/MM/YYYY, unlike the month-first convention common in the United States.

Dates should include leap days, invalid month lengths, and daylight-saving transitions used in Sydney and Melbourne. Brisbane’s fixed offset provides a useful contrast when calculations cross local midnight. A shape-only date expression can confirm the separators and digit counts, but actual calendar validity still needs date parsing or semantic validation.

Addresses need more than four arbitrary digits. Australian postcodes are four digits and are associated with states or territories, so pair examples such as ACT 2600, NSW 2000, VIC 3000, and QLD 4000 with matching locality fields. Vary postcode formatting only if the application actually accepts it. This catches permissive expressions that accept a plausible shape but attach it to the wrong region.

The local market also supplies business-specific fields. Prices may be denominated in AUD, GST is normally 10%, and an Australian Business Number has an 11-digit format with checksum rules. Local domains commonly end in .com.au, while checkout forms may request a company name, delivery address, and state. Generate synthetic ABNs and test data rather than using customer records, and separate these market conventions from personal identifiers.

Building a repeatable JMH benchmark

JMH is the best default for comparing Java expression variants because it controls warm-up, forks the JVM, and prevents dead-code elimination. Keep Pattern instances outside the timed method unless compilation itself is under test. Put file loading and corpus selection in setup code, and use realistic values large enough to exceed timing noise.

@BenchmarkMode(Mode.AverageTime)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 5, time = 1)
@Measurement(iterations = 10, time = 1)
@Fork(2)
@State(Scope.Benchmark)
public class RegexBenchmark {
    private static final Pattern REFERENCE =
            Pattern.compile("\\A[A-Z]{2}\\d{6}\\z");

    @Param({"ORD123456", "ORD12345X", "ORD123456789"})
    public String reference;

    @Benchmark
    public boolean matchesReference() {
        return REFERENCE.matcher(reference).matches();
    }
}

This small benchmark deliberately covers a valid value, a near miss, and an overlong value. A fuller harness can read named corpora from files during setup and expose the corpus name as a parameter. Run each expression against the same corpus, keep JVM flags identical, and repeat the run on the CI platform because JIT behaviour and available processors can change the ranking.

Do not average unrelated scenarios into one number. Report each input class separately, then select a weighted workload only if production traffic provides defensible weights. Include a genuinely adversarial but bounded input to detect exponential growth, yet keep the benchmark deterministic. A random generator without a fixed seed makes regressions difficult to compare across builds.

Measuring latency throughput and allocation

Latency is the first signal. JMH’s average-time mode is useful for throughput-oriented comparisons, while sample-time mode provides a distribution that can be inspected for outliers. Service-level objectives usually concern a percentile, so use a profiler or load tool to measure p95 and p99 at the HTTP boundary rather than assuming a nanosecond-level regex benchmark predicts them.

Throughput requires a second view. Run the matcher through the real allocation path and test concurrency with representative payload sizes. Pattern is safe to share, but each concurrent operation still needs its own Matcher; contention on a shared matcher is a correctness bug, not a performance tuning technique. JMH thread annotations can expose scaling limits, while an integration test should reveal contention in caches and request handling.

Allocation can dominate short matches. A profiler such as JMH’s GC profiler, Java Flight Recorder, or a sampling CPU profiler can show whether matcher creation, character buffers, or captured groups drive cost. Returned match objects and group extraction also matter when the expression is only part of a larger parsing pipeline.

Compare the optimised pattern with the original as a paired experiment. Record operations per second, latency distribution, allocation per operation, and CPU time. A change that improves a synthetic valid case but doubles p99 under mixed traffic has not improved the service. Keep the raw JMH JSON, profiler output, corpus version, and JVM version with the results.

A practical optimisation checklist

Apply these checks in order so that optimisation does not obscure a faulty benchmark or a semantic change:

Each recommendation protects a different part of the result: relevance, isolation, semantics, safety, and repeatability. The list is deliberately operational; a clever pattern is not useful if its behaviour cannot be demonstrated across realistic inputs.

Validating behaviour before deployment

Optimisation is only complete when the expression remains correct in context. Keep behaviour-based tests that feed the same valid and invalid records to the old and new implementations, including records captured from synthetic boundary cases. For date, amount, phone, and postcode fields, prefer parsers that validate real calendars and ranges when a regular expression alone would accept impossible values.

Run the final candidate through the application’s real path: request decoding, validation, error mapping, logging, and response serialisation. Generate a controlled load burst and watch p95, p99, CPU, heap allocation, and error rates. This catches issues that an isolated regex benchmark cannot, such as a retained matcher, duplicate validation, or logging that magnifies rejected input.

Make the evidence reproducible by storing the corpus generator, benchmark source, baseline JSON, and accepted latency budget in the repository. Wire the benchmark into continuous integration as a non-blocking report first; once its variance is understood, promote the agreed threshold to a blocking check for material regressions.

The concrete next step is to profile the slowest production validation pattern, add one JMH benchmark with valid and near-miss Australian fixtures, and record a baseline before changing the expression.