test_detective

Testing Java annotation processing with compile-time tests

Java annotation processors run inside javac, before the JVM starts. They inspect Element and TypeElement instances, react to supported annotations, and may ask the Filer to create source files or resources. A defect can therefore make a build fail, emit the wrong diagnostic, or silently omit generated code even when every ordinary unit test passes.

Compile-time testing exercises that path directly: it compiles small synthetic programs, captures compiler diagnostics, and inspects generated output. The goal is not to mirror a whole application, but to prove that the processor recognises an annotation, reacts to valid and invalid inputs, and produces the artefacts expected by downstream code. Test Detective’s editorial policy explains how its technical examples are selected and reviewed.

Why processor behaviour needs a dedicated test strategy

Because most tests start only after javac has finished, they can miss processor-specific faults entirely. A runtime test might prove that a generated DTO retains the correct value, yet say nothing about why the source was generated, whether a duplicate class was suppressed, or whether the compiler reported an actionable error.

Concern Runtime-oriented test Compile-time test
When behaviour occurs After production code compiles Within the javac invocation
Primary assertion Values and runtime behaviour Compiler outcome, diagnostics and generated files
Defects exposed Retention and runtime integration Annotation discovery, processing rounds and Filer behaviour
Main limitation Cannot directly observe source generation Requires controlled compiler inputs and fixtures

A compile-time test supplies a source file, invokes the processor and checks the compiler result. It can prove that an annotation on an invalid type produces a diagnostic with a stable code, rather than relying on a human to interpret an unexpected failure from a Maven or Gradle build.

Keep each test focused on one processor responsibility. Separate tests can cover annotation discovery, validation, source generation, resource generation and interaction with other processors. A full application build can then confirm that the generated classes integrate correctly, but it should complement rather than replace the faster compiler-level checks.

Designing a reliable compilation test harness

Use either the JDK compiler API or a library such as Google Compile Testing. The JDK route provides direct access to JavaCompiler, DiagnosticCollector and StandardJavaFileManager; the library route supplies concise matchers for successful compilation, diagnostics and generated files. Either approach should register the processor explicitly, use a fresh output directory and avoid accidentally discovering another processor through META-INF/services.

Google Compile Testing keeps most fixtures in memory and allows assertions against the compilation result:

Compilation compilation = Compiler.javac()
    .withProcessors(new FixtureProcessor())
    .compile(JavaFileObjects.forSourceString(
        "sample.Order", "@sample.Generated class Order {}"));

assertThat(compilation).succeeded();
assertThat(compilation)
    .generatedSourceFile("sample.OrderFactory")
    .contentsAsUtf8String()
    .contains("class OrderFactory");

The harness should specify the source and target release rather than inheriting an IDE default. It should also use deterministic encoding, locale and time zone settings, while ensuring that generated files never depend on the developer’s home directory. Temporary output must be cleaned between cases so one compilation cannot influence the next.

Fixtures should be minimal but realistic. Use separate sources for a valid annotation, an invalid annotation value, an unsupported target and a type that triggers multiple processing rounds. Names such as Order, Customer and Invoice are easier to diagnose than generic strings, but test data must remain synthetic and must not resemble real customer records.

Asserting diagnostics and generated artefacts

Diagnostic assertions should verify the compiler error code, diagnostic kind and a stable fragment of the message. Checking the entire text can make a test fail after a JDK update changes punctuation or wording. Counting diagnostics is equally important: an invalid input that produces one useful error and several incidental warnings may still be unsuitable for automated builds.

Generated Java deserves structural assertions rather than a blind comparison with a large expected string. Check the package, visibility, constructors, annotations and important method signatures. Snapshot comparisons work well when output is deterministic, while targeted assertions are better when the processor legitimately changes formatting. Generated resources such as schemas or metadata need separate checks for location, encoding and content.

Processor edge cases deserve dedicated fixtures. Cover annotations on classes, records, fields and methods; annotations that refer to types from the classpath; and generated sources that contain further annotations requiring another round. Also test duplicate generation and malformed values, because incorrect Filer usage often appears as a FilerException rather than a useful domain diagnostic.

Fitting compile-time checks into Australian delivery pipelines

Australian teams often combine Gradle or Maven with hosted Git repositories and issue tracking. Jira and Bitbucket, both built by Sydney-based Atlassian, have a strong local footprint, so a compile-time test task should appear in the same pipeline status as ordinary unit tests. Keep the compiler fixtures in the test source set and run the processor explicitly instead of relying on IDE annotation-processing settings.

Teams may span Sydney, Melbourne, Brisbane and Perth. Daylight-saving changes affect Australia’s eastern states, while Perth remains on UTC+8, so scheduled verification jobs should not assume one nationwide clock. A common Australian office rhythm is a brief stand-up and coffee break, but pipeline reliability must come from an automated gate rather than someone noticing a failed compilation later.

Keep test sources synthetic wherever possible. If a fixture contains personal information, the Privacy Act 1988 and Australian Privacy Principles may apply, while OAIC de-identification guidance provides a safer approach to test-data handling. In consumer-facing systems, a successful processor run also says nothing about whether generated fee calculations, terms or disclosures comply with the Australian Consumer Law.

Pin the JDK version used for compilation tests, then run a smaller matrix against supported Java releases. In Gradle, separate a fast processor-test task from slower build integration tests; in Maven, use a dedicated test execution with controlled compiler arguments. Caching is safe for deterministic fixtures, but changes to the processor, compiler version or annotation definitions should invalidate the relevant results.

A practical path to durable compile-time coverage

A useful adoption sequence starts with the failure modes that ordinary tests cannot observe, then expands towards build integration:

The next build step is to add one focused test that compiles an invalid annotated type and asserts its diagnostic code and message fragment, then extend the same fixture to verify the expected generated file.