Testing Custom Spring Boot Auto-Configuration and Starters

A Spring Boot starter can make a library feel effortless: add one dependency, set a property, and the application receives correctly configured beans. That convenience depends on conditional configuration behaving precisely. A bean created too early, a property ignored, or a user-defined component accidentally replaced can turn a small starter into a difficult production incident.

Testing custom Spring Boot auto-configuration requires more than starting one sample application. A reliable test suite checks when configuration activates, when it stays dormant, how it responds to application properties, and whether it backs away when the consuming application provides its own implementation. The goal is to test the contract between the starter and its users.

Separate The Starter From Auto-Configuration

A modern starter commonly consists of at least two modules. The starter module provides a convenient dependency for application developers, while the auto-configuration module contains configuration classes, conditional annotations, metadata, and implementation dependencies. Keeping those responsibilities separate makes dependency management clearer and lets the auto-configuration be tested directly.

For Spring Boot 3, an auto-configuration class is usually annotated with @AutoConfiguration and registered in META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports. Spring Boot 2 projects generally use @Configuration with spring.factories. Tests need to reflect the version supported by the library; a migration can otherwise leave the application working locally while the packaged JAR is never discovered.

A minimal configuration might look like this:

@AutoConfiguration
@EnableConfigurationProperties(ClientProperties.class)
@ConditionalOnClass(RemoteClient.class)
@ConditionalOnProperty(
    prefix = "example.client",
    name = "enabled",
    havingValue = "true",
    matchIfMissing = true
)
public class ClientAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    RemoteClient remoteClient(ClientProperties properties) {
        return new RemoteClient(properties.getBaseUrl());
    }
}

The important behaviour is expressed through conditions. The test suite should verify each meaningful condition rather than merely checking that a bean exists in the happy path.

Use ApplicationContextRunner For Focused Tests

ApplicationContextRunner is usually the best starting point for auto-configuration tests. It creates a small application context, applies selected auto-configurations, and closes the context after each test. This is quicker and more diagnostic than launching the complete application with @SpringBootTest.

A basic test can be written with JUnit 5 and AssertJ:

private final ApplicationContextRunner contextRunner =
    new ApplicationContextRunner()
        .withConfiguration(
            AutoConfigurations.of(ClientAutoConfiguration.class)
        );

@Test
void createsClientWhenEnabled() {
    contextRunner
        .withPropertyValues(
            "example.client.enabled=true",
            "example.client.base-url=https://api.example"
        )
        .run(context -> {
            assertThat(context).hasSingleBean(RemoteClient.class);
            assertThat(context.getBean(RemoteClient.class).getBaseUrl())
                .isEqualTo("https://api.example");
        });
}

Keep the runner field close to the tests and customise it per scenario. withPropertyValues is useful for configuration properties, while withBean, withUserConfiguration, and withClassLoader allow tests to model application code and classpath differences.

ApplicationContextRunner also exposes startup failures clearly. For an expected failure, use runFailure or inspect the thrown exception and its cause. This is particularly valuable for invalid URLs, missing credentials, incompatible properties, and deliberately strict configuration rules.

Cover Activation And Back-Off Behaviour

Every condition deserves a test for activation and non-activation. If the configuration is guarded by @ConditionalOnClass, test the context with the required library present and absent. A FilteredClassLoader can simulate a missing dependency without modifying the test classpath:

@Test
void doesNotActivateWithoutClientLibrary() {
    contextRunner
        .withClassLoader(new FilteredClassLoader(RemoteClient.class))
        .run(context ->
            assertThat(context).doesNotHaveBean(RemoteClient.class)
        );
}

For @ConditionalOnProperty, cover the enabled value, disabled value, missing property, and invalid value where those cases have distinct meanings. Be explicit about matchIfMissing; an accidental default can activate a network client in an application that never intended to use it.

Back-off is just as important. A starter should normally respect a bean supplied by the consuming application:

@Test
void backsOffWhenApplicationProvidesClient() {
    RemoteClient customClient = new RemoteClient("http://localhost:9090");

    contextRunner
        .withBean(RemoteClient.class, () -> customClient)
        .run(context -> {
            assertThat(context).hasSingleBean(RemoteClient.class);
            assertThat(context.getBean(RemoteClient.class))
                .isSameAs(customClient);
        });
}

Also test multiple beans if your condition uses @ConditionalOnSingleCandidate, and test bean names when @ConditionalOnMissingBean(name = "...") is involved. Conditions that appear obvious often change meaning after another auto-configuration contributes a bean.

Scenario	Useful Fixture	Main Assertion
Required class exists	Normal test classpath	Auto-configuration is applied
Required class is absent	FilteredClassLoader	Configuration backs off
Feature is disabled	withPropertyValues	No managed bean is created
User supplies an implementation	withBean or test configuration	Starter keeps the user bean
Invalid settings are supplied	Bad property values	Context fails with a useful cause
Full application wiring is needed	@SpringBootTest	End-to-end behaviour works

Verify Properties And Defaults

@ConfigurationProperties tests should check binding, defaults, relaxed naming, and validation. A property such as example.client.connect-timeout=2s must result in the expected Duration, while an omitted property should produce the documented default. Testing the Java field alone does not prove that Boot’s binder reaches it correctly.

When properties are validated with @Validated, include invalid values such as a blank endpoint or a negative timeout. The test should assert that the context fails for the right reason, rather than accepting a generic startup exception. Clear failure messages matter when a library is used by teams that did not write its internals.

Use a dedicated properties test when binding rules are complicated, but also test through the auto-configuration context. The latter catches mistakes such as forgetting @EnableConfigurationProperties, using the wrong prefix, or injecting a different properties class than the one being bound.

Configuration defaults should be treated as compatibility commitments. If a starter defaults to enabled, verify that choice. If a later release changes it to disabled, record that as an intentional behavioural change and test both migration paths where necessary.

Add Integration Tests For The Packaged Starter

Focused context tests prove conditional wiring, but they do not always prove that the published artifact is usable. Add a small integration fixture that consumes the starter as a real application would. It can use @SpringBootTest, @ImportAutoConfiguration, or a sample application module, depending on the behaviour under test.

A useful integration test exercises the complete route from dependency resolution to application startup. It can verify an HTTP endpoint, a message listener, a scheduled task, or a client making a request to a controlled server. WireMock, MockWebServer, or a local test server is preferable to a live external service.

For a REST client starter, combine context assertions with an actual request against a disposable server. Confirm headers, timeouts, error mapping, and retry settings. For asynchronous components, use Awaitility with bounded time limits and verify that an event is processed exactly as intended. Avoid arbitrary sleeps, which make CI failures slow and difficult to diagnose.

Packaging tests are especially useful for libraries published through Maven Central or an internal Nexus or Artifactory repository. Australian teams often have separate repositories for public dependencies and company-approved artifacts, so a source-set test can pass while the generated JAR omits AutoConfiguration.imports or declares an incorrect dependency. Inspect the built JAR in CI and run the sample application against the packaged version.

Make Compatibility And Delivery Observable

Spring Boot auto-configuration is sensitive to framework versions, Java versions, and optional dependencies. A library supporting Boot 2 and Boot 3 may need separate source sets or release lines because registration files and annotations differ. A CI matrix should test the combinations that the project promises, rather than relying on one local version.

Test condition reports when diagnosis is important. ConditionEvaluationReport can show why an auto-configuration matched or backed off. This is more useful than asserting only that the final bean count is zero, particularly when several conditions can prevent activation.

Use @ImportAutoConfiguration to limit integration tests to the configuration under review. This prevents unrelated Boot auto-configurations from hiding missing imports or creating accidental dependencies. Conversely, maintain one full application test to detect conflicts with common infrastructure such as Jackson, validation, web MVC, security, or actuator.

Runtime diagnostics can help library users too. A starter that logs its effective configuration without exposing secrets gives support teams better evidence when an application behaves differently in Sydney, Melbourne, or Perth deployments. Tests should ensure credentials and tokens never appear in logs. Teams operating across AEST and AEDT should also verify timeout and scheduling properties with explicit units and zones; a test that assumes the developer’s local timezone can pass in winter and fail after daylight saving changes.

A practical quality gate includes unit tests for the implementation, ApplicationContextRunner tests for conditions, packaged-artifact tests, and a small number of end-to-end scenarios. The suite should run quickly enough for every pull request, with slower Testcontainers or compatibility jobs separated where appropriate.

A custom Spring Boot starter is a public extension point, even when maintained inside one organisation. Test its activation rules, property binding, defaults, failure modes, user-bean back-off, packaging metadata, and behaviour under supported Boot versions. The key thing to remember is that a good starter test suite proves when the configuration should act, when it must stay out of the way, and whether the resulting application behaves safely in the environments that consume it.