How to test Java spell checkers and NLP libraries with sample data

Spell checking looks simple until a checker meets “colour”, “defence”, a product code or an Australian street name. The practical side of how to write tests for Java spell checker or NLP libraries with sample data starts by separating token detection, dictionary membership, correction suggestions and language rules. Those behaviours can change independently, so a broad assertion may conceal the actual failure.

A dependable suite combines a small hand-curated corpus, generated edge cases and assertions tied to the chosen engine’s documented contract. It also treats locale, dictionary version, ignore lists and normalisation as configuration. That discipline matters to Java teams across Sydney, Melbourne and Brisbane, whose English-language products commonly serve Australian English, multilingual customers and specialised terminology.

Frame the behaviour under test

Begin by identifying the public capability your application relies on. A tokeniser, dictionary checker and grammar engine need different evidence, and combining them in one assertion creates brittle tests.

Engine or component Primary behaviour Sample data that matters Assertion to prefer
Apache Commons Text JSpellChecker Dictionary membership and candidate suggestions Known words, unknown words and locale-specific terms Accepted or rejected, plus an expected candidate
java-spellchecker Edit-distance correction suggestions Single-token typos and near-valid words Expected correction is included
Apache Lucene analyzers Tokenisation and normalisation Punctuation, possessives, hyphenated text and acronyms Exact token stream or normalised terms
LanguageTool Grammar and spelling rules Complete sentences and rule metadata Expected rule, category or text location

Different engines expose different boundaries. Lucene analyzers are useful for token-stream tests, while JSpellChecker and similar libraries focus on dictionary decisions and nearby candidates. LanguageTool is better suited to complete sentences and rule identifiers. APIs and dictionary support vary by version, so confirm the behaviour in the dependency your service actually uses.

The same separation of concerns appears in broader quality engineering guidance: test the public behaviour you depend on, not every internal ranking detail. That advice prevents a harmless suggestion-order change from becoming a release blocker.

Build a representative sample-data corpus

Store the corpus as versioned test resources rather than large literals in test classes. Separate known words, deliberate misspellings, domain terms, ignore-list entries and sentence-level examples; for example, keep files under src/test/resources/spell/en-AU/. Record the expected result beside each case, but keep it independent from the input file so a typo in fixture preparation does not silently validate itself.

Australian English makes locale selection observable. Include expectations such as organise, realise, and licence as a noun, while retaining licence-related exceptions where the product uses the verb. Place names such as Canberra, Parramatta and Geelong reveal whether proper nouns come from a custom dictionary. Products used by Australian government, education or financial organisations may also contain local names, formal wording and acronyms alongside ordinary prose.

Local users do not write only one kind of English. Test Māori and Torres Strait Islander place names, accented family names and code-switched passages with the appropriate language scope. If the library does not claim multilingual accuracy, assert only the boundaries it supports instead of turning the corpus into an unsupported translation test.

Make assertions deterministic and meaningful

Make each assertion answer one question. A word expected to be accepted should not also imply anything about correction rank. A rejected word should normally have a narrowly defined candidate, and a sentence-level test should identify the exact rule category or offset when the library exposes it.

For example, an isolated JUnit test using Apache Commons Text can load a tiny fixture dictionary and verify both acceptance and suggestion membership:

@Test
void detectsTypoAndSuggestsExpectedWord() {
    Set<String> dictionary = Set.of(
        "organisation", "realise", "licence", "spelling");
    SpellChecker checker = new SpellCheckerBuilder()
        .setDictionary(dictionary)
        .build();

    assertTrue(checker.spell("organisation"));
    assertFalse(checker.spell("speling"));
    assertThat(checker.suggest("speling")).contains("spelling");
}

The custom dictionary keeps this unit test independent of the machine’s locale and the library’s bundled word list. A separate integration test can exercise the stock en-AU dictionary where the dependency supports it. Pin the locale, library version and fixture version in the build environment; never let Locale.getDefault() decide which English variant CI tests.

Exercise Unicode and tokenisation boundaries

Unicode tests should compare equivalent representations, not merely different-looking strings. A composed café and decomposed café with a combining acute accent can produce different tokens if normalisation is absent. Include examples with é, ø, Greek characters and Devanagari text, then assert the library’s documented normalisation result.

Token boundaries create another class of defects. Test possessive forms such as team’s, hyphenated terms such as well-known, abbreviations including NSW, URLs, email addresses, decimal values and product codes. A sentence-oriented library may split differently from a word checker, so derive expected tokens from the specific analyser rather than assuming whitespace equals a word.

Context should also be isolated. Include a misspelling in the middle of a normal clause, near a full stop and beside punctuation-heavy content. For multilingual input, verify that unsupported language is ignored or flagged according to policy rather than silently producing corrections from an English dictionary.

Separate spelling errors from acceptable suggestions

Suggestions need expectations proportional to their stability. Assert that an expected correction is present instead of snapshotting the entire list, because distance ordering, frequency data and dictionary versions can alter it. If your product promises a first suggestion, test that ranking separately and document the engine configuration that makes the promise meaningful.

Keep domain vocabulary in explicit fixtures. Terms such as iOS, JIT, SaaS, ABN, Melbourne or an internal product name should be accepted because the application chose to add them, not because every checker happens to know them. Test the custom dictionary and ignore list independently so failures identify the responsible configuration.

Useful metamorphic checks include preserving acceptance when only letter case changes and preserving a valid token when surrounding punctuation changes, provided the tokenizer’s contract permits those equivalences. For a deliberate single-character typo, require a correction within a documented edit distance. Avoid applying metamorphic rules blindly across libraries with different tokenisation semantics.

Automate corpus checks and property tests

Move repetitive cases into JUnit 5 parameterised tests, and use a property-based library such as jqwik for generated inputs. The curated suite should stay fast enough for every commit; a larger randomised corpus can run on a schedule or in a dedicated continuous-delivery job.

Useful fixture patterns are:

Generated inputs work best when they check invariants rather than guessed exact outputs. These properties are useful:

Run both layers in Maven or Gradle with en-AU, UTC and a pinned dependency graph. Archive the corpus revision, dictionary version and generated seed with CI artefacts. When a generated case reveals a genuine defect, promote a reduced version into the curated corpus; keep random failures as reproducible evidence rather than allowing a flaky suite to train the team to ignore red builds.

Ship the suite as a quality gate

A useful spelling-quality suite makes each failure explainable. It should reveal whether the regression came from normalisation, tokenisation, a missing dictionary term, an altered suggestion ranking or a language rule. Separate unit fixtures from stock-dictionary integration tests, and reserve exact snapshots for output that your own product contract controls.

For continuous delivery, the fast suite should block a pull request, while broader corpus runs reveal low-frequency issues before release. The practical takeaway is to commit a small representative corpus, pin the locale and dictionary versions, assert stable public behaviour rather than incidental rankings, and turn every credible generated failure into a reproducible curated case.