Compare Backends

Warning: This is an alpha feature. APIs and behaviors may change; use in production with care.

The compare backends feature runs the same input through multiple CLTK NLP backends and produces a structured diff report. Use it to spot disagreements in tokenization, lemma, UPOS, features, head, and dependency relations.

Python API

from cltk.evaluation.compare_backends import compare_backends, report_to_markdown

report = compare_backends(
    "lati1261",
    "Gallia est omnis divisa in partes tres.",
    ["stanza", "openai"],
)
print(report_to_markdown(report))

CLI

python -m cltk.compare_backends \
  --language lati1261 \
  --text "Gallia est omnis divisa in partes tres." \
  --backends stanza,openai \
  --out report.md

To write JSON, Markdown, and CSV outputs in a directory:

python -m cltk.compare_backends \
  --language lati1261 \
  --text "Gallia est omnis divisa in partes tres." \
  --backends stanza,openai \
  --out-dir ./reports