Skip to content
PtahDocs
latest

Graphic preview

100%
Page type: status

Conformance

Where Atlas compatibility evidence lives and how to interpret it.

Atlas compatibility evidence is maintained in stokaro/ptah-atlas-conformance.

The conformance repository keeps Atlas Apache-2.0 fixtures outside Ptah’s MIT source tree and imports Ptah as the system under test:

ptah-atlas-conformance -> ptah
ptah !-> ptah-atlas-conformance

The authoritative current numbers live in the conformance repository reports:

Those reports are the live source for the observation counts, so this page names them rather than restating a number that changes without it. gaps-live.md carries the live database round-trip corpus and gaps-diff.md the Atlas CE differential corpus.

Green conformance reports mean that the current measured corpus has no red results. They do not, by themselves, prove every Atlas OSS command, flag, dialect feature, and output mode. Use the Feature matrix for product and coverage gaps that are outside the current measured corpus; each row that is partial names the issue that owns it.

A green documentation build proves that the documentation site builds and that its internal links resolve. It is not parity evidence.

Atlas CE comparisons run the compatibility binary with PTAH_ATLAS_STRICT_COMPAT=1. That opt-in policy constructs the CE command and flag tree and refuses extension inputs or inspected live objects before output or mutation. The normal ptah-compat surface does not set it and retains implemented Atlas Pro-like and best-effort capabilities.

Strict inspection removes PostgreSQL’s server-installed plpgsql extension and baseline PUBLIC USAGE grant from the snapshot it renders. Full mode keeps the original reader snapshot. Strict cleanup executes the validated and confirmed plan itself. On PostgreSQL it locks every planned table, repeats the strict inventory through the transaction session, compares the rebuilt cleanup plan with the confirmed plan, and refuses catalog drift before the first drop. A trigger, policy, view, or foreign key created while the prompt is open cannot disappear with its table.

Strict schema workflows refuse YAML sources and an authored schema apply lint policy that the CE execution path cannot enforce. Commands that execute, convert, or replay migration bodies refuse Atlas txtar, every Ptah directive — including malformed or bare -- +ptah markers — and SQL templates; checksum-only reads preserve those bytes. Desired migration directories are captured and validated before a target or dev database is opened or a migration lock is acquired, and replay uses that same stable snapshot. Default mode keeps the extensions.

After a target connection opens, strict schema apply inventories an explicit --schema scope before acquiring the apply lock or replaying a desired migration directory. Without that explicit scope, a PostgreSQL-family target inventories the user realm because desired replay may name a schema beyond the URL’s search_path. The locked planning phase validates it again before producing or executing a plan. Default mode performs no supplemental strict-policy inventory.

Strict process startup also rejects both Atlas-facing PTAH_* flag bindings and native aliases consumed after forwarding, such as PTAH_MIGRATIONS_DIR. Ordinary environment variables used by getenv in atlas.hcl remain available.

A required companion change in the separate conformance harness (stokaro/ptah-atlas-conformance#277) must keep the two environments separate: CE parity probes inject strict mode into each subprocess, while Pro-retention and native Ptah probes leave it absent. Until that change lands, invoke CE probes with per-process injection rather than enabling strict mode for the whole harness. Strict mode still keeps deliberate safety and correctness improvements, so a green result never depends on copying a CE behavior that silently drops authored content or corrupts migration state.

ptah-compat schema inspect --format '{{ sql . }}' writes the plan that creates the inspected schema on a new database, as the pinned Atlas CE v1.3.0 binary does, measured on PostgreSQL 18 and SQLite. Each statement is on one line under the comment that names its change. A foreign key sits inside the CREATE TABLE of its table, and that table comes after the one it references. An indent argument lays each statement out, and an empty database writes 0 bytes.

The bytes are those ptah-compat schema diff writes from a new database to the inspected one, so both commands differ from the community binary in the same places. A SQLite identifier is quoted with " where that binary writes a backtick. A PostgreSQL text default keeps its cast, 'x'::text. A read of the whole database writes no CREATE SCHEMA IF NOT EXISTS "public" and no comment for the schema a new database already holds. The plan also keeps what that binary leaves out of its own: extensions, standalone sequences, row-level security policies and SQLite virtual tables.

A read scoped to some schemas, by --schema or by a URL that names one, plans what those schemas hold. What belongs to the database rather than to a schema is left out: an extension installed in a schema outside the scope, and a grant on the database itself. One note on standard error names each of them. So --schema app on a database with pgcrypto in public writes no CREATE EXTENSION, as the community binary writes none, and the note names pgcrypto. An extension installed in a scoped schema is created with it.

The selectors and a whole read treat these objects as follows:

  • A read of the whole database plans every grant on the database and every extension the comparison does not ignore (it ignores plpgsql).
  • --include leaves out a grant on the database, as --schema does. It keeps every schema the read holds, so an extension installed in one of them is planned.
  • --exclude removes what it names, an extension installed in an excluded schema with it, and leaves the rest of the read whole.

Native ptah schema inspect --format sql writes the rendered schema instead.

The conformance repository separates regression budgets from full parity:

Gate type Meaning
Regression budget No new gaps beyond the accepted budget for that contour. Should stay green.
Full conformance Every checked case in that contour passes. May stay red while the measured corpus still has non-OK results.

A green regression-budget check does not mean Ptah has full Atlas OSS parity. A red full-conformance check is expected while the report still lists measured non-OK results.

Even when both regression-budget and full-conformance checks are green, the claim is limited to the corpus represented by the generated reports. Expanding live and differential coverage is tracked in stokaro/ptah-atlas-conformance#167.

Findings 3.1 and 3.2 in stokaro/ptah#1235 were measured against the pinned Atlas CE v1.3.0 binary on August 11, 2026. Both tools exit 0 and write the same migration and atlas.sum artifacts; only their process output differed.

Directory layout Pinned binary Ptah before Ptah now
Atlas Stdout and stderr are byte-empty Stdout names the migration by absolute path Both streams are byte-empty
Converted Stdout and stderr are byte-empty Stdout names one or two migrations by absolute path Both streams are byte-empty

The change is limited to the ptah-compat migrate new adapter. Migration names, file contents, atlas.sum, editor execution, warnings, and failure diagnostics remain unchanged. Native ptah migrations create still reports the paths it creates.

Re-run the focused evidence from the Ptah repository:

Terminal window
go test ./internal/cli/atlas -run '^TestCompatMigrateNew' -count=1
go test ./cmd/ptah-compat -run '^TestCompatBinaryMigrateNew' -count=1
go test ./cmd/ptah -run '^TestPtahNativeMigrationsCreateKeepsSuccessReport$' -count=1

Each workflow below states the native Ptah command, the Atlas-compatible surface, what Atlas CE does, and the evidence. This table is the index; the sections carry the detail.

Workflow Native Ptah Atlas CE
Declarative migration and schema tests ptah migrations test, ptah schema test Cannot run either command
Migration directory maintenance ptah migrations edit, rebase, rm Cannot run any of the three
Verified and reported rollback ptah migrations down --shadow-db migrate down is absent
Pre-approved direct schema plans Same engine as schema apply schema plan aborts

This is a workflow-parity record, not a claim of full Atlas Pro compatibility. For the code-by-code status of the analyzer checks Atlas marks as Pro, see Lint rules. Four verbs sit outside the pin entirely and are triaged rather than measured here; see Verbs beyond the CE pin.

Native Ptah. ptah migrations test and ptah schema test

Atlas-compatible Ptah surface. ptah-compat migrate test and ptah-compat schema test forward to the native runners with Atlas-shaped flags and exit codes. Schema-file variables come from an explicit --var or from the selected data.hcl_schema source’s isolated vars scope.

Atlas CE. Cannot run either command; the framework is outside the open-source core

Evidence. Unit coverage includes explicit and project-scoped HCL variables through the Atlas-compatible forward. Integration-tagged PostgreSQL tests cover the live runners. This workflow is not counted as a schema-object fixture.

Native Ptah. ptah migrations edit, rebase, and rm

Atlas-compatible Ptah surface. ptah-compat migrate edit, rebase, and rm forward to the native commands with Atlas-shaped flags and {name | version} positionals; the --edit flags on migrate new, migrate diff, and schema apply open the operator’s editor

Atlas CE. Cannot run any of the three verbs; they abort with the community-version boundary

Evidence. Unit coverage with hermetic editor scripts, including ptah migrations validate passing on the mutated directory; not counted as a schema-object fixture

Native Ptah. ptah migrations down --shadow-db replays the rollback plan on a disposable shadow database before the target is touched

Atlas-compatible Ptah surface. ptah-compat migrate down --dev-url maps to the shadow verification, and --format renders an Atlas Go-template down report (.Env, .Planned, .Reverted, .Current, .Target, .Total, .Error); real rollbacks never read stdin, matching Atlas, while native ptah migrations down keeps its prompt; the forward reads the directory in the layout migrate apply reads and refuses --dir-format; it defaults to the Atlas revision-table layout (--revision-format atlas) but retains Ptah’s recoverable failed-down bookkeeping, with the native --revision-format ptah pass-through as the layout escape hatch; --to-tag resolves against the tags ptah migrations tag records in the database, --skip-checks bypasses the pre-migration checks the down bodies carry, and --plan derives the rollback from the schema difference instead of running those bodies.

Atlas CE. migrate down does not exist in the community binary; the CE notice lists down migrations among excluded features

Evidence. Unit coverage over live SQLite: verification success and pre-target abort on both paths, report rendering including partial-failure reports, waiver rejections, non-interactive execution with EOF stdin, rejection of the non-Atlas --confirm flag, byte-identical execution output against a pre-approved native run, and revision-format regressions proving a bare ptah-compat migrate down reverts revisions written by ptah-compat migrate apply. A subprocess test runs the built ptah-compat binary with EOF stdin and checks the SQLite end state

Native Ptah. Ptah plans and applies direct schema changes through the same engine that powers schema apply

Atlas-compatible Ptah surface. ptah-compat schema plan saves the computed plan in the Atlas .plan.hcl format by default (a .json output path keeps the native fingerprinted JSON plan); ptah-compat schema apply --plan file://<path> reads both formats, Atlas-authored files included — JSON plans execute only after the live database matches the plan’s source fingerprint, Atlas-format plans require --to and are verified by dev-database replay plus the always-on post-apply end-state check; registry planning flags are recorded waivers; schema plan new, schema plan validate and schema plan lint are implemented — the last verifies the plan file and then reports what Ptah’s migration lint rules find in its SQL, without letting findings decide the exit code unless PTAH_ATLAS_PLAN_LINT_FAIL_ON_ERROR=1 asks it to — while approve, list, pull, push, rm and test stay boundary stubs

Atlas CE. schema plan aborts with the community-version boundary; the plan/approval flow is bound to the Atlas Pro registry

Evidence. Unit coverage over live SQLite: plan computation and save, plan execution with schema assertions, stale-plan refusal after target drift, dry-run, dialect mismatch, malformed documents, and waiver rejections. Validation consumes a versioned Atlas-authored plan bundle independently of Ptah’s writer and rejects source, desired-schema, SQL, statement-set, HCL, and malformed-hash mutations without changing target schema or rows. The bundle records known capture provenance and artifact hashes; destructive dev-database guards cover percent-encoded/path/query-option/symlink/hard-link aliases, driver endpoint/database overrides, and fail-closed comparison errors. This workflow is not counted as a schema-object fixture

atlas migrate ls, atlas migrate show, atlas schema stats, and atlas schema validate appear in current Atlas documentation but are not registered in the pinned conformance Atlas CE binary. Each prints the help of its command group and exits 0, which is what that binary does for any subcommand it does not register, rather than a community-version abort stub. So they are outside the CLI-surface parity target. Being outside that target is not a reason to leave a capability unbuilt: all four are Ptah verbs, reachable through the compatibility spelling and omitted only under PTAH_ATLAS_STRICT_COMPAT=1, which is the profile that mirrors the pin.

Verb Where it is answered
migrate ls ptah-compat migrate ls, forwarding to native ptah migrations ls. Not ptah migrations status: status needs --db-url and reports applied and pending state, while listing a directory is a local question with no database in it.
migrate show ptah-compat migrate show, forwarding to native ptah migrations show, which prints a stored migration’s SQL.
schema validate ptah-compat schema validate and native ptah schema validate report every structural problem in a desired state without a database.
schema stats ptah-compat schema stats inspect and native ptah schema stats, which count the objects in a live schema and emit them as OpenMetrics.

Revisit the classification when a pinned build registers one of them. stokaro/ptah#1618 records it.

From the Ptah repository:

Terminal window
make conformance

Ptah’s own CI also builds the pinned Atlas CE oracle from an immutable source archive. The build verifies the release tag’s locked commit, the archive SHA-256, and exact version output. It runs every night, and on any pull request that changes the lock, the build scripts or the Go toolchain. Other runs reuse the built binary after checking the version it prints. CI then runs the migration-directory query, migrate-apply interoperability, and Flyway revision-identity controls through both command-line processes, runs differential migration-sum tests, regenerates the recorded corpus, and fails if the committed corpus changes. This is a black-box executable used only by tests; Atlas source and compiled code are not imported, vendored, or linked into Ptah.

Atlas Cloud and commercial binaries are outside this oracle workflow.

Terminal window
scripts/build-atlas-ce-reference.sh
GOWORK=off \
PTAH_ATLAS_REFERENCE="$PWD/bin/atlas-ce-reference" \
go test -tags=integration -count=1 \
./integration/atlasreference/migratedirquery
GOWORK=off \
PTAH_ATLAS_REFERENCE="$PWD/bin/atlas-ce-reference" \
go test -tags=integration -count=1 \
./integration/atlasreference/migrateapply
GOWORK=off \
PTAH_ATLAS_REFERENCE="$PWD/bin/atlas-ce-reference" \
go test -tags=integration -count=1 \
./integration/atlasreference/flywayrevision
GOWORK=off \
PTAH_ATLAS_REFERENCE="$PWD/bin/atlas-ce-reference" \
PTAH_ATLAS_FUZZ_N=200 \
go test -count=1 \
-run '^TestSumFileNamesDifferentialFuzz(RealisticFlyway|OtherFormats)?$' \
./internal/atlasmigrateimport

From ptah-atlas-conformance:

Terminal window
make probe
make budget
make gate
make probe-live
make budget-live
make gate-live
make probe-diff
make budget-diff
make probe-orm-providers
make budget-orm-providers
make gate-orm-providers
make probe-cli-surface
make budget-cli-surface
make gate-cli-surface

Live and differential probes require real database URLs. Differential probes also require an Atlas CE binary built from the pinned Atlas version in the conformance repository. CLI surface probes use the same pinned Atlas CE binary to compare command paths, help boundaries, flags, and runtime classifications.

Update conformance after Ptah changes that affect Atlas command behavior, schema parsing/rendering, migration directory semantics, live database round-trips, or public compatibility APIs. Bump the Ptah module version in the conformance repository, run go mod tidy, regenerate the relevant reports, and let both regression and full-conformance checks show the expected state.

The deterministic offline report includes a 20-observation external-schema workflow. It measures static SQL; external programs that emit SQL, HCL, and YAML; trust denial without side effects for render, compare, drift, plan, and generate; configuration and explicit CLI sources; migration generation and application to ephemeral SQLite; table, primary-key, unique-index, and cascading-foreign-key facts; and converged compare, drift, plan, and generate results.

Pinned GORM and SQLAlchemy providers run in a separate tier so network-backed dependency installation cannot weaken the deterministic corpus. The tier has independent regression-budget and zero-gap full-conformance jobs.

This coverage measures Ptah’s native external-program source. Atlas HCL data.external_schema evaluation is a separate project-language feature and is not implied by a green native provider report.