Pull from an HTTPS URL
Pull a desired schema or a migration directory from an HTTPS URL whose bytes are pinned by their SHA-256, with its content digest.
ptah schema pull and ptah migrations pull read from an https:// URL as
well as from an OCI registry and a Git repository. The URL names one download:
the schema file itself, or a tar.gz or zip archive. The download is pinned
by the SHA-256 of its exact bytes, and Ptah reads none of it before that digest
matches, so a server that serves other bytes reaches no archive reader.
Ptah Operator delivers schemas and migrations from HTTPS through these commands, so what a pull reports here is what the operator binds a plan to.
Pull a schema file
Section titled “Pull a schema file”ptah schema pull https://artifacts.example.com/inventory/1.4.0/schema \ --format file --path schema.hcl \ --expect-download-digest sha256:5f3c... --out schema.hclWith --format file the download is the schema file, and --path is the one
name it is given: the name in the manifest, whose extension tells the schema’s
format. The bytes are written as served, so give --out the same extension.
Pulled https://artifacts.example.com/inventory/1.4.0/schema to /work/schema.hclDownload digest: sha256:5f3c...Content digest: sha256:dd0b...A schema file can come in an archive too: --format tar.gz or --format zip,
with --path the member to read, such as db/schema.hcl.
Pull a migration directory
Section titled “Pull a migration directory”ptah migrations pull https://artifacts.example.com/orders/orders-1.4.0.tar.gz \ --format tar.gz --path orders-1.4.0/db/migrations \ --expect-download-digest sha256:9b71... --out migrationsA migration directory comes in an archive, tar.gz or zip. Every file under
--path is written under --out, at its path relative to the selected
directory; --path . selects the archive’s root. A leading ./ on an entry’s
name is ignored, as tar -C dir -czf orders.tar.gz . writes it.
Pin the download
Section titled “Pin the download”--expect-download-digest is required. It is sha256: and the SHA-256 of the
exact bytes the URL serves, which the job that publishes the archive can print:
shasum -a 256 orders-1.4.0.tar.gzThe pull writes the body to a private temporary file, hashing it as it writes, and compares the digest before it reads the file as an archive. A mismatch names both digests. The download is deleted once the selection is in memory, before the output is written, so the temporary directory holds one of the two at a time.
--expect-digest holds the selected files to their content digest as well, as
for Git. The content digest names
the files, so an archive packed again with the same files keeps it while its
download digest changes.
Read the report as JSON
Section titled “Read the report as JSON”--json prints one document instead of the text:
{ "source": "https", "reference": "https://artifacts.example.com/orders/orders-1.4.0.tar.gz", "format": "tar.gz", "download_digest": "sha256:9b71...", "download_size": 18234, "path": "orders-1.4.0/db/migrations", "content_digest": "sha256:9a4e...", "files": [ {"path": "0000000001_create_orders.up.sql", "size": 120, "sha256": "..."} ], "written": ["/work/migrations"]}files lists every file read, sorted by path, so two pulls of the same content
print the same document.
Authenticate and pin the server
Section titled “Authenticate and pin the server”Credentials come from the environment, never from a flag or the URL:
| Variable | Used for |
|---|---|
PTAH_HTTPS_USERNAME and PTAH_HTTPS_PASSWORD |
Basic authentication |
PTAH_HTTPS_TOKEN |
a Bearer token |
Any other combination is refused: a username without a password, a password without a username, or a token beside either.
--ca-file names the only roots the server’s certificate may chain to; without
it, the system’s roots are used. TLS 1.2 or later is required, and no flag
turns verification off. The transport reads HTTPS_PROXY and NO_PROXY from
the environment.
A pull follows up to 10 redirects, never to http:// and never to a host
with a byte outside printable ASCII. While a credential is set, a redirect to
another origin (scheme, host and port) is refused, so the credential is never
sent to a server it was not meant for. Hosts are compared ignoring ASCII case
only: Unicode case folding reads gİthub.com as github.com, while the
connection goes to xn--github-qyd.com. Without a credential, a redirect to
another origin is followed, and the digest still pins the bytes.
What a pull refuses
Section titled “What a pull refuses”The URL and the request:
- an
http://URL, a URL with a user, a query or a fragment, and one with no path; - a host with a byte outside printable ASCII; write an internationalized name
in its
xn--form; - a status other than
200, named in the message; the response body is never printed; - a
Content-Encodingother than identity: the digest names the bytes as served, and the pull asks for them unencoded; - a declared length over 64 MiB, before the body is read, and a body that runs past 64 MiB or ends before its declared length.
Every entry of an archive, inside the selection or not:
- an absolute name, a
..element, a backslash, a drive letter or a NUL, whateverGODEBUGsays about insecure paths; - more than 10,000 entries, or a
tar.gzthat expands past 1 GiB.
Inside the selection:
- a symbolic link, a hard link, a device, a FIFO or a sparse file, each named in the message;
- a duplicate path, or a path that is both a file and a directory;
- a path with an empty,
.or..element, a control character or invalid UTF-8; - an encrypted zip entry, a zip method other than store or deflate, and a CRC that does not match; a truncated gzip stream, or a trailer that does not match;
- more than 1,000 files or more than 64 MiB of selected content;
- a selected path that is not there, a directory where a schema selects a file,
and
--format filefor a migration directory.
Entries outside the selection are skipped: a zip entry there is not inflated.
The output path, as for every pull, must not exist.