Skip to content
PtahDocs
latest

Graphic preview

100%
Page type: how-to

Pull from an HTTPS URL

Pull a desired schema or a migration directory from an HTTPS URL whose bytes are pinned by their SHA-256, with its content digest.

ptah schema pull and ptah migrations pull read from an https:// URL as well as from an OCI registry and a Git repository. The URL names one download: the schema file itself, or a tar.gz or zip archive. The download is pinned by the SHA-256 of its exact bytes, and Ptah reads none of it before that digest matches, so a server that serves other bytes reaches no archive reader.

Ptah Operator delivers schemas and migrations from HTTPS through these commands, so what a pull reports here is what the operator binds a plan to.

Terminal window
ptah schema pull https://artifacts.example.com/inventory/1.4.0/schema \
--format file --path schema.hcl \
--expect-download-digest sha256:5f3c... --out schema.hcl

With --format file the download is the schema file, and --path is the one name it is given: the name in the manifest, whose extension tells the schema’s format. The bytes are written as served, so give --out the same extension.

Pulled https://artifacts.example.com/inventory/1.4.0/schema to /work/schema.hcl
Download digest: sha256:5f3c...
Content digest: sha256:dd0b...

A schema file can come in an archive too: --format tar.gz or --format zip, with --path the member to read, such as db/schema.hcl.

Terminal window
ptah migrations pull https://artifacts.example.com/orders/orders-1.4.0.tar.gz \
--format tar.gz --path orders-1.4.0/db/migrations \
--expect-download-digest sha256:9b71... --out migrations

A migration directory comes in an archive, tar.gz or zip. Every file under --path is written under --out, at its path relative to the selected directory; --path . selects the archive’s root. A leading ./ on an entry’s name is ignored, as tar -C dir -czf orders.tar.gz . writes it.

--expect-download-digest is required. It is sha256: and the SHA-256 of the exact bytes the URL serves, which the job that publishes the archive can print:

Terminal window
shasum -a 256 orders-1.4.0.tar.gz

The pull writes the body to a private temporary file, hashing it as it writes, and compares the digest before it reads the file as an archive. A mismatch names both digests. The download is deleted once the selection is in memory, before the output is written, so the temporary directory holds one of the two at a time.

--expect-digest holds the selected files to their content digest as well, as for Git. The content digest names the files, so an archive packed again with the same files keeps it while its download digest changes.

--json prints one document instead of the text:

{
"source": "https",
"reference": "https://artifacts.example.com/orders/orders-1.4.0.tar.gz",
"format": "tar.gz",
"download_digest": "sha256:9b71...",
"download_size": 18234,
"path": "orders-1.4.0/db/migrations",
"content_digest": "sha256:9a4e...",
"files": [
{"path": "0000000001_create_orders.up.sql", "size": 120, "sha256": "..."}
],
"written": ["/work/migrations"]
}

files lists every file read, sorted by path, so two pulls of the same content print the same document.

Credentials come from the environment, never from a flag or the URL:

Variable Used for
PTAH_HTTPS_USERNAME and PTAH_HTTPS_PASSWORD Basic authentication
PTAH_HTTPS_TOKEN a Bearer token

Any other combination is refused: a username without a password, a password without a username, or a token beside either.

--ca-file names the only roots the server’s certificate may chain to; without it, the system’s roots are used. TLS 1.2 or later is required, and no flag turns verification off. The transport reads HTTPS_PROXY and NO_PROXY from the environment.

A pull follows up to 10 redirects, never to http:// and never to a host with a byte outside printable ASCII. While a credential is set, a redirect to another origin (scheme, host and port) is refused, so the credential is never sent to a server it was not meant for. Hosts are compared ignoring ASCII case only: Unicode case folding reads gİthub.com as github.com, while the connection goes to xn--github-qyd.com. Without a credential, a redirect to another origin is followed, and the digest still pins the bytes.

The URL and the request:

  • an http:// URL, a URL with a user, a query or a fragment, and one with no path;
  • a host with a byte outside printable ASCII; write an internationalized name in its xn-- form;
  • a status other than 200, named in the message; the response body is never printed;
  • a Content-Encoding other than identity: the digest names the bytes as served, and the pull asks for them unencoded;
  • a declared length over 64 MiB, before the body is read, and a body that runs past 64 MiB or ends before its declared length.

Every entry of an archive, inside the selection or not:

  • an absolute name, a .. element, a backslash, a drive letter or a NUL, whatever GODEBUG says about insecure paths;
  • more than 10,000 entries, or a tar.gz that expands past 1 GiB.

Inside the selection:

  • a symbolic link, a hard link, a device, a FIFO or a sparse file, each named in the message;
  • a duplicate path, or a path that is both a file and a directory;
  • a path with an empty, . or .. element, a control character or invalid UTF-8;
  • an encrypted zip entry, a zip method other than store or deflate, and a CRC that does not match; a truncated gzip stream, or a trailer that does not match;
  • more than 1,000 files or more than 64 MiB of selected content;
  • a selected path that is not there, a directory where a schema selects a file, and --format file for a migration directory.

Entries outside the selection are skipped: a zip entry there is not inflated.

The output path, as for every pull, must not exist.