# Build a minimal provider

Build an executable provider for a new engine with the provider SDK, check it with Ptah, and apply a schema through it.

Source: https://docs.ptah.run/edge/extend/write-a-provider/tutorial/

Ptah reaches a database engine it does not ship through a provider: a separate
executable that Ptah starts, describes, and calls over a local connection. You
build one here from an empty Go module. It serves an engine named `tutorial`,
and at the end `ptah schema apply` creates a table through it.

A `tutorial` database is a SQLite file, so you need no database server. Where
the provider calls `database/sql` with a SQLite driver, a provider for your
engine calls that engine's client. The finished module is
[`examples/minimal-provider` in the latest development source](https://github.com/stokaro/ptah/blob/master/examples/minimal-provider/README.md),
and every Go block below is an excerpt of one of its files.

## Prerequisites

- A Go toolchain at least as new as the `go` line in the `go.mod` of step 1.
- `ptah` on your `PATH`. See [Install Ptah](../../../start/install/).
- `shasum`, or another program that prints the SHA-256 digest of a file.

## 1. Create the module

Create an empty module and add the provider SDK and the SQLite driver:

```bash
mkdir ptah-tutorial-provider && cd ptah-tutorial-provider
go mod init example.com/ptah-tutorial-provider
go get ptah.run/provider-sdk@v0.40.0 modernc.org/sqlite@v1.61.1
```

The SDK is a Go module of its own, `ptah.run/provider-sdk`,
released under tags named `provider-sdk/vX.Y.Z`. It imports nothing from Ptah,
and your provider imports nothing else from Ptah either: the two meet only at
the process boundary. The provider owns its database driver.

Keep the module path while you follow along, because the excerpts import
`example.com/ptah-tutorial-provider/backend`. After `go mod tidy` in step 6,
the direct requirements match the example's:

```text title="go.mod"
module example.com/ptah-tutorial-provider

go 1.26.5

require (
	github.com/frankban/quicktest v1.14.6
	google.golang.org/grpc v1.86.0-dev
	google.golang.org/protobuf v1.36.12
	modernc.org/sqlite v1.61.1
	ptah.run/provider-sdk v0.40.0
)
```

The example's tests add quicktest; a module without tests does not need it.

## 2. Describe the provider

Each of the next steps adds one file to a `backend` package and shows its
central declarations. The complete file, with its package clause, imports and
helpers, is in the example: copy it rather than assembling it from the
excerpts.

Ptah's first call to a provider is `Describe`. The reply says who the provider
is, which engines it serves, and which services it offers. Create
`backend/description.go`, which also declares `version` and
`identifierRules`:

```go title="backend/description.go"
const (
	// ProviderID identifies the provider. A manifest entry names the same
	// value, and Ptah refuses an executable that describes another one.
	ProviderID = "example.com/ptah-tutorial-provider"
	// Engine is the engine the provider serves. It is also the scheme of the
	// addresses Open accepts: tutorial:///absolute/path/to/file.db.
	Engine = "tutorial"
	// defaultSchema is the one schema a tutorial database has.
	defaultSchema = "main"
)

// Description is what the provider tells Ptah when Ptah starts it: who it
// is, the engine it serves and the services it offers. Ptah starts a
// provider only when it offers inspection, rendering, execution and SQL
// analysis.
func Description() *protocol.Description {
	return &protocol.Description{
		Revision:         provider.Revision,
		ProviderId:       ProviderID,
		Version:          version,
		SemanticIdentity: "tutorial-1",
		Engines: []*protocol.Engine{{
			Id:            Engine,
			DefaultSchema: defaultSchema,
			Identifiers:   identifierRules(),
			// Ptah plans the tables and columns of this engine with its
			// common planner and asks the provider to render each change.
			Features: []string{provider.CommonTableIdentityFeature, provider.LocksFeature},
		}},
		Features: []string{
			provider.InspectFeature,
			provider.RenderFeature,
			provider.ExecuteFeature,
			provider.AnalyzeSQLFeature,
			// Ptah takes a lock before it applies a plan.
			provider.LocksFeature,
			// A failed call carries a reason the provider wrote.
			provider.ErrorReasonsFeature,
		},
	}
}
```

The fields carry the provider's identity and its promises:

- `Revision` is the contract's major version. Ptah refuses a provider of
  another major before it opens a session.
- `ProviderId` is a namespaced name. A manifest entry names the same value,
  and Ptah refuses an executable that reports another one.
- `SemanticIdentity` names what the provider's values and plans mean. A saved
  plan binds to it, so change it when the same input would mean something else.
- `Engines[].Id` is the engine name. It is also the URL scheme a command uses
  to reach the engine, as in `tutorial:///path/app.db`, and `Aliases` adds
  more schemes.
- `Identifiers` says how the engine compares names. Ptah never infers it.

Feature names select services. Ptah starts a provider only when it lists
`schema.inspect`, `schema.render`, `units.execute` and `sql.analyze`. The
engine's `schema.common-table-identity.v1` asks Ptah to plan the engine's
tables and columns with its common planner, so the provider renders each
change rather than planning it. `sessions.locks` offers the lock that
`ptah schema apply` takes, on the provider and on the engine. `errors.reasons`
lets a failed call carry a reason the provider wrote.

The description sets no minimum contract, which means any host of contract 25
can start the provider. [How a provider works](../architecture/#contract-versions-and-features)
explains contract minors and features.

## 3. Open sessions and read the database

Ptah opens a session for each database a command connects to. The address
travels in the `Open` request, never on a command line, because it can carry a
credential. Create `backend/sessions.go`. Its core is the `Provider` type and
`Open`:

```go title="backend/sessions.go"
// Provider serves the tutorial engine. The zero value is ready to use, and
// one value serves every session of the process.
type Provider struct {
	mu       sync.Mutex
	sessions map[string]*session
}

// session is one open database. Its mutex serializes the calls that use it.
type session struct {
	mu    sync.Mutex
	db    *sql.DB
	path  string
	locks map[string]*lock
	// lost is set once a lock the session held is gone. It never clears:
	// the session runs nothing more.
	lost bool
}

// Open connects to the database file the address names and returns a session
// bound to it. SQLite creates a missing file on the first connection.
func (p *Provider) Open(ctx context.Context, request *protocol.OpenRequest) (*protocol.Session, error) {
	if request.GetEngine() != Engine {
		return nil, runtime.ErrorWithReason(codes.InvalidArgument, "the provider serves the tutorial engine only")
	}
	path, err := databasePath(request.GetAddress())
	if err != nil {
		return nil, err
	}
	db, err := sql.Open("sqlite", path)
	if err != nil {
		return nil, runtime.ErrorWithReason(codes.InvalidArgument, "the database file cannot be opened")
	}
	db.SetMaxOpenConns(1)
	var serverVersion string
	if err := db.QueryRowContext(ctx, "SELECT sqlite_version()").Scan(&serverVersion); err != nil {
		_ = db.Close()
		return nil, runtime.ErrorWithReason(codes.Unavailable, "the database file cannot be read")
	}
	id := rand.Text()
	p.mu.Lock()
	defer p.mu.Unlock()
	if p.sessions == nil {
		p.sessions = make(map[string]*session)
	}
	p.sessions[id] = &session{db: db, path: path}
	return &protocol.Session{
		Id:            id,
		Engine:        Engine,
		Database:      path,
		DefaultSchema: defaultSchema,
		ServerVersion: serverVersion,
		Identifiers:   identifierRules(),
	}, nil
}
```

A reason the provider writes never quotes the address. The file also holds
`Close`, which releases a session and its locks, `lookup`, and
`databasePath`, which turns `tutorial:///path` into a file path.

A read of the database is `Inspect`. The snapshot it returns is complete, or
the call fails. Ptah compares what a read reports with the desired schema, so
an object a read leaves out is an object Ptah plans to create. Create
`backend/inspect.go`:

```go title="backend/inspect.go"
// Inspect reads every table of the database with its columns. The snapshot is
// complete or the call fails: an object the engine cannot describe, such as a
// view or a table with a CHECK constraint, fails the read rather than
// disappearing from it, because Ptah would plan to create what a read leaves
// out.
func (p *Provider) Inspect(ctx context.Context, request *protocol.InspectRequest) (*protocol.Schema, error) {
	current, err := p.lookup(request.GetSessionId())
	if err != nil {
		return nil, err
	}
	for _, schema := range request.GetSchemas() {
		if schema != defaultSchema {
			return nil, runtime.ErrorWithReason(codes.InvalidArgument, "a tutorial database has the schema main only")
		}
	}
	current.mu.Lock()
	defer current.mu.Unlock()
	if err := refuseOtherObjects(ctx, current.db); err != nil {
		return nil, err
	}
	names, err := tableNames(ctx, current.db)
	if err != nil {
		return nil, err
	}
	snapshot := &protocol.Schema{
		Complete: true,
		Engine:   Engine,
		Coverage: &protocol.ModelCoverage{Representation: protocol.ValueInput_OBSERVED},
	}
	for _, name := range names {
		table, err := readTable(ctx, current, name)
		if err != nil {
			return nil, err
		}
		snapshot.Tables = append(snapshot.Tables, table)
	}
	// A read narrowed to schemas says which namespace each selector names.
	for _, selector := range request.GetSchemas() {
		snapshot.ScopeBindings = append(snapshot.ScopeBindings, &protocol.InspectionScopeBinding{
			Selector: selector,
			Namespaces: []*protocol.ModelNamespace{{
				Database: &protocol.NamespaceComponent{Source: current.path, Normalized: current.path},
				Catalog:  &protocol.NamespaceComponent{},
				Schema:   &protocol.NamespaceComponent{},
			}},
		})
	}
	return snapshot, nil
}
```

The helpers in the same file read the catalog. `readTable` proves that a read
is complete: it renders the columns it read back into a `CREATE TABLE`
statement and compares it with the definition SQLite stored, so a CHECK
constraint or a default the engine does not model fails the read. A catalog
query that fails reports a reason of the provider's own and never the driver's
text, which can quote a value.

## 4. Render, run and analyze changes

Ptah plans a change and then asks the provider to write it. `Render` connects
to nothing. It returns one unit for each change, in the order the changes
arrive, or refuses the whole batch. Create `backend/render.go`:

```go title="backend/render.go"
// Render writes one SQL statement for each change Ptah planned, in the order
// the changes arrive. It connects to nothing. A change it cannot write
// refuses the whole batch, so Ptah never receives part of a plan.
func (*Provider) Render(_ context.Context, request *protocol.RenderRequest) (*protocol.RenderResult, error) {
	result := &protocol.RenderResult{Complete: true}
	for _, change := range request.GetChanges() {
		statement, err := renderChange(change)
		if err != nil {
			return nil, err
		}
		result.Units = append(result.Units, &protocol.Unit{
			ChangeId: change.GetId(),
			Body:     &protocol.Unit_Sql{Sql: statement},
		})
	}
	return result, nil
}

// renderChange writes the statement of one change.
func renderChange(change *protocol.Change) (string, error) {
	object := change.GetObject()
	if object.GetName() == "" || object.GetSchema() != "" {
		return "", refuse("a tutorial table has a name and no schema qualifier")
	}
	table := quote(object.GetName())
	switch operation := change.GetOperation().(type) {
	case *protocol.Change_CreateTable:
		return createTable(table, operation.CreateTable)
	case *protocol.Change_DropTable:
		return "DROP TABLE " + table + ";", nil
	case *protocol.Change_AddColumn:
		if operation.AddColumn.GetPrimaryKey() {
			return "", refuse("the tutorial engine cannot add a primary key column to an existing table")
		}
		column, err := columnDefinition(operation.AddColumn)
		if err != nil {
			return "", err
		}
		return "ALTER TABLE " + table + " ADD COLUMN " + column + ";", nil
	case *protocol.Change_DropColumn:
		return "ALTER TABLE " + table + " DROP COLUMN " + quote(operation.DropColumn) + ";", nil
	default:
		return "", refuse("the tutorial engine creates and drops tables and adds and drops columns, nothing else")
	}
}
```

```go title="backend/render.go"
// refuse fails a call with a reason Ptah shows to the operator. A reason
// names what failed and why. It never quotes a statement, a value, an address
// or a credential.
func refuse(reason string) error {
	return runtime.ErrorWithReason(codes.InvalidArgument, reason)
}
```

The same file writes the column definitions and quotes names. `Inspect` reuses
`createTable` for its completeness check.

`Execute` runs the units and reports how many the database applied. Create
`backend/execute.go`:

```go title="backend/execute.go"
// Execute runs the units of one request in order and reports how many of them
// the database applied. It never runs a request again because of its ID: when
// the reply is lost, Ptah reads the database to learn what happened.
//
// Each unit is one SQLite statement in autocommit mode, so a statement the
// database refuses applied nothing, and the units before it stay applied. A
// canceled call may have stopped inside a statement, so its outcome is
// UNKNOWN.
func (p *Provider) Execute(ctx context.Context, request *protocol.ExecuteRequest) (*protocol.Execution, error) {
	current, err := p.lookup(request.GetSessionId())
	if err != nil {
		return nil, err
	}
	if request.GetTransactionId() != "" {
		return nil, runtime.ErrorWithReason(codes.InvalidArgument, "the tutorial engine offers no transactions")
	}
	current.mu.Lock()
	defer current.mu.Unlock()
	units := request.GetUnits()
	var applied uint32
	for _, unit := range units {
		statement, ok := unit.GetBody().(*protocol.Unit_Sql)
		if !ok {
			return nil, runtime.ErrorWithReason(codes.InvalidArgument, "the tutorial engine runs SQL units only")
		}
		if !current.guard(ctx) {
			return stoppedAt(applied, fmt.Sprintf("the session lost its lock before unit %d of %d", applied+1, len(units))), nil
		}
		if _, err := current.db.ExecContext(ctx, statement.Sql); err != nil {
			return failedAt(ctx, applied, len(units)), nil
		}
		applied++
	}
	return &protocol.Execution{Outcome: protocol.Execution_APPLIED, AppliedCount: applied}, nil
}
```

The outcome is a contract. `APPLIED` confirms the whole batch,
`NOT_APPLIED` confirms that no unit applied, `PARTIAL` confirms a prefix, and
`UNKNOWN` says nobody knows. The helpers `failedAt` and `stoppedAt` in the
same file choose between them, and each result that is not `APPLIED` carries a
reason that names the unit and never its text.

Ptah reads SQL of the engine through `AnalyzeSQL` wherever it judges a
statement, such as in the data-safety check before a plan runs. Create
`backend/analyze.go`:

```go title="backend/analyze.go"
// AnalyzeSQL splits each source into statements and says what each one is.
// Ptah reads SQL through it wherever it judges a statement of the engine,
// such as the data-safety check before a plan runs. Every byte of a source is
// part of a statement or white space between statements, and a statement the
// engine does not know refuses the whole request.
func (*Provider) AnalyzeSQL(ctx context.Context, request *protocol.AnalyzeSQLRequest) (*protocol.SQLAnalysis, error) {
	if request.GetEngine() != Engine {
		return nil, refuse("the provider analyzes the SQL of the tutorial engine only")
	}
	result := &protocol.SQLAnalysis{Complete: true}
	for _, source := range request.GetSources() {
		if err := ctx.Err(); err != nil {
			return nil, err
		}
		file, err := analyzeSource(source)
		if err != nil {
			return nil, err
		}
		result.Files = append(result.Files, file)
	}
	return result, nil
}
```

```go title="backend/analyze.go"
// analyzedStatement describes the statement that spans source[start:end].
// The engine knows the four statements Render writes. Each is common DDL:
// Ptah judges what a common statement does to the schema, and a DROP is
// destructive in its judgment, not in the provider's.
func analyzedStatement(source string, start, end int, words []string) (*protocol.SQLStatement, error) {
	canonical := make([]string, len(words))
	for i, word := range words {
		canonical[i] = word
		if !strings.HasPrefix(word, `"`) {
			canonical[i] = strings.ToUpper(word)
		}
	}
	if !knownStatement(canonical) {
		return nil, refuse("the tutorial engine runs CREATE TABLE, DROP TABLE and ALTER TABLE ADD COLUMN or DROP COLUMN only")
	}
	line := 1 + strings.Count(source[:start], "\n")
	return &protocol.SQLStatement{
		Start:       uint32(start), // #nosec G115 -- analyzeSource refuses a source longer than math.MaxUint32
		End:         uint32(end),   // #nosec G115 -- analyzeSource refuses a source longer than math.MaxUint32
		Line:        uint32(line),  // #nosec G115 -- a line number is at most the source length
		Words:       canonical,
		SourceWords: words,
		Canonical:   strings.Join(canonical, " "),
		Safety:      protocol.SQLStatement_COMMON_DDL,
	}, nil
}
```

Every byte of a source belongs to a statement or to the white space between
statements. `COMMON_DDL` tells Ptah to judge the statement with its own rules
for common DDL; it does not mean that the statement is safe. The tokenizer
`analyzeSource` and `tokenEnd` complete the file.

## 5. Hold the apply lock

`ptah schema apply` takes a lock before it reads the target, so two runs never
change one database at once. The provider owns how the engine excludes a
second session, and Ptah owns how long it waits. Create `backend/locks.go`:

```go title="backend/locks.go"
// lock is one lock ID of a session. Ptah takes a lock before it applies a
// plan, so two runs never change one database at once.
//
// The tutorial engine holds a lock as a write transaction on a sidecar file
// beside the database, one file per lock name. SQLite lets one connection at
// a time hold a write transaction on a file, in any process, so a second
// attempt is BUSY. The transaction never commits: rolling it back releases
// the lock, and so does the end of the provider process.
type lock struct {
	name    string
	outcome protocol.LockResult_Outcome
	db      *sql.DB
	tx      *sql.Tx
}

// TryLock makes one attempt to take a lock and does not wait. A lock ID is
// used once: a second request with the same ID gets the answer of the first.
func (p *Provider) TryLock(ctx context.Context, request *protocol.LockRequest) (*protocol.LockResult, error) {
	current, err := p.lookup(request.GetScope().GetSessionId())
	if err != nil {
		return nil, err
	}
	current.mu.Lock()
	defer current.mu.Unlock()
	record, fresh, err := current.reserve(request.GetScope())
	if err != nil {
		return nil, err
	}
	switch {
	case current.lost:
		return receipt(request, protocol.LockResult_LOST), nil
	case !fresh:
		current.verify(ctx, record)
		return receipt(request, record.outcome), nil
	}
	if err := record.acquire(ctx, current.path); err != nil {
		return nil, err
	}
	return receipt(request, record.outcome), nil
}
```

```go title="backend/locks.go"
// guard verifies every lock the session holds. Execute calls it before each
// unit, because a statement must not run after the lock that protects it is
// gone.
func (s *session) guard(ctx context.Context) bool {
	for _, record := range s.locks {
		s.verify(ctx, record)
	}
	return !s.lost
}
```

A lock ID is used once, and each receipt echoes the request's scope and
request ID. The file also holds `CheckLock`, which proves ownership now rather
than repeating an earlier answer, and `ReleaseLock`, which fences an ID that
never took the lock. In `acquire`, the write transaction starts under
`context.WithoutCancel`, because the lock outlives the call that took it.
`Execute` calls `guard` before each unit, so a statement never runs after the
lock that protects it is gone.

## 6. Serve the provider and check it

Join the description with its services in `backend/server.go`:

```go title="backend/server.go"
// Server joins the description with the services that implement it. Each
// service the description offers needs an implementation, and each
// implementation needs its feature in the description: runtime.Serve refuses
// a server where the two differ.
func Server() runtime.Server {
	tutorial := &Provider{}
	return runtime.Server{
		Description: Description(),
		Sessions:    tutorial,
		Inspector:   tutorial,
		Renderer:    tutorial,
		Executor:    tutorial,
		SQLAnalyzer: tutorial,
		Locks:       tutorial,
	}
}
```

Serve it from `main.go`:

```go title="main.go"
func main() {
	if err := runtime.Serve(backend.Server()); err != nil {
		fmt.Fprintln(os.Stderr, err)
		os.Exit(1)
	}
}
```

`runtime.Serve` refuses a server that offers a service it does not implement,
or implements one it does not offer. It reserves standard output for the
transport, and Ptah discards what the provider writes to standard error. The
executable runs only under Ptah: started by hand, it exits with an error.

Tidy the module and build the provider:

```bash
go mod tidy
go build -o ptah-tutorial-provider .
```

Ptah starts an executable provider only when a manifest names its absolute
path and the SHA-256 digest of its bytes. Write `providers.json` for the build:

```bash
digest=$(shasum -a 256 ptah-tutorial-provider | cut -d ' ' -f 1)
cat > providers.json <<EOF
{
  "format_version": 2,
  "providers": [
    {
      "executable": "$PWD/ptah-tutorial-provider",
      "sha256": "$digest",
      "provider_id": "example.com/ptah-tutorial-provider",
      "engines": [{"id": "tutorial", "aliases": []}],
      "environment": []
    }
  ]
}
EOF
```

Every build has a new digest, so write the manifest again after each one.
Start the provider without opening a database:

```bash
ptah providers check --provider-config providers.json --provider-engine tutorial
```

Expected output on standard output:

```text
example.com/ptah-tutorial-provider 0.0.0-dev: compatible protocol revision 25 (/path/to/ptah-tutorial-provider/ptah-tutorial-provider)
```

Ptah verified the digest, started the process, negotiated the contract and
compared the provider ID and the engine with the manifest entry. Add `--json`
to see the features the two sides agreed on.

## 7. Apply a schema

Create `schema.yaml`:

```yaml title="schema.yaml"
tables:
  notes:
    columns:
      id:
        type: INTEGER
        primary: true
        not_null: true
      body:
        type: TEXT
```

Read the empty database. SQLite creates the file on the first connection:

```bash
export DATABASE_URL="tutorial://$PWD/app.db"
ptah schema inspect --provider-config providers.json --db-url "$DATABASE_URL"
```

Expected output on standard output:

```text
// Code generated by ptah; DO NOT EDIT.
// ptah:not-described object_owner reason=not-inspected provenance=observed
// ptah:not-described role_membership reason=not-inspected provenance=observed

schema "main" {
}
```

The `not-described` lines say what the read did not inspect, so a later
comparison does not treat those objects as absent. Plan the schema and save
the plan:

```bash
ptah schema plan --provider-config providers.json \
  --db-url "$DATABASE_URL" --schema-file schema.yaml --output change.plan.json
```

Expected output on standard output:

```text
Planned schema changes:
CREATE TABLE "notes" ("id" INTEGER NOT NULL PRIMARY KEY, "body" TEXT);
Plan saved to file://change.plan.json
```

Ptah compared the read with the YAML schema, planned one table creation, and
asked the provider to render it. Apply the saved plan:

```bash
ptah schema apply --provider-config providers.json \
  --db-url "$DATABASE_URL" --plan change.plan.json --tx-mode none --auto-approve
```

Expected output on standard output:

```text
Planned schema changes:
CREATE TABLE "notes" ("id" INTEGER NOT NULL PRIMARY KEY, "body" TEXT);
Auto-approval enabled; applying schema changes.
Schema apply completed successfully.
```

Ptah took the lock through `TryLock`, ran the statement through `Execute`, and
released the lock. The provider offers no transactions, so the apply runs with
`--tx-mode none`; the default mode is refused before any statement runs.

Plan the same schema again to verify that the database matches it:

```bash
ptah schema plan --provider-config providers.json \
  --db-url "$DATABASE_URL" --schema-file schema.yaml --dry-run --json
```

Expected output on standard output:

```text
{
  "contract_version": 3,
  "command": "schema plan",
  "outcome": "no-changes"
}
```

A declaration the engine cannot hold is refused with the provider's reason.
Change the type of `body` in `schema.yaml` to `VARCHAR(80)` and plan again:

```text illustration
error: compare database schema: provider render failed (InvalidArgument): the tutorial engine stores the column types INTEGER, TEXT, REAL and BLOB only
```

The text after the code is the reason `refuse` wrote. Without
`errors.reasons`, Ptah prints the code alone.

## Clean up

Leave the module and remove it:

```bash
cd ..
rm -rf ptah-tutorial-provider
```

## Next steps

- To decide which services your engine needs next, read
  [How a provider works](../architecture/).
- To prove each service before Ptah calls it, follow
  [Test a provider](../testing/).
- To publish builds that `ptah providers install` and a manifest can trust, follow
  [Package and release a provider](../release/).