Testing locally

The problem with push-debug-push

The fastest way to find out whether a package works is to push it and install it. That loop has a cost: every push bakes a new digest into the registry, forces a re-download on every consumer, and leaves a trail of half-finished tags in the cascade. A typo in metadata.json can mean three push-install cycles before the env surface looks right.

ocx package test cuts that loop. It runs the same install pipeline that consumers see — dep resolution, layer extraction, env composition — but writes the result to a temp directory instead of the registry. No network round-trip. No new digest. The temp directory disappears when the command exits.

Analogy

npm pack + npm install ./pkg.tgz is the closest analogue: materialize locally, run the thing, throw away the scratch directory. cargo publish --dry-run does a rebuild-from-scratch instead of reusing the archive — ocx package test uses the archive you already have.

Basic usage

The argument shape mirrors ocx package push: identifier as -i/--identifier, then layers, then an optional --platform. Like push, --platform defaults to the platform ocx package create recorded in the metadata sidecar; pass it explicitly only to assert it matches (a mismatch is rejected).

ocx package test -i acme/mytool:1.0.0 mytool-1.0.0.tar.xz -- mytool --version

The -- separator marks the end of package arguments and the start of the command to run. Everything after -- is passed verbatim to the child process. If you need to run scripted assertions instead of a single command, use --script instead of -- — the two forms are mutually exclusive.

OCX will:

Read metadata.json (inferred from the layer filename or supplied via -m).
Auto-install any declared dependencies into the regular package store.
Assemble the package in a temp directory under $OCX_HOME/temp/test/.
Compose the env exactly as ocx package exec would.
Exec the trailing command in that env.
Delete the temp directory on exit — whether the command succeeds or fails.

The child's exit code is forwarded unchanged. A failing test command (exit 7) gives you exit code 7.

What the command name resolves to

A bare command name is resolved against the package's own directories before the host PATH, and the package's copy of a name is never quietly passed over. The point of package test is to test what you are about to publish, so a name your package ships has to be the thing that runs.

That matters most when the file is there but cannot be executed. A binary that lost its executable bit — a common casualty of zip, of a COPY in a build image, or of a checkout on a filesystem that drops the bit — fails with exit code 65, naming the path and its mode. It is not passed over in favour of a same-named host binary: doing so would test something the package does not contain, and report a pass for a package nobody can run.

Names your package does not ship are unaffected: sh, grep, curl and the rest still resolve on the host PATH, with a warning on stderr naming the package directories that were searched. So a typo'd tool name is visible in the log rather than silently testing a host binary. A name carrying a path separator (./tool, an absolute path) addresses a file directly and skips this entirely.

To catch a missing executable bit before you even get here, pass --bin-scan to ocx package create: it verifies the binaries claim against the content tree at authoring time.

Identifier constraints

The identifier must be in tag form — repo:tag or registry/repo:tag. An explicit @digest suffix is rejected with a usage error (exit 64), because the digest is computed locally from the layers you supply and would conflict with any pre-committed value.

# good
ocx package test -p linux/amd64 -i acme/mytool:1.0.0 mytool.tar.xz -- true

# bad — digest rejected
ocx package test -p linux/amd64 -i acme/mytool:1.0.0@sha256:abc… mytool.tar.xz -- true

Keeping the build dir for inspection

When a command fails you often want to inspect the materialized layout — check which files landed where, whether entrypoints were generated, whether resolve.json is correct. Pass --keep to preserve the temp directory. OCX prints its path to stderr just before executing the command:

ocx package test -p linux/amd64 --keep -i acme/mytool:1.0.0 mytool.tar.xz -- mytool --version
# stderr: kept at /home/user/.ocx/temp/test/test-a1b2c3d4

The directory persists whether the command succeeds or fails. Without --keep, the temp directory is deleted in both cases.

Writing to a specific directory

--output DIR materializes the package to a directory you control instead of an auto-managed temp dir. The directory must not exist or must be empty — OCX creates it.

ocx package test -p linux/amd64 --output ./build -i acme/mytool:1.0.0 mytool.tar.xz -- mytool --version

The directory is never deleted by OCX. --output implies keep — omitting --keep is fine, combining them is an error.

Same filesystem required

--output DIR must reside on the same filesystem as $OCX_HOME/layers/. OCX assembles packages via hardlinks from the layer store; copying across filesystem boundaries is not supported. Passing a path on a different filesystem (for example /tmp/… when $OCX_HOME is on a separate partition) exits with code 74 (IoError) and a clear message.

Windows: --output must be under $OCX_HOME

On Windows, --output must point to a directory under $OCX_HOME/. Placing the output on a different volume — for example a separate drive letter — is not currently supported. Cross-volume hardlink support is planned for a future release.

Testing the private env surface

By default, ocx package test composes the interface surface — the env vars marked public or interface that consumers see. To compose the private surface (what the package sees when its own launchers run), pass --self:

ocx package test -p linux/amd64 --self -i acme/mytool:1.0.0 mytool.tar.xz \
  -- sh -c 'echo $MY_PRIVATE_VAR'

The --self flag mirrors the same flag on ocx package exec and ocx package env.

Stripping the parent env

By default the composed env inherits the parent shell's variables. Pass --clean to strip everything except the OCX_* config keys and the package-declared vars:

ocx package test -p linux/amd64 --clean -i acme/mytool:1.0.0 mytool.tar.xz \
  -- sh -c 'env | sort'

Useful when you want to verify the package supplies all required env on its own, without relying on ambient values from the developer's shell.

Digest layer references

Layer arguments can be file paths or digest references, exactly like ocx package push:

# base layer already in registry; only the top layer is local
ocx package test -p linux/amd64 -i acme/mytool:1.0.1 \
  sha256:<hex>.tar.xz ./newtool.tar.xz -- mytool --version

Digest layers are fetched from the registry on demand when not already cached locally. In --offline mode, a missing digest layer exits with code 81 (PolicyBlocked).

The inner pre-push loop

A typical authoring session looks like this:

# 1. Build the archive — --platform is recorded in the sidecar for the
#    steps below to read back.
ocx package create build -m metadata.json -o mytool-1.0.0.tar.xz -p linux/amd64

# 2. Test it locally — no registry involved.
ocx package test -i acme/mytool:1.0.0 \
  mytool-1.0.0.tar.xz -- mytool --version

# 3. Something wrong? Keep the dir and inspect.
ocx package test --keep -i acme/mytool:1.0.0 \
  mytool-1.0.0.tar.xz -- mytool --version
ls "$HOME/.ocx/temp/test/"*/

# 4. Happy with it? Push.
ocx package push -i acme/mytool:1.0.0 mytool-1.0.0.tar.xz

Testing a package locally before pushing

Scripted tests

The -- CMD form works well when the package ships its own test runner. Tool packages — cmake, shellcheck, goreleaser — do not. They need sh -c '...' on the host, which breaks on Windows without WSL or Git Bash.

--script PATH solves this. Instead of exec'ing a command, OCX interprets a Starlark script against the materialized package environment. The script has no access to the host shell or runtime. It runs identically on linux/*, macos/*, and windows/*.

What is Starlark?

Starlark is a deterministic, Python-like scripting language designed as an embedded configuration and scripting language for build tools. It is used by Bazel and Buck2. No while loops, deterministic iteration order, empty sandbox by default — properties that make it safe to run in a package manager context.

Invocation

# Read script from a file
ocx package test -p linux/amd64 -i shfmt/shfmt:3.8.0 shfmt.tar.xz --script smoke.star

# Read script source from stdin (the value `-` means stdin)
printf 'r = ocx.run("shfmt", "--version")\nexpect.ok(r)\n' \
  | ocx package test -p linux/amd64 -i shfmt/shfmt:3.8.0 shfmt.tar.xz --script -

--script and -- CMD are mutually exclusive. Supplying both exits with code 64. Supplying neither exits with code 64.

When --script - is used, OCX reads the script source from stdin. A read failure (broken pipe, closed stream) exits with code 74.

Host API — `ocx.*`

The ocx.* module gives the script access to the materialized package environment.

Function	Returns	Purpose
`ocx.run(prog, args, , env=None, cwd=None, stdin=None)`	`RunResult`	Spawn a binary from the composed package env. `env` is a dict overlaid on top of the composed env for this call only. `cwd` defaults to the scratch root. `stdin` is a string written to the child's stdin.
`ocx.env(name)`	`str \| None`	Read one variable from the composed package env. Returns `None` if the variable is not set.
`ocx.target_platform`	`Platform`	Attribute (no parens) — per-run constant. Typed value with attributes `is_any: bool`, `os: os \| None`, `arch: arch \| None`. Reflects the `-p` flag passed to the command (the platform the package was built for), not the host.
`ocx.content_root`	`str`	Attribute (no parens) — per-run constant. Path to the bundle's own files, i.e. what the package ships (read-only).
`ocx.package_root`	`str`	Attribute (no parens) — per-run constant. Path to the materialized package directory holding it (read-only).
`ocx.scratch_root`	`str`	Attribute (no parens) — per-run constant. Path to the writable scratch directory.
`ocx.read_file(path, *, max_bytes=1048576)`	`str`	Read a file within `{scratch_root, content_root}`.
`ocx.write_file(path, content)`	—	Write a file within `scratch_root` only. Parent directories must exist.
`ocx.exists(path)`	`bool`	Check whether a path exists within `{scratch_root, content_root}`.
`ocx.mkdir(path)`	—	Create a directory and its parents within `scratch_root` (idempotent).

ocx.run returns a typed RunResult value with attributes exit_code: int, stdout: str, stderr: str, duration_ms: int, truncated: bool. A non-zero exit code does not raise — the script decides whether to fail. A program that cannot be spawned at all is different: nothing ran, there is no exit code, and the script fails outright. Ask what a bundle contains with ocx.exists, not by spawning and inspecting the failure.

ocx.target_platform is a typed Platform attribute (no parens). The OS / architecture constants live in companion namespaces (ocx.os.{Linux,Darwin,Windows}, ocx.arch.{Amd64,Arm64}). Compare typed values directly:

python

p = ocx.target_platform
if p.os == ocx.os.Linux:
    expect.eq(p.arch, ocx.arch.Amd64)

Full reference for every typed value and every host function is in the Script Host API reference.

ocx.env(name) reads only the composed package env. Host credentials such as OCX_AUTH_* are not readable from scripts.

Read paths are relative and are resolved against the scratch root first, then the bundle's content_root — so a file the package ships at bin/javac is read as bin/javac:

python

if ocx.exists("bin/javac"):
    expect.ok(ocx.run("javac", "-version"))

Path arguments use / as separator on all platforms. Absolute paths and .. escapes are rejected — including an absolute path built from ocx.content_root, which names a location inside the sandbox. Use the relative spelling.

Assertion API — `expect.*`

Function	Purpose
`expect.ok(result, msg=None)`	Assert `result.exit_code == 0`. On failure, the message automatically includes the captured stderr — no boilerplate needed.
`expect.eq(actual, expected, msg=None)`	Assert equality.
`expect.ne(actual, expected, msg=None)`	Assert inequality.
`expect.true(cond, msg=None)`	Assert truthiness.
`expect.false(cond, msg=None)`	Assert falsiness.
`expect.contains(haystack, needle, msg=None)`	Substring check for strings; membership check for lists.
`expect.matches(text, pattern, msg=None)`	Regex match using Rust `regex` syntax. An invalid pattern exits with code 65.
`expect.fail(msg)`	Unconditional failure.

The builtin Starlark fail(msg) is also available.

A typical smoke test looks like this:

python

r = ocx.run("shfmt", "--version")
expect.ok(r)
expect.contains(r.stdout, "v3")

# Verify the package exported the expected env var
expect.true(ocx.env("SHFMT_ROOT") != None, "SHFMT_ROOT should be set")

Sandbox model

The script runs inside a bounded environment:

Writable area: a scratch directory created as a sibling of the package root. The ocx.scratch_root attribute holds its path. Files written here survive --keep.
Read area: the bundle's own files (ocx.content_root) are readable but not writable. The package directory around them is storage structure, not a script contract — a script never has to name it.
Symlink containment: every path is validated for symlink escape after lexical normalization. A symlink inside scratch_root that points outside is refused on access, not just at creation.
Path portability: use / as the separator in all paths — it works on all platforms.

Spawned binaries are not sandboxed

The sandbox applies to the ocx.* host API only — file reads, writes, and path resolution. Binaries launched via ocx.run run with normal host OS privileges, exactly as -- CMD does. A binary can write anywhere the OS allows. This matches the existing trailing-command form and is a documented v1 scope limit.

Re-entrant ocx invocations are refused in v1. ocx.run("ocx", ...) exits with code 1 and a message explaining the limitation.

ocx.run resolves a bare program name the same way the trailing-command form does — against the package first. A name your package ships but cannot execute fails the script rather than running the host's copy of that tool, so a package with a 644 binary no longer passes its own smoke test. Names the package does not ship still resolve on the host PATH.

Output format

Pass --format json to get a structured envelope alongside the exit code:

ocx package test -p linux/amd64 -i shfmt/shfmt:3.8.0 shfmt.tar.xz \
  --script smoke.star --format json

The envelope has three top-level keys — all stable v1 contract:

json

{
  "status": "passed|failed|usage|script_error|io|timeout",
  "assertion": { "kind": "ok|eq|ne|true|false|contains|matches|fail|other|unknown", "message": "…" },
  "run":       { "exit_code": 0, "stdout": "…", "stderr": "…", "duration_ms": 12, "truncated": false }
}

assertion and run are null when not applicable (for example, assertion is null on a passing run). assertion.kind reflects which expect.* function triggered the failure and is the stable machine field for tooling. assertion.message prose is not stable. Exit code remains the primary machine signal.

Exit codes

Code	Meaning
0	All expectations passed
1	An expectation failed, `expect.fail` was called, or a host API returned a failure
64	Usage error — both `--script` and `-- CMD` supplied; neither supplied; script file not found
65	Script syntax, type, or arity error
74	I/O error — stdin read failure (`--script -`), scratch directory creation failure

Editor integration

For .star syntax highlighting in VS Code, add the vscode-bazel extension to your workspace — it provides .star file syntax highlighting.

Exit codes

Code	Meaning
Child's exit code	The command ran; forwarded unchanged
64	Usage error — bad identifier, conflicting flags, or (for `--script`) missing/extra arguments
65	Data error — malformed metadata or script syntax error
74	I/O error — `--output` on wrong filesystem, filesystem failure, or stdin read failure
81	Policy blocked — a local policy (`--offline` or `--frozen`) refused the operation (e.g. digest layer missing and `--offline` set)

Testing locally ​

The problem with push-debug-push ​

Basic usage ​

What the command name resolves to ​

Identifier constraints ​

Keeping the build dir for inspection ​

Writing to a specific directory ​

Testing the private env surface ​

Stripping the parent env ​

Digest layer references ​

The inner pre-push loop ​

Scripted tests ​

Invocation ​

Host API — ocx.* ​

Assertion API — expect.* ​

Sandbox model ​

Output format ​

Exit codes ​

Editor integration ​

Exit codes ​

See also ​

Testing locally

The problem with push-debug-push

Basic usage

What the command name resolves to

Identifier constraints

Keeping the build dir for inspection

Writing to a specific directory

Testing the private env surface

Stripping the parent env

Digest layer references

The inner pre-push loop

Scripted tests

Invocation

Host API — `ocx.*`

Assertion API — `expect.*`

Sandbox model

Output format

Exit codes

Editor integration

Exit codes

See also