test262 Conformance

zig-js is scored against the real tc39/test262 corpus from the pinned test262/ submodule through the configured runner. Progress is a number, not a vibe, but the number only covers tests the configured runner actually scores.

Sources. Numbers come from docs/.data/test262.json; runner scope (subtrees, skips, exclusions, timeouts) from conformance/test262.zig. Regenerate both with bun run docs:data.

How a test is run

The harness (conformance/test262.zig) walks the corpus and, for each .js file:

1. Reads the frontmatter — the /*--- … ---*/ YAML block declaring flags (raw, onlyStrict, async, module), includes: harness files, and negative: expectations.
2. Assembles the source — prepends "use strict" when onlyStrict, loads the real upstream harness/sta.js + assert.js plus any includes:, then appends the test body.
3. Runs it in an isolated subprocess worker under a 30-second backstop timeout (120 s for a few known-slow sweeps), where a timeout is charged as a host failure. Each worker streams one <index>:<outcome> line per test. If a worker crashes or times out, the parent respawns it just past the failure — so a single segfault or hang can never zero out a run. The parent exits non-zero if any aggregate parse, runtime, host, negative, or skip count is non-zero.
4. Scores the outcome on one of two axes.

Two axes

Valid and negative tests measure different things, so they are scored separately:

• VALID — "can we run it?" pass + parse-fail + runtime-fail + host-fail. This is the headline metric.
• NEGATIVE — "do we reject bad input?" pass-negative + fail-negative.

Skipped tests are excluded from both denominators. The configured runner currently has zero unsupported metadata skips; docs/.data/test262-skips.tsv is still generated as the audit file.

The configured runner also has zero excluded files; docs/.data/test262-excluded.tsv is still generated as the audit file. Two non-normative SpiderMonkey staging files (sm/regress/regress-602621.js and sm/lexical-environment/block-scoped-functions-annex-b-arguments.js) are removed from the configured corpus definition before scoring because their pending Annex B block-scoped-function expectations contradict the normative Annex B tests in test/annexB. They are not skip-list or exclusion-list entries. Proper-tail-call coverage, dynamic-import catch-target behavior, import-defer async-module behavior, plain modules, module+async/top-level-await graph-ordering tests, plain async tests, and CanBlockIsFalse tests are part of the runner where supported.

The denominator is the configured subtree list in conformance/test262.zig; directories not enumerated there are outside the measurement entirely — distinct from skipped (unsupported metadata) and excluded/removed (corpus-definition) files. So 100% means 100% of the configured corpus, not of every path under test262/test.

Current numbers

Failure shape on the VALID axis: 0 parse-fail · 0 runtime-fail · 0 host-fail. Skipped: 0.

Per-suite breakdown

The table below is populated only when docs/.data/test262.json was regenerated from a saved run that includes subtree lines. If the suite list is empty, do not invent rows; rerun or parse a saved transcript with bun run docs:data -- --from run.txt.