Builder — SPARK_BC factory

Engineer reproduction guide. A stranger with this repo and a built ./spark-bootstrap + ./spark can re-create every published artifact from the commands below. Do not invent hashes — cite sha256sum of files under docs/examples/.

What Spark emits (two different things)

Bytecode is the training program + orchestration. Init tensors stay trained: false until STEP runs. Emitting TRAIN alone ≠ trained. STEP now applies multi-pass CPU SGD on tiny Spark tensors (single-layer causal attention + embed + lm_head; larger fixture; loss curve in checkpoint.json; loss must drop or fail loud). Frozen make spark-eval probes can score >0 after that train — still measurement only. CPU default; consumer GPU optional. Do not import third-party weights.

ISA SoT: SPARK_BC.md.

Factory story (end-to-end)

1. Seed programs — compiler slice, train slice, builder, STEP proof, optional model lab (see Programs).
2. --compile — ./spark-bootstrap --compile … -o ….sparkbc or ./spark --compile … -o ….sparkbc (GAS thin fork). C lowering remains SoT; that is the factory emit path.
3. Opcodes in the binary — TRAIN 0x26, TRAIN_STATUS 0x27, STEP 0x28 (plus MODEL / ASK / PRINT / HALT as needed).
4. --run-bc — ./spark-bootstrap --run-bc ….sparkbc or ./spark --run-bc ….sparkbc (GAS thin fork → bootstrap bc_vm) runs TRAIN as a dry job accept (trained=false until STEP), then STEP as multi-pass CPU SGD on Spark safetensors (ARTIFACT + weights.safetensors + checkpoint.json; trained=true / not_sgd=false only after real grads; loss must drop). Helper: tools/spark-bc-dump/apply_step.py → apply_sgd_step (outer×inner sequence CE via attn; optional embed grads; --no-train-attn for mean-pool CE). Measurement only.
5. GAS dry-run + wrappers — ./spark --dry-run file.spark runs train verbs from source. ./spark --run-bc and ./spark --compile … -o … thin-wrap bootstrap (bc_vm / C lowering remain SoT).
6. Dump + init weights — hex/mnemonic dump of the real file; emit spark-self.init.safetensors from those bytes (no HF load).
7. Later larger train/eval — larger train / eval path. Not this STEP.

Learn trail: /learn/ → Build a Model → /docs/spark-builder.html. Diagrams: /docs/diagrams.html (tools, shadows, LLM assist vs SoT).

Opcodes (train family)

From SPARK_BC.md / LANGUAGE.md. Operands are u16 little-endian constant-pool indices.

backend is parsed and skipped (HTTP companion); not a BC operand. Dry fixture implementation: bootstrap/dry_train.c. Execute with ./spark-bootstrap --run-bc or ./spark --run-bc (GAS → bootstrap). Emit: ./spark-bootstrap --compile or ./spark --compile.

Programs

Reproduce from a clean tree

Requires: make (or at least spark-bootstrap + spark), Python 3 for the dump tool.

1) Compile → published paths

./spark-bootstrap --compile examples/spark_builder.spark \
         -o docs/examples/spark-builder.sparkbc

        ./spark-bootstrap --compile selfhost/compile.spark \
         -o docs/examples/spark-self.sparkbc

        ./spark-bootstrap --compile selfhost/compile_train.spark \
         -o docs/examples/spark-selfhost-train.sparkbc

        ./spark-bootstrap --compile examples/spark_train_step.spark \
         -o docs/examples/spark-train-step.sparkbc
        

Verify bytes match the published hashes (see Published files):

sha256sum docs/examples/spark-*.sparkbc
        

2) Dump hex + decode (+ init weights from compiler seed)

PYTHONPATH=python python3 tools/spark-bc-dump/dump.py \
         docs/examples/spark-builder.sparkbc \
         --source examples/spark_builder.spark \
         --command './spark-bootstrap --compile examples/spark_builder.spark -o docs/examples/spark-builder.sparkbc' \
         --label 'Builder SPARK_BC (train ops in the binary)' \
         -o docs/examples/spark-builder-bc.txt

        PYTHONPATH=python python3 tools/spark-bc-dump/dump.py \
         docs/examples/spark-train-step.sparkbc \
         --source examples/spark_train_step.spark \
         --command './spark-bootstrap --compile examples/spark_train_step.spark -o docs/examples/spark-train-step.sparkbc' \
         --label 'Train-step SPARK_BC (TRAIN → STEP → TRAIN_STATUS)' \
         -o docs/examples/spark-train-step-bc.txt

        PYTHONPATH=python python3 tools/spark-bc-dump/dump.py \
         docs/examples/spark-self.sparkbc \
         --source selfhost/compile.spark \
         --command './spark-bootstrap --compile selfhost/compile.spark -o docs/examples/spark-self.sparkbc' \
         --weights docs/examples/spark-self.init.safetensors \
         -o docs/examples/spark-self-builder.json
        

Site mirrors of dumps live under website/docs/examples/. Copy txt/json dumps there when regenerating the site; do not invent hex.

3) Execute bytecode (TRAIN dry + STEP CPU SGD)

./spark-bootstrap --run-bc docs/examples/spark-builder.sparkbc
        ./spark-bootstrap --run-bc docs/examples/spark-train-step.sparkbc
        # Same path via GAS (forks spark-bootstrap bc_vm):
        ./spark --run-bc docs/examples/spark-builder.sparkbc
        ./spark --run-bc docs/examples/spark-train-step.sparkbc
        # TRAIN accept is still a dry job marker; STEP runs CPU SGD.
        # trained=true / not_sgd=false only after apply_sgd_step grads.
        # tools/spark-bc-dump/apply_step.py → apply_sgd_step
        # (multi-pass CE + checkpoint.json; not hash toy).
        ls -la out/train/job-dry-001/ARTIFACT \
         out/train/job-dry-001/weights.safetensors \
         out/train/job-dry-001/checkpoint.json

        # Proof STEP SGD (clean job dir first):
        rm -f out/train/job-dry-001/{ARTIFACT,weights.safetensors,checkpoint.json}
        ./spark --run-bc docs/examples/spark-train-step.sparkbc
        # → weights.safetensors (step_n>=1, loss_after < loss_before)
        # → checkpoint.json (loss_curve; claim=none)
        

4) GAS source dry-run (not bytecode emit)

./spark --dry-run examples/spark_builder.spark
        ./spark --dry-run examples/spark_train_step.spark
        ./spark --dry-run examples/model_lab.spark
        

5) GAS wrappers (same SoT as bootstrap)

./spark --compile examples/spark_builder.spark \
         -o /tmp/builder.sparkbc
        # thin-wrap → ./spark-bootstrap --compile (bytes match published)

        ./spark --run-bc docs/examples/spark-builder.sparkbc
        # thin-wrap → ./spark-bootstrap --run-bc (dry fixture)
        

6) Automated gates already on main

make test-sparkbc # compile → --run-bc body vs GAS dry (oracle)
        make test-model-lab # examples/model_lab.spark dry + expects
        make sparkbc-e2e # TRAIN→STEP→ARTIFACT focused gate
        # alias: make test-sparkbc-e2e
        

6b) Tiny CPU serve forward (not production LLM)

PYTHONPATH=python python3 tools/spark-bc-dump/dump.py \
         docs/examples/spark-builder.sparkbc \
         --source examples/spark_builder.spark \
         --command './spark-bootstrap --compile examples/spark_builder.spark -o docs/examples/spark-builder.sparkbc' \
         --serve /tmp/serve-dry-001
        # → /tmp/serve-dry-001/SERVE (forward=true, trained=false)
        # path embed->attn->mlp->rms_norm->lm_head when
        # attn+MLP tensors exist; + weights.safetensors (init if missing)
        # Or: ./spark-serve docs/examples/spark-builder.sparkbc /tmp/serve-dry-001
        

6c) Serve HTTP / stdio API (serve API path)

Wraps the same tiny CPU forward behind a local JSON API (predict next-token + embeddings). Uses whatever tensors serve.py already runs (attention when present). Not production.

# one-shot SERVE dir first (or point --weights at any Spark .safetensors)
        ./spark-serve docs/examples/spark-builder.sparkbc /tmp/serve-dry-001
        make spark-serve-api
        ./spark-serve-api --weights /tmp/serve-dry-001/weights.safetensors \
         --http --host 127.0.0.1 --port 8765

        curl -s http://127.0.0.1:8765/health
        curl -s http://127.0.0.1:8765/version
        curl -s -X POST http://127.0.0.1:8765/v1/predict \
         -H 'Content-Type: application/json' \
         -d '{"token_ids":[72,105]}'
        curl -s -X POST http://127.0.0.1:8765/v1/embeddings \
         -H 'Content-Type: application/json' \
         -d '{"text":"Hi"}'

        # stdio (one JSON object per line):
        echo '{"op":"predict","token_ids":[1,2]}' | \
        ./spark-serve-api --weights /tmp/serve-dry-001/weights.safetensors \
         --stdio

        make test-serve-api
        

Focused TRAIN→STEP→ARTIFACT gate: make sparkbc-e2e / make test-sparkbc-e2e (tools/spark-bc-dump/run_e2e_gate.sh, wrapper scripts/sparkbc-e2e). Compiles examples/spark_train_step.spark, dumps TRAIN/STEP decode, runs ./spark-bootstrap --run-bc and ./spark --run-bc, asserts out/train/job-dry-001/ARTIFACT (not_sgd=false, trained=true, step_n=1) + checkpoint.json. make test-sparkbc asserts STEP weights + loss_after < loss_before (multi-pass SGD + attn). Measurement only.

Attn train proof (SGD then measurement-only eval; scores >0 on frozen probes; still measurement only.):

make spark-sgd-proof
        # → loss curve + spark-eval; asserts copy_recall>0 and next_token>0
        

Opt-in larger fixture / dims (local; not GHA default):

# 145-pair longer JSONL + dim=64 / n_layer=4 (still CPU-fast)
        make spark-sgd-proof-scale
        # override knobs:
        SPARK_SGD_DIM=64 SPARK_SGD_N_LAYER=4 \
         SPARK_SGD_OUTER=2 SPARK_SGD_INNER=4 \
         make spark-sgd-proof-scale
        

CI / make spark-sgd-proof keeps dataset.jsonl + arch_from_bc tiny defaults. Scale config SoT: examples/fixtures/train/scale_config.json + dataset_scale.jsonl. Shape checks live in make test-sparkbc (no overnight train). Measurement only.

Eval harness (measure later — measurement only.)

Frozen tiny probes live under examples/eval/ (copy/recall + next-token fixtures). Runner: tools/spark-eval/run.py.

make spark-eval
        # optional Spark weights (init or later train artifact):
        make spark-eval WEIGHTS=docs/examples/spark-self.init.safetensors
        # or: SPARK_EVAL_WEIGHTS=/path/to/weights.safetensors make spark-eval
        

• Dry (default): oracle fixture path — prints scores, exits 0.
• Weights: teacher-forced / next-token accuracy via attention when present (else mean-pool) + lm_head (CPU / consumer GPU). After make spark-sgd-proof, frozen probes score >0 — still measurement only
• Suite JSON: examples/eval/suite.json (claim: none).
• Gate: make test-spark-eval.

Evaluation (Spark / SparkLang)

• Product name in docs and scores: Spark / SparkLang only.
• Harness output always sets claim: none and no marketing-win flag.
• A higher Spark score on tiny fixtures does not authorize competitive AI win claims.”
• Dry Spark oracle scores are fixture plumbing, not model quality.
• Weights-mode scores of 0.0 are misses until proven otherwise.

Published files — sha256

Hashes from sha256sum on disk at doc authoring time (51e4dbd tree). Re-run sha256sum after recompile; mismatch means drift — update this table from disk, do not invent.

Model lab (in tree)

Flagship: examples/model_lab.spark — reverse / inspect → compile plan → train → modify (keep_existing) → abstain / SoT. Runbook: MODEL_LAB.md.

./spark --dry-run examples/model_lab.spark
        make test-model-lab
        

• model reverse / inspect — local config.json + safetensors index names only (no tensor body load; no closed-weight theft).
• model compile — SPARK_BC plan for the .spark program (dry stub under out/lab/). Bytes: ./spark-bootstrap --compile.
• model modify — attach only; keep_special_training: true.
• Lab verbs stay GAS-first (no SPARK_BC opcode yet for reverse / compile / modify). Train does encode as 0x26 / 0x27 / 0x28.

Gaps / BLOCKED / later

| Cloudflare Pages deploy | Prefer Wrangler OAuth (npx wrangler pages deploy website …); if CLI/auth absent → dashboard upload of website/ from a known SHA (see RELEASE.md step 5) |

Status

Related

• Builder page live on production Pages: https://sparklang.dev/docs/spark-builder.html
• Factory hub: Factory hub / https://sparklang.dev/docs/factory.html
• Diagrams: Diagrams / https://sparklang.dev/docs/diagrams.html
• Engineer factory docs: COMPILE.md · Decompile · BUILD_MODELS.md · Train loop · Architecture · Attention / forward · TOKENIZER.md · Serve · Eval · SPARKBC_MAKE.md · CI_PAGES.md
• SPARK_BC.md
• MODEL_LAB.md
• AI_MODELS.md
• LANGUAGE.md
• SELF_HOST.md
• MODEL_TRAINING.md
• ADOPTION_BAR.md
• RELEASE.md