Spark

SparkLang

From .spark to SPARK_BC — then prove it

One reviewable language surface for compile → dump → train → serve → share. Dry-run fixtures in CI. Owned spark-coder stays tiny — no "competitive AI win" claims.

Offline-first: ./spark --dry-run. Live jobs hit SPARK_TRAIN_URL. Five CPU reference methods: spark_distill_cpu, spark_pref_pack, spark_playbook_fit, spark_faq_index, spark_reply_pack.

Download Spark The Spark loop Playground

Why a language?

Library glue scatters prompts, training scripts, and eval harnesses across repos. A .spark program is one reviewable surface — same file for dry-run CI and live ops.

Typical Python SDK soup

client = OpenAI(...)
resp = client.chat.completions.create(...)
# separate train script, eval notebook,
# CI secrets, ad-hoc shell glue…

One .spark program

model train dataset "data.jsonl" base "base" out "out/job-1" backend "http" -> job
model status "job-1" -> status
expect contains job fixture "…/want_accepted.txt"
expect contains status fixture "…/want_succeeded.txt"

# ./spark --dry-run train_eval.spark

The wedge

Dry-run by default

./spark --dry-run uses offline fixtures — no keys, no network. Pass --live only when you want real calls. 

./spark --dry-run examples/model_train.spark

Train backends wired

model train / model build submit via backend "http" to your trainer at SPARK_TRAIN_URL (POST …/jobs, GET …/jobs/{id}). Dry-run uses fixtures; live needs a service that implements the Trainer HTTP contract — in-repo reference: tools/spark-train-ref ships five CPU methods: spark_distill_cpu, spark_pref_pack, spark_playbook_fit, spark_faq_index, spark_reply_pack (real weights.pt / pref_pack.json+ranker / playbooks.json+router / faq_index.json+encoder / replies.json+gate — not LoRA). Captures: live-train-capture.txt, live-train-methods-capture.txt. 

model train dataset "…" base "…" out "…" backend "http" -> job
model status "job-id" -> status

Eval in the same file

After train, assert bound job and status fields with expect contains … fixture (exit 0/1) — gate of this job, not unrelated gateway aliases. 

expect contains job fixture "…/want_accepted.txt"
expect contains status fixture "…/want_succeeded.txt"

Train → status → expect

Dry-run writes fixture artifact paths (no GPU). Live POSTs to the trainer at SPARK_TRAIN_URL (backend "http"), or an allowlisted local-yield unit.

  1. 1 Train Submit dataset, base, and output path
  2. 2 Status Poll the job and bind artifact JSON
  3. 3 Expect assert bound job/status (exit 0/1) 
# examples/train_eval.spark — dry-run first
model train dataset "examples/fixtures/train/dataset.jsonl" base "fixture-base" out "out/train/job-dry-001" backend "http" -> job
model status "job-dry-001" -> status
expect contains job fixture "examples/fixtures/train/want_accepted.txt"
expect contains status fixture "examples/fixtures/train/want_succeeded.txt"

Fail path: examples/train_eval_fail.spark (exit 1). Live HTTP proof (five CPU methods, via) ./spark --live / ./spark-train-http --method): live-train-methods-capture.txt. Model training docs · Build a Model · What's shipping today.

Also included

Beyond the train → status → expect wedge: local spark-analyze, grounded asks with ./spark-ground, voice ask / voice easy, the three-pane IDE (spark-bc-gui), weight gallery, and decompile scoreboard. Details live in the factory hub, language reference, and what's shipping today.

Learn Spark

Follow the trail: install → first dry-run → train→status→expect → gate a job. Field concepts live in the AI knowledge hive.

Try it

# Dry-run train → status → expect — no network, no API key
./spark --dry-run examples/train_eval.spark

# Optional live submit — your trainer at SPARK_TRAIN_URL
export SPARK_TRAIN_URL=http://127.0.0.1:8090/v1
./spark --live examples/model_train.spark

See programming guide and language reference.