The pipeline

Overview

The flow, and which package owns each step:

The connective tissue is a COCO file (annotations.coco.json) carrying two extra keys per annotation — review and source — that gate which labels are allowed to train. That gate is the spine of the whole pipeline; step 3 and Internals › the review gate cover it.

1 · Synthesize

Generate images from a text prompt. Two backends: mock (deterministic, torch-free, runs anywhere — renders seeded coil images with exact ground-truth boxes) and gemini (Nano Banana, gemini-2.5-flash-image — a hosted API call, no local GPU, reads GEMINI_API_KEY).

Iterate on a prompt first with a contact sheet:

uv run tensor-factory-synth --backend gemini sample \
  --prompt "extreme macro of a helicoil in machined aluminum" \
  --n 9 --cols 3 --out samples.png

Then generate and (with the gemini backend) auto-label a full dataset in one shot:

uv run tensor-factory-synth --backend gemini dataset \
  --prompt "extreme macro of a helicoil in machined aluminum" \
  --features helicoil --n 500 --out data/

That writes data/images/, data/annotations.coco.json, and a data/label_studio.json of pre-annotations. With --no-label it skips auto-labeling; --skip-errors logs and skips per-image failures instead of aborting a long run.

The mock backend's boxes are exact synthetic ground truth, so they are marked approved on creation and train without review — the fastest way to exercise the whole pipeline with no API key.

For application-matched data, the helicoils example ships batch scripts that condition generation on a real part photo (and synthesize negatives). See CLI › batch scripts.

2 · Auto-label

For the gemini backend, the dataset command auto-labels with GroundingDINO (open-vocabulary detection via transformers), turning your --features into detection phrases. Each box it proposes is written as review=pending, source=groundingdino — an AI guess, not yet trainable.

See what awaits review at any time:

uv run tensor-factory-synth triage --data data/
# images: 500 total, ...
# annotations: 487 total -- 0 approved, 487 pending, 0 rejected (trainable: 0)

The labeler resolves its device cuda → mps → cpu, so it runs on this Mac's MPS and scales out to a CUDA box for bulk runs. It lives behind the gpu extra.

3 · Review (the human gate)

By default, AI-labeled data is not trainable until a human validates it. There are two ways through the gate.

Path A — Label Studio round-trip

Push the candidates (with their boxes as pre-annotations), correct them in the UI, and pull the result back. The pull stamps everything review=approved, source=human — now trainable.

# push candidates into a new Label Studio project
uv run tensor-factory-label push --data data/ \
  --title "helicoil v1" --image-base http://localhost:8081

# ...correct boxes in the UI, then pull them back to COCO
uv run tensor-factory-label pull --project <id> --out data/annotations.coco.json

It reads LABEL_STUDIO_URL (default http://localhost:8080) and LABEL_STUDIO_API_KEY from the environment. The helicoils example bundles examples/helicoils/relabel.sh [DATA_DIR], which brings up a shared Label Studio plus a per-dataset image server so several datasets can be labeled concurrently.

Path B — fast path (skip hand-labeling)

Hand-correcting every box is optional. --allow-unreviewed at training time trains straight on the raw GroundingDINO labels. QC them by rendering a contact-sheet of boxes — eyeball one image instead of clicking hundreds. Use Label Studio only when you want to fix specific bad labels, not as a gate.

3b · Negatives (optional, for present/absent)

A box-only detector always emits a box, even on a part with no helicoil. To let it report absent, synthesize negatives — machined parts with holes but no insert — and add them at training time:

uv run --with google-genai python \
  packages/tensor-factory-synth/scripts/gen_negatives.py \
  --n 110 --out examples/helicoils/images/negatives_pool

Negatives carry no box and are trainable by construction (a known absence — no review gate). At training, --negatives DIR turns on the presence head and trains its objectness toward absent (no class), masking the box loss for box-less images. The box output contract is unchanged.

4 · Train

Fit the tiny detector on the COCO dataset and export an int8 ONNX model:

uv run tensor-factory-train fit --data data/ --out model.onnx \
  --epochs 45 --device mps \
  --negatives examples/helicoils/images/negatives_pool \
  --allow-unreviewed

The essentials:

• Only approved annotations train by default; an all-pending dataset is refused. --allow-unreviewed overrides it.
• Device resolves cuda → mps → cpu; export runs on CPU, then quantizes weights to int8.
• With --val-frac the best-scoring checkpoint is exported — not the last, possibly-overfit epoch.

See CLI › train for every flag, and Internals › the model for how the soft-argmax detector, its learnable gain, and the int8 export actually work.

5 · Run

Run the exported model on CPU from the CLI:

uv run tensor-factory detect --model model.onnx --image frame.png
uv run tensor-factory bench  --model model.onnx

Or serve it. A presence-head model also reports present / class_name over MCP and HTTP:

uv run tensor-factory-mcp                          # stdio MCP, --model defaults to bundled
uv run tensor-factory-http --model model.onnx     # POST raw bytes to /detect

See Getting started › serve and the CLI reference.

Pointing the factory at a new object

The helicoils example is the template. To target something else:

1. Write a prompt that generates your object, and pick --features phrases GroundingDINO can find.
2. Generate + auto-label a dataset (or use the mock backend to prototype).
3. QC the labels — contact-sheet for the fast path, Label Studio for precision.
4. Optionally synthesize negatives for present/absent.
5. Train, export, and serve. Same five packages, same commands.