API and CLI

Helios serves the UI, the annotated feed, and its control API from one local FastAPI process.

HTTP endpoints

GET /feed

Returns multipart/x-mixed-replace; boundary=frame: a continuous MJPEG stream of JPEG frames with boxes, labels, overlays, and the mode banner already drawn in. Frames are paced to target_fps.

When no frame is available yet — or a fatal model error is set — the stream serves a dark placeholder frame rendered with the current status message, so the browser always gets a real image rather than a broken stream. An optional ?frames=N query parameter bounds the stream to N frames.

GET /api/status

{
  "ready": true,
  "error": null,
  "message": null,
  "active_concepts": ["person", "cup"],
  "device": "cuda:0",
  "fps": 29.4,
  "live_counts": { "person": 2 },
  "cumulative_counts": { "cup": 7 },
  "counting": false,
  "mode": "presence",
  "required_ppe": [],
  "zone": null,
  "tiling_enabled": false
}

error is null or one of the three fatal states below, with message carrying its user-facing text. mode is presence, ppe, or zone. zone is null or a four-element array of normalized coordinates. The UI polls this roughly every 3 seconds.

POST /api/chat

Request body, capped at 2000 characters:

{ "text": "anything someone could trip over" }

In presence mode the response carries the mapped concepts plus the prompt history:

{
  "concepts": ["bag", "cable", "shoe", "box", "chair"],
  "history": [{ "prompt": "anything someone could trip over", "concepts": ["bag", "cable"] }]
}

In PPE mode the same endpoint parses a safety policy and also returns the required-PPE keys:

{
  "required_ppe": ["hard_hat", "safety_vest"],
  "concepts": ["person", "hard hat", "safety vest"],
  "history": [{ "prompt": "hard hat and safety vest", "concepts": ["person", "hard hat"] }]
}

Failure responses:

POST /api/mode

{ "mode": "ppe" }

Responds with the new mode and the vocabulary Helios staged for it:

{ "mode": "ppe", "active_concepts": ["person", "hard hat", "safety vest"] }

An unrecognised value returns 400 BAD_MODE with the list of valid modes. Switching modes also resets the temporal smoothers.

POST /api/zone

Normalized [0,1] coordinates; each is clamped into range server-side.

{ "x1": 0.12, "y1": 0.4, "x2": 0.66, "y2": 0.95 }

{ "zone": [0.12, 0.4, 0.66, 0.95] }

POST /api/zone/clear takes no body and responds with { "zone": null }.

POST /api/tiling

Requires a real JSON boolean; anything else is 400 INVALID_TILING.

{ "enabled": true }

{ "tiling_enabled": true }

The count endpoints

POST /api/count/start, /api/count/stop, and /api/count/reset all take no body and return the same shape:

{ "counting": true, "cumulative_counts": { "cup": 7 } }

POST /api/shutdown

Backs the UI's ⏻ Stop button. Responds 200 first, then ends the open MJPEG streams, stops the capture loop (releasing the camera), and exits the process. This exists because long-lived MJPEG connections otherwise leave the server hanging on shutdown.

{ "status": "shutting down" }

The three fatal error states

Helios has exactly three fatal states, each rendered as a full-pane error card with this exact text. There is no degraded path.

MODEL_UNAVAILABLE also covers an unexpected fault inside the capture loop, so readers never keep seeing a frozen "ready" frame. API_KEY_INVALID blocks only the chat pane — the feed keeps running on the starter concepts.

CLI entry points

python -m app.capture --list                    # list cameras as 'index -> device name'
python -m app.vision --selftest                 # live GPU smoke test: load, bind, capture, infer
python -m app.vision --selftest --camera 700    # target a specific camera index
python -m app.vision --selftest --tiling        # route the smoke test through the tiling path

The self-test prints one line per frame with the device, model, tracker, tiling flag, frame shape, and detection count, then a line per detection with its label, confidence, box, and track_id. It is also the way to pre-warm the model-weight downloads.

Pure-logic modules ship with unit tests that need no GPU (pytest). The GPU and camera paths — live inference, the capture loop, the MJPEG feed — are verified by live smoke on the CUDA machine rather than mocked, so that no CPU stand-in creeps into a no-degraded-path design.

Troubleshooting