Modes

Helios has three explicit, presenter-selected modes. The mode is a visible segmented switch — never auto-classified, because predictability beats magic in a live demo — and is drawn server-side as a banner on the frame itself, so it reads correctly on a projector even with the browser chrome off-screen.

All three modes reuse the same YOLOE open-vocabulary detection plus Ultralytics tracking. The two newer modes are pure post-inference logic over the detection list; there is no extra model path.

Presence (default)

The original demo: chat in plain language for the concepts to detect, with live per-label counts and cumulative line counting. The count overlay and count controls appear in presence mode only. See Object counting.

The chat placeholder reads "What should I look for?", and the empty-state hint suggests prompts like "anything someone could trip over" or "things on the desk".

PPE compliance

Type a safety policy in chat ("hard hat and safety vest"). Claude parses it into a required-PPE set; Helios detects people and PPE, associates each PPE box with a person, and flags anyone missing required gear:

• Green box — compliant: all required gear was found on that person.
• Red box — violation, labelled with the missing item ("no hard hat").

In this mode the chat placeholder becomes "Type the safety policy (e.g. hard hat and vest)", and the active bar is labelled Required PPE.

The supported catalog

Chat maps free text onto exactly these three items — nothing else:

PPE that the catalog does not support is omitted, even if your policy mentions it. A policy with no supported items returns an inline chat error rather than a degraded guess.

Helios never asserts compliance without a policy

Entering PPE mode with no policy set applies the default required set — hard_hat and safety_vest — so the mode always enforces a real policy out of the box.

If the required set is empty (for example, deliberately cleared), Helios does not mark everyone "OK". It draws neutral PERSON boxes and an explicit banner:

PPE COMPLIANCE - NO POLICY SET

That is the only correct behavior for an empty policy: no policy means no compliance claim.

The person-association heuristic

Each PPE box is bound to a person when the PPE box center falls inside that person's matching body band — a head band for hard hat and glasses, a torso band for the vest. Bands are fractional offsets from the top of the person box (0.0 = top, 1.0 = bottom):

A match also requires the PPE center's x to lie within the person box's x range. This is deliberately simple for a one-or-two-person scene: any matching person satisfies the item, and there is no global exclusive assignment, so one PPE box can satisfy more than one person. The band fractions live in app/modes.py (HEAD_BAND / TORSO_BAND).

Danger zone (mouse-drawn)

Switch to Danger Zone and drag a rectangle directly on the live feed to draw a no-go region — a digital light curtain. The region is drawn server-side onto the frame as a translucent red rectangle with a NO-GO ZONE chip in its top-left corner; the border and fill intensify while someone is inside.

• A person whose feet point — the bottom-center of their bounding box — is inside the region is flagged red and labelled INSIDE.
• Everyone else is labelled safe.
• Clear Zone removes the region.

Feet point, not box overlap: a person is "in" the zone when they are standing in it, not merely when their box clips its corner.

Chat is disabled in this mode — the region is drawn, not typed — and the composer is replaced with the hint "Draw a no-go region on the feed". This mouse-drawn region is a deliberate departure from the otherwise chat-only input model, for danger-zone mode only.

Coordinates are normalized

The rectangle is transported as normalized [0,1] coordinates, computed against the rendered image area under object-fit: contain (so it is letterbox-aware). That makes it resolution-independent: the backend converts it to pixels against the live frame every frame, so the drawn box lines up with your mouse drag at any feed resolution. Coordinates are clamped into [0,1] server-side, and a zero-area rectangle is rejected.

If the drawn rectangle does not line up with the drag, the mapping ran before the feed image had natural dimensions — wait for the first frame to show, then draw again.

Stable, non-strobing state

Both PPE and danger-zone modes judge state over time, not per frame, so a violation or intrusion reads cleanly on a projector instead of strobing. A rolling-window majority vote per track ID decides the displayed state:

• PPE keys the window on a (track_id, item) pair; the zone keys it on the track_id.
• A brand-new track reports its first sample immediately (fast onset).
• Once the window fills, a lone dissenting frame cannot flip an established state.
• A person without a track ID falls back to their instantaneous state — the same tolerance counting uses.

Smoother state is owned by the capture thread and is reset on every mode change, so stale history never bleeds across modes, and reset whenever the zone is set, cleared, or redrawn, so a prior "inside" history cannot keep a red box up after the zone moves out from under someone.

The window is tunable per machine — see Temporal smoothing.