Almost every “video AI” product is a batch job wearing a real-time costume: upload a clip, wait, get JSON back. That shape is fine for a library of finished footage and useless for a camera, because a camera never finishes. The stream is unbounded, and the moment your processing is slower than the source you are no longer doing perception — you are accumulating a backlog.
The naïve fix is to send every frame to a vision-language model. This does not work twice over: it is far too slow to keep up with a live feed, and the cost is unbounded in exactly the same way the stream is. Argus exists to answer the question that sits between those two failures — how do you get continuous, queryable understanding of a live scene, with bounded latency and bounded cost, on hardware you own?
The answer is to stop treating all understanding as one thing. Argus is a three-tier cascade, and each tier runs at the frequency its job actually needs.
sources ──► Tier 1: always-on perception (real-time, no LLM)
WebRTC detection + multi-object tracking, on-device
RTSP / SRT │
HLS / robot ▼
Tier 2: gated semantic reasoning (~1 fps, triggered)
vision-language model — "what is happening", in words
│
▼
Tier 3: scene state = source of truth
live scene graph · entities · zones · activity
event log is *derived* from state transitions
│
▼
real-time sync ──► dashboard · chat · robot plannerTier 1 is geometry, and it never stops. NMS-free end-to-end object detection with persistent multi-object tracking (ByteTrack), running real-time on Apple Silicon through MLX and Metal. It produces stable identities, motion, and zone or tripwire logic at zero marginal inference cost beyond the local GPU.
Tier 2 is language, and it fires only when something happened. A vision-language model describes the scene in words, triggered by Tier 1 events plus a heartbeat floor so the description never goes stale — never on every frame. The model sits behind an interface, so a local on-device VLM and a cloud model are hot-swappable without touching the pipeline.
Tier 3 is the part that makes the other two useful. A continuously-updated scene graph — entities, zones, activity, semantic notes — is the single source of truth, and the event log is event-sourced from state transitions rather than written alongside them. Entity entered, zone breached, activity shifted. This inverts the usual arrangement, where events are emitted by hand and the state is reconstructed by replaying them, and it means the log and the state cannot disagree.
The governing design rule is drop stale, never lag. Every hand-off between stages is a backpressure-aware, size-one latest-frame slot: a slow consumer sees fewer, fresher frames, never a growing queue. This is a deliberate trade of completeness for latency. Argus will silently discard frames it cannot process in time, and that is the correct behaviour for perception — a robot needs to know what is in front of it now, not what was there four seconds ago. It would be the wrong behaviour for forensic recording, and Argus is not that.
The second trade is running on the edge at all. Inference happens on your hardware; no frames leave the box unless you send them. That buys privacy, predictable cost, and independence from a network, and it costs raw model quality — an on-device VLM is not a frontier cloud model. Making the model layer swappable is how that stays a choice rather than a ceiling.
Everything is built to fail soft. If the semantic layer drops, geometric perception stays live. Sources reconnect with exponential backoff. One stream crashing never takes down another. A perception system that stops perceiving when one component dies is worse than no perception system, because you were relying on it.
Argus runs against a webcam, an iPhone, an RTSP or SRT IP camera, an HLS feed, or a robot's own cameras, and streams live state and events local-first to whatever consumes them — an ops dashboard, a conversational query layer, or a planner. Detectors and VLMs sit behind clean abstractions, so swapping YOLO for a transformer detector is a config change rather than a rewrite.
It is open source under MIT, with a test suite, and it is the project on this list I most want other people to build on: new sources, detectors, VLM backends, and sync targets all plug into interfaces that already exist.