Jatagan · Engineering Note

Jatagan Filter — second-stage AI verification

A YOLOv8 model on a Hailo-8 accelerator that decides whether a camera's tripwire alarm deserves a human's attention, before an email reaches the monitoring agents.

Status running against four cameras, verifying real events Hardware Hailo-8, 26 TOPS · yolov8x Latency ~8 s from crossing to email

The problem

The camera is not lying. It is answering an easier question than the one we care about.

A Dahua camera with IVS rules reports CrossLineDetection whenever something crosses a tripwire. It reports it for shadows sweeping across pavement, for headlights at night, for rain, for branches moving in wind. Every one of those becomes an email, and every email costs a monitoring agent the seconds it takes to open it, look, and dismiss it. Enough of them and the alerts stop being read.

The obvious fix is to run object detection on the alarm snapshot: if there is no person in the frame, drop it. That removes the shadows and the rain. It does not remove the most common false positive, and that is the whole reason this project has the shape it does.

Why one frame is not enough

Presence and crossing are different questions, and only one of them fits in a photograph.

So the field agent does not send a snapshot. It sends a window: about two seconds before the event and two after, sampled at 4 fps. The office side detects objects in every frame, links them into tracks, and tests whether a track's path actually intersects the rule geometry.

Architecture

Neither end can accept inbound connections, so both dial out.

Why not MQTT, and why not object storage

MQTT is the right tool for telemetry — trailer alive, camera down, temperature — and it will likely carry exactly that later. It is the wrong tool for a megabyte of JPEGs per event: large payloads work against the protocol, and it means operating a broker for something HTTP already does.

Object storage as the primary route (upload to R2, have the office poll) adds a hop and its latency to a path whose whole purpose is getting an operator's attention quickly. The failure it protects against — office unreachable — is covered more cheaply by a disk-backed retry queue on the trailer. Storage remains attractive later as an archive, not as transport.

The event package

What the office needs in order to rule on a crossing.

event.json        rule geometry, timestamps, what the camera itself claimed
frames/0000.jpg   analysis window, ~4 fps, 640 px long side, q80
hd.jpg            full-quality frame, for the operator's email

Analysis frames are downscaled deliberately. The network resizes everything to 640×640 regardless, so shipping 4K over a metered link buys pixels that are thrown away before inference. Only the operator's frame keeps full quality. Rule geometry travels in Dahua's fixed 8192×8192 space, exactly as the camera config reports it, so it stays valid at any stream resolution.

The verdicts

Each answers a different question. Every one of these exists because a real event exposed the need for it.

Crossing is tested at the object's ground point — the bottom-centre of its box, where feet meet the floor. Using the box centre would register a tall person as having crossed while they are still on the near side.

Resolution: why more pixels did not help

The most expensive lesson of the first morning, and the least intuitive.

The filter was missing real people. A camera reported a person crossing, we detected nothing, and the evidence photo looked empty — so the verdict was recorded as the camera hallucinating. It was not. Someone who knew the site looked at the same picture and saw the person immediately.

The obvious response is to send higher-resolution frames. It does nothing. The detector resizes whatever it receives to 640×640 before inference, so a 1080p frame and a 480p frame arrive at the model identically, and a distant person shrinks back to a handful of pixels either way. Measured on the same scene, the substream at 640, the substream at native size and the main stream at native size all returned exactly the same detections.

What works is cropping instead of scaling. The event tells us where the camera saw something; the agent cuts that region out of the main stream at native scale and ships it alongside the frames. The crop is small in bytes precisely because it is small in area, and the person inside it is no longer 45 pixels tall but around 125.

One consequence worth planning for: raw frames at 1080p cost about 6 MB each, and an eight-second buffer per camera became 750 MB. Four agents exhausted the machine's paging file. The buffer now retains 5 fps instead of 15 — every frame is still read so the decoder stays current, but only the ones we would actually ship are kept. Four cameras now run in 1.5 GB and under 6% CPU.

Worked examples

Six real events from the first morning: what the trailer sent, what the office concluded, and whether it was right. Amber is what the camera claimed; green is what the Hailo found; red is the rule.

The accelerator, and what it cost to get working

Everything below was discovered the hard way and is written down so it is discovered only once.

Measured

The verdict logic is covered by a synthetic test that pans a still image to fabricate motion. It asserts both outcomes that matter: a moving object is confirmed as crossing, and a stationary one beside the line is rejected with the "never crossed" reason. Both pass.

Where the eight seconds go

Most of the delay is a deliberate wait, not slowness. Knowing which part is which is what makes it safe to leave alone.

The three seconds spent building the package are the price of the whole design. After an event arrives, the agent waits two more seconds so the frames following the crossing land in the buffer. Without them there is no trajectory, and without a trajectory we are back to being unable to tell someone crossing a line from someone walking beside it. Cutting that wait would make the system faster and worse.

Two of the remaining stages are recoverable without touching decision quality: the SMTP session is rebuilt on every send rather than held open, and there is a half-second safety margin after the window that could be trimmed. Together they would bring this closer to five or six seconds. That work is deferred on purpose — eight seconds from a crossing to a verified email with a photograph is comfortably inside what monitoring needs, and the camera's own unverified alerts do not arrive faster.

Components

Dependencies were held to numpy and opencv on purpose. After the fight with a missing C compiler, a server that needs nothing installed is one less thing to break at three in the morning.

Open questions

Thresholds are untuned. Confidence sits at 0.30 with yolov8l — more sensitive than it will end up, chosen deliberately after the missed person. Extra detections can be discarded by the trajectory test; something never detected cannot be recovered. The server archives every rejection and now the raw event package too, so thresholds can be raised against recorded evidence instead of intuition.

Verdicts are only spot-checked, and the checks themselves have erred. Of the first events, six were audited against their images. Five verdicts were correct. But two of those five were initially recorded here as failures, and one genuine failure was initially recorded as a success — each corrected only when the site operator, who knew what had actually happened, contradicted the write-up. Reviewing a proper sample is the next real work, and it should be done by someone who knows the location.

Deduplication is not built yet. When it is, it must key on the track or the event — never on position. Position-based dedup was already tried on a related system and it silently discarded legitimate new arrivals.

Night performance is unknown. Every measurement so far is daylight or synthetic. IR illumination, motion blur and rain all change detection confidence, and the thresholds that work at noon may not hold at 3am.