A CRC tells you the bits arrived intact. It doesn't tell you they're true.

02 — The attack

Every check passes. The rocket is somewhere else.

Source: vbw/link/codec.py · apps/site/public/data/ses9_ppo.json

03 — Detection

Four checks. Only one of them is about the bits.

L0 asks whether the frame arrived intact. L1 asks whether the motion is physically possible. L2 asks whether it matches how this vehicle flies. L3 asks whether it looks like what attackers produce.

Source: vbw/link/codec.py · vbw/detector/physics.py · l2.py · l3.py · alerting.py

04 — Attack ladder

From a dumb offset to an attacker that learned the detector.

Six held-out launches. Each attack starts at 35% of the flight, above the atmosphere, and runs to the end. Latency is the mean time to the first alert. The lie is the mean of each run's largest altitude error before that alert.

Source: results/detector_eval_r2.json

05 — Arms race

Each new attacker hid longer. Each new defender caught both.

PPO round 1 walks 12.3 km past the base detector. Hardened r1 cuts that to 66 s. Retrained against r1, PPO round 2 survives it 2.5× longer (167 s versus 66 s) with a smaller lie. Hardened r2, trained on that output and now shipped, catches both.

The first reward charged the detector's score on every step, so the best policy was to never lie, or to get caught immediately. Charging only for score above a margin, and making an alert much more expensive than a slow lie, is what produced an attacker that actually lies. A policy that was also allowed to nudge downrange was caught in seconds by the tightest rule, so the shipped agent lies about altitude and the velocities that go with it.

Source: results/arms_race.json

06 — Replay

Scrub SES-9. The shaded gap is the lie.

Holdout launch, seed 0, the same frames the detector scores. PPO is armed at T+190 s. Bias, for contrast, is a 3 km step at T+60 s. This is one flight, not the six-launch mean.

This replay: base alerts at 315 s with a 23.8 km lie; hardened alerts at 34 s with a 0.61 km lie. Onset is T+190.0 s, the live demo's arm time. The six-launch eval starts each attack at 35% of that flight and records SES-9 base 315 s / 23.8 km and hardened 34 s / 0.6 km. Alert marks are the persisted decision, not the first sample over τ.

Downsampled from 10 Hz to about 2 Hz. Alert edges are kept. Arrow keys move the slider. Drag a chart. 1,099 samples from 5,450 frames.

Source: apps/site/public/data/ses9_ppo.json · ses9_bias.json

07 — Results

Held-out launches. Zero false alerts. The cost of a frame.

• Mean latency is over the 5 launches that alerted. Base never alerted on JCSAT-16.
• Mean latency is over the 3 launches that alerted. Base never alerted on SES-9, JCSAT-16, IRIDIUM-NEXT-5.
• Mean latency is over the 3 launches that alerted. Base never alerted on SES-9, JCSAT-16, IRIDIUM-NEXT-5.

Display rounding: latency to 0.1 s, then to the nearest second from that tenth once it is at least 20 (40.5 rounds to 41). Lies of 2 km or more to 0.1 km; smaller lies to 0.01 km. Freeze is 0.52 km here; the README prints 0.5 km. Unrounded values stay in the JSON files named under each section.

Source: results/detector_eval_r2.json

08 — Engineering

A laptop GPU, a 30-microsecond core, and a benchmark we lose.

TensorRT is a fleet benchmark, not the live path

The demo scores one vehicle on ONNX Runtime, on CPU. NVIDIA GeForce RTX 5060 Laptop GPU, TensorRT 11.4.0.106, plugged in. FP16 scores 16,384 streams of L2 and L3 together in 15.9 ms, inside a 20 ms frame at 50 Hz: 35× on L2 and 8.7× on L3 versus the CPU, with 0 detection decisions changed. INT8 (minmax calibration) is slower than FP16 on these small models and flips 3.0% of L3 decisions. On battery the same FP16 pair takes 21.5 ms at 16,384 streams and 6.0 ms at 4,096. One stream at a time the GPU loses: L2 is 567.8 µs on FP16 versus 57.2 µs on CPU, and L3 is 239.5 µs versus 38.9 µs.

C++ edge monitor

The base detector (L1 + L2, not L3) reimplemented in C++. Bit-exact with Python on 16,766 frames: max absolute error 0, alert-frame mismatches 0. 30 µs per frame p50 (L1 3.1 µs, L2 26.4 µs), 32,550 frames/s on one core. The Python reference in that bench is 1.19 ms/frame, about 40× slower. Not measured yet: live UDP end to end (scripts/edge_eval.py with vbw_edge.exe + scripts/udp_replay.py); test_l1.cpp per-rule parity.

NASA SMAP / MSL

82 labeled streams, 81 unique channels, 105 labeled anomalies. Event-level scoring, the Telemanom protocol (Telemanom (Hundman et al., KDD 2018)). Combined we are 10.8 F1 points below the paper. That shows the learned layer is a working anomaly detector. It is not a state-of-the-art claim.

Hundman et al., KDD 2018. We do not quote point-adjusted F1. It is in the JSON and it is optimistic.

Tests and calibration

221 Python tests collected in the default environment. The TensorRT module (FP32 and FP16) is skipped unless those packages and a GPU are present. 75 console test cases. 3 C++ suites: frame, config, and detector parity. τ = 0.66 was set from clean training and validation flights only. The test half never moved a threshold.

Source: results/trt_bench.json · results/edge_bench.json · results/bench_summary.json

09 — Architecture

From a real launch to a spoken alert.

Sources are encoded, a man-in-the-middle may rewrite them and re-sign a valid CRC, and four layers judge the frame. PPO drives the attacker and, after training, the discriminator.

• Live path. Python 3.12, ONNX Runtime on CPU, FastAPI and a WebSocket.
• Attacker. Scripted ladder plus Stable-Baselines3 PPO, two rounds.
• Console. Next.js mission console. Cartesia speaks alerts when a key is set.
• Store. Tiger Data / TimescaleDB hypertables and a one-second aggregate. Verified live.
• GPU. TensorRT FP16/INT8 bench for L2 and L3 only. Not in the live demo.
• Edge. C++ base detector, bit-exact on the golden frames. Live UDP not yet measured.

Diagram of the README "How it works" flow. No Kubernetes. Vultr was not deployed.

10 — What actually ran

Sponsor tech, only where it is true.

Source: docs/INTEGRATIONS.md

11 — Limits

What this does not show.

The interesting claims are next to the places they fail. These are the ones we will not sand off.

12 — Team

Rajeev Hotlani and Kapilan Karunakaran.

Cal Hacks 13.0.

• Rajeev Hotlani
• Kapilan Karunakaran
• github.com/kapilank1/valid-but-wrong
• Devpost — URL not added yet
• Demo video — URL not added yet