Building GaitGuard's on-device digital twin in 24 hours

GaitGuard is a wearable rehab system that gives patients real-time feedback on their movement between physical therapy visits. We built it in 24 hours at YHack 2026, where it placed 3rd overall out of 212 projects and won the "Built with Zed" track.

The pitch is simple: most rehab happens alone, at home, with no one watching your form. GaitGuard watches for you and taps you when you drift.

The interesting part is how it decides what "drift" means.

More: Devpost writeup · GitHub repo · Cornell Engineering feature

Not a threshold. A digital twin.

The naive version of this is threshold-based: pick some angle ranges for a "correct" squat, buzz when the patient leaves them. It doesn't work. A post-surgery knee, a stroke patient, and a healthy 20-year-old have wildly different baselines. A generic population norm is wrong for everyone it's supposed to help.

So we flipped it. Instead of encoding correct form as fixed numbers, we learn it per patient:

1. The physical therapist demonstrates the correct movement while the patient wears the rig, a handful of clean reps.
2. Each rep gets segmented automatically, low-pass filtered at 6Hz, and time-normalized to 100 points so reps at different speeds compare fairly.
3. That becomes training data for a two-layer stacked PyTorch LSTM, a model of this patient's correct form.
4. At runtime the model watches the first part of a rep and predicts what the rest should look like. We compare that prediction against the live sensor readings.

The per-timestep deviation between predicted and actual collapses into a single Gait Health Score from 0 to 100. When it drops below a clinical threshold, knee collapsing inward, not enough range of motion, wrong timing, the haptic motor fires with a pattern that maps to the specific error.

That trained model is the "digital twin": a model of the correct-form self, not an average of strangers. It's also not limited to walking. Swap the demo movement and the same pipeline covers whatever exercise the therapist prescribes.

The hardware

Three wearable units down the leg, each in a custom SolidWorks enclosure printed in PLA on a Bambu X1C:

• Shin (main unit): ESP32 + an MPU-6050 IMU + a 3.7V 300mAh LiPo + the haptic vibration motor. This is the brain of the wearable.
• Ankle: a second MPU-6050 in its own housing, wired to the shin unit through protected sheathing.
• Foot: an ESP32-S3 with a built-in IMU and a 1.69" touch display, on its own battery.

Two of the three attach with velcro for quick on/off. The ESPs do exactly one job, sample the IMU at 50Hz and push raw accel + gyro over WiFi. No ML on the microcontrollers.

Everything lands on a Raspberry Pi 5, which does the real work: sensor fusion, model inference, and a web page for the live sensor preview. We ran QNX on the Pi instead of vanilla Raspberry Pi OS, a real-time OS for predictable interrupt timing, because haptic feedback that lags feels broken.

Joint angles without a Kalman filter

Raw IMU data isn't joint angles. Integrating the gyro alone drifts; the accelerometer alone is noisy. The standard fix is a complementary filter, which trusts the gyro over short windows and the accelerometer over long ones:

# fuse gyro (fast, drifts) with accel (noisy, absolute)
angle = alpha * (angle + gyro_rate * dt) + (1 - alpha) * accel_angle

One line, one tuning knob (alpha = 0.98), and you get stable shin/ankle/foot angles without the overhead of a full Kalman filter. For a 24-hour build that tradeoff is the whole game.

What actually ate the time

The idea was clear by hour two. The rest was the usual hardware tax:

• Two MPU-6050s on one bus. They ship with the same I2C address and fight. The fix (pull AD0 high to get 0x69) is well known and still cost us real time at 2am.
• WiFi latency vs. instant haptics. Streaming at 50Hz over WiFi has variable latency, and the feedback has to feel immediate. Synchronizing multiple ESP32s without dropped packets or timestamp mismatches (which quietly corrupt the joint-angle math) shaped a lot of the timing code.
• Training a model in-situ. Collecting therapist-demo data, training the LSTM, and deploying it back to the Pi inside the same night is legitimately fast-cycle ML. No pretrained weights to lean on, and only a few reps of calibration data to learn from, so the normalization and filtering had to earn their keep.
• Three enclosures that survive movement. Shin, ankle, and foot each needed a housing that fit the body and didn't shake loose mid-squat. The press-fit parts came out right on the first print.

Takeaway

The lesson I keep relearning at hackathons: the model is rarely the bottleneck. The LSTM was the easy part. Getting clean data off two conflicting sensors, over WiFi, through a filter, into inference, and back out to a motor fast enough to feel instant, that's the build. I owned the mechanical side of GaitGuard, the enclosures, wiring, and hardware integration, and it's exactly that layer where the product either works on a real leg or doesn't. Physical AI is mostly plumbing, and the plumbing is where the product lives.

Credits

GaitGuard was built by Jimmy Mulosmani (embedded + Pi pipeline + web dashboard), Samuel Rosen and Paul Trusov (the ML pipeline behind the digital twin), and me, Arsh Singh (mechanical design, hardware assembly, system integration). Built with Zed. The code is on GitHub, and more detail plus the full component list are on the Devpost.