MED1stMR

MED1stMR was a European research project focused on mixed-reality training for medical first responders. The project combined physical patient simulators, medical equipment, haptic feedback, and virtual emergency environments to make training for mass-casualty and disaster situations more realistic and repeatable.

Green Manikin Prototype

Within MED1stMR, I developed the first prototype of what later became the Green Manikin. It started as a quick XR demo I hacked together one late night for the Varjo XR-3 headset we had just acquired: a low-poly city where users could stand around, look at buildings, and generally enjoy the novelty of seeing the real and virtual world. As in the other projects I worked on the fun thing is that you can see you own hands embedded in the scene.

That little scene then started to grow. I added a virtual patient avatar created in Character Creator 4, taught him how to breathe, gave him proper eye movement, added wounds, and layered in sound cues. With QR code markers I tracked a real mannequin torso in the room and overlaid the virtual body onto it. The torso had to be green for the first tracking setup, so the earliest version was literally a medical torso wrapped in green tape. It looked rough, but it worked, and the name Green Manikin basically wrote itself.

I also tracked more QR code markers around the physical scene so I could pin virtual user interfaces into the room. For those interfaces I reused the C++ and Dear ImGui tooling I had already built for im.FLUGE and TeleOperationStation, embedded the rendered UI back into Unity, and anchored it in mixed reality. That made it possible to show virtual vital signs like heart rate.

Later versions replaced the taped-together torso with a much better mannequin, now with hands and feet. Even later, Rodrigo Gutierrez added AI-based speech interaction, so the patient could actually talk to trainees. A few years after this began as a headset demo in a low-poly city, the improved version won 3rd place at the Houskapreis 2026 as Green Manikin.

Rodrigo Gutierrez added AI-based speech interaction, so the patient could actually talk to trainees. A few years after this began as a headset demo in a low-poly city, the improved version won 3rd place at the Houskapreis 2026 as Green Manikin.

One of the first images I have of me showing the Green Manikin.

The old mannequin was a bit of a mess, but in 2023 he was replaced with a much more realistic model.

The first video of the Green Manikin moving with random noise and some controls for breathing.