Using Unity and Houdini alongside C#, HLSL, and Shadergraph, I made a hand interaction projection mapped playground. Without holding a controller, the user is able to use their hands to interact with objects while the result is projected onto the wall; the scene looks like an extrusion into the wall in real life.

I used a Leap Motion Controller 2 for the hand tracking. I made a script that uses the positions of the player's fingers and hands to determine which interaction the player is executing. The player can make a fist to grab & throw objects, double tap their fingers to create objects, pinch to attract objects, clap to change the color palette, and tap their middle finger create explosions.