Hand Gesture Music Control: Map Both Hands to Sound

Left-Hand Scale Degree Controls

Raise one, two, three, four or five fingers on the left hand to select degrees I through V. Degree VI uses an index-and-little-finger shape. Degree VII adds the thumb to that shape. The scale guide displays the current degree and note names for the selected key.

When major/minor control is assigned to the wrist, tilting the left wrist changes chord world. If you need predictable practice, choose and lock the chord world in Settings instead of relying on wrist angle.

Run a recognition ladder before attempting a progression. Select C, lock Fixed Major, mute the master output and present degrees I through V in order. Continue with the index-and-little-finger shape for VI, then add the thumb for VII. Read the degree indicator after every pose, then briefly close the left hand or move it out of frame between attempts. If only one degree fails, rotate the palm toward the lens or increase finger separation without changing wrist height. Repeat the ladder in reverse and note whether VI and VII remain distinct; those two shapes share the same outer fingers, so a hidden thumb can make VII look like VI. This silent check isolates left-hand recognition from key, chord-quality and audio problems so you do not confuse them during a song.

The selected key changes the note attached to each degree, not the shape itself. For example, degree V points to G when the key is C and D when the key is G. Use the on-screen note name to verify a transposition, then keep the same left-hand pose while switching keys in Settings. Hearing the root move while the Roman numeral stays fixed is a practical way to learn what the degree control represents.

• I: one raised finger
• II–V: two to five raised fingers
• VI: index and little finger
• VII: index, little finger and thumb

Right-Hand Harmony and Expression Controls

Right-hand finger count selects the chord voicing. The available shapes include a basic triad, inversions and seventh-chord colours. Watch the on-screen chord label while learning; do not assume every raised-finger count maps to the same musical note.

Vertical position controls output volume, wrist angle controls low-pass filter brightness and an extended thumb lowers the voicing by an octave. These values are smoothed to reduce clicks and sudden changes.

Calibrate the expression controls while the left hand holds degree I. Start with the basic triad and place the right hand around the middle of the frame; move it upward until the level is comfortable, then keep that height while slowly rotating the wrist from a darker to a brighter filter position. Return the wrist to centre before extending the thumb for the lower octave. The chord label should keep the same degree and quality while loudness, tone and register change. If the quality changes during the sweep, improve finger separation before continuing. This one-chord calibration reveals which motion caused each audible result and reduces accidental jumps when the same controls are combined in a progression.

Theremin Mode Uses Position Instead of Chord Shapes

In Theremin mode, the right hand moves through a continuous pitch range and the left hand sets volume. Finger-count chord mappings no longer choose fixed harmony, so slow vertical motion produces a glide.

This is a webcam-based virtual instrument, not a physical Theremin. It does not sense capacitance around antennas and cannot reproduce the exact field or technique of acoustic Theremin performance.

Make Gesture Controls More Stable

Keep fingertips, palms and wrists inside the camera frame. Avoid overlapping hands, strong backlight and patterned backgrounds that reduce contrast. Hold a new shape briefly so the tracker can stabilize before expecting a musical change.

Use the master mute if a pose produces an unexpectedly loud chord. If recognition remains slow, enable low-performance mode and close other camera-heavy tabs or applications.

Practise the controls in order → · Use the four-track generator →