Make Music with a Gesture Music Generator

How the Gesture Music Generator Works

In Gesture mode, the left hand selects degrees I through VII. The right hand chooses a triad, inversion or seventh-chord variation, while hand height sets volume. Wrist angle opens or closes the filter and an extended thumb lowers the harmony by an octave.

The generator turns those controls into oscillator frequencies in real time. It does not upload camera frames, listen through the microphone or generate a song on a remote AI service. Sound is synthesized locally after you press Start.

• Hold one to five left-hand fingers for degrees I through V.
• Use the left index-and-little-finger shape for degree VI; add the thumb for VII.
• Raise or lower the right hand for volume and tilt the wrist for filter tone.
• Keep both palms separated from the background for steadier tracking.

Record and Edit a Four-Track Gesture Loop

Choose a tempo, bar count and time signature before recording. A one-measure count-in gives you time to place both hands, then recognized notes are quantized to the visible step grid. When the loop wraps, recording stops and playback continues.

Each track has independent volume, mute and solo controls. Select an empty track for a second layer, or edit the grid directly when a camera gesture was recognized on the wrong step. The loop stays in the current tab and is not saved to an account.

Run a Repeatable Four-Bar Generator Experiment

Before layering a full arrangement, make one controlled four-bar test. Select C major, 4/4 time and a slow tempo, then use only degrees I and V with the basic right-hand triad. Listen for an even attack and watch the recognized chord label before each change. This separates hand-recognition errors from arrangement decisions: if the label is correct but the sound is too quiet or dull, adjust hand height or wrist angle instead of changing the finger count.

Record the same two-chord pass on track one, play it back, and inspect the step grid. Correct a misplaced event in the grid rather than recording the entire pass again. Add track two only after the first loop wraps cleanly. A useful second layer is the same degree pattern with a first inversion or a lower octave, because the timing stays comparable while the harmony changes. Export the short test and play the downloaded file before beginning a longer composition.

• Change one control at a time so the audible result has a clear cause.
• Confirm the recognized chord label before judging the synthesized sound.
• Use grid editing for an isolated timing mistake and rerecord only when the performance itself needs work.
• Verify one local export early instead of waiting until a four-track arrangement is finished.

Export Music Without Uploading a Performance

Select Export after the loop contains at least one event. Supported browsers render a WAV file locally. If offline WAV rendering is unavailable, the app attempts a WebM audio fallback and explains the result in the interface.

Export contains synthesized audio from the loop grid. It never contains webcam video or microphone audio because the app requests video-only camera permission and does not request a microphone track.

What This Generator Can and Cannot Do

GestureSynth is a playable browser instrument and loop builder. It does not create complete AI songs from text, separate stems from an uploaded recording or import commercial tracks. Tracking quality also depends on lighting, camera angle and device performance.

For a predictable first result, use front-facing light, keep both wrists in frame and hold each gesture briefly. Enable low-performance mode in Settings if the camera view is smooth but landmark updates lag.

Learn every hand control → · Follow the beginner tutorial →