I want to develop further on my sound interface experiment.

https://jfz4p.csb.app/

Sound Scrpbook

Libraries used: poseNet, Draggable.js, Tone.js, shaderbooth

• Tokens overlapped - sound is playing together and color changed (red + blue = purple)
• A button that switch play mode - single/duet
• Position of tokens >> change of sound effect
• Handpose / poseNet

  1. Grab a token(following hand)

     Hand: Rock-Paper >> Volume down-up

     Hand: Moving >> position-sound effect

  2. Controll overall sound effect (poseNet)

One hand manipulate tokens by mouse, the other hand manually manipulate one token.

• Buttons to record audio or screenrecord and download.(generate something and let user get something eventually)

Conflicts between two hands controlling could be interesting

choose tokens with two hands: circle, triangle, square. — automatically append to screen

on point of finger - like mouse

one hand - one token

rock-paper >> control volume