Camera off · on device
A curious little thing that lives in your computer and looks at you through the camera. It follows your face, your hands and your body, guesses your mood, and names the things you hold up. Everything runs in your browser: the picture never leaves your device.
- Tap
- study what's there, up close
- Drag
- draw a box around something to study
- Hold it up
- keep a thing still for two seconds and it takes a closer look by itself
How it’s built
Six MediaPipe models take turns on the camera frames in two workers, a quick one and a slow one, and Segment Anything takes a closer look in a third. The page itself only draws. Nothing leaves your device.
- Senses that take turns. Silhouette, face, hands, body, things and names each have their own rhythm. The worker gets a new frame only when it’s done with the last one, so a slow phone sees less often but the picture keeps its pace.
- A closer look. SlimSAM through Transformers.js on WebGPU, in a worker of its own. A phone, or a browser without WebGPU, gets MediaPipe’s Magic Touch there instead: lighter, a little rougher, and it leaves the GPU to the screen.
- One look, then follow. SAM studies a thing once. After that the object detector does the following, and the mask moves and scales with its box.
- Two ways to name things. The object detector knows 80 everyday things. An ImageNet classifier with 1,000 labels guesses what’s in your hand and what SAM just studied, with a question mark.
- A mood, guessed. From the face’s 52 blendshapes, smoothed, with a few sillier guesses on top. On purpose a bit off.