Agent Draw creates and updates TLDraw canvases from voice descriptions

The Neuron · 3 days ago

Agent Draw uses voice descriptions to automatically create and update drawings on tldraw, an infinite-canvas SDK, while a user presents by speaking descriptions into rectangular areas they define. Claude Opus 3.5 demonstrated the strongest results, producing a fully realized cricket scene with the pen tool, while smaller models like Claude Haiku and Gemini Flash settled for simpler primitive shapes. The implementation queues multiple capture requests, transcribes speech via Mistral's Voxtral model, and removes unnecessary agent actions like camera repositioning and shape-overlap review to reduce model calls by roughly half per drawing.