· Updated
GWM-1 (General World Model) to process live visual feeds. Characters can now see and respond to what is happening on a user's screen or webcam.This shift moves AI agents beyond text and audio into the visual domain. By integrating vision into the agent loop (the iterative cycle of observing, reasoning, and acting), Runway enables collaborative AI. Agents can now provide context-aware guidance based on what they observe in the physical or digital world.
You can implement these features using the updated React SDK, which includes built-in components like AvatarCall and ControlBar. New hooks, such as useLocalMedia, allow for programmatic control over streams. The capability is available now via the Runway API for building interactive demos, games, or design tools.
Runway Characters now supports camera and screen sharing. Users can share a live feed and the Character will see, understand and respond to what's on screen. All in real time via the Runway API. https://t.co/c5OX4lizuP
More like this
Runway18h ago
PikaApr 30
OpenAIMar 15
HeyGen18h ago