NVIDIA says it created a real-time generative surgical simulator

NVIDIA introduced Cosmos3-H-Dreams, an interactive generative simulator specialized in cholecystectomy. The company says it was trained on video recordings and kinematic data from clinical procedures performed on CMR Surgical’s Versius platform and is, to its knowledge, the first real-time generative surgical simulator trained on such recordings to support live kinematic control.

Built on Cosmos 3 and accelerated by the FlashDreams library, the system runs interactively on a single NVIDIA RTX PRO 6000 GPU. The model uses 28 action dimensions, including pose, orientation and gripper commands for both robotic arms, along with energy controls and joystick inputs. NVIDIA notes that visual realism does not prove biomechanical accuracy: the simulator must also respond coherently to instrument movements while preserving anatomical and tissue consistency over time.