Sevan Brodjian

PhD Student, Computation and Neural Systems, California Institute of Technology

Video games feel solid, but if you look closely enough you soon reach the edge of what was built. My aim is for virtual worlds where that edge disappears. Looking closer should keep revealing new structure, in what things are and in how they behave, and the world should hold each discovery persistently once it is revealed. Generative models can invent without end, but they do not reliably track state or hold laws. Simulators hold both, but only for what they were designed around.

I suspect we will develop a mature theory of generative models as a class of simulators, rediscovering many principles of classical simulators in a new neural, latent form. My research investigates how these models represent world state and compute over it, to design better simulation without getting in the way of scalable learning. My recent work found that whether a model’s latents act as a simulator’s state depends more on its training environment than its architecture. It suggests simulation can be designed into these models, as hybrid generative simulators.

To me, world models don’t stop at the screen. Next I want to apply them to more immersive interfaces, using VR, BCI and neuroscience to study what makes a world feel meaningful and how people will author their own. To that end I also currently work with real neural data.