World Labs Launches AI to Build 3D Worlds with Spatial Intelligence

World Labs, a spatial intelligence company co-founded by renowned AI researcher Fei-Fei Li, is on a mission to revolutionize how AI models understand and interact with 3D environments. Fei-Fei Li, often called the "Godmother of AI," has teamed up with experts Justin Johnson, Christoph Lassner, and Ben Mildenhall to launch the company, which aims to bridge the gap between AI's current 2D capabilities and the complex spatial relationships of the 3D world.

The Concept of Large World Models (LWMs)

At the heart of World Labs' mission is the development of Large World Models (LWMs), a new class of AI models designed to perceive, generate, and interact with 3D environments. LWMs move AI beyond static, 2D imagery, enabling it to model real-world physics and complex spatial relationships. By integrating both physics and semantics into these models, World Labs is pushing AI to reason about objects, places, and interactions within 3D space and time.

World Labs believes that artificial intelligence will help humans build better worlds. The progress in AI has been rapid, but they have only scratched the surface of what’s possible in the generative AI revolution. To advance beyond current capabilities, spatially intelligent AI is needed to model and understand the world in three dimensions. The potential applications are vast, from improving real-world environments to enabling more efficient and dynamic design processes in various industries.

What is Spatial Intelligence?

Defining Spatial Intelligence and Its Importance

Spatial intelligence allows us to visualize, create, and interact with the physical world. It helps us bring the pictures in our mind’s eye into reality, whether we’re designing a towering skyscraper or constructing a simple sandcastle. We use it to reason, move, and invent, providing the foundation for everything from navigation to architecture.

In AI, spatial intelligence will enable machines to not only recognize objects but also understand how these objects relate to each other in space. This is crucial for AI to interact with physical environments in a meaningful way, whether it's simulating the real-world effects of a design or assisting with complex, real-world tasks such as urban planning, manufacturing, and autonomous vehicles.

Applications and Real-World Use Cases

Empowering Professionals with 3D Modeling and Spatial Intelligence

World Labs is initially focusing on generating limitless 3D worlds, creating and editing virtual spaces that incorporate physics, semantics, and control. These models are expected to unlock new capabilities for creative professionals such as architects, designers, developers, and engineers, allowing them to simulate real-world conditions and create dynamic, interactive environments.

In real-world applications, spatial intelligence in AI can have a transformative impact. For instance:

• Autonomous Vehicles: Spatially intelligent AI can help self-driving cars understand and navigate their surroundings in real time, accounting for the movements of other vehicles, pedestrians, and environmental factors.

• Manufacturing and Robotics: In industrial settings, AI with spatial intelligence can optimize assembly lines, manage inventory, and operate complex machinery, all while maintaining real-time spatial awareness of its environment.

• Urban Planning and Construction: AI models that understand 3D spaces could revolutionize city planning by simulating how different layouts would affect traffic flow, energy usage, and even social interaction in newly designed urban areas.

• Healthcare and Surgery: In the medical field, AI-powered 3D modeling can assist in planning complex surgeries, providing surgeons with highly detailed, spatially accurate representations of the human body.

While these real-world applications are being developed, AR/VR remains an important focus for World Labs, offering immersive experiences in gaming, design, and virtual collaboration. However, the broader goal is to use spatial intelligence to drive innovation across a range of industries and day-to-day environments.

World Labs' Ambitious Mission

From Research to Product Development

World Labs’ founders are not only visionaries in AI research but also experts in systems engineering and product design. Their approach is to create a tight feedback loop between their research into spatially intelligent foundation models and the products they develop, empowering users to leverage these innovations in practical ways.

Building Toward the Future of AI

World Labs will continue to develop spatially intelligent Large World Models (LWMs) that can reason about the 3D world from images and other modalities. Over time, the company expects to train more powerful models with broader capabilities that can be applied in a variety of fields, working seamlessly alongside humans. This focus on spatial intelligence is expected to open new doors for creativity, problem-solving, and innovation.

Bridging the Gap: From Text-Based AI to Spatial Understanding

Moving Beyond Language Models

Much of today’s AI technology relies on language models that understand and process text-based information. However, World Labs recognizes that AI must go beyond this, especially in areas where spatial reasoning is key. By bridging the gap between 2D perception and 3D understanding, World Labs is paving the way for AI that can truly comprehend and interact with physical environments.

In real estate, for example, AI could simulate how a building will interact with its surrounding environment over time. In supply chain management, AI could dynamically model warehouse layouts and predict potential disruptions in inventory flows. These examples highlight how spatially intelligent AI can offer new insights and solutions for industries that depend on physical space.

The Road Ahead: Unlocking the Full Potential of AI

Innovating with LWMs for Real-World Impact

World Labs is at the forefront of transforming AI through spatial intelligence. By training increasingly sophisticated LWMs, the company is working to unlock new possibilities for AI to assist humans in everything from creating virtual worlds to solving complex, real-world problems.

A Partnership for Progress

With over $230 million in funding and the backing of visionary investors, World Labs is set to push the boundaries of what AI can achieve. With their funding being led by Martin Casado and Sarah Wang of Andreessen Horowitz, Scott Sandell of NEA, and Jordan Jacobs of Radical Ventures, World Labs has attracted significant attention from top investors. Other contributors include Marc Benioff, Geoffrey Hinton, Reid Hoffman, Andrej Karpathy, Sound Ventures co-founder Ashton Kutcher, Eric Schmidt, Ram Shriram, as well as Adobe Ventures, AMD Ventures, Databricks Ventures, NVentures (NVIDIA’s venture arm), and Shinrai Investments LLC.

The support of figures like Marc Benioff, Geoffrey Hinton, and Reid Hoffman highlights the confidence in World Labs’ mission to transform AI’s interaction with the world.

Conclusion: Ushering in a New Era of AI

World Labs’ development of spatial intelligence marks the beginning of a new chapter in AI. By focusing on real-world applications and 3D modeling, World Labs aims to bring AI closer to human-like understanding of space and time. These innovations will not only reshape industries like autonomous driving, healthcare, and urban planning but also push the boundaries of what’s possible in AI.

To follow World Labs’ progress and stay updated on their journey, connect with them on LinkedIn and X.

World Labs is actively growing its team, with several open roles available for those interested in joining their innovative journey.