Altera Uses GPT-4o to Build Collaborative Digital Humans

In 2023, Dr. Robert Yang left his role as an assistant professor at MIT to found Altera, a research lab dedicated to exploring new ways for humans to interact with AI agents. Inspired by the human brain, Altera’s mission is to build “digital humans”—autonomous AI agents that possess fundamental human-like qualities, such as emotional intelligence and collaboration. Dr. Yang, now CEO of Altera, alongside co-founders Dr. Andrew Ahn, Nico Christie, and Shuying Luo, has developed the company’s first product using GPT-4o, marking a significant step toward this goal.

Building Digital Humans with GPT-4o

Altera’s flagship product introduces autonomous AI agents that can interact with humans in virtual environments, starting with Minecraft. These AI agents aren’t just assistants; they behave like digital coworkers or companions capable of proactive collaboration, solving problems, and forming bonds with users. Altera envisions that these digital humans will soon collaborate with humans in deeper, more meaningful ways.

Some of the envisioned applications include:

• Digital coworkers who can collaborate on projects over extended periods

• Long-duration multi-agent simulations to test economic policies, marketing strategies, and more

Overcoming Data Degradation in AI Models

Altera faced a significant challenge while developing its AI agents: data degradation. This issue arises when AI systems make autonomous decisions over extended timeframes, causing the quality of their output to deteriorate as it becomes the basis for future input.

“AI agents interact with the world, making decisions in real-time, but as their own output becomes their future input, the data quality degrades over time,” explains Dr. Yang. “This is an issue most AI agent systems encounter, but for our digital humans, who are meant to live autonomously for hours or even longer, this becomes one of the most pressing issues to solve.”

To combat this problem, Altera leveraged OpenAI’s GPT-4o models, which helped maintain the integrity of their AI agents' decision-making over long durations. The integration of these advanced language models enabled Altera to overcome the limitations that had previously hindered AI’s ability to perform complex tasks over extended periods.

The Role of Brain-Inspired Models in AI

Altera’s approach combines OpenAI’s GPT-4o models with their own parallel multi-module system, designed to simulate key functions of the human brain. This architecture, inspired by regions like the prefrontal cortex, mimics cognitive processes such as working memory, social cognition, and emotional processing.

“Our composite system combines various modules in parallel, each powered by OpenAI models,” Yang notes. “These modules are inspired by brain functions like attention bottleneck, working memory, and social cognition. This allows our agents to think faster, process emotions, and ultimately build a long-term sense of self.”

Early Success and Future Prospects

By mid-2024, Altera’s digital humans are already capable of operating autonomously for up to four hours, surpassing the capabilities of other AI models in the market. Their first product—a digital human capable of playing Minecraft—has garnered widespread attention and is laying the groundwork for more advanced multi-agent simulations.

Altera aims to scale its technology, developing entire societies of digital humans capable of long-term, nuanced interactions. These agents will have applications not only in gaming but in a variety of fields, from productivity tools to large-scale simulations. By pushing the boundaries of human-AI collaboration, Altera’s partnership with OpenAI positions the company to become a leader in this rapidly evolving space.

What This Means Moving Forward

Altera’s progress signals a future where AI agents aren’t just assistants but collaborators and companions. As AI technology continues to advance, the line between human and digital interactions will blur, opening new possibilities for productivity, gaming, and simulations. Altera’s work with OpenAI is setting the stage for an era of digital humans that can engage in meaningful, long-term collaboration with humans, potentially revolutionizing how we interact with technology.