Meta Connect 2024: Orion AR Glasses, Quest 3S, Meta AI Upgrades & More

In this comprehensive article, we explore the highlights of Meta Connect 2024, where CEO Mark Zuckerberg showcased Meta's latest advancements in hardware and software, supporting the company’s ambitions in AI and the metaverse. We previewed some of these in yesterday’s article, but this is the full breakdown with all the exciting announcements from the event.

Orion AR Glasses: A Glimpse into the Future of Augmented Reality

One of the most anticipated reveals was the introduction of Orion, Meta’s prototype for what it claims to be “the most advanced glasses the world has ever seen.” These fully holographic AR glasses are designed with features like hand-tracking, eye-tracking, and a groundbreaking neural interface that allows users to control the device through subtle wrist movements. This interface leverages technology from Meta’s acquisition of CTRL-labs, making it possible to operate the glasses using a wristband that detects neural signals, setting a new standard in augmented reality.

While the glasses are still in the concept phase, Zuckerberg highlighted their potential, mentioning that they have been a decade in the making. Orion utilizes tiny projectors embedded in the temples to create a heads-up display, similar to a futuristic version of Google Glass. The glasses were tested by notable figures such as Nvidia CEO Jensen Huang, but Zuckerberg emphasized that there’s still significant work to be done before they are ready for the consumer market.

The Orion glasses will be initially available to developers, following the path of previous AR and XR devices, and are seen as a successor to Meta's current Ray-Ban Meta glasses. With a neural interface for seamless interaction and the promise of enhanced functionality, Orion aims to revolutionize how we interact with the digital and physical worlds.

Meta Quest 3S Headset: Affordable Mixed Reality

Meta also unveiled the Quest 3S, a more affordable alternative to the Quest 3, starting at $299 for the 128GB version and $399 for 256GB. The headset supports the full library of Quest apps and games and offers a compelling mixed-reality experience. With the launch of the Quest 3S, the price of the Quest 3 will be reduced from $649 to $499. Meta has also announced the phase-out of the Quest 2 and Quest Pro, which will be available until the end of the year or until current stock is sold out. “Quest 3S is the best headset for those new to mixed reality and immersive experiences and for those who might’ve been waiting for a low-cost upgrade from Quest and Quest 2,” Meta

This price drop positions Meta competitively against other headsets like HTC Vive, reinforcing its strategy of capturing market share through accessible pricing. Preorders for the Meta Quest 3S began on Wednesday September 25, with shipping set to start on October 15.

Meta AI Enhancements: A Leap Towards Voice Integration and Visual Understanding

Meta AI is evolving beyond text responses, now offering simulated vocal interactions across platforms like Instagram, Messenger, WhatsApp, and Facebook. Users can choose from multiple voice options, including those of celebrities like Dame Judi Dench, John Cena, Awkwafina, Kristen Bell, and Keegan-Michael Key. These updates mark a significant step towards making Meta AI one of the most widely used digital assistants globally, with nearly 500 million monthly active users.

In addition to voice integration, Meta AI now features advanced visual capabilities powered by the Llama 3.2 models. These enhancements allow users to interact with their photos in innovative ways. Meta AI can analyze images, identify objects, and even answer questions about visual content. For instance, users can upload a photo of a flower and ask Meta AI to identify the species or share a picture of a dish and request the recipe.

Meta AI also enables real-time photo editing through natural language prompts. Users can ask the assistant to make changes such as adding or removing objects, altering the background, or modifying colors. These capabilities extend to generating new images based on prompts directly from social media feeds, bringing AI-driven creativity to Facebook, Instagram, and Messenger.

Additionally, Meta is testing a translation tool for Instagram Reels that can automatically dub a creator's voice into different languages while syncing lip movements. This feature aims to make content more accessible and engaging for a global audience. With these multimodal capabilities, Meta AI is poised to offer a more immersive and interactive user experience across all its platforms.

Llama 3.2: Expanding Meta’s AI Capabilities

Meta announced the release of Llama 3.2, the latest addition to its Llama family of AI models, now equipped with advanced multimodal capabilities. This groundbreaking update enables the model to process both text and images, allowing it to understand, generate, and interact with various types of content. The multimodal functionality opens up new possibilities for developers and users alike, from creating augmented reality apps that provide real-time visual analysis to developing more intuitive virtual assistants that can interpret and respond to visual cues.

Llama 3.2 includes multiple versions optimized for different use cases. The flagship models, Llama 3.2 11B and 90B, boast impressive capabilities, such as interpreting complex charts and graphs, captioning images, and even providing detailed visual explanations. For example, users can share a graph depicting a company’s financial performance, and Llama can instantly identify trends, outliers, and key insights. Additionally, the models can analyze maps to offer detailed information about terrain, routes, and distances.

The smaller versions of Llama 3.2, with 1 billion and 3 billion parameters, are specifically designed to run efficiently on mobile devices, enabling AI-powered applications to function seamlessly on smartphones and tablets. These lightweight models are optimized for Qualcomm and MediaTek hardware, making it easier for developers to integrate sophisticated AI capabilities into portable devices.

Llama 3.2’s capabilities are not limited to understanding visual content. It can also perform intricate image editing tasks based on user instructions, such as removing or adding objects, enhancing image quality, or even generating entirely new visuals. These features, combined with its natural language processing strengths, make Llama 3.2 a versatile tool for everything from content creation to data analysis.

One notable aspect of Llama 3.2 is its potential impact on the development of AI-powered augmented reality devices, such as Meta’s Ray-Ban Meta glasses. By integrating Llama’s vision models, these devices could offer real-time understanding of the user’s environment, enabling applications like visual search, object recognition, and interactive navigation.

As Meta continues to innovate with Llama 3.2, the company is positioning itself at the forefront of multimodal AI research and application, bridging the gap between text, visuals, and real-world interaction. This latest model update represents a significant step forward in making AI more accessible and useful across a wide range of platforms and devices.

Ray-Ban Meta Glasses: Real-Time AI Video and Smart Features

Meta’s partnership with Ray-Ban continues to push the boundaries of smart eyewear. The latest update to the Ray-Ban Meta glasses introduces several cutting-edge features, including real-time AI video processing. This enhancement allows users to ask questions about what they are seeing through the glasses, and Meta AI will provide instant, audible responses. For example, users can ask for information about a landmark or a menu item while looking at it, and the glasses will relay the details directly to them.

The Ray-Ban Meta glasses also support live language translation between English and multiple languages, including French, Italian, and Spanish. This feature enables real-time conversation translation, making it easier for users to communicate with people who speak different languages. Imagine traveling abroad and engaging in seamless, fluid conversations without a language barrier—Meta aims to make this a reality.

In addition to real-time translations, the glasses now integrate with popular music streaming services like Amazon Music, Audible, and iHeartRadio. Users can control their playlists, listen to audiobooks, or enjoy live radio broadcasts directly through the glasses’ built-in speakers, all while keeping their hands free.

The glasses are also equipped with a new reminder feature, where users can ask Meta AI to remember specific visual details, such as an outfit they see or a location they want to revisit. Later, they can retrieve these memories by simply asking the glasses to recall the saved information, making the device a virtual assistant for daily life.

One of the most innovative upgrades is the ability to scan QR codes or phone numbers from the glasses. By simply looking at a QR code and giving a voice command, users can have the link or information open on their smartphone automatically, streamlining interactions between the physical and digital worlds.

The glasses also feature a range of new Transitions lenses that adapt to changing light conditions, making them more versatile for indoor and outdoor use. This enhancement, combined with the device’s lightweight design and sleek aesthetic, positions the Ray-Ban Meta glasses as a stylish yet functional piece of tech that blends seamlessly into everyday activities.

Meta has ambitious plans to further develop the Ray-Ban Meta glasses with additional AI-powered features. Future updates could include advanced visual search capabilities, allowing users to point the glasses at an object and receive detailed information about it, similar to Google Lens. As Meta continues to innovate, these glasses could become an indispensable tool for navigating and interacting with both digital and physical environments.

Understanding the Differences: Orion Prototype vs. Ray-Ban Meta Glasses

While both the Orion prototype glasses and Ray-Ban Meta glasses were highlighted during Meta Connect 2024, they serve distinct purposes and cater to different user needs.

• Orion Prototype Glasses: Designed for a full augmented reality (AR) experience, Orion glasses are built to overlay digital content onto the physical world using advanced holographic displays. They feature a neural interface, allowing users to control the device through subtle wrist movements, which sets a new standard for hands-free interaction. Although still in the prototype phase, Orion aims to revolutionize how we interact with digital and physical environments, making it a promising tool for developers and AR enthusiasts.

• Ray-Ban Meta Glasses: In contrast, the Ray-Ban Meta glasses focus on integrating practical AI features into a familiar, stylish form factor. They do not have a built-in display but provide useful functions like real-time video processing, live language translation, and voice-activated reminders. Designed for everyday use, these glasses are available to consumers and offer a more accessible entry into the world of smart eyewear.

Key Takeaway: While Orion represents the future of immersive AR technology with its advanced capabilities and developer focus, Ray-Ban Meta glasses are positioned as a consumer-friendly, AI-powered accessory that blends technology with everyday wear. This dual approach showcases Meta’s commitment to both pioneering cutting-edge AR experiences and enhancing daily life with intelligent, stylish devices.

Meta Connect 2024: What’s Next?

Meta Connect 2024 highlighted the company’s continued focus on integrating AI and the metaverse into daily life. From groundbreaking AR glasses to affordable mixed-reality headsets, and advanced AI capabilities, Meta is positioning itself as a leader in the next wave of digital innovation.