Apple Unveils Visual Intelligence for Instant Object Recognition

Apple has unveiled a new AI feature called Visual Intelligence, which will be part of the Apple Intelligence suite in iOS 18. The feature aims to help users quickly identify and learn about real-world objects, similar to offerings from Google Lens and other AI systems.

How Visual Intelligence Works

Visual Intelligence allows users to “instantly learn about everything you see,” according to Craig Federighi, Apple’s Senior Vice President of Software Engineering. During Apple’s September event, Federighi explained that the feature is powered by a new capacitive camera button called Camera Control, available on the iPhone 16 and iPhone 16 Pro. To activate Visual Intelligence, users simply press and hold this button while pointing the phone’s camera at the object they want to identify.

On-Device AI for Privacy and Functionality

The feature uses a combination of on-device intelligence and Apple services, ensuring that no images are stored while Visual Intelligence processes the information. This allows users to take a photo of a restaurant and instantly receive details like its hours of operation, or scan a flyer to automatically record information such as the title, date, and location.

Integration with Third-Party Tools

Federighi also noted that Visual Intelligence can integrate with third-party models. For example, users could point their camera at a bicycle and use external services to search for more information or capture study notes to receive help with a particular concept.

Release Timeline

Although no specific launch date was provided, Apple confirmed that Visual Intelligence will debut as part of Camera Control “later this year,” bringing enhanced object recognition capabilities to iPhones.