ChatGPT for macOS Now Works with Desktop Coding Apps

OpenAI has introduced a new feature in its ChatGPT macOS app, allowing the AI to interact with select developer tools such as VS Code, Xcode, TextEdit, Terminal, and iTerm2. Known as Work with Apps, the feature enables ChatGPT to automatically read code from these applications and use it as context to provide tailored responses to user prompts.

This new capability removes the need for developers to copy and paste code manually into ChatGPT, streamlining workflows and enhancing usability. However, unlike AI tools such as GitHub Copilot or Cursor, ChatGPT cannot yet write code directly into these applications on behalf of users.

How It Works

OpenAI’s Work with Apps relies on macOS’s accessibility API, primarily the screen reader technology that powers Apple’s VoiceOver feature. This API allows ChatGPT to extract text from applications and use it as input, though its functionality is limited to text-based elements and cannot process visual content like images or video.

Key features include:

• Sending the last 200 lines of code or the entire text in the foremost window to ChatGPT alongside the user’s prompt.

• Allowing users to highlight specific code sections to refine the chatbot’s focus.

• Requiring extensions for certain apps, such as VS Code, to facilitate communication with ChatGPT.

Despite these capabilities, limitations exist. For example, ChatGPT cannot interpret visual layouts or graphical elements, and its reliance on text-based inputs means it’s restricted to specific types of applications for now.

Early Access and Future Rollout

The feature is currently available to Plus and Teams users, with plans to expand access to Enterprise and Edu users in the coming weeks. Initially focused on coding tools, OpenAI envisions expanding Work with Apps to support other text-based applications, such as word processors, for writing tasks.

In a demonstration shared with TechCrunch, OpenAI showcased ChatGPT’s ability to assist with a coding project in Xcode. The AI analyzed a project simulating the solar system and generated a code snippet to add a missing planet, which the user then manually copied into the Xcode environment.

“This isn’t meant to be an agent; it’s a way to collaborate with coding tools to start, and there will be more tools coming soon,” said Alexander Embiricos, OpenAI’s desktop product lead, during a briefing with TechCrunch.

A Step Toward General AI Agents

OpenAI describes Work with Apps as a “key building block” toward the development of general-purpose AI agents. Current AI agents, such as Anthropic’s screenshot-based system, aim to analyze entire desktop environments but are often slow and error-prone. In contrast, OpenAI’s targeted approach emphasizes precision by focusing on text-based interactions through APIs.

This incremental progress aligns with reports that OpenAI is working on a broader AI agent, codenamed Operator, expected to launch in early 2025. Operator aims to rival existing agent systems from competitors like Anthropic and Google.

What’s Next?

OpenAI is prioritizing macOS for this rollout, with Windows compatibility potentially following later. The announcement comes just ahead of Apple’s planned integration of ChatGPT into its systems, set for release in December.

As OpenAI refines Work with Apps, the feature could eventually extend beyond coding tools, making AI even more integral to productivity workflows. For developers, this update already offers a significant improvement in efficiency by automating repetitive steps and enabling faster problem-solving.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing, image, and idea-generation support from ChatGPT, an AI assistant. However, the final perspective and editorial choices are solely Alicia Shapiro’s. Special thanks to ChatGPT for assistance with research and editorial support in crafting this article.