On Windows 11 (and 10), users in the United States can now use Copilot Vision, an AI assistant that helps with almost anything you're doing on your computer. Want to know if it really works? Well, I gave it a shot, and in this guide, I'll break down exactly what the feature offers and how you can get it, and I'll share my personal experience with it.
In the Copilot app, "Vision" is a feature that allows you to share the screen with a chatbot, allowing it to see and interpret what's on your screen. It acts like a second pair of eyes, providing real-time assistance, answers, and insights based on the content displayed in your apps or browser.
It's also an opt-in feature, so you must enable it manually in the Copilot app version 1.25061.104.0 and higher releases. If you use the Microsoft Edge browser, you can also access the feature through the Copilot integration.
It's important to note that the AI can view virtually any content as long as it's not DRM-protected material or another type of restricted content.
The feature is available for anyone without a Copilot Pro subscription on Windows 10 and 11. You can also access Copilot Vision on iOS and Android, but you'll need a subscription to use it on these mobile devices.
In this guide, I'll outline the steps to get started with the feature and share my experience.
How to get and enable Copilot Vision on Windows 11
Copilot Vision is available in the United States, starting with the Copilot app version 1.25061.104.0 and higher releases. As a result, the first step is to open Microsoft Store > Downloads and click the "Check for updates" button to ensure the latest app version is installed on your computer.
You can get it from the Microsoft Store app if you previously uninstalled the app.
Microsoft says that Vision is an opt-in feature, which is technically accurate since you have to give it permission to share the screen. However, it's actually available by default since there's no option to turn off the feature completely on the settings page.
In the Copilot settings page, you'll only find an option to turn "Highlights" on and off, but this feature only controls the ability for the AI to point out actions on the screen visually. It doesn't control the availability of the feature in the app.
If you're not comfortable with the idea of Copilot, it's best to remove it from Settings > Apps > Installed apps and click the "Uninstall" option for the "Copilot" entry.
How was my experience using Copilot Vision on Windows 11
The first time I interacted with Copilot Vision, it felt a little weird because it's something I've never seen before. Instead of having to create a text or voice prompt with content, the chatbot already knew the context, and it was able to assist.
Although it's not completely accurate, it's an early preview of the future of working on a computer.
Getting started
Once I knew that Copilot Vision was available on my computer, I opened a few apps, launched the Copilot app from the Start menu, clicked the "Vision" (glasses) button, selected the app I wanted to share, and turned on the "Share" toggle switch.
App question test
While in the Notepad app, I asked Copilot to show me how to change the default font for the application, but it gave the wrong answer.
Copilot's answer was to look into the "View" menu, but the option wasn't in the location because, by design, on the settings page, you can access it from the "Edit" menu or by clicking the "Gear" button from the top-right.
Eventually, it got it right using the process of elimination. The interesting part is that the chatbot said it made a mistake because it confused the versions of Notepad, and I don't remember Notepad ever having the font settings in the "View" menu.
Settings question test
As part of the testing process, I launched the Settings app on Windows 11 and then asked Copilot to update my computer with the latest system updates.
In this instance, Microsoft Copilot with Vision correctly identified that I was in the Settings app. It accurately directed me to the "Windows Update" section and highlighted the "Check for updates" button.
Next, I gave it a less common (but still relevant) prompt. I asked how to prevent my computer from uploading updates to other devices during the update process. While not a particularly complex request, it tested the assistant's ability to interpret more subtle user intent.
On the first attempt, Vision misunderstood the query and defaulted to generic instructions for updating the device. On the second attempt, after refining my phrasing, it did recognize the question correctly.
However, its guidance was inconsistent with what I saw on screen. For example, it indicated that I had successfully disabled the update-sharing feature, even though I hadn't taken any action. This suggested that the assistant was making assumptions based on expected behavior rather than actually analyzing the live system state.
Recognizing objects test
For the next test, I wanted to see if Copilot was able to recognize objects on the screen, so I opened an image and asked it to determine the specific object.
In this instance, I had an image of a red jacket, and the chatbot was able to identify and describe the product while providing more contextual information upon request.
However, although the chatbot was aware that the jacket was available on Amazon, even though I opened the image on a separate tab, it wasn't aware that I wasn't on the actual product page, and it was unable to provide the product page information.
Extracting text test
Another thing you can do in Copilot Vision is extract text from an image. In the past, the operating system could not detect and extract text from an image, and now, you have many ways to complete this task.
For example, you have the text extractor feature available through PowerToys, Snipping Tool, and Click to Do, and now, you can even use Copilot Vision.
For this text test, I opened the Settings app on the Game Mode page and asked if it could extract the text. The chatbot was able to read aloud everything on the page.
The only caveat was that it wasn't able to copy the extracted text to the clipboard or let me select the text, as is possible in Click to Do. However, the chatbot transcribed everything in the Copilot app as part of the chat history.
Writing text test
You can technically ask any question about anything on the screen. If you're on a file, you can prompt the AI to analyze a chart or any other type of data.
You can ask to describe a scene or an image, determine a landmark or location, and virtually anything you can imagine.
If you're working on a piece of text, you can ask to read the text and suggest ideas to improve.
For my final test, I had some text in Notepad, and I asked the chatbot to make it longer, and it was able to suggest an alternative version.
While it was impressive how the assistant was able to understand the text on the screen and shared an alternative version, it wasn't very accurate in understanding follow-up actions.
Although I knew it wasn't possible to copy and replace the text with the suggestion, I asked if it was possible to copy and paste the alternative version of the text, but it pointed to a section in the file to do the insertion instead.
The trick here is to open and close Copilot Vision and, from the Copilot interface, select and copy the text from the chat history.
Final thoughts
From the point of view of someone who has been writing guides for many years, the technology is impressive in making sense of the content on the screen and providing assistance.
However, it makes a lot of mistakes, and it may not be very useful if you don't already have some understanding of the action you want to take.
Also, it wants to act like it knows, but it's still an AI that repeats content already available online.
At times, using Copilot Vision feels like talking to tech support over the phone because even though the chatbot can view the screen, it guides you as a tech support agent would assist you, and it usually doesn't understand if you completed the task.
Furthermore, in most cases, you have to be extremely specific with your instructions, often down to the exact action you want to perform. This essentially defeats the purpose of having an AI that's supposed to understand user intent naturally. It's important to remember that this feature is primarily designed for non-technical users, as tech-savvy individuals are less likely to rely on it.
Overall, while Vision shows promise of becoming your personal assistant, its deeper contextual awareness and accuracy still need refinement. It felt more like it was "guessing" based on the command rather than actively observing and responding to the current state of the content on the screen.
Lastly, it's important to note that I'm not listing the specific questions I used for this guide because they are irrelevant. The idea of a chatbot is to have a regular conversation as if you were talking to another human.
Also, this feature cannot take action for you. It can only analyze the contents on the screen. To take action, the AI has to be an agent, but currently, only Copilot+ PCs have this capability in a limited capacity in the Settings app.
Finally, even though the feature is free, it's available with limits. For instance, while I do have a Microsoft 365 subscription, after a few interactions, the app prompts me to upgrade to Copilot Pro.
So, if you're troubleshooting a problem and are in the middle of it, you may be asked to pay to have the AI assistant finish fixing the issue.
More resources
Explore more in-depth how-to guides, troubleshooting advice, and essential tips to get the most out of Windows 11 and 10. Start browsing here:
You must confirm your public display name before commenting
Please logout and then login again, you will then be prompted to enter your display name.
