It can be difficult to keep up with all the new artificial intelligence tools. You may even have access to numerous AI features and resources that you haven’t had the time to notice.

For example, Copilot Vision is now free for everyone to use in Microsoft Edge. That’s perhaps not something you’ve realized if you don’t use Edge as your regular browser, or if you’ve never heard of Copilot Vision.

It’s an extension of Microsoft’s Copilot AI that can essentially ‘see’ what you’re looking at on the web. Now, that’s probably already sounding a few privacy alarm bells in your head: Do you really want AI following your every move online?

Well, perhaps, if you find it useful. It means you can ask Copilot questions about what you’re looking at and get advice—whether you’re trying to find the best pour-over coffee makers or put together a perfect itinerary for your next road trip.

Microsoft says nothing you chat with Copilot Vision about is permanently stored, and the data from your session is deleted as soon as you finish. If you’re happy with those guardrails, here’s how to get started with the feature.

How to launch Copilot Vision

Make sure you’re using the latest version of Microsoft Edge by launching the browser on WIndows, then clicking the three dots (top right), and Help and feedback > About Microsoft Edge. You also need to be signed into your Microsoft account via the profile avatar button just to the left of the three dots on the toolbar.

You then need to navigate to a website, video, or PDF file in a browser tab that you want to chat to Copilot Vision about. (Note that the AI will refuse to engage on websites containing harmful or adult content.) Launch Copilot by clicking on the Copilot icon, which is on the far right of the toolbar (to the right of the three dots).

Next, click the microphone icon in the lower right corner of the Copilot side panel. If Copilot Vision is available, you’ll need to click to Accept it, and then you’ll get a short spoken spiel explaining how the feature works. The Copilot Vision panel sits in the center of the browser, and the color of the surrounding frame changes to show you that the AI feature is active.

There are four buttons here: Click the X to quit Copilot Vision, the mic button to mute and unmute your microphone, the glasses icon to turn Copilot Vision on or off, and the gear icon to access the settings for the tool (there’s only one setting available right now—you get to choose the voice style of the AI).

With Copilot Vision initiated, you just need to start talking. You can chat about anything you like, as you normally would with Copilot, as well as anything that’s on the page. You can’t get the AI to take actions for you though—it won’t scroll through pages or click on links, just chat with you via voice.

For now, Copilot Vision remains experimental, and this was backed up by my own testing. It’s accurate a lot of the time, but it’s not always going to understand everything you say perfectly, or give you reliable answers 100 percent of the time—as usual with these AI bots, you need to double-check anything important.

How to use Copilot Vision

Now that you know how to turn Copilot Vision on, how do you actually go about using it? Let’s say that you’re looking at a webpage with a lot of different news stories on it, for example: You can ask the AI to summarize all the headlines and read them out to you, to save scrolling and reading. Copilot Vision then asks if you’re interested in any story in particular, so you can continue the conversation.

Maybe you’re looking at a webpage for a venue, business, or organization—the summary tools can come in handy here as well. You might want to ask if a place is suitable for kids, or check when the opening hours are, or see if there are any special offers you can take advantage of. You can do all this just by reading the page of course, but when there’s a lot of text and info to take in, Copilot Vision can save you some time.

Another way to use it is to identify images, something AI has been good at for a long time. You can ask it about the species of plants you’re looking at online, for example, or about where in the world famous landmarks are (assuming these aren’t properly captioned, in which case you’ll already know). You can even ask Copilot Vision to give you an opinion on art, architecture, or whatever else you’re viewing.

Shopping is another use case. If you’re browsing through a lot of options on screen, you can get Copilot Vision to recommend something based on its looks, its specs, or on your personal tastes (the AI will prompt you about these, if needed). Again though, it can’t do the clicking and scrolling for you—you have to do all that.

Copilot Vision can even help with gaming on the web, if you’re playing lightweight games inside your web browser. You can ask for tips and strategy ideas, or instructions on how to play a game, and the AI will help you out. On certain web games, such as GeoGuessr, Copilot Vision can lead you towards the right answer.

From the testing I’ve done with Copilot Vision, its answers can often combine what’s on the page with more general knowledge from the web and its training data—so you can break off to ask questions about something that isn’t on screen. When you close down Copilot, you’ll be returned to the standard Copilot panel at the side of Edge.

 

More deals, reviews, and buying guides

The PopSci team has tested hundreds of products and spent thousands of hours trying to find the best gear and gadgets you can buy.