Google is putting more of its Gemini AI into many of its products, and the next target in its sights is Photos. At its I/O developer conference today, the company’s CEO Sundar Pichai announced a feature called Ask Photos, which is designed to help you find specific images in your gallery by talking to Gemini.
Ask Photos will appear as a new tab at the bottom of your Google Photos app. It will first roll out to One subscribers, starting in US English in the coming months. When you tap this panel, you’ll see a Gemini star icon and a welcome message above a bar prompting you to “search or ask for photos.”
According to Google, you can ask things like “show me the best photo of any national park I’ve visited,” which not only relies on GPS information, but also requires the AI to exercise some judgment to determine which is ” the best”. The company’s vice president of photography, Shimrit Ben-Yair, told Engadget that you’ll be able to provide feedback to the AI and let it know which photos you prefer instead. “Learning is key,” Ben-Yair said.
You can also ask Photos to find your best photos from a recent vacation and generate a caption to describe them so you can more quickly share them on social media. Again, if you don’t like what Gemini suggests, you can also make changes later.
For now, you’ll have to type your request into Ask Photos — voice input isn’t supported yet. And as the feature rolls out, those who opt in to use it will see their existing search feature “upgrade” to Ask. However, Google said that “key search functionality, such as quick access to your face groups or map view, will not be lost.”
The company explained that there are three parts to the Ask Photos process: “Understanding your question,” “drafting an answer,” and “ensuring safety and remembering corrections.” Although safety is mentioned only at the last stage, it should be baked all the time. The company acknowledges that “the information in your photos can be deeply personal, and we take our responsibility to protect it very seriously.”
For this purpose, requests are not stored anywhere, although they are processed in the cloud (not on the device). People won’t review conversations or personal data on Ask Photos except “in rare cases to address abuse or harm.” Google also said it does not train “any generative AI product outside of Google Photos on this personal data, including other Gemini models and products.”
Your media continues to be protected by the same security and privacy measures that cover the use of Google Photos. That’s a good thing, because one of the potentially more useful ways to use Ask Photos could be to get information like passport or license expiration dates from photos you may have taken years ago. It uses Gemini’s multimodal capabilities to read text in images to find and answer.
Of course, AI isn’t new to Google Photos. You’ve always had the ability to search the app for things like “credit card” or a specific friend using the company’s face and object recognition algorithms. But Gemini AI offers generative processing, so Photos can do a lot more than just deliver photos with certain people or items in them.
Other applications include getting photos to tell you what themes you might have used for the last few birthday parties you threw for your partner or child. Gemini AI works here to study your photos and understand what themes you have already adopted.
There are many promising use cases for Ask Photos, which is currently an experimental feature and “rolling out soon.” Like other photo tools, it may start as a premium feature for One subscribers and Pixel owners before rolling out to everyone using the free app. There is no official word yet on when or if this might happen.
Catch up on all the news from Google I/O 2024 right now here!
