Apple Enables 'Enhanced Visual Search' Without User Consent, Raising Privacy Alarms

Apple has enabled a new 'Enhanced Visual Search' feature for Photos on iOS 18 and macOS Sequoia 15 that sends data from your images back to Apple for analysis. The feature, which was spotted yesterday by developer Jeff Johnson, is enabled by default.

The settings toggle for 'Enhanced Visual Search' reads:

"Allow this device to privately match places in your photos with a global index maintained by Apple so you can search by almost any landmark or point of interest."




Johnson was only able to find two documents that referenced the feature on Apple's website. The first is a legal note.

Enhanced Visual Search in Photos allows you to search for photos using landmarks or points of interest. Your device privately matches places in your photos to a global index Apple maintains on our servers. We apply homomorphic encryption and differential privacy, and use an OHTTP relay that hides IP address. This prevents Apple from learning about the information in your photos.

The second is a Machine Learning Research blog post titled "Combining Machine Learning and Homomorphic Encryption in the Apple Ecosystem." The post reveals that Apple begins with an on-device machine learning model to analyze your photos and determine if they contain a "region of interest" (ROI) featuring a landmark. If an ROI is detected, a vector embedding for that region of the image is calculated and sent for remote identification. Apple states it uses an anonymization network to hide the device's source IP address before the request reaches its server infrastructure.


The fleet of servers that handle these queries leverage Apple's existing ML infrastructure, including a vector database of global landmark image embeddings, expressed as an inverted index. The server identifies the relevant shard based on the index in the client query and uses HE to compute the embedding similarity in this encrypted space. The encrypted scores and set of corresponding metadata (such as landmark names) for candidate landmarks are then returned to the client.

Despite Apple's efforts to anonymize the process, this is a "phone home" feature that some users may find uncomfortable. It also directly contradicts Apple's marketing claim that "What happens on your iPhone, stays on your iPhone."

"It ought to be up to the individual user to decide their own tolerance for the risk of privacy violations," says Johnson. "In this specific case, I have no tolerance for risk, because I simply have no interest in the Enhanced Visual Search feature, even if it happened to work flawlessly. There's no benefit to outweigh the risk. By enabling the "feature" without asking, Apple disrespects users and their preferences. I never wanted my iPhone to phone home to Apple. Remember this advertisement? "What happens on your iPhone, stays on your iPhone." That was demonstrably a lie."

You can disable the feature by toggling it OFF on your device...

● iPhone: Open the Settings app > Apps > Photos > Enhanced Visual Search
● Mac: Open the Photos app > Settings > General > Enhanced Visual Search

Please download the iClarified app or follow iClarified on Twitter, Facebook, YouTube, and RSS for more updates!