Street exploration helps artificial intelligence locate where photographs were taken

Allowing vision systems to explore virtual streets cuts localization errors from hundreds of kilometers to single digits for top models, though regional biases and navigation failures persist.

A photograph of an unlabelled dirt road or an ordinary city intersection contains dozens of subtle clues about its place on Earth. Determining where that photograph was captured requires a methodical process of elimination. An observer might first examine the angle of sunlight, the soil color, or the foliage to narrow the location down to a continent or hemisphere. The observer then looks for human-made infrastructure, such as road signs, utility pole designs, or architectural styles, to identify a specific nation. Finally, moving along the roadway might reveal a town name, an area code on a delivery truck, or a distinct local dialect on a shopfront, which pins the snapshot to an exact province or neighborhood.

For years, automated geolocalization software worked without that third step. Artificial intelligence systems evaluated solitary, stationary snapshots, treating the deduction as a single instant of pattern matching. In a preprint posted to the arXiv server on September 1, 2026, researchers Arka Mukherjee, Soham Roy, Kartikeya Trivedi, and Shreya Ghosh introduced a benchmark called GeoAgent to change that approach. By embedding vision-language models into interactive Street View environments, the authors gave software agents the ability to move through an area, examine alternative angles, and gather successive visual evidence before registering a coordinate guess.¹

Reliable location deduction carries immediate consequences beyond recreational guessing games. Emergency personnel rely on automated image geolocalization during natural disasters to confirm where damaged bridges or flooded roads sit when communications fail.¹ Human rights investigators and intelligence analysts depend on open-source intelligence verification to determine whether combat footage matches a claimed village or city square.¹ At the same time, privacy advocates track automated location recognition to understand how easily photographs shared online can reveal an individual's personal residence.¹

Why does wandering a street help identify a location?

Sequential exploration allows an artificial intelligence agent to search deliberately for identifying details that a single frame hides. When an automated agent receives only one photograph, a tree branch can block a street name, or a generic stretch of pavement can mimic roads across several continents.¹ In the GeoAgent testing environment, multimodal models autonomously traverse surrounding pathways, turning corners and inspecting municipal markers across consecutive frames.¹ The authors report that modern vision-language models score well above human baselines on image geolocalization, and giving these models the freedom of agentic navigation significantly improves their accuracy across established measurement metrics compared with static image baselines.^contributed

Gathering evidence step by step mirrors how skilled human analysts confirm ambiguous scenes. An agent might observe a vehicle driving on the left side of the road, reject jurisdictions that mandate right-hand driving, and then move forward until a billboard confirms the surrounding language. By combining visual cues gathered over time, the software refines its internal estimate through sequential reasoning before committing to a final point on the globe.¹

Mukherjee, the lead author, explained how this iterative reasoning loop operates across camera positions. Mukherjee said in response to questions from Primary that the setup implements a sequential reasoning pipeline where each step relies on the ReAct framework to collect and interpret clues before executing an action.^contributed "Through sequential reasoning, we built a pipeline in which the LLM refined its hypothesis about its geolocalization," Mukherjee said.^contributed "Each step used the ReAct pipeline to gather and reason about new evidence, then act on its current guess and choose the next move to continue the loop. If the model gathers enough evidence to challenge its theory about its location, our pipeline naturally allows it to fix course."^contributed

However, the absolute gains delivered by physical exploration depend heavily on the underlying reasoning capacity of the model. Mukherjee told Primary that the reduction in localization error is highly model-dependent.^contributed In tests measuring coordinate discrepancies, Gemini 3.0 Flash reduced its median error distance from 118.5 kilometers on static images down to 5.0 kilometers when granted exploration turns.^contributed By contrast, a weaker model, Llama 4 Scout, saw its mean error worsen from 458 kilometers in static mode to 579.7 kilometers during navigation.^contributed Mukherjee explained that while exploration assists stronger architectures, "it also adds a planning and management overhead that weaker LLMs were naturally set to benefit less from."^contributed

Where do frontier vision models still fall short?

Frontier vision models struggle to pinpoint local districts even when they correctly deduce the broader country and continent. The authors observed that while frontier architectures routinely classify broad national boundaries, they falter when asked to separate neighboring regions or provinces with similar topographies.¹ Knowing that a gravel road runs through Brazil, for instance, does not reliably help an agent distinguish between its southern grasslands and northern rainforests.

Mukherjee noted that current evaluation sets frequently over-represent iconic landmarks. The GeoAgent benchmark was specifically designed to balance these distributions with "less weight on well-known views such as Mount Fuji or Times Square, and greater focus on fine-grained locations in the Nigerian countryside, say."^contributed Mechanistically, Mukherjee observed that models require stronger location-based scene decomposition, though testing such granular reasoning currently encounters an enormous computational bottleneck.^contributed

The models also exhibit poor self-correction once they form an initial misconception.¹ If an automated agent begins its exploration with an inaccurate prior assumption about its surroundings, subsequent observations rarely dislodge the error.¹ Instead of adjusting its hypothesis when incoming street details contradict the initial guess, the software often clings to its mistaken coordinate estimate.

Mukherjee pointed to recurring misclassifications across Asian territories as clear evidence of this stubbornness. In commentary provided to Primary, Mukherjee observed that several frontier models frequently conflated locations in Taiwan and Japan, categorizing scenes from both territories as Japan despite obvious visual counter-evidence.^contributed Mukherjee noted that this failure to self-correct is probably an artifact of imbalances in how web-scale pre-training data is curated.^contributed

What does the benchmark reveal about geographic bias?

Frontier artificial intelligence models display severe performance discrepancies between developed and developing regions.¹ The authors identified pronounced disparities across architectures, with systems achieving far higher localization success in wealthier nations than in lower-income areas.¹ Mukherjee confirmed that raw guessing accuracy correlates with whether an architecture ingested visual data from a territory during training, meaning that without supplemental tooling, model performance in developing regions naturally craters.^contributed

Active exploration can narrow this geographic gap, but only when models know how to move effectively through an environment. Mukherjee noted that embodied navigation helped improve performance discrepancies between developed and developing regions directionally across the test suite.^contributed Conversely, systems with poor spatial navigation strategies deteriorated. Mukherjee highlighted that Llama 4 Scout frequently spent its exploration turns rotating in-place rather than traversing roadways, causing its spatial accuracy to degrade when given navigation commands.^contributed

These limitations emphasize what the benchmark can and cannot prove about automated deduction. The paper is an academic preprint that has not yet completed formal peer review. Furthermore, the experiments assess autonomous agents inside simulated Street View environments, where movement is constrained to recorded camera trajectories rather than unrestricted real-world locomotion.¹ The recorded results measure relative benchmark accuracy under programmatic rules, which cannot guarantee how an agent will behave when confronted with unmapped disaster zones or degraded imagery.¹

Addressing the gap between virtual streets and crisis zones, Mukherjee cautioned against assuming these results immediately apply to emergency logistics. Mukherjee stated that the benchmark relies on Google Street View, which assumes idealized conditions for automated navigation.^contributed Real-world emergency footage from disaster areas presents far messier visual inputs, where performance drops could easily widen. Mukherjee added that navigating actual disaster environments would require additional agent capabilities that the current benchmark does not evaluate, including dynamic camera zooming, multi-step trajectory planning, and cross-modal database lookups.^contributed

For practitioners in disaster response and forensic verification, the findings indicate that automated agents cannot yet operate without close human supervision. The presence of regional bias means that critical logistics tools risk failing in the communities that need assistance most urgently. To encourage broader evaluation of spatial deduction and embodied exploration, Mukherjee, Roy, Trivedi, and Ghosh released their code and the GeoAgent simulation platform publicly at geoagent-benchmark.github.io.¹

What this rests on

33 sentences trace to 1 source and 1 contributor.

All 2Documents 1Records 0Reporting 0People 1

1. 1 AM Arka Mukherjee, Soham Roy, Kartikeya Trivedi, Shreya Ghosh Paper · arxiv.org · 1 Sep 2026 GeoAgent: Evaluating VLM Geolocalization Through Embodied Navigation Preprint · may not have been peer reviewed See the source
2. 2 AM Arka Mukherjee, lead author Contributor · interview 29 Sep 2026 · in the article since 5 Oct 2026 Contribution — Arka Mukherjee 24 statements added to this article

Article history

1. Published 5 Oct 2026, 13:22
   Assembled by the Primary desk from 1 source · 1 contributor · 33 cited sentences
2. AM Arka Mukherjee, lead author 24 statements added 29 Sep 2026 7 days before publication

   1. Go to the sentence: The authors report that modern vision-language models score well above human baselines on image geolocalization, and giving these models the freedom of agentic navigation significantly improves their accuracy across established measurement metrics compared with static image baselines.↑
   2. Go to the sentence: By combining visual cues gathered over time, the software refines its internal estimate through sequential reasoning before committing to a final point on the globe.↑
   3. Go to the sentence: Mukherjee said in response to questions from Primary that the setup implements a sequential reasoning pipeline where each step relies on the ReAct framework to collect and interpret clues before executing an action.↑
   4. Go to the sentence: "Through sequential reasoning, we built a pipeline in which the LLM refined its hypothesis about its geolocalization," Mukherjee said.↑
   5. Go to the sentence: "Each step used the ReAct pipeline to gather and reason about new evidence, then act on its current guess and choose the next move to continue the loop. If the model gathers enough evidence to challenge its theory about its location, our pipeline naturally allows it to fix course."↑
   6. Go to the sentence: Mukherjee told Primary that the reduction in localization error is highly model-dependent.↑
   7. Go to the sentence: In tests measuring coordinate discrepancies, Gemini 3.0 Flash reduced its median error distance from 118.5 kilometers on static images down to 5.0 kilometers when granted exploration turns.↑
   8. Go to the sentence: By contrast, a weaker model, Llama 4 Scout, saw its mean error worsen from 458 kilometers in static mode to 579.7 kilometers during navigation.↑
   9. Go to the sentence: Mukherjee explained that while exploration assists stronger architectures, "it also adds a planning and management overhead that weaker LLMs were naturally set to benefit less from."↑
   10. Go to the sentence: The authors observed that while frontier architectures routinely classify broad national boundaries, they falter when asked to separate neighboring regions or provinces with similar topographies.↑
   11. Go to the sentence: The GeoAgent benchmark was specifically designed to balance these distributions with "less weight on well-known views such as Mount Fuji or Times Square, and greater focus on fine-grained locations in the Nigerian countryside, say."↑
   12. Go to the sentence: Mechanistically, Mukherjee observed that models require stronger location-based scene decomposition, though testing such granular reasoning currently encounters an enormous computational bottleneck.↑
   13. Go to the sentence: If an automated agent begins its exploration with an inaccurate prior assumption about its surroundings, subsequent observations rarely dislodge the error.↑
   14. Go to the sentence: In commentary provided to Primary, Mukherjee observed that several frontier models frequently conflated locations in Taiwan and Japan, categorizing scenes from both territories as Japan despite obvious visual counter-evidence.↑
   15. Go to the sentence: Mukherjee noted that this failure to self-correct is probably an artifact of imbalances in how web-scale pre-training data is curated.↑
   16. Go to the sentence: Frontier artificial intelligence models display severe performance discrepancies between developed and developing regions.↑
   17. Go to the sentence: The authors identified pronounced disparities across architectures, with systems achieving far higher localization success in wealthier nations than in lower-income areas.↑
   18. Go to the sentence: Mukherjee confirmed that raw guessing accuracy correlates with whether an architecture ingested visual data from a territory during training, meaning that without supplemental tooling, model performance in developing regions naturally craters.↑
   19. Go to the sentence: Mukherjee noted that embodied navigation helped improve performance discrepancies between developed and developing regions directionally across the test suite.↑
   20. Go to the sentence: Mukherjee highlighted that Llama 4 Scout frequently spent its exploration turns rotating in-place rather than traversing roadways, causing its spatial accuracy to degrade when given navigation commands.↑
   21. Go to the sentence: Furthermore, the experiments assess autonomous agents inside simulated Street View environments, where movement is constrained to recorded camera trajectories rather than unrestricted real-world locomotion.↑
   22. Go to the sentence: The recorded results measure relative benchmark accuracy under programmatic rules, which cannot guarantee how an agent will behave when confronted with unmapped disaster zones or degraded imagery.↑
   23. Go to the sentence: Mukherjee stated that the benchmark relies on Google Street View, which assumes idealized conditions for automated navigation.↑
   24. Go to the sentence: Mukherjee added that navigating actual disaster environments would require additional agent capabilities that the current benchmark does not evaluate, including dynamic camera zooming, multi-step trajectory planning, and cross-modal database lookups.↑