Mapping the modern world: How S2Vec learns the language of our cities

Google Research just published a paper on S2Vec, a system that treats the built environment — roads, shops, apartment blocks, bus stops — like a photograph. Instead of feeding a model raw coordinates, the team carves the planet into a hierarchy of cells using the S2 Geometry library, then counts what's inside each one. Three coffee shops and a park in a cell become pixels of a sort, layered into a multi-channel image the model can actually look at.

Once geography looks like a picture, computer vision tricks that already work well on photos suddenly apply. S2Vec leans on masked autoencoding, the same self-supervised idea behind a lot of recent image and language models: hide part of the scene, make the network guess what's missing from context. Show it a subway stop next to high-rises with a blank patch nearby, and it learns to predict there's probably a grocery store there. Do that across millions of locations worldwide and the model starts to internalize which features cluster together, without anyone ever telling it what a 'financial district' or a 'suburb' is.

The payoff shows up in benchmarks. Against rivals like SATCLIP, GeoCLIP, and Hex2vec, S2Vec came out ahead specifically at zero-shot extrapolation — guessing median income or population density in regions it never saw during training. That's the harder, more useful version of the test; interpolating within familiar data is comparatively easy. Stack S2Vec's embeddings alongside satellite imagery and the combo beats either approach alone, which tells you the two data types are capturing genuinely different signals rather than duplicating each other.

Where it stumbles is environmental prediction — tree cover, elevation, that kind of thing. Counting buildings and businesses just doesn't tell you much about vegetation or terrain, and Google admits S2Vec needs satellite imagery bolted on to be competitive there. The company is positioning this as one piece of a larger Earth AI push that already includes a population dynamics model and a remote-sensing vision-language model, with the pitch being that urban planners and climate researchers eventually get a general-purpose geographic embedding instead of a pile of hand-built indicators for every new question.