GitHub Copilot is going local — but Microsoft won’t say what gets sent to the cloud

GitHub is about to let Copilot decide, task by task, whether work runs on the device or gets handed to cloud models. Automatic routing is expected by the end of October, and Microsoft says the system will look at task context and cache state as it switches between local and cloud inference, even in multi-turn sessions.

That shift comes with a useful-sounding label, but not much clarity. Microsoft hasn’t said how much repository context Auto sends to cloud models, whether developers can inspect those routing calls, or whether they can force Copilot to stay local. And despite the “local” framing, the company says local inference does not mean the session is offline.

The hardware story is just as constrained. Microsoft’s local model, MAI Code 1.1 Flash, is a mixture-of-experts system with 137 billion total parameters and 6.8 billion active. It was shrunk with mixed-precision quantization to 53GB, about an 80% cut from the bfloat16 cloud version, and then paired with speculative decoding to speed up output. Even so, Microsoft measured peak memory use of 75.5GB at a 256K-token context on an NVIDIA RTX Spark Windows PC such as Surface Laptop Ultra, which offers up to 128GB of unified memory.

That means this is local inference for a fairly specific class of machines, not a blanket escape hatch for developer laptops. The OS, apps, runtime, and growing key-value cache all need space too, so longer sessions will demand more memory than the model weights alone suggest. Microsoft’s own benchmark numbers are decent but not dramatic: 70.8% on SWE-Bench Verified versus 72.6% for the full-precision version, and 66.29% on Terminal-Bench 2.1 versus 62.9% on a dataset of 89 tasks.

GitHub is also tightening sandboxing, though the protections vary by tool. Shell commands and local MCP servers get OS-level restrictions through Microsoft’s open source Execution Containers library, while built-in file tools depend on harness checks inside the agent process. Remote MCP servers sit outside the local sandbox, which means anyone who wants a truly sealed-off setup will still have to control what those tools can reach.

The company’s demo doesn’t exactly calm the nerves either. Microsoft describes it as an offline workflow, but the prompt uses GitHub issue and pull request metadata, and the post doesn’t say whether that data came from the network or local cache. For a product leaning hard into local control, that is a very familiar sort of omission.