Gemini 1.5 Pro: How 1M Tokens Changed Coding

Google's 1.5 Pro was the first widely available foundation model to serve a million-token context window for production coding work. That window is large enough for tens of thousands of lines in a single prompt. A two-million-token developer tier followed the same year.

Table of Contents:

1. Key Takeaways
2. Introduction
3. Why Did Google DeepMind Abandon the 32k Window?
4. How Does Gemini Hold a Whole Repo in Memory?
5. What Did Needle Tests and SWE-bench Actually Measure?
6. How Did Gemini Change Client-Repo Work for Agency Developers?
7. Why Did Claude 3.5 Sonnet Still Win Isolated Coding Contests?
8. What Breaks When You Paste 800,000 Tokens Into One Prompt?
9. FAQ

Key Takeaways

• The 1.5 Pro context jump was a hardware story as much as a model story, and TPU optical interconnects were the quiet, under-credited hero behind million-token production prompts.
• Dumping a whole repo into context did not retire RAG for every shop, because attention still gets sloppy when five constraints hide in 800k tokens of client code.
• Valeo's 35 percent assisted-code figure says more about how teams wired Code Assist into daily workflow than it says about a HumanEval leaderboard screenshot.

Introduction

The first time I watched a model chew through 746,152 tokens of the JAX library and still land on the autodiff path, I stopped treating context like a sticky note. Agency developers who babysit client monorepos felt that same jolt when Gemini stopped asking them to carve the tree into 500-token twigs.

That was not a personality upgrade. It was a sparse Mixture-of-Experts stack on Google TPU pods, plus Context Caching that made repeat queries on a frozen repo cheap enough to run all week. Before 1.5 Pro, most shops lived in RAG: split the tree, embed the scraps, hope cosine similarity understood inheritance.

I will walk the February-to-October timeline, the Valeo production-code share, the $1.25 input price, and the cases where I would still feed the model three files instead of the whole monorepo.

Why Did Google DeepMind Abandon the 32k Window?

Google Brain and DeepMind merged in April 2023 under Demis Hassabis, then shipped the 1.0 family in December with a 32,000-token cap. Quadratic attention made bigger windows painful, so the team pivoted to sparse Mixture-of-Experts and announced 1.5 Pro on February 15, 2024 via Google AI Studio and Vertex AI.

Piper, Google's internal monorepo, weighed about 86 terabytes, an estimated 37.9 trillion tokens. Training a natively multimodal rival to OpenAI's GPT-4 on that pile made a 32k visor look silly. Dense attention scales as O(N²). A bigger dense net was not going to remember a whole payroll service.

The public dates are blunt:

• April 2023: Brain and DeepMind combine, Hassabis as CEO.
• December 2023: 1.0 ships as Nano, Pro, and Ultra, still at 32,000 tokens.
• February 15, 2024: Sundar Pichai and Hassabis open 1.5 Pro private preview.
• March 8, 2024: Reid et al. publish the 1.5 technical report, with research runs out to 10 million tokens.
• April 1, 2024: Logan Kilpatrick leaves OpenAI developer relations and joins to lead AI Studio and the API.
• May 14, 2024 (Google I/O): million-token 1.5 Pro goes generally available across 35 languages, Flash debuts, 2 million tokens enter preview.
• June 27, 2024: extended window, Context Caching, and a sandboxed Python interpreter reach all developers.
• September–October 2024: Pro 002 and Flash 002 cut prices 50 to 64 percent, drop time-to-first-token about 3x, double throughput, and raise Pro to 1,000 requests per minute.

It feels almost impossible that I would be talking about hundreds of thousands of tokens in context length, let alone millions.

Oriol Vinyals, VP of Research at Google DeepMind, said that in February 2024, after a career that ran from Shannon-style bi-grams to LSTMs that topped out near 200 tokens. Google did not stretch context by stacking denser attention. It changed the wiring and the chips.

This sits on top of how generative code actually started and the November 2022 ChatGPT shock. 1.5 Pro did not invent code models. It changed the unit of work from a snippet to a repository.

How Does Gemini Hold a Whole Repo in Memory?

1.5 Pro is a multimodal sparse Mixture-of-Experts transformer. A learned router activates only a slice of experts per token, so knowledge can grow without running every parameter on every token. TPU pods linked by optical networks split the KV cache across machines, which is how a million-token context window stays practical.

If I am paying per million tokens, I cache the repo once and query it like a boring database. Context Caching writes the precomputed KV state of a static prefix onto TPU memory. Later calls skip the prefill tax. Google lists that path as cutting cached token processing cost by 64 percent, with cached input around $0.31 to $0.625 per million tokens and cache storage at $4.50 per million tokens per hour.

The production spec sheet is less mystical than the demos:

• Window: 1,000,000 tokens in production (about 750,000 words, or roughly 40,000 to 60,000 lines of code); 2,000,000 on the extended developer path; 10,000,000 in the Reid et al. research ceiling.
• Modalities: text, code, high-res images, audio up to 107 hours, video up to 10.5 hours, interleaved in one sequence.
• Execution: optional non-networked Python sandbox so the model can run a snippet, read stdout, and retry before it answers you.
• Hardware: Google TPU pods with a high-speed optical mesh.

It would honestly have been difficult to do at decent latency without TPUs and their interconnect.

Sholto Douglas, a Google DeepMind research engineer, said that in February 2024. The interconnect is the unglamorous part of vibe coding: you talk to a model that can actually see the tree, not a chatbot guessing from three files you remembered to paste.

What Did Needle Tests and SWE-bench Actually Measure?

Synthetic retrieval is the easy headline. According to the Reid et al. technical report, single-needle recall stayed above 99.7 percent out to a million tokens across text, audio, and video. Real GitHub issues are harder: unscaffolded SWE-bench Verified sat at 38.0 percent, then climbed when agents and compiler tests joined the loop.

The Needle in the Haystack test is a hide-and-seek game. You bury a factual string at random depth inside a mountain of distractor text (or audio, or video) and ask the model to fetch it. Jeff Dean, Chief Scientist at Google DeepMind, described a nastier variant: 100 needles in the same million-token pile.

We also created a generalized version of the needle in a haystack test, where the model must retrieve 100 different needles hidden in the context window.

Dean noted that 1.5 Pro stayed steadier across that full million than GPT-4 Turbo, which faded after about 64,000 tokens. Claude 2.1's 200,000-token window and GPT-4 Turbo's 128,000 were simply out of range for the 10 million token research runs, where recall stayed above 99 percent.

The coding scoreboard is a different sport:

• HumanEval (0-shot pass@1): 84.1 percent for 1.5 Pro, versus 93.7 percent for Claude 3.5 Sonnet and 83.5 percent for Qwen2.5-14B.
• SWE-bench Verified, unscaffolded: 38.0 percent DirectSolve, versus 36.2 percent for GPT-4o in an Agentless setup. Bare prompting also showed an 11.0 percent floor depending on the harness.
• Two-stage setup: 48.6 percent when combined with Claude 3.7.
• Scaffolded agents (SWE-agent, Devin-class loops, test execution): 61.8 to 80.6 percent, against Claude 3.5 Sonnet at 64.0 to 88.6 percent and OpenAI o3 near 72 percent.
• MATH (4-shot Minerva): 86.5 percent, ahead of GPT-4o at 83.3 percent and Flash 002 at 77.9 percent.
• MMLU-Pro (0-shot): 75.8 percent, with Flash 002 at 67.3 percent.

The JAX demo is the one I quote to people who still think long context is a party trick. DeepMind stuffed the entire 746,152-token JAX library into one prompt. The model localized autodiff internals, traced call graphs, and drafted docs with no vector index. It did the same with Flax's 41,070 lines. Perfect needle recall is a retrieval score, not a promise that five-hop refactors will land clean.

How Did Gemini Change Client-Repo Work for Agency Developers?

Agency developers used to split client trees into 200-to-1,000-token chunks, embed them, and hope the retriever kept interfaces next to implementations. In-context repository ingestion loads the tree, the docs, and the configs in one prompt, then traces call graphs the way a senior would with every file tab open.

That old RAG stack was not "a bit lossy." It smashed abstract syntax trees into bag-of-tokens similarity. Chunking isolated type signatures from the classes that implemented them, which is how you get a confident patch that cannot compile. 1.5 Pro's bet was a global attention graph over 500,000 to 1,500,000 tokens, cached, then queried for multi-file diffs and migration maps.

Whole-repo attention beats cosine similarity when the bug lives three files away from the similar-looking string. Agency retainers feel this most on inherited PHP and old Spring services. According to Google Cloud's published gen AI case studies, shops actually shipped on this pattern:

• Valeo: Code Assist across global software teams; more than 35 percent of production code generated or assisted by AI.
• Renault Group / Ampere: the same assistant on EV embedded monorepos, used to keep conventions consistent across a huge tree.
• Domina: Vertex AI for delivery-return predictions, with an 80 percent lift in real-time data access and less manual report wrangling.
• LUXGEN: Vertex conversational agents that cut human support load by 30 percent.

According to Google Cloud surveys, 61 percent of surveyed organizations had already put generative AI into production. According to Sundar Pichai's Google I/O 2024 keynote, more than 1.5 million developers were building on the models across Google's tools. The consumer subscription, launched as Advanced, crossed 1 million subscribers in its first 90 days.

This is also why the gap between no-code builders and vibe-code prompts got weird in 2024. You could still drag a site together. You could also paste the client's repo and talk the model through a rewrite. Neither move requires a vector database on day one.

Why Did Claude 3.5 Sonnet Still Win Isolated Coding Contests?

Long context is not the same skill as writing a tight algorithm on a blank page. Claude 3.5 Sonnet scored 93.7 percent zero-shot pass@1 on HumanEval. 1.5 Pro scored 84.1 percent. If the job is one nasty function with no repo around it, the bigger window does not automatically win.

I will be the wet blanket here. Capacity is not taste. Early feedback said 1.5 Pro needed more scaffolding and more loops to match Anthropic on idiomatic, single-pass logic. GPT-4o's HumanEval band sat around 88 to 90 percent. Cursor-class agent IDEs still leaned on Claude for the "write this file like a picky staff engineer" motion, while the Google stack won the "here is the entire tree, find the rot" motion.

According to the Google Developers Blog on the 002 checkpoints, Pro input prices fell 64 percent (output 52 percent, cached tokens 64 percent) for prompts under 128k, effective October 1, 2024. Flash 002 sat at $0.13 input and $0.38 output. Flash-8B undercut that again at $0.0375 and $0.15. 002 also lifted MMLU-Pro about 7 percent and MATH about 20 percent.

A bigger window is not a better first draft when the file does not exist yet. If you live in AI app builders that talk a UI into existence, that distinction matters: use the long-context model to read the mess you already have, not to pretend it is the best blank-page intern in the room.

What Breaks When You Paste 800,000 Tokens Into One Prompt?

Uncached prefill is slow. Time-to-first-token can sit between 2,000 milliseconds and several seconds, which wrecks inline completion. Independent work also showed multi-hop reasoning can fade when five constraints hide across 800,000 tokens, even when single-needle tests look almost perfect. I still isolate the three hot files when the task is a five-hop refactor.

The "lost in the middle" problem did not vanish because a press demo found one needle. Attention dilutes. A smaller prompt that contains only the three relevant modules often writes cleaner code than a heroic dump of 800,000 tokens of noise. Sub-300ms autocomplete engines should not call the million-token path at all. Use Flash, or a tiny specialist, and save Pro for the batch job that can wait.

The financial burden to build with AI is one of the rate limiters of this technology being accessible.

Logan Kilpatrick said that in September 2024, after the 002 cuts. Caching is the practical answer to the cost objection: repeated queries against a frozen tree at roughly $0.31 per million cached tokens often beat the bill for embeddings, a vector database, and a reranker. The hallucination objection is sloppier. Single-needle recall stayed above 99.7 percent. Multi-constraint synthesis is where people get burned, and file-boundary markup in the prompt helps more than another pep talk.

What 1.5 Pro is bad at, in plain language:

• Inline completion: uncached million-token prefill is too slow for keystroke UX.
• Greenfield puzzles: Claude 3.5 Sonnet still led HumanEval-style isolated synthesis.
• Five-hop logic in a haystack: retrieve-the-fact scores do not equal assemble-the-patch scores.
• Uncached novelty: if the repo changes every prompt, you pay full prefill and you wait.

Later checkpoints kept the window and added test-time compute. Unscaffolded SWE-bench Verified moved from 38.0 percent on 1.5 Pro to 50.8 percent on 2.5 Pro, and 64.0 percent at pass@8. For trees under a couple million tokens, that path is already eating classic code RAG. For everything else, I still chunk. It depends.

Start exploring launch-ready no-code templates here!

FAQ

What is a million-token context window?

A million-token context window is the amount of input a model can attend to in one request, including code, docs, and chat history. For 1.5 Pro, that production window maps to roughly 750,000 words, or about 40,000 to 60,000 lines of code, which is why whole-library prompts became a real tactic.

How much does 1.5 Pro cost per million tokens?

On the 002 pricing table, input is $1.25 per million tokens below 128k and $2.50 above that, with output at $5.00 and $10.00. Cached input drops to about $0.31 per million tokens, which is why teams freeze a repo once and query it all week.

Does 1.5 Pro beat Claude 3.5 Sonnet for all coding?

No. Claude 3.5 Sonnet led isolated synthesis on HumanEval at 93.7 percent versus 84.1 percent for 1.5 Pro, and it still wins a lot of single-file first drafts. 1.5 Pro's edge is the million-token window, cheaper cached input, and native video plus audio sitting next to the source tree.

Can you mix screenshots and code in one prompt?

Yes. 1.5 Pro natively interleaves text, code, images, audio streams (up to 107 hours), and video (up to 10.5 hours) in a single sequence. That is the multimodal debugging case: a screencast, a stack trace, and the module that threw it, without a separate vision pipeline.

What is Context Caching on the API?

Context Caching stores the precomputed KV attention state of a static prefix, like a 500,000-token repo, on TPU memory so later calls skip the expensive prefill. According to Google, that path cuts cached token processing cost by 64 percent, and cached input pricing sits near $0.31 per million tokens.