Separation of storage and compute#
In 2006, AWS launched a simple product without fanfare. It let developers store + retrieve unlimited numbers of files, and it promised not to lose track of them:
All software does is basically manipulate files, and S3 (Simple Storage Service) was a bottomless filing cabinet.
Renting it was much cheaper than buying your own filing cabinet of merely moderate size.
AWS launched S3’s soulmate, EC2 (Elastic Compute Cloud) in a similarly low-energy way in August of 2006. It let you rent as many servers as you needed, whenever you needed them.
This pairing (the foundation of the Cloud) made developers rather thirsty. By separating compute from storage as separate cloud services, you could reimagine stodgy, ubiquitous infrastructure like databases.
The creators of Snowflake were two of those developers, and the analytics warehouse was a ripe first target. As Snowflake co-founder Benoit Dageville put it:
"I always said that as older software guys we were at the mercy of hardware and its resources. The cloud was a game changer for analytic workloads that has lots of peaks and troughs in usage. It really needed that elasticity the cloud offered."
Him and his co-founder Thierry Cruanes left Oracle and started Snowflake in 2012, which today has an ~$80B market cap. Larry Ellison had no comment.
These two AWS launches from 2006 unlocked an untold number of novel data infrastructure companies to be built.
- Analytics + machine learning: Databricks, Snowflake’s nemesis for data engineering workloads
- Transactional databases: Neon, the transactional database (acquired by Databricks)
- Streaming: WarpStream, the real-time event streaming service (acquired by their competitor Kafka)
- Search: Turbopuffer
All have succeeded by rebuilding foundational data tooling on top of object storage like S3.
Enter vector search#
AI agents make analytics workloads look quaint by comparison. Just the vector search piece is a perfect storm of how to run up a cloud bill:
- Embeddings are big. If a chunk of text is 1KB, an embedding vector representing the meaning of that text could be 6KB.
- Agents search more than humans. Way more. If you watch a coding agent work, it runs tons of searches. I usually only search “Oprah net worth” once a week.
- Agents produce more vectors. More
slop code + documents means the search index grows faster than it would if it was just people making stuff.
Search engines like Algolia or Elastic at this time were designed for a person searching in a search box on a website. They needed instant feedback, so data was stored in RAM. High cost was a tradeoff for low latency.
For agents using vector search, the tradeoff needed to be different. High cost was untenable, so the solution would be to trade a little latency.
There’s debate as to whether agents need vector search, or whether they can just use keyword search (the “grep” tool) to hunt-and-peck their way through your codebase or documents.
Claude Code famously (currently) doesn’t actually index your codebase into a searchable vector index.
This is a pretty silly debate: not every use case is the same.
