What are code sandboxes?

Sandboxes.

Though for most of my readers (who for some reason I assume were born in the 90’s) this word may evoke a quaint yet hauntingly-long-ago visage of their suburban childhoods, it means something else entirely to a software developer.

The TL;DR#

Sandboxes are isolated computing environments where you can run code that you don’t entirely trust. Think of them as solitary confinement, but for a program.

• Much of software today requires you to run untrusted code – code written by a Large Language Model or even a human you don’t know.
• Running untrusted code on your system is risky: it can breach your security, delete your data, and generally just mess with things.
• A sandbox puts iron walls around this untrusted code so it can’t mess with things, while still allowing it to run and (hopefully) accomplish its task.
• Sandboxes are absolutely exploding in popularity because of several different LLM-related use cases.

Sandboxes have been around for as long as I’ve been in tech. For years, products like Retool (my former employer) have needed ways to let their users run arbitrary code…without being subject to the risks of their users running arbitrary code.

But with LLMs taking the center stage, they’ve gotten a second wind for the ages. There are now dozens (dozens!) of sandboxes for LLM startups vying for the business of small companies and enterprises alike who need a place to run random code. And I will explain them to you.

Coding agents, RL training, and other forms of untrusted code#

To understand why sandboxes are having such a moment, we must first detour to understand this concept of “untrusted code” a little more deeply.

Simply put, untrusted code is code that you, yourself, have not examined closely line by line. Because you haven’t examined it closely line by line, you don’t know what it does. And because you don’t know what it does, running the code in the same places that you have your other code running is very risky. Who knows what it’s going to do?

Imagine you’re a prison warden (if I have any readers who are prison wardens, let me know in the comments). You’ve got your hundreds of inmates who are, for the most part, playing pretty nicely together. Then you get a call that a new prisoner is on his way; you don’t know anything about him, just that he’s rumored to be a high-profile gangbanger. Is he well behaved? Could he incite a riot? Will he try to escape? Who knows.

With that uncertainty, you’re certainly not going to put him in Cell Block A with everyone else. Instead, you’ll cordon him off in solitary confinement where whatever he does won’t impact the orderly conduct of the rest of your subordinates.

In software engineering this is referred to as isolation. The prison is your infrastructure – the server your existing code runs on, the database it’s backed by, the APIs powering it. Everything for the most part is working well, and you don’t want to risk a new inmate fucking around with things.

So who are the gangbangers in our analogy? There are a few:

• Internal coding agents – developers using LLMs to generate code for new features, fixing bugs, etc. This code is untrusted, so it needs a sandbox to test it before merging into the rest of the codebase.
• Reinforcement Learning (RL) – when teams train their own coding agents, they need environments where the agents can play around and learn how to get better. Since the code they’re generating and running is (very) untrusted, they need a sandbox.
• (old school) Users – sometimes you want your users to be able to write code and run it on your systems.

Sandboxes give developers an isolated environment to run and test these outputs – these gangbangers if you will – where they physically cannot negatively impact other parts of your stack. If they mess up, if they turn malicious, if they start a riot, the scope of damage can never extend beyond the sandbox itself.

By the way, this gangbanging is not theoretical. Code can absolutely get nasty and mess with your stuff. You’ve probably seen examples on X of coding agents “accidentally” deleting someone’s entire database. And for RL, logs of coding agents in sandboxes often show them attempting destructive behavior because they’re basically try anything (that’s how RL works).