What's a Package Manager?

Intro to package management

The vast, vast majority of software, be it consumer apps like Snapchat and Strava, or B2B platforms like Salesforce and Workday, use open source software as building blocks. This is to say, they use code written and made public by other people to save themselves time when developing their own applications.

A classic example of a use case for leveraging open source code is if your app has anything to do with time, such as displaying the date and time to a user. Time is surprisingly complex: what time zone are users in, are they in daylight savings, do you want to display the day of the week as well? You could code up all this logic yourself, but it's much easier to use code someone else already wrote to do this.

The world of package managers emerges when you ask: how do you actually use someone else’s code?

One naive way might be copy-pasting code from someone else's app or website into your own codebase, line for line. This would work, but is very inefficient. What happens if the author makes changes? What if there’s a security vulnerability discovered down the road?

For these reasons and more, the most common way that developers of open source software distribute their code is by bundling it into packages and publishing them for the world to use via a package manager. You can think of a package as a nice wrapper around the code that someone has authored, boxing up your little (or a lot) bit of code nicely so you can ship it to others without things getting messed up.

Package managers are programs that deal with the logistics of these “packages” of code: in particular, fetching and installing these open-source software packages. They are used almost anywhere people (or robots) write, test, or run code. The key elements of a package management system are the package manager and the package registry. Let's quickly review how they work.

If I'm a developer starting a new project and I want to use someone else’s code for something (like handling dates), one of the first things I'm going to do is ensure I have a package manager installed on my system. The package manager makes it extremely easy to install the packages that I need. Each language maintains one or more official package manager, such as pip for Python, and npm for Javascript.

For the example where I’m coding up an app where I need to display time to my user, if I was building in Python I’d use pip by opening up my terminal and typing pip install pendulum (pendulum is the package name). The package manager does a few things:

First, the package manager checks my system to see if pendulum is installed already. If it is, fantastic, it does nothing. If it is not, it goes out and downloads the package. But where does it go to get the package? It goes to the registry.

The package registry is a public, trusted repository of open source software packages on the internet. Each language maintains one or more official registries, such as PyPI (Python Package Index) for Python, not to be confused with pip, and the npm registry for JavaScript.

You can think of the registry as the warehouse where all of the boxed software goes to be shipped out. The authors of the software are (ideally) constantly uploading new and improved versions of their packages so users like you can benefit from the great wheels of progress.

This general process of ensuring I have the packages I need is referred to as dependency resolution – a dependency just means a package that your software needs in order to run – and it's the core job of the package manager. And it's a full-time job: as I code up more features, my dependencies will change, and the package manager needs to make sure what I need is installed at all times, and what I no longer need is purged from the system.

Over time, many package managers grew beyond just fetching code. See, going from code to real running software requires a few other steps. For example, after code is written and external packages are installed, something called a compiler or bundler needs to translate that code into a language that’s readable by the machines. After that, a different program actually needs to run the code, called the runtime. At least that's how it was for a long time — different tools for different stages of the pipeline.

But newer tools like Bun in the JavaScript world started collapsing this entire toolchain into one fast program that handles everything. This bundling is important context for understanding what AI labs actually bought, which we'll get to shortly.

The thing I want you to know is that for the entirety of my time building software, and for decades before that, the package manager was a thing you used manually, slowly, and pretty infrequently. Starting or pulling a new project, updating a package, or running some tests locally. It was not software that anyone loved or waxed poetic about, it was just a thing you had to do.

This is because coding — making your software do what you want it to do by typing out instructions — takes a long time when humans do it. For human engineers, package management only has to be as fast as humans can decide what packages they need and can use them in their development cycle. Until recently, this was fine, because humans were the only entities that could write code.