Why do developers choose different types of databases?

You wouldn’t think there’d be so much fanfare – and such colossal budgets – around data storage. And yet there are 6 public companies in the U.S. that have built billion+ dollar database businesses, from $ORCL to $ESTC to $SNOW. Not to mention the public clouds that sell database products ($AMZN, $GOOG, $DOCN, etc.), plus supporting tooling for infrastructure ($DDOG, $NEWR, etc.).

How should you understand the differences between all of these seemingly similar databases? What do developers actually use databases for anyway? And why would one be better than another?

This post will run through everything that an investor needs to know to effectively understand the difference between different databases and how they’re positioned in the market. So far in this category, we’ve covered Elastic, MongoDB, Snowflake, and Confluent, plus AWS.

If you’re still getting up to speed on what databases are, you’ll want to read Technically’s beginner's guide to databases. We also have more in depth posts on a few of the categories listed there, like production databases,relational databases, NoSQL databases, and more.

You can also refer back to the database database if you like having a cheat sheet.

What developers use different databases for#

There’s an incredible diversity of database use cases, and it can get overwhelming to understand them all. Why would you use Redis for this? Couldn’t you just use Postgres? Do I need a Vector Database?

If you’re trying to understand a new data store, or the difference between two, ask these two questions.

How does the data get used?#

The first question to ask is: what’s the point of this data in the first place? What apps does it power, what does it help a developer accomplish? Is it for:

A SaaS-style app, like Gmail or Salesforce? This is a production database, and you have many options, from relational databases like PostgreSQL and MySQL to NoSQL platforms like MongoDB or Firebase.

Long analytics queries from a data team? This is a data warehouse, and you might be picking between Snowflake, Redshift, BigQuery, and Clickhouse.

A real-time credit card fraud detection system? This could be any combination of things, including an in-memory database like Redis, a streaming solution like Kafka and Confluent, and an operational data store like Materialize.

A RAG pipeline for a Large Language Model? This could be a dedicated Vector Database like Pinecone, or one of the large platform databases like PostgreSQL (with Timescale, perhaps).

Typically, earlier in a company lifecycle, they’ll have a simpler, smaller set of use cases. Any company with a web application is going to need a database to store their user and app data. Then maybe as they grow, they add a data team who is curious about analytical trends over time. At huge scale, they want to start checking for payment fraud. It’s mostly larger companies who have many of these use cases at once. This is why it’s not always MongoDB vs. Redis for a large organization: they could conceivably be using a dozen different databases for different use cases all at once.

It’s important to note that you can use a regular old relational database for all of these things – it just won’t work as well as dedicated solutions. PostgreSQL can be a data warehouse, and was for many years; but analytics is important enough to most companies to pay up for Snowflake. The reverse is not true: Snowflake cannot be a production database for your SaaS app.

What does the data look like?#

“Data” is not monolithic. Depending on what you’re using the data for (see above), the shape, size, and nature of data is wildly different. A few questions worth asking:

• Is the data large, or small?
• Is it well organized, or haphazardly put together?
• Does the data need to be 100% correct at all times, or can it be mostly correct?
• Does it need to be near real-time, or can queries take 10 seconds?
• Are there any regulatory constraints like HIPAA it needs to adhere to?

Data that’s highly organized, needs fast queries, and is of moderate size is a good fit for relational databases. Data that’s small and needs to be real-time is a good fit for in-memory databases. NoSQL is a good fit for huge, disorganized data. The list goes on.

In the beginner’s guide to databases, I organized all database use cases into 3 categories:

1. Databases that power a user-facing app – production databases that store the data you need for your app to run.
2. Databases that power analytics – databases for analysis, machine learning, and anything a data team does.
3. Databases that power operations – databases for monitoring, logs, security, and any internal processes that enable the above.

I’m writing this post a couple of years later and I think the lines are more blurry, especially with the “one database to rule them all” wave happening. Speaking of which…

One database to rule them all#

What has made life difficult for analysts over the past few years – and honestly, developers too – is that larger database companies have been expanding the scope of their use cases. Products like MongoDB are trying to be all in one databases, solving for multiple different use cases instead of the initial one they were created for (in MongoDB’s case, transactional NoSQL). Look at MongoDB’s homepage, and how their messaging reflects the “one database to rule them all” idea:

From developers I’ve spoken to, a lot of this is just marketing – MongoDB is still severely feature limited in some of these areas like time series or AI. But it doesn’t have to be as good as use-case-specific competitors, it just needs to be good enough for someone who is already (considering) using MongoDB for something else.