Is Redis' Throne Threatened? An Analysis of the New Generation of In-Memory Databases

💡 TL;DR (Too Long; Didn't Read)

For years, Redis was the default choice for cache and in-memory data storage, largely due to its simplicity and performance in a single-threaded model. However, modern hardware evolved (multiple cores), and Redis' model became a bottleneck. DragonflyDB emerges as a modern competitor, built from scratch with a multi-threaded, shared-nothing architecture, promising orders of magnitude better performance on the same hardware. At the same time, Redis' controversial license change (from BSD to SSPL) alienated part of the community and gave birth to Valkey, an open-source fork maintained by the Linux Foundation and backed by giants like AWS and Google. Redis' throne isn't lost, but for the first time, it faces an existential threat on two fronts: one technological and one philosophical.

---

The Unquestioned King and the Single-Threaded Ivory Tower

Fellow engineers,

There's a small group of technologies that have become part of our mental infrastructure. Just as we use git for versioning and ssh for access, when we need a fast cache, a simple message broker, or a remote data structure, we use Redis. For over a decade, Redis has been the default answer, the unquestioned king of in-memory databases.

Redis' beauty has always been in its elegant simplicity. Its single-threaded event processing model, combined with non-blocking I/O, was a design masterpiece. It eliminated concurrency complexity (no locks, no deadlocks) and, in an era when processors scaled in clock speed, was incredibly efficient. Redis gave us predictable performance and an API that was a pleasure to use.

However, the fundamental law of hardware changed. Clock speed stagnated, and the industry moved to processors with multiple cores. Suddenly, Redis' single-threaded fortress began to look like an ivory tower — elegant, but isolated from the modern world of massive parallelism. Running multiple Redis instances on a single machine to use all cores became a common pattern, but it always felt like a workaround, not a solution.

Today, Redis' throne is under attack. And it's not just any attack. It's a two-front offensive: one challenging its fundamental architecture and the other, its very open-source soul. Let's coldly analyze the rise of DragonflyDB and Valkey and understand if we're witnessing a king's twilight.

The Technological Threat: DragonflyDB and the Multi-threaded Revolution

DragonflyDB isn't just a "faster version of Redis". It's a complete rewrite, designed from scratch for 21st-century hardware. The team behind it looked at Redis' Achilles heel — the single-thread bottleneck — and built a fundamentally different architecture.

Dragonfly's magic lies in two main concepts:

1. Shared-Nothing Architecture: Unlike simply adding threads, DragonflyDB implements a model where each thread manages its own slice of the data dictionary. A key "user:123" belongs to a specific thread, and only that thread can modify it. This eliminates the need for global locks, which kill performance in multi-threaded systems. Communication between threads is done through message queues, a much more scalable pattern.
2. Efficient Data Structures: DragonflyDB uses more advanced data structures, like the Dash Table, which are optimized for concurrent access and offer better CPU cache performance. This allows it to achieve much higher throughput and lower latencies, especially in write-heavy workloads.

Benchmarks: Smoke or Fire?

The benchmarks published by DragonflyDB are impressive, claiming to be up to 25 times faster than Redis. The community, rightly, received these numbers with a healthy dose of skepticism. Database benchmarks are notoriously difficult to replicate and can be tuned to favor one system over another.

However, discussion on Hacker News and engineering blogs reveals a more subtle truth: even if the "25x" number is marketing, the performance advantage on modern hardware with many cores is real and significant. Companies that migrated report consolidation of dozens of Redis instances into a single Dragonfly node, with drastically lower infrastructure costs and improved P99 latency.

DragonflyDB's threat is clear: it offers an order of magnitude improvement in performance and cost efficiency, transforming Redis' architectural argument from a virtue into a limitation.

The Philosophical Threat: The License Change and Valkey's Birth

While DragonflyDB attacked the technological front, Redis Inc. (the company behind Redis) itself opened a second battle front: the philosophical one. In March 2024, they announced that future Redis versions would no longer be released under the BSD license, but rather under the SSPL (Server Side Public License), a "source-available" license that prohibits cloud providers from offering Redis as a service without a commercial agreement.

The justification was to protect the project from "exploitation" by large cloud providers (read: AWS). The consequence, however, was a fracture in the open-source community. For many, Redis ceased to be truly "open source".

The response was immediate and decisive. Led by the Linux Foundation and with support from former Redis maintainers and industry giants like AWS, Google Cloud, and Oracle, a Redis fork was created: Valkey.

Valkey is essentially a continuation of Redis 7.0.12 under the original BSD license. It promises:

• Open Source Continuity: Maintain the project under an open and community governance model.
• Total Compatibility: Be a drop-in replacement for Redis.
• Active Development: They're already planning to add new features, like more robust clustering and search vectors, which may not be in Redis' commercial version.

Valkey's threat is existential. If the industry's biggest players and the developer community adopt Valkey as the "true" successor to open-source Redis, Redis Inc. may find itself isolated, maintaining a niche commercial product while the rest of the world moves on with the fork.

Conclusion: The Wavering Throne

So, is Redis dead? No, far from it. The installed base is gigantic, the documentation is impeccable, and for many workloads, its performance is more than "sufficient". Redis continues to be a fantastic and reliable tool.

However, the era of its unquestioned sovereignty is over. As engineers, we can no longer simply type apt-get install redis without thinking. Now we have a choice to make, and that choice depends on the problem we're solving:

• Do you need maximum performance and cost efficiency on modern hardware? DragonflyDB deserves serious analysis. The performance gain may justify adopting a new technology.
• Do you value pure open source and want a guarantee that your infrastructure won't be locked into a restrictive license? Valkey is now the safest path and the spiritual successor to the Redis we knew and loved.
• Do you already have a massive Redis infrastructure and don't have an immediate performance bottleneck? Staying with your current Redis version (pre-SSPL) is perfectly reasonable, but a migration plan to Valkey should be on your radar.

Redis' throne isn't empty, but it's wavering. Competition and controversy are signs of a healthy and evolving ecosystem. For us, engineers, this is great news.

It is a reminder to evaluate our defaults with fresh eyes. It means more options, better tools, and most importantly, it forces us to think critically about the technologies we choose, instead of simply following the king.