Skip to main content

Caching with Redis

MongoDB stores everything that needs to last forever. Redis stores everything that needs to be fast.

This page walks through the four jobs Redis does in Dashify, why each one belongs in Redis and not MongoDB, and how the platform stays correct even when caches get out of sync.

What Redis is​

Redis is an in-memory key-value store. It keeps its data entirely in RAM, which makes it dramatically faster than a disk-backed database — typical operations finish in tens of microseconds. It can also persist its data to disk so a restart does not wipe everything, but the read and write paths are pure memory.

The trade-off is that it is not built for the kind of complex, queryable data MongoDB handles. Redis is best at simple, fast operations on simple, fast data.

What Redis does in Dashify​

Four roles, in order of importance.

1. Session storage​

When a user logs in, the API creates a session — a small record with the user id, their tenant id, when the session was created, and from which IP. The session is keyed by a long random session token, and the session token is what gets returned to the browser as a cookie.

Storing sessions in Redis (rather than in MongoDB) matters for two reasons. First, sessions are read on every request — fast lookups are essential. Second, sessions are short-lived; Redis can expire them automatically with a built-in TTL (time-to-live), so we never have to write a cleanup job.

2. Rate limiting​

To stop brute-force attacks and abuse, sensitive routes (login, password reset, signup, API token verify) are rate-limited. The implementation needs to answer "how many times has this IP tried to log in in the last 60 seconds?" fast, atomically, on every attempt.

Redis is purpose-built for this. The rate-limiter-flexible library uses Redis to maintain a counter per IP per route, with automatic expiration. If the counter exceeds the limit, the request is rejected with a 429.

3. Cache for hot lookups​

Some pieces of data are read on essentially every request: the current tenant's package, the role-permission matrix, the tenant settings. Reading them from MongoDB on every request would be wasteful.

The API caches these in Redis with a short TTL (a few minutes). The first request hits MongoDB and warms the cache; the next many requests hit Redis. When the underlying document changes (a SuperAdmin updates a package, an Org Admin changes settings), the cache is invalidated explicitly so the next read sees fresh data.

The cache is conservative — only data that rarely changes and is read often is cached at all. A stale cache is worse than no cache.

4. Pub/Sub for real-time fan-out​

When the platform scales horizontally — multiple API instances behind a load balancer — a real-time event that originates on one instance needs to reach the user's browser, which might be connected to a different instance.

Redis Pub/Sub is the bridge. The Socket.IO Redis adapter publishes every event to a Redis channel; every API instance is subscribed; every instance forwards the event to the browsers it is currently connected to.

User A and User B are both watching the same chat channel. They are connected to different API instances. Without Redis Pub/Sub, A's message would never reach B. With it, B sees the message a few milliseconds after A sends it.

5. Queue backend (BullMQ)​

This is technically a fifth role. BullMQ is the job queue library Dashify uses, and Redis is its storage backend. The API enqueues a job by writing a small record to a Redis list; the worker process consumes jobs by reading from the list. Job state, retry counters, and scheduled triggers all live in Redis.

The queue chapter on Scalability has the full story. For Redis's purposes here, it is enough to know that BullMQ is one of its biggest tenants.

Why not just use MongoDB for all of this?​

MongoDB could technically handle these jobs. The reasons it shouldn't are:

  • Speed. MongoDB reads and writes go to disk (eventually). Redis stays in RAM. For the request-path operations that Redis handles, the speed difference is between "imperceptible" and "noticeable."
  • TTL semantics. Redis has clean, native expiration. MongoDB has TTL indexes but they only sweep periodically, which is unsuitable for sessions and rate limits.
  • Pub/Sub. Redis Pub/Sub is real-time. MongoDB Change Streams exist but are heavier and not designed for this kind of dense, ephemeral fan-out.

Using MongoDB and Redis together is the standard pattern for any reasonably scaled SaaS, and Dashify follows it.

Connection management​

The API and the worker each open a handful of Redis connections — one for normal commands, one for Pub/Sub subscriptions (Pub/Sub blocks the connection so it gets its own), and one or more for BullMQ. The ioredis client manages reconnection with exponential backoff if Redis briefly goes away, so a transient Redis blip does not crash the platform.

What if Redis goes down?​

Two failure modes:

Brief outage (seconds). ioredis reconnects. Sessions stay valid (they are still in Redis when it comes back). Rate limits are temporarily not enforced for that window. Real-time messages queued during the outage are lost (Pub/Sub is fire-and-forget).

Extended outage (minutes or more). Sessions cannot be loaded, so users are effectively logged out. Background jobs cannot be enqueued, so anything that depends on them (audit logs, emails, indexer) is delayed. Real-time stops working. The platform is degraded but the data layer (MongoDB) is unaffected — once Redis is back, everything resumes.

In production this is mitigated by running Redis in a managed cluster with replication and automatic failover. The local-dev stack runs a single Redis instance because the consequences of a brief restart are tolerable.

Memory pressure​

Redis uses RAM. If the dataset grows beyond the RAM Redis has, it starts evicting keys based on the configured policy (allkeys-lru is sensible for a cache). Dashify's keyspaces are bounded by TTL, so memory pressure is rare — but in a heavy production deployment, watch the redis_memory_used_bytes metric on the Grafana dashboard.

Key takeaways​

  • Redis is an in-memory key-value store used for sessions, rate-limits, hot caches, real-time fan-out, and the BullMQ queue backend.
  • It is the right tool for fast, ephemeral, atomic operations — the things MongoDB is not optimised for.
  • Caches are kept conservatively: only rarely-changing, often-read data, with explicit invalidation.
  • A brief Redis outage is recoverable; an extended one degrades the platform but does not corrupt data.