Cache and Redis Explained: A Complete System Design Guide
Caching is one of the most powerful techniques in system design. It can turn a slow, overloaded application into a fast, scalable one — without changing the underlying database. In this guide, you will learn what a cache is, why we need it, the different types of cache (from CPU to Redis), and how to choose the right one for your use case.
Whether you are a beginner learning system design or an experienced developer preparing for interviews, this guide covers everything with clear explanations and Java examples.
What is a Cache?
A cache is a temporary storage layer that holds frequently accessed data closer to where it is needed. Instead of fetching the same data repeatedly from a slow source (like a database, disk, or remote API), the application retrieves it from a fast, nearby storage layer.
Think of a cache as a cheat sheet — it stores the answers you use most often, so you don't have to look them up every time.
Why Do We Need a Cache?
Caching improves system performance by storing frequently accessed data in a fast-access storage layer. It reduces latency, minimizes database and backend service load, increases throughput, and enhances scalability. Here's why caching matters:
-
Reduced Latency:
Frequently accessed data is served from a high-speed storage layer (cache) instead of slower storage systems such as databases, disks, or external services, resulting in lower response times.
-
Lower Backend Load:
Caching reduces the number of requests hitting databases, APIs, and downstream services, decreasing CPU, memory, and I/O utilization on backend systems.
-
Higher Throughput:
Since fewer resources are spent retrieving the same data repeatedly, the system can process a larger number of requests per second (RPS).
-
Improved Scalability:
By offloading read traffic from the primary data source, caching enables the application to handle increased user traffic without requiring proportional scaling of backend infrastructure.
Types of Cache
There are many ways to classify caches, but in software engineering and system design, the most common types are:
- CPU Cache (L1, L2, L3)
- Browser Cache
- Application Cache
- Database Cache
- CDN Cache
- Distributed Cache (Redis, Memcached)
1. CPU Cache (L1, L2, L3)
CPU cache is a small, extremely fast memory located inside or very close to the CPU. It stores recently accessed data and instructions so the processor does not have to fetch them repeatedly from slower memory (RAM). There are three primary levels:
L1 Cache — Fastest, Smallest
- Fastest cache level
- Smallest in size
- Located inside each CPU core
- Stores the most frequently accessed instructions and data
Example in Java:
public class L1Tutorial {
public static void main(String[] args) {
int total = 0;
for (int i = 0; i < 1000; i++) {
total += i;
}
System.out.println(total);
}
}
In the above program, the variables i and total are accessed
repeatedly. Due to temporal locality, the CPU can keep the associated data
in registers and cache lines, reducing the need for repeated RAM accesses.
Note: The CPU does not store Java variables directly in L1 cache. The JVM and CPU manage memory automatically using registers, cache lines, and memory hierarchies.
L2 Cache — Larger, Slightly Slower
- Larger than L1 cache
- Slightly slower than L1 but still very fast
- Typically dedicated to each CPU core
Example in Java:
public class L2Tutorial {
public static void main(String[] args) {
int size = 50_000_000;
int[] arr = new int[size];
for (int i = 0; i < size; i++) {
arr[i] = i;
}
long total = 0;
// Sequential access → good cache utilization
for (int i = 0; i < size; i++) {
total += arr[i];
}
System.out.println(total);
}
}L2 cache benefits from sequential memory access because the CPU loads memory in cache lines (typically 64 bytes). When adjacent elements are accessed, there is a high probability that they are already available in the cache.
L3 Cache — Largest, Shared
- Largest CPU cache level
- Slower than L1 and L2
- Typically shared among multiple CPU cores
Example in Java:
import java.util.concurrent.*;
public class L3CacheExample {
private static final int SIZE = 50_000_000;
private static final int THREADS = 4;
public static void main(String[] args) throws Exception {
int[] data = new int[SIZE];
for (int i = 0; i < SIZE; i++) {
data[i] = 1;
}
ExecutorService executor = Executors.newFixedThreadPool(THREADS);
long start = System.currentTimeMillis();
int chunk = SIZE / THREADS;
for (int t = 0; t < THREADS; t++) {
int startIdx = t * chunk;
int endIdx = (t == THREADS - 1) ? SIZE : startIdx + chunk;
executor.submit(() -> {
long sum = 0;
for (int i = startIdx; i < endIdx; i++) {
sum += data[i];
}
System.out.println(
Thread.currentThread().getName() + " sum = " + sum);
});
}
executor.shutdown();
executor.awaitTermination(1, TimeUnit.MINUTES);
long end = System.currentTimeMillis();
System.out.println("Total time: " + (end - start) + " ms");
}
}A large array is stored in RAM, and multiple threads read different portions of it. As elements are accessed, the CPU loads nearby data into cache lines. Since the threads only read the data, cache-coherency overhead is minimal.
2. Browser Cache
The browser cache stores frequently used website files on the user's device so they can be loaded locally on subsequent visits.
- Stores files such as images, CSS, JavaScript, fonts, and logos on the user's device.
- When the user revisits the same website, the browser first checks its local cache before downloading files again.
- If the files are already cached and haven't changed, the browser loads them directly from the device.
- This significantly reduces network requests and download time.
- Web pages load faster and consume less internet bandwidth.
- It also reduces the load on the web server because fewer files need to be transferred.
Note: Browser cache improves user experience (faster page loads) and reduces server load at the same time — a win-win.
3. Application Cache (Java)
Application cache refers to data stored directly within the memory space of a running Java application. The cache resides inside the application's process (JVM) and is used to reduce database calls, API calls, or expensive computations by serving frequently accessed data from memory.
Types of Java Application Cache
a) HashMap Cache
Basic in-memory cache using HashMap.
Map<String, String> cache = new HashMap<>();
cache.put("user1", "John");
String value = cache.get("user1");Pros:
- Very simple and fast (O(1) average)
- No external libraries needed
- Good for single-threaded use cases
Cons:
- Not thread-safe
- No eviction (memory keeps growing)
- No expiration support
- Risk of memory leaks in long-running apps
b) ConcurrentHashMap Cache
Thread-safe version of HashMap.
Map<String, String> cache = new ConcurrentHashMap<>();
cache.put("user1", "John");
String value = cache.get("user1");Pros:
- Thread-safe (good for concurrent apps)
- High performance under multi-threading
- No locking on full map
Cons:
- Still no eviction policy
- No TTL (time-based expiry)
- Can grow indefinitely
c) LRU Cache
Evicts the least recently accessed entry when the cache is full.
class LRUCache extends LinkedHashMap<Integer, Integer> {
private final int capacity;
public LRUCache(int capacity) {
super(capacity, 0.75f, true); // accessOrder = true
this.capacity = capacity;
}
@Override
protected boolean removeEldestEntry(Map.Entry<Integer, Integer> eldest) {
return size() > capacity;
}
}Pros:
- Automatically removes least used items
- Prevents memory overflow
- Simple implementation
Cons:
- Not highly scalable under heavy concurrency
- No TTL support by default
- Needs customization for production use
d) Spring Boot Cache
When developers start learning caching in Spring Boot, it's common to assume that Spring Boot
provides multiple cache implementations out of the box. However, that's not entirely accurate.
If you enable caching without adding any external cache library, Spring Boot automatically
configures a Simple Cache Manager. Internally, it uses
ConcurrentHashMap to store cached data in the application's JVM memory.
Characteristics of Simple Cache Manager:
- Stores data in JVM memory.
- Thread-safe because it uses ConcurrentHashMap.
- No cache expiration (TTL).
- No automatic eviction policy.
- Cache data is lost when the application restarts.
- Suitable for development and small applications.
- Best for single-instance deployments.
Example:
@EnableCaching
@Service
public class UserService {
@Cacheable("users")
public User getUser(Long id) {
return userRepository.findById(id);
}
}4. Database Cache (Hibernate)
Database cache stores frequently executed query results, pages, or records to improve database performance.
Types of Hibernate Cache
a) First-Level Cache
- Default cache in Hibernate
- Works at Session level
- Cannot be disabled
Session session = sessionFactory.openSession();
User u1 = session.get(User.class, 1L);
User u2 = session.get(User.class, 1L);Note: Hibernate First-Level Cache stores entities inside a Session so repeated access to the same entity does not trigger multiple database queries.
b) Second-Level Cache
- Optional cache
- Shared across multiple sessions
- Requires configuration + cache provider (e.g., Ehcache, Redis)
c) Query Cache
- Caches query results instead of entities
Query query = session.createQuery(
"FROM User WHERE status = 'ACTIVE'"
);5. CDN Cache
A CDN (Content Delivery Network) cache stores content at edge locations closer to users. Instead of every user fetching from your origin server, they fetch from the nearest CDN node.
Commonly Cached Content
- Images
- Videos
- HTML pages
- CSS / JS files
Not Suitable For
- Highly dynamic content
- Real-time transactions
- Database queries
6. Distributed Cache (Redis)
A distributed cache stores data across multiple servers (e.g., Redis, Memcached). It provides scalability, high availability, and shared access across multiple application instances.
This is the type of cache used in modern microservices and large-scale systems, because it is not tied to a single JVM.
Why Redis is So Popular
Redis is the most widely used distributed cache today. Here's why:
- Very fast (in-memory, microsecond latency)
- Works as both cache and database
- Supports multiple data structures (strings, lists, sets, hashes, sorted sets, streams)
- Distributed cache (works across multiple servers)
- Highly scalable (supports clustering and replication)
- Data persistence option (can save data to disk if needed)
- Built-in TTL (auto expiry of cached data)
- Atomic operations (safe in concurrent systems)
- Supports pub/sub messaging (real-time communication)
- Useful for real-time applications (chat, gaming, notifications, leaderboards)
- Reduces load on database significantly
- More flexible than simple caches like Memcached
- Better for dynamic data compared to CDN caching
- Can handle high-traffic workloads efficiently
Frequently Asked Questions
What is a cache in system design?
A cache is a temporary storage layer that holds frequently accessed data closer to where it is needed. It reduces latency, lowers backend load, increases throughput, and improves scalability by avoiding repeated access to slower data sources such as databases or remote servers.
What is Redis and why is it used as a cache?
Redis is an in-memory data store used as a distributed cache. It is very fast (microsecond latency), supports multiple data structures (strings, lists, sets, hashes, sorted sets), provides TTL-based expiration, atomic operations, replication, clustering, and optional persistence — making it ideal for high-traffic applications.
What are the different types of cache?
The main types of cache are CPU cache (L1, L2, L3), browser cache, application cache (HashMap, ConcurrentHashMap, LRU, Spring Boot Cache), database cache (Hibernate first-level, second-level, query cache), CDN cache, and distributed cache (Redis, Memcached).
What is the difference between HashMap, ConcurrentHashMap, and LRU cache in Java?
HashMap is fast but not thread-safe and has no eviction.
ConcurrentHashMap is thread-safe and high-performance under multi-threading but
still lacks eviction and TTL. LRU cache
(LinkedHashMap with accessOrder=true) evicts the least recently
used entry when full, preventing memory overflow.
What is the difference between first-level and second-level cache in Hibernate?
First-level cache is enabled by default and works at the Session level — it cannot be disabled. Second-level cache is optional, shared across multiple sessions, and requires configuration plus a cache provider such as Ehcache or Redis.
Conclusion
Caching is a foundational concept in system design. From CPU L1 cache to Redis distributed cache, each layer serves a specific purpose — reducing latency, lowering backend load, and enabling systems to scale.
For beginners: start by understanding why caching exists and how a simple
HashMap or Spring Boot @Cacheable works. For experts: choose the
right cache layer for each problem — LRU for in-memory, Hibernate second-level for
database-heavy apps, CDN for static assets, and Redis for distributed, high-traffic systems.
Related topics: Redis Data Structures, Spring Boot Caching, Hibernate Second-Level Cache, Cache Eviction Policies (LRU, LFU, FIFO), Cache Stampede, Cache Invalidation Strategies
No comments:
Post a Comment