{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiesbtmknmrqbvdi2gykgwbudr6x5kspafhsjmiganom5kisc5clpi",
    "uri": "at://did:plc:25rdn5elo5izoxrmtis34zuk/app.bsky.feed.post/3monva4hcxet2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreibecfyr2ljnx2hi6kcxkcnq4h3cysnudz5bey2nfrdljt44u5oxwi"
    },
    "mimeType": "image/webp",
    "size": 64040
  },
  "path": "/abhinov007/building-the-in-memory-store-strings-lists-hashes-and-expiry-301g",
  "publishedAt": "2026-06-19T17:20:10.000Z",
  "site": "https://dev.to",
  "tags": [
    "backend",
    "database",
    "javascript",
    "tutorial"
  ],
  "textContent": "In the previous post, I wrote about RESP — the protocol layer that lets a Redis server understand commands coming over TCP.\n\nBut parsing a command is only the first step.\n\nOnce the server receives something like:\n\n\n\n    SET name Alice\n\n\nand the RESP parser converts it into:\n\n\n\n    [\"SET\", \"name\", \"Alice\"]\n\n\nthe next question is:\n\n**Where does this data actually live?**\n\nThat is what this post is about.\n\nIn this part of my Redis clone, I built the in-memory storage layer that supports:\n\n  * Strings\n  * Lists\n  * Hashes\n  * Key expiry\n  * Type validation\n  * Lazy expiry on access\n\n\n\nThis is the layer where the server starts feeling like an actual database.\n\n##  Why the Storage Layer Matters\n\nAt first, a Redis clone sounds like it could just be:\n\n\n\n    const store = {};\n\n\nor:\n\n\n\n    const store = new Map();\n\n\nAnd for the most basic version, that is true.\n\nIf all you want is:\n\n\n\n    SET name Alice\n    GET name\n\n\nthen a simple key-value map works.\n\nBut Redis is not just a plain key-value object.\n\nRedis has data types.\n\nA key can store a string.\n\nAnother key can store a list.\n\nAnother key can store a hash.\n\nSome keys may expire after 10 seconds.\n\nSome commands should only work on specific data types.\n\nSo the storage layer needs to track more than just:\n\n\n\n    key -> value\n\n\nIt needs to track:\n\n\n\n    key -> value\n    key -> type\n    key -> expiry time\n\n\nThat is what turns a simple JavaScript object into a Redis-like in-memory database.\n\n##  The Basic Data Model\n\nThe simplest mental model for the store is:\n\n\n\n    database:\n      name      -> Alice\n      queue     -> [task1, task2, task3]\n      user:1    -> { name: Bob, age: 30 }\n\n    expiry:\n      session   -> expires at timestamp\n\n\nSo internally, the system has two major responsibilities:\n\n  1. Store the actual data.\n  2. Track when keys should expire.\n\n\n\nI kept these concerns separate.\n\nThe database module focuses on storing and retrieving values.\n\nThe expiry module focuses on TTL metadata.\n\nThis separation makes the design easier to reason about.\n\n##  Strings\n\nStrings are the simplest Redis data type.\n\nExample:\n\n\n\n    SET name Alice\n    GET name\n\n\nThe server stores:\n\n\n\n    name -> Alice\n\n\nWhen the client runs:\n\n\n\n    GET name\n\n\nthe server checks:\n\n  1. Does the key exist?\n  2. Has the key expired?\n  3. Is the key storing a string?\n  4. If yes, return the value.\n\n\n\nThe response is encoded back in RESP format.\n\nFor example, if the value is `Alice`, the server returns:\n\n\n\n    $5\\r\\nAlice\\r\\n\n\n\nIf the key does not exist, it returns a null bulk string:\n\n\n\n    $-1\\r\\n\n\n\nThe string data type is simple, but it becomes important because many other commands depend on the same storage rules:\n\n  * type checking\n  * expiry checking\n  * persistence\n  * replication\n  * response encoding\n\n\n\nA simple `GET` command still passes through multiple layers of the system.\n\n##  SET with Expiry\n\nRedis allows setting keys with expiry:\n\n\n\n    SET session abc EX 60\n\n\nThis means:\n\n\n\n    Store session = abc\n    Expire it after 60 seconds\n\n\nIn my Redis clone, the value is stored in the main database, while the expiry timestamp is stored separately.\n\nSo conceptually:\n\n\n\n    database:\n      session -> abc\n\n    expiry:\n      session -> currentTime + 60 seconds\n\n\nThis separation makes expiry easier to manage.\n\nThe value does not need to know about its own expiry.\n\nThe expiry system handles that concern.\n\n##  Lazy Expiry\n\nThere are two common ways to expire keys:\n\n\n\n    Active expiry\n    Lazy expiry\n\n\nActive expiry means a background process keeps scanning for expired keys and deletes them.\n\nLazy expiry means the server checks whether a key has expired only when someone tries to access it.\n\nFor example:\n\n\n\n    SET token abc EX 10\n\n\nAfter 10 seconds, the key is expired.\n\nBut instead of deleting it immediately in the background, the server can wait until a client runs:\n\n\n\n    GET token\n\n\nAt that moment, the server checks:\n\n\n\n    Does token have an expiry timestamp?\n    Is the current time greater than the expiry timestamp?\n    If yes, delete token and return null.\n\n\nSo the flow becomes:\n\n\n\n    GET token\n       ↓\n    check expiry\n       ↓\n    expired?\n       ↓\n    delete key + expiry metadata\n       ↓\n    return null\n\n\nThis approach keeps the system simpler.\n\nIt also taught me an important database design idea:\n\n> Sometimes cleanup does not have to happen immediately. It just has to happen before the data is observed.\n\n##  Why Expiry Affects Everything\n\nExpiry sounds like a small feature, but it touches many parts of the database.\n\nFor example:\n\n###  GET\n\nBefore returning a key, the server must check whether it is expired.\n\n###  DEL\n\nWhen a key is deleted, its expiry metadata should also be removed.\n\n###  FLUSHALL\n\nWhen the database is cleared, the expiry store should also be cleared.\n\n###  Persistence\n\nIf a key has expiry metadata, that information needs to be preserved or handled correctly during save/load.\n\n###  Replication\n\nIf the master writes a key with expiry, the replica needs to receive the same write behavior.\n\n###  Sandbox\n\nIf a key has TTL, the UI should show the countdown clearly.\n\nSo expiry is not just an extra field.\n\nIt becomes a cross-cutting concern.\n\n##  Lists\n\nThe next data type I implemented was Lists.\n\nRedis lists are useful for queues, stacks, timelines, and task buffers.\n\nSupported commands include:\n\n\n\n    LPUSH queue a b c\n    RPUSH queue d e\n    LPOP queue\n    RPOP queue\n    LLEN queue\n    LRANGE queue 0 -1\n\n\nA list is stored internally as an ordered array-like structure.\n\nExample:\n\n\n\n    LPUSH queue task1\n    LPUSH queue task2\n\n\nThe list becomes:\n\n\n\n    queue -> [task2, task1]\n\n\nBecause `LPUSH` inserts on the left.\n\nIf we run:\n\n\n\n    RPUSH queue task3\n\n\nthe list becomes:\n\n\n\n    queue -> [task2, task1, task3]\n\n\nThis was useful because it forced me to think about command semantics.\n\n`LPUSH` and `RPUSH` sound similar, but they mutate different ends of the list.\n\n`LPOP` and `RPOP` also remove from different ends.\n\n##  LRANGE and Index Handling\n\nOne of the more interesting list commands is:\n\n\n\n    LRANGE queue 0 -1\n\n\nThis means:\n\n\n\n    Return all elements from index 0 to the last element.\n\n\nRedis supports negative indexes.\n\nSo:\n\n\n\n    LRANGE queue -2 -1\n\n\nmeans:\n\n\n\n    Return the last two elements.\n\n\nThat means the clone needs to normalize indexes.\n\nThe server has to convert:\n\n\n\n    start = -2\n    stop = -1\n\n\ninto real array indexes based on the list length.\n\nThis small feature makes the implementation more realistic.\n\nIt is not just pushing and popping.\n\nIt is matching Redis-like behavior.\n\n##  Hashes\n\nHashes are another important Redis data type.\n\nThey let you store field-value pairs under a single key.\n\nExample:\n\n\n\n    HSET user:1 name Bob age 30\n    HGET user:1 name\n    HGETALL user:1\n\n\nConceptually:\n\n\n\n    user:1 -> {\n      name: Bob,\n      age: 30\n    }\n\n\nThis is useful for storing object-like data.\n\nIn the clone, hashes support commands like:\n\n\n\n    HSET user:1 name Bob age 30\n    HGET user:1 name\n    HDEL user:1 age\n    HGETALL user:1\n    HLEN user:1\n    HEXISTS user:1 name\n\n\nThe interesting part here was handling multiple fields in one command.\n\nFor example:\n\n\n\n    HSET user:1 name Bob age 30 city Delhi\n\n\nThis command contains multiple field-value pairs.\n\nThe command handler needs to validate that fields and values are paired correctly.\n\nIf there is a missing value, the command should return an error.\n\nThat kind of validation is what makes the command engine feel closer to a real Redis server.\n\n##  Type Checking\n\nOne of the most important parts of the storage layer is type checking.\n\nImagine this:\n\n\n\n    SET name Alice\n    LPUSH name Bob\n\n\nThis should not work.\n\nThe key `name` already stores a string.\n\nA list command should not be allowed on it.\n\nSo the server must return a wrong type error instead of silently converting the value.\n\nThis means each key needs an associated type.\n\nConceptually:\n\n\n\n    name:\n      type: string\n      value: Alice\n\n    queue:\n      type: list\n      value: [task1, task2]\n\n    user:1:\n      type: hash\n      value: { name: Bob }\n\n\nBefore executing a command, the handler checks whether the key has the expected type.\n\nFor example:\n\n\n\n    GET expects string\n    LPUSH expects list\n    HGET expects hash\n\n\nIf the key does not exist, some commands create it.\n\nIf the key exists with the wrong type, the command returns an error.\n\nThis was one of the key differences between a simple map and a Redis-like store.\n\n##  Command Flow Example: SET\n\nLet’s walk through what happens when a client sends:\n\n\n\n    SET name Alice EX 60\n\n\nThe flow looks like this:\n\n\n\n    RESP parser receives raw bytes\n            ↓\n    Parser emits [\"SET\", \"name\", \"Alice\", \"EX\", \"60\"]\n            ↓\n    Command router identifies SET\n            ↓\n    SET handler validates arguments\n            ↓\n    Database stores name = Alice\n            ↓\n    Type is marked as string\n            ↓\n    Expiry store records TTL\n            ↓\n    AOF persistence records the write\n            ↓\n    RDB snapshot is updated\n            ↓\n    Replication layer can propagate the write\n            ↓\n    Server returns +OK\n\n\nSo one command touches:\n\n  * protocol parsing\n  * command validation\n  * storage\n  * expiry\n  * persistence\n  * replication\n  * response encoding\n\n\n\nThat is why even “simple” Redis commands are great for learning systems design.\n\n##  Command Flow Example: LPUSH\n\nNow take:\n\n\n\n    LPUSH queue task1 task2\n\n\nThe server does:\n\n\n\n    Parse command\n            ↓\n    Check whether queue exists\n            ↓\n    If it does not exist, create a new list\n            ↓\n    If it exists, verify it is a list\n            ↓\n    Push values to the left\n            ↓\n    Persist the write\n            ↓\n    Return the new list length\n\n\nIf `queue` does not exist, it is created.\n\nIf `queue` already stores a list, the command mutates it.\n\nIf `queue` stores a string or hash, the command returns a type error.\n\nThis type behavior keeps the database predictable.\n\n##  Command Flow Example: HSET\n\nFor:\n\n\n\n    HSET user:1 name Bob age 30\n\n\nthe server does:\n\n\n\n    Parse command\n            ↓\n    Check whether user:1 exists\n            ↓\n    If not, create a hash\n            ↓\n    Validate field-value pairs\n            ↓\n    Set name = Bob\n            ↓\n    Set age = 30\n            ↓\n    Return number of new fields added\n\n\nHashes were interesting because they introduced nested storage.\n\nA string stores one value.\n\nA list stores an ordered sequence.\n\nA hash stores a map inside the main map.\n\nThat means the storage layer becomes:\n\n\n\n    database map\n       ↓\n    key\n       ↓\n    hash map\n       ↓\n    field -> value\n\n\nThis helped me understand why Redis data types are powerful.\n\nThey let you model different kinds of data without leaving the key-value model.\n\n##  Storage and Persistence\n\nThe storage layer does not live alone.\n\nEvery successful write needs to be persisted.\n\nFor example:\n\n\n\n    SET name Alice\n    LPUSH queue task1\n    HSET user:1 name Bob\n\n\nThese commands should update memory, but they should also be recorded by the persistence system.\n\nThat way, if the server restarts, the database can be rebuilt.\n\nThis is where AOF and RDB connect to storage.\n\nThe storage layer owns the current state.\n\nThe persistence layer records or snapshots that state.\n\nThe relationship looks like:\n\n\n\n    Command handler\n       ↓\n    Storage update\n       ↓\n    AOF append\n       ↓\n    RDB snapshot\n\n\nThis made the architecture cleaner because storage does not need to know how commands arrived.\n\nIt only needs to expose operations that command handlers can use.\n\n##  Storage and Replication\n\nReplication also depends on successful writes.\n\nWhen a write happens on the master, it needs to be propagated to replicas.\n\nSo after a command updates storage, the server can send the same write command to replica connections.\n\nThe flow becomes:\n\n\n\n    Client writes to master\n            ↓\n    Master updates in-memory store\n            ↓\n    Master persists write\n            ↓\n    Master sends command to replicas\n            ↓\n    Replicas apply the same write\n\n\nThis is where command design matters.\n\nIf the command is represented clearly, it can be reused for:\n\n  * local execution\n  * persistence replay\n  * replication propagation\n\n\n\nThat was an important architectural learning.\n\n##  Storage and the React Sandbox\n\nThe React sandbox made the storage layer much easier to understand visually.\n\nInstead of only seeing:\n\n\n\n    +OK\n\n\nafter running a command, the sandbox shows the internal changes.\n\nFor example:\n\n\n\n    SET name Alice EX 60\n\n\nupdates:\n\n\n\n    Database tab\n    Expiry tab\n    AOF log\n    RDB snapshot\n\n\nFor a list command:\n\n\n\n    LPUSH queue task1 task2\n\n\nthe database view shows the list changing.\n\nFor a hash command:\n\n\n\n    HSET user:1 name Bob age 30\n\n\nthe database view shows the hash fields.\n\nThis made the project more explainable.\n\nIt is one thing to say “the database changed.”\n\nIt is much better to see how it changed.\n\n##  What I Learned\n\n###  1. A key-value store is not always simple\n\nAt the beginning, I thought the storage layer would be the easiest part.\n\nBut once data types, expiry, type checking, persistence, and replication are added, it becomes much more interesting.\n\n###  2. Types matter\n\nRedis commands are simple because Redis is strict about what each key contains.\n\nA list command should not work on a string.\n\nA hash command should not work on a list.\n\nThat strictness makes the system predictable.\n\n###  3. Expiry is a system-wide concern\n\nTTL is not just attached to `SET`.\n\nIt affects reads, deletes, persistence, replication, and UI visualization.\n\n###  4. Command semantics are important\n\nCommands like `LPUSH`, `RPUSH`, `LRANGE`, `HSET`, and `HGETALL` all have small behavior details that need to be handled carefully.\n\n###  5. Separating storage and expiry keeps the design clean\n\nThe database stores values.\n\nThe expiry module stores TTL metadata.\n\nThis makes both parts easier to test and reason about.\n\n###  6. A database is a collection of layers\n\nThe in-memory store is only one layer.\n\nIt sits between protocol parsing, command execution, persistence, replication, and client responses.\n\n##  Final Thought\n\nBefore building this, I thought of Redis mainly as:\n\n\n\n    key -> value\n\n\nAfter building the storage layer, I started seeing it more like:\n\n\n\n    key -> typed value\n    key -> expiry metadata\n    commands -> validated mutations\n    writes -> persistence + replication\n\n\nThat shift matters.\n\nA Redis-like database is not just a map.\n\nIt is a carefully designed system where every command has rules, every key has type behavior, and every write can affect persistence, replication, and expiry.\n\nThat is what made this part of the project so valuable.\n\nIn the next post, I’ll go deeper into persistence: how AOF and RDB work, why both exist, and what I learned while implementing them.\n\nRepo:\n\n\n\n    https://github.com/Abhinov007/redis_clone\n\n\nLive sandbox:\n\n\n\n    https:https://redis-clone.vercel.app/\n",
  "title": "Building the In-Memory Store: Strings, Lists, Hashes and Expiry"
}