# Redis — full corpus # LLM Wiki An open-source template for building LLM-powered knowledge bases, following [Andrej Karpathy's "LLM Wiki" pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f). You provide raw sources. The LLM reads them, writes structured wiki pages, cross-links everything, and maintains it over time. You never edit the wiki directly — you curate sources and ask questions. ## How It Works The system has three layers: ``` raw/ Sources you collect (articles, transcripts, notes, PDFs) wiki/ LLM-written & maintained pages (summaries, concepts, entities, syntheses) CLAUDE.md Schema that tells the LLM how to structure everything ``` Three operations drive the workflow: | Operation | Trigger | What happens | |-----------|---------|--------------| | **Ingest** | "ingest raw/my-source.txt" | LLM reads the source, creates a summary page, creates/updates concept and entity pages, adds cross-links, updates the index and log | | **Query** | Ask any question | LLM searches the wiki, synthesizes an answer with citations, optionally creates a synthesis page for novel insights | | **Lint** | "lint" or "health check" | LLM audits all pages for orphans, contradictions, missing links, incomplete sections, and low-confidence claims — fixes what it can, reports the rest | ## Quick Start 1. **Clone this repo** ```bash git clone https://github.com/YOUR_USERNAME/llm-wiki.git my-knowledge-base cd my-knowledge-base ``` 2. **Customize CLAUDE.md** for your domain - Update the Purpose section with your topic - Replace the placeholder tagging taxonomy with your own categories - Adjust confidence level descriptions if needed - Everything else (workflows, page formats, linking rules) works as-is 3. **Drop sources into `raw/`** - Text files, transcripts, articles, notes — any plain text - These are immutable once added; the LLM never modifies them 4. **Tell the LLM to ingest** ``` ingest raw/my-first-source.txt ``` The LLM will create summary pages, concept pages, entity pages, cross-links, and update the index. 5. **Ask questions** ``` What are the key differences between X and Y? ``` The LLM answers from the wiki, citing specific pages. 6. **Run health checks** ``` lint ``` The LLM audits the wiki and fixes issues. ## Directory Structure ``` . ├── CLAUDE.md # Schema — the LLM's instructions ├── raw/ # Your source documents (immutable) └── wiki/ ├── index.md # Master catalog of all pages ├── log.md # Append-only activity log ├── dashboard.md # Dataview dashboard (Obsidian) ├── analytics.md # Charts View analytics (Obsidian) ├── flashcards.md # Spaced repetition cards ├── summaries/ # One page per source document ├── concepts/ # Concept and framework pages ├── entities/ # People, tools, organizations, etc. ├── syntheses/ # Cross-cutting analyses and comparisons ├── journal/ # Research/session journal entries │ └── template.md # Journal entry template └── presentations/ # Marp slide decks ``` ## Enhancements This template includes several extras beyond the core wiki pattern: ### Dataview Dashboard (`wiki/dashboard.md`) Live queries that surface low-confidence pages, recent updates, concepts by tag, and pages with the most sources. Requires the [Dataview](https://github.com/blacksmithgu/obsidian-dataview) Obsidian plugin. ### Charts View Analytics (`wiki/analytics.md`) Visual analytics with pie charts, bar charts, and word clouds. Requires the [Charts View](https://github.com/caronchen/obsidian-chartsview-plugin) Obsidian plugin. ### Mermaid Diagrams Use Mermaid code blocks in any wiki page to create flowcharts, sequence diagrams, or concept maps. Native support in Obsidian and GitHub. ### Marp Slides (`wiki/presentations/`) Create slide decks from markdown using [Marp](https://marp.app/). Drop presentation files in this directory. ### Research Journal (`wiki/journal/`) Track your research sessions, experiments, or applied work with the included template. The LLM can reference journal entries when answering queries. ### Spaced Repetition (`wiki/flashcards.md`) Flashcards in the format used by the [Spaced Repetition](https://github.com/st3v3nmw/obsidian-spaced-repetition) Obsidian plugin. Ask the LLM to generate flashcards from any wiki page. ### MCP Server This repo works with Claude Code's MCP server capabilities. Point an MCP-compatible client at this repo and the LLM can read/write the wiki programmatically. ## Customizing for Your Domain The schema in `CLAUDE.md` is domain-agnostic. To adapt it: 1. **Purpose** — Describe your knowledge domain in one paragraph 2. **Tagging taxonomy** — Replace placeholder categories with your own (e.g., for a cooking KB: `cuisine`, `technique`, `ingredient`, `equipment`) 3. **Confidence levels** — Adjust the descriptions to match your domain's evidence standards 4. **Entity types** — Update the entity page description to match what entities mean in your domain (people, tools, companies, etc.) 5. **Journal template** — Customize `wiki/journal/template.md` for your workflow Everything else — page format, linking conventions, workflows, rules — is universal and works across domains. ## Example Domains This template works for any knowledge-intensive topic: - **Research notes** — papers, experiments, methodologies - **Book analysis** — themes, characters, author techniques - **Competitive analysis** — companies, products, market trends - **Course notes** — lectures, readings, key concepts - **Personal development** — frameworks, habits, book summaries - **Technical documentation** — APIs, architectures, design patterns - **Hobby deep-dives** — any subject you want to master ## License MIT --- title: "Redis KB — Master Index" type: index updated: 2026-06-25 redis_version: "8.8" --- # Redis KB — Master Index **Domain:** Redis — the in-memory data store: data types, key management & expiration/eviction, pub/sub, transactions, programmability (Lua + Functions), the RESP protocol, client-side caching, the redis-cli, persistence, and operations (replication, Sentinel, Cluster, ACL). **Corpus:** 38 provenance-stamped sources in `raw/` — the core OSS Redis docs (gathered as per-page Markdown from redis.io) plus release notes (to Redis 8.8). Dev Hub tutorials and Cloud/Enterprise content were deliberately excluded. **Pages:** 17 (14 concepts · 1 entity · 1 summary · 1 synthesis). ## Concepts — data types - [[concepts/data-types-overview]] — the data-type model and key concepts - [[concepts/strings]] — strings, plus bitmaps and bitfields (`SET`/`GET`, `SETBIT`/`BITCOUNT`, `BITFIELD`) - [[concepts/lists]] — lists (`LPUSH`/`RPUSH`/`LRANGE`/`LMOVE`, blocking ops) - [[concepts/hashes]] — hashes (`HSET`/`HGET`/`HGETALL`, field expiration) - [[concepts/sets-and-sorted-sets]] — sets and sorted sets (`SADD`, `ZADD`/`ZRANGEBYSCORE`) - [[concepts/streams]] — streams and consumer groups (`XADD`/`XREAD`/`XREADGROUP`/`XACK`) - [[concepts/probabilistic-and-geospatial]] — HyperLogLog (`PFADD`/`PFCOUNT`) and geospatial (`GEOADD`/`GEOSEARCH`) - [[concepts/json]] — Redis JSON module (`JSON.SET`/`JSON.GET`) ## Concepts — interaction & operations - [[concepts/keys-expiration-eviction]] — keyspace, `EXPIRE`/`TTL`/`SCAN`, `maxmemory` eviction policies - [[concepts/pubsub]] — publish/subscribe (`SUBSCRIBE`/`PUBLISH`/`PSUBSCRIBE`, sharded) - [[concepts/transactions]] — `MULTI`/`EXEC`/`DISCARD`/`WATCH` - [[concepts/scripting-and-functions]] — Lua scripting (`EVAL`/`EVALSHA`) and Redis Functions (`FUNCTION LOAD`/`FCALL`) - [[concepts/persistence]] — durability: RDB snapshots and AOF (`appendonly`/`appendfsync`) - [[concepts/high-availability-and-scaling]] — replication, Sentinel, and Cluster ## Entities - [[entities/redis-cli]] — the `redis-cli` command-line client: options and special modes ## Summaries - [[summaries/reference-catalog]] — map of the RESP protocol, configuration, ACL security, and client-side caching (command reference mapped, not paged) ## Syntheses - [[syntheses/data-types-and-caching-patterns]] — choosing the right data type and using Redis as a cache (TTL, eviction, invalidation) ## Statistics - **Total pages**: 17 - **Concepts**: 14 · **Entities**: 1 · **Summaries**: 1 · **Syntheses**: 1 - **Sources ingested**: 38 (raw/, immutable) - **High confidence**: 16 · **Medium confidence**: 1 · **Low confidence**: 0 ## Coverage notes Strong: all core data types, key/expiration/eviction, pub/sub, transactions, scripting & Functions, persistence, and HA/scaling, plus the redis-cli and the RESP/config/ACL reference. The latest mirrored release note is Redis 8.8.0; freshness = source fetch date 2026-06-25. Mapped, not paged (see [[summaries/reference-catalog]]): the exhaustive per-command reference, Redis Cloud/Enterprise, Redis Stack modules beyond JSON (Search/Query, Time Series, Bloom), and client-library specifics. For those, use redis.io. --- title: "Redis Data Types Overview" type: concept tags: [data-types, overview, redis] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md.md] --- # Redis Data Types Overview Redis is a data structure server. At its core, Redis provides a collection of native data types that help solve a wide variety of problems, from caching to queuing to event processing. Each value you store is associated with a key. ## General-purpose data types - **[[concepts/strings]]** — the most basic type, a sequence of bytes. Often used for caching; also supports counters and bitwise operations. Folds in **bitmaps** (bit-oriented operations on a string treated as a bit vector) and **bitfields** (encode multiple integer counters in a string with atomic get/set/increment and overflow policies). - **[[concepts/lists]]** — lists of strings sorted by insertion order; used for stacks and queues. - **[[concepts/hashes]]** — record types modeled as collections of field-value pairs, resembling dictionaries/maps. - **[[concepts/sets-and-sorted-sets]]** — sets are unordered collections of unique strings with O(1) add/remove/membership; sorted sets are unique strings ordered by an associated score. - **[[concepts/streams]]** — append-only log structures that record events in order for later processing. - **[[concepts/json]]** — structured, hierarchical arrays and key-value objects matching the JSON format. ## Specialized data types - **Geospatial indexes** — find locations within a geographic radius or bounding box (see [[concepts/probabilistic-and-geospatial]]). - **Probabilistic data types** — gather statistics approximately but efficiently: Bloom filter, Count-min sketch, Cuckoo filter, HyperLogLog, t-digest, and Top-K (see [[concepts/probabilistic-and-geospatial]]). - **Arrays** — sparse, index-addressable sequences of strings. - **Time series** — store and query timestamped data points. - **Vector sets** — manage high-dimensional vector data for similarity search, supporting the HNSW algorithm and cosine similarity, with native hybrid search combining vector similarity and structured filters. See the [Redis command reference](https://redis.io/commands/) for the full command set per type, and [[summaries/reference-catalog]] for a curated catalog. ## Adding extensions To extend the included data types, you can write custom server-side functions in Lua (see [[concepts/scripting-and-functions]]) or write a Redis module using the modules API. Related: [[syntheses/data-types-and-caching-patterns]]. --- title: "Redis Hashes" type: concept tags: [hashes, field-value, objects, field-expiration] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-5.md] --- # Redis Hashes Redis hashes are record types structured as collections of field-value pairs. Use them to represent basic objects and to store groupings of counters. ## Core commands - `HSET key field value [field value ...]` — create or modify one or more fields; O(1) per pair. - `HGET key field` — value of a single field, O(1). - `HGETALL key` — all fields and values, O(N). - `HMGET key field...` — values of multiple fields, O(N). - `HDEL`, `HEXISTS`, `HLEN`, `HKEYS`, `HVALS`, `HSETNX`, `HSTRLEN`, `HSCAN`, `HRANDFIELD`. ``` > HSET bike:1 model Deimos brand Ergonom type 'Enduro bikes' price 4972 (integer) 4 > HGET bike:1 model "Deimos" > HGETALL bike:1 1) "model" 2) "Deimos" 3) "brand" 4) "Ergonom" 5) "type" 6) "Enduro bikes" 7) "price" 8) "4972" ``` ## Hashes as counters `HINCRBY` increments the integer value of a field (using 0 as the initial value if the field doesn't exist), O(1); `HINCRBYFLOAT` does the same for floating-point values. ## Field expiration Redis 7.4 introduced the ability to set an expiration time or TTL on individual hash fields, comparable to key expiration: - `HEXPIRE` — set the remaining TTL in seconds; `HPEXPIRE` — in milliseconds. - `HEXPIREAT` / `HPEXPIREAT` — set expiration to an absolute Unix timestamp (seconds / milliseconds). - `HEXPIRETIME` / `HPEXPIRETIME` — get the expiration timestamp (seconds / milliseconds). - `HTTL` / `HPTTL` — get the remaining TTL (seconds / milliseconds). - `HPERSIST` — remove a field's expiration. These commands are O(N) in the number of specified fields. Use cases include event tracking (count events in the last hour via per-field TTL + `HLEN`) and fraud detection (hourly counters with a 48-hour TTL). Redis 8.0 added `HGETEX` and `HSETEX`, which get/set field values and optionally set their expiration in a single call. ## Limits and performance Every hash can store up to 4,294,967,295 (2^32 - 1) field-value pairs, limited in practice only by available memory. Most hash commands are O(1); `HKEYS`, `HVALS`, `HGETALL`, and most expiration-related commands are O(n) in the number of field-value pairs. For key-level expiration semantics see [[concepts/keys-expiration-eviction]]. As alternatives for structured data, consider strings ([[concepts/strings]]) or [[concepts/json]]. Related: [[concepts/data-types-overview]], [[entities/redis-cli]]. --- title: "High Availability and Scaling" type: concept tags: [replication, sentinel, cluster, failover, high-availability] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-25.md, raw/web_community-index-html-md-26.md, raw/web_community-index-html-md-27.md] --- # High Availability and Scaling Redis provides three layered mechanisms: **replication** (the base copy mechanism), **Sentinel** (automatic failover for non-clustered setups), and **Redis Cluster** (horizontal sharding with built-in availability). ## Replication Leader-follower (master-replica) replication makes replicas exact copies of a master. Replication is **asynchronous**: the master does not wait for replica acknowledgement before replying to clients, so acknowledged writes can still be lost on failover. Configure a replica in `redis.conf` or at runtime: ``` replicaof 192.168.1.1 6379 > REPLICAOF 192.168.1.1 6379 ``` A replica reconnects automatically and attempts a **partial resynchronization** using `PSYNC` (its old replication ID + offset); if the backlog is gone it does a full resync (the master `BGSAVE`s an RDB and streams subsequent writes). Each master is identified by a `(replication ID, offset)` pair. Replicas are **read-only** by default (`replica-read-only`); writable replicas exist only for historical reasons and risk inconsistency. Key points: - `WAIT` requests synchronous acknowledgement from N replicas but does not give strong consistency. - `min-replicas-to-write` / `min-replicas-max-lag` let a master refuse writes unless N replicas are within a lag bound. - `masterauth ` lets a replica authenticate to a password-protected master. - Replicas don't expire keys themselves; the master synthesizes a `DEL` and propagates it. - By default a replica ignores `maxmemory` (`replica-ignore-maxmemory no` to change). Strongly prefer **persistence on** for masters with auto-restart off — a master that auto-restarts empty will wipe its replicas. ## Sentinel Sentinel provides HA for non-clustered Redis: **monitoring**, **notification**, **automatic failover**, and acting as a **configuration provider** for clients. Run it on TCP port 26379: ``` redis-sentinel /path/to/sentinel.conf sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 60000 sentinel failover-timeout mymaster 180000 sentinel parallel-syncs mymaster 1 ``` The `quorum` (last arg of `sentinel monitor`) is the number of Sentinels that must agree a master is down (SDOWN → ODOWN). A failover additionally requires the vote of a **majority** of Sentinels. **Deploy at least three Sentinels in three independent boxes** — two can never break a partition safely. Query and control via `SENTINEL get-master-addr-by-name`, `SENTINEL master`, `SENTINEL failover`, `SENTINEL CKQUORUM`, and subscribe to events like `+sdown`, `+odown`, `switch-master`. Replica promotion favors a lower `replica-priority` (0 = never promote), then most replication offset, then lowest run ID. ## Redis Cluster Cluster shards data automatically across nodes and tolerates partial failures. There are **16384 hash slots**; a key's slot is `CRC16(key) mod 16384`. Each master owns a slot range; multi-key operations require all keys in the same slot — force this with hash tags (`user:{123}:profile`). Each node uses two ports: the client port (e.g. 6379) and a cluster bus port (client port + 10000, e.g. 16379, or `cluster-port`). Minimum is three masters; six nodes (three masters + three replicas) is recommended. Key `redis.conf` directives: `cluster-enabled yes`, `cluster-config-file nodes.conf`, `cluster-node-timeout`, `cluster-require-full-coverage`, `cluster-migration-barrier`. Create and operate via `redis-cli`: ``` redis-cli --cluster create 127.0.0.1:7000 ... --cluster-replicas 1 redis-cli --cluster reshard 127.0.0.1:7000 redis-cli --cluster add-node / del-node / check > CLUSTER FAILOVER # manual failover, run on a replica > CLUSTER NODES ``` Cluster does **not** guarantee strong consistency (asynchronous replication; writes to a minority-partition master are lost after node-timeout). See [[entities/redis-cli]], [[concepts/persistence]], and [[concepts/scripting-and-functions]]. --- title: "Redis JSON" type: concept tags: [json, redisjson, jsonpath, data-types] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-12.md] --- # Redis JSON The JSON capability (the RedisJSON module, part of Redis Open Source) lets you store, update, and retrieve JSON values in Redis like any other type. Documents are stored as binary data in a tree structure for fast access to sub-elements, with typed atomic operations for every JSON value type. It supports the full JSON standard and uses [JSONPath](http://goessner.net/articles/JsonPath/) syntax (the root is `$`) to select and update elements. It also integrates with Redis Search to index and query JSON documents. ## Setting and getting values `JSON.SET` writes a value at a path; `JSON.GET` reads it; `JSON.TYPE` reports the type: ``` > JSON.SET bike $ '"Hyperion"' OK > JSON.GET bike $ "[\"Hyperion\"]" > JSON.TYPE bike $ 1) "string" ``` ## Typed operations String operations with `JSON.STRLEN` and `JSON.STRAPPEND`: ``` > JSON.STRLEN bike $ 1) (integer) 8 > JSON.STRAPPEND bike $ '" (Enduro bikes)"' 1) (integer) 23 ``` Atomic numeric arithmetic with `JSON.NUMINCRBY` and `JSON.NUMMULTBY`: ``` > JSON.SET crashes $ 0 OK > JSON.NUMINCRBY crashes $ 1 "[1]" > JSON.NUMMULTBY crashes $ 24 "[24]" ``` Arrays and nested objects, addressing elements by JSONPath, and deleting with `JSON.DEL`: ``` > JSON.SET newbike $ '["Deimos", {"crashes": 0}, null]' OK > JSON.GET newbike $[1].crashes "[0]" > JSON.DEL newbike $[-1] (integer) 1 ``` Other commands include `JSON.ARRAPPEND`, `JSON.ARRINSERT`, `JSON.ARRTRIM`, `JSON.ARRPOP` (arrays); `JSON.OBJLEN`, `JSON.OBJKEYS` (objects); `JSON.MERGE`, `JSON.MGET`, `JSON.MSET`, and `JSON.TOGGLE`. ## Formatting and limits Run `redis-cli --raw` and add `INDENT`, `NEWLINE`, and `SPACE` keywords to `JSON.GET` for human-readable output: ```bash $ redis-cli --raw > JSON.GET obj INDENT "\t" NEWLINE "\n" SPACE " " $ ``` A JSON value passed to a command can have a nesting depth of up to 128; deeper objects or arrays cause the command to return an error. See [[concepts/data-types-overview]] and [[summaries/reference-catalog]]. --- title: "Keys, Expiration, and Eviction" type: concept tags: [keyspace, expiration, ttl, eviction, maxmemory] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-18.md, raw/web_community-index-html-md-21.md] --- # Keys, Expiration, and Eviction Every value in Redis is stored under a unique **key**. Keys are binary-safe (any byte sequence, the empty string included) with a maximum size of 512 MB. A common convention splits keys into sections with a colon, e.g. `user:1000:followers`. Avoid both very long keys (costly lookups) and overly short, cryptic keys. ## Querying the keyspace Type-agnostic commands work on keys of any type: ``` > EXISTS mykey # 1 if present, 0 otherwise > DEL mykey # returns count removed > TYPE mykey # e.g. "string", or "none" if missing ``` To iterate keys safely in production, use `SCAN` — it iterates incrementally, returning a few elements per call, and never blocks the server for long. Avoid `KEYS` outside debugging: it blocks until all matching keys are returned and can ruin performance on large databases. Both support glob-style patterns (`h?llo`, `h*llo`, `h[ae]llo`, `h[^e]llo`, `h[a-b]llo`; escape with `\`). ### Hashtags Wrapping part of a key in braces — a *hashtag* — makes only that part determine the hash slot. `{person}:1` and `{person}:2` map to the same slot, enabling multi-key operations in a clustered database (see [[concepts/high-availability-and-scaling]]). Use sparingly: too many keys in one slot harms performance. ## Key expiration Expiration sets a key's **time to live** (TTL); when it elapses the key is destroyed automatically. Expires can be set in seconds or milliseconds (resolution is always 1 ms), and are replicated and persisted (the absolute expire instant is saved, so time passes even while the server is stopped). ``` > EXPIRE key 5 # set TTL in seconds > TTL key # remaining seconds > SET key 100 EX 10 # SET option sets value + TTL atomically > PERSIST key # remove the expire, make key permanent ``` Millisecond variants: `PEXPIRE`, `PTTL`. ## Eviction (maxmemory) When used as a cache, Redis can evict keys once memory hits a limit. Set the limit with `maxmemory` (in `redis.conf` or at runtime): ``` maxmemory 100mb > CONFIG SET maxmemory 100mb ``` `maxmemory 0` means no limit (default on 64-bit; 32-bit implies 3GB). When a write would exceed the limit, Redis applies the policy set by `maxmemory-policy`: - `noeviction` — return an error on writes that add data (reads still work). - `allkeys-lru` / `volatile-lru` — evict least recently used (all keys / only keys with a TTL). - `allkeys-lfu` / `volatile-lfu` — least frequently used (since Redis 4.0). - `allkeys-lrm` / `volatile-lrm` — least recently modified (since Redis 8.6; updates timestamp on write only). - `allkeys-random` / `volatile-random` — random keys. - `volatile-ttl` — evict the keys with the shortest remaining TTL. `volatile-*` policies behave like `noeviction` if no key has a TTL. `allkeys-lru` is a good default (Pareto access patterns). LRU and LFU are **approximated** by sampling; tune with `maxmemory-samples 5`. LFU has `lfu-log-factor` and `lfu-decay-time`. With replication or persistence, leave RAM headroom because replication/AOF buffers are not counted toward `maxmemory`. Monitor cache health via `INFO`: `keyspace_hits`, `keyspace_misses`, `evicted_keys`, `expired_keys`. See [[syntheses/data-types-and-caching-patterns]] and [[concepts/persistence]]. --- title: "Redis Lists" type: concept tags: [lists, queues, stacks, blocking] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-3.md] --- # Redis Lists Redis lists are linked lists of string values, sorted by insertion order. They are frequently used to implement stacks and queues, and to build queue management for background worker systems. ## Adding and removing elements - `LPUSH key element...` — prepend one or more elements (creates the key); O(1) per element. - `RPUSH key element...` — append one or more elements (creates the key); O(1) per element. - `LPUSHX` / `RPUSHX` — push only when the list already exists. - `LPOP` / `RPOP` — remove and return elements from the head/tail; deletes the list when the last element is popped. - `LLEN` — length of the list, O(1). Use `LPUSH` + `RPOP` for a FIFO queue: ``` > LPUSH bikes:repairs bike:1 (integer) 1 > LPUSH bikes:repairs bike:2 (integer) 2 > RPOP bikes:repairs "bike:1" > RPOP bikes:repairs "bike:2" ``` ## Accessing and modifying - `LRANGE key start stop` — return a range of elements, O(S+N). - `LINDEX key index` — element by index, O(N). - `LINSERT key BEFORE|AFTER pivot element` — insert relative to another element, O(N). - `LSET key index element` — set an element by index, O(N). - `LREM`, `LTRIM` — remove elements / trim the list to a range. - `LMOVE` / `RPOPLPUSH` — atomically pop from one list and push to another. - `LPOS` — index of matching elements. ## Blocking operations Lists support blocking pops, making them a building block for producer/consumer systems without polling. `BLPOP` and `BRPOP` are blocking versions of `LPOP`/`RPOP`: they return only when an element is added or a user-specified timeout is reached. ``` > RPUSH bikes:repairs bike:1 bike:2 (integer) 2 > BRPOP bikes:repairs 1 1) "bikes:repairs" 2) "bike:2" > BRPOP bikes:repairs 1 (nil) (2.01s) ``` Related blocking commands: `BLMOVE`, `BRPOPLPUSH`, `BLMPOP`, `LMPOP`. ## Limits and performance The maximum length of a list is 2^32 - 1 (4,294,967,295) elements. Operations at the head or tail are O(1); commands that manipulate interior elements (`LINDEX`, `LINSERT`, `LSET`) are O(N) — use caution on large lists. Consider [[concepts/streams]] as an alternative when you need to store and process an indeterminate series of events. Related: [[concepts/data-types-overview]], [[concepts/pubsub]], [[entities/redis-cli]]. --- title: "Persistence (RDB and AOF)" type: concept tags: [persistence, rdb, aof, durability, backup] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-23.md] --- # Persistence (RDB and AOF) Persistence writes data to durable storage so it survives restarts. Redis offers four options: **RDB** (point-in-time snapshots), **AOF** (append-only log of writes), **no persistence** (caching only), or **RDB + AOF combined** in the same instance. ## RDB snapshotting RDB saves a compact, single-file point-in-time snapshot (`dump.rdb`). It is ideal for backups and disaster recovery, maximizes performance (the parent process only forks; the child does all disk I/O via copy-on-write), and allows faster restarts on big datasets. The downside: you can lose the data written since the last snapshot if Redis stops unexpectedly. Configure automatic save points — dump if at least M changes occur within N seconds — or trigger manually: ``` save 60 1000 # save if >=1000 keys changed in 60 seconds > SAVE # synchronous, blocks the server > BGSAVE # fork a child, save in background ``` ## Append-only file (AOF) AOF logs every write command (in the Redis protocol format) and replays it at startup to rebuild state — a fully durable strategy. Enable it: ``` appendonly yes ``` Durability is governed by the fsync policy: - `appendfsync always` — fsync on every write batch. Very safe, very slow. - `appendfsync everysec` — fsync once per second (default). Fast and you lose at most one second of data on disaster. - `appendfsync no` — let the OS flush (≈ every 30s on Linux). Fastest, least safe. Since Redis 7.0 AOF is **multi-part**: a base file (an RDB or AOF-format snapshot) plus incremental files, tracked by a manifest, kept in the `appenddirname` directory. ### Log rewriting The AOF grows with every write. `BGREWRITEAOF` rewrites it in the background into the shortest command sequence that reproduces the current dataset, using copy-on-write; since Redis 2.4 this is also triggered automatically. A truncated AOF (e.g. crash mid-write) loads anyway when `aof-load-truncated` is enabled; a corrupted AOF is repaired with `redis-check-aof --fix `. ## Choosing a strategy - Use **both** RDB + AOF for durability comparable to PostgreSQL. - Use **RDB alone** if a few minutes of data loss is acceptable. - AOF alone is discouraged — keep periodic RDB snapshots for backups and faster restarts. When both are enabled and Redis restarts, the **AOF is used** to reconstruct data, since it is the most complete. Redis ≥ 2.4 avoids running an AOF rewrite and a `BGSAVE` simultaneously. To switch a live RDB-only server to AOF without data loss, enable it at runtime first, then persist the config: ``` redis-cli config set appendonly yes redis-cli config set save "" # optionally disable RDB ``` then update `redis.conf` (or use `CONFIG REWRITE`) before restarting. ## Backups and disaster recovery RDB files are backup-friendly — they're immutable once produced, so copying while the server runs is safe. Cron hourly/daily snapshots and transfer at least one copy off-site daily (e.g. encrypted to S3 with `gpg -c`, or via SCP). For AOF backups, disable rewrites first (`CONFIG SET auto-aof-rewrite-percentage 0`, confirm `aof_rewrite_in_progress` is 0 in `INFO persistence`), copy the `appenddirname` directory, then re-enable. Related: [[concepts/high-availability-and-scaling]], [[concepts/keys-expiration-eviction]], [[entities/redis-cli]] (`--rdb` remote backups). --- title: "HyperLogLog and Geospatial Indexes" type: concept tags: [hyperloglog, geospatial, probabilistic, data-types] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-10.md, raw/web_community-index-html-md-11.md] --- # HyperLogLog and Geospatial Indexes Two specialized Redis data structures: HyperLogLog for approximate cardinality counting, and geospatial indexes for location queries. ## HyperLogLog HyperLogLog (HLL) is a probabilistic data structure that estimates the cardinality (number of unique members) of a set, trading perfect accuracy for space efficiency. The Redis implementation uses up to 12 KB of memory with a standard error of 0.81%, regardless of how many items are counted. HLLs are encoded as Redis strings, so `GET`/`SET` can serialize/deserialize them. Add observed elements with `PFADD`, get the estimated count with `PFCOUNT`, and combine HLLs with `PFMERGE`: ``` > PFADD bikes Hyperion Deimos Phoebe Quaoar (integer) 1 > PFCOUNT bikes (integer) 4 > PFADD commuter_bikes Salacia Mimas Quaoar (integer) 1 > PFMERGE all_bikes bikes commuter_bikes OK > PFCOUNT all_bikes (integer) 6 ``` `PFADD` and `PFCOUNT` run in O(1); `PFMERGE` is O(n) in the number of sketches. An HLL can estimate cardinalities up to 2^64 members. Typical use cases: counting unique daily visitors to a page, unique plays of a song, or unique search queries — without storing personally identifying values. ## Geospatial indexes A geospatial index stores coordinates (longitude, latitude) and lets you search for members within a radius or bounding box. It is built on a sorted set, so members are unique. Add points with `GEOADD` (note the longitude-then-latitude order): ``` > GEOADD bikes:rentable -122.27652 37.805186 station:1 (integer) 1 > GEOADD bikes:rentable -122.2674626 37.8062344 station:2 (integer) 1 ``` Search with `GEOSEARCH` (since 6.2.0). Specify the center with `FROMLONLAT` (or `FROMMEMBER`) and the area with `BYRADIUS` or `BYBOX`. `WITHDIST` returns the distance to each match: ``` > GEOSEARCH bikes:rentable FROMLONLAT -122.2612767 37.7936847 BYRADIUS 5 km WITHDIST 1) 1) "station:1" 2) "1.8523" 2) 1) "station:2" 2) "1.4979" ``` Related commands include `GEODIST` (distance between two members), `GEOPOS` (coordinates of members), `GEOHASH`, and `GEOSEARCHSTORE` (store results in a key). This data type is intended for simpler use cases; for richer geo querying see Redis Search. See [[concepts/data-types-overview]], [[concepts/sets-and-sorted-sets]], and [[summaries/reference-catalog]]. --- title: "Redis Pub/Sub" type: concept tags: [pubsub, messaging, channels, sharded] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-13.md] --- # Redis Pub/Sub Pub/Sub implements the publish/subscribe messaging paradigm: publishers send messages to named channels without knowing who (if anyone) is subscribed, and subscribers express interest in channels and receive only matching messages. This decoupling of senders and receivers allows greater scalability and a more dynamic topology. Subscribers receive messages in the order they are published. ## Subscribing and publishing Subscribe to one or more channels with `SUBSCRIBE`, and send messages with `PUBLISH`: ```bash SUBSCRIBE channel11 ch:00 ``` ``` > PUBLISH second Hello ``` A client subscribed to channels shouldn't issue regular commands (under RESP2 only a few are allowed, such as `PING`, `SUBSCRIBE`, `UNSUBSCRIBE`, `PSUBSCRIBE`, `PUNSUBSCRIBE`, `QUIT`, `RESET`); under RESP3 (see `HELLO`) any command is permitted. Note that in `redis-cli`'s subscribed mode you cannot run `UNSUBSCRIBE`/`PUNSUBSCRIBE` — only `Ctrl-C` exits. ## Message format Each pushed message is a three-element array. The first element is the kind: - `subscribe` / `unsubscribe` — acknowledges (un)subscription; the third element is the current subscription count. The client leaves Pub/Sub state when this count reaches zero. - `message` — a payload from a `PUBLISH`; the second element is the channel, the third is the payload. ## Pattern-matching subscriptions Subscribe to glob-style patterns with `PSUBSCRIBE` (and leave them with `PUNSUBSCRIBE`): ``` PSUBSCRIBE news.* ``` This receives messages sent to `news.art.figurative`, `news.music.jazz`, etc. Pattern-matched deliveries use the `pmessage` type, whose elements are the matched pattern, the originating channel, and the payload. A client subscribed to both a channel and a matching pattern receives the message twice (once as `message`, once as `pmessage`). ## Delivery semantics Pub/Sub exhibits at-most-once delivery: once the server sends a message there is no retransmission, and if a subscriber is disconnected the message is lost. For stronger guarantees and persistence, use [[concepts/streams]], which support both at-most-once and at-least-once delivery. Pub/Sub has no relation to the keyspace and ignores database numbers — publishing on db 10 is heard by a subscriber on db 1. For scoping, prefix channel names (e.g. by environment). ## Sharded Pub/Sub Since Redis 7.0, sharded Pub/Sub assigns shard channels to slots by the same algorithm used for keys, using `SSUBSCRIBE`, `SUNSUBSCRIBE`, and `SPUBLISH`. It restricts message propagation to within a cluster shard, limiting cluster-bus traffic and letting Pub/Sub scale horizontally by adding shards. Related: [[concepts/streams]], [[concepts/high-availability-and-scaling]], [[entities/redis-cli]]. --- title: "Scripting and Functions" type: concept tags: [programmability, lua, functions, eval] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-15.md, raw/web_community-index-html-md-16.md, raw/web_community-index-html-md-17.md] --- # Scripting and Functions Redis programmability lets you execute custom server-side logic for *data locality* — processing data where it lives, reducing network round trips. Redis ships a single embedded scripting engine, the **Lua 5.1** interpreter, run inside a sandbox. There are two ways to run scripts: ephemeral **Eval scripts** (since Redis 2.6) and **Redis Functions** (since Redis 7.0, which supersedes `EVAL` for complex use cases). Script and function execution is **atomic**: it blocks all other server activity for its entire runtime, with the same semantics as [[concepts/transactions]]. Because of this, scripts should be fast — slow scripts block every connected client. ## Eval scripts `EVAL` runs an ad-hoc Lua script. Its second argument is the number of key-name arguments that follow; those populate the `KEYS` table, the rest populate `ARGV`. ``` > EVAL "return 'Hello, scripting!'" 0 "Hello, scripting!" > EVAL "return { KEYS[1], KEYS[2], ARGV[1] }" 2 key1 key2 arg1 ``` **Important:** every key a script accesses must be passed explicitly as a key argument — never access programmatically-generated key names. Inside scripts, call Redis commands with `redis.call()` (errors propagate to the client) or `redis.pcall()` (errors returned to the script). ``` > EVAL "return redis.call('SET', KEYS[1], ARGV[1])" 1 foo bar OK ``` ## Script cache and SCRIPT Sending the source on every `EVAL` wastes bandwidth, so Redis caches scripts by their SHA1 digest. Load once with `SCRIPT LOAD`, then run by digest with `EVALSHA`: ``` > SCRIPT LOAD "return 'cached'" "c664a3bf70bd1d45c4284ffebb65a6f2299bfc9f" > EVALSHA c664a3bf70bd1d45c4284ffebb65a6f2299bfc9f 0 ``` If the digest is missing the server returns `NOSCRIPT`, and the app must reload. The cache is **volatile** — cleared on restart, failover, or `SCRIPT FLUSH`. Other subcommands: `SCRIPT EXISTS`, `SCRIPT KILL` (interrupts a long-running read-only script), `SCRIPT DEBUG`. ## Redis Functions Functions are first-class database elements: named, persisted to AOF, and replicated, so applications no longer need to reload them. Every function belongs to a **library** loaded with `FUNCTION LOAD`; the payload starts with a shebang declaring engine and library name. Call functions with `FCALL`: ```lua #!lua name=mylib redis.register_function('knockknock', function() return 'Who\'s there?' end) ``` ``` redis> FCALL knockknock 0 "Who's there?" ``` Reload a library with the `REPLACE` modifier; manage with `FUNCTION LIST` and `FUNCTION DELETE`. In a Redis Cluster, functions must be loaded to all master nodes manually. ## Read-only scripts and flags A read-only script executes no writes and can run on replicas, never fails with OOM, and can always be killed. Use the dedicated `EVAL_RO`/`EVALSHA_RO`/`FCALL_RO` variants, or declare the `no-writes` flag. Since Redis 7.0, Eval scripts can declare flags via a shebang line (e.g. `#!lua flags=no-writes,allow-stale`); functions declare flags at registration. Scripts have a max execution time controlled by `busy-reply-threshold` (default 5s); past it, Redis replies `BUSY` and only `SCRIPT KILL`, `FUNCTION KILL`, or `SHUTDOWN NOSAVE` are accepted. Since Redis 5.0 replication is effects-based (only the resulting write commands are replicated); verbatim script replication was removed in 7.0. See also [[entities/redis-cli]] (`--eval`, `--functions-rdb`) and [[concepts/high-availability-and-scaling]]. --- title: "Redis Sets and Sorted Sets" type: concept tags: [sets, sorted-sets, leaderboards, set-operations] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-4.md, raw/web_community-index-html-md-6.md] --- # Redis Sets and Sorted Sets ## Sets A Redis set is an unordered collection of unique strings (members). Sets efficiently track unique items, represent relations, and perform set operations. Adding, removing, and membership tests are O(1). - `SADD key member...` — add members (creates the key); duplicates are silently ignored. - `SREM`, `SCARD` (count, O(1)), `SISMEMBER` / `SMISMEMBER` (membership, O(1)), `SMEMBERS` (all members, O(N)), `SSCAN` (iterative), `SPOP`, `SRANDMEMBER`, `SMOVE`. - Combinations: `SINTER`, `SUNION`, `SDIFF` (and `SINTERCARD` plus the `...STORE` variants that save results to a key). ``` > SADD bikes:racing:france bike:1 bike:2 bike:3 (integer) 3 > SADD bikes:racing:usa bike:1 bike:4 (integer) 2 > SINTER bikes:racing:france bikes:racing:usa 1) "bike:1" > SDIFF bikes:racing:france bikes:racing:usa 1) "bike:3" 2) "bike:2" ``` Max set size is 2^32 - 1 (4,294,967,295) members. `SMEMBERS` is O(n) and returns the whole set — prefer `SSCAN` for large sets. For approximate membership with low memory, consider a Bloom or Cuckoo filter (see [[concepts/probabilistic-and-geospatial]]). ## Sorted sets A sorted set is a collection of unique strings (members) ordered by an associated floating-point score; members with equal scores are ordered lexicographically. Use cases include leaderboards and sliding-window rate limiters. - `ZADD key score member [score member ...]` — add members or update scores (creates the key); O(log(N)) per item. - `ZSCORE` / `ZMSCORE`, `ZINCRBY`, `ZCARD`, `ZRANK` / `ZREVRANK`, `ZCOUNT`, `ZREM`. ``` > ZADD racer_scores 10 "Norem" (integer) 1 > ZADD racer_scores 8 "Sam-Bodden" 10 "Royce" 6 "Ford" 14 "Prickett" (integer) 4 ``` ### Operating on ranges - `ZRANGE key start stop` — members by index range (with `BYSCORE`, `BYLEX`, `REV`, `LIMIT` options); O(log(N)+M). - `ZRANGEBYSCORE key min max` — members within a score range (`-inf`/`+inf` supported). - `ZRANGEBYLEX` / `ZLEXCOUNT` — lexicographical range queries. - `ZREVRANGE`, `ZRANGEBYSCORE`, `ZPOPMIN` / `ZPOPMAX` (and blocking `BZPOPMIN`/`BZPOPMAX`/`BZMPOP`), `ZREMRANGEBY*`. ``` > ZRANGEBYSCORE racer_scores -inf 10 1) "Ford" 2) "Sam-Bodden" 3) "Norem" 4) "Royce" ``` Most sorted set operations are O(log(n)); `ZRANGE` is O(log(n)+m) — use caution with large return values. Related: [[concepts/data-types-overview]], [[concepts/probabilistic-and-geospatial]], [[concepts/json]], [[entities/redis-cli]]. --- title: "Redis Streams" type: concept tags: [streams, consumer-groups, messaging, data-types] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-7.md] --- # Redis Streams A Redis stream is an append-only log data structure. Each entry consists of one or more field-value pairs (like a small hash) and is identified by an auto-generated, monotonically increasing ID of the form `-`. Adding an entry is O(1); streams are implemented as radix trees, giving efficient inserts and reads. Unlike [[concepts/pubsub]], stream entries are persisted and support both at-most-once and at-least-once delivery. ## Adding and reading entries Append with `XADD`, using `*` to auto-generate the ID: ``` > XADD race:france * rider Castilla speed 30.2 position 1 location_id 1 "1692632086370-0" > XLEN race:france (integer) 3 ``` Query a range by ID with `XRANGE` (use `-` and `+` for min/max, `COUNT` to limit); `XREVRANGE` reverses the order. An exclusive start is written with a leading `(`: ``` > XRANGE race:france - + COUNT 2 > XRANGE race:france (1692632094485-0 + COUNT 2 ``` Read new entries with `XREAD`. `STREAMS` lists keys then start IDs; `$` means "only entries arriving after this call", and `BLOCK ` waits (0 = forever): ``` > XREAD COUNT 2 STREAMS race:france 0 > XREAD BLOCK 0 STREAMS race:france $ ``` ## Consumer groups Consumer groups let multiple consumers cooperatively split a stream, with each entry delivered to exactly one consumer in the group. Create a group with `XGROUP CREATE`; add `MKSTREAM` to create the stream if absent: ``` > XGROUP CREATE race:france france_riders $ OK > XGROUP CREATE race:italy italy_riders $ MKSTREAM ``` Consumers fetch undelivered entries with `XREADGROUP` using the special ID `>`. Each delivered entry enters the group's Pending Entries List (PEL) until acknowledged with `XACK`: ``` > XREADGROUP GROUP italy_riders Alice COUNT 1 STREAMS race:italy > > XACK race:italy italy_riders 1692632639151-0 ``` Passing an explicit ID (e.g. `0`) instead of `>` re-reads that consumer's still-pending entries. Inspect pending work with `XPENDING`, and reassign stalled entries to another consumer with `XCLAIM` or `XAUTOCLAIM`: ``` > XPENDING race:italy italy_riders > XAUTOCLAIM race:italy italy_riders Alice 60000 0-0 COUNT 1 ``` ## Observability and capping `XINFO STREAM `, `XINFO GROUPS `, and `XINFO CONSUMERS ` report structure and group/consumer state. Cap stream length with `MAXLEN` on `XADD`; `~` makes trimming approximate (more efficient): ``` > XADD race:italy MAXLEN 2 * rider Henshaw > XADD race:italy MAXLEN ~ 1000 * field value ``` Use streams when you need persistence and replay; for fire-and-forget fan-out use [[concepts/pubsub]]. Related: [[concepts/data-types-overview]], [[summaries/reference-catalog]]. --- title: "Redis Strings, Bitmaps, and Bitfields" type: concept tags: [strings, bitmaps, bitfields, counters] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-2.md, raw/web_community-index-html-md-8.md, raw/web_community-index-html-md-9.md] --- # Redis Strings, Bitmaps, and Bitfields Redis strings store sequences of bytes — text, serialized objects, or binary arrays (e.g. a JPEG). They are the simplest value type. A single string can be a maximum of **512 MB**. ## Core string commands `SET` assigns a value (replacing any existing value, even of another type); `GET` retrieves it. Both are O(1). ``` > SET bike:1 Deimos OK > GET bike:1 "Deimos" ``` `SET` options give atomic compare-and-set behavior: `NX` succeeds only if the key does not exist, `XX` only if it already exists. ``` > SET bike:1 bike NX (nil) > SET bike:1 bike XX OK ``` Other commands: `GETSET` (set new value, return old), `MSET`/`MGET` (atomic multi-key set/get, O(N), reduce round trips), `APPEND`, `STRLEN`, `GETDEL`, `GETEX`, `SETEX`/`PSETEX` (set with expiry), `SETNX`. Random-access commands `SUBSTR`, `GETRANGE`, and `SETRANGE` are O(N) — use caution on large strings. Most other string operations are O(1). ## Strings as counters `INCR` parses the value as an integer, increments by one, and stores the result; it is atomic, so concurrent clients never race. Related: `INCRBY`, `DECR`, `DECRBY`, `INCRBYFLOAT`. ``` > SET total_crashes 0 > INCR total_crashes (integer) 1 > INCRBY total_crashes 10 (integer) 11 ``` ## Bitmaps Bitmaps are not a distinct type but a set of bit-oriented operations on a string treated as a bit vector (up to 2^32 bits). Use cases include compact set representations and object permissions. - `SETBIT key offset value` — set/clear the bit at an offset (auto-grows the string), O(1). - `GETBIT key offset` — return a bit's value (out-of-range bits read as 0), O(1). - `BITCOUNT` — population count of set bits, O(N). - `BITPOS` — find the first 0 or 1 bit, O(N). - `BITOP destkey op key...` — bitwise op storing the result; operators are `AND`, `OR`, `XOR`, `NOT`, `DIFF`, `DIFF1`, `ANDOR`, and `ONE`, O(N). ``` > SETBIT pings:2024-01-01-00:00 123 1 (integer) 0 > GETBIT pings:2024-01-01-00:00 123 1 > BITCOUNT pings:2024-01-01-00:00 (integer) 1 ``` ## Bitfields Bitfields let you set, increment, and get integer values of arbitrary bit length (unsigned 1-bit to signed 63-bit), stored in a binary-encoded string, with atomic read/write/increment and overflow policies. `BITFIELD` is O(n) in the number of counters accessed; `BITFIELD_RO` is the read-only variant. ``` > BITFIELD bike:1:stats SET u32 #0 1000 1) (integer) 0 > BITFIELD bike:1:stats INCRBY u32 #0 -50 INCRBY u32 #1 1 1) (integer) 950 2) (integer) 1 > BITFIELD bike:1:stats GET u32 #0 GET u32 #1 1) (integer) 950 2) (integer) 1 ``` If storing structured data as a serialized string, consider [[concepts/hashes]] or [[concepts/json]] instead. For expiry, see [[concepts/keys-expiration-eviction]]. Related: [[concepts/data-types-overview]], [[entities/redis-cli]]. --- title: "Redis Transactions" type: concept tags: [transactions, multi-exec, watch, optimistic-locking] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-14.md] --- # Redis Transactions Redis transactions let you execute a group of commands as a single step, centered on `MULTI`, `EXEC`, `DISCARD`, and `WATCH`. They make two guarantees: all commands in a transaction are serialized and executed sequentially (no other client's request is served in the middle), and `EXEC` triggers execution of all queued commands as a single isolated operation. If the connection is lost before `EXEC`, none of the operations run. ## MULTI / EXEC `MULTI` starts a transaction (replies `OK`). Subsequent commands are not executed but queued, each replying `QUEUED`. `EXEC` runs them all and returns an array of replies in order: ``` > MULTI OK > INCR foo QUEUED > INCR bar QUEUED > EXEC 1) (integer) 1 2) (integer) 1 ``` `DISCARD` aborts the transaction, flushing the queue and restoring the connection to normal: ``` > MULTI OK > INCR foo QUEUED > DISCARD OK ``` ## Errors and rollbacks There are two kinds of error. A command that fails to be queued (e.g. a syntax error) is detected before `EXEC`; since Redis 2.6.5 the server then refuses to run the transaction and `EXEC` returns an error, discarding it. A command that fails at execution time (e.g. a wrong-type operation) does not stop the others — Redis processes the rest of the queue regardless. Redis does **not** support rollbacks, because supporting them would harm its simplicity and performance. ## Optimistic locking with WATCH `WATCH` adds check-and-set (CAS) behavior. Watched keys are monitored; if any watched key is modified before `EXEC`, the whole transaction aborts and `EXEC` returns a Null reply. This lets you build safe read-modify-write sequences: ``` WATCH mykey val = GET mykey val = val + 1 MULTI SET mykey $val EXEC ``` If another client changed `mykey` between `WATCH` and `EXEC`, the transaction fails and you retry. Modifications by Redis itself (expiration or eviction) also trip the watch. All keys are unwatched when `EXEC` is called (whether or not it aborted) and when the connection closes; `UNWATCH` flushes watched keys manually. From Redis 8.4, single-command CAS is also available via `SET` with `IFEQ`/`IFNE`/`IFDEQ`/`IFDNE` and compare-and-delete via `DELEX`. ## Scripting alternative Anything you can do with a transaction you can also do with a [[concepts/scripting-and-functions]] script, which is also transactional and is usually both simpler and faster. Related: [[concepts/strings]], [[concepts/keys-expiration-eviction]], [[summaries/reference-catalog]]. --- title: "redis-cli" type: entity tags: [cli, tools, redis-cli, debugging] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-22.md] --- # redis-cli `redis-cli` is the Redis command-line interface, installed with every Redis Open Source distribution (or built with `make redis-cli`, producing `src/redis-cli`). It runs one-off commands, an interactive REPL, and several special diagnostic modes. ## Command-line usage Run a command directly by passing it as arguments: ``` $ redis-cli INCR mycounter (integer) 7 ``` Replies are typed; the type hint (e.g. `(integer)`) is shown only when stdout is a tty. When piped or redirected, *raw output mode* auto-enables. Force it with `--raw`, or force human output with `--no-raw`. ## Connection options ``` $ redis-cli -h -p PING $ redis-cli -a PING # or REDISCLI_AUTH env var (safer) $ redis-cli -n INCR a # select a database $ redis-cli -u redis://user:password@host:port/dbnum PING ``` Defaults are host `127.0.0.1`, port `6379`. Use `--user`/`--pass` for ACL-style auth, `-t ` for connection timeout, `-4`/`-6` for DNS preference. For TLS use `--tls` with `--cacert`/`--cacertdir`, and `--cert`/`--key` for client certificates (scheme `rediss` in URIs). ## Interactive mode Launch with no command. The prompt shows host:port (and `[db]` when not DB 0). Features: line editing (linenoise), persistent history (`.rediscli_history`), TAB completion, syntax hints (`:set hints` / `:set nohints`), reverse search (Ctrl-R), and — since Redis 8.8 — word-jump navigation (`Alt/Ctrl + ←/→`). `CONNECT host port` switches servers; `HELP @` / `HELP ` shows online help; `CLEAR` clears the screen. Prefix a command with a number to repeat it (`5 INCR mycounter`). ## Input, scripting, and output ``` $ redis-cli -x SET net_services < /etc/services # value from stdin $ cat commands.txt | redis-cli # batch of commands $ redis-cli --csv LRANGE mylist 0 -1 # CSV output $ redis-cli --eval /tmp/script.lua key1 , arg1 # run a Lua script ``` With `--eval`, keys and args are separated by a comma (no key count needed). Also `--json`/`--quoted-json`, and `-3`/`-2` to choose RESP3/RESP2 (see [[summaries/reference-catalog]]). ## Repetition and special modes - `-r -i ` — run a command N times (`-1` = forever) with a delay. - `--stat` — continuous rolling server stats (keys, mem, clients, requests). - `--bigkeys` / `--memkeys` / `--keystats` — scan the keyspace (via `SCAN`) for large keys, memory usage, and distribution; work on cluster replicas. - `--scan [--pattern ] [--count ]` — list keys non-blockingly. - `--latency` / `--latency-history` / `--latency-dist` — measure server latency; `--intrinsic-latency ` measures the local kernel scheduler latency (run on the server host). - `--rdb ` — transfer a remote RDB backup locally; `--functions-rdb ` extracts only functions. - `--replica` — print the replication stream a primary sends. - `--lru-test ` — simulate an 80-20 cache workload (needs a `maxmemory` cap). - `--pipe` — mass-insert raw protocol from stdin; `--cluster ` — cluster manager. Pub/Sub and `MONITOR` modes activate automatically when you issue `SUBSCRIBE`/`PSUBSCRIBE` or `MONITOR`. Related: [[concepts/scripting-and-functions]], [[concepts/high-availability-and-scaling]], [[concepts/persistence]]. --- title: "Activity Log" type: log --- # Activity Log Append-only record of all wiki changes. ## Format Each entry follows this format: ``` ### YYYY-MM-DD HH:MM — [Action Type] - **Source/Trigger**: what initiated the action - **Pages created**: list of new pages - **Pages updated**: list of updated pages - **Notes**: any contradictions flagged, decisions made ``` --- ### 2026-04-08 00:00 — Setup - **Source/Trigger**: Repository initialized - **Pages created**: index.md, log.md, dashboard.md, analytics.md, flashcards.md - **Pages updated**: none - **Notes**: Empty knowledge base ready for first source ingestion --- ### 2026-06-25 — Initial curation (factory build) - **Source/Trigger**: `new_wiki.py init redis` — 38 sources (28 core OSS doc pages via redis.io per-page markdown + 10 release notes to Redis 8.8) - **Pages created**: 17 — 14 concepts (data-types-overview, strings, lists, hashes, sets-and-sorted-sets, streams, probabilistic-and-geospatial, json, keys-expiration-eviction, pubsub, transactions, scripting-and-functions, persistence, high-availability-and-scaling), 1 entity (redis-cli), 1 summary (reference-catalog), 1 synthesis (data-types-and-caching-patterns) - **Pages updated**: index.md, log.md - **Notes**: First gather via llms_txt was discarded — redis.io's llms.txt is dominated by Dev Hub tutorials AND stale (listed old `/develop/interact/*` paths that 404). Re-gathered a curated core-doc list via the `/index.html.md` endpoint; current paths confirmed (programmability → `/develop/programmability/`, pub/sub → `/develop/pubsub/`, bitmaps/bitfields under `/develop/data-types/strings/`). Filenames are opaque (`index-html-md-N`); curators mapped them by `source_url`. Scope = OSS core; Cloud/Enterprise and modules beyond JSON excluded. --- title: "Reference Catalog" type: summary tags: [reference, resp, protocol, config, acl, catalog] updated: 2026-06-25 confidence: high sources: [raw/web_community-index-html-md-19.md, raw/web_community-index-html-md-20.md, raw/web_community-index-html-md-24.md, raw/web_community-index-html-md-28.md, raw/github_release-8-8-0.md] --- # Reference Catalog A map of the core Redis reference material: the wire protocol (RESP), configuration, the ACL security model, and client-side caching. This page points to the right area rather than reproducing the full command reference. Versions noted reference Redis 8.x (8.8 is the current GA release; configuration files exist per minor version 8.0–8.8). ## RESP (serialization protocol) Clients talk to Redis over **RESP**, the Redis Serialization Protocol, on a TCP connection (default port 6379). A client sends a command as an array of bulk strings; the server replies with a type-specific RESP value. The first byte identifies the type. **RESP2** (since Redis 2.0) is the default; **RESP3** (Redis 6+, opt-in via `HELLO `) adds richer types. | Type | Version | First byte | | --- | --- | --- | | Simple string | RESP2 | `+` | | Simple error | RESP2 | `-` | | Integer | RESP2 | `:` | | Bulk string | RESP2 | `$` | | Array | RESP2 | `*` | | Null | RESP3 | `_` | | Boolean | RESP3 | `#` | | Double | RESP3 | `,` | | Big number | RESP3 | `(` | | Bulk error | RESP3 | `!` | | Verbatim string | RESP3 | `=` | | Map | RESP3 | `%` | | Attribute | RESP3 | `\|` | | Set | RESP3 | `~` | | Push | RESP3 | `>` | `\r\n` (CRLF) terminates every part. RESP2 represents null as `$-1\r\n` (null bulk string) or `*-1\r\n` (null array). Bulk strings are limited to 512 MB (`proto-max-bulk-len`). RESP uses prefixed lengths, so parsing needs no escaping. Pipelining lets a client send many commands before reading replies. New connections should begin with `HELLO`; an unsupported version returns `-NOPROTO`. ## Configuration Configure via `redis.conf` (directives are `keyword arg1 arg2 ...`). Since Redis 8, Redis Open Source ships two files: `redis.conf` (server only) and `redis-full.conf` (server + Search, Time Series, probabilistic data structures, JSON; its first line is `include redis.conf`). Pass directives on the command line by prefixing `--` (e.g. `./redis-server --port 6380 --replicaof 127.0.0.1 6379`). Change settings at runtime with `CONFIG SET` / `CONFIG GET` (not all directives are dynamic); persist runtime changes back to disk with `CONFIG REWRITE`. Key directive families covered on other pages: `maxmemory` / `maxmemory-policy` ([[concepts/keys-expiration-eviction]]); `save`, `appendonly`, `appendfsync` ([[concepts/persistence]]); `replicaof`, `cluster-*`, Sentinel directives ([[concepts/high-availability-and-scaling]]). ## ACL (security) The Access Control List restricts which commands and keys a connection may use. A connection authenticates with `AUTH ` (the legacy `AUTH ` targets the `default` user). A fresh instance has one user: `user default on nopass ~* &* +@all`. Manage users with `ACL SETUSER`, inspect with `ACL LIST` / `ACL GETUSER`, list categories with `ACL CAT`, generate secrets with `ACL GENPASS`. ``` > ACL SETUSER alice on >p1pp0 ~cached:* +get > ACL SETUSER antirez on +@all -@dangerous ~* >secret ``` Rule syntax: `on`/`off`; `+cmd`/`-cmd` and `+cmd|subcmd`; `+@category`/`-@category` (e.g. `@admin`, `@read`, `@write`, `@dangerous`, 29 categories total); key patterns `~pattern` (plus read/write `%R~`, `%W~` since 7.0); Pub/Sub channels `&pattern`; passwords `>pass`, ``, `nopass`; **selectors** `(...)` (7.0+). `ACL SETUSER` is incremental — it never resets an existing user. Passwords are stored SHA-256 hashed. Users can live in `redis.conf` or an external `aclfile` (loaded with `ACL LOAD`, saved with `ACL SAVE`). Since Redis 8.8, `ACL LOAD` allows `#` comment lines. ## Client-side caching Server-assisted caching ("Tracking") since Redis 6 lets clients cache values locally and receive **invalidation messages** when keys change. Enable with `CLIENT TRACKING ON` (default mode remembers accessed keys server-side; `BCAST` mode broadcasts by key prefix with no server memory cost). Options: `REDIRECT ` (send invalidations to a second connection, required for RESP2 via the `__redis__:invalidate` Pub/Sub channel), `OPTIN`/`OPTOUT` (with `CLIENT CACHING YES`/`CLIENT UNTRACKING`), `NOLOOP`. See [[syntheses/data-types-and-caching-patterns]]. Cross-references: [[concepts/data-types-overview]], [[entities/redis-cli]], [[concepts/scripting-and-functions]], [[concepts/transactions]], [[concepts/pubsub]]. --- title: "Data Types and Caching Patterns" type: synthesis tags: [data-types, caching, eviction, client-side-caching, patterns] updated: 2026-06-25 confidence: medium sources: [raw/web_community-index-html-md.md, raw/web_community-index-html-md-21.md, raw/web_community-index-html-md-20.md] --- # Data Types and Caching Patterns Redis is a data structure server: the right cache design starts with choosing the data type that fits the access pattern, then layering eviction and invalidation strategies on top. This page synthesizes those three concerns; the underlying facts live in [[concepts/data-types-overview]], [[concepts/keys-expiration-eviction]], and [[summaries/reference-catalog]]. ## Pick the data type for the workload Redis Open Source implements [[concepts/strings]] (with [[concepts/data-types-overview|bitmaps and bitfields]]), [[concepts/lists]], [[concepts/hashes]], [[concepts/sets-and-sorted-sets|sets and sorted sets]], [[concepts/streams]], [[concepts/json]], [[concepts/probabilistic-and-geospatial|probabilistic and geospatial]] types, plus time series and vector sets. A few caching-relevant mappings: - **Strings** — the most basic cache value (a serialized object, a rendered fragment, a counter). Bitmaps/bitfields pack many flags or counters into one value. - **Hashes** — model a record as field-value pairs, so you can update one field without rewriting the whole object — well suited to caching structured entities like `user:1234`. - **Sets / Sorted sets** — membership tests in O(1), or ranked/leaderboard caches via scores. - **Probabilistic types** (Bloom/Cuckoo filters, HyperLogLog, Count-min sketch, Top-K) — approximate but highly memory-efficient when exact answers are not required (cardinality, frequency, presence). Caching everything as opaque strings works, but choosing a richer type usually means smaller memory footprint and cheaper partial updates. ## Eviction as a cache discipline When Redis is a cache, cap memory with `maxmemory` and let it evict rather than error. Match `maxmemory-policy` to the access pattern (see [[concepts/keys-expiration-eviction]]): ``` maxmemory 2mb maxmemory-policy allkeys-lru ``` - `allkeys-lru` is the pragmatic default — most caches follow a Pareto (hot-subset) distribution. - `allkeys-lfu` favors keys accessed *frequently* over merely recently. - `allkeys-lrm` (Redis 8.6+) keeps frequently-read data but evicts what hasn't been *modified*, useful for read-heavy workloads. - `volatile-ttl` suits apps that assign short TTLs to good eviction candidates. Note that setting an explicit `EXPIRE` costs memory, so `allkeys-lru` can be more memory-efficient than relying on TTLs alone. With this configuration Redis behaves much like memcached, evicting under pressure without the app managing TTLs. Watch `keyspace_hits`/`keyspace_misses` and `evicted_keys` from `INFO` to tune. ## Invalidation: TTLs vs. server-assisted tracking The hard part of caching is invalidation. Two complementary patterns: 1. **TTL-based** — accept a maximum staleness by setting a TTL on cached entries. Simple and bandwidth-free, but can serve stale data within the window. 2. **Client-side caching (Tracking)** — Redis 6+ lets application servers cache values in their own memory and receive **invalidation messages** when keys change (see [[summaries/reference-catalog]]). Default mode tracks per-client accessed keys server-side; `BCAST` mode broadcasts by key prefix (`object:`, `user:`) with zero server memory cost but more messages. Tracking trades the classic Pub/Sub invalidation fan-out for targeted messages. Practical guidance from the reference: put a max TTL on every locally cached key even when using tracking (guards against lost-connection staleness), flush the local cache if the invalidation connection drops, and beware the race where an invalidation arrives before a slow `GET` reply — populate a "caching-in-progress" placeholder to avoid storing stale values. Cache keys that are requested often and change at a reasonable rate; avoid caching rapidly-changing or rarely-read keys. ## Putting it together A typical read-through cache: store entities as hashes or serialized strings keyed `type:id`, cap memory with `maxmemory` + `allkeys-lru`, give entries a max TTL, and enable client-side tracking (`OPTIN` with `REDIRECT`) so application servers hold hot values locally and drop them on invalidation. For approximate analytics layered on the same instance, reach for probabilistic types instead of materializing full sets. > Confidence is *medium*: data-type-to-pattern mappings and the combined design are an editorial synthesis; the per-type facts, eviction policies, and tracking protocol are each sourced directly.