# Docker — full corpus # LLM Wiki An open-source template for building LLM-powered knowledge bases, following [Andrej Karpathy's "LLM Wiki" pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f). You provide raw sources. The LLM reads them, writes structured wiki pages, cross-links everything, and maintains it over time. You never edit the wiki directly — you curate sources and ask questions. ## How It Works The system has three layers: ``` raw/ Sources you collect (articles, transcripts, notes, PDFs) wiki/ LLM-written & maintained pages (summaries, concepts, entities, syntheses) CLAUDE.md Schema that tells the LLM how to structure everything ``` Three operations drive the workflow: | Operation | Trigger | What happens | |-----------|---------|--------------| | **Ingest** | "ingest raw/my-source.txt" | LLM reads the source, creates a summary page, creates/updates concept and entity pages, adds cross-links, updates the index and log | | **Query** | Ask any question | LLM searches the wiki, synthesizes an answer with citations, optionally creates a synthesis page for novel insights | | **Lint** | "lint" or "health check" | LLM audits all pages for orphans, contradictions, missing links, incomplete sections, and low-confidence claims — fixes what it can, reports the rest | ## Quick Start 1. **Clone this repo** ```bash git clone https://github.com/YOUR_USERNAME/llm-wiki.git my-knowledge-base cd my-knowledge-base ``` 2. **Customize CLAUDE.md** for your domain - Update the Purpose section with your topic - Replace the placeholder tagging taxonomy with your own categories - Adjust confidence level descriptions if needed - Everything else (workflows, page formats, linking rules) works as-is 3. **Drop sources into `raw/`** - Text files, transcripts, articles, notes — any plain text - These are immutable once added; the LLM never modifies them 4. **Tell the LLM to ingest** ``` ingest raw/my-first-source.txt ``` The LLM will create summary pages, concept pages, entity pages, cross-links, and update the index. 5. **Ask questions** ``` What are the key differences between X and Y? ``` The LLM answers from the wiki, citing specific pages. 6. **Run health checks** ``` lint ``` The LLM audits the wiki and fixes issues. ## Directory Structure ``` . ├── CLAUDE.md # Schema — the LLM's instructions ├── raw/ # Your source documents (immutable) └── wiki/ ├── index.md # Master catalog of all pages ├── log.md # Append-only activity log ├── dashboard.md # Dataview dashboard (Obsidian) ├── analytics.md # Charts View analytics (Obsidian) ├── flashcards.md # Spaced repetition cards ├── summaries/ # One page per source document ├── concepts/ # Concept and framework pages ├── entities/ # People, tools, organizations, etc. ├── syntheses/ # Cross-cutting analyses and comparisons ├── journal/ # Research/session journal entries │ └── template.md # Journal entry template └── presentations/ # Marp slide decks ``` ## Enhancements This template includes several extras beyond the core wiki pattern: ### Dataview Dashboard (`wiki/dashboard.md`) Live queries that surface low-confidence pages, recent updates, concepts by tag, and pages with the most sources. Requires the [Dataview](https://github.com/blacksmithgu/obsidian-dataview) Obsidian plugin. ### Charts View Analytics (`wiki/analytics.md`) Visual analytics with pie charts, bar charts, and word clouds. Requires the [Charts View](https://github.com/caronchen/obsidian-chartsview-plugin) Obsidian plugin. ### Mermaid Diagrams Use Mermaid code blocks in any wiki page to create flowcharts, sequence diagrams, or concept maps. Native support in Obsidian and GitHub. ### Marp Slides (`wiki/presentations/`) Create slide decks from markdown using [Marp](https://marp.app/). Drop presentation files in this directory. ### Research Journal (`wiki/journal/`) Track your research sessions, experiments, or applied work with the included template. The LLM can reference journal entries when answering queries. ### Spaced Repetition (`wiki/flashcards.md`) Flashcards in the format used by the [Spaced Repetition](https://github.com/st3v3nmw/obsidian-spaced-repetition) Obsidian plugin. Ask the LLM to generate flashcards from any wiki page. ### MCP Server This repo works with Claude Code's MCP server capabilities. Point an MCP-compatible client at this repo and the LLM can read/write the wiki programmatically. ## Customizing for Your Domain The schema in `CLAUDE.md` is domain-agnostic. To adapt it: 1. **Purpose** — Describe your knowledge domain in one paragraph 2. **Tagging taxonomy** — Replace placeholder categories with your own (e.g., for a cooking KB: `cuisine`, `technique`, `ingredient`, `equipment`) 3. **Confidence levels** — Adjust the descriptions to match your domain's evidence standards 4. **Entity types** — Update the entity page description to match what entities mean in your domain (people, tools, companies, etc.) 5. **Journal template** — Customize `wiki/journal/template.md` for your workflow Everything else — page format, linking conventions, workflows, rules — is universal and works across domains. ## Example Domains This template works for any knowledge-intensive topic: - **Research notes** — papers, experiments, methodologies - **Book analysis** — themes, characters, author techniques - **Competitive analysis** — companies, products, market trends - **Course notes** — lectures, readings, key concepts - **Personal development** — frameworks, habits, book summaries - **Technical documentation** — APIs, architectures, design patterns - **Hobby deep-dives** — any subject you want to master ## License MIT --- title: "Docker Knowledge Base — Index" type: index updated: 2026-07-07 --- # Docker — Knowledge Base Docker packages applications into containers: portable, isolated processes that share the host kernel. This wiki covers the open-source Docker Engine and CLI, the Dockerfile, image building (BuildKit), Docker Compose, container runtime, networking, storage, the daemon, the Engine API, and security. Scope is the open-source engine plus Compose - not Docker Desktop, Scout, Hub, or enterprise features. Master catalog - every page appears here. ## Concepts ### Core - [[concepts/what-is-docker]] - containers vs VMs, the client/daemon/registry architecture - [[concepts/images-and-registries]] - image layers, tags, digests, pull/push, registries - [[concepts/installation]] - installing Docker Engine per platform ### Building - [[concepts/dockerfile-reference]] - every Dockerfile instruction and its syntax - [[concepts/building-images]] - docker build, BuildKit, multi-stage, cache, secrets - [[concepts/docker-cli]] - the `docker` CLI, global options, key commands ### Running - [[concepts/containers-and-runtime]] - the container lifecycle and `docker run` flags - [[concepts/networking]] - network drivers, port publishing, service discovery - [[concepts/storage-and-volumes]] - volumes, bind mounts, tmpfs ### Compose - [[concepts/docker-compose]] - the `docker compose` CLI and project model - [[concepts/compose-file-reference]] - the compose.yaml specification ### Engine & operations - [[concepts/docker-engine-and-daemon]] - dockerd, daemon.json, storage/log drivers, rootless - [[concepts/engine-api]] - the Docker Engine REST API and versioning - [[concepts/security]] - namespaces/cgroups, capabilities, rootless, secrets, trust ## Syntheses - [[syntheses/dockerfile-best-practices]] - layer caching, multi-stage, image size - [[syntheses/volumes-vs-bind-mounts]] - volumes vs bind mounts vs tmpfs - [[syntheses/run-vs-compose]] - docker run vs Docker Compose ## XL edition (Pro) Deeper per-item reference is available to Pro subscribers under this wiki's XL edition (not listed above): full CLI flag tables (containers, images, build, network, volume, system, swarm), the complete Compose file specification, every Dockerfile instruction, and the Engine API reference — 23 pages. Subscribe at /pro. ## Statistics - **Total pages**: 17 - **Concepts**: 14 - **Entities**: 0 - **Summaries**: 0 - **Syntheses**: 3 --- title: "Building Images (docker build, BuildKit, buildx)" type: concept tags: [docker, build, buildkit, buildx, multi-stage, cache, well-established] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-build.md", "raw/llms_txt_doc-dockerfile.md", "raw/llms_txt_doc-docker-cli.md"] confidence: high --- ## Definition Docker Build is one of Docker Engine's most-used features: whenever you create an image, you are using Docker Build. It packages and bundles your code so you can ship it anywhere. Build is more than a single command - it is an ecosystem of tools and features (BuildKit, buildx, multi-stage builds, multi-platform images, build drivers, exporters, build caching, and Bake) that support both common workflows and advanced scenarios. The input is a [[concepts/dockerfile-reference|Dockerfile]]; the output is an [[concepts/images-and-registries|image]]. ## How It Works `docker build` reads a Dockerfile and a build context (the set of files sent to the builder, usually the current directory `.`) and produces an image. The modern build backend is **BuildKit**, the open-source build engine, invoked through **buildx** (`docker buildx build`, also the default behind `docker build` on current Engine). BuildKit parallelizes independent build steps, skips unused stages, and manages a content-addressable cache. Basic invocation: ``` docker build -t name:tag . docker buildx build -t name:tag . ``` Common flags: `-t`/`--tag` (name the image), `-f`/`--file` (Dockerfile path), `--build-arg` (set an `ARG`), `--target` (stop at a named stage), `--no-cache` (ignore cache), `--pull` (always fetch a newer base), `--platform` (target architecture(s)), `--secret`, `--ssh`, `--push`/`--load`/`--output` (where to send the result). Progress output type is controlled by `BUILDKIT_PROGRESS` (`auto`, `plain`, `tty`, `rawjson`). **Multi-stage builds** keep images small and secure with minimal dependencies. Each `FROM` starts a new stage; name a stage with `AS`, then copy only the needed artifacts forward with `COPY --from=`: ``` FROM golang:1.22 AS builder WORKDIR /src COPY . . RUN go build -o /app FROM alpine:3.20 COPY --from=builder /app /app ENTRYPOINT ["/app"] ``` Only the final stage's layers ship, so build-time toolchains never reach the runtime image. `--target builder` builds just that stage (useful for a test or debug target). **Build cache.** Each Dockerfile instruction produces a cache layer; on rebuild, BuildKit reuses layers whose inputs are unchanged and rebuilds from the first change downward. Ordering instructions from least- to most-frequently-changing (dependencies before source) maximizes reuse. `RUN --mount=type=cache,target=` persists a cache directory (for example a package manager cache) across builds without baking it into the image. Cache can also be exported/imported across machines via cache backends (inline, registry, local, GitHub Actions, S3, Azure Blob). ## Key Parameters **Build arguments** parameterize a build. Declare with `ARG` in the Dockerfile and pass at build time: ``` docker build --build-arg VERSION=1.2.3 -t app:1.2.3 . ``` `ARG`s declared before the first `FROM` are global and usable in `FROM` lines. Note that build args are visible in image history and should not carry secrets. **Build secrets** provide credentials to a build without persisting them in the image or its layers. Mount a secret into a single `RUN` step: ``` RUN --mount=type=secret,id=mytoken \ TOKEN=$(cat /run/secrets/mytoken) some-command ``` ``` docker build --secret id=mytoken,src=./token.txt . ``` The secret is available only during that `RUN` and is never written to the final image. `RUN --mount=type=ssh` similarly forwards an SSH agent for fetching private dependencies. **Multi-platform images** let you build, push, pull, and run images across architectures (for example `linux/amd64` and `linux/arm64`) from a single command: ``` docker buildx build --platform linux/amd64,linux/arm64 -t name:tag --push . ``` `DOCKER_DEFAULT_PLATFORM` sets a default for commands that take `--platform`. BuildKit exposes built-in build args such as `BUILDKIT_INLINE_CACHE` (embed cache metadata) and `BUILDKIT_CONTEXT_KEEP_GIT_DIR`. **Exporters and drivers.** Build drivers configure where and how builds run (the default `docker` driver, or `docker-container`/`kubernetes`/`remote` for advanced features like multi-platform and cache export). Exporters let you output artifacts other than a Docker image (e.g. a local tarball or filesystem) via `--output`. Bake (`docker buildx bake`) orchestrates multiple builds from a declarative file. ## When To Use - **Any time you produce an image** from source - `docker build` is the entry point. - **Multi-stage** whenever build tools (compilers, dev headers) should not ship in the runtime image, or to produce test/lint/debug targets from one Dockerfile. - **Build secrets** for registry tokens, SSH keys, or API credentials needed only during the build. - **Multi-platform** when the same image must run on both x86 and ARM (Apple Silicon, Graviton, Raspberry Pi). - **Cache mounts and remote cache** to speed up CI, where the local layer cache is cold on every run. ## Risks & Pitfalls - **Secrets in `--build-arg` or `ENV` leak.** Build args and env values are recoverable from image history; use `--secret` / `RUN --mount=type=secret` instead. See [[concepts/security]]. - **A large build context is slow.** Everything in the context directory is sent to the builder; use a `.dockerignore` file to exclude `node_modules`, `.git`, and build output. - **Cache invalidation cascades.** Changing an early instruction (e.g. `COPY . .` before installing dependencies) busts the cache for every step after it. Copy dependency manifests and install before copying the full source. - **`--platform` without an emulator or the right driver fails.** Cross-building non-native architectures needs QEMU/binfmt and often the `docker-container` driver. - **`--push` vs `--load`.** Multi-platform results cannot be loaded into the local single-arch image store; you must `--push` to a registry (or `--output`). - **Stale base images.** `docker build` reuses a cached base; add `--pull` to force fetching a newer base image. ## Related Concepts - [[concepts/dockerfile-reference]] - [[concepts/images-and-registries]] - [[concepts/docker-cli]] - [[syntheses/dockerfile-best-practices]] - [[concepts/what-is-docker]] ## Sources - `raw/llms_txt_doc-docker-build.md` - `raw/llms_txt_doc-dockerfile.md` - `raw/llms_txt_doc-docker-cli.md` --- title: "Compose File Reference" type: concept tags: [compose, yaml, reference, services, configuration] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-compose-file.md", "raw/llms_txt_doc-compose-cli.md"] source_url: "https://docs.docker.com/reference/compose-file/" confidence: high --- ## Definition The Compose file (`compose.yaml`) configures a Docker application's services, networks, volumes, and more. The **Compose Specification** is the latest and recommended version of the Compose file format. Legacy versions 2.x and 3.x of the format were merged into the Compose Specification, which is implemented in versions 1.27.0 and above of the Docker Compose CLI (also known as Compose v2). By default Compose looks for `compose.yaml` or `docker-compose.yaml`. ## Top-level elements A Compose file is organized into these top-level keys: - `version` and `name` - version and name attributes for the Compose file - `services` - the containers that make up the application (the primary element) - `networks` - networks the services attach to - `volumes` - named volumes for persistent data - `configs` - configuration data made available to services - `secrets` - sensitive data made available to services ## The `services` element Each entry under `services` names one service and configures the container(s) that run it. Common service keys: - `image` - the image to run, e.g. `image: examples/web` - `build` - build context / instructions when the image is built locally, e.g. `build: .` - `ports` - published ports, in `"HOST:CONTAINER"` form, e.g. `- "8000:8000"` - `environment` - environment variables, e.g. `- DEBUG=1` - `volumes` - mounts, e.g. `- "/data"` (also named volumes and bind mounts) - `depends_on` - express startup/shutdown dependencies between services - `healthcheck` - define a health probe; other services can wait on it with `depends_on` - `deploy` - deployment/runtime constraints (replicas, resource limits, restart policy in Swarm) - `restart` - restart policy for the service's containers - `networks` - which top-level networks this service joins - `secrets` / `configs` - grant the service access to declared secrets/configs - `profiles` - assign the service to one or more profiles so it starts only when that profile is enabled A minimal example combining several keys (from the Compose CLI reference): ```yaml services: webapp: image: examples/web ports: - "8000:8000" volumes: - "/data" ``` Overriding the same service across multiple files - matching fields override the previous file, new values add to the service configuration: ```yaml services: webapp: build: . environment: - DEBUG=1 ``` ## Merging and overriding files When you supply multiple files with `-f`, Compose combines them into a single configuration in the order supplied; subsequent files override and add to their predecessors. If a later file specifies the same service, any matching fields override the earlier file and new values add to the service. All paths in the files are relative to the first configuration file specified with `-f` (override with `--project-directory`). This layering is the mechanism behind `compose.override.yaml` and environment-specific overrides (for example a base `compose.yaml` plus `compose.admin.yaml`). ## Startup order Startup and shutdown order is controlled with `depends_on` together with health checks: a `healthcheck` on one service lets dependent services wait until it reports healthy before starting. In the dry-run example from the CLI docs, the `backend` and `proxy` services wait until the `db` service is healthy before starting. ## Profiles Services can be tagged with `profiles`. A service assigned to a profile only starts when that profile is enabled (via `--profile`, `COMPOSE_PROFILES`, or by being a dependency of an enabled service). Services with no profile always start. This keeps optional components (debug tooling, seed jobs, alternate frontends) out of the default `up`. ## Project name in the file The top-level `name:` key sets the project name from within the file. It sits below the `-p` flag and `COMPOSE_PROJECT_NAME` in precedence but above the directory basename. When multiple files are merged, the last `name:` wins. ## Related Concepts - [[concepts/docker-compose]] - the `docker compose` CLI that reads this file - [[concepts/containers-and-runtime]] - the underlying container runtime keys map to `docker run` flags - [[concepts/networking]] - the `networks` top-level element - [[concepts/storage-and-volumes]] - the `volumes` top-level element - [[concepts/dockerfile-reference]] - what `build:` compiles - [[syntheses/run-vs-compose]] - declarative Compose vs imperative `docker run` ## Sources - raw/llms_txt_doc-compose-file.md - raw/llms_txt_doc-compose-cli.md --- title: "Containers and the Runtime" type: concept tags: [containers, runtime, docker-run, lifecycle, cli] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-concepts.md", "raw/llms_txt_doc-docker-cli.md", "raw/llms_txt_doc-what-is-docker.md"] source_url: "https://docs.docker.com/reference/cli/docker/" confidence: high --- ## Definition A container is a runnable instance of an image - an isolated process with all of the files it needs to run. Containers are self-contained, isolated, independent, and portable: each has everything it needs to function with no reliance on pre-installed host dependencies, runs in isolation from the host and other containers, is independently managed (deleting one doesn't affect others), and runs the same way anywhere. Unlike a virtual machine, a container does not carry its own OS kernel; multiple containers share the host kernel, letting you run more applications on less infrastructure. You can create, start, stop, move, or delete a container using the Docker API or CLI, connect it to one or more networks, attach storage to it, or create a new image based on its current state. ## What happens on `docker run` Running `docker run -i -t ubuntu /bin/bash` (assuming default registry configuration): 1. If the `ubuntu` image isn't local, Docker pulls it from the configured registry (as if you ran `docker pull ubuntu`). 2. Docker creates a new container (as if you ran `docker container create`). 3. Docker allocates a read-write filesystem to the container as its final layer, so the running container can create or modify files. 4. Docker creates a network interface to connect the container to the default network and assigns it an IP address (since no networking options were specified). By default, containers can reach external networks using the host's network connection. 5. Docker starts the container and executes `/bin/bash`. Because of `-i` and `-t`, the container runs interactively attached to your terminal. When you `exit`, the container stops but isn't removed - you can start it again or remove it. When a container is removed, any changes to its state that aren't in persistent storage disappear. ## Container lifecycle commands The base run syntax is: ``` docker run [OPTIONS] IMAGE [COMMAND] [ARG...] ``` Start a detached container publishing a port: ``` $ docker run -d -p 8080:80 docker/welcome-to-docker ``` `-d` runs the container in detached (background) mode; the command returns the full container ID. List running containers (add `-a` to include stopped ones): ``` $ docker ps $ docker ps -a ``` Stop a container by ID or name: ``` $ docker stop ``` When referencing a container by ID you only need enough of the ID to be unique, e.g. `docker stop a1f`. The `docker container` management command groups the full lifecycle. Subcommands include: `attach`, `commit`, `cp`, `create`, `diff`, `exec`, `export`, `inspect`, `kill`, `logs`, `ls`, `pause`, `port`, `prune`, `rename`, `restart`, `rm`, `run`, `start`, `stats`, `stop`, `top`, `unpause`, `update`, and `wait`. The legacy top-level aliases (`docker run`, `docker ps`, `docker stop`, `docker rm`, `docker exec`, `docker logs`, `docker inspect`, ...) map onto these. ## Key `docker run` flags - `-d, --detach` - run the container in the background - `-p HOST:CONTAINER` - publish a container port to the host (e.g. `-p 8080:80`) - `-v` - mount a volume or bind mount (e.g. `docker run -v /host:/container example/mysql`) - `-e` - set an environment variable in the container - `--name` - assign a name to the container (a string option, specified once) - `--network` - connect the container to a specific network - `--restart` - set a restart policy for the container - `--rm` - automatically remove the container when it exits - `-i, --interactive` - keep STDIN open; `-t, --tty` - allocate a pseudo-TTY Single-character options can be combined: instead of `docker run -i -t --name test busybox sh`, write `docker run -it --name test busybox sh`. Boolean options default as shown in help text; specifying the flag without a value sets it to `true` (so `docker run -d` runs detached). Options that default to `true` (e.g. `docker build --rm=true`) are turned off with `--rm=false`. Multi-value options like `-a` and `-v` can be repeated in one command line. ## Detaching and re-attaching Once attached to a container, detach and leave it running with the `CTRL-p CTRL-q` key sequence. The sequence is customizable via the `detachKeys` property in `config.json` or per-command with `--detach-keys` on `docker attach`, `docker exec`, `docker run`, or `docker start`. ## Related Concepts - [[concepts/docker-cli]] - the full `docker` command surface and environment variables - [[concepts/images-and-registries]] - the images containers instantiate - [[concepts/networking]] - `--network`, `-p`, and container networking - [[concepts/storage-and-volumes]] - `-v` / `--mount` and persistence - [[concepts/docker-compose]] - declarative multi-container runtime - [[syntheses/run-vs-compose]] - `docker run` vs Compose ## Sources - raw/llms_txt_doc-docker-concepts.md - raw/llms_txt_doc-docker-cli.md - raw/llms_txt_doc-what-is-docker.md --- title: "Docker CLI" type: concept tags: [docker, cli, commands, reference, well-established] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-cli.md", "raw/llms_txt_doc-docker-concepts.md"] confidence: high --- ## Definition `docker` is the base command for the Docker CLI and the primary way most users interact with Docker. When you run a command such as `docker run`, the client sends it to the daemon (`dockerd`) over the Docker API, and the daemon carries it out. The client can talk to more than one daemon. To list help on any command, append `--help`, for example `docker run --help`. ## How It Works **Global options** apply to the `docker` command itself and select which daemon to talk to and how: ``` --config Location of client config files (default /root/.docker) -c, --context Name of the context to use (overrides DOCKER_HOST) -D, --debug Enable debug mode -H, --host Daemon socket to connect to -l, --log-level Set logging level (debug, info, warn, error, fatal; default info) --tls / --tlsverify / --tlscacert / --tlscert / --tlskey ``` The `-H`/`--host` flag accepts `unix://`, `tcp://`, `ssh://`, and `npipe://` schemes, e.g. `docker -H ssh://user@192.168.64.5 ps` or `docker -H tcp://174.17.0.1:2376 ps`. By convention the daemon uses port `2376` for secure TLS and `2375` for insecure connections. Default sockets are `unix:///var/run/docker.sock` on macOS/Linux and `npipe:////./pipe/docker_engine` on Windows. See [[concepts/docker-engine-and-daemon]]. **Option types.** Single-character options can be combined (`docker run -it` == `docker run -i -t`). Boolean flags default as shown in help; specifying the flag without a value sets it true, and flags that default to true are disabled explicitly, e.g. `docker build --rm=false .`. **Environment variables** that control the client include `DOCKER_HOST` (daemon socket), `DOCKER_CONTEXT` (context name, overrides `DOCKER_HOST`), `DOCKER_CONFIG` (config directory), `DOCKER_API_VERSION`, `DOCKER_DEFAULT_PLATFORM` (default for `--platform`), `DOCKER_TLS_VERIFY`, and `BUILDKIT_PROGRESS` (`auto`, `plain`, `tty`, `rawjson`). Go's `HTTP_PROXY`/`HTTPS_PROXY`/`NO_PROXY` are also honored. **Configuration file.** The CLI stores config in `~/.docker/config.json`. You can set default output formats (`psFormat`, `imagesFormat`, `statsFormat`, and more), credential stores (`credsStore`, `credHelpers`), automatic container/build proxies (`proxies`), and the detach key sequence (`detachKeys`, default `CTRL-p CTRL-q`). ## Key Parameters Docker groups functionality under management commands (`docker container`, `docker image`, `docker network`, `docker volume`, `docker system`, `docker builder`/`buildx`, `docker compose`, `docker context`, etc.). The most-used commands, grouped by function: **Run and manage container lifecycle:** ``` docker run [OPTIONS] IMAGE [COMMAND] [ARG...] # create and run a new container docker ps # list running containers docker ps -a # list all containers (incl. stopped) docker stop docker start docker rm # remove a container ``` Common `docker run` flags: `-d` (detached/background), `-i` (keep STDIN open), `-t` (allocate a TTY), `-p host:container` (publish a port), `-P` (publish all exposed ports), `-v` / `--mount` (attach storage), `-e` (set env var), `--name` (name the container), `--rm` (remove on exit), `--network`, `--restart`. Example from the docs: `docker run -d -p 8080:80 docker/welcome-to-docker`, and `docker run -i -t ubuntu /bin/bash`. **Inspect and observe:** ``` docker logs # fetch container logs (add -f to follow) docker exec -it sh # run a command in a running container docker inspect # low-level JSON info on any Docker object docker stats # live resource usage of containers docker cp # copy files between host and container docker top # running processes in a container ``` **Images and registries** (see [[concepts/images-and-registries]]): ``` docker build -t name:tag . # build an image from a Dockerfile docker images # list local images (alias: docker image ls) docker pull docker push docker tag SOURCE TARGET docker rmi # remove an image docker login / docker logout ``` **Housekeeping:** ``` docker system prune # remove unused data docker system df # show Docker disk usage docker version / docker info ``` When referencing a container by ID you only need enough of the prefix to be unique (`docker stop a1f`). ## When To Use Use the CLI for interactive and scripted control of individual containers, images, networks, and volumes on a single daemon. For multi-container applications defined declaratively, prefer [[concepts/docker-compose]] (`docker compose up`). See the [[syntheses/run-vs-compose]] comparison for choosing between ad hoc `docker run` and Compose. ## Risks & Pitfalls - **`docker ps` hides stopped containers.** Add `-a` to see all containers, otherwise a stopped container looks "gone." - **`sudo` and the `docker` group.** You may need `sudo` for each command, or membership in the `docker` group; that group grants root-equivalent access to the host. See [[concepts/security]]. - **Precedence of settings.** Command-line options override environment variables, which override `config.json` properties; `--config` overrides `DOCKER_CONFIG`. - **Wrong daemon.** `DOCKER_HOST`, `DOCKER_CONTEXT`, or a `-H` flag can silently point the CLI at a different (or remote) daemon. Confirm your context with `docker context` when results look wrong. - **`config.json` holds secrets.** It can contain registry credentials and proxy passwords; do not commit it. - **`-t` with `-a stderr`.** Avoid combining them due to pty limitations; in pty mode all stderr goes to stdout. ## Related Concepts - [[concepts/what-is-docker]] - [[concepts/containers-and-runtime]] - [[concepts/images-and-registries]] - [[concepts/building-images]] - [[concepts/docker-compose]] - [[concepts/docker-engine-and-daemon]] ## Sources - `raw/llms_txt_doc-docker-cli.md` - `raw/llms_txt_doc-docker-concepts.md` --- title: "Docker Compose" type: concept tags: [compose, cli, orchestration, multi-container, yaml] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-compose.md", "raw/llms_txt_doc-compose-cli.md"] source_url: "https://docs.docker.com/reference/cli/docker/compose/" confidence: high --- ## Definition Docker Compose is a tool for defining and running multi-container applications. It lets you manage services, networks, and volumes for an entire application stack from a single YAML configuration file (`compose.yaml`), and then create and start everything with a single command. Compose works in all environments - production, staging, development, testing, and CI workflows - and has commands for the whole application lifecycle: start, stop, and rebuild services; view running-service status; stream log output; and run one-off commands on a service. In Compose v2 (CLI versions 1.27.0 and above), Compose is invoked as the `docker compose` subcommand of the Docker CLI (space, not the legacy `docker-compose` hyphenated binary). ## Project model Each Compose configuration has a **project name**. Compose sets it using the following mechanisms, in order of precedence: 1. The `-p` command line flag 2. The `COMPOSE_PROJECT_NAME` environment variable 3. The top-level `name:` variable from the config file (or the last `name:` from a series of files specified with `-f`) 4. The `basename` of the project directory containing the config file (or of the first config file specified with `-f`) 5. The `basename` of the current directory if no config file is specified Project names must contain only lowercase letters, decimal digits, dashes, and underscores, and must begin with a lowercase letter or decimal digit. ## The `docker compose` CLI Top-level options (from `docker compose --help`): - `-f, --file` - Compose configuration files - `-p, --project-name` - Project name - `--profile` - Specify a profile to enable - `--project-directory` - Specify an alternate working directory (default: the path of the first specified Compose file) - `--env-file` - Specify an alternate environment file - `--parallel` (default `-1`) - Control max parallelism, `-1` for unlimited - `--compatibility` - Run compose in backward compatibility mode - `--dry-run` - Execute command in dry run mode - `--ansi` (default `auto`) - Control when to print ANSI control characters (`never`|`always`|`auto`) - `--progress` - Set type of progress output (auto, tty, plain, json, quiet) - `--all-resources` - Include all resources, even those not used by services ### Core subcommands - `docker compose up` - Create and start containers - `docker compose down` - Stop and remove containers, networks - `docker compose ps` - List containers - `docker compose logs` - View output from containers - `docker compose build` - Build or rebuild services - `docker compose exec` - Execute a command in a running container - `docker compose run` - Run a one-off command on a service - `docker compose start` / `stop` / `restart` - Start / stop / restart services - `docker compose create` - Creates containers for a service - `docker compose rm` - Removes stopped service containers - `docker compose pull` / `push` - Pull / push service images - `docker compose config` - Parse, resolve and render compose file in canonical format - `docker compose ls` - List running compose projects - `docker compose images` - List images used by the created containers - `docker compose kill` - Force stop service containers - `docker compose pause` / `unpause` - Pause / unpause services - `docker compose top` - Display the running processes - `docker compose stats` - Display a live stream of container(s) resource usage statistics - `docker compose wait` - Block until containers of all (or specified) services stop - `docker compose watch` - Watch build context and rebuild/refresh containers when files are updated - `docker compose version` - Show the Docker Compose version information ## Specifying Compose files with `-f` The `-f` flag is optional. If you don't provide it, Compose traverses the working directory and its parent directories looking for a `compose.yaml` or `docker-compose.yaml` file. You can supply multiple `-f` files; Compose combines them into a single configuration in the order you supply them, with subsequent files overriding and adding to their predecessors: ``` $ docker compose -f compose.yaml -f compose.admin.yaml run backup_db ``` When you use multiple files, all paths are relative to the first configuration file specified with `-f` (override with `--project-directory`). Use `-f -` to read the configuration from stdin. `-f` also accepts an `oci://` prefix (a Compose file published to an OCI registry via `docker compose publish`), plain HTTPS/SSH git URLs, and git refs/subdirectories: ``` $ docker compose -f oci://registry.example.com/my-compose-project:latest up $ docker compose -f https://github.com/user/repo.git@v1.0.0 up $ docker compose -f https://github.com/user/repo.git#main:path/to/compose.yaml up ``` ## Profiles and optional services `docker compose --profile frontend up` starts the services with the profile `frontend` plus services without any specified profiles. Enable multiple profiles by repeating the flag: `docker compose --profile frontend --profile debug up`. Profiles can also be set via the `COMPOSE_PROFILES` environment variable. ## Environment-variable equivalents Several flags have environment-variable equivalents: `COMPOSE_FILE` (= `-f`), `COMPOSE_PROJECT_NAME` (= `-p`), `COMPOSE_PROFILES` (= `--profiles`), and `COMPOSE_PARALLEL_LIMIT` (= `--parallel`). If a flag is explicitly set on the command line, the associated environment variable is ignored. `COMPOSE_IGNORE_ORPHANS=true` stops Compose from detecting orphaned containers; `COMPOSE_MENU=false` disables the helper menu in attached `up`. ## Dry Run mode Use the `--dry-run` flag to test a command without changing your application stack state; it shows every step Compose would apply (for example `docker compose --dry-run up --build -d`). Dry Run works with almost all commands but not with read-only commands like `ps`, `ls`, or `logs`. ## Related Concepts - [[concepts/compose-file-reference]] - the `compose.yaml` specification the CLI consumes - [[concepts/containers-and-runtime]] - the single-container `docker run` equivalent - [[concepts/docker-cli]] - the parent `docker` CLI - [[syntheses/run-vs-compose]] - when to use `docker run` vs Compose ## Sources - raw/llms_txt_doc-docker-compose.md - raw/llms_txt_doc-compose-cli.md --- title: "Docker Engine and the dockerd Daemon" type: concept tags: [engine, dockerd, daemon, configuration, storage-driver, logging, rootless, operations] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-engine.md", "raw/llms_txt_doc-daemon-cli-dockerd.md", "raw/llms_txt_doc-what-is-docker.md"] source_url: "https://docs.docker.com/reference/cli/dockerd/" confidence: high --- # Docker Engine and the dockerd Daemon ## Definition Docker Engine is an open source containerization technology for building and containerizing applications. It is licensed under the Apache License, Version 2.0. Docker Engine acts as a client-server application with three parts: a server with a long-running daemon process (`dockerd`); APIs which specify interfaces that programs use to talk to and instruct the Docker daemon; and a command line interface (CLI) client (`docker`). `dockerd` is the persistent process that manages containers; Docker uses different binaries for the daemon and client. To run the daemon you type `dockerd`. ## How It Works The daemon listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. It relies on an OCI-compliant runtime (invoked via the `containerd` daemon) as its interface to the Linux kernel namespaces, cgroups, and SELinux. By default the daemon uses `runc` as the container runtime and automatically starts `containerd`. The daemon can listen for Engine API requests via three socket types: `unix`, `tcp`, and `fd`. By default a `unix` domain socket is created at `/var/run/docker.sock`, requiring either root permission or `docker` group membership. Remote access requires enabling the tcp socket; convention uses port `2375` for un-encrypted and `2376` for encrypted (TLS) communication. Bind sockets with `-H`, e.g. `dockerd -H unix:///var/run/docker.sock -H tcp://192.168.59.106`. Persistent state lives under `--data-root` (default `/var/lib/docker`); execution state under `--exec-root` (default `/var/run/docker`). To run with debug output use `dockerd --debug` or add `"debug": true` to `daemon.json`. Enable experimental features with `--experimental` or `"experimental": true`. ## Key Parameters Configuration file (`--config-file`, default `/etc/docker/daemon.json` on Linux; `%programdata%\docker\config\daemon.json` on Windows). It uses the same flag names as keys, except plural forms for multi-entry flags (e.g. `labels` for `--label`). The daemon fails to start if an option is duplicated between the file and flags. Validate without starting via `dockerd --validate --config-file=/tmp/valid-config.json`. Common `daemon.json` keys (verbatim): - `"data-root"`, `"exec-root"`, `"pidfile"`, `"group"` (unix socket group, default `docker`) - `"storage-driver"` / `"storage-opts"` -- Linux drivers: `overlay2` (preferred, default), `fuse-overlayfs`, `btrfs`, `zfs`; Windows only supports `windowsfilter` - `"log-driver"` (default `json-file`), `"log-opts"` (e.g. `max-size`, `max-file`), `"log-format"` (`text` default, or `json`), `"log-level"` (`debug`|`info`|`warn`|`error`|`fatal`, default `info`) - `"default-runtime"` (default `runc`), `"runtimes"` (register OCI runtimes) - `"insecure-registries"`, `"registry-mirrors"` - `"live-restore"`, `"experimental"`, `"debug"` - `"features"` -- e.g. `"containerd-snapshotter": true` (use containerd snapshotters instead of classic storage drivers), `"cdi": true` (Container Device Interface, enabled by default since Engine 28.3.0) - `"proxies"` with `"http-proxy"`, `"https-proxy"`, `"no-proxy"` (Engine 23.0+) - `"userns-remap"`, `"no-new-privileges"`, `"seccomp-profile"`, `"selinux-enabled"` (security hardening) - `"default-address-pools"` (with `base` and `size`), `"default-ulimits"`, `"default-shm-size"` (default `64M`) The cgroup driver is set via `--exec-opt native.cgroupdriver`, which accepts only `cgroupfs` or `systemd` (default: `cgroupfs` on cgroup v1, `systemd` on cgroup v2 hosts with systemd available). ## When To Use Configure `dockerd` and `daemon.json` when operating Docker as a host service: choosing a storage driver, setting default logging, registering alternative runtimes (e.g. gVisor/`io.containerd.runsc.v1`, Kata), exposing the daemon over TCP/TLS or SSH, setting proxies, advertising GPUs via `"node-generic-resources"`, or serving Prometheus metrics via `--metrics-addr 127.0.0.1:9323`. Use rootless mode (`--rootless`, typically with RootlessKit) to run Docker without root privileges. Some options reload without a restart via `SIGHUP` on Linux (reconfigurable set includes `debug`, `labels`, `live-restore`, `max-concurrent-downloads`, `default-runtime`, `runtimes`, `insecure-registries`, `registry-mirrors`, `features`); apply with `sudo systemctl reload dockerd`. ## Risks & Pitfalls - Binding the daemon to a TCP port or the `docker` user group introduces security risks: anyone with access to that port has full Docker access and may gain root on the host. Do not expose `0.0.0.0:2375` on an open network; secure with the built-in HTTPS socket or a web proxy. - On systemd-based systems `-H` is already set, so you cannot use the `hosts` key in `daemon.json` to add listening addresses (use a systemd drop-in instead). - `--insecure-registry` allows un-encrypted/untrusted communication and creates security vulnerabilities; prefer adding the registry CA to the system trust store. - Running multiple daemons on one host is experimental; each needs distinct `--bridge`, `--exec-root`, `--data-root`, `--pidfile`, and `-H`. ## Related Concepts - [[concepts/what-is-docker]] - [[concepts/containers-and-runtime]] - [[concepts/engine-api]] - [[concepts/security]] - [[concepts/storage-and-volumes]] - [[concepts/installation]] ## Sources - raw/llms_txt_doc-docker-engine.md - raw/llms_txt_doc-daemon-cli-dockerd.md - raw/llms_txt_doc-what-is-docker.md --- title: "Dockerfile Reference" type: concept tags: [docker, dockerfile, build, instructions, reference, well-established] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-dockerfile.md"] confidence: high --- ## Definition A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image. Docker builds images automatically by reading its instructions. Instructions are not case-sensitive, but convention is UPPERCASE to distinguish them from arguments. Docker runs instructions in order, and a Dockerfile must begin with a `FROM` instruction (which may only be preceded by parser directives, comments, and globally scoped `ARG`s). ## How It Works The format is `# Comment` then `INSTRUCTION arguments`. BuildKit treats a line beginning with `#` as a comment unless it is a valid parser directive. Parser directives must be at the top of the file, in the form `# directive=value`, and each may be used once. The main ones are: - `# syntax=docker/dockerfile:1` - pull the latest stable Dockerfile frontend before building. - `# escape=\` (default) or `` # escape=` `` - set the escape character (backtick is useful on Windows where `\` is the path separator). Most instructions accept two forms: - **Exec form** - a JSON array with double quotes, e.g. `ENTRYPOINT ["/bin/bash", "-c", "echo hello"]`. It does not invoke a shell, so shell processing such as variable substitution does not happen (`RUN [ "echo", "$HOME" ]` will not expand `$HOME`; use `RUN [ "sh", "-c", "echo $HOME" ]`). - **Shell form** - e.g. `CMD command param1 param2`, which runs via a shell (`/bin/sh -c` on Linux). ## Key Parameters Verbatim instruction syntax: ``` FROM [--platform=] [:] [AS ] FROM [--platform=] [@] [AS ] RUN (shell form) RUN ["executable","param1","param2"] (exec form) CMD ["executable","param1","param2"] (exec form) CMD ["param1","param2"] (default params to ENTRYPOINT) CMD command param1 param2 (shell form) ENTRYPOINT ["executable", "param1"] (exec form) ENTRYPOINT command param1 (shell form) LABEL = ... ENV = ... ADD [--chown=] [--chmod=] ... COPY [--from=] [--chown=] [--chmod=] ... EXPOSE [/...] VOLUME ["/var/log/"] USER [:] WORKDIR /path/to/workdir ARG [=] STOPSIGNAL signal SHELL ["executable", "parameters"] HEALTHCHECK [OPTIONS] CMD command HEALTHCHECK NONE ONBUILD ``` Notable behaviors: - **`FROM`** initializes a new build stage and sets the base image. `AS ` names the stage for later `COPY --from=` (the basis of [[concepts/building-images|multi-stage builds]]). - **`RUN`** executes build commands and supports mounts: `RUN --mount=type=cache,...`, `RUN --mount=type=secret,...`, `RUN --mount=type=bind,...`, `RUN --mount=type=ssh,...`, plus `RUN --network=` and `RUN --security=`. - **`CMD` vs `ENTRYPOINT`** - there can be only one of each (the last wins). `ENTRYPOINT` sets the executable that always runs; `CMD` provides default arguments that can be overridden at runtime. Use exec form for `ENTRYPOINT` combined with `CMD` for default args. - **`COPY` / `ADD`** - both copy into the image. `ADD` additionally supports remote URLs and auto-extraction of local tar archives; `COPY` is preferred for plain file copies. Flags include `--chown`, `--chmod`, `--link`, `--from`, `--exclude`, and (for `ADD`) `--checksum`, `--keep-git-dir`, `--unpack`. - **`ENV`** sets persistent environment variables in the image; if `ENV` overrides an `ARG` of the same name, the constant `ENV` value is what the image includes. - **`ARG`** defines a build-time variable passed with `docker build --build-arg =`. `ARG`s before `FROM` are global and usable in `FROM` lines only. - **`EXPOSE`** documents which ports the container listens on (default TCP; add `/udp`). It does not publish the port - use `-p` or `-P` on `docker run`. - **`WORKDIR`** sets the working directory for subsequent `RUN`, `CMD`, `ENTRYPOINT`, `COPY`, and `ADD`; it is created if it does not exist. Relative paths are relative to the previous `WORKDIR`. - **`HEALTHCHECK`** options are `--interval=DURATION` (default 30s), `--timeout=DURATION` (default 30s), `--start-period=DURATION` (default 0s), `--start-interval=DURATION` (default 5s, requires Engine 25.0+), `--retries=N` (default 3). The check command should exit 0 (healthy) or 1 (unhealthy); exit 2 is reserved. Example: `HEALTHCHECK --interval=5m --timeout=3s CMD curl -f http://localhost/ || exit 1`. - **`SHELL`** overrides the default shell (Linux `["/bin/sh", "-c"]`, Windows `["cmd", "/S", "/C"]`); must be JSON form and may appear multiple times. - **`ONBUILD`** registers an instruction to run later, when the image is used as the base for another build. - **`MAINTAINER`** is deprecated; use `LABEL` instead. ## When To Use Write a Dockerfile whenever you need a reproducible, versioned recipe for an image rather than committing container state by hand. It is the input to [[concepts/building-images|docker build]] and the canonical way to define a base image, install dependencies, copy in code, set the runtime user and working directory, declare ports, and define the default command. ## Risks & Pitfalls - **Shell vs exec form changes signal handling and variable expansion.** Exec form does not run a shell, so `$VAR` is not expanded and the process is PID 1 (signals reach it directly). - **Only the last `CMD`, `ENTRYPOINT`, and `HEALTHCHECK` take effect.** Earlier ones are silently ignored. - **`EXPOSE` does not publish ports.** It is documentation only; publishing needs `-p`/`-P` at run time. See [[concepts/networking]]. - **Parser directives are position-sensitive.** They must be at the very top; once any comment, blank line, or instruction is processed, BuildKit stops looking for them. - **`ADD` with remote URLs and archives is surprising.** Prefer `COPY` unless you specifically need `ADD`'s extraction or URL fetching. - **Each instruction is a cache layer.** Order instructions from least- to most-frequently-changing to maximize [[concepts/building-images|build cache]] reuse. ## Related Concepts - [[concepts/building-images]] - [[concepts/images-and-registries]] - [[concepts/docker-cli]] - [[syntheses/dockerfile-best-practices]] - [[concepts/what-is-docker]] ## Sources - `raw/llms_txt_doc-dockerfile.md` --- title: "Docker Engine API" type: concept tags: [engine, api, rest, sdk, versioning, socket, operations] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-engine-api.md", "raw/llms_txt_doc-daemon-cli-dockerd.md"] source_url: "https://docs.docker.com/reference/api/engine/" confidence: high --- # Docker Engine API ## Definition Docker provides an API for interacting with the Docker daemon (called the Docker Engine API), as well as SDKs for Go and Python. The SDKs let you efficiently build and scale Docker apps and solutions; if Go or Python don't work for you, you can use the Docker Engine API directly. The Docker Engine API is a RESTful API accessed by an HTTP client such as `wget` or `curl`, or the HTTP library which is part of most modern programming languages. ## How It Works The daemon exposes the API over a socket. By default a `unix` domain socket is created at `/var/run/docker.sock` (root or `docker` group required). The daemon can also listen on `tcp` and `fd` sockets; conventionally port `2375` is used for un-encrypted and `2376` for encrypted (TLS) traffic. The Docker client honors the `DOCKER_HOST` environment variable to set the `-H` flag, and can connect to a remote daemon over SSH, e.g. `docker -H ssh://me@example.com ps` (public key authentication only; password auth is not supported). When using `curl` directly, specify the version as the first part of the URL. For instance, if the endpoint is `/containers/` you can use `/v1.55/containers/`. The `docker` CLI uses the Docker API for commands like `docker run`, and the client can communicate with more than one daemon. ## Key Parameters Versioning. A given version of the SDK supports a specific version of the API plus all earlier versions; breaking changes are documented prominently. A new API version is released when new features are added, and the API is backward-compatible, so you don't need to update code unless you need new features. To see the highest and lowest version your daemon and client support, use `docker version` (reports e.g. `API version: 1.55 (minimum version 1.40)`). Selecting a version: - When using the SDK, use the latest version, or at minimum the version incorporating the API features you need. - When using `curl` directly, prefix the path with the version (`/v1.55/containers/`). - Set `DOCKER_API_VERSION` to force the CLI or SDKs to use an older API version than reported by `docker version`, e.g. `DOCKER_API_VERSION=1.54`. This works on Linux, Windows, and macOS. While set, that version is used even if the daemon supports newer, and it disables API version negotiation, so use it only when you must pin a version or for debugging. - The Go SDK can enable API version negotiation, automatically selecting a version supported by both client and Engine. You can also specify the API version programmatically as a parameter to the `client` object. Version negotiation. The API server and client support version negotiation: a client connecting to an older Engine negotiates the highest version supported by both, downgrading if necessary. When downgrading, features from later versions are disabled and requests/responses are adjusted. Compatibility is "best effort." API version matrix (selected, verbatim). Docker 29.5 -> max API 1.54 / min 1.40; 29.2 -> 1.53 / 1.44; 29.0 -> 1.52 / 1.44; 28.5 -> 1.51 / 1.24; 28.0 -> 1.48 / 1.24; 27.0 -> 1.46 / 1.24; 25.0 -> 1.44 / 1.24; 24.0 -> 1.43 / 1.12; 20.10 -> 1.41 / 1.12; 19.03 -> 1.40 / 1.12. API versions before v1.40 are deprecated and no longer supported by current Engine and CLI; archived docs for deprecated versions are in the code repository on GitHub. Daemon/client version skew. The daemon and client need not be the same version. If the daemon is newer than the client, the client doesn't know about new features or deprecated endpoints. If the client is newer than the daemon, the client can request endpoints the daemon doesn't know about. ## When To Use Use the Engine API (or the Go/Python SDKs) to build tools that create and control containers, images, and exec sessions programmatically; to script against a remote daemon over TCP/TLS or SSH; or to integrate Docker into orchestration and monitoring systems. Pin `DOCKER_API_VERSION` when a tool must target a specific Engine API contract. ## Risks & Pitfalls - Exposing the API over an un-encrypted TCP socket gives unauthenticated, direct access to the daemon; anyone reaching the port has full Docker (and effectively root) access. Secure it with the built-in HTTPS socket or a proxy. - Pinning `DOCKER_API_VERSION` disables negotiation; a pinned version that the daemon lacks will fail rather than downgrade gracefully. - Only TLS 1.0 and higher is supported on the HTTPS socket; SSLv3 and below are rejected. ## Related Concepts - [[concepts/docker-engine-and-daemon]] - [[concepts/docker-cli]] - [[concepts/containers-and-runtime]] - [[concepts/security]] ## Sources - raw/llms_txt_doc-engine-api.md - raw/llms_txt_doc-daemon-cli-dockerd.md --- title: "Images and Registries" type: concept tags: [docker, images, registries, layers, tags, foundational, well-established] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-what-is-docker.md", "raw/llms_txt_doc-docker-concepts.md", "raw/llms_txt_doc-docker-cli.md"] confidence: high --- ## Definition A Docker image is a read-only template with instructions for creating a container. Often an image is based on another image with some additional customization. A registry stores images. Docker Hub is a public registry that anyone can use, and Docker looks for images on Docker Hub by default. You can also run your own private registry. Images move between your local machine and a registry with `docker pull` (download) and `docker push` (upload). ## How It Works **Layers.** Each instruction in a [[concepts/dockerfile-reference|Dockerfile]] creates a layer in the image. When you change the Dockerfile and rebuild, only the layers that changed are rebuilt. This layer reuse is part of what makes images lightweight, small, and fast compared with other virtualization technologies. Layers are stacked; a running container adds a thin read-write layer on top of the read-only image layers. **Image references.** An image is named `repository:tag`, for example `ubuntu:24.04`. If you omit the tag, Docker assumes `latest`. Images also have a content-addressable digest (`repository@sha256:...`) that pins an exact immutable image regardless of what a tag currently points to. You can reference an image by tag or by digest wherever an image name is accepted, including the `FROM` instruction and `docker run`. **Registries and pulling.** When you use `docker pull` or `docker run`, Docker pulls the required images from your configured registry. When you use `docker push`, Docker pushes your image to your configured registry. If an image is not present locally when you `docker run`, Docker pulls it automatically as though you had run `docker pull` manually. See [[concepts/what-is-docker]] for how the client, daemon, and registry fit together. ## Key Parameters Verbatim commands for working with images and registries: ``` docker pull ubuntu docker push /: docker images docker tag SOURCE_IMAGE[:TAG] TARGET_IMAGE[:TAG] docker rmi docker login docker logout docker search ``` - `docker pull IMAGE[:TAG|@DIGEST]` downloads an image from a registry. Docker Hub is the default source. - `docker images` (alias of `docker image ls`) lists local images. Its default output format can be customized with the `imagesFormat` property in `~/.docker/config.json`, for example `"table {{.ID}}\t{{.Repository}}\t{{.Tag}}\t{{.CreatedAt}}"`. - `docker tag` creates an additional name that points at an existing image; it is how you rename an image for a target registry before pushing. - `docker push` uploads an image to the registry named in the tag; you must be authenticated with `docker login` for private repositories. - `docker rmi` removes a local image; `docker image prune` reclaims space from dangling images. - `docker search` searches Docker Hub for images from the CLI. Credentials are handled by `docker login`, which stores tokens in the `auths` property of `~/.docker/config.json` or, when `credsStore` is set, in an external `docker-credential-` binary on `$PATH`. Per-registry helpers can be set with `credHelpers`. The config directory may contain sensitive authentication information, so review it before sharing and do not commit it to version control. ## When To Use - **Consume published images.** Pull base images and off-the-shelf application images from Docker Hub or a private registry rather than building from scratch. - **Distribute your own builds.** After [[concepts/building-images|building an image]], tag it for your registry and `docker push` it so colleagues, CI, and production can pull the identical artifact. - **Pin for reproducibility.** Reference images by digest (`@sha256:...`) in `FROM` and deployment manifests when you need byte-for-byte reproducibility instead of a moving tag. - **Promote across environments.** The image is the unit that moves from development through test to production unchanged; pushing the updated image is how a fix reaches customers. ## Risks & Pitfalls - **`latest` is not "newest" - it is just the default tag.** It moves whenever someone pushes to it, so builds and deployments that rely on `latest` are not reproducible. Prefer explicit tags or digests. - **Tags are mutable; digests are not.** A tag can be repointed at a different image at any time. Use a digest when immutability matters. - **Pushing requires authentication and a fully qualified name.** To push to a non-Docker-Hub registry, tag the image with the registry host (`registry.example.com/team/app:1.0`) first. - **Local image store grows.** Unused images and layers accumulate. Reclaim space with `docker image prune` or `docker system prune`. - **Credential exposure.** `~/.docker/config.json` can hold registry credentials. Guard it and prefer a credential store. See [[concepts/security]]. ## Related Concepts - [[concepts/what-is-docker]] - [[concepts/dockerfile-reference]] - [[concepts/building-images]] - [[concepts/docker-cli]] - [[concepts/containers-and-runtime]] ## Sources - `raw/llms_txt_doc-what-is-docker.md` - `raw/llms_txt_doc-docker-concepts.md` - `raw/llms_txt_doc-docker-cli.md` --- title: "Installing Docker" type: concept tags: [installation, get-docker, docker-engine, docker-desktop, linux, setup] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-get-docker.md", "raw/llms_txt_doc-get-started.md", "raw/llms_txt_doc-docker-engine.md", "raw/llms_txt_doc-what-is-docker.md"] source_url: "https://docs.docker.com/get-started/get-docker/" confidence: medium --- # Installing Docker ## Definition Docker is an open platform for developing, shipping, and running applications. You can download and install Docker on multiple platforms. There are two installation paths: Docker Desktop (a native application delivering all Docker tools for Mac, Windows, or Linux) and Docker Engine (the open source containerization engine installed directly on a Linux distribution). Choose the best installation path for your setup. ## How It Works Docker Desktop is an easy-to-install application for Mac, Windows, or Linux that includes the Docker daemon (`dockerd`), the Docker client (`docker`), Docker Compose, Docker Content Trust, Kubernetes, and Credential Helper: - Docker Desktop for Mac -- a native application using the macOS sandbox security model that delivers all Docker tools to your Mac. - Docker Desktop for Windows -- a native Windows application that delivers all Docker tools to your Windows computer. It can also be installed from the Microsoft Store. - Docker Desktop for Linux -- a native Linux application that delivers all Docker tools to your Linux computer. Docker Engine is the open source engine you install for your distribution when you don't want the full Desktop application (for example, on a headless Linux server). Docker Engine can be installed on the following platforms (per-distribution guides): CentOS, Debian, Fedora, Raspberry Pi OS (32-bit / armhf), RHEL, and Ubuntu. Docker Engine can also be installed from binaries. If you are looking for information on how to install Docker Engine rather than Docker Desktop, see the Docker Engine installation overview. ## Key Parameters Post-install and configuration: - The Docker CLI defaults to connecting to the daemon at `unix:///var/run/docker.sock` on Linux, and `tcp://127.0.0.1:2376` on Windows. - Depending on your Docker system configuration, you may be required to preface each `docker` command with `sudo`. To avoid using `sudo`, a system administrator can create a Unix group called `docker` and add users to it. (Note: adding a user to the `docker` group grants root-equivalent access to the host.) - On Linux, the daemon configuration file lives at `/etc/docker/daemon.json`; on Windows at `%programdata%\docker\config\daemon.json`. If the file does not exist you must create it. - Persistent Docker state is stored under `/var/lib/docker` (`--data-root`). Verify the installation. After installing, confirm the client and daemon are talking and check versions: ``` $ docker version Client: Docker Engine - Community Version: 29.6.1 API version: 1.55 ... Server: Docker Engine - Community Engine: Version: 29.6.1 API version: 1.55 (minimum version 1.40) ``` Run a first container to confirm end to end, e.g. `docker run -d -p 8080:80 docker/welcome-to-docker` (then visit `http://localhost:8080`), or interactively `docker run -i -t ubuntu /bin/bash`. List running containers with `docker ps` (add `-a` to list all, including stopped). ## When To Use Install Docker Desktop for local development on Mac, Windows, or Linux when you want the full toolset (Compose, Kubernetes, GUI dashboard) in one package. Install Docker Engine directly on a Linux distribution for servers, CI runners, and production hosts where you want only the engine and CLI. See [[concepts/docker-engine-and-daemon]] for configuring the daemon after installation. ## Risks & Pitfalls - Commercial use of Docker Desktop in larger enterprises (more than 250 employees OR more than $10 million USD in annual revenue) requires a paid subscription. The same threshold applies to commercial use of Docker Engine obtained via Docker Desktop. - Adding users to the `docker` group is equivalent to granting them root on the host; treat group membership as a privileged grant. - Per-distribution package steps differ by OS; follow the specific installation guide for CentOS, Debian, Fedora, Raspberry Pi OS, RHEL, or Ubuntu rather than assuming one command set works everywhere. ## Related Concepts - [[concepts/what-is-docker]] - [[concepts/docker-engine-and-daemon]] - [[concepts/docker-cli]] - [[concepts/docker-compose]] ## Sources - raw/llms_txt_doc-get-docker.md - raw/llms_txt_doc-get-started.md - raw/llms_txt_doc-docker-engine.md - raw/llms_txt_doc-what-is-docker.md --- title: "Container Networking" type: concept tags: [networking, bridge, overlay, ports, dns, drivers] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-concepts.md", "raw/llms_txt_doc-what-is-docker.md", "raw/llms_txt_doc-llms-full-txt.md", "raw/llms_txt_doc-docker-cli.md"] source_url: "https://docs.docker.com/engine/network/" confidence: medium --- ## Definition Docker networking manages the network connections between containers, and between containers and the outside world. The Docker daemon (`dockerd`) creates and manages network objects alongside images, containers, and volumes. When you run a container without specifying networking options, Docker creates a network interface to connect the container to the default network, assigns it an IP address, and - by default - lets the container reach external networks using the host machine's network connection. You can control how isolated a container's network is from other containers and from the host. ## Network drivers Docker's built-in network drivers each serve a different topology (source descriptions from the Docker Engine network docs): - **bridge** - user-defined bridge networks and the default bridge. The default driver for standalone containers on a single host; containers on the same user-defined bridge can reach each other. - **host** - exposes containers on the Docker host's network directly, with no network isolation between the container and the host. - **overlay** - connects containers across multiple Docker daemon hosts, used for Swarm services and multi-host communication. - **ipvlan** - makes your containers appear like physical machines on the network, giving full control over IPv4/IPv6 addressing. - **macvlan** - makes your containers appear like physical machines on the network by assigning a MAC address to each container. - **none** - isolates the networking stack of a container; the container gets no external network interface. Network driver plugins can extend this set with third-party drivers. ## `docker network` commands The `docker network` management command administers network objects: - `docker network create` - create a network (choose the driver with `--driver`) - `docker network ls` - list networks - `docker network inspect` - show low-level details of one or more networks - `docker network connect` - connect a running container to a network - `docker network disconnect` - disconnect a container from a network - `docker network rm` - remove one or more networks - `docker network prune` - remove all unused networks ## Port publishing By default a container's ports are reachable only from within its network. To make a container port accessible from the host, publish it with `-p` on `docker run` (or `ports:` in Compose). For example: ``` $ docker run -d -p 8080:80 docker/welcome-to-docker ``` maps host port `8080` to container port `80`. `docker ps` then shows the mapping in the `PORTS` column: ``` PORTS 0.0.0.0:8080->80/tcp ``` The frontend is then accessible on the host at `http://localhost:8080`. Publishing a port is what lets you connect through the isolated environment of the container. ## DNS and service discovery Containers connect to one or more networks and are assigned IP addresses by the daemon. On user-defined networks, containers can address each other, which is the basis of service-to-service communication. In Docker Compose, Compose sets up networking between the services in a project automatically so that services can reach one another; see "Networking in Compose" in the Docker docs. ## Container-to-container communication Because a container can be connected to one or more networks, multi-container applications typically place their components (for example a frontend, backend, and database) on a shared user-defined network so they can communicate while remaining isolated from unrelated containers. This is what Compose automates for a project, and what `docker network create` + `docker network connect` do manually. ## Related Concepts - [[concepts/containers-and-runtime]] - the `-p` / `--network` run flags - [[concepts/docker-compose]] - Compose sets up per-project networking automatically - [[concepts/compose-file-reference]] - the `networks` top-level element - [[concepts/docker-engine-and-daemon]] - the daemon that manages network objects - [[concepts/storage-and-volumes]] - the storage counterpart to networking ## Sources - raw/llms_txt_doc-docker-concepts.md - raw/llms_txt_doc-what-is-docker.md - raw/llms_txt_doc-llms-full-txt.md - raw/llms_txt_doc-docker-cli.md --- title: "Docker Container Security" type: concept tags: [security, namespaces, cgroups, rootless, seccomp, secrets, isolation, hardening] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-security.md", "raw/llms_txt_doc-daemon-cli-dockerd.md", "raw/llms_txt_doc-what-is-docker.md"] source_url: "https://docs.docker.com/security/" confidence: high --- # Docker Container Security ## Definition Docker helps you protect your local environments, infrastructure, and networks with developer-level security features. Security spans two layers: kernel-level isolation of the containers themselves (namespaces, cgroups, capabilities, seccomp, user namespaces) enforced by the Engine, and workflow/supply-chain controls (access management, vulnerability scanning, secrets, image trust) used during development and delivery. ## How It Works Kernel isolation. Docker takes advantage of several Linux kernel features. It uses namespaces to provide the isolated workspace called the container: when you run a container, Docker creates a set of namespaces for it, and each aspect of a container runs in a separate namespace with access limited to that namespace. The daemon relies on an OCI-compliant runtime (via `containerd`) as its interface to the Linux kernel namespaces, cgroups, and SELinux. By default a container is relatively well isolated from other containers and its host; you control how isolated its network, storage, and other subsystems are. User namespaces. Linux kernel user namespace support gives a process (and therefore a container) a unique range of user and group IDs outside the host's traditional range. The key improvement: by default, container processes running as `root` have the expected administrative privileges (with some restrictions) inside the container, but are effectively mapped to an unprivileged `uid` on the host. Enable via the daemon's `--userns-remap` flag or `"userns-remap"` key. Rootless mode. Run Docker without root privileges using `dockerd --rootless` (typically used with RootlessKit), or the `"rootless"` configuration. This reduces the impact of a daemon or runtime compromise. ## Key Parameters Daemon-level hardening (verbatim flags / `daemon.json` keys): - `--seccomp-profile` / `"seccomp-profile"` -- path to a seccomp profile; set to `"unconfined"` to disable the default seccomp profile (default `"builtin"`). - `--no-new-privileges` / `"no-new-privileges"` -- set no-new-privileges by default for new containers. - `--selinux-enabled` / `"selinux-enabled"` -- enable SELinux support. - `--userns-remap` / `"userns-remap"` -- user/group setting for user namespaces. - `--authorization-plugin` / `"authorization-plugins"` -- load authorization plugins; once installed, requests through the CLI or Engine API are allowed or denied by the plugin, and with multiple plugins each must allow a request in order for it to complete. - `--icc` -- inter-container communication for the default bridge (default true). Registry trust. A secure registry uses TLS with a copy of its CA certificate placed at `/etc/docker/certs.d/myregistry:5000/ca.crt`. Marking a registry insecure (`--insecure-registry`) allows plain-HTTP or untrusted-TLS communication; because it creates security vulnerabilities it should only be enabled for testing. Local registries in the `127.0.0.0/8` range are automatically treated as insecure. Developer and supply-chain features (from the Security manual): - Two-factor authentication (2FA) and personal access tokens as an alternative to your password, to manage account access. - Docker Scout for static vulnerability scanning: automatically run a point-in-time scan on images for vulnerabilities. - Secrets in Docker Compose to integrate secrets securely into your development stack. - Suppress CVEs with VEX (suppress non-applicable or fixed vulnerabilities found in images). - Docker Hardened Images to enhance software supply chain security (distroless variants minimize attack surface and remove unnecessary components). - Docker Content Trust (bundled in Docker Desktop) for image signing. ## When To Use Apply kernel isolation controls (seccomp, user namespaces, no-new-privileges, rootless) when hardening the runtime surface of a host, especially for multi-tenant or untrusted workloads. Use scanning (Docker Scout), signing (Content Trust), VEX, and Hardened Images to secure the build-and-ship pipeline. Use authorization plugins to gate daemon access in regulated environments. ## Risks & Pitfalls - Disabling the default seccomp profile (`"unconfined"`) widens the syscall attack surface. - Binding the daemon to a TCP port or the `docker` group can allow non-root users to gain root on the host; access to the socket is equivalent to root. - `--insecure-registry` disables TLS verification; prefer adding the CA to the system trust store. - Container `root` without user-namespace remapping maps to host `root`; enable `userns-remap` or rootless mode to break that mapping. ## Related Concepts - [[concepts/docker-engine-and-daemon]] - [[concepts/engine-api]] - [[concepts/images-and-registries]] - [[concepts/containers-and-runtime]] - [[syntheses/dockerfile-best-practices]] ## Sources - raw/llms_txt_doc-security.md - raw/llms_txt_doc-daemon-cli-dockerd.md - raw/llms_txt_doc-what-is-docker.md --- title: "Storage and Volumes" type: concept tags: [storage, volumes, bind-mounts, tmpfs, persistence] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-concepts.md", "raw/llms_txt_doc-what-is-docker.md", "raw/llms_txt_doc-llms-full-txt.md", "raw/llms_txt_doc-docker-cli.md"] source_url: "https://docs.docker.com/engine/storage/" confidence: medium --- ## Definition By default, a container's writable data lives in the read-write filesystem layer that Docker allocates as the container's final layer when it runs. This layer is tied to the container's lifecycle: when a container is removed, any changes to its state that aren't stored in persistent storage disappear. To persist data beyond the life of a single container - or to share data with the host or between containers - Docker provides mounts. The Docker daemon manages volumes as first-class objects alongside images, containers, and networks. ## Mount types Docker offers three ways to mount data into a container (from the Docker Engine storage docs): - **Volumes** - the preferred mechanism for persisting data generated and used by Docker. Docker creates and manages the storage location; you create, manage, and use volumes instead of bind mounts. Volumes are stored in a part of the host filesystem managed by Docker and are the recommended way to persist data. - **Bind mounts** - mount a file or directory from the host filesystem directly into the container. The path is host-controlled, so a bind mount depends on the host's directory structure. - **tmpfs mounts** - store data in the host's memory only; the data is never written to the host filesystem and does not persist after the container stops. Useful for temporary or sensitive data. ## `-v` and `--mount` syntax Attach storage at run time with `-v` (or the more explicit `--mount`). The `-v` flag can be specified multiple times and takes a `HOST:CONTAINER` value: ``` $ docker run -v /host:/container example/mysql ``` Using a named volume in place of a host path attaches a Docker-managed volume. Anonymous volumes can also be requested in a Compose service with a single in-container path (for example `- "/data"`). In a Dockerfile, the `VOLUME` instruction declares a mount point (for example `VOLUME /var/log` or `VOLUME ["/var/log/"]`) so that data written there is kept outside the image's writable layer. ## `docker volume` commands The `docker volume` management command administers volume objects: - `docker volume create` - create a volume - `docker volume ls` - list volumes - `docker volume inspect` - display detailed information on one or more volumes - `docker volume rm` - remove one or more volumes - `docker volume prune` - remove all unused local volumes - `docker volume update` - update a volume (cluster volumes) ## Persistence model Because the container's writable layer is discarded on removal, stateful workloads (databases, uploads, caches you want to survive restarts) must write to a volume or bind mount. Volumes are the recommended choice because they are managed by Docker, are decoupled from the host's directory layout, and can be shared across containers and defined declaratively in Compose (`volumes:` service key and the top-level `volumes` element). Bind mounts are typically used for development, when you want a container to see live host files (for example mounting source code into a container). tmpfs mounts are used when data should never touch disk. ## Storage drivers vs. volumes Volumes, bind mounts, and tmpfs mounts control how data is mounted into containers. Separately, the storage driver (for example OverlayFS) governs how the image and container layers themselves are stored on the host. Persisting application data should always use a mount rather than relying on the writable layer, regardless of storage driver. ## Related Concepts - [[concepts/containers-and-runtime]] - the writable layer and the `-v` run flag - [[concepts/compose-file-reference]] - the `volumes` top-level element and service key - [[concepts/dockerfile-reference]] - the `VOLUME` instruction - [[concepts/docker-engine-and-daemon]] - the daemon that manages volume objects - [[syntheses/volumes-vs-bind-mounts]] - choosing between volumes, bind mounts, and tmpfs ## Sources - raw/llms_txt_doc-docker-concepts.md - raw/llms_txt_doc-what-is-docker.md - raw/llms_txt_doc-llms-full-txt.md - raw/llms_txt_doc-docker-cli.md --- title: "What is Docker?" type: concept tags: [docker, architecture, containers, foundational, well-established] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-what-is-docker.md", "raw/llms_txt_doc-introduction.md", "raw/llms_txt_doc-docker-concepts.md"] confidence: high --- ## Definition Docker is an open platform for developing, shipping, and running applications. It lets you separate your applications from your infrastructure so you can deliver software quickly. Docker packages and runs an application in a loosely isolated environment called a container. Containers are lightweight and contain everything needed to run the application, so you do not need to rely on what is installed on the host. You can share a container and be sure that everyone you share with gets the same container that works the same way. ## How It Works Docker uses a client-server architecture. The Docker client talks to the Docker daemon, which does the heavy lifting of building, running, and distributing your containers. The client and daemon can run on the same system, or you can connect a client to a remote daemon. They communicate using a REST API, over UNIX sockets or a network interface. - **The Docker daemon** (`dockerd`) listens for Docker API requests and manages Docker objects such as images, containers, networks, and volumes. A daemon can also communicate with other daemons to manage services. See [[concepts/docker-engine-and-daemon]]. - **The Docker client** (`docker`) is the primary way many users interact with Docker. When you run a command such as `docker run`, the client sends it to `dockerd`, which carries it out. The `docker` command uses the Docker API. See [[concepts/docker-cli]]. - **Docker Compose** is another client that lets you work with applications consisting of a set of containers. See [[concepts/docker-compose]]. - **Docker registries** store images. Docker Hub is a public registry, and Docker looks for images there by default. `docker pull` or `docker run` pulls required images from your configured registry; `docker push` pushes an image to it. See [[concepts/images-and-registries]]. Docker is written in Go and takes advantage of Linux kernel features. It uses a technology called namespaces to provide the isolated workspace called the container: when you run a container, Docker creates a set of namespaces that limit each aspect of the container to its own namespace, providing a layer of isolation. **Images vs. containers.** An image is a read-only template with instructions for creating a container. Often an image is based on another image with some additional customization (for example, an Ubuntu base image plus the Apache web server and your app). Each instruction in a [[concepts/dockerfile-reference|Dockerfile]] creates a layer in the image; when you change the Dockerfile and rebuild, only the changed layers are rebuilt. A container is a runnable instance of an image. You can create, start, stop, move, or delete a container using the API or CLI. See [[concepts/containers-and-runtime]]. **Containers vs. virtual machines.** A VM is an entire operating system with its own kernel, hardware drivers, programs, and applications, so spinning one up just to isolate a single application is a lot of overhead. A container is simply an isolated process with all the files it needs to run. If you run multiple containers, they all share the same host kernel, letting you run more applications on less infrastructure. Containers and VMs are often used together: a VM with a container runtime can run multiple containerized applications, increasing resource utilization. ## Key Parameters Containers are described by four properties: - **Self-contained** - each container has everything it needs to function, with no reliance on pre-installed dependencies on the host. - **Isolated** - containers run in isolation, with minimal influence on the host and other containers, increasing security. - **Independent** - each container is independently managed; deleting one container does not affect any others. - **Portable** - a container that runs on your development machine works the same way in a data center or the cloud. The Docker platform manages the container lifecycle: develop your application and its components using containers, use the container as the unit for distributing and testing, then deploy into production as a container or an orchestrated service, whether that environment is a local data center, a cloud provider, or a hybrid. ## When To Use - **Fast, consistent delivery** - standardized environments using local containers, well suited to CI/CD workflows. - **Responsive deployment and scaling** - highly portable workloads that run on a laptop, physical or virtual machines, or the cloud, and scale up or tear down in near real time. - **Running more workloads on the same hardware** - a lightweight, cost-effective alternative to hypervisor-based VMs, good for high-density environments and small-to-medium deployments. A minimal first run: ``` $ docker run -i -t ubuntu /bin/bash ``` If the `ubuntu` image is not local, Docker pulls it (as if you ran `docker pull ubuntu`), creates a container, allocates a read-write filesystem as its final layer, attaches a network interface, and starts the container running `/bin/bash` interactively (`-i` keeps STDIN open, `-t` allocates a terminal). When you `exit`, the container stops but is not removed. ## Risks & Pitfalls - **Docker Desktop is out of scope here.** Docker Desktop is a separate install for Mac, Windows, or Linux that bundles the daemon, client, Compose, Kubernetes, and helpers. This KB covers the open-source Engine, CLI, Compose, and Build, not Desktop, Scout, or Hub features. - **Ephemeral state.** When a container is removed, any changes to its state that are not stored in persistent storage disappear. Use volumes for data you must keep. See [[concepts/storage-and-volumes]]. - **Isolation is not a security boundary by default.** Containers share the host kernel; control how isolated a container's network, storage, and subsystems are. See [[concepts/security]]. - **`sudo` requirement.** Depending on configuration, you may need to preface each `docker` command with `sudo`, or have an administrator add your user to the `docker` group. ## Related Concepts - [[concepts/containers-and-runtime]] - [[concepts/images-and-registries]] - [[concepts/docker-cli]] - [[concepts/dockerfile-reference]] - [[concepts/docker-engine-and-daemon]] - [[concepts/installation]] ## Sources - `raw/llms_txt_doc-what-is-docker.md` - `raw/llms_txt_doc-introduction.md` - `raw/llms_txt_doc-docker-concepts.md` --- title: "Activity Log" type: log --- # Activity Log Append-only record of all wiki changes. ## Format Each entry follows this format: ``` ### YYYY-MM-DD HH:MM — [Action Type] - **Source/Trigger**: what initiated the action - **Pages created**: list of new pages - **Pages updated**: list of updated pages - **Notes**: any contradictions flagged, decisions made ``` --- ### 2026-04-08 00:00 — Setup - **Source/Trigger**: Repository initialized - **Pages created**: index.md, log.md, dashboard.md, analytics.md, flashcards.md - **Pages updated**: none - **Notes**: Empty knowledge base ready for first source ingestion --- title: "Dockerfile Best Practices" type: synthesis tags: [dockerfile, build, multi-stage, cache, dockerignore, image-size, best-practices] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-dockerfile.md", "raw/llms_txt_doc-docker-build.md", "raw/llms_txt_doc-what-is-docker.md"] confidence: high --- # Dockerfile Best Practices A decision guide for writing Dockerfiles that build fast, stay small, and ship securely. Whenever you create an image you are using Docker Build; Build is a whole ecosystem of tools, not just a command for packaging code. ## Comparison | Technique | What it does | Primary payoff | Cost / trade-off | |-----------|--------------|----------------|------------------| | Layer ordering + cache | Each Dockerfile instruction creates a layer; only changed layers are rebuilt | Fast incremental rebuilds | Mis-ordering invalidates cache early | | Multi-stage builds | `FROM ... AS build` stage compiles; final stage `COPY --from=build` copies only artifacts | Small, secure final image with minimal dependencies | More complex Dockerfile | | `.dockerignore` | Excludes files and directories from the build context | Smaller context, faster uploads, avoids leaking secrets | Must be maintained as the repo grows | | Cache mounts (`RUN --mount=type=cache`) | Caches compiler/package-manager directories across builds | Skips repeated costly installs | BuildKit-only syntax | | Secret mounts (`RUN --mount=type=secret`) | Exposes a secret to one `RUN` only | Keeps credentials out of image layers | Requires passing secrets at build time | | Non-root `USER` | Runs the container process as an unprivileged user | Reduced runtime attack surface | May need file-permission adjustments | ## Analysis Cache is driven by layer ordering. Because each instruction creates a layer and rebuilds touch only changed layers plus everything after them, the ordering rule is: put the least-frequently-changing instructions first. Install dependencies before copying application source, so a code edit does not invalidate the dependency-install layer. Copy the dependency manifest (e.g. `package.json`, `requirements.txt`, `go.mod`) and install, then copy the rest of the source. Multi-stage builds keep your images small and secure with minimal dependencies. Compilers, build tools, and dev headers live only in a build stage; the final runtime stage starts from a slim base and pulls in just the built artifact via `COPY --from=`. This removes the entire build toolchain from the shipped image, cutting both size and attack surface. Build caching avoids unnecessary repetitions of costly operations such as package installs. Beyond layer cache, `RUN --mount=type=cache` lets the build container cache directories for compilers and package managers across builds; the `id` option identifies separate caches and defaults to the value of `target` (the mount path). Written data on a `readwrite` mount is discarded after the `RUN` completes and is not committed to the image layer. The `.dockerignore` file excludes files and directories from the build context, which shrinks the context Docker sends to the daemon and prevents accidentally copying secrets, `.git`, and local build output into the image. Secrets belong in mounts, not layers. Use `RUN --mount=type=secret` to expose a credential to a single `RUN` instruction so it never lands in an image layer; never `COPY` secrets or bake them into `ENV`. ## Recommendations - Order instructions from least- to most-frequently changing; copy dependency manifests and install before copying full source, to preserve the cache. - Use multi-stage builds for any compiled or bundled application: build in a fat stage, `COPY --from=build` only the artifact into a slim final stage. - Add a `.dockerignore` early; exclude `.git`, local artifacts, and secrets from the build context. - Use `RUN --mount=type=cache` for package managers/compilers to avoid re-downloading and re-installing on every build. - Use `RUN --mount=type=secret` for build-time credentials; keep them out of layers, `ENV`, and `ARG`. - Prefer minimal base images (distroless/Hardened Images) and set a non-root `USER` in the final stage to reduce runtime attack surface. See [[concepts/security]]. ## Pages Compared - [[concepts/dockerfile-reference]] - [[concepts/building-images]] - [[concepts/images-and-registries]] - [[concepts/security]] --- title: "docker run vs Docker Compose" type: synthesis tags: [docker-run, compose, cli, orchestration, multi-container, workflow] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-cli.md", "raw/llms_txt_doc-docker-compose.md", "raw/llms_txt_doc-compose-cli.md", "raw/llms_txt_doc-what-is-docker.md"] confidence: high --- # docker run vs Docker Compose Both start containers, but they operate at different levels. `docker run` is an imperative command for a single container; Docker Compose is a declarative tool for defining and running multi-container applications from a YAML file. This page helps you choose. ## Comparison | Dimension | `docker run` | Docker Compose | |-----------|--------------|----------------| | Model | Imperative -- one command, one container | Declarative -- a YAML file describes the whole stack | | Scope | A single container from an image | Multiple services, networks, and volumes together | | Definition | Flags on the command line (`-d`, `-p`, `-v`, ...) | `compose.yaml` (`services:`, `networks:`, `volumes:`) | | Startup | One container per invocation | One command creates and starts all services | | State of config | Lives in shell history / scripts | Version-controlled config file | | Client | `docker` CLI talking to `dockerd` | `docker compose` (another client on top of the API) | ## Analysis `docker run` is the imperative primitive. `docker run [OPTIONS] IMAGE [COMMAND] [ARG...]` creates and runs a new container from an image; the `docker` client sends the command to `dockerd`, which carries it out. Example: `docker run -d -p 8080:80 docker/welcome-to-docker` starts a detached container publishing port 8080, or `docker run -i -t ubuntu /bin/bash` runs an interactive shell. Everything about the container -- ports, mounts, environment, network -- is expressed as flags on that single command. This is perfect for one-off containers and quick experiments, but reproducing a multi-container setup means remembering and re-running several long commands in the right order. Docker Compose is the declarative layer. Compose is a tool for defining and running multi-container applications. It simplifies control of your entire application stack, making it easy to manage services, networks, and volumes in a single YAML configuration file. Then, with a single command, you create and start all the services from your configuration file. Compose also has commands for managing the whole lifecycle: start, stop, and rebuild services; view the status of running services; stream the log output of running services; and run a one-off command on a service. Another Docker client is Docker Compose, which lets you work with applications consisting of a set of containers. Compose scales the configuration cleanly. You supply the config with `-f`, e.g. `docker compose -f compose.yaml -f compose.admin.yaml run backup_db`; when you supply multiple files Compose combines them in order, with subsequent files overriding and adding to their predecessors. This lets you layer environment-specific overrides on a shared base -- something `docker run` has no equivalent for. Compose works in all environments: production, staging, development, testing, and CI workflows. ## Recommendations - Running a single container, a quick test, or a one-off task -> use `docker run` with the flags you need. - Any application with more than one container (for example a frontend, an API, and a database) -> define it in a `compose.yaml` and bring it up with one Compose command. - Configuration you want version-controlled, repeatable, and shareable across a team or CI -> use Compose; the YAML file is the source of truth, not shell history. - Environment-specific variation (dev vs prod) -> keep a base Compose file and layer overrides with multiple `-f` files rather than branching `docker run` scripts. - Prototyping that later becomes a stack -> start with `docker run`, then graduate to Compose once a second service appears. ## Pages Compared - [[concepts/docker-cli]] - [[concepts/docker-compose]] - [[concepts/compose-file-reference]] - [[concepts/containers-and-runtime]] --- title: "Volumes vs Bind Mounts vs tmpfs" type: synthesis tags: [storage, volumes, bind-mounts, tmpfs, persistence, data] created: 2026-07-07 updated: 2026-07-07 sources: ["raw/llms_txt_doc-docker-concepts.md", "raw/llms_txt_doc-llms-full-txt.md", "raw/llms_txt_doc-what-is-docker.md"] confidence: medium --- # Volumes vs Bind Mounts vs tmpfs A container's writable layer is ephemeral: when a container is removed, any changes to its state that aren't stored in persistent storage disappear. Docker offers three mount types for getting data into and out of containers -- volumes, bind mounts, and tmpfs mounts. This page is a decision guide for picking the right one. ## Comparison | Dimension | Volumes | Bind mounts | tmpfs mounts | |-----------|---------|-------------|--------------| | What it is | Storage managed by Docker, stored under Docker's data area (`/var/lib/docker`) | A file or directory on the host filesystem mounted into the container | An in-memory mount that never touches the host disk | | Managed by | Docker (`docker volume` lifecycle) | You (any host path) | The container runtime (memory) | | Persistence | Persists after the container is removed | Persists (it is your host path) | Cleared when the container stops; nothing is written to disk | | Portability | Portable across hosts and Compose projects; host-path-independent | Tied to a specific host directory layout | Not persisted, so not portable | | Best for | Databases and application data Docker should own | Live source code / config during development | Secrets and scratch data that must never persist | ## Analysis Volumes are the recommended default for persistence. The official guidance is to use volumes instead of bind mounts for persisting data generated and used by Docker. Because Docker manages a volume's storage location (under its data root, default `/var/lib/docker`), you don't hard-code host paths, the volume survives container removal, and it moves cleanly between environments and Compose projects. This is why the standard pattern for "persist the DB" is a named volume rather than a host directory. Bind mounts share local files with containers. A bind mount maps an exact file or directory on the host into the container, so changes are visible in both directions immediately. That is ideal for the inner development loop -- editing source on the host and having it reflected live inside a running container -- but it couples the container to the host's directory structure and permissions, which reduces portability. tmpfs mounts keep data off disk entirely. A tmpfs mount lives in memory and is never written to the host filesystem, so its contents vanish when the container stops. Use it for sensitive values you don't want persisted, or for high-churn scratch data where avoiding disk I/O is desirable. Storage type is distinct from the storage driver. The mount type (volume/bind/tmpfs) controls where a specific piece of data lives; the daemon's storage driver (`overlay2` by default on Linux, `windowsfilter` on Windows) controls how the container's own image and writable layers are stored. See [[concepts/docker-engine-and-daemon]]. ## Recommendations - Persisting application/database data that Docker should own -> use a volume. Prefer named volumes; they persist beyond the container and are portable. - Developing with live code or injecting host config files -> use a bind mount for the immediate host-to-container reflection. - Handling secrets or throwaway scratch data that must never hit disk -> use a tmpfs mount. - In Compose, declare a named volume (top-level `volumes:` plus a service-level mount) rather than a host path when you want portability across machines. See [[concepts/docker-compose]]. - Avoid bind mounts in production where host layout differs from development; the coupling to a specific host path is a portability and reproducibility risk. ## Pages Compared - [[concepts/storage-and-volumes]] - [[concepts/containers-and-runtime]] - [[concepts/docker-engine-and-daemon]] - [[concepts/docker-compose]]