containerd storage recovery

containerd snapshot metadata can lose the map to writable layers even when snapshot directories still contain application files.

containerd separates persistent content, metadata, plugin data, and snapshots from ephemeral runtime state. Damage to Bolt metadata, an overlay snapshot database, the backing filesystem, or cleanup state can make images and containers disappear while portions of their data remain on disk.

Understand the symptom

Snapshot directories and metadata databases must agree on identity and parent relationships.

WHY

What is happening

containerd documents persistent snapshots, content, and plugin metadata under its root and warns that these directories are implementation details that should not be manipulated externally. Overlay snapshotters maintain their own metadata database beside snapshot content.

What you may see

Symptoms that change the recovery path.

01

Snapshots not found

Metadata no longer resolves a key or numeric snapshot directory.

02

Failed to get snapshot info

The snapshotter database is unavailable or inconsistent.

03

Containers return with older files

The expected active upper snapshot may not be attached.

04

Cleanup reports busy or stale handles

Runtime mounts and persistent state may disagree.

Protect the source

Actions that can make the situation worse.

01

Garbage collection can remove orphaned data

Unreferenced-looking blobs or snapshots may contain the only recent files.

02

Manual renames break numeric mappings

Directory names alone do not reconstruct metadata relationships.

03

Restarting workloads creates new layers

New writes can reuse space and blur the original state.

Do this now

Safe steps before evaluation.

01

Stop containerd and orchestrators

Prevent starts, pulls, commits, cleanup, and rescheduling writes.

02

Capture both storage roots

Keep persistent root and relevant runtime state with ownership and xattrs.

03

Collect namespace and workload data

Preserve CRI, Kubernetes, image digests, pod specs, volumes, and logs.

04

Identify writable-layer data

Separate bind mounts and volumes from files stored only inside snapshots.

NEPFOSYS process

How a data-first evaluation proceeds.

01

Snapshotter inventory

NEPFOSYS maps namespaces, metadata databases, content digests, snapshot IDs, parent chains, and host health.

02

Copy-based layer extraction

Readable upper layers, content blobs, and persistent volumes are collected without starting workloads.

03

Application validation

Priority databases, uploads, configuration, ownership, timestamps, and expected paths are reviewed.

A clear next step

Preserve the original device before another repair or recovery attempt.

Start a case with the exact symptoms, incident timeline, prior attempts, encryption details, and the files that matter most. NEPFOSYS will route the case to the appropriate DFW intake or nationwide mail-in path.

Start with a case review

Questions, answered

What customers ask before starting.

Is containerd data the same as Docker overlay2?

They can use related overlay concepts, but containerd has its own namespaces, metadata, content store, and snapshotter paths.

Can I rebuild meta.db from directory names?

Not reliably without additional metadata and content relationships.

Should Kubernetes be restarted first?

No. Preserve node and storage state before the orchestrator creates replacement workloads.

DFW local · Nationwide mail-in

Protect the device. Start with the facts.

Start my recovery case