Elasticsearch shard recovery

A corrupt Elasticsearch translog can block shard recovery even when most Lucene segments remain readable.

Elasticsearch uses Lucene segments for indexed data and a translog to recover acknowledged operations not yet safely committed to a segment. A damaged translog, shard store or cluster-state decision can leave a primary unassigned or repeatedly failing.

Understand the symptom

Shard recovery chooses a history using allocation IDs, sequence numbers, primary terms and retained operations.

WHY

What is happening

Recovery must distinguish translog-only damage from corrupt Lucene segments, filesystem faults and an incomplete replica. Tools that remove corrupt translog entries may allow a shard to open, but acknowledged operations can be lost; a good replica or snapshot may provide a safer source.

What you may see

Symptoms that change the recovery path.

01

Translog corruption exception

Checksum, truncation or generation errors stop shard initialization.

02

Primary shard remains unassigned

Allocation retries fail on each candidate copy.

03

Shard loops between initializing and failed

Recovery repeatedly reaches the same damaged store or history conflict.

04

Node loss preceded the error

Not every shard copy or latest operation history may still be present.

Protect the source

Actions that can make the situation worse.

01

Translog truncation can lose acknowledged writes

Removing corrupt operations creates a new endpoint that must be documented and validated.

02

Force allocation can discard a better copy

Allocating a stale primary may define an older history as authoritative.

03

Deleting shard data is irreversible

The store may contain unique segments even when Elasticsearch will not open it.

Do this now

Safe steps before evaluation.

01

Stop automated allocation changes

Pause repeated reroutes, deletions, translog tools and node cleanup.

02

Capture cluster and shard state

Record allocation explain output, index UUID, shard ID, primary terms, sequence checkpoints and logs.

03

Preserve every copy

Snapshot or copy all relevant node data paths and repository snapshots before modification.

04

Identify data priorities

Document affected indices, time ranges and source systems that may support reindexing.

NEPFOSYS process

How a data-first evaluation proceeds.

01

Shard-copy comparison

NEPFOSYS maps histories, allocation IDs, checkpoints, translog generations and segment health.

02

Protected recovery testing

Replica, snapshot, segment extraction and narrowly scoped translog options are evaluated on copies.

03

Index validation

Document counts, time ranges, mappings and sampled records are checked before reintroduction.

A clear next step

Preserve the original device before another repair or recovery attempt.

Start a case with the exact symptoms, incident timeline, prior attempts, encryption details, and the files that matter most. NEPFOSYS will route the case to the appropriate DFW intake or nationwide mail-in path.

Start with a case review

Questions, answered

What customers ask before starting.

Can elasticsearch-translog truncate fix the shard?

It can remove corrupt translog content so a shard may open, but acknowledged operations in that content can be lost. Preserve and compare all copies first.

Is a replica always current?

No. Compare sequence checkpoints, primary terms and allocation history before choosing a copy.

Can a corrupt shard be rebuilt from the source application?

Sometimes, if the authoritative source and complete mappings are available. Recovery still needs to account for records or updates that exist only in Elasticsearch.

DFW local · Nationwide mail-in

Protect the device. Start with the facts.

Start my recovery case