Apache ORC recovery

An ORC file may retain readable stripes even when its tail metadata or one stripe is damaged.

ORC stores rows in self-contained stripes plus file metadata, a footer, and a postscript at the tail. Tail truncation can prevent readers from discovering stripes that remain physically present.

Understand the symptom

The postscript describes footer lengths, compression, version, and other information needed to parse the file.

WHY

What is happening

Each ORC stripe contains indexes, data, and a stripe footer. The file tail contains metadata, a file footer, postscript, and its length byte. Damage may be localized to the tail, a stripe footer, a compressed stream, or a column within one stripe.

What you may see

Symptoms that change the recovery path.

01

Malformed ORC file

Postscript, footer, version, or compression metadata may be invalid.

02

Unexpected end of file

The tail or last stripe may be truncated.

03

One stripe cannot be read

A column stream, index, or stripe footer may be damaged.

04

Wrong row count

File statistics or table inventory may not match surviving stripes.

Protect the source

Actions that can make the situation worse.

01

Compaction replaces original stripes

Freeze table maintenance and retention jobs.

02

Dropping a partition removes evidence

Preserve metastore and object versions.

03

Silent row skipping changes analytics

Every excluded stripe must be documented.

Do this now

Safe steps before evaluation.

01

Copy the full partition set

Keep ORC objects, table metadata, manifests, and transaction state.

02

Record writer details

Note ORC version, compression, schema, encryption, timezone, and application.

03

Define expected data

List partitions, columns, row counts, and critical ranges.

04

Protect the source

Retain immutable object versions or image local disks before recovery reads.

NEPFOSYS process

How a data-first evaluation proceeds.

01

Tail and stripe mapping

NEPFOSYS identifies postscript, footer, stripe boundaries, streams, and readable ranges.

02

Copy-based extraction

Readable stripes or columns are written to new files.

03

Table validation

Schema, partitions, rows, statistics, timestamps, and priority columns are reconciled.

A clear next step

Preserve the original device before another repair or recovery attempt.

Start a case with the exact symptoms, incident timeline, prior attempts, encryption details, and the files that matter most. NEPFOSYS will route the case to the appropriate DFW intake or nationwide mail-in path.

Start with a case review

Questions, answered

What customers ask before starting.

Can one damaged stripe make the whole ORC file fail?

Yes. Some readers stop on a stripe or tail error even when earlier stripes remain readable.

Is the ORC postscript compressed?

No. The specification places it at the end and leaves it uncompressed.

Should we repair the production table in place?

No. Preserve files and metastore state, then extract into a separate controlled destination.

DFW local · Nationwide mail-in

Protect the device. Start with the facts.

Start my recovery case