Apache Parquet recovery

A Parquet file can retain readable column pages even when its footer or one row group is damaged.

Parquet is a column-oriented format whose footer describes schema, row groups, column chunks, encodings, and statistics. Truncation near the end can remove the metadata needed to locate otherwise present pages.

Understand the symptom

Readers use the final magic bytes and footer length to locate serialized file metadata.

WHY

What is happening

Plaintext-footer Parquet files use PAR1 at the beginning and end, with footer length stored before the final magic. Encrypted-footer files use PARE. Row groups and column pages can fail independently, but accurate extraction requires schema and page boundaries.

What you may see

Symptoms that change the recovery path.

01

Magic bytes not found in footer

The file may be truncated or the tail overwritten.

02

Could not read footer

Footer length, metadata, encryption context, or final bytes may be invalid.

03

One row group fails

Damage may be limited to a column chunk or data page.

04

Partition count is short

Some objects may be missing, zero bytes, or replaced.

Protect the source

Actions that can make the situation worse.

01

Dataset rewrite can replace good objects

Freeze compaction, vacuum, and overwrite jobs.

02

Skipping corrupt pages changes results

Document every discarded row group or column page.

03

Encrypted metadata needs keys

Do not rotate or lose footer and column keys.

Do this now

Safe steps before evaluation.

01

Freeze the partition set

Copy files, manifests, transaction logs, catalog entries, and object versions.

02

Capture schema and writer context

Keep application, Parquet version, codecs, page format, and encryption settings.

03

Record expected rows

Note partition keys, file sizes, row counts, and critical columns.

04

Protect source storage

Image local media or retain immutable object versions before parsing.

NEPFOSYS process

How a data-first evaluation proceeds.

01

File-tail and schema assessment

NEPFOSYS maps magic, footer, row groups, chunks, pages, codecs, and encryption context.

02

Selective column extraction

Readable pages are decoded into new files without changing sources.

03

Dataset validation

Schema, partition keys, row counts, null counts, statistics, and priority columns are compared.

A clear next step

Preserve the original device before another repair or recovery attempt.

Start a case with the exact symptoms, incident timeline, prior attempts, encryption details, and the files that matter most. NEPFOSYS will route the case to the appropriate DFW intake or nationwide mail-in path.

Start with a case review

Questions, answered

What customers ask before starting.

Can data survive a missing Parquet footer?

Potentially, but row-group and column-page boundaries must be reconstructed from reliable evidence.

What does PARE mean?

It marks an encrypted-footer Parquet file; authorized keys and crypto metadata are required.

Should we rerun compaction?

No until the affected objects and catalog state are preserved; compaction may replace evidence.

DFW local · Nationwide mail-in

Protect the device. Start with the facts.

Start my recovery case