Skip to content

[Variant] Initial API for reading Variant data and metadata#7535

Merged
alamb merged 44 commits intoapache:mainfrom
mkarbo:mkar/7423-read-variant-api
May 31, 2025
Merged

[Variant] Initial API for reading Variant data and metadata#7535
alamb merged 44 commits intoapache:mainfrom
mkarbo:mkar/7423-read-variant-api

Conversation

@mkarbo
Copy link
Copy Markdown
Contributor

@mkarbo mkarbo commented May 21, 2025

Which issue does this PR close?

Rationale for this change

We need to agree on an API for reading Variant metadata. Based on the work and discussions in #7452, in this PR we propose an API plus an implementation (WIP while draft) for reading variant metadata in the parquet-variant crate.
A lot of the work is based on the work in #7452 by @PinkCrow007 and feedback from @alamb, @scovich, and @Weijun-H.

What changes are included in this PR?

  • Adds Variant enum (and associated structs)
  • Adds an API for parsing and reading metadata
  • Adds an API for parsing and reading Variant values of various types

We attempt to be result- and validation driven while ensuring zero-allocations, and we do so avoiding serde_json.

We tried to keep the Variant API similar to the Json::Value api.

Are there any user-facing changes?

The new API's added in parquet-variant will be user facing.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Variant] Rust API to Read Variant Values

6 participants