Using arrow to write parquet (ends up with an empty table)

[dragan224]

Hi all,

I have this piece of code which produces a valid parquet file (using an arrow table), however the contents of the file are empty.

Code: https://ideone.com/DJkKUF

When I run "parquet-tools cat test.parquet" and actually try to read the file, i get an empty output, shouldn't the file have 5 entries?

Edit: Furthermore when i run "parquet-tools schema test.parquet" the schema prints correctly:

message schema {
optional group MyStruct {
required binary str (UTF8);
}
}

[wesm]

This is really a Parquet issue, can you open an issue with this information on the Apache Parquet JIRA?

[wesm]

FWIW, It looks like the non-nullable case might not be handled properly. Does it work if you make the string field nullable?

[dragan224]

Nope.

And I just noticed, that it throws this error: "lemented: Level generation for Struct not supported yet". Sorry, didn't realize it before, but for some reason it still creates a file with a valid schema (and after that it throws that exception).

I'll go ask parquet folks. My guess is that they intended for structs to be written differently than what I'm doing there?

Edit: Follow-up JIRA link for anyone that comes across this in the future: https://issues.apache.org/jira/browse/PARQUET-1352

[wesm]

Thanks. It'll be the same people responding there but we'll see what's up