Describe the bug, including details regarding any error messages, version, and platform.
In the main branch, DecimalVector and Decimal256Vector are losing nullability information during a zero-copy transfer. It currently forces a nullable FieldType onto the "to" ValueVector.
The deficiency in the code is in DecimalVector.TransferImpl class when trying to get a transfer getTransferPair(String, BufferAllocator) creating a DecimalVector with a nullable FieldType (although the "from" ValueVector's FieldType is non-nullable):
|
private class TransferImpl implements TransferPair { |
|
DecimalVector to; |
|
|
|
public TransferImpl(String ref, BufferAllocator allocator) { |
|
to = |
|
new DecimalVector(ref, allocator, DecimalVector.this.precision, DecimalVector.this.scale); |
|
} |
Note that all the other (at least the primitive) ValueVectors are transferred without losing nullability information by using the "from" ValueVector's field. E.g.:
|
private class TransferImpl implements TransferPair { |
|
BigIntVector to; |
|
|
|
public TransferImpl(String ref, BufferAllocator allocator) { |
|
to = new BigIntVector(ref, field.getFieldType(), allocator); |
|
} |
To reproduce the issue: add the following test in src/test/java/org/apache/arrow/vector/TestDecimalVector.java to see it fail.
@Test
public void testGetTransferPairWithoutFieldNonNullable() {
final FieldType decimalNonNullableType =
new FieldType(false, new ArrowType.Decimal(10, scale), null);
final DecimalVector fromVector =
new DecimalVector("decimal", decimalNonNullableType, allocator);
final TransferPair transferPair =
fromVector.getTransferPair(fromVector.getField().getName(), allocator);
final DecimalVector toVector = (DecimalVector) transferPair.getTo();
// These asserts fail
assertSame(fromVector.getField().isNullable(), toVector.getField().isNullable());
assertSame(fromVector.getField().getFieldType(), toVector.getField().getFieldType());
}
Describe the bug, including details regarding any error messages, version, and platform.
In the main branch,
DecimalVectorandDecimal256Vectorare losing nullability information during a zero-copy transfer. It currently forces a nullableFieldTypeonto the "to"ValueVector.The deficiency in the code is in
DecimalVector.TransferImplclass when trying to get a transfergetTransferPair(String, BufferAllocator)creating aDecimalVectorwith a nullableFieldType(although the "from" ValueVector's FieldType is non-nullable):arrow-java/vector/src/main/java/org/apache/arrow/vector/DecimalVector.java
Lines 563 to 569 in 48a20ba
Note that all the other (at least the primitive)
ValueVectors are transferred without losing nullability information by using the "from"ValueVector's field. E.g.:arrow-java/vector/src/main/java/org/apache/arrow/vector/BigIntVector.java
Lines 319 to 324 in 48a20ba
To reproduce the issue: add the following test in
src/test/java/org/apache/arrow/vector/TestDecimalVector.javato see it fail.