Skip to content

Regression while planning plans with periods in their name like tag.one: Plan("Aggregate requires at least one grouping or aggregate expression") #8223

@alamb

Description

@alamb

Describe the bug

We found a bug while upgrading IOx to use the latest datafusion

Some of our tests began failing like this:

"Optimizer rule 'push_down_projection' failed"

To Reproduce

in https://github.com/apache/arrow-datafusion/blob/8bd70d691cba1e730ec0d5fdce5cd544acf5008d/datafusion/optimizer/src/push_down_projection.rs#L608-L631

Add this test

    #[test]
    fn aggregate_with_periods() -> Result<()> {
        let schema = Schema::new(vec![Field::new("tag.one", DataType::Utf8, false)]);

        // Build a plan that looks as follows (note "tag.one" is a column named
        // "tag.one", not a column named "one" in a table named "tag"):
        //
        // Projection: tag.one
        //   Aggregate: groupBy=[], aggr=[MAX("tag.one") AS "tag.one"]
        //    TableScan
        let plan = table_scan(Some("m4"), &schema, None)?
            .aggregate(
                Vec::<Expr>::new(),
                vec![max(col(Column::new_unqualified("tag.one"))).alias("tag.one")],
            )?
            .project([col(Column::new_unqualified("tag.one"))])?
            .build()?;

        let expected = "\
        Aggregate: groupBy=[[]], aggr=[[MAX(m4.tag.one) AS tag.one]]\
        \n  TableScan: m4 projection=[tag.one]";

        assert_optimized_plan_eq(&plan, expected)
    }

The test will fail like:

thread 'push_down_projection::tests::aggregate_with_periods' panicked at datafusion/optimizer/src/push_down_projection.rs:1123:45:
failed to optimize plan: Plan("Aggregate requires at least one grouping or aggregate expression")

Expected behavior

The test should pass

Additional context

The regression was introduced in #7981

It appears to be related to columns that have periods in their names

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions