List view

Is it possible to support PyArrow backed UDFs in Comet natively?

enhancementNew feature or requestNew feature or request

Status: Open.

#957 In apache/datafusion-comet;

· SemyonSinchenko opened on Sep 21, 2024

Register CometArrowAllocator as a Spark MemoryConsumer for JVM-UDF dispatch

area:expressionsExpression evaluationExpression evaluation

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4174 In apache/datafusion-comet;

· andygrove opened on May 1, 2026

Eliminate remaining row<->Arrow round-trip in CometPythonMapInArrowExec

area:udf

enhancementNew feature or requestNew feature or request

priority:lowMinor issues, test failures, tooling, cosmeticMinor issues, test failures, tooling, cosmetic

Status: Open.

#4240 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

Add native implementations ofRegExpExtract and RegExpExtractAll

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

good first issueGood for newcomersGood for newcomers

Status: Open.

#2708 In apache/datafusion-comet;

· andygrove opened on Nov 6, 2025

Re-enable COUNT for mixed Spark partial / Comet final aggregate execution

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4242 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

Bloom filter intermediate aggregate buffers are not compatible between Spark and Comet

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

bugSomething isn't workingSomething isn't working

crashNative engine crash/panic/segfaultNative engine crash/panic/segfault

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#2889 In apache/datafusion-comet;

· andygrove opened on Dec 12, 2025

Mismatch in Data Types Between Partial and Final Stages in Aggregations

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

bugSomething isn't workingSomething isn't working

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#1287 In apache/datafusion-comet;

· andygrove opened on Jan 14, 2025

[hdfs-opendal] Got an reader got too little data error on hdfs-opendal

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#2400 In apache/datafusion-comet;

· wForget opened on Sep 16, 2025

ExecutionMemoryPool errors releasing more memory than allocated

bugSomething isn't workingSomething isn't working

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

Status: Open.

#2453 In apache/datafusion-comet;

· andygrove opened on Sep 25, 2025

Dropping a GlobalRef in a detached thread

area:ffiArrow FFI / JNI boundaryArrow FFI / JNI boundary

bugSomething isn't workingSomething isn't working

priority:highCrashes, panics, segfaults, major functional breakageCrashes, panics, segfaults, major functional breakage

Status: Open.

#2470 In apache/datafusion-comet;

· andygrove opened on Sep 26, 2025

Fix issues found when documenting current expression incompatibilities

area:expressionsExpression evaluationExpression evaluation

bugSomething isn't workingSomething isn't working

priority:criticalData corruption, silent wrong results, security issuesData corruption, silent wrong results, security issues

Status: Open.

#4074 In apache/datafusion-comet;

· andygrove opened on Apr 24, 2026

Enable windowed aggregates and fix correctness issues

area:aggregationHash aggregates, aggregate expressionsHash aggregates, aggregate expressions

correctness

enhancementNew feature or requestNew feature or request

expressionSpark expression relatedSpark expression related

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4007 In apache/datafusion-comet;

· andygrove opened on Apr 20, 2026

feat: add experimental support for Spark regexp expressions via JVM UDF framework

Status: Open (in progress).

apache/datafusion-cometnumber 4239

#4239 In apache/datafusion-comet;

· andygrove opened on May 6, 2026

feat: support Spark expression slice

Status: Open (in progress).

apache/datafusion-cometnumber 4149

#4149 In apache/datafusion-comet;

· andygrove opened on Apr 29, 2026

[native_datafusion] Add support for reading row index metadata columns

enhancementNew feature or requestNew feature or request

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

Status: Open.

#3432 In apache/datafusion-comet;

· andygrove opened on Feb 6, 2026

chore: DataFusion 54.0.0

buildBuild environmentBuild environment

dependenciesPull requests that update a dependency filePull requests that update a dependency file

enhancementNew feature or requestNew feature or request

Status: Open.

#3978 In apache/datafusion-comet;

· comphead opened on Apr 17, 2026

feat: vendor-pluggable S3 credentials for native scans

Status: Open (in progress).

apache/datafusion-cometnumber 4309

#4309 In apache/datafusion-comet;

· mbutrovich opened on May 13, 2026

Implement JVM UDFs for all date/time expressions

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4311 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

feat: add experimental support for JSON expressions using Comet JVM UDF Framework

Status: Draft (not ready).

apache/datafusion-cometnumber 4305

#4305 In apache/datafusion-comet;

· andygrove opened on May 12, 2026

Implement JVM UDFs for JSON expressions

area:expressionsExpression evaluationExpression evaluation

enhancementNew feature or requestNew feature or request

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

Status: Open.

#4313 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

native_datafusion more permissive than Spark 3.x when reading Parquet TimestampNTZ columns

area:scanParquet scan / data readingParquet scan / data reading

bugSomething isn't workingSomething isn't working

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

priority:mediumFunctional bugs, performance regressions, broken featuresFunctional bugs, performance regressions, broken features

spark 3.x

Status: Open.

#4219 In apache/datafusion-comet;

· andygrove opened on May 5, 2026

native_datafusion: ParquetSchemaConvert error does not include the file path

native_datafusionSpecific to native_datafusion scan typeSpecific to native_datafusion scan type

Status: Open.

#4316 In apache/datafusion-comet;

· andygrove opened on May 13, 2026

Upgrade Spark 4.1.1 to 4.1.2

enhancementNew feature or requestNew feature or request

requires-triage

Status: Open.

#4380 In apache/datafusion-comet;

· andygrove opened on May 21, 2026