Skip to content

feat: add data_converter_utils, field_type_utils, file_type and decim…#38

Merged
leaves12138 merged 2 commits into
apache:mainfrom
lszskye:p3.7
Jun 4, 2026
Merged

feat: add data_converter_utils, field_type_utils, file_type and decim…#38
leaves12138 merged 2 commits into
apache:mainfrom
lszskye:p3.7

Conversation

@lszskye
Copy link
Copy Markdown
Contributor

@lszskye lszskye commented Jun 2, 2026

Purpose

Introduce utility classes for data type conversion, field type introspection, file type classification, and decimal operations.

Changes

DataConverterUtils: Provides bidirectional converters between string values and BinaryRow fields

FieldTypeUtils: Utility for FieldType, e.g., integer type checking, scale comparison

FileType: Enum class defining file types: kMeta, kData, kBucketIndex, kGlobalIndex, kFileIndex.

DecimalUtils: validates Arrow decimal type constraints

Tests

  • DataConverterUtilsTest
  • FieldTypeUtilsTest
  • FileTypeTest
  • DecimalUtilsTest

Copy link
Copy Markdown
Contributor

@leaves12138 leaves12138 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the migration. I found one blocker in DataConverterUtils: non-legacy partition string conversion for negative FLOAT/DOUBLE values uses scientific notation because the fixed-format branch only accepts positive values. This breaks round-trip/path compatibility for negative floating partition values.

Comment thread src/paimon/common/utils/data_converter_utils.h Outdated
Copy link
Copy Markdown
Contributor

@leaves12138 leaves12138 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed PR #38. Given the current scope does not support FLOAT/DOUBLE as partition fields, the removal of that conversion path is acceptable. I did not find other blockers.

@leaves12138 leaves12138 merged commit 9fd9edb into apache:main Jun 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants