[VL] Fix Iceberg writer aggregate-init build failure on macOS C++20 libc++#12115
Open
jackylee-ch wants to merge 1 commit into
Open
[VL] Fix Iceberg writer aggregate-init build failure on macOS C++20 libc++#12115jackylee-ch wants to merge 1 commit into
jackylee-ch wants to merge 1 commit into
Conversation
…0 libc++
Two call sites in cpp/velox/compute/iceberg/IcebergWriter.cc construct
aggregate types via paren-init with multiple arguments:
fields.emplace_back(name, type, transform, parameter); // line 311
return WriteStats(numWrittenBytes, // line 262
numWrittenFiles, writeIOTimeNs,
writeWallNs);
Both target types are pure aggregate structs without a user-defined
constructor:
// Velox: velox/connectors/hive/iceberg/PartitionSpec.h
struct IcebergPartitionSpec::Field {
std::string name;
TypePtr type;
TransformType transformType;
std::optional<int32_t> parameter;
};
// Gluten: cpp/velox/compute/iceberg/IcebergWriter.h
struct WriteStats {
uint64_t numWrittenBytes{0};
uint32_t numWrittenFiles{0};
uint64_t writeIOTimeNs{0};
uint64_t writeWallNs{0};
...
};
C++20 P0960R3 made aggregates initializable via paren-init at the
language level, but the standard library implementation of
std::construct_at (which std::vector::emplace_back routes through to do
in-place construction) is uneven across stdlibs:
- libstdc++ (Linux GCC): construct_at accepts aggregate paren-init,
so emplace_back(a, b, c, d) compiles for aggregate types.
- libc++ (Apple Clang 17, macOS): construct_at still requires a real
constructor, so the substitution fails:
error: no matching function for call to 'construct_at'
note: candidate template ignored: substitution failure
[with _Tp = IcebergPartitionSpec::Field, _Args = ...]:
no matching constructor for initialization of
'IcebergPartitionSpec::Field'
Result: builds with --enable_enhanced_features=ON on macOS abort
hard at IcebergWriter.cc.cpp.o (~156/217) before producing
libgluten.dylib, blocking any macOS user from running the Iceberg
writer integration that this gate adds.
The fix is a minimal style change at the two call sites — switch
from paren-init to brace-init, which is aggregate initialization on
every C++17/20 toolchain (libstdc++, libc++, MSVC):
fields.push_back({name, type, transform, parameter});
return WriteStats{numWrittenBytes, numWrittenFiles, ...};
Linux libstdc++ continues to compile cleanly: brace-init resolves to
the same aggregate initialization the paren-init form selected via
P0960. Zero runtime change, identical field-by-field initialization.
Verification:
- macOS 14 arm64 + Apple Clang 17 + --enable_enhanced_features=ON:
cpp build now reaches 217/217 (was 156/217 before this fix).
- cpp ctest matrix on the same configuration: 5539 / 5539 pass,
including the enhanced-only velox_iceberg_test executable.
- Linux x86_64 build remains green; brace-init for aggregates is
unchanged in libstdc++.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
The bug
Builds with
--enable_enhanced_features=ON(which compiles the Icebergwriter integration in
cpp/velox/compute/iceberg/) fail to compile onmacOS Apple Clang 17 + libc++ with
-std=c++20. The build aborts atcpp/velox/compute/iceberg/IcebergWriter.cc.cpp.o(~156 / 217 in thegluten cpp ninja graph) with:
Root cause
IcebergWriter.ccconstructs aggregate types with paren-init attwo call sites:
return WriteStats(bytes, files, ioNs, wallNs)gluten::WriteStatscpp/velox/compute/iceberg/IcebergWriter.hfields.emplace_back(name, type, transform, parameter)IcebergPartitionSpec::Field(Velox)velox/connectors/hive/iceberg/PartitionSpec.hBoth target types are pure aggregate structs with no user-defined
constructor:
C++20 P0960R3
made aggregates initializable via paren-init at the language level, but
standard library implementations of
std::construct_at(whichstd::vector::emplace_back,std::make_shared,std::optional::emplaceall route through for in-place construction) have not all caught up:
std::construct_atfor aggregatesSo the same code that passes on Linux blocks the macOS build with
--enable_enhanced_features=ON.The fix
Two minimal edits — switch paren-init to brace-init at the two call
sites:
Brace-init invokes aggregate initialization, which is supported on
every C++17/20 standard library. It does not depend on
std::construct_ataccepting paren-init for aggregates, so theresult is portable.
Why this matters
dev/builddeps-veloxbe.sh --enable_enhanced_features=ONon macOStoday hits a hard compile error before any binary is produced.
macOS build fixes land (gflags load-time abort, Arrow vendored
datetime,
/usr/localisolation), enhanced mode is the next thingmacOS users will try — this PR removes the last compile-time
blocker on that path.
produce identical field-by-field initialization on all
toolchains. Linux libstdc++ still compiles cleanly. The change is
purely build portability.
of
cpp/velox/already uses brace-init for aggregate construction.These two paren-init call sites are the outliers.
How was this patch tested?
-std=c++20+--enable_enhanced_features=ON:IcebergWriter.cc.cpp.o(~156 / 217 ninja steps) with the
construct_atsubstitutionfailure.
velox_iceberg_testexecutable produced under the enhanced-featuresgate.
velox_iceberg_test): 5539 / 5539 pass with this fix.aggregates is the same lowering as the prior paren-init under
libstdc++'s P0960 implementation, so behavior is identical.
Was this patch authored or co-authored using generative AI tooling?
co-auth: Claude (Sonnet/Opus) via Claude Code 1.x