[AMORO-1503][mixed-hive] Add INFO logs for partition operations#4200
[AMORO-1503][mixed-hive] Add INFO logs for partition operations#4200lintingbin wants to merge 1 commit into
Conversation
Add INFO/WARN logging on the Mixed Hive Table partition operation paths (create / drop / alter location) so that issues like missing or mis-located partitions can be diagnosed from the logs. Logging added at two layers: * HivePartitionUtil: per-partition LOG.info on create, drop, and update-location paths, plus matching LOG.warn on failures. The alterPartition log now reports both old and new locations. * UpdateHiveFiles / ReplaceHivePartitions: per-batch LOG.info for the drop / create / alter sets emitted by commitPartitionedTable, with a count and a sampled list of partitions to avoid log explosion on large commits, plus matching LOG.warn on commit failures. Each log includes the table identifier, partition values, location (old -> new for alter), and transaction id where available.
|
Hi @zhoujinsong, could you help take a look at this PR when you have time? Thanks! Quick note on CI: the |
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@amoro.apache.org list. Thank you for your contributions. |
Summary
Closes #1503.
Adds INFO/WARN logs on the Mixed Hive Table partition operation paths
(create / drop / alter location). These operations are infrequent, so
logging them at INFO level makes after-the-fact diagnosis of missing
or mis-located partitions much easier with negligible runtime cost.
Logs are added at two layers:
HivePartitionUtil— per-partitionLOG.infooncreatePartitionIfAbsent,dropPartition, andupdatePartitionLocation, plus matchingLOG.warnonthe failure paths. The existing
alterPartitionlog now reports both oldand new locations.
UpdateHiveFiles/ReplaceHivePartitions— per-batchLOG.infofor thedrop / create / alter sets emitted by
commitPartitionedTable, with acount and a sampled list of partitions (capped at 5, with
...for therest) to avoid log explosion on large commits, plus matching
LOG.warnon commit failures.
Each log line carries the table identifier, partition values, location
(old → new on alter), and transaction id where available, so a single
log record is enough to identify the partition and trace it back to a
commit.
Sample logs
Per-partition (single-partition path,
HivePartitionUtil):Per-batch (transaction commit path,
UpdateHiveFiles):Failure path:
Tests
No new unit tests; existing tests in the module continue to pass:
mvn spotless:checkandmvn checkstyle:checkare clean for thetouched module.