MDEV-38735 Support --hex-blob option for mysqldump --tab/--dir and mariadb-import to preserve binary data#4610
Open
jendis wants to merge 2 commits intoMariaDB:mainfrom
Open
MDEV-38735 Support --hex-blob option for mysqldump --tab/--dir and mariadb-import to preserve binary data#4610jendis wants to merge 2 commits intoMariaDB:mainfrom
jendis wants to merge 2 commits intoMariaDB:mainfrom
Conversation
…data Problem: When using mysqldump with --tab or --dir to export table data to separate .txt files, BLOB and BINARY columns containing null bytes (0x00) or other binary data get corrupted or truncated because the tab-separated text format cannot properly represent binary data. Solution: When both --hex-blob and --tab/--dir are specified, automatically wrap BLOB and BINARY columns with HEX() in the SELECT statement. This ensures binary data is exported as hexadecimal strings that can be safely stored in text files and later imported without data loss. Implementation: - Query INFORMATION_SCHEMA.COLUMNS to get data_type and character_set_name - Detect BLOB/BINARY columns (binary charset + blob/binary data types) - Wrap detected columns with HEX(column_name) AS column_name in SELECT - Works for: BINARY, VARBINARY, TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB This enables proper round-trip export/import of binary data when used with mariadb-import --hex-blob. Example usage: mysqldump --hex-blob --tab=/tmp/backup test mytable # Creates mytable.txt with hex-encoded BLOB data
…data Problem: When exporting tables with mysqldump --hex-blob --tab/--dir, BLOB and BINARY columns are hex-encoded to preserve binary data (including null bytes) in text files. However, mariadb-import had no corresponding option to decode this hex data back to binary during import, making the round-trip export/import workflow impossible for binary data. Solution: Implement --hex-blob option for mariadb-import that automatically detects BLOB and BINARY columns and applies UNHEX() transformation during import. Implementation: When --hex-blob is specified: 1. Query server's INFORMATION_SCHEMA.COLUMNS to get table structure: - column_name, data_type, character_set_name 2. Identify BLOB/BINARY columns (binary charset + blob/binary types) 3. Modify LOAD DATA statement to use user variables for hex data: - LOAD DATA INFILE ... (id, @blob_col_hex, varchar_col) 4. Add SET clause to convert hex to binary: - SET blob_col=UNHEX(@blob_col_hex) Supports all binary column types: - BLOB family: TINYBLOB, BLOB, MEDIUMBLOB, LONGBLOB - BINARY family: BINARY(n), VARBINARY(n) Works with: - Explicit --columns parameter (subset of columns) - All columns (default) - Mixed tables (BLOB and non-BLOB columns) - NULL and empty values - Both standalone .txt files and --dir mode Example workflow: # Export with hex-encoded BLOBs mysqldump --hex-blob --tab=/tmp/backup test mytable # Import with automatic UNHEX() conversion mariadb-import --hex-blob test /tmp/backup/mytable.txt # Binary data including null bytes preserved perfectly Test coverage (mysql-test/main/mariadb-import.test): - Round-trip for all BLOB types with embedded null bytes - BINARY and VARBINARY types - Mixed tables (BLOB + non-BLOB columns) - Import with --columns parameter - NULL and empty blob handling - Comparison with/without --hex-blob (data mismatch detection) - Directory-based import with --dir option - Data integrity verification via direct SQL comparison This completes the --hex-blob round-trip functionality, enabling reliable backup and restore of tables containing binary data.
gkodinov
added
the
External Contribution
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extend the existing --hex-blob option to work with --tab/--dir mode for both mysqldump and mariadb-import:
mysqldump --hex-blob --tab/--dir:
mariadb-import --hex-blob:
Together, these changes enable a complete round-trip export/import workflow for tables containing binary data.