Transfer logs show e.g. "Loaded 29376 cleaned" but "Upserting 1000 MinorTraceChemistry records". The only way to get 1000 rows is either (1) deduplication by nma_GlobalID leaving 1000 unique IDs, or (2) many rows skipped in _row_to_dict. There is no logging that distinguishes these.
Requested action: In transfers/minor_trace_chemistry_transfer.py, after building row_dicts and calling _dedupe_rows(rows), log both len(row_dicts) and len(rows) (e.g. "Converted N rows to dicts, M unique by nma_GlobalID"). This makes it clear whether the reduction is from dedupe or from skipped rows and avoids confusion about whether the transfer is truncating data.
Transfer logs show e.g. "Loaded 29376 cleaned" but "Upserting 1000 MinorTraceChemistry records". The only way to get 1000 rows is either (1) deduplication by
nma_GlobalIDleaving 1000 unique IDs, or (2) many rows skipped in_row_to_dict. There is no logging that distinguishes these.Requested action: In
transfers/minor_trace_chemistry_transfer.py, after buildingrow_dictsand calling_dedupe_rows(rows), log bothlen(row_dicts)andlen(rows)(e.g. "Converted N rows to dicts, M unique by nma_GlobalID"). This makes it clear whether the reduction is from dedupe or from skipped rows and avoids confusion about whether the transfer is truncating data.