Fix/recipe duplicates last line of csv apis#64
Fix/recipe duplicates last line of csv apis#64alexbourret wants to merge 7 commits intofeature/sniffing-csv-dialectfrom
Conversation
Ellana42
left a comment
Ellana42
left a comment
There was a problem hiding this comment.
The first comment is my biggest issue - otherwise works great
| # Todo: check api_response key is free and add something overwise | ||
| base_row = copy.deepcopy(metadata) | ||
| if is_raw_output: | ||
| assert_json(json_response) |
There was a problem hiding this comment.
This feels dangerous to me because the assert json is not a perfect match with the json.dumps api -> it could make workflows that worked previously failed. I'd prefer catching the error from the dump and then trying other workflows
Accepted object reference
| json_response = decode_csv_data(json_response) | ||
| for row in json_response: | ||
| decoded_csv_data = decode_csv_data(json_response) | ||
| is_api_returning_dict = False |
There was a problem hiding this comment.
This breaks compatibility too but I believe this whole functionnality was introduced in an alpha for a client so probably not problematic
| decoded_csv_data = [ | ||
| { | ||
| DKUConstants.API_RESPONSE_KEY: "{}".format( | ||
| decode_bytes(json_response) | ||
| ) | ||
| } |
There was a problem hiding this comment.
| decoded_csv_data = [ | |
| { | |
| DKUConstants.API_RESPONSE_KEY: "{}".format( | |
| decode_bytes(json_response) | |
| ) | |
| } | |
| decoded_csv_data = [ | |
| { | |
| DKUConstants.API_RESPONSE_KEY: decode_bytes(json_response) | |
| } |
| ## [Version 1.2.7](https://github.com/dataiku/dss-plugin-api-connect/releases/tag/v1.2.7) - Feature - 2026-02-18 | ||
|
|
||
| - Detecting dialect for better csv decoding | ||
| - Fixing duplication of last line in csv APIs using the recipe |
There was a problem hiding this comment.
Missing mention of the new string dump
| def assert_json(variable_to_check): | ||
| if isinstance(variable_to_check, dict) or isinstance(variable_to_check, list): | ||
| return | ||
| raise Exception("Returned data is not JSON format. Try again with 'Raw JSON output' un-checked.") |
There was a problem hiding this comment.
nit: A more specific exception might be better
2 participants
https://app.shortcut.com/dataiku/story/308025