#51 Submission Validation in a worker thread#197
Conversation
| }): Promise<number> => { | ||
| const { submissionId, submissionData, username } = input; | ||
|
|
||
| const performDataValidation = async (submissionId: number): Promise<number> => { |
There was a problem hiding this comment.
This function is being executed by the worker thread, changed the params to receive only the submissionId it will load submission data from db within the worker thread
| workerpool.worker({ | ||
| initializeWorker, | ||
| commitSubmission, | ||
| performDataValidation, |
There was a problem hiding this comment.
adding a function to the workerpool
| submissionData: { | ||
| inserts: submissionData.inserts, | ||
| deletes: submissionData.deletes, | ||
| updates: submissionData.updates, | ||
| }, |
There was a problem hiding this comment.
After validation is complete we do not need to update Submission data, instead it's passing an array of errors (schemaErrors), this function updates the submission status to INVALID where there are errors , or VALID if no errors
joneubank
left a comment
joneubank
left a comment
There was a problem hiding this comment.
Looks good.
I have a question about the state transition to OPEN before running validation... Have we properly considered how this interacts with receiving additional data to the active submission while the validation runs?
| await submissionRepository.update(submission.id, { | ||
| data: updatedActiveSubmissionData, | ||
| updatedBy: username, | ||
| status: 'OPEN', |
There was a problem hiding this comment.
Is this the correct state to set to while we run validation? Do we want to allow new data to be submitted while the validation runs?
There was a problem hiding this comment.
I agree, there should be a blocking state for a submission between OPEN and VALID states, for example: VALIDATING.
Similarly, that state along with COMMITTING should be restricting any update on the submission (submit, edit, delete, commit). I will update this PR to add the proposed change.
| // Updating the Submission with the new data and 'VALIDATING' status before validation starts | ||
| await submissionRepository.update(submission.id, { | ||
| data: updatedActiveSubmissionData, | ||
| updatedBy: username, | ||
| status: 'VALIDATING', | ||
| }); |
There was a problem hiding this comment.
update the submission status to VALIDATING
| if (!isSubmissionActive(activeSubmission.status)) { | ||
| throw new StatusConflict(`Existing submission with status '${activeSubmission.status}' cannot be modified`); | ||
| } |
There was a problem hiding this comment.
Throwing an error to avoid overriding a submission that is not OPEN, VALID or INVALID
|
PR Updated to add new submission status |
| ForceDeleteSubmission: | ||
| description: Optional query parameter to force the deletion of a submission. Default value is false | ||
| name: force | ||
| in: query | ||
| schema: | ||
| type: boolean | ||
| required: false |
There was a problem hiding this comment.
Flag added to the DELETE submission endpoint.
2 participants
Summary
This PR improves the performance of the submission data validation process.
Issues
Description of Changes
Data validation
POST /submission/category/{categoryId}/dataandPOST /submission/category/{categoryId}/filesDELETE /submission/{submissionId}/{actionType}PUT /submission/category/{categoryId}/dataDELETE /submission/category/{categoryId}/data/{systemId}VALIDATINGstatus, it prevents the Submission to be altered by other request during this state.Database
VALIDATINGon Submission statusClose Submission
forceon toDELETE /submission/{submissionId}. By design aVALIDATINGorCOMMITTINGsubmissions cannot be deleted, unless this flag is provided.