feat(ci): enhance test notifications with automated incident reporting

[jwu-labs]

Pull Request Template

Type of Change

• New feature
• Bug fix
• Documentation
• Refactor
• Chore

Changes

This PR upgrades the notifications job in the testsPython.yml workflow from a placeholder script to a fully functional automated incident reporting system.

• Automated Email Notifications: Integrated action-send-mail to automatically email a detailed incident report to designated recipients whenever the Python unit tests fail.
• Structured Incident Reporting: Implemented the automated generation of a structured incident report for failed test runs, including the repository name, branch name, commit SHA, triggering user, a summary of failed tests, root causes, and a direct link to the full logs.
• Automated Log Analysis: Added a custom parsing script that uses the GitHub CLI to fetch logs, and automatically extract error summaries and parse the output for common Python errors (e.g., NameError, TypeError, ValueError).
• Visual Feedback: Added GitHub Step Summaries with Markdown-formatted status messages on the Action summary page for immediate visibility into test outcomes (passed, failed, or cancelled).
• Improved Reliability: Added 'if: always()' to ensure notifications are sent even when the preceding test job fails, ensuring constant monitoring.
• Workflow Permission Restriction: Added explicit read-only permissions (contents and actions) for the notifications job to enforce least-privilege access while still allowing log retrieval for incident reporting.
• Short SHA Output: Added a step to generate a 7-character Short SHA for cleaner reporting.

Testing

• Email Notification Dispatch: Validation was performed to ensure the automated email is successfully dispatched to designated recipients upon a detected test failure.
• Incident Report Accuracy: The generated report was verified to contain accurate repository name, branch name, 7-character commit SHA, trigger actor, a section for error summary, a section with root cause, and a correct link to full logs.
• Log Parsing and Cleanup: The log parsing logic was confirmed to successfully strip ANSI color codes and timestamps from raw runner logs to accurately identify specific Python errors like ImportError and ValueError.
• UI Summary Verification: The GitHub Actions summary page was verified to display the correct Markdown status banners (e.g., ❌ Build Failed) and direct commit links for failed test runs.
• Job Execution Reliability: The if: always() logic was validated by ensuring the notification job runs to completion even when the preceding unit test job fails.
• Security and Access Validation: The job was verified to successfully retrieve logs via the GitHub API while operating under restricted read-only permissions.
• Short SHA Integration: The 7-character commit SHA generation was confirmed and verified to be properly integrated into both the email subject lines and the incident report headers.

Screenshots

GitHub Actions Summary:

GitHub Actions Summary - Build Failed

GitHub Actions Summary - Build Cancelled

Email Notifications:

Email Notification - GitHub Actions Notifications

Email Notification - Incident Report

Workflow Steps:

Notify on failed test results (actions summary)

Build incident report

Notify on failed test results (email)

Dependencies

• GitHub Actions:
  • actions/checkout@v6: Used for repository checkout.
  • dawidd6/action-send-mail@v3: Used for dispatching the incident report via SMTP.
• Tools:
  • GitHub CLI (gh): Utilized within the workflow to fetch job logs via the GitHub API.
  • Standard Linux Utilities: grep, sed, and sort are used for log cleaning and error parsing.
• Repository Secrets:
  • EMAIL_USERNAME: The SMTP login email.
  • EMAIL_PASSWORD: The SMTP app-specific password.
  • NOTIFY_EMAIL: The recipient address for failure alerts.

Breaking Changes

• Secret Requirements: This PR introduces a dependency on three new repository secrets (EMAIL_USERNAME, EMAIL_PASSWORD, and NOTIFY_EMAIL). If these are not configured in the repository settings, the email notification step will fail.
• Workflow Permissions: The workflow now requires explicit contents: read and actions: read permissions to allow the GitHub CLI to retrieve logs. These have been added to the YAML, but maintainers should ensure no organization-level policies override these settings.