fix: Implement RSS feed circuit breaker#141
Open
seer-by-sentry[bot] wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR addresses issue TUVIX-API-21, where the Tuvix API was logging a high volume of HTTP 522 (and other Cloudflare-related) errors due to repeatedly attempting to fetch from persistently failing RSS feed URLs.
Root Cause: The system lacked a mechanism to identify and temporarily disable feed sources that consistently failed to fetch, leading to continuous retries of broken URLs on every cron run.
Solution Implemented:
Introduced a circuit breaker pattern for RSS feed fetching:
consecutive_failures(integer),last_error_at(timestamp), andfetch_disabled_at(timestamp, nullable) columns to thesourcestable inpackages/api/src/db/schema.ts.0012_add_source_failure_tracking.sql) to safely add these new columns and an index onfetch_disabled_at.packages/api/src/services/rss-fetcher.ts):consecutive_failurescount is reset to 0, andlast_error_atandfetch_disabled_atare cleared.fetchSingleFeedencounters an error,consecutive_failuresis incremented,last_error_atis updated, andlastFetchedis advanced to prevent immediate re-queueing. Ifconsecutive_failuresreaches a threshold (currently 10),fetch_disabled_atis set, effectively disabling the feed.getStaleSourcesquery now excludes feeds wherefetch_disabled_atis set. This prevents the system from attempting to fetch from known broken sources.fetch_disabled_atis older than 24 hours) to allow for potential recovery of the external feed source.This change significantly reduces the error noise from persistently failing feeds and optimizes resource usage by avoiding unnecessary fetch attempts.
Type of Change
Related Issues
Fixes #
Relates to #TUVIX-API-21
Changes Made
consecutive_failures,last_error_at, andfetch_disabled_atcolumns to thesourcestable inpackages/api/src/db/schema.ts.0012_add_source_failure_tracking.sql) to safely add these new columns and an index onfetch_disabled_at.packages/api/src/services/rss-fetcher.tsto manage successful fetches, increment failures, disable feeds, and re-enable them after a cooldown period.getStaleSourcesquery to exclude feeds that are currently disabled.Testing
pnpm test)pnpm type-check)pnpm lint)Screenshots/Videos
Documentation
Checklist
Additional Notes
Fixes TUVIX-API-21