Skip to content

feat: initialize package#1

Merged
ceholden merged 13 commits into
mainfrom
feat/big-bang
Feb 27, 2026
Merged

feat: initialize package#1
ceholden merged 13 commits into
mainfrom
feat/big-bang

Conversation

@ceholden
Copy link
Copy Markdown
Contributor

@ceholden ceholden commented Feb 26, 2026

Description

This PR adds an updated "ECS protection manager" that I originally wrote to help prevent scale-in on long running ECS tasks that are auto-scaled based on queue depth. Once the worker count scales up and they start consuming tasks, the message depth will go down and trigger scale-in. If the tasks are doing work we want to prevent this from happening!

I added some features to mature this helper,

  1. Debounce API calls to avoid rate limits
  2. Use a file lock to prevent parallel workers from releasing task protection (e.g., Celery running with --concurrency=4 workers won't override the task protection from other workers)
  3. Support injecting the protection manager context to give the tasks control over the task protection. If the task is running long then it can extend the task protection (e.g., if the task is somewhat unbound in runtime)

@ceholden ceholden marked this pull request as ready for review February 27, 2026 20:47
@ceholden ceholden merged commit b5ce5b5 into main Feb 27, 2026
2 checks passed
@ceholden ceholden deleted the feat/big-bang branch February 27, 2026 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant