Skip to content

Move CrewAI pipeline to task queue (Celery/RQ) #11

@freesoft

Description

@freesoft

Phase 4: Server Architecture

CrewAI pipelines run in threading.Thread (web_tutor.py:857). Not scalable and subject to GIL.

Tasks

  • Set up Celery or RQ with Redis broker
  • Move generate_problem_bg, check_answer_bg, generate_scaffold_problem_bg to async tasks
  • Keep SSE notification mechanism working with external workers

Files

  • web_tutor.py

Metadata

Metadata

Assignees

No one assigned

    Labels

    phase4Phase 4: Server Architecture

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions