Motivation
The lack of proper async support in platform has been a constant hindrance for over a decade. It's kept us from being able to use websockets and has forced to:
- Use suboptimal polling solutions for things like course import and timed exams.
- Caused awkward UX issues like delayed content indexing after import on libraries, for which we need to make hacky workarounds.
- Skip features altogether because they are impractical to implement, like real-time notifications in the LMS when using a browser.
This lack of platform capability has also narrowed our vision of what we can do in terms of collaborative tools and group interactions. When we talk about Teams, group projects, or plugin possibilities, there's a whole space of solutions that we write off in advance because we don't have the ability to do real-time updates effectively.
More recently, @regisb's proposed PXC will require the platform to have async capabilities and be served via some ASGI server (likely granian).
Current Status
We currently have a few gaps:
- Our platform still uses
threading.local in a number of places—something that only isolates that var to the OS thread, not to the overall execution context (i.e. a thread or asyncio task). This must be addressed first.
- Our platform uses explicit thread locking in places. I'm not clear about these cases, but each use would need to be investigated.
- We would need a new settings file for an ASGI deployment.
- Tutor would need to be updated to deploy a new backend for platform that would route and serve ASGI requests.
Scope of work
To make the platform safe to run in async, we should examine all existing uses of threading within openedx-platform and its dependencies and either remove them (if there are safe alternatives), or convert them to use contextvars equivalents instead. Alternatively, we can try to make use of Django's sync_to_async wrapper where it's appropriate.
I'm currently ignoring threading code that only shows up in tests.
Out of Scope
Actually making the ASGI deployment settings and deployment via granian is out of scope for this Epic. This is just about making things safe to be async, as a precursor to that work.
Motivation
The lack of proper async support in platform has been a constant hindrance for over a decade. It's kept us from being able to use websockets and has forced to:
This lack of platform capability has also narrowed our vision of what we can do in terms of collaborative tools and group interactions. When we talk about Teams, group projects, or plugin possibilities, there's a whole space of solutions that we write off in advance because we don't have the ability to do real-time updates effectively.
More recently, @regisb's proposed PXC will require the platform to have async capabilities and be served via some ASGI server (likely granian).
Current Status
We currently have a few gaps:
threading.localin a number of places—something that only isolates that var to the OS thread, not to the overall execution context (i.e. a thread or asyncio task). This must be addressed first.Scope of work
To make the platform safe to run in async, we should examine all existing uses of
threadingwithinopenedx-platformand its dependencies and either remove them (if there are safe alternatives), or convert them to usecontextvarsequivalents instead. Alternatively, we can try to make use of Django's sync_to_async wrapper where it's appropriate.I'm currently ignoring threading code that only shows up in tests.
Out of Scope
Actually making the ASGI deployment settings and deployment via granian is out of scope for this Epic. This is just about making things safe to be async, as a precursor to that work.