feat: add kubernetes support

[kyteinsky]

No description provided.

[marcelklehr]

I would leave the threads running, regardless of whether the app is enabled, app_enabled can be checked inside the threads

[kyteinsky]

yes it would be simpler, although I imagine we wouldn't have the guarantee that the thread completed its processes until we hold the line, which is to say, we wait for the thread to complete before allowing the lifespan of the fastapi app to proceed with the shutdown.
app_enabled would signal the thread but the app would still proceed with the cleanup.

the thread.wait() is kind of the equivalent of SHUTDOWN_CLEAR.wait() in the llm2 PR.

this pointed out an important fact, one event for each thread is overkill, only one event for all the threads should suffice and why not let it be the app_enabled one as you suggest.

[marcelklehr]

app_enabled=False and shutdown is not the same thing though, I think

[kyteinsky]

app_api switched to a new behaviour recently (not sure when exactly) where disabling the app from the GUI/occ command first sends the enable=False request and then stops the container

[marcelklehr]

mh, ok. but we can't get rid of the entire app_enabled route because of backwards compatibility, probably?

[kyteinsky]

maybe we can guess that the app stopping event (the yield in lifespan) is the result of app being disabled if we remove the app_enabled route but it's not ideal, so would be nice if we keep it for now.

app_enabled=False and shutdown is not the same thing though, I think

and you were right :)
added a dedicated event.

[marcelklehr]

Shall we set the context_chat php app to the branch of the corresponding PR here, so we can use CI properly? (temporarily)

[kyteinsky]

Shall we set the context_chat php app to the branch of the corresponding PR here, so we can use CI properly? (temporarily)

good idea!