Skip to content

Added an exit if the socket is closed for 10 min#101

Merged
thomst08 merged 7 commits into
thomst08:masterfrom
tacoroumen:disconnect-no-reconnect
May 21, 2026
Merged

Added an exit if the socket is closed for 10 min#101
thomst08 merged 7 commits into
thomst08:masterfrom
tacoroumen:disconnect-no-reconnect

Conversation

@tacoroumen
Copy link
Copy Markdown

As per #99

@thomst08
Copy link
Copy Markdown
Owner

Hey @tacoroumen,

Oh yeah.... Yeah, that should do it :)
Awesome work, will check it out (after the other commits)

@gabemeola
Copy link
Copy Markdown

@tacoroumen I'm not familiar with this code, but if it's simple enough to detect and exit, could it attempt to reconnect instead?

We could still exit if it couldn't reconnect after say 3 times.

@tacoroumen
Copy link
Copy Markdown
Author

Hey @gabemeola I'm assuming it would already try and reconnect. But I'm not sure maybe @thomst08 knows this? If it doesn't reconnect after a disconnect we could also add this yes.

@thomst08
Copy link
Copy Markdown
Owner

But I'm not sure maybe @thomst08 knows this? If it doesn't reconnect after a disconnect we could also add this yes.

This is a little tricky to know exactly.... Short answer is I don't know.
There has been times when D#+ crashes in the background and does not trigger any calls to functions I have, so I do not know when the bot is dead but still running. I am not sure if in this case it will catch it or not, it depends.... Honestly, only testing it will know for sure.... We will need to know how to crash it consistently and is predictable ways.

We could still exit if it couldn't reconnect after say 3 times.

I like the idea and know where your coming from, but if D#+ crashes in the background, I haven't had success restarting the bot without closing the program and reopening it, even when rebuilding the bot without a restart, it seems to crash in the library in the backend and I couldn't find a way of reloading it. But this might be unrelated to this issue, just a note.

@tacoroumen, you have got the check in the right spot, but I am unsure if that's what we want to look for, what happens when the bot crashes in other ways? Will it kill it when it's about to restart? I am not sure at the moment. But at a guess, it's a good idea and worth exploring.

- Updated DSharpPlus packages
- Added waits to notification calls when sending messages on startup, this should help prevent the bot hitting rate limits
- Help prevent rate limits being hit at startup
@thomst08
Copy link
Copy Markdown
Owner

Hey guys,

I added some changes to the PR, basically some small waits on calls to Discord, just to help prevent rate limits. Was setup on PR #81, but only the waits without all the logging and other changes.

I also push this into a docker container for testing, you can find the image here: thomst08/requestrr:dev-ratelimit

@thomst08
Copy link
Copy Markdown
Owner

Just testing the changes on my live system, these are the logs I am now getting.

warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-14 07:09:33 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord bot has been disconnected for over 10 minutes without reconnecting. Exiting process to allow container restart.
Application is shutting down...
Starting Requestrr - build 'v2.1.9'
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
Hosting environment: Production
Content root path: /root
Now listening on: http://[::]:4545
Application started. Press Ctrl+C to shut down.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: 4000): 
[2026-05-14 08:09:40 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (4000, ''), reconnecting
fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord bot has been disconnected for over 10 minutes without reconnecting. Exiting process to allow container restart.
Application is shutting down...
Starting Requestrr - build 'v2.1.9'
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
Hosting environment: Production
Content root path: /root
Now listening on: http://[::]:4545
Application started. Press Ctrl+C to shut down.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-14 08:45:50 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord bot has been disconnected for over 10 minutes without reconnecting. Exiting process to allow container restart.
Application is shutting down...
Starting Requestrr - build 'v2.1.9'
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
Hosting environment: Production
Content root path: /root
Now listening on: http://[::]:4545
Application started. Press Ctrl+C to shut down.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-14 09:13:46 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
fail: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord bot has been disconnected for over 10 minutes without reconnecting. Exiting process to allow container restart.
Application is shutting down...
Starting Requestrr - build 'v2.1.9'
warn: Microsoft.AspNetCore.DataProtection.Repositories.FileSystemXmlRepository[60]
      Storing keys in a directory '/root/.aspnet/DataProtection-Keys' that may not be persisted outside of the container. Protected data will be unavailable when container is destroyed.
Hosting environment: Production
Content root path: /root
Now listening on: http://[::]:4545
Application started. Press Ctrl+C to shut down.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot configuration changed: restarting bot

It looks like things are restarting every 1-2 hours....I wonder if for some reason, a reconnect isn't noted and it's resetting too often....will have to look into this before knowing if it is OK to pull in...
Any ideas @tacoroumen?

thomst08 added 2 commits May 14, 2026 22:44
- Added check for heartbeat and to restart based off that last connected time
@thomst08
Copy link
Copy Markdown
Owner

I checked and tried to emulate a socket disconnect, I don't think there was a way to detect when a socket connection came back, so, if it failed once, it would restart in ten mins regardless it reconnected....
So I added a function to remove the time when it reconnects.

But I also found it does not handle when the heartbeat drops, so I added a check for that too and restarts when the heartbeat fails for 4 mins. This should help correct issues with internet dropouts.

@thomst08
Copy link
Copy Markdown
Owner

This is the log after running the bot for about 8 hours:

warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Bot has been restarted.
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-15 00:21:48 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 00:21:48)
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-15 02:26:46 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 02:26:47)
[2026-05-15 04:50:13 +10:00] [111 /ConnectionCl] [Crit ]warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: 4000): 
 Connection terminated (4000, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 04:50:14)
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: 4000): 
[2026-05-15 05:30:58 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (4000, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 05:30:58)
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-15 06:11:52 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 06:11:52)
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-15 06:33:00 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 06:33:01)
[2026-05-15 08:45:08 +10:00] [111 /ConnectionCl] [Crit ] Connection terminated (-1, ''), reconnecting
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket closed (code: -1): 
[2026-05-15 08:45:18 +10:00] [102 /ConnectionFa] [Error] Connection attempt failed, retrying in 7s
System.Net.WebSockets.WebSocketException (0x80004005): Unable to connect to the remote server
 ---> System.Net.Http.HttpRequestException: Resource temporarily unavailable (gateway.discord.gg:443)
 ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
   at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
   at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
   at System.Net.WebSockets.WebSocketHandle.ConnectAsync(Uri uri, CancellationToken cancellationToken, ClientWebSocketOptions options)
   at System.Net.WebSockets.ClientWebSocket.ConnectAsyncCore(Uri uri, CancellationToken cancellationToken)
   at DSharpPlus.Net.WebSocket.WebSocketClient.ConnectAsync(Uri uri)
   at DSharpPlus.DiscordClient.InternalConnectAsync()
   at DSharpPlus.DiscordClient.ConnectAsync(DiscordActivity activity, Nullable`1 status, Nullable`1 idlesince)
warn: Requestrr.WebApi.RequestrrBot.ChatBot[0]
      Discord socket reconnected (05/15/2026 08:45:38)

The bot appears to drop the socket and then reconnect, the correction to the code seems to correct fix the issue with not noticing a reconnect on the socket.
The heartbeat monitoring also appears to be stable.

- Changed socket logs to debug
@thomst08
Copy link
Copy Markdown
Owner

Happy with the changes made here, everything seems to be testing well.
Good work @tacoroumen :)

@thomst08 thomst08 merged commit 9a05b8a into thomst08:master May 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants