RDSC-4633 How to perform HA failover#2818
Open
ilianiliev-redis wants to merge 1 commit intoredis:mainfrom
Open
RDSC-4633 How to perform HA failover#2818ilianiliev-redis wants to merge 1 commit intoredis:mainfrom
ilianiliev-redis wants to merge 1 commit intoredis:mainfrom
Conversation
ilianiliev-redis
force-pushed
the
RDSC-4633-ha-failover-setup
branch
from
dcdc014 to
7cba538
Compare
andy-stark-redis
approved these changes
Feb 27, 2026
Contributor
andy-stark-redis
left a comment
andy-stark-redis
left a comment
There was a problem hiding this comment.
Just a few style suggestions, but otherwise LGTM.
| rdi-reloader-77df5f7854-lwmvz 1/1 Running 0 71m | ||
| ``` | ||
|
|
||
| 2. Identify the leader node - this is the one that has a running `collector-source` pod |
Contributor
There was a problem hiding this comment.
Suggested change
| 2. Identify the leader node - this is the one that has a running `collector-source` pod | |
| 2. Identify the leader node - this is the one that has a running `collector-source` pod. |
|
|
||
| To perform HA, you can simulate a connection failure between the leader and the RDI database by blocking the network traffic. You can do this by running the following command on the leader node: | ||
|
|
||
| 1. Identify the database IP (if you are using it with hostname): |
Contributor
There was a problem hiding this comment.
Suggested change
| 1. Identify the database IP (if you are using it with hostname): | |
| 1. Identify the database IP (replace `<hostname>` with your own hostname): |
|
|
||
| ## Performing the HA Failover Testing | ||
|
|
||
| To perform HA, you can simulate a connection failure between the leader and the RDI database by blocking the network traffic. You can do this by running the following command on the leader node: |
Contributor
There was a problem hiding this comment.
Suggested change
| To perform HA, you can simulate a connection failure between the leader and the RDI database by blocking the network traffic. You can do this by running the following command on the leader node: | |
| To perform HA, you can simulate a connection failure between the leader and the RDI database by blocking the network traffic. You can do this by running the following commands on the leader node: |
| 54.78.220.161 | ||
| ``` | ||
|
|
||
| 2. For each of the IPs returned by the above command, run the following command to block the traffic: |
Contributor
There was a problem hiding this comment.
Does this mean that you expect dig to return more than one IP address for a hostname? (Presumably you only need to run the command once on the leader node.) If so, maybe say that explicitly in step 1, because it currently says "Identify the database IP", which makes it sound like there is only one address, but "IP" might potentially be plural here.
Comment on lines
+73
to
+74
| In about 10 second you will start seeing logs from the leader that it could not acquire the leadership. | ||
| Once the leader lock expires, the second node will acquire the leadership and you will see logs from the second node indicating that it has become the leader. |
Contributor
There was a problem hiding this comment.
Suggested change
| In about 10 second you will start seeing logs from the leader that it could not acquire the leadership. | |
| Once the leader lock expires, the second node will acquire the leadership and you will see logs from the second node indicating that it has become the leader. | |
| In about 10 seconds you will start seeing log entries from the leader saying that it could not acquire the leadership. | |
| When the leader lock expires, the second node will acquire the leadership and you will see log entries from the second node indicating that it has become the leader. |
|
|
||
| ## Cleanup | ||
|
|
||
| To clean up after the test, you can remove the iptables rule that you added to block the traffic: |
Contributor
There was a problem hiding this comment.
Suggested change
| To clean up after the test, you can remove the iptables rule that you added to block the traffic: | |
| To clean up after the test, remove the `iptables` rule that you added to block the traffic: |
| @@ -0,0 +1,82 @@ | |||
| --- | |||
| Title: How to perform HA failover testing | |||
Contributor
There was a problem hiding this comment.
Your original title was fine, but this fits our usual style a bit more closely.
Suggested change
| Title: How to perform HA failover testing | |
| Title: Test HA failover |
| description: Learn how to perform HA failover testing for Redis Data Integration (RDI) to ensure high availability and reliability of your data integration setup. | ||
| group: di | ||
| hideListLinks: false | ||
| linkTitle: Testing HA failover |
Contributor
There was a problem hiding this comment.
Suggested change
| linkTitle: Testing HA failover | |
| linkTitle: Test HA failover |
| will take place. After the failover, the secondary instance will become the primary one, | ||
| and the RDI pipeline will be active on that VM. | ||
|
|
||
| You can see how to test HA failover in the [HA failover testing page]({{< relref "/integrate/redis-data-integration/installation/ha-test" >}}). |
Contributor
There was a problem hiding this comment.
Suggested change
| You can see how to test HA failover in the [HA failover testing page]({{< relref "/integrate/redis-data-integration/installation/ha-test" >}}). | |
| You may find it useful to trigger a failover deliberately to check that RDI is correctly configured to handle it. See [Test HA failover]({{< relref "/integrate/redis-data-integration/installation/ha-test" >}}) to learn how to do this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: https://redislabs.atlassian.net/browse/RDSC-4633
Document how to perform an HA failover test.