Register Citus secondaries in pg_dist_node (#2755)

1. All nodes with role == 'replica' and state == 'running' are are registered. In case is state isn't running the node is removed.
2. In case of failover/switchover we always first update the primary
3. When switching to a registered secondary we call citus_update_node() three times: rename primary to primary-demoted, put the primary name to a promoted secondary row and put the promoted secondary name to the primary row

State transitions are produced by the transition() method. First of all the method makes sure that the actual primary is registered in the metadata. In case if for a given group the primary didn't change, the method registers new secondaries and removes secondaries that are gone. It prefers to use citus_update_node() UDF to replace gone secondaries with added.

Communication protocol between primary nodes remains the same and all old features work without any changes.
This commit is contained in:
Alexander Kukushkin
2024-08-13 09:12:03 +02:00
committed by GitHub
parent 384705ad97
commit 0fa41502f1
4 changed files with 664 additions and 119 deletions
+7
View File
@@ -11,7 +11,9 @@ Feature: citus
Then replication works from postgres0 to postgres1 after 15 seconds
Then replication works from postgres2 to postgres3 after 15 seconds
And postgres0 is registered in the postgres0 as the primary in group 0 after 5 seconds
And postgres1 is registered in the postgres0 as the secondary in group 0 after 5 seconds
And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres3 is registered in the postgres0 as the secondary in group 1 after 5 seconds
Scenario: coordinator failover updates pg_dist_node
Given I run patronictl.py failover batman --group 0 --candidate postgres1 --force
@@ -19,11 +21,13 @@ Feature: citus
And "members/postgres0" key in a group 0 in DCS has state=running after 15 seconds
And replication works from postgres1 to postgres0 after 15 seconds
And postgres1 is registered in the postgres2 as the primary in group 0 after 5 seconds
And postgres0 is registered in the postgres2 as the secondary in group 0 after 15 seconds
And "sync" key in a group 0 in DCS has sync_standby=postgres0 after 15 seconds
When I run patronictl.py switchover batman --group 0 --candidate postgres0 --force
Then postgres0 role is the primary after 10 seconds
And replication works from postgres0 to postgres1 after 15 seconds
And postgres0 is registered in the postgres2 as the primary in group 0 after 5 seconds
And postgres1 is registered in the postgres2 as the secondary in group 0 after 15 seconds
And "sync" key in a group 0 in DCS has sync_standby=postgres1 after 15 seconds
Scenario: worker switchover doesn't break client queries on the coordinator
@@ -35,6 +39,7 @@ Feature: citus
And "members/postgres2" key in a group 1 in DCS has state=running after 15 seconds
And replication works from postgres3 to postgres2 after 15 seconds
And postgres3 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres2 is registered in the postgres0 as the secondary in group 1 after 15 seconds
And "sync" key in a group 1 in DCS has sync_standby=postgres2 after 15 seconds
And a thread is still alive
When I run patronictl.py switchover batman --group 1 --force
@@ -42,6 +47,7 @@ Feature: citus
And postgres2 role is the primary after 10 seconds
And replication works from postgres2 to postgres3 after 15 seconds
And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres3 is registered in the postgres0 as the secondary in group 1 after 15 seconds
And "sync" key in a group 1 in DCS has sync_standby=postgres3 after 15 seconds
And a thread is still alive
When I stop a thread
@@ -55,6 +61,7 @@ Feature: citus
And postgres2 role is the primary after 10 seconds
And replication works from postgres2 to postgres3 after 15 seconds
And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres3 is registered in the postgres0 as the secondary in group 1 after 15 seconds
And a thread is still alive
When I stop a thread
Then a distributed table on postgres0 has expected rows