Register Citus secondaries in pg_dist_node

1. All nodes with role == 'replica' and state == 'running' are
   are registered. In case is state isn't running the node is removed.
2. In case of failover/switchover we always first update the primary
3. When switching to a registered secondary we call citus_update_node()
   three times: rename primary to primary-demoted, put the primary name
   to a promoted secondary row and put the promoted secondary name to
   the primary row

State transitions are produced by the transition() method. First of all
the method makes sure that the actual primary is registered in the
metadata. In case if for a given group the primary didn't change, the
method registers new secondaries and removes secondaries that are gone.
It prefers to use citus_update_node() UDF to replace gone secondaries
with added.

Communication protocol between primary nodes remains the same and all
old features work without any changes.
This commit is contained in:
Alexander Kukushkin
2023-07-13 15:14:07 +02:00
parent a4d29eb99e
commit 9ba40f0a25
4 changed files with 638 additions and 109 deletions
+7
View File
@@ -11,7 +11,9 @@ Feature: citus
Then replication works from postgres0 to postgres1 after 15 seconds
Then replication works from postgres2 to postgres3 after 15 seconds
And postgres0 is registered in the postgres0 as the primary in group 0 after 5 seconds
And postgres1 is registered in the postgres0 as the secondary in group 0 after 5 seconds
And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres3 is registered in the postgres0 as the secondary in group 1 after 5 seconds
Scenario: coordinator failover updates pg_dist_node
Given I run patronictl.py failover batman --group 0 --candidate postgres1 --force
@@ -19,11 +21,13 @@ Feature: citus
And "members/postgres0" key in a group 0 in DCS has state=running after 15 seconds
And replication works from postgres1 to postgres0 after 15 seconds
And postgres1 is registered in the postgres2 as the primary in group 0 after 5 seconds
And postgres0 is registered in the postgres2 as the secondary in group 0 after 15 seconds
And "sync" key in a group 0 in DCS has sync_standby=postgres0 after 15 seconds
When I run patronictl.py switchover batman --group 0 --candidate postgres0 --force
Then postgres0 role is the primary after 10 seconds
And replication works from postgres0 to postgres1 after 15 seconds
And postgres0 is registered in the postgres2 as the primary in group 0 after 5 seconds
And postgres1 is registered in the postgres2 as the secondary in group 0 after 15 seconds
And "sync" key in a group 0 in DCS has sync_standby=postgres1 after 15 seconds
Scenario: worker switchover doesn't break client queries on the coordinator
@@ -35,6 +39,7 @@ Feature: citus
And "members/postgres2" key in a group 1 in DCS has state=running after 15 seconds
And replication works from postgres3 to postgres2 after 15 seconds
And postgres3 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres2 is registered in the postgres0 as the secondary in group 1 after 15 seconds
And "sync" key in a group 1 in DCS has sync_standby=postgres2 after 15 seconds
And a thread is still alive
When I run patronictl.py switchover batman --group 1 --force
@@ -42,6 +47,7 @@ Feature: citus
And postgres2 role is the primary after 10 seconds
And replication works from postgres2 to postgres3 after 15 seconds
And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres3 is registered in the postgres0 as the secondary in group 1 after 15 seconds
And "sync" key in a group 1 in DCS has sync_standby=postgres3 after 15 seconds
And a thread is still alive
When I stop a thread
@@ -55,6 +61,7 @@ Feature: citus
And postgres2 role is the primary after 10 seconds
And replication works from postgres2 to postgres3 after 15 seconds
And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
And postgres3 is registered in the postgres0 as the secondary in group 1 after 15 seconds
And a thread is still alive
When I stop a thread
Then a distributed table on postgres0 has expected rows