gitops/patroni
1
0
Fork 0
mirror of https://github.com/outbackdingo/patroni.git synced 2026-08-25 14:53:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Start postgres not in recovery in some cases (#2726)

Browse Source 
If we know for sure that a few moments ago postgres was still running as a primary and we still have the leader lock and can successfully update it, in this case we can safely start postgres back not in recovery. That will allow to avoid bumping timeline without a reason and hopefully improve reliability because it will address issues similar to #2720.

In addition to that remove `if self.state_handler.is_starting()` check from the `recover()` method. This branch could never be reached because the `starting` state is handled earlier in the `_run_cycle()`. Besides that remove redundant `self._crash_recovery_executed`.

P.S. now we do not cover cases when Patroni was killed along with Postgres.
Lets consider that we just started Patroni, there is no leader, and `pg_controldata` reports `Database cluster state` as `shut down`. It feels logical to use `Latest checkpoint location` and `Latest checkpoint's TimeLineID` to do a usual leader race and start directly as a primary, but it could be totally wrong. The thing is that we run `postgres --single` if standby wasn't shut down cleanly before executing `pg_rewind`. As a result `Database cluster state` transition from `in archive recovery` to `shut down`, but if such a node becomes a leader the timeline must be increased.
This commit is contained in:
Alexander Kukushkin
2023-07-12 09:42:34 +02:00
committed by GitHub
parent b8cff3515a
commit 6e96db173f
4 changed files with 71 additions and 19 deletions
Download Patch File Download Diff File Expand all files Collapse all files
features/basic_replication.feature
+1 -6
View File
@@ -72,16 +72,11 @@ Feature: basic replication
Then table bar is present on postgres1 after 20 seconds
And Response on GET http://127.0.0.1:8010/config contains master_start_timeout after 10 seconds
Scenario: check immediate failover when master_start_timeout=0
Given I kill postmaster on postgres2
Then postgres1 is a leader after 10 seconds
And postgres1 role is the primary after 10 seconds
Scenario: check rejoin of the former primary with pg_rewind
Given I add the table splitbrain to postgres0
And I start postgres0
Then postgres0 role is the secondary after 20 seconds
When I add the table buz to postgres1
When I add the table buz to postgres2
Then table buz is present on postgres0 after 20 seconds
Scenario: check graceful rejection when two nodes have the same name
features/recovery.feature
+24
View File
@@ -0,0 +1,24 @@
Feature: recovery
We want to check that crashed postgres is started back
Scenario: check that timeline is not incremented when primary is started after crash
Given I start postgres0
Then postgres0 is a leader after 10 seconds
And there is a non empty initialize key in DCS after 15 seconds
When I start postgres1
And I add the table foo to postgres0
Then table foo is present on postgres1 after 20 seconds
When I kill postmaster on postgres0
Then postgres0 role is the primary after 10 seconds
When I issue a GET request to http://127.0.0.1:8008/
Then I receive a response code 200
And I receive a response role master
And I receive a response timeline 1
Scenario: check immediate failover when master_start_timeout=0
Given I issue a PATCH request to http://127.0.0.1:8008/config with {"master_start_timeout": 0}
Then I receive a response code 200
And Response on GET http://127.0.0.1:8008/config contains master_start_timeout after 10 seconds
When I kill postmaster on postgres0
Then postgres1 is a leader after 10 seconds
And postgres1 role is the primary after 10 seconds