gitops/patroni
1
0
Fork 0
mirror of https://github.com/outbackdingo/patroni.git synced 2026-08-25 14:53:37 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use target_session_attrs only when multiple hosts in standby_cluster (#3040)

Browse Source 
Actually comment in the code was already saying that, but on practice it didn't happen.

It should help #3039
This commit is contained in:
Alexander Kukushkin
2024-04-02 11:59:57 +02:00
committed by GitHub
parent ceb2965ab8
commit d7454f7bcd
7 changed files with 87 additions and 63 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/index.rst
+1
View File
@@ -28,6 +28,7 @@ Currently supported PostgreSQL versions: 9.3 to 16.
patronictl
replica_bootstrap
replication_modes
standby_cluster
watchdog
pause
dcs_failsafe_mode
docs/replica_bootstrap.rst
-56
View File
@@ -219,59 +219,3 @@ and
- waldir: /pg-wal-mount/external-waldir
If all replica creation methods fail, Patroni will try again all methods in order during the next event loop cycle.
.. _standby_cluster:
Standby cluster
---------------
Another available option is to run a "standby cluster", that contains only of
standby nodes replicating from some remote node. This type of clusters has:
* "standby leader", that behaves pretty much like a regular cluster leader,
except it replicates from a remote node.
* cascade replicas, that are replicating from standby leader.
Standby leader holds and updates a leader lock in DCS. If the leader lock
expires, cascade replicas will perform an election to choose another leader
from the standbys.
There is no further relationship between the standby cluster and the primary
cluster it replicates from, in particular, they must not share the same DCS
scope if they use the same DCS. They do not know anything else from each other
apart from replication information. Also, the standby cluster is not being
displayed in :ref:`patronictl_list` or :ref:`patronictl_topology` output on the
primary cluster.
For the sake of flexibility, you can specify methods of creating a replica and
recovery WAL records when a cluster is in the "standby mode" by providing
`create_replica_methods` key in `standby_cluster` section. It is distinct from
creating replicas, when cluster is detached and functions as a normal cluster,
which is controlled by `create_replica_methods` in `postgresql` section. Both
"standby" and "normal" `create_replica_methods` reference keys in `postgresql`
section.
To configure such cluster you need to specify the section ``standby_cluster``
in a patroni configuration:
.. code:: YAML
bootstrap:
dcs:
standby_cluster:
host: 1.2.3.4
port: 5432
primary_slot_name: patroni
create_replica_methods:
- basebackup
Note, that these options will be applied only once during cluster bootstrap,
and the only way to change them afterwards is through DCS.
Patroni expects to find `postgresql.conf` or `postgresql.conf.backup` in PGDATA
of the remote primary and will not start if it does not find it after a
basebackup. If the remote primary keeps its `postgresql.conf` elsewhere, it is
your responsibility to copy it to PGDATA.
If you use replication slots on the standby cluster, you must also create the corresponding replication slot on the primary cluster. It will not be done automatically by the standby cluster implementation. You can use Patroni's permanent replication slots feature on the primary cluster to maintain a replication slot with the same name as ``primary_slot_name``, or its default value if ``primary_slot_name`` is not provided.
docs/standby_cluster.rst
+78
View File
@@ -0,0 +1,78 @@
.. _standby_cluster:
Standby cluster
---------------
Patroni also support running cascading replication to a remote datacenter
(region) using a feature that is called "standby cluster". This type of
clusters has:
* "standby leader", that behaves pretty much like a regular cluster leader,
except it replicates from a remote node.
* cascade replicas, that are replicating from standby leader.
Standby leader holds and updates a leader lock in DCS. If the leader lock
expires, cascade replicas will perform an election to choose another leader
from the standbys.
There is no further relationship between the standby cluster and the primary
cluster it replicates from, in particular, they must not share the same DCS
scope if they use the same DCS. They do not know anything else from each other
apart from replication information. Also, the standby cluster is not being
displayed in :ref:`patronictl_list` or :ref:`patronictl_topology` output on the
primary cluster.
For the sake of flexibility, you can specify methods of creating a replica and
recovery WAL records when a cluster is in the "standby mode" by providing
:ref:`create_replica_methods <custom_replica_creation>` key in
`standby_cluster` section. It is distinct from creating replicas, when cluster
is detached and functions as a normal cluster, which is controlled by
`create_replica_methods` in `postgresql` section. Both "standby" and "normal"
`create_replica_methods` reference keys in `postgresql` section.
To configure such cluster you need to specify the section ``standby_cluster``
in a patroni configuration:
.. code:: YAML
bootstrap:
dcs:
standby_cluster:
host: 1.2.3.4
port: 5432
primary_slot_name: patroni
create_replica_methods:
- basebackup
Note, that these options will be applied only once during cluster bootstrap,
and the only way to change them afterwards is through DCS.
Patroni expects to find `postgresql.conf` or `postgresql.conf.backup` in PGDATA
of the remote primary and will not start if it does not find it after a
basebackup. If the remote primary keeps its `postgresql.conf` elsewhere, it is
your responsibility to copy it to PGDATA.
If you use replication slots on the standby cluster, you must also create the
corresponding replication slot on the primary cluster. It will not be done
automatically by the standby cluster implementation. You can use Patroni's
permanent replication slots feature on the primary cluster to maintain a
replication slot with the same name as ``primary_slot_name``, or its default
value if ``primary_slot_name`` is not provided.
In case the remote site doesn't provide a single endpoint that connects to a
primary, one could list all hosts of the source cluster in the
``standby_cluster.host`` section. When ``standby_cluster.host`` contains
multiple hosts separated by commas, Patroni will:
* add ``target_session_attrs=read-write`` to the ``primary_conninfo`` on the
standby leader node.
* use ``target_session_attrs=read-write`` when trying to determine whether we
need to run ``pg_rewind`` or when executing ``pg_rewind`` on all nodes of the
standby cluster.
There is also a possibility to replicate the standby cluster from another
standby cluster or from a standby member of the primary cluster: for that, you
need to define a single host in the ``standby_cluster.host`` section. However,
you need to beware that in this case ``pg_rewind`` will fail to execute on the
standby cluster.
patroni/postgresql/config.py
+1 -2
View File
@@ -639,8 +639,7 @@ class ConfigHandler(object):
# We are a standby leader and are using a replication slot. Make sure we connect to
# the leader of the main cluster (in case more than one host is specified in the
# connstr) by adding 'target_session_attrs=read-write' to primary_conninfo.
if is_remote_member and 'target_sesions_attrs' not in primary_conninfo and\
self._postgresql.major_version >= 100000:
if is_remote_member and ',' in primary_conninfo['host'] and self._postgresql.major_version >= 100000:
primary_conninfo['target_session_attrs'] = 'read-write'
recovery_params['primary_conninfo'] = primary_conninfo
patroni/postgresql/rewind.py
+4 -3
View File
@@ -209,9 +209,10 @@ class Rewind(object):
ret = member.conn_kwargs(auth)
if not ret.get('dbname'):
ret['dbname'] = self._postgresql.database
# Add target_session_attrs in case more than one hostname is specified
# (libpq client-side failover) making sure we hit the primary
if 'target_session_attrs' not in ret and self._postgresql.major_version >= 100000:
# Add target_session_attrs to make sure we hit the primary.
# It is not strictly necessary for starting from PostgreSQL v14, which made it possible
# to rewind from standby, but doing it from the real primary is always safer.
if self._postgresql.major_version >= 100000:
ret['target_session_attrs'] = 'read-write'
return ret
tests/test_postgresql.py
+1 -1
View File
@@ -364,7 +364,7 @@ class TestPostgresql(BaseTestPostgresql):
@patch.object(Postgresql, 'start', Mock())
def test_follow(self):
self.p.call_nowait(CallbackAction.ON_START)
m = RemoteMember('1', {'restore_command': '2', 'primary_slot_name': 'foo', 'conn_kwargs': {'host': 'bar'}})
m = RemoteMember('1', {'restore_command': '2', 'primary_slot_name': 'foo', 'conn_kwargs': {'host': 'foo,bar'}})
self.p.follow(m)
with patch.object(Postgresql, 'ensure_major_version_is_known', Mock(return_value=False)):
self.assertIsNone(self.p.follow(m))
tests/test_rewind.py
+2 -1
View File
@@ -98,7 +98,8 @@ class TestRewind(BaseTestPostgresql):
self.r.rewind_or_reinitialize_needed_and_possible(self.leader)
@patch.object(CancellableSubprocess, 'call', mock_cancellable_call)
@patch.object(Postgresql, 'checkpoint', side_effect=['', '1'],)
@patch.object(Postgresql, 'get_guc_value', Mock(return_value=''))
@patch.object(Postgresql, 'checkpoint', side_effect=['', '1'])
@patch.object(Postgresql, 'stop', Mock(return_value=False))
@patch.object(Postgresql, 'start', Mock())
def test_execute(self, mock_checkpoint):