mirror of
https://github.com/outbackdingo/patroni.git
synced 2026-08-25 14:53:37 +00:00
Merge branch 'master' of github.com:zalando/patroni into feature/failover-switchover-definition
This commit is contained in:
Alexander Kukushkin
@@ -46,9 +46,9 @@ In order to change the dynamic configuration you can use either ``patronictl edi
|
||||
- **archive\_cleanup\_command**: cleanup command for standby leader
|
||||
- **recovery\_min\_apply\_delay**: how long to wait before actually apply WAL records on a standby leader
|
||||
|
||||
- **slots**: define permanent replication slots. These slots will be preserved during switchover/failover. The logical slots are copied from the primary to a standby with restart, and after that their position advanced every **loop_wait** seconds (if necessary). Copying logical slot files performed via ``libpq`` connection and using either rewind or superuser credentials (see **postgresql.authentication** section). There is always a chance that the logical slot position on the replica is a bit behind the former primary, therefore application should be prepared that some messages could be received the second time after the failover. The easiest way of doing so - tracking ``confirmed_flush_lsn``. Enabling permanent logical replication slots requires **postgresql.use_slots** to be set and will also automatically enable the ``hot_standby_feedback``. Since the failover of logical replication slots is unsafe on PostgreSQL 9.6 and older and PostgreSQL version 10 is missing some important functions, the feature only works with PostgreSQL 11+.
|
||||
- **slots**: define permanent replication slots. These slots will be preserved during switchover/failover. Permanent slots that don't exist will be created by Patroni. The physical slots are maintained only in the current primary. The logical slots are copied from the primary to a standby with restart, and after that their position advanced every **loop_wait** seconds (if necessary). Copying logical slot files performed via ``libpq`` connection and using either rewind or superuser credentials (see **postgresql.authentication** section). There is always a chance that the logical slot position on the replica is a bit behind the former primary, therefore application should be prepared that some messages could be received the second time after the failover. The easiest way of doing so - tracking ``confirmed_flush_lsn``. Enabling permanent logical replication slots requires **postgresql.use_slots** to be set and will also automatically enable the ``hot_standby_feedback``. Since the failover of logical replication slots is unsafe on PostgreSQL 9.6 and older and PostgreSQL version 10 is missing some important functions, the feature only works with PostgreSQL 11+.
|
||||
|
||||
- **my\_slot\_name**: the name of replication slot. If the permanent slot name matches with the name of the current primary it will not be created. Everything else is the responsibility of the operator to make sure that there are no clashes in names between replication slots automatically created by Patroni for members and permanent replication slots.
|
||||
- **my\_slot\_name**: the name of the permanent replication slot. If the permanent slot name matches with the name of the current leader it will not be created. Please note that Patroni does not make checks for permanent slot names added to this configuration matching those that Patroni creates automatically for members. If those names are added, Patroni will ensure that any slots that were created are not removed even if the member becomes unresponsive, situation which would normally result in the slot's removal by Patroni. Although this can be useful in some situations, such as when importing existing members to a new Patroni cluster (see :ref:`Convert a Standalone to a Patroni Cluster <existing_data>` for details), caution should be exercised by the operator that these clashes in names are not persisted in the DCS due to its effect on normal functioning of Patroni.
|
||||
|
||||
- **type**: slot type. Could be ``physical`` or ``logical``. If the slot is logical, you have to additionally define ``database`` and ``plugin``.
|
||||
- **database**: the database name where logical slots should be created.
|
||||
|
||||
+56
-16
@@ -10,18 +10,58 @@ To deploy a Patroni cluster without using a pre-existing PostgreSQL instance, se
|
||||
Procedure
|
||||
---------
|
||||
|
||||
A Patroni cluster can be started with a data directory from a single-node PostgreSQL database. This is achieved by following closely these steps:
|
||||
You can find below an overview of steps for converting an existing Postgres cluster to a Patroni managed cluster. In the steps we assume all nodes that are part of the existing cluster are currently up and running, and that you *do not* intend to change Postgres configuration while the migration is ongoing. The steps:
|
||||
|
||||
1. Manually start PostgreSQL daemon
|
||||
2. Create Patroni superuser and replication users as defined in the :ref:`authentication <postgresql_settings>` section of the Patroni configuration. If this user is created in SQL, the following queries achieve this:
|
||||
#. Create the Postgres users as explained for :ref:`authentication <postgresql_settings>` section of the Patroni configuration. You can find sample SQL commands to create the users in the code block below, in which you need to replace the usernames and passwords as per your environment. If you already have the relevant users, then you can skip this step.
|
||||
|
||||
.. code-block:: sql
|
||||
.. code-block:: sql
|
||||
|
||||
CREATE USER $PATRONI_SUPERUSER_USERNAME WITH SUPERUSER ENCRYPTED PASSWORD '$PATRONI_SUPERUSER_PASSWORD';
|
||||
CREATE USER $PATRONI_REPLICATION_USERNAME WITH REPLICATION ENCRYPTED PASSWORD '$PATRONI_REPLICATION_PASSWORD';
|
||||
-- Patroni superuser
|
||||
-- Replace PATRONI_SUPERUSER_USERNAME and PATRONI_SUPERUSER_PASSWORD accordingly
|
||||
CREATE USER PATRONI_SUPERUSER_USERNAME WITH SUPERUSER ENCRYPTED PASSWORD 'PATRONI_SUPERUSER_PASSWORD';
|
||||
|
||||
3. Start Patroni (e.g. ``patroni /etc/patroni/patroni.yml``). It automatically detects that PostgreSQL daemon is already running but its configuration might be out-of-date.
|
||||
4. Ask Patroni to restart the node with ``patronictl restart cluster-name node-name``. This step is only required if PostgreSQL configuration is out-of-date.
|
||||
-- Patroni replication user
|
||||
-- Replace PATRONI_REPLICATION_USERNAME and PATRONI_REPLICATION_PASSWORD accordingly
|
||||
CREATE USER PATRONI_REPLICATION_USERNAME WITH REPLICATION ENCRYPTED PASSWORD 'PATRONI_REPLICATION_PASSWORD';
|
||||
|
||||
-- Patroni rewind user, if you intend to enable use_pg_rewind in your Patroni configuration
|
||||
-- Replace PATRONI_REWIND_USERNAME and PATRONI_REWIND_PASSWORD accordingly
|
||||
CREATE USER PATRONI_REWIND_USERNAME WITH ENCRYPTED PASSWORD 'PATRONI_REWIND_PASSWORD';
|
||||
GRANT EXECUTE ON function pg_catalog.pg_ls_dir(text, boolean, boolean) TO PATRONI_REWIND_USERNAME;
|
||||
GRANT EXECUTE ON function pg_catalog.pg_stat_file(text, boolean) TO PATRONI_REWIND_USERNAME;
|
||||
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text) TO PATRONI_REWIND_USERNAME;
|
||||
GRANT EXECUTE ON function pg_catalog.pg_read_binary_file(text, bigint, bigint, boolean) TO PATRONI_REWIND_USERNAME;
|
||||
|
||||
#. Perform the following steps on all Postgres nodes. Perform all steps on one node before proceeding with the next node. Start with the primary node, then proceed with each standby node:
|
||||
|
||||
#. If you are running Postgres through systemd, then disable the Postgres systemd unit. This is performed as Patroni manages starting and stopping the Postgres daemon.
|
||||
|
||||
#. Create a YAML configuration file for Patroni.
|
||||
|
||||
* **Note (specific for the primary node):** If you have replication slots being used for replication between cluster members, then it is recommended that you enable ``use_slots`` and configure the existing replication slots as permanent via the ``slots`` configuration item. Be aware that Patroni automatically creates replication slots for replication between members, and drops replication slots that it does not recognize, when ``use_slots`` is enabled. The idea of using permanent slots here is to allow your existing slots to persist while the migration to Patroni is in progress. See :ref:`YAML Configuration Settings <yaml_configuration>` for details.
|
||||
|
||||
#. Start Patroni using the ``patroni`` systemd service unit. It automatically detects that Postgres is already running and starts monitoring the instance.
|
||||
|
||||
#. Hand over Postgres "start up procedure" to Patroni. In order to do that you need to restart the cluster members through ``patronictl restart cluster-name member-name`` command. For minimal downtime you might want to split this step into:
|
||||
|
||||
#. Immediate restart of the standby nodes.
|
||||
#. Scheduled restart of the primary node within a maintenance window.
|
||||
|
||||
#. If you configured permanent slots in step ``1.2.``, then you should remove them from ``slots`` configuration through ``patronictl edit-config cluster-name member-name`` command once the ``restart_lsn`` of the slots created by Patroni is able to catch up with the ``restart_lsn`` of the original slots for the corresponding members. By removing the slots from ``slots`` configuration you will allow Patroni to drop the original slots from your cluster once they are not needed anymore. You can find below an example query to check the ``restart_lsn`` of a couple slots, so you can compare them:
|
||||
|
||||
.. code-block:: sql
|
||||
|
||||
-- Assume original_slot_for_member_x is the name of the slot in your original
|
||||
-- cluster for replicating changes to member X, and slot_for_member_x is the
|
||||
-- slot created by Patroni for that purpose. You need restart_lsn of
|
||||
-- slot_for_member_x to be >= restart_lsn of original_slot_for_member_x
|
||||
SELECT slot_name,
|
||||
restart_lsn
|
||||
FROM pg_replication_slots
|
||||
WHERE slot_name IN (
|
||||
'original_slot_for_member_x',
|
||||
'slot_for_member_x'
|
||||
)
|
||||
|
||||
.. _major_upgrade:
|
||||
|
||||
@@ -30,14 +70,14 @@ Major Upgrade of PostgreSQL Version
|
||||
|
||||
The only possible way to do a major upgrade currently is:
|
||||
|
||||
1. Stop Patroni
|
||||
2. Upgrade PostgreSQL binaries and perform `pg_upgrade <https://www.postgresql.org/docs/current/pgupgrade.html>`_ on the primary node
|
||||
3. Update patroni.yml
|
||||
4. Remove the initialize key from DCS or wipe complete cluster state from DCS. The second one could be achieved by running ``patronictl remove <cluster-name>``. It is necessary because pg_upgrade runs initdb which actually creates a new database with a new PostgreSQL system identifier.
|
||||
5. If you wiped the cluster state in the previous step, you may wish to copy patroni.dynamic.json from old data dir to the new one. It will help you to retain some PostgreSQL parameters you had set before.
|
||||
6. Start Patroni on the primary node.
|
||||
7. Upgrade PostgreSQL binaries, update patroni.yml and wipe the data_dir on standby nodes.
|
||||
8. Start Patroni on the standby nodes and wait for the replication to complete.
|
||||
#. Stop Patroni
|
||||
#. Upgrade PostgreSQL binaries and perform `pg_upgrade <https://www.postgresql.org/docs/current/pgupgrade.html>`_ on the primary node
|
||||
#. Update patroni.yml
|
||||
#. Remove the initialize key from DCS or wipe complete cluster state from DCS. The second one could be achieved by running ``patronictl remove <cluster-name>``. It is necessary because pg_upgrade runs initdb which actually creates a new database with a new PostgreSQL system identifier.
|
||||
#. If you wiped the cluster state in the previous step, you may wish to copy patroni.dynamic.json from old data dir to the new one. It will help you to retain some PostgreSQL parameters you had set before.
|
||||
#. Start Patroni on the primary node.
|
||||
#. Upgrade PostgreSQL binaries, update patroni.yml and wipe the data_dir on standby nodes.
|
||||
#. Start Patroni on the standby nodes and wait for the replication to complete.
|
||||
|
||||
Running pg_upgrade on standby nodes is not supported by PostgreSQL. If you know what you are doing, you can try the rsync procedure described in https://www.postgresql.org/docs/current/pgupgrade.html instead of wiping data_dir on standby nodes. The safest way is however to let Patroni replicate the data for you.
|
||||
|
||||
|
||||
@@ -32,8 +32,11 @@ Configuration
|
||||
|
||||
Patroni Kubernetes :ref:`settings <kubernetes_settings>` and :ref:`environment variables <kubernetes_environment>` are described in the general chapters of the documentation.
|
||||
|
||||
.. _kubernetes_role_values:
|
||||
|
||||
Customize role label
|
||||
^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
By default, Patroni will set corresponding labels on the pod it runs in based on node's role, such as ``role=master``.
|
||||
The key and value of label can be customized by `kubernetes.role_label`, `kubernetes.leader_label_value`, `kubernetes.follower_label_value` and `kubernetes.standby_leader_label_value`.
|
||||
|
||||
|
||||
@@ -3,6 +3,84 @@
|
||||
Release notes
|
||||
=============
|
||||
|
||||
Version 3.1.0
|
||||
-------------
|
||||
|
||||
**Breaking changes**
|
||||
|
||||
- Changed semantic of ``restapi.keyfile`` and ``restapi.certfile`` (Alexander Kukushkin)
|
||||
|
||||
Previously Patroni was using ``restapi.keyfile`` and ``restapi.certfile`` as client certificates as a fallback if there were no respective configuration parameters in the ``ctl`` section.
|
||||
|
||||
.. warning::
|
||||
If you enabled client certificates validation (``restapi.verify_client`` is set to ``required``), you also **must** provide **valid client certificates** in the ``ctl.certfile``, ``ctl.keyfile``, ``ctl.keyfile_password``. If not provided, Patroni will not work correctly.
|
||||
|
||||
|
||||
**New features**
|
||||
|
||||
- Make Pod role label configurable (Waynerv)
|
||||
|
||||
Values could be customized using ``kubernetes.leader_label_value``, ``kubernetes.follower_label_value`` and ``kubernetes.standby_leader_label_value`` parameters. This feature will be very useful when we change the ``master`` role to the ``primary``. You can read more about the feature and migration steps :ref:`here <kubernetes_role_values>`.
|
||||
|
||||
|
||||
**Improvements**
|
||||
|
||||
- Various improvements of ``patroni --validate-config`` (Alexander Kukushkin)
|
||||
|
||||
Improved parameter validation for different DCS, ``bootstrap.dcs`` , ``ctl``, ``restapi``, and ``watchdog`` sections.
|
||||
|
||||
- Start Postgres not in recovery if it crashed during recovery while Patroni is running (Alexander Kukushkin)
|
||||
|
||||
It may reduce recovery time and will help to prevent unnecessary timeline increments.
|
||||
|
||||
- Avoid unnecessary updates of ``/status`` key (Alexander Kukushkin)
|
||||
|
||||
When there are no permanent logical slots Patroni was updating the ``/status`` on every heartbeat loop even when LSN on the primary didn't move forward.
|
||||
|
||||
- Don't allow stale primary to win the leader race (Alexander Kukushkin)
|
||||
|
||||
If Patroni was hanging during a significant time due to lack of resources it will additionally check that no other nodes promoted Postgres before acquiring the leader lock.
|
||||
|
||||
- Implemented visibility of certain PostgreSQL parameters validation (Alexander Kukushkin, Feike Steenbergen)
|
||||
|
||||
If validation of ``max_connections``, ``max_wal_senders``, ``max_prepared_transactions``, ``max_locks_per_transaction``, ``max_replication_slots``, or ``max_worker_processes`` failed Patroni was using some sane default value. Now in addition to that it will also show a warning.
|
||||
|
||||
- Set permissions for files and directories created in ``PGDATA`` (Alexander Kukushkin)
|
||||
|
||||
All files created by Patroni had only owner read/write permissions. This behaviour was breaking backup tools that run under a different user and relying on group read permissions. Now Patroni honors permissions on ``PGDATA`` and correctly sets permissions on all directories and files it creates inside ``PGDATA``.
|
||||
|
||||
|
||||
**Bugfixes**
|
||||
|
||||
- Run ``archive_command`` through shell (Waynerv)
|
||||
|
||||
Patroni might archive some WAL segments before doing crash recovery in a single-user mode or before ``pg_rewind``. If the archive_command contains some shell operators, like ``&&`` it didn't work with Patroni.
|
||||
|
||||
- Fixed "on switchover" shutdown checks (Polina Bungina)
|
||||
|
||||
It was possible that specified candidate is still streaming and didn't received shut down checking but the leader key was removed because some other nodes were healthy.
|
||||
|
||||
- Fixed "is primary" check (Alexander Kukushkin)
|
||||
|
||||
During the leader race replicas were not able to recognize that Postgres on the old leader is still running as a primary.
|
||||
|
||||
- Fixed ``patronictl list`` (Alexander Kukushkin)
|
||||
|
||||
The Cluster name field was missing in ``tsv``, ``json``, and ``yaml`` output formats.
|
||||
|
||||
- Fixed ``pg_rewind`` behaviour after pause (Alexander Kukushkin)
|
||||
|
||||
Under certain conditions, Patroni wasn't able to join the false primary back to the cluster with ``pg_rewind`` after coming out of maintenance mode.
|
||||
|
||||
- Fixed bug in Etcd v3 implementation (Alexander Kukushkin)
|
||||
|
||||
Invalidate internal KV cache if key update performed using ``create_revision``/``mod_revision`` field due to revision mismatch.
|
||||
|
||||
- Fixed behaviour of replicas in standby cluster in pause (Alexander Kukushkin)
|
||||
|
||||
When the leader key expires replicas in standby cluster will not follow the remote node but keep ``primary_conninfo`` as it is.
|
||||
|
||||
|
||||
Version 3.0.4
|
||||
-------------
|
||||
|
||||
|
||||
Reference in New Issue
Block a user