diff --git a/.github/workflows/tests.yaml b/.github/workflows/tests.yaml index dacfb0fb..2056d492 100644 --- a/.github/workflows/tests.yaml +++ b/.github/workflows/tests.yaml @@ -186,7 +186,7 @@ jobs: - uses: jakebailey/pyright-action@v2 with: - version: 1.1.371 + version: 1.1.378 docs: runs-on: ubuntu-latest diff --git a/docs/patronictl.rst b/docs/patronictl.rst index 938b30e1..7aefae8b 100644 --- a/docs/patronictl.rst +++ b/docs/patronictl.rst @@ -1940,25 +1940,25 @@ Get version of ``patronictl`` only: .. code:: bash $ patronictl -c postgres0.yml version - patronictl version 3.1.0 + patronictl version 4.0.0 Get version of ``patronictl`` and of all members of cluster ``batman``: .. code:: bash $ patronictl -c postgres0.yml version batman - patronictl version 3.1.0 + patronictl version 4.0.0 - postgresql0: Patroni 3.1.0 PostgreSQL 15.2 - postgresql1: Patroni 3.1.0 PostgreSQL 15.2 - postgresql2: Patroni 3.1.0 PostgreSQL 15.2 + postgresql0: Patroni 4.0.0 PostgreSQL 16.4 + postgresql1: Patroni 4.0.0 PostgreSQL 16.4 + postgresql2: Patroni 4.0.0 PostgreSQL 16.4 Get version of ``patronictl`` and of members ``postgresql1`` and ``postgresql2`` of cluster ``batman``: .. code:: bash $ patronictl -c postgres0.yml version batman postgresql1 postgresql2 - patronictl version 3.1.0 + patronictl version 4.0.0 - postgresql1: Patroni 3.1.0 PostgreSQL 15.2 - postgresql2: Patroni 3.1.0 PostgreSQL 15.2 + postgresql1: Patroni 4.0.0 PostgreSQL 16.4 + postgresql2: Patroni 4.0.0 PostgreSQL 16.4 diff --git a/docs/releases.rst b/docs/releases.rst index a3d06390..7fab4249 100644 --- a/docs/releases.rst +++ b/docs/releases.rst @@ -3,6 +3,98 @@ Release notes ============= +Version 4.0.0 +------------- + +Released 2024-08-29 + +.. warning:: + - This version completes work on getting rid of the "master" term, in favor of "primary". This means a couple of breaking changes, please read the release notes carefully. Upgrading to the Patroni 4+ will work reliably only if you run Patroni 3.1.0 or newer. Upgrading from an older version directly to 4+ is possible but may lead to unexpected behavior if the primary fails while the rest of the nodes are running on other Patroni versions. + + +**Breaking changes** + +- The following breaking changes were introduced when getting rid of the non-inclusive "master" term in the Patroni code: + + - On Kubernetes, Patroni by default will set ``role`` label to ``primary``. In case if you want to keep the old behavior and avoid downtime or lengthy complex migrations, you can configure parameters ``kubernetes.leader_label_value`` and ``kubernetes.standby_leader_label_value`` to ``master``. Read more :ref:`here `. + - Patroni role is written to DCS as ``primary`` instead of ``master``. + - Patroni role returned by Patroni REST API has been changed from ``master`` to ``primary``. + - Patroni REST API no longer accepts ``role=master`` in requests to ``/switchover``, ``/failover``, ``/restart`` endpoints. + - ``/metrics`` REST API endpoint will no longer report ``patroni_master`` metric. + - ``patronictl`` no longer accepts ``--master`` option for any command. ``--leader`` or ``--primary`` options should be used instead. + - ``no_master`` option in the declarative configuration of custom replica creation methods is no longer treated as a special option, please use ``no_leader`` instead. + - ``patroni_wale_restore`` script doesn't accept ``--no_master`` option anymore. + - ``patroni_barman`` script doesn't accept ``--role=master`` option anymore. + - All callback scripts are executed with ``role=primary`` option passed instead of ``role=master``. + +- ``patronictl failover`` does not accept ``--leader`` option that was deprecated since Patroni 3.2.0. + +- User creation functionality (``bootstrap.users`` configuration section) deprecated since Patroni 3.2.0 has been removed. + + +**New features** + +- Quorum-based failover (Ants Aasma, Alexander Kukushkin) + + The feature implements quorum-based synchronous replication (available from PostgreSQL v10) which helps to reduce worst-case latencies, even during normal operation, as a higher latency of replicating to one standby can be compensated by other standbys. Patroni implements additional safeguards to prevent any user-visible data loss by choosing a failover candidate based on the latest transaction received. + +- Register Citus secondaries in ``pg_dist_node`` (Alexander Kukushkin) + + Patroni now maintains the list of nodes with ``role==replica``, ``state==running`` and without ``noloadbalance`` :ref:`tag ` in ``pg_dist_node``. + +- Configurable retention of members' replication slots (Alexander Kukushkin) + + Implements support of ``member_slots_ttl`` global configuration parameter that controls for how long member replication slots should be kept around when the member key is absent. + +- Make permissions of log files created by Patroni configurable (Alexander Kukushkin) + + Allows to set specific permissions for log files created by Patroni. If not specified, permissions are set based on the current ``umask`` value. + +- Compatibility with PostgreSQL 17 beta3 (Alexander Kukushkin) + + GUC's validator rules were extended. Patroni handles all the new auxiliary backends during shutdown and sets ``dbname`` in ``primary_conninfo``, as it is required for logical replication slots synchronization. + +- Implement ``--ignore-listen-port`` option for Patroni config validation (Sahil Naphade) + + Make it possible to ignore already bound ports when running ``patroni --validate-config``. + + +**Improvements** + +- Make ``wal_log_hints`` configurable (Paul_Kim) + + Allows to avoid the overhead of ``wal_log_hints`` configuration being enabled in case ``use_pg_rewind`` is set to ``off``. + +- Log ``pg_basebackup`` command in ``DEBUG`` level (Waynerv) + + Facilitates failed initialization debugging. + + +**Bugfixes** + +- Advance permanent slots for cascading nodes while in failsafe (Alexander Kukushkin) + + Ensure that slots for cascading replicas are properly advanced on the primary when failsafe mode is activated. It is done by extending replicas response on ``POST /failsafe`` REST API request with their ``xlog_location``. + +- Don't let the current node be chosen as synchronous (Alexander Kukushkin) + + There may be "something" streaming from the current primary node with ``application_name`` that matches the name of the current primary. Patroni was not properly handling this situation, which could end up in the primary being declared as a synchronous node and consequently was blocking switchovers. + +- Ignore ``restapi.allowlist_include_members`` for POST /failsafe (Alexander Kukushkin) + +- Improve GUCs validation (Polina Bungina) + + Due to additional validation through running ``postgres --describe-config`` command, it was previously not possible to set GUCs not listed there through Patroni configuration. This limitation is now removed. + +- Add line with ``localhost`` to ``.pgpass`` file when unix sockets are detected (Alexander Kukushkin) + + Patroni will add an additional line to ``.pgpass`` file if ``host`` parameter specified starts with ``/`` character. This allows to cover a corner case when ``host`` matches the default socket directory path. + +- Fix logging issues (Waynerv) + + Defined proper request URL in failsafe handling logs and fixed the order of timestamps in postmaster check log. + + Version 3.3.2 ------------- diff --git a/docs/rest_api.rst b/docs/rest_api.rst index 37e51dc3..f97bd170 100644 --- a/docs/rest_api.rst +++ b/docs/rest_api.rst @@ -103,7 +103,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be $ curl -s http://localhost:8008/patroni | jq . { "state": "running", - "postmaster_start_time": "2024-08-18 11:03:37.966359+00:00", + "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00", "role": "primary", "server_version": 160004, "xlog": { @@ -147,7 +147,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be $ curl -s http://localhost:8008/patroni | jq . { "state": "running", - "postmaster_start_time": "2024-08-18 11:09:08.615242+00:00", + "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00", "role": "replica", "server_version": 160004, "xlog": { @@ -195,7 +195,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be $ curl -s http://localhost:8008/patroni | jq . { "state": "running", - "postmaster_start_time": "2024-08-18 11:09:08.615242+00:00", + "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00", "role": "replica", "server_version": 160004, "xlog": { @@ -241,7 +241,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be $ curl -s http://localhost:8008/patroni | jq . { "state": "running", - "postmaster_start_time": "2024-08-18 11:09:08.615242+00:00", + "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00", "role": "replica", "server_version": 160004, "xlog": { @@ -267,7 +267,7 @@ The ``GET /patroni`` is used by Patroni during the leader race. It also could be } ], "pause": true, - "dcs_last_seen": 1692356928, + "dcs_last_seen": 1724874295, "tags": { "clonefrom": true }, @@ -293,7 +293,7 @@ Retrieve the Patroni metrics in Prometheus format through the ``GET /metrics`` e patroni_postgres_running{scope="batman",name="patroni1"} 1 # HELP patroni_postmaster_start_time Epoch seconds since Postgres started. # TYPE patroni_postmaster_start_time gauge - patroni_postmaster_start_time{scope="batman",name="patroni1"} 1657656955.179243 + patroni_postmaster_start_time{scope="batman",name="patroni1"} 1724873966.352526 # HELP patroni_primary Value is 1 if this node is the leader, 0 otherwise. # TYPE patroni_primary gauge patroni_primary{scope="batman",name="patroni1"} 1 @@ -344,7 +344,7 @@ Retrieve the Patroni metrics in Prometheus format through the ``GET /metrics`` e patroni_postgres_timeline{scope="batman",name="patroni1"} 24 # HELP patroni_dcs_last_seen Epoch timestamp when DCS was last contacted successfully by Patroni. # TYPE patroni_dcs_last_seen gauge - patroni_dcs_last_seen{scope="batman",name="patroni1"} 1677658321 + patroni_dcs_last_seen{scope="batman",name="patroni1"} 1724874235 # HELP patroni_pending_restart Value is 1 if the node needs a restart, 0 otherwise. # TYPE patroni_pending_restart gauge patroni_pending_restart{scope="batman",name="patroni1"} 1 @@ -492,12 +492,21 @@ Let's check that the node processed this configuration. First of all it should s $ curl -s http://localhost:8008/patroni | jq . { - "pending_restart": true, "database_system_identifier": "6287881213849985952", - "postmaster_start_time": "2024-08-18 13:13:05.211 CEST", + "postmaster_start_time": "2024-08-28 19:39:26.352526+00:00", "xlog": { "location": 2197818976 }, + "timeline": 1, + "dcs_last_seen": 1724874545, + "database_system_identifier": "7408277255830290455", + "pending_restart": true, + "pending_restart_reason": { + "max_connections": { + "old_value": "100", + "new_value": "101" + } + }, "patroni": { "version": "4.0.0", "scope": "batman", diff --git a/patroni/version.py b/patroni/version.py index 69a0d8c2..ceb3b227 100644 --- a/patroni/version.py +++ b/patroni/version.py @@ -2,4 +2,4 @@ :var __version__: the current Patroni version. """ -__version__ = '3.3.2' +__version__ = '4.0.0' diff --git a/tests/test_bootstrap.py b/tests/test_bootstrap.py index 792924d9..9924c81a 100644 --- a/tests/test_bootstrap.py +++ b/tests/test_bootstrap.py @@ -175,7 +175,7 @@ class TestBootstrap(BaseTestPostgresql): with patch('subprocess.call', Mock(return_value=1)): self.assertFalse(self.b.bootstrap({})) - config = {'users': {'replicator': {'password': 'rep-pass', 'options': ['replication']}}} + config = {} with patch.object(Postgresql, 'is_running', Mock(return_value=False)), \ patch.object(Postgresql, 'get_major_version', Mock(return_value=140000)), \ @@ -253,8 +253,12 @@ class TestBootstrap(BaseTestPostgresql): self.assertFalse(task.result) self.p.config._config.pop('pg_hba') - self.b.post_bootstrap({}, task) - self.assertTrue(task.result) + with patch('patroni.postgresql.bootstrap.logger.error', new_callable=Mock()) as mock_logger: + self.b.post_bootstrap({'users': 1}, task) + self.assertEqual(mock_logger.call_args_list[0][0][0], + 'User creation is not be supported starting from v4.0.0. ' + 'Please use "boostrap.post_bootstrap" script to create users.') + self.assertTrue(task.result) self.b.bootstrap(config) with patch.object(Postgresql, 'pending_restart_reason', diff --git a/tests/test_citus.py b/tests/test_citus.py index 63bbf330..42cf6d2f 100644 --- a/tests/test_citus.py +++ b/tests/test_citus.py @@ -22,7 +22,7 @@ class TestCitus(BaseTestPostgresql): self.cluster = get_cluster_initialized_with_leader() self.cluster.workers[1] = self.cluster - @patch('time.time', Mock(side_effect=[100, 130, 160, 190, 220, 250, 280, 310, 340, 370])) + @patch('time.time', Mock(side_effect=[100, 130, 160, 190, 220, 250, 280, 310, 340, 370, 400, 430, 460, 490])) @patch('patroni.postgresql.mpp.citus.logger.exception', Mock(side_effect=SleepException)) @patch('patroni.postgresql.mpp.citus.logger.warning') @patch('patroni.postgresql.mpp.citus.PgDistTask.wait', Mock())