Merge branch 'master' of github.com:zalando/patroni into feature/quorum-commit

.github/workflows/tests.yaml

@@ -173,4 +173,4 @@ jobs:
 
     - uses: jakebailey/pyright-action@v1
       with:
-        version: 1.1.309
+        version: 1.1.316

.gitignore

@@ -57,3 +57,9 @@ docs/source/_templates/
 
 #VSCode IDE
 .vscode/
+
+# Virtual environment
+venv*/
+
+# Default test data directory
+data/

.readthedocs.yaml

@@ -0,0 +1,21 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.11"
+
+# Build documentation in the docs/ directory with Sphinx
+sphinx:
+  configuration: docs/conf.py
+
+formats:
+  - epub
+  - pdf
+  - htmlzip

Dockerfile.citus

@@ -25,7 +25,7 @@ RUN set -ex \
             | grep -Ev '^python3-(sphinx|etcd|consul|kazoo|kubernetes)' \
             | xargs apt-get install -y vim curl less jq locales haproxy sudo \
                             python3-etcd python3-kazoo python3-pip busybox \
-                            net-tools iputils-ping --fix-missing \
+                            net-tools iputils-ping lsb-release --fix-missing \
     && if [ $(dpkg --print-architecture) = 'arm64' ]; then \
         apt-get install -y postgresql-server-dev-$PG_MAJOR \
                            git gcc make autoconf \
@@ -40,7 +40,7 @@ RUN set -ex \
         echo "deb [signed-by=/etc/apt/trusted.gpg.d/citusdata_community.gpg] https://packagecloud.io/citusdata/community/debian/ $(lsb_release -cs) main" > /etc/apt/sources.list.d/citusdata_community.list \
         && curl -sL https://packagecloud.io/citusdata/community/gpgkey | gpg --dearmor > /etc/apt/trusted.gpg.d/citusdata_community.gpg \
         && apt-get update -y \
-        && apt-get -y install postgresql-$PG_MAJOR-citus-11.3 \
+        && apt-get -y install postgresql-$PG_MAJOR-citus-11.3; \
     fi \
     && pip3 install dumb-init \
 \

README.rst

@@ -8,7 +8,7 @@ You can find a version of this documentation that is searchable and also easier
 
 There are many ways to run high availability with PostgreSQL; for a list, see the `PostgreSQL Documentation <https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling>`__.
 
-Patroni is a template for you to create your own customized, high-availability solution using Python and - for maximum accessibility - a distributed configuration store like `ZooKeeper <https://zookeeper.apache.org/>`__, `etcd <https://github.com/coreos/etcd>`__, `Consul <https://github.com/hashicorp/consul>`__ or `Kubernetes <https://kubernetes.io>`__. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in the datacenter-or anywhere else-will hopefully find it useful.
+Patroni is a template for high availability (HA) PostgreSQL solutions using Python. For maximum accessibility, Patroni supports a variety of distributed configuration stores like `ZooKeeper <https://zookeeper.apache.org/>`__, `etcd <https://github.com/coreos/etcd>`__, `Consul <https://github.com/hashicorp/consul>`__ or `Kubernetes <https://kubernetes.io>`__. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in datacenters - or anywhere else - will hopefully find it useful.
 
 We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.
 

docs/ENVIRONMENT.rst

@@ -136,6 +136,13 @@ PostgreSQL
 -  **PATRONI\_POSTGRESQL\_DATA\_DIR**: The location of the Postgres data directory, either existing or to be initialized by Patroni.
 -  **PATRONI\_POSTGRESQL\_CONFIG\_DIR**: The location of the Postgres configuration directory, defaults to the data directory. Must be writable by Patroni.
 -  **PATRONI\_POSTGRESQL\_BIN_DIR**: Path to PostgreSQL binaries. (pg_ctl, initdb, pg_controldata, pg_basebackup, postgres, pg_isready, pg_rewind) The  default value is an empty string meaning that PATH environment variable will be used to find the executables.
+-  **PATRONI\_POSTGRESQL\_BIN\_PG\_CTL**: (optional) Custom name for ``pg_ctl`` binary.
+-  **PATRONI\_POSTGRESQL\_BIN\_INITDB**: (optional) Custom name for ``initdb`` binary.
+-  **PATRONI\_POSTGRESQL\_BIN\_PG\_CONTROLDATA**: (optional) Custom name for ``pg_controldata`` binary.
+-  **PATRONI\_POSTGRESQL\_BIN\_PG\_BASEBACKUP**: (optional) Custom name for ``pg_basebackup`` binary.
+-  **PATRONI\_POSTGRESQL\_BIN\_POSTGRES**: (optional) Custom name for ``postgres`` binary.
+-  **PATRONI\_POSTGRESQL\_BIN\_IS\_READY**: (optional) Custom name for ``pg_isready`` binary.
+-  **PATRONI\_POSTGRESQL\_BIN\_PG\_REWIND**: (optional) Custom name for ``pg_rewind`` binary.
 -  **PATRONI\_POSTGRESQL\_PGPASS**: path to the `.pgpass <https://www.postgresql.org/docs/current/static/libpq-pgpass.html>`__ password file. Patroni creates this file before executing pg\_basebackup and under some other circumstances. The location must be writable by Patroni.
 -  **PATRONI\_REPLICATION\_USERNAME**: replication username; the user will be created during initialization. Replicas will use this user to access the replication source via streaming replication
 -  **PATRONI\_REPLICATION\_PASSWORD**: replication password; the user will be created during initialization.

docs/README.rst

@@ -4,7 +4,7 @@
 Introduction
 ============
 
-Patroni originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.
+Patroni is a template for high availability (HA) PostgreSQL solutions using Python. Patroni originated as a fork of `Governor <https://github.com/compose/governor>`__, the project from Compose. It includes plenty of new features.
 
 For an example of a Docker-based deployment with Patroni, see `Spilo <https://github.com/zalando/spilo>`__, currently in use at Zalando.
 

docs/index.rst

@@ -6,7 +6,7 @@
 Introduction
 ============
 
-Patroni is a template for you to create your own customized, high-availability solution using Python and - for maximum accessibility - a distributed configuration store like `ZooKeeper <https://zookeeper.apache.org/>`__, `etcd <https://github.com/coreos/etcd>`__, `Consul <https://github.com/hashicorp/consul>`__ or `Kubernetes <https://kubernetes.io>`__. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in the datacenter-or anywhere else-will hopefully find it useful.
+Patroni is a template for high availability (HA) PostgreSQL solutions using Python. For maximum accessibility, Patroni supports a variety of distributed configuration stores like `ZooKeeper <https://zookeeper.apache.org/>`__, `etcd <https://github.com/coreos/etcd>`__, `Consul <https://github.com/hashicorp/consul>`__ or `Kubernetes <https://kubernetes.io>`__. Database engineers, DBAs, DevOps engineers, and SREs who are looking to quickly deploy HA PostgreSQL in datacenters — or anywhere else — will hopefully find it useful.
 
 We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely. There are many ways to run high availability with PostgreSQL; for a list, see the `PostgreSQL Documentation <https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling>`__.
 

features/citus.feature

@@ -10,19 +10,19 @@ Feature: citus
     And I start postgres3 in citus group 1
     Then replication works from postgres0 to postgres1 after 15 seconds
     Then replication works from postgres2 to postgres3 after 15 seconds
-    And postgres0 is registered in the postgres0 as the worker in group 0 after 5 seconds
-    And postgres2 is registered in the postgres0 as the worker in group 1 after 5 seconds
+    And postgres0 is registered in the postgres0 as the primary in group 0 after 5 seconds
+    And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
 
   Scenario: coordinator failover updates pg_dist_node
     Given I run patronictl.py failover batman --group 0 --candidate postgres1 --force
     Then postgres1 role is the primary after 10 seconds
     And replication works from postgres1 to postgres0 after 15 seconds
-    And postgres1 is registered in the postgres2 as the worker in group 0 after 5 seconds
+    And postgres1 is registered in the postgres2 as the primary in group 0 after 5 seconds
     And "sync" key in a group 0 in DCS has sync_standby=postgres0 after 15 seconds
     When I run patronictl.py failover batman --group 0 --candidate postgres0 --force
     Then postgres0 role is the primary after 10 seconds
     And replication works from postgres0 to postgres1 after 15 seconds
-    And postgres0 is registered in the postgres2 as the worker in group 0 after 5 seconds
+    And postgres0 is registered in the postgres2 as the primary in group 0 after 5 seconds
     And "sync" key in a group 0 in DCS has sync_standby=postgres1 after 15 seconds
 
   Scenario: worker switchover doesn't break client queries on the coordinator
@@ -32,14 +32,14 @@ Feature: citus
     Then I receive a response returncode 0
     And postgres3 role is the primary after 10 seconds
     And replication works from postgres3 to postgres2 after 15 seconds
-    And postgres3 is registered in the postgres0 as the worker in group 1 after 5 seconds
+    And postgres3 is registered in the postgres0 as the primary in group 1 after 5 seconds
     And "sync" key in a group 1 in DCS has sync_standby=postgres2 after 15 seconds
     And a thread is still alive
     When I run patronictl.py switchover batman --group 1 --force
     Then I receive a response returncode 0
     And postgres2 role is the primary after 10 seconds
     And replication works from postgres2 to postgres3 after 15 seconds
-    And postgres2 is registered in the postgres0 as the worker in group 1 after 5 seconds
+    And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
     And "sync" key in a group 1 in DCS has sync_standby=postgres3 after 15 seconds
     And a thread is still alive
     When I stop a thread
@@ -52,7 +52,7 @@ Feature: citus
     Then I receive a response returncode 0
     And postgres2 role is the primary after 10 seconds
     And replication works from postgres2 to postgres3 after 15 seconds
-    And postgres2 is registered in the postgres0 as the worker in group 1 after 5 seconds
+    And postgres2 is registered in the postgres0 as the primary in group 1 after 5 seconds
     And a thread is still alive
     When I stop a thread
     Then a distributed table on postgres0 has expected rows
@@ -64,8 +64,7 @@ Feature: citus
     When I run patronictl.py edit-config batman --group 2 -s ttl=20 --force
     Then I receive a response returncode 0
     And I receive a response output "+ttl: 20"
-    When I sleep for 2 seconds
-    Then postgres4 is registered in the postgres2 as the worker in group 2 after 5 seconds
+    Then postgres4 is registered in the postgres2 as the primary in group 2 after 5 seconds
     When I shut down postgres4
     Then There is a transaction in progress on postgres0 changing pg_dist_node
     When I run patronictl.py restart batman postgres2 --group 1 --force

features/environment.py

@@ -595,7 +595,7 @@ class KubernetesController(AbstractExternalDcsController):
                 api_process = 'kube-apiserver'
             elif context.startswith('k3d-'):
                 container = '{0}-server-0'.format(context)
-                api_process = 'k3s'
+                api_process = 'k3s server'
             else:
                 return super(KubernetesController, self)._is_running()
             try:

features/steps/citus.py

@@ -44,18 +44,19 @@ def start_citus(context, name, group):
     return context.pctl.start(name, custom_config={"citus": {"database": "postgres", "group": int(group)}})
 
 
-@step('{name1:w} is registered in the {name2:w} as the worker in group {group:d} after {time_limit:d} seconds')
-def check_registration(context, name1, name2, group, time_limit):
+@step('{name1:w} is registered in the {name2:w} as the {role:w} in group {group:d} after {time_limit:d} seconds')
+def check_registration(context, name1, name2, role, group, time_limit):
     time_limit *= context.timeout_multiplier
     max_time = time.time() + int(time_limit)
-    pg_dist_value = None
+
     worker_port = int(context.pctl.query(name1, "SHOW port").fetchone()[0])
+
     while time.time() < max_time:
         try:
-            r = context.pctl.query(name2,
-                                   "SELECT nodeport FROM pg_catalog.pg_dist_node WHERE groupid = {0}".format(group))
-            pg_dist_value = r.fetchone()[0]
-            if pg_dist_value == worker_port:
+            cur = context.pctl.query(name2, "SELECT nodeport, noderole"
+                                            " FROM pg_catalog.pg_dist_node WHERE groupid = {0}".format(group))
+            mapping = {r[0]: r[1] for r in cur}
+            if mapping.get(worker_port) == role:
                 return
         except Exception:
             pass

kubernetes/Dockerfile.citus

@@ -7,7 +7,7 @@ RUN export DEBIAN_FRONTEND=noninteractive \
     && apt-get upgrade -y \
     && apt-cache depends patroni | sed -n -e 's/.* Depends: \(python3-.\+\)$/\1/p' \
             | grep -Ev '^python3-(sphinx|etcd|consul|kazoo|kubernetes)' \
-            | xargs apt-get install -y busybox vim-tiny curl jq less locales git python3-pip python3-wheel \
+            | xargs apt-get install -y busybox vim-tiny curl jq less locales git python3-pip python3-wheel lsb-release \
     ## Make sure we have a en_US.UTF-8 locale available
     && localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 \
     && echo "deb [signed-by=/etc/apt/trusted.gpg.d/citusdata_community.gpg] https://packagecloud.io/citusdata/community/debian/ $(lsb_release -cs) main" > /etc/apt/sources.list.d/citusdata_community.list \

patroni.spec

@@ -16,7 +16,10 @@ def hiddenimports():
 a = Analysis(['patroni/__main__.py'],
              pathex=[],
              binaries=None,
-             datas=None,
+             datas=[
+                ('patroni/postgresql/available_parameters/*.yml', 'patroni/postgresql/available_parameters'),
+                ('patroni/postgresql/available_parameters/*.yaml', 'patroni/postgresql/available_parameters'),
+            ],
              hiddenimports=hiddenimports(),
              hookspath=[],
              runtime_hooks=[],

patroni/__init__.py

@@ -1,3 +1,10 @@
+"""Define general variables and functions for :mod:`patroni`.
+
+:var PATRONI_ENV_PREFIX: prefix for Patroni related configuration environment variables.
+:var KUBERNETES_ENV_PREFIX: prefix for Kubernetes related configuration environment variables.
+:var MIN_PSYCOPG2: minimum version of :mod:`psycopg2` required by Patroni to work.
+"""
+
 import sys
 
 from typing import Any, Callable, Iterator, Tuple
@@ -8,12 +15,40 @@ MIN_PSYCOPG2 = (2, 5, 4)
 
 
 def fatal(string: str, *args: Any) -> None:
-    sys.stderr.write('FATAL: ' + string.format(*args) + '\n')
-    sys.exit(1)
+    """Write a fatal message to stderr and exit with code ``1``.
+
+    :param string: message to be written before exiting.
+    """
+    sys.exit('FATAL: ' + string.format(*args))
 
 
 def parse_version(version: str) -> Tuple[int, ...]:
+    """Convert *version* from human-readable format to tuple of integers.
+
+    .. note::
+        Designed for easy comparison of software versions in Python.
+
+    :param version: human-readable software version, e.g. ``2.5.4``.
+
+    :returns: tuple of *version* parts, each part as an integer.
+
+    :Example:
+
+        >>> parse_version('2.5.4')
+        (2, 5, 4)
+    """
     def _parse_version(version: str) -> Iterator[int]:
+        """Yield each part of a human-readable version string as an integer.
+
+        :param version: human-readable software version, e.g. ``2.5.4``.
+
+        :yields: each part of *version* as an integer.
+
+        :Example:
+
+            >>> tuple(_parse_version('2.5.4'))
+            (2, 5, 4)
+        """
         for e in version.split('.'):
             try:
                 yield int(e)
@@ -22,9 +57,22 @@ def parse_version(version: str) -> Tuple[int, ...]:
     return tuple(_parse_version(version.split(' ')[0]))
 
 
-# We pass MIN_PSYCOPG2 and parse_version as arguments to simplify usage of check_psycopg from the setup.py
 def check_psycopg(_min_psycopg2: Tuple[int, ...] = MIN_PSYCOPG2,
                   _parse_version: Callable[[str], Tuple[int, ...]] = parse_version) -> None:
+    """Ensure at least one among :mod:`psycopg2` or :mod:`psycopg` libraries are available in the environment.
+
+    .. note::
+        We pass ``MIN_PSYCOPG2`` and :func:`parse_version` as arguments to simplify usage of :func:`check_psycopg` from
+        the ``setup.py``.
+
+    .. note::
+        Patroni chooses :mod:`psycopg2` over :mod:`psycopg`, if possible.
+
+        If nothing meeting the requirements is found, then exit with a fatal message.
+
+    :param _min_psycopg2: minimum required version in case :mod:`psycopg2` is chosen.
+    :param _parse_version: function used to parse :mod:`psycopg2`/:mod:`psycopg` version into a comparable object.
+    """
     min_psycopg2_str = '.'.join(map(str, _min_psycopg2))
 
     # try psycopg2

patroni/api.py

@@ -1549,11 +1549,11 @@ class RestApiServer(ThreadingMixIn, HTTPServer, Thread):
         if self.__ssl_options.get('certfile'):
             import ssl
             try:
-                ctx = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
-                crts = ctx.load_verify_locations(self.__ssl_options['certfile'])
-                if crts:
-                    return crts[0].get('serialNumber')
-            except Exception as e:
+                crt: Dict[str, Any] = ssl._ssl._test_decode_cert(self.__ssl_options['certfile'])  # pyright: ignore
+                if TYPE_CHECKING:  # pragma: no cover
+                    assert isinstance(crt, dict)
+                return crt.get('serialNumber')
+            except ssl.SSLError as e:
                 logger.error('Failed to get serial number from certificate %s: %r', self.__ssl_options['certfile'], e)
 
     def reload_local_certificate(self) -> Optional[bool]:

patroni/async_executor.py

@@ -1,3 +1,4 @@
+"""Implement facilities for executing asynchronous tasks."""
 import logging
 
 from threading import Event, Lock, RLock, Thread
@@ -13,14 +14,23 @@ class CriticalTask(object):
     """Represents a critical task in a background process that we either need to cancel or get the result of.
 
     Fields of this object may be accessed only when holding a lock on it. To perform the critical task the background
-    thread must, while holding lock on this object, check `is_cancelled` flag, run the task and mark the task as
-    complete using `complete()`.
+    thread must, while holding lock on this object, check ``is_cancelled`` flag, run the task and mark the task as
+    complete using :func:`complete`.
 
     The main thread must hold async lock to prevent the task from completing, hold lock on critical task object,
-    call cancel. If the task has completed `cancel()` will return False and `result` field will contain the result of
-    the task. When cancel returns True it is guaranteed that the background task will notice the `is_cancelled` flag.
+    call :func:`cancel`. If the task has completed :func:`cancel` will return ``False`` and ``result`` field will
+    contain the result of the task. When :func:`cancel` returns ``True`` it is guaranteed that the background task will
+    notice the ``is_cancelled`` flag.
+
+    :ivar is_cancelled: if the critical task has been cancelled.
+    :ivar result: contains the result of the task, if it has already been completed.
     """
+
     def __init__(self) -> None:
+        """Create a new instance of :class:`CriticalTask`.
+
+        Instantiate the lock and the task control attributes.
+        """
         self._lock = Lock()
         self.is_cancelled = False
         self.result = None
@@ -28,37 +38,59 @@ class CriticalTask(object):
     def reset(self) -> None:
         """Must be called every time the background task is finished.
 
-        Must be called from async thread. Caller must hold lock on async executor when calling."""
+        .. note::
+            Must be called from async thread. Caller must hold lock on async executor when calling.
+        """
         self.is_cancelled = False
         self.result = None
 
     def cancel(self) -> bool:
-        """Tries to cancel the task, returns True if the task has already run.
+        """Tries to cancel the task.
 
-        Caller must hold lock on async executor and the task when calling."""
+        .. note::
+            Caller must hold lock on async executor and the task when calling.
+
+        :returns: ``False`` if the task has already run, or ``True`` it has been cancelled.
+        """
         if self.result is not None:
             return False
         self.is_cancelled = True
         return True
 
     def complete(self, result: Any) -> None:
-        """Mark task as completed along with a result.
+        """Mark task as completed along with a *result*.
 
-        Must be called from async thread. Caller must hold lock on task when calling."""
+        .. note::
+            Must be called from async thread. Caller must hold lock on task when calling.
+        """
         self.result = result
 
     def __enter__(self) -> 'CriticalTask':
+        """Acquire the object lock when entering the context manager."""
         self._lock.acquire()
         return self
 
     def __exit__(self, exc_type: Optional[Type[BaseException]],
                  exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None:
+        """Release the object lock when exiting the context manager."""
         self._lock.release()
 
 
 class AsyncExecutor(object):
+    """Asynchronous executor of (long) tasks.
+
+    :ivar critical_task: a :class:`CriticalTask` instance to handle execution of critical background tasks.
+    """
 
     def __init__(self, cancellable: CancellableSubprocess, ha_wakeup: Callable[..., None]) -> None:
+        """Create a new instance of :class:`AsyncExecutor`.
+
+        Configure the given *cancellable* and *ha_wakeup*, initializes the control attributes, and instantiate the lock
+        and event objects that are used to access attributes and manage communication between threads.
+
+        :param cancellable: a subprocess that supports being cancelled.
+        :param ha_wakeup: function to wake up the HA loop.
+        """
         self._cancellable = cancellable
         self._ha_wakeup = ha_wakeup
         self._thread_lock = RLock()
@@ -70,9 +102,22 @@ class AsyncExecutor(object):
 
     @property
     def busy(self) -> bool:
+        """``True`` if there is an action scheduled to occur, else ``False``."""
         return self.scheduled_action is not None
 
     def schedule(self, action: str) -> Optional[str]:
+        """Schedule *action* to be executed.
+
+        .. note::
+            Must be called before executing a task.
+
+        .. note::
+            *action* can only be scheduled if there is no other action currently scheduled.
+
+        :param action: action to be executed.
+
+        :returns: ``None`` if *action* has been successfully scheduled, or the previously scheduled action, if any.
+        """
         with self._scheduled_action_lock:
             if self._scheduled_action is not None:
                 return self._scheduled_action
@@ -83,14 +128,32 @@ class AsyncExecutor(object):
 
     @property
     def scheduled_action(self) -> Optional[str]:
+        """The currently scheduled action, if any, else ``None``."""
         with self._scheduled_action_lock:
             return self._scheduled_action
 
     def reset_scheduled_action(self) -> None:
+        """Unschedule a previously scheduled action, if any.
+
+        .. note::
+            Must be called once the scheduled task finishes or is cancelled.
+        """
         with self._scheduled_action_lock:
             self._scheduled_action = None
 
-    def run(self, func: Callable[..., Any], args: Tuple[Any, ...] = ()) -> Optional[bool]:
+    def run(self, func: Callable[..., Any], args: Tuple[Any, ...] = ()) -> Optional[Any]:
+        """Run *func* with *args*.
+
+        .. note::
+            Expected to be executed through a thread.
+
+        :param func: function to be run. If it returns anything other than ``None``, HA loop will be woken up at the end
+            of :func:`run` execution.
+        :param args: arguments to be passed to *func*.
+
+        :returns: ``None`` if *func* execution has been cancelled or faced any exception, otherwise the result of
+            *func*.
+        """
         wakeup = False
         try:
             with self:
@@ -114,15 +177,36 @@ class AsyncExecutor(object):
                 self._ha_wakeup()
 
     def run_async(self, func: Callable[..., Any], args: Tuple[Any, ...] = ()) -> None:
+        """Start an async thread that runs *func* with *args*.
+
+        :param func: function to be run. Will be passed through args to :class:`~threading.Thread` with a target of
+            :func:`run`.
+        :param args: arguments to be passed along to :class:`~threading.Thread` with *func*.
+
+        """
         Thread(target=self.run, args=(func, args)).start()
 
     def try_run_async(self, action: str, func: Callable[..., Any], args: Tuple[Any, ...] = ()) -> Optional[str]:
+        """Try to run an async task, if none is currently being executed.
+
+        :param action: name of the task to be executed.
+        :param func: actual function that performs the task *action*.
+        :param args: arguments to be passed to *func*.
+
+        :returns: ``None`` if *func* was scheduled successfully, otherwise an error message informing of an already
+            ongoing task.
+        """
         prev = self.schedule(action)
         if prev is None:
             return self.run_async(func, args)
         return 'Failed to run {0}, {1} is already in progress'.format(action, prev)
 
     def cancel(self) -> None:
+        """Request cancellation of a scheduled async task, if any.
+
+        .. note::
+            Wait until task is cancelled before returning control to caller.
+        """
         with self:
             with self._scheduled_action_lock:
                 if self._scheduled_action is None:
@@ -137,9 +221,15 @@ class AsyncExecutor(object):
             self.reset_scheduled_action()
 
     def __enter__(self) -> 'AsyncExecutor':
+        """Acquire the thread lock when entering the context manager."""
         self._thread_lock.acquire()
         return self
 
     def __exit__(self, exc_type: Optional[Type[BaseException]],
                  exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None:
+        """Release the thread lock when exiting the context manager.
+
+        .. note::
+            The arguments are not used, but we need them to match the expected method signature.
+        """
         self._thread_lock.release()

patroni/collections.py

@@ -1,73 +1,202 @@
+"""Patroni custom object types somewhat like :mod:`collections` module.
+
+Provides a case insensitive :class:`dict` and :class:`set` object types.
+"""
 from collections import OrderedDict
 from typing import Any, Collection, Dict, Iterator, MutableMapping, MutableSet, Optional
 
 
 class CaseInsensitiveSet(MutableSet[str]):
-    """A case-insensitive ``set``-like object.
+    """A case-insensitive :class:`set`-like object.
 
-    Implements all methods and operations of :class:``MutableSet``. All values are expected to be strings.
+    Implements all methods and operations of :class:`~typing.MutableSet`. All values are expected to be strings.
     The structure remembers the case of the last value set, however, contains testing is case insensitive.
     """
+
     def __init__(self, values: Optional[Collection[str]] = None) -> None:
+        """Create a new instance of :class:`CaseInsensitiveSet` with the given *values*.
+
+        :param values: values to be added to the set.
+        """
         self._values: Dict[str, str] = {}
         for v in values or ():
             self.add(v)
 
     def __repr__(self) -> str:
+        """Get a string representation of the set.
+
+        Provide a helpful way of recreating the set.
+
+        :returns: representation of the set, showing its values.
+
+        :Example:
+
+            >>> repr(CaseInsensitiveSet(('1', 'test', 'Test', 'TESt', 'test2')))  # doctest: +ELLIPSIS
+            "<CaseInsensitiveSet('1', 'TESt', 'test2') at ..."
+        """
         return '<{0}{1} at {2:x}>'.format(type(self).__name__, tuple(self._values.values()), id(self))
 
     def __str__(self) -> str:
+        """Get set values for printing.
+
+        :returns: set of values in string format.
+
+        :Example:
+
+            >>> str(CaseInsensitiveSet(('1', 'test', 'Test', 'TESt', 'test2')))  # doctest: +SKIP
+            "{'TESt', 'test2', '1'}"
+        """
         return str(set(self._values.values()))
 
     def __contains__(self, value: str) -> bool:
+        """Check if set contains *value*.
+
+        The check is performed case-insensitively.
+
+        :param value: value to be checked.
+
+        :returns: ``True`` if *value* is already in the set, ``False`` otherwise.
+        """
         return value.lower() in self._values
 
     def __iter__(self) -> Iterator[str]:
+        """Iterate over the values in this set.
+
+        :yields: values from set.
+        """
         return iter(self._values.values())
 
     def __len__(self) -> int:
+        """Get the length of this set.
+
+        :returns: number of values in the set.
+
+        :Example:
+
+        >>> len(CaseInsensitiveSet(('1', 'test', 'Test', 'TESt', 'test2')))
+        3
+        """
         return len(self._values)
 
     def add(self, value: str) -> None:
+        """Add *value* to this set.
+
+        Search is performed case-insensitively. If *value* is already in the set, overwrite it with *value*, so we
+        "remember" the last case of *value*.
+
+        :param value: value to be added to the set.
+        """
         self._values[value.lower()] = value
 
     def discard(self, value: str) -> None:
+        """Remove *value* from this set.
+
+        Search is performed case-insensitively. If *value* is not present in the set, no exception is raised.
+
+        :param value: value to be removed from the set.
+        """
         self._values.pop(value.lower(), None)
 
     def issubset(self, other: 'CaseInsensitiveSet') -> bool:
+        """Check if this set is a subset of *other*.
+
+        :param other: another set to be compared with this set.
+        :returns: ``True`` if this set is a subset of *other*, else ``False``.
+        """
         return self <= other
 
 
 class CaseInsensitiveDict(MutableMapping[str, Any]):
-    """A case-insensitive ``dict``-like object.
+    """A case-insensitive :class:`dict`-like object.
 
-    Implements all methods and operations of :class:``MutableMapping`` as well as dict's :func:``copy``.
-    All keys are expected to be strings. The structure remembers the case of the last key to be set,
-    and ``iter(instance)``, ``keys()``, ``items()``, ``iterkeys()``, and ``iteritems()`` will contain
-    case-sensitive keys. However, querying and contains testing is case insensitive.
+    Implements all methods and operations of :class:`~typing.MutableMapping` as well as :class:`dict`'s
+    :func:`~dict.copy`. All keys are expected to be strings. The structure remembers the case of the last key to be set,
+    and :func:`iter`, :func:`dict.keys`, :func:`dict.items`, :func:`dict.iterkeys`, and :func:`dict.iteritems` will
+    contain case-sensitive keys. However, querying and contains testing is case insensitive.
     """
+
     def __init__(self, data: Optional[Dict[str, Any]] = None) -> None:
+        """Create a new instance of :class:`CaseInsensitiveDict` with the given *data*.
+
+        :param data: initial dictionary to create a :class:`CaseInsensitiveDict` from.
+        """
         self._values: OrderedDict[str, Any] = OrderedDict()
         self.update(data or {})
 
     def __setitem__(self, key: str, value: Any) -> None:
-        # Use the lowercase key for lookups, but store the actual key alongside the value.
+        """Assign *value* to *key* in this dict.
+
+        *key* is searched/stored case-insensitively in the dict. The corresponding value in the dict is a tuple of:
+            * original *key*;
+            * *value*.
+
+        :param key: key to be created or updated in the dict.
+        :param value: value for *key*.
+        """
         self._values[key.lower()] = (key, value)
 
     def __getitem__(self, key: str) -> Any:
+        """Get the value corresponding to *key*.
+
+        *key* is searched case-insensitively in the dict.
+
+        .. note:
+            If *key* is not present in the dict, :class:`KeyError` will be triggered.
+
+        :param key: key to be searched in the dict.
+
+        :returns: value corresponding to *key*.
+        """
         return self._values[key.lower()][1]
 
-    def __delitem__(self, key: str) -> Any:
+    def __delitem__(self, key: str) -> None:
+        """Remove *key* from this dict.
+
+        *key* is searched case-insensitively in the dict.
+
+        .. note:
+            If *key* is not present in the dict, :class:`KeyError` will be triggered.
+
+        :param key: key to be removed from the dict.
+        """
         del self._values[key.lower()]
 
     def __iter__(self) -> Iterator[str]:
+        """Iterate over keys of this dict.
+
+        :yields: each key present in the dict. Yields each key with its last case that has been stored.
+        """
         return iter(key for key, _ in self._values.values())
 
     def __len__(self) -> int:
+        """Get the length of this dict.
+
+        :returns: number of keys in the dict.
+
+        :Example:
+
+        >>> len(CaseInsensitiveDict({'a': 'b', 'A': 'B', 'c': 'd'}))
+        2
+        """
         return len(self._values)
 
     def copy(self) -> 'CaseInsensitiveDict':
+        """Create a copy of this dict.
+
+        :return: a new dict object with the same keys and values of this dict.
+        """
         return CaseInsensitiveDict({v[0]: v[1] for v in self._values.values()})
 
     def __repr__(self) -> str:
+        """Get a string representation of the dict.
+
+        Provide a helpful way of recreating the dict.
+
+        :returns: representation of the dict, showing its keys and values.
+
+        :Example:
+
+            >>> repr(CaseInsensitiveDict({'a': 'b', 'A': 'B', 'c': 'd'}))  # doctest: +ELLIPSIS
+            "<CaseInsensitiveDict{'A': 'B', 'c': 'd'} at ..."
+        """
         return '<{0}{1} at {2:x}>'.format(type(self).__name__, dict(self.items()), id(self))

patroni/config.py

@@ -395,6 +395,11 @@ class Config(object):
                                     'dir', 'file_size', 'file_num', 'loggers'])
         _set_section_values('raft', ['data_dir', 'self_addr', 'partner_addrs', 'password', 'bind_addr'])
 
+        for binary in ('pg_ctl', 'initdb', 'pg_controldata', 'pg_basebackup', 'postgres', 'pg_isready', 'pg_rewind'):
+            value = _popenv('POSTGRESQL_BIN_' + binary)
+            if value:
+                ret['postgresql'].setdefault('bin_name', {})[binary] = value
+
         for first, second in (('restapi', 'allowlist_include_members'), ('ctl', 'insecure')):
             value = ret.get(first, {}).pop(second, None)
             if value:

patroni/dcs/consul.py

@@ -67,7 +67,7 @@ class HTTPClient(object):
         if ca_cert:
             kwargs['ca_certs'] = ca_cert
         kwargs['cert_reqs'] = ssl.CERT_REQUIRED if verify or ca_cert else ssl.CERT_NONE
-        self.http = urllib3.PoolManager(num_pools=10, maxsize=10, **kwargs)
+        self.http = urllib3.PoolManager(num_pools=10, maxsize=10, headers={}, **kwargs)
         self._ttl = 30
 
     def set_read_timeout(self, timeout: float) -> None:

patroni/dcs/kubernetes.py

@@ -75,7 +75,7 @@ class K8sConfig(object):
         pass
 
     def __init__(self) -> None:
-        self.pool_config: Dict[str, Union[str, int]] = {'maxsize': 10, 'num_pools': 10}  # urllib3.PoolManager config
+        self.pool_config: Dict[str, Any] = {'maxsize': 10, 'num_pools': 10}  # urllib3.PoolManager config
         self._token_expires_at = datetime.datetime.max
         self._headers: Dict[str, str] = {}
         self._make_headers()
@@ -277,12 +277,13 @@ class K8sClient(object):
 
         def _get_api_servers(self, api_servers_cache: List[str]) -> List[str]:
             _, per_node_timeout, per_node_retries = self._calculate_timeouts(len(api_servers_cache))
-            kwargs = {'headers': self._make_headers({}), 'preload_content': True, 'retries': per_node_retries,
+            headers = self._make_headers({})
+            kwargs = {'preload_content': True, 'retries': per_node_retries,
                       'timeout': urllib3.Timeout(connect=max(1.0, per_node_timeout / 2.0), total=per_node_timeout)}
             path = self._API_URL_PREFIX + 'default/endpoints/kubernetes'
             for base_uri in api_servers_cache:
                 try:
-                    response = self.pool_manager.request('GET', base_uri + path, **kwargs)
+                    response = self.pool_manager.request('GET', base_uri + path, headers=headers, **kwargs)
                     endpoint = self._handle_server_response(response, True)
                     if TYPE_CHECKING:  # pragma: no cover
                         assert isinstance(endpoint, K8sObject)

patroni/exceptions.py

@@ -1,33 +1,55 @@
+"""Implement high-level Patroni exceptions.
+
+More specific exceptions can be found in other modules, as subclasses of any exception defined in this module.
+"""
 from typing import Any
 
 
 class PatroniException(Exception):
+    """Parent class for all kind of Patroni exceptions.
 
-    """Parent class for all kind of exceptions related to selected distributed configuration store"""
+    :ivar value: description of the exception.
+    """
 
     def __init__(self, value: Any) -> None:
+        """Create a new instance of :class:`PatroniException` with the given description.
+
+        :param value: description of the exception.
+        """
         self.value = value
 
 
 class PatroniFatalException(PatroniException):
+    """Catastrophic exception that prevents Patroni from performing its job."""
+
     pass
 
 
 class PostgresException(PatroniException):
+    """Any exception related with Postgres management."""
+
     pass
 
 
 class DCSError(PatroniException):
+    """Parent class for all kind of DCS related exceptions."""
+
     pass
 
 
 class PostgresConnectionException(PostgresException):
+    """Any problem faced while connecting to a Postgres instance."""
+
     pass
 
 
 class WatchdogError(PatroniException):
+    """Any problem faced while managing a watchdog device."""
+
     pass
 
 
 class ConfigParseError(PatroniException):
+    """Any issue identified while loading or validating the Patroni configuration."""
+
     pass

patroni/postgresql/__init__.py

@@ -26,6 +26,7 @@ from .slots import SlotsHandler
 from .sync import SyncHandler
 from .. import psycopg
 from ..async_executor import CriticalTask
+from ..collections import CaseInsensitiveSet
 from ..dcs import Cluster, Leader, Member
 from ..exceptions import PostgresConnectionException
 from ..utils import Retry, RetryFailedError, polling_loop, data_directory_is_empty, parse_int
@@ -216,6 +217,11 @@ class Postgresql(object):
 
         return ("SELECT " + self.TL_LSN + ", {2}").format(self.wal_name, self.lsn_name, extra)
 
+    @property
+    def available_gucs(self) -> CaseInsensitiveSet:
+        """GUCs available in this Postgres server."""
+        return self._get_gucs()
+
     def _version_file_exists(self) -> bool:
         return not self.data_directory_empty() and os.path.isfile(self._version_file)
 
@@ -1270,3 +1276,13 @@ class Postgresql(object):
         self.slots_handler.schedule()
         self.citus_handler.schedule_cache_rebuild()
         self._sysid = ''
+
+    def _get_gucs(self) -> CaseInsensitiveSet:
+        """Get all available GUCs based on ``postgres --describe-config`` output.
+
+        :returns: all available GUCs in the local Postgres server.
+        """
+        cmd = [self.pgcommand('postgres'), '--describe-config']
+        return CaseInsensitiveSet({
+            line.split('\t')[0] for line in subprocess.check_output(cmd).decode('utf-8').strip().split('\n')
+        })

patroni/postgresql/citus.py

@@ -76,8 +76,9 @@ class CitusHandler(Thread):
         self._connection = Connection()
         self._pg_dist_node: Dict[int, PgDistNode] = {}  # Cache of pg_dist_node: {groupid: PgDistNode()}
         self._tasks: List[PgDistNode] = []  # Requests to change pg_dist_node, every task is a `PgDistNode`
-        self._condition = Condition()  # protects _pg_dist_node, _tasks, and _schedule_load_pg_dist_node
         self._in_flight: Optional[PgDistNode] = None  # Reference to the `PgDistNode` being changed in a transaction
+        self._schedule_load_pg_dist_node = True  # Flag that "pg_dist_node" should be queried from the database
+        self._condition = Condition()  # protects _pg_dist_node, _tasks, _in_flight, and _schedule_load_pg_dist_node
         self.schedule_cache_rebuild()
 
     def is_enabled(self) -> bool:
@@ -117,7 +118,8 @@ class CitusHandler(Thread):
         except Exception as e:
             logger.error('Exception when executing query "%s", (%s): %r', sql, params, e)
             self._connection.close()
-            self._in_flight = None
+            with self._condition:
+                self._in_flight = None
             self.schedule_cache_rebuild()
             raise e
 
@@ -215,17 +217,20 @@ class CitusHandler(Thread):
     def process_task(self, task: PgDistNode) -> bool:
         """Updates a single row in `pg_dist_node` table, optionally in a transaction.
 
-        The transaction is started if we do a demote of the worker node
-        or before promoting the other worker if there is not transaction
-        in progress. And, the transaction it is committed when the
-        switchover/failover completed.
+        The transaction is started if we do a demote of the worker node or before promoting the other worker if
+        there is no transaction in progress. And, the transaction is committed when the switchover/failover completed.
 
-        This method returns `True` if node was updated (optionally,
-        transaction was committed) as an indicator that
-        the `self._pg_dist_node` cache should be updated.
+        .. note:
+            The maximum lifetime of the transaction in progress is controlled outside of this method.
 
-        The maximum lifetime of the transaction in progress
-        is controlled outside of this method."""
+        .. note:
+            Read access to `self._in_flight` isn't protected because we know it can't be changed outside of our thread.
+
+        :param task: reference to a :class:`PgDistNode` object that represents a row to be updated/created.
+        :returns: `True` if the row was succesfully created/updated or transaction in progress
+            was committed as an indicator that the `self._pg_dist_node` cache should be updated,
+            or, if the new transaction was opened, this method returns `False`.
+        """
 
         if task.event == 'after_promote':
             # The after_promote may happen without previous before_demote and/or
@@ -236,7 +241,6 @@ class CitusHandler(Thread):
                 self.update_node(task)
             if self._in_flight:
                 self.query('COMMIT')
-                self._in_flight = None
             return True
         else:  # before_demote, before_promote
             if task.timeout:
@@ -244,11 +248,11 @@ class CitusHandler(Thread):
             if not self._in_flight:
                 self.query('BEGIN')
             self.update_node(task)
-            self._in_flight = task
         return False
 
     def process_tasks(self) -> None:
         while True:
+            # Read access to `_in_flight` isn't protected because we know it can't be changed outside of our thread.
             if not self._in_flight and not self.load_pg_dist_node():
                 break
 
@@ -259,11 +263,17 @@ class CitusHandler(Thread):
                 update_cache = self.process_task(task)
             except Exception as e:
                 logger.error('Exception when working with pg_dist_node: %r', e)
-                update_cache = False
+                update_cache = None
             with self._condition:
                 if self._tasks:
                     if update_cache:
                         self._pg_dist_node[task.group] = task
+
+                    if update_cache is False:  # an indicator that process_tasks has started a transaction
+                        self._in_flight = task
+                    else:
+                        self._in_flight = None
+
                     if id(self._tasks[i]) == id(task):
                         self._tasks.pop(i)
             task.wakeup()
@@ -293,11 +303,19 @@ class CitusHandler(Thread):
         with self._condition:
             i = self.find_task_by_group(task.group)
 
-            # task.timeout is None is an indicator that it was scheduled
-            # from the sync_pg_dist_node() and we don't want to override
-            # already existing task created from REST API.
-            if task.timeout is None and (i is not None or self._in_flight and self._in_flight.group == task.group):
-                return False
+            # The `PgDistNode.timeout` == None is an indicator that it was scheduled from the sync_pg_dist_node().
+            if task.timeout is None:
+                # We don't want to override the already existing task created from REST API.
+                if i is not None and self._tasks[i].timeout is not None:
+                    return False
+
+                # There is a little race condition with tasks created from REST API - the call made "before" the member
+                # key is updated in DCS. Therefore it is possible that :func:`sync_pg_dist_node` will try to create a
+                # task based on the outdated values of "state"/"role". To solve it we introduce an artificial timeout.
+                # Only when the timeout is reached new tasks could be scheduled from sync_pg_dist_node()
+                if self._in_flight and self._in_flight.group == task.group and self._in_flight.timeout is not None\
+                        and self._in_flight.deadline > time.time():
+                    return False
 
             # Override already existing task for the same worker group
             if i is not None:

patroni/postgresql/config.py

@@ -39,13 +39,14 @@ def conninfo_uri_parse(dsn: str) -> Dict[str, str]:
     for netloc in r.netloc.split('@')[-1].split(','):
         host = None
         if '[' in netloc and ']' in netloc:
-            host = netloc.split(']')[0][1:]
-        tmp = netloc.split(':', 1)
+            tmp = netloc.split(']') + ['']
+            host = tmp[0][1:]
+            netloc = ':'.join(tmp[:2])
+        tmp = netloc.rsplit(':', 1)
         if host is None:
             host = tmp[0]
         hosts.append(host)
-        if len(tmp) == 2:
-            ports.append(tmp[1])
+        ports.append(tmp[1] if len(tmp) == 2 else '')
     if hosts:
         ret['host'] = ','.join(hosts)
     if ports:
@@ -113,9 +114,9 @@ def parse_dsn(value: str) -> Optional[Dict[str, str]]:
     and sets the `sslmode`, 'gssencmode', and `channel_binding` to `prefer` if it is not present in
     the connection string. This is necessary to simplify comparison of the old and the new values.
 
-    >>> r = parse_dsn('postgresql://u%2Fse:pass@:%2f123,[%2Fhost2]/db%2Fsdf?application_name=mya%2Fpp&ssl=true')
-    >>> r == {'application_name': 'mya/pp', 'host': ',/host2', 'sslmode': 'require',\
-              'password': 'pass', 'port': '/123', 'user': 'u/se', 'gssencmode': 'prefer', 'channel_binding': 'prefer'}
+    >>> r = parse_dsn('postgresql://u%2Fse:pass@:%2f123,[::1]/db%2Fsdf?application_name=mya%2Fpp&ssl=true')
+    >>> r == {'application_name': 'mya/pp', 'host': ',::1', 'sslmode': 'require',\
+              'password': 'pass', 'port': '/123,', 'user': 'u/se', 'gssencmode': 'prefer', 'channel_binding': 'prefer'}
     True
     >>> r = parse_dsn(" host = 'host' dbname = db\\\\ name requiressl=1 ")
     >>> r == {'host': 'host', 'sslmode': 'require', 'gssencmode': 'prefer', 'channel_binding': 'prefer'}
@@ -412,7 +413,8 @@ class ConfigHandler(object):
             include = self._config.get('custom_conf') or self._postgresql_base_conf_name
             f.writeline("include '{0}'\n".format(ConfigWriter.escape(include)))
             for name, value in sorted((configuration).items()):
-                value = transform_postgresql_parameter_value(self._postgresql.major_version, name, value)
+                value = transform_postgresql_parameter_value(self._postgresql.major_version, name, value,
+                                                             self._postgresql.available_gucs)
                 if value is not None and\
                         (name != 'hba_file' or not self._postgresql.bootstrap.running_custom_bootstrap):
                     f.write_param(name, value)
@@ -534,7 +536,8 @@ class ConfigHandler(object):
                     self._passfile_mtime = mtime(self._pgpass)
                 value = self.format_dsn(value)
             else:
-                value = transform_recovery_parameter_value(self._postgresql.major_version, name, value)
+                value = transform_recovery_parameter_value(self._postgresql.major_version, name, value,
+                                                           self._postgresql.available_gucs)
                 if value is None:
                     continue
             fd.write_param(name, value)

patroni/postgresql/validator.py

@@ -1,9 +1,13 @@
 import abc
+from copy import deepcopy
 import logging
+import os
+import yaml
 
-from typing import Any, MutableMapping, Optional, Tuple, Union
+from typing import Any, Dict, Iterator, List, MutableMapping, Optional, Tuple, Type, Union
 
-from ..collections import CaseInsensitiveDict
+from ..collections import CaseInsensitiveDict, CaseInsensitiveSet
+from ..exceptions import PatroniException
 from ..utils import parse_bool, parse_int, parse_real
 
 logger = logging.getLogger(__name__)
@@ -11,10 +15,20 @@ logger = logging.getLogger(__name__)
 
 class _Transformable(abc.ABC):
 
-    def __init__(self, version_from: int, version_till: Optional[int]) -> None:
+    def __init__(self, version_from: int, version_till: Optional[int] = None) -> None:
         self.__version_from = version_from
         self.__version_till = version_till
 
+    @classmethod
+    def get_subclasses(cls) -> Iterator[Type['_Transformable']]:
+        """Recursively get all subclasses of :class:`_Transformable`.
+
+        :yields: each subclass of :class:`_Transformable`.
+        """
+        for subclass in cls.__subclasses__():
+            yield from subclass.get_subclasses()
+            yield subclass
+
     @property
     def version_from(self) -> int:
         return self.__version_from
@@ -43,8 +57,8 @@ class Bool(_Transformable):
 
 class Number(_Transformable):
 
-    def __init__(self, version_from: int, version_till: Optional[int],
-                 min_val: Union[int, float], max_val: Union[int, float], unit: Optional[str]) -> None:
+    def __init__(self, *, version_from: int, version_till: Optional[int] = None, min_val: Union[int, float],
+                 max_val: Union[int, float], unit: Optional[str] = None) -> None:
         super(Number, self).__init__(version_from, version_till)
         self.__min_val = min_val
         self.__max_val = max_val
@@ -99,7 +113,8 @@ class Real(Number):
 
 class Enum(_Transformable):
 
-    def __init__(self, version_from: int, version_till: Optional[int], possible_values: Tuple[str, ...]) -> None:
+    def __init__(self, *, version_from: int, version_till: Optional[int] = None,
+                 possible_values: Tuple[str, ...]) -> None:
         super(Enum, self).__init__(version_from, version_till)
         self.__possible_values = possible_values
 
@@ -128,456 +143,366 @@ class String(_Transformable):
 
 
 # Format:
-#  key - parameter name
-#  value - tuple or multiple tuples if something was changing in GUC across postgres versions
-parameters = CaseInsensitiveDict({
-    'allow_in_place_tablespaces': Bool(150000, None),
-    'allow_system_table_mods': Bool(90300, None),
-    'application_name': String(90300, None),
-    'archive_command': String(90300, None),
-    'archive_library': String(150000, None),
-    'archive_mode': (
-        Bool(90300, 90500),
-        EnumBool(90500, None, ('always',))
-    ),
-    'archive_timeout': Integer(90300, None, 0, 1073741823, 's'),
-    'array_nulls': Bool(90300, None),
-    'authentication_timeout': Integer(90300, None, 1, 600, 's'),
-    'autovacuum': Bool(90300, None),
-    'autovacuum_analyze_scale_factor': Real(90300, None, 0, 100, None),
-    'autovacuum_analyze_threshold': Integer(90300, None, 0, 2147483647, None),
-    'autovacuum_freeze_max_age': Integer(90300, None, 100000, 2000000000, None),
-    'autovacuum_max_workers': (
-        Integer(90300, 90600, 1, 8388607, None),
-        Integer(90600, None, 1, 262143, None)
-    ),
-    'autovacuum_multixact_freeze_max_age': Integer(90300, None, 10000, 2000000000, None),
-    'autovacuum_naptime': Integer(90300, None, 1, 2147483, 's'),
-    'autovacuum_vacuum_cost_delay': (
-        Integer(90300, 120000, -1, 100, 'ms'),
-        Real(120000, None, -1, 100, 'ms')
-    ),
-    'autovacuum_vacuum_cost_limit': Integer(90300, None, -1, 10000, None),
-    'autovacuum_vacuum_insert_scale_factor': Real(130000, None, 0, 100, None),
-    'autovacuum_vacuum_insert_threshold': Integer(130000, None, -1, 2147483647, None),
-    'autovacuum_vacuum_scale_factor': Real(90300, None, 0, 100, None),
-    'autovacuum_vacuum_threshold': Integer(90300, None, 0, 2147483647, None),
-    'autovacuum_work_mem': Integer(90400, None, -1, 2147483647, 'kB'),
-    'backend_flush_after': Integer(90600, None, 0, 256, '8kB'),
-    'backslash_quote': EnumBool(90300, None, ('safe_encoding',)),
-    'backtrace_functions': String(130000, None),
-    'bgwriter_delay': Integer(90300, None, 10, 10000, 'ms'),
-    'bgwriter_flush_after': Integer(90600, None, 0, 256, '8kB'),
-    'bgwriter_lru_maxpages': (
-        Integer(90300, 100000, 0, 1000, None),
-        Integer(100000, None, 0, 1073741823, None)
-    ),
-    'bgwriter_lru_multiplier': Real(90300, None, 0, 10, None),
-    'bonjour': Bool(90300, None),
-    'bonjour_name': String(90300, None),
-    'bytea_output': Enum(90300, None, ('escape', 'hex')),
-    'check_function_bodies': Bool(90300, None),
-    'checkpoint_completion_target': Real(90300, None, 0, 1, None),
-    'checkpoint_flush_after': Integer(90600, None, 0, 256, '8kB'),
-    'checkpoint_segments': Integer(90300, 90500, 1, 2147483647, None),
-    'checkpoint_timeout': (
-        Integer(90300, 90600, 30, 3600, 's'),
-        Integer(90600, None, 30, 86400, 's')
-    ),
-    'checkpoint_warning': Integer(90300, None, 0, 2147483647, 's'),
-    'client_connection_check_interval': Integer(140000, None, 0, 2147483647, 'ms'),
-    'client_encoding': String(90300, None),
-    'client_min_messages': Enum(90300, None, ('debug5', 'debug4', 'debug3', 'debug2',
-                                              'debug1', 'log', 'notice', 'warning', 'error')),
-    'cluster_name': String(90500, None),
-    'commit_delay': Integer(90300, None, 0, 100000, None),
-    'commit_siblings': Integer(90300, None, 0, 1000, None),
-    'compute_query_id': (
-        EnumBool(140000, 150000, ('auto',)),
-        EnumBool(150000, None, ('auto', 'regress'))
-    ),
-    'config_file': String(90300, None),
-    'constraint_exclusion': EnumBool(90300, None, ('partition',)),
-    'cpu_index_tuple_cost': Real(90300, None, 0, 1.79769e+308, None),
-    'cpu_operator_cost': Real(90300, None, 0, 1.79769e+308, None),
-    'cpu_tuple_cost': Real(90300, None, 0, 1.79769e+308, None),
-    'cursor_tuple_fraction': Real(90300, None, 0, 1, None),
-    'data_directory': String(90300, None),
-    'data_sync_retry': Bool(90400, None),
-    'DateStyle': String(90300, None),
-    'db_user_namespace': Bool(90300, None),
-    'deadlock_timeout': Integer(90300, None, 1, 2147483647, 'ms'),
-    'debug_discard_caches': Integer(150000, None, 0, 0, None),
-    'debug_pretty_print': Bool(90300, None),
-    'debug_print_parse': Bool(90300, None),
-    'debug_print_plan': Bool(90300, None),
-    'debug_print_rewritten': Bool(90300, None),
-    'default_statistics_target': Integer(90300, None, 1, 10000, None),
-    'default_table_access_method': String(120000, None),
-    'default_tablespace': String(90300, None),
-    'default_text_search_config': String(90300, None),
-    'default_toast_compression': Enum(140000, None, ('pglz', 'lz4')),
-    'default_transaction_deferrable': Bool(90300, None),
-    'default_transaction_isolation': Enum(90300, None, ('serializable', 'repeatable read',
-                                                        'read committed', 'read uncommitted')),
-    'default_transaction_read_only': Bool(90300, None),
-    'default_with_oids': Bool(90300, 120000),
-    'dynamic_library_path': String(90300, None),
-    'dynamic_shared_memory_type': (
-        Enum(90400, 120000, ('posix', 'sysv', 'mmap', 'none')),
-        Enum(120000, None, ('posix', 'sysv', 'mmap'))
-    ),
-    'effective_cache_size': Integer(90300, None, 1, 2147483647, '8kB'),
-    'effective_io_concurrency': Integer(90300, None, 0, 1000, None),
-    'enable_async_append': Bool(140000, None),
-    'enable_bitmapscan': Bool(90300, None),
-    'enable_gathermerge': Bool(100000, None),
-    'enable_hashagg': Bool(90300, None),
-    'enable_hashjoin': Bool(90300, None),
-    'enable_incremental_sort': Bool(130000, None),
-    'enable_indexonlyscan': Bool(90300, None),
-    'enable_indexscan': Bool(90300, None),
-    'enable_material': Bool(90300, None),
-    'enable_memoize': Bool(150000, None),
-    'enable_mergejoin': Bool(90300, None),
-    'enable_nestloop': Bool(90300, None),
-    'enable_parallel_append': Bool(110000, None),
-    'enable_parallel_hash': Bool(110000, None),
-    'enable_partition_pruning': Bool(110000, None),
-    'enable_partitionwise_aggregate': Bool(110000, None),
-    'enable_partitionwise_join': Bool(110000, None),
-    'enable_seqscan': Bool(90300, None),
-    'enable_sort': Bool(90300, None),
-    'enable_tidscan': Bool(90300, None),
-    'escape_string_warning': Bool(90300, None),
-    'event_source': String(90300, None),
-    'exit_on_error': Bool(90300, None),
-    'extension_destdir': String(140000, None),
-    'external_pid_file': String(90300, None),
-    'extra_float_digits': Integer(90300, None, -15, 3, None),
-    'force_parallel_mode': EnumBool(90600, None, ('regress',)),
-    'from_collapse_limit': Integer(90300, None, 1, 2147483647, None),
-    'fsync': Bool(90300, None),
-    'full_page_writes': Bool(90300, None),
-    'geqo': Bool(90300, None),
-    'geqo_effort': Integer(90300, None, 1, 10, None),
-    'geqo_generations': Integer(90300, None, 0, 2147483647, None),
-    'geqo_pool_size': Integer(90300, None, 0, 2147483647, None),
-    'geqo_seed': Real(90300, None, 0, 1, None),
-    'geqo_selection_bias': Real(90300, None, 1.5, 2, None),
-    'geqo_threshold': Integer(90300, None, 2, 2147483647, None),
-    'gin_fuzzy_search_limit': Integer(90300, None, 0, 2147483647, None),
-    'gin_pending_list_limit': Integer(90500, None, 64, 2147483647, 'kB'),
-    'hash_mem_multiplier': Real(130000, None, 1, 1000, None),
-    'hba_file': String(90300, None),
-    'hot_standby': Bool(90300, None),
-    'hot_standby_feedback': Bool(90300, None),
-    'huge_pages': EnumBool(90400, None, ('try',)),
-    'huge_page_size': Integer(140000, None, 0, 2147483647, 'kB'),
-    'ident_file': String(90300, None),
-    'idle_in_transaction_session_timeout': Integer(90600, None, 0, 2147483647, 'ms'),
-    'idle_session_timeout': Integer(140000, None, 0, 2147483647, 'ms'),
-    'ignore_checksum_failure': Bool(90300, None),
-    'ignore_invalid_pages': Bool(130000, None),
-    'ignore_system_indexes': Bool(90300, None),
-    'IntervalStyle': Enum(90300, None, ('postgres', 'postgres_verbose', 'sql_standard', 'iso_8601')),
-    'jit': Bool(110000, None),
-    'jit_above_cost': Real(110000, None, -1, 1.79769e+308, None),
-    'jit_debugging_support': Bool(110000, None),
-    'jit_dump_bitcode': Bool(110000, None),
-    'jit_expressions': Bool(110000, None),
-    'jit_inline_above_cost': Real(110000, None, -1, 1.79769e+308, None),
-    'jit_optimize_above_cost': Real(110000, None, -1, 1.79769e+308, None),
-    'jit_profiling_support': Bool(110000, None),
-    'jit_provider': String(110000, None),
-    'jit_tuple_deforming': Bool(110000, None),
-    'join_collapse_limit': Integer(90300, None, 1, 2147483647, None),
-    'krb_caseins_users': Bool(90300, None),
-    'krb_server_keyfile': String(90300, None),
-    'krb_srvname': String(90300, 90400),
-    'lc_messages': String(90300, None),
-    'lc_monetary': String(90300, None),
-    'lc_numeric': String(90300, None),
-    'lc_time': String(90300, None),
-    'listen_addresses': String(90300, None),
-    'local_preload_libraries': String(90300, None),
-    'lock_timeout': Integer(90300, None, 0, 2147483647, 'ms'),
-    'lo_compat_privileges': Bool(90300, None),
-    'log_autovacuum_min_duration': Integer(90300, None, -1, 2147483647, 'ms'),
-    'log_checkpoints': Bool(90300, None),
-    'log_connections': Bool(90300, None),
-    'log_destination': String(90300, None),
-    'log_directory': String(90300, None),
-    'log_disconnections': Bool(90300, None),
-    'log_duration': Bool(90300, None),
-    'log_error_verbosity': Enum(90300, None, ('terse', 'default', 'verbose')),
-    'log_executor_stats': Bool(90300, None),
-    'log_file_mode': Integer(90300, None, 0, 511, None),
-    'log_filename': String(90300, None),
-    'logging_collector': Bool(90300, None),
-    'log_hostname': Bool(90300, None),
-    'logical_decoding_work_mem': Integer(130000, None, 64, 2147483647, 'kB'),
-    'log_line_prefix': String(90300, None),
-    'log_lock_waits': Bool(90300, None),
-    'log_min_duration_sample': Integer(130000, None, -1, 2147483647, 'ms'),
-    'log_min_duration_statement': Integer(90300, None, -1, 2147483647, 'ms'),
-    'log_min_error_statement': Enum(90300, None, ('debug5', 'debug4', 'debug3', 'debug2', 'debug1', 'info',
-                                                  'notice', 'warning', 'error', 'log', 'fatal', 'panic')),
-    'log_min_messages': Enum(90300, None, ('debug5', 'debug4', 'debug3', 'debug2', 'debug1', 'info',
-                                           'notice', 'warning', 'error', 'log', 'fatal', 'panic')),
-    'log_parameter_max_length': Integer(130000, None, -1, 1073741823, 'B'),
-    'log_parameter_max_length_on_error': Integer(130000, None, -1, 1073741823, 'B'),
-    'log_parser_stats': Bool(90300, None),
-    'log_planner_stats': Bool(90300, None),
-    'log_recovery_conflict_waits': Bool(140000, None),
-    'log_replication_commands': Bool(90500, None),
-    'log_rotation_age': Integer(90300, None, 0, 35791394, 'min'),
-    'log_rotation_size': Integer(90300, None, 0, 2097151, 'kB'),
-    'log_startup_progress_interval': Integer(150000, None, 0, 2147483647, 'ms'),
-    'log_statement': Enum(90300, None, ('none', 'ddl', 'mod', 'all')),
-    'log_statement_sample_rate': Real(130000, None, 0, 1, None),
-    'log_statement_stats': Bool(90300, None),
-    'log_temp_files': Integer(90300, None, -1, 2147483647, 'kB'),
-    'log_timezone': String(90300, None),
-    'log_transaction_sample_rate': Real(120000, None, 0, 1, None),
-    'log_truncate_on_rotation': Bool(90300, None),
-    'maintenance_io_concurrency': Integer(130000, None, 0, 1000, None),
-    'maintenance_work_mem': Integer(90300, None, 1024, 2147483647, 'kB'),
-    'max_connections': (
-        Integer(90300, 90600, 1, 8388607, None),
-        Integer(90600, None, 1, 262143, None)
-    ),
-    'max_files_per_process': (
-        Integer(90300, 130000, 25, 2147483647, None),
-        Integer(130000, None, 64, 2147483647, None)
-    ),
-    'max_locks_per_transaction': Integer(90300, None, 10, 2147483647, None),
-    'max_logical_replication_workers': Integer(100000, None, 0, 262143, None),
-    'max_parallel_maintenance_workers': Integer(110000, None, 0, 1024, None),
-    'max_parallel_workers': Integer(100000, None, 0, 1024, None),
-    'max_parallel_workers_per_gather': Integer(90600, None, 0, 1024, None),
-    'max_pred_locks_per_page': Integer(100000, None, 0, 2147483647, None),
-    'max_pred_locks_per_relation': Integer(100000, None, -2147483648, 2147483647, None),
-    'max_pred_locks_per_transaction': Integer(90300, None, 10, 2147483647, None),
-    'max_prepared_transactions': (
-        Integer(90300, 90600, 0, 8388607, None),
-        Integer(90600, None, 0, 262143, None)
-    ),
-    'max_replication_slots': (
-        Integer(90400, 90600, 0, 8388607, None),
-        Integer(90600, None, 0, 262143, None)
-    ),
-    'max_slot_wal_keep_size': Integer(130000, None, -1, 2147483647, 'MB'),
-    'max_stack_depth': Integer(90300, None, 100, 2147483647, 'kB'),
-    'max_standby_archive_delay': Integer(90300, None, -1, 2147483647, 'ms'),
-    'max_standby_streaming_delay': Integer(90300, None, -1, 2147483647, 'ms'),
-    'max_sync_workers_per_subscription': Integer(100000, None, 0, 262143, None),
-    'max_wal_senders': (
-        Integer(90300, 90600, 0, 8388607, None),
-        Integer(90600, None, 0, 262143, None)
-    ),
-    'max_wal_size': (
-        Integer(90500, 100000, 2, 2147483647, '16MB'),
-        Integer(100000, None, 2, 2147483647, 'MB')
-    ),
-    'max_worker_processes': (
-        Integer(90400, 90600, 1, 8388607, None),
-        Integer(90600, None, 0, 262143, None)
-    ),
-    'min_dynamic_shared_memory': Integer(140000, None, 0, 2147483647, 'MB'),
-    'min_parallel_index_scan_size': Integer(100000, None, 0, 715827882, '8kB'),
-    'min_parallel_relation_size': Integer(90600, 100000, 0, 715827882, '8kB'),
-    'min_parallel_table_scan_size': Integer(100000, None, 0, 715827882, '8kB'),
-    'min_wal_size': (
-        Integer(90500, 100000, 2, 2147483647, '16MB'),
-        Integer(100000, None, 2, 2147483647, 'MB')
-    ),
-    'old_snapshot_threshold': Integer(90600, None, -1, 86400, 'min'),
-    'operator_precedence_warning': Bool(90500, 140000),
-    'parallel_leader_participation': Bool(110000, None),
-    'parallel_setup_cost': Real(90600, None, 0, 1.79769e+308, None),
-    'parallel_tuple_cost': Real(90600, None, 0, 1.79769e+308, None),
-    'password_encryption': (
-        Bool(90300, 100000),
-        Enum(100000, None, ('md5', 'scram-sha-256'))
-    ),
-    'plan_cache_mode': Enum(120000, None, ('auto', 'force_generic_plan', 'force_custom_plan')),
-    'port': Integer(90300, None, 1, 65535, None),
-    'post_auth_delay': Integer(90300, None, 0, 2147, 's'),
-    'pre_auth_delay': Integer(90300, None, 0, 60, 's'),
-    'quote_all_identifiers': Bool(90300, None),
-    'random_page_cost': Real(90300, None, 0, 1.79769e+308, None),
-    'recovery_init_sync_method': Enum(140000, None, ('fsync', 'syncfs')),
-    'recovery_prefetch': EnumBool(150000, None, ('try',)),
-    'recursive_worktable_factor': Real(150000, None, 0.001, 1e+06, None),
-    'remove_temp_files_after_crash': Bool(140000, None),
-    'replacement_sort_tuples': Integer(90600, 110000, 0, 2147483647, None),
-    'restart_after_crash': Bool(90300, None),
-    'row_security': Bool(90500, None),
-    'search_path': String(90300, None),
-    'seq_page_cost': Real(90300, None, 0, 1.79769e+308, None),
-    'session_preload_libraries': String(90400, None),
-    'session_replication_role': Enum(90300, None, ('origin', 'replica', 'local')),
-    'shared_buffers': Integer(90300, None, 16, 1073741823, '8kB'),
-    'shared_memory_type': Enum(120000, None, ('sysv', 'mmap')),
-    'shared_preload_libraries': String(90300, None),
-    'sql_inheritance': Bool(90300, 100000),
-    'ssl': Bool(90300, None),
-    'ssl_ca_file': String(90300, None),
-    'ssl_cert_file': String(90300, None),
-    'ssl_ciphers': String(90300, None),
-    'ssl_crl_dir': String(140000, None),
-    'ssl_crl_file': String(90300, None),
-    'ssl_dh_params_file': String(100000, None),
-    'ssl_ecdh_curve': String(90400, None),
-    'ssl_key_file': String(90300, None),
-    'ssl_max_protocol_version': Enum(120000, None, ('', 'tlsv1', 'tlsv1.1', 'tlsv1.2', 'tlsv1.3')),
-    'ssl_min_protocol_version': Enum(120000, None, ('tlsv1', 'tlsv1.1', 'tlsv1.2', 'tlsv1.3')),
-    'ssl_passphrase_command': String(110000, None),
-    'ssl_passphrase_command_supports_reload': Bool(110000, None),
-    'ssl_prefer_server_ciphers': Bool(90400, None),
-    'ssl_renegotiation_limit': Integer(90300, 90500, 0, 2147483647, 'kB'),
-    'standard_conforming_strings': Bool(90300, None),
-    'statement_timeout': Integer(90300, None, 0, 2147483647, 'ms'),
-    'stats_fetch_consistency': Enum(150000, None, ('none', 'cache', 'snapshot')),
-    'stats_temp_directory': String(90300, 150000),
-    'superuser_reserved_connections': (
-        Integer(90300, 90600, 0, 8388607, None),
-        Integer(90600, None, 0, 262143, None)
-    ),
-    'synchronize_seqscans': Bool(90300, None),
-    'synchronous_commit': (
-        EnumBool(90300, 90600, ('local', 'remote_write')),
-        EnumBool(90600, None, ('local', 'remote_write', 'remote_apply'))
-    ),
-    'synchronous_standby_names': String(90300, None),
-    'syslog_facility': Enum(90300, None, ('local0', 'local1', 'local2', 'local3',
-                                          'local4', 'local5', 'local6', 'local7')),
-    'syslog_ident': String(90300, None),
-    'syslog_sequence_numbers': Bool(90600, None),
-    'syslog_split_messages': Bool(90600, None),
-    'tcp_keepalives_count': Integer(90300, None, 0, 2147483647, None),
-    'tcp_keepalives_idle': Integer(90300, None, 0, 2147483647, 's'),
-    'tcp_keepalives_interval': Integer(90300, None, 0, 2147483647, 's'),
-    'tcp_user_timeout': Integer(120000, None, 0, 2147483647, 'ms'),
-    'temp_buffers': Integer(90300, None, 100, 1073741823, '8kB'),
-    'temp_file_limit': Integer(90300, None, -1, 2147483647, 'kB'),
-    'temp_tablespaces': String(90300, None),
-    'TimeZone': String(90300, None),
-    'timezone_abbreviations': String(90300, None),
-    'trace_notify': Bool(90300, None),
-    'trace_recovery_messages': Enum(90300, None, ('debug5', 'debug4', 'debug3', 'debug2',
-                                                  'debug1', 'log', 'notice', 'warning', 'error')),
-    'trace_sort': Bool(90300, None),
-    'track_activities': Bool(90300, None),
-    'track_activity_query_size': (
-        Integer(90300, 110000, 100, 102400, None),
-        Integer(110000, 130000, 100, 102400, 'B'),
-        Integer(130000, None, 100, 1048576, 'B')
-    ),
-    'track_commit_timestamp': Bool(90500, None),
-    'track_counts': Bool(90300, None),
-    'track_functions': Enum(90300, None, ('none', 'pl', 'all')),
-    'track_io_timing': Bool(90300, None),
-    'track_wal_io_timing': Bool(140000, None),
-    'transaction_deferrable': Bool(90300, None),
-    'transaction_isolation': Enum(90300, None, ('serializable', 'repeatable read',
-                                                'read committed', 'read uncommitted')),
-    'transaction_read_only': Bool(90300, None),
-    'transform_null_equals': Bool(90300, None),
-    'unix_socket_directories': String(90300, None),
-    'unix_socket_group': String(90300, None),
-    'unix_socket_permissions': Integer(90300, None, 0, 511, None),
-    'update_process_title': Bool(90300, None),
-    'vacuum_cleanup_index_scale_factor': Real(110000, 140000, 0, 1e+10, None),
-    'vacuum_cost_delay': (
-        Integer(90300, 120000, 0, 100, 'ms'),
-        Real(120000, None, 0, 100, 'ms')
-    ),
-    'vacuum_cost_limit': Integer(90300, None, 1, 10000, None),
-    'vacuum_cost_page_dirty': Integer(90300, None, 0, 10000, None),
-    'vacuum_cost_page_hit': Integer(90300, None, 0, 10000, None),
-    'vacuum_cost_page_miss': Integer(90300, None, 0, 10000, None),
-    'vacuum_defer_cleanup_age': Integer(90300, None, 0, 1000000, None),
-    'vacuum_failsafe_age': Integer(140000, None, 0, 2100000000, None),
-    'vacuum_freeze_min_age': Integer(90300, None, 0, 1000000000, None),
-    'vacuum_freeze_table_age': Integer(90300, None, 0, 2000000000, None),
-    'vacuum_multixact_failsafe_age': Integer(140000, None, 0, 2100000000, None),
-    'vacuum_multixact_freeze_min_age': Integer(90300, None, 0, 1000000000, None),
-    'vacuum_multixact_freeze_table_age': Integer(90300, None, 0, 2000000000, None),
-    'wal_buffers': Integer(90300, None, -1, 262143, '8kB'),
-    'wal_compression': (
-        Bool(90500, 150000),
-        EnumBool(150000, None, ('pglz', 'lz4', 'zstd'))
-    ),
-    'wal_consistency_checking': String(100000, None),
-    'wal_decode_buffer_size': Integer(150000, None, 65536, 1073741823, 'B'),
-    'wal_init_zero': Bool(120000, None),
-    'wal_keep_segments': Integer(90300, 130000, 0, 2147483647, None),
-    'wal_keep_size': Integer(130000, None, 0, 2147483647, 'MB'),
-    'wal_level': (
-        Enum(90300, 90400, ('minimal', 'archive', 'hot_standby')),
-        Enum(90400, 90600, ('minimal', 'archive', 'hot_standby', 'logical')),
-        Enum(90600, None, ('minimal', 'replica', 'logical'))
-    ),
-    'wal_log_hints': Bool(90400, None),
-    'wal_receiver_create_temp_slot': Bool(130000, None),
-    'wal_receiver_status_interval': Integer(90300, None, 0, 2147483, 's'),
-    'wal_receiver_timeout': Integer(90300, None, 0, 2147483647, 'ms'),
-    'wal_recycle': Bool(120000, None),
-    'wal_retrieve_retry_interval': Integer(90500, None, 1, 2147483647, 'ms'),
-    'wal_sender_timeout': Integer(90300, None, 0, 2147483647, 'ms'),
-    'wal_skip_threshold': Integer(130000, None, 0, 2147483647, 'kB'),
-    'wal_sync_method': Enum(90300, None, ('fsync', 'fdatasync', 'open_sync', 'open_datasync')),
-    'wal_writer_delay': Integer(90300, None, 1, 10000, 'ms'),
-    'wal_writer_flush_after': Integer(90600, None, 0, 2147483647, '8kB'),
-    'work_mem': Integer(90300, None, 64, 2147483647, 'kB'),
-    'xmlbinary': Enum(90300, None, ('base64', 'hex')),
-    'xmloption': Enum(90300, None, ('content', 'document')),
-    'zero_damaged_pages': Bool(90300, None)
-})
+#  key  - parameter name
+#  value - variable length tuple of `_Transformable` objects. Each object in the tuple represents a different
+#          validation of the GUC across postgres versions. If a GUC validation has never changed over time, then it will
+#          have a single object in the tuple. For example, `password_encryption` used to be a boolean GUC up to Postgres
+#          10, at which point it started being an enum. In that case the value of `password_encryption` would be a tuple
+#          of 2 `_Transformable` objects (`Bool` and `Enum`, respectively), each one reprensenting a different
+#          validation rule.
+parameters = CaseInsensitiveDict()
+recovery_parameters = CaseInsensitiveDict()
 
 
-recovery_parameters = CaseInsensitiveDict({
-    'archive_cleanup_command': String(90300, None),
-    'pause_at_recovery_target': Bool(90300, 90500),
-    'primary_conninfo': String(90300, None),
-    'primary_slot_name': String(90400, None),
-    'promote_trigger_file': String(120000, None),
-    'recovery_end_command': String(90300, None),
-    'recovery_min_apply_delay': Integer(90400, None, 0, 2147483647, 'ms'),
-    'recovery_target': Enum(90400, None, ('immediate', '')),
-    'recovery_target_action': Enum(90500, None, ('pause', 'promote', 'shutdown')),
-    'recovery_target_inclusive': Bool(90300, None),
-    'recovery_target_lsn': String(100000, None),
-    'recovery_target_name': String(90400, None),
-    'recovery_target_time': String(90300, None),
-    'recovery_target_timeline': String(90300, None),
-    'recovery_target_xid': String(90300, None),
-    'restore_command': String(90300, None),
-    'standby_mode': Bool(90300, 120000),
-    'trigger_file': String(90300, 120000)
-})
+class ValidatorFactoryNoType(PatroniException):
+    """Raised when a validator spec misses a type."""
 
 
-def _transform_parameter_value(validators: MutableMapping[str, Union[_Transformable, Tuple[_Transformable, ...]]],
-                               version: int, name: str, value: Any) -> Optional[Any]:
-    name_validators = validators.get(name)
-    if name_validators:
-        for validator in (name_validators if isinstance(name_validators, tuple) else (name_validators,)):
+class ValidatorFactoryInvalidType(PatroniException):
+    """Raised when a validator spec contains an invalid type."""
+
+
+class ValidatorFactoryInvalidSpec(PatroniException):
+    """Raised when a validator spec contains an invalid set of attributes."""
+
+
+class ValidatorFactory:
+    """Factory class used to build Patroni validator objects based on the given specs."""
+
+    TYPES: Dict[str, Type[_Transformable]] = {cls.__name__: cls for cls in _Transformable.get_subclasses()}
+
+    def __new__(cls, validator: Dict[str, Any]) -> _Transformable:
+        """Parse a given Postgres GUC *validator* into the corresponding Patroni validator object.
+
+        :param validator: a validator spec for a given parameter. It usually comes from a parsed YAML file.
+
+        :returns: the Patroni validator object that corresponds to the specification found in *validator*.
+
+        :raises :class:`ValidatorFactoryNoType`: if *validator* contains no ``type`` key.
+        :raises :class:`ValidatorFactoryInvalidType`: if ``type`` key from *validator* contains an invalid value.
+        :raises :class:`ValidatorFactoryInvalidSpec`: if *validator* contains an invalid set of attributes for the
+            given ``type``.
+
+        :Example:
+
+            If a given validator was defined as follows in the YAML file:
+
+            ```yaml
+            - type: String
+            version_from: 90300
+            version_till: null
+            ```
+
+            Then this method would receive *validator* as:
+
+            ```python
+            {
+                'type': 'String',
+                'version_from': 90300,
+                'version_till': None
+            }
+            ```
+
+            And this method would return a :class:`String`:
+
+            ```python
+            String(90300, None)
+            ```
+        """
+        validator = deepcopy(validator)
+        try:
+            type_ = validator.pop('type')
+        except KeyError as exc:
+            raise ValidatorFactoryNoType('Validator contains no type.') from exc
+
+        if type_ not in cls.TYPES:
+            raise ValidatorFactoryInvalidType(f'Unexpected validator type: `{type_}`.')
+
+        for key, value in validator.items():
+            # :func:`_transform_parameter_value` expects :class:`tuple` instead of :class:`list`
+            if isinstance(value, list):
+                tmp_value: List[Any] = value
+                validator[key] = tuple(tmp_value)
+
+        try:
+            return cls.TYPES[type_](**validator)
+        except Exception as exc:
+            raise ValidatorFactoryInvalidSpec(
+                f'Failed to parse `{type_}` validator (`{validator}`): `{str(exc)}`.') from exc
+
+
+def _get_postgres_guc_validators(config: Dict[str, Any], parameter: str) -> Tuple[_Transformable, ...]:
+    """Get all validators of *parameter* from *config*.
+
+    Loop over all validators specs of *parameter* and return them parsed as Patroni validators.
+
+    :param config: Python object corresponding to an YAML file, with values of either ``parameters`` or
+        ``recovery_parameters`` key.
+    :param parameter: name of the parameter found under *config* which validators should be parsed and returned.
+
+    :rtype: yields any exception that is faced while parsing a validator spec into a Patroni validator object.
+    """
+    validators: List[_Transformable] = []
+    for validator_spec in config.get(parameter, []):
+        try:
+            validator = ValidatorFactory(validator_spec)
+            validators.append(validator)
+        except (ValidatorFactoryNoType, ValidatorFactoryInvalidType, ValidatorFactoryInvalidSpec) as exc:
+            logger.warning('Faced an issue while parsing a validator for parameter `%s`: `%r`', parameter, exc)
+
+    return tuple(validators)
+
+
+class InvalidGucValidatorsFile(PatroniException):
+    """Raised when reading or parsing of a YAML file faces an issue."""
+
+
+def _read_postgres_gucs_validators_file(file: str) -> Dict[str, Any]:
+    """Read an YAML file and return the corresponding Python object.
+
+    :param file: path to the file to be read. It is expected to be encoded with ``UTF-8``, and to be a YAML document.
+
+    :returns: the YAML content parsed into a Python object. If any issue is faced while reading/parsing the file, then
+        return ``None``.
+
+    :raises :class:`InvalidGucValidatorsFile`: if faces an issue while reading or parsing *file*.
+    """
+    try:
+        with open(file, encoding='UTF-8') as stream:
+            return yaml.safe_load(stream)
+    except Exception as exc:
+        raise InvalidGucValidatorsFile(
+            f'Unexpected issue while reading parameters file `{file}`: `{str(exc)}`.') from exc
+
+
+def _load_postgres_gucs_validators() -> None:
+    """Load all Postgres GUC validators from YAML files.
+
+    Recursively walk through ``available_parameters`` directory and load validators of each found YAML file into
+    ``parameters`` and/or ``recovery_parameters`` variables.
+
+    Walk through directories in top-down fashion and for each of them:
+        * Sort files by name;
+        * Load validators from YAML files that were found.
+
+    Any problem faced while reading or parsing files will be logged as a ``WARNING`` by the child function, and the
+    corresponding file or validator will be ignored.
+
+    By default Patroni only ships the file ``0_postgres.yml``, which contains Community Postgres GUCs validators, but
+    that behavior can be extended. For example: if a vendor wants to add GUC validators to Patroni for covering a custom
+    Postgres build, then they can create their custom YAML files under ``available_parameters`` directory.
+
+    Each YAML file may contain either or both of these root attributes, here called sections:
+        * ``parameters``: general GUCs that would be written to ``postgresql.conf``;
+        * ``recovery_parameters``: recovery related GUCs that would be written to ``recovery.conf`` (Patroni later
+            writes them to ``postgresql.conf`` if running PG 12 and above).
+
+    Then, each of these sections, if specified, may contain one or more attributes with the following structure:
+        * key: the name of a GUC;
+        * value: a list of validators. Each item in the list must contain a ``type`` attribute, which must be one among:
+            * ``Bool``; or
+            * ``Integer``; or
+            * ``Real``; or
+            * ``Enum``; or
+            * ``EnumBool``; or
+            * ``String``.
+
+            Besides the ``type`` attribute, it should also contain all the required attributes as per the corresponding
+            class in this module.
+
+    .. seealso::
+        * :class:`Bool`;
+        * :class:`Integer`;
+        * :class:`Real`;
+        * :class:`Enum`;
+        * :class:`EnumBool`;
+        * :class:`String`.
+
+    :Example:
+
+        This is a sample content for an YAML file based on Postgres GUCs, showing each of the supported types and
+        sections:
+
+        ```yaml
+        parameters:
+          archive_command:
+          - type: String
+            version_from: 90300
+            version_till: null
+          archive_mode:
+          - type: Bool
+            version_from: 90300
+            version_till: 90500
+          - type: EnumBool
+            version_from: 90500
+            version_till: null
+            possible_values:
+            - always
+          archive_timeout:
+          - type: Integer
+            version_from: 90300
+            version_till: null
+            min_val: 0
+            max_val: 1073741823
+            unit: s
+          autovacuum_vacuum_cost_delay:
+          - type: Integer
+            version_from: 90300
+            version_till: 120000
+            min_val: -1
+            max_val: 100
+            unit: ms
+          - type: Real
+            version_from: 120000
+            version_till: null
+            min_val: -1
+            max_val: 100
+            unit: ms
+          client_min_messages:
+          - type: Enum
+            version_from: 90300
+            version_till: null
+            possible_values:
+            - debug5
+            - debug4
+            - debug3
+            - debug2
+            - debug1
+            - log
+            - notice
+            - warning
+            - error
+        recovery_parameters:
+          archive_cleanup_command:
+          - type: String
+            version_from: 90300
+            version_till: null
+        ```
+    """
+    conf_dir = os.path.join(
+        os.path.dirname(os.path.abspath(__file__)),
+        'available_parameters',
+    )
+    yaml_files: List[str] = []
+
+    for root, _, files in os.walk(conf_dir):
+        for file in sorted(files):
+            full_path = os.path.join(root, file)
+            if file.lower().endswith(('.yml', '.yaml')):
+                yaml_files.append(full_path)
+            else:
+                logger.info('Ignored a non-YAML file found under `available_parameters` directory: `%s`.', full_path)
+
+    for file in yaml_files:
+        try:
+            config: Dict[str, Any] = _read_postgres_gucs_validators_file(file)
+        except InvalidGucValidatorsFile as exc:
+            logger.warning(str(exc))
+            continue
+
+        logger.debug(f'Parsing validators from file `{file}`.')
+
+        mapping = {
+            'parameters': parameters,
+            'recovery_parameters': recovery_parameters,
+        }
+
+        for section in ['parameters', 'recovery_parameters']:
+            section_var = mapping[section]
+
+            config_section = config.get(section, {})
+            for parameter in config_section.keys():
+                section_var[parameter] = _get_postgres_guc_validators(config_section, parameter)
+
+
+_load_postgres_gucs_validators()
+
+
+def _transform_parameter_value(validators: MutableMapping[str, Tuple[_Transformable, ...]],
+                               version: int, name: str, value: Any,
+                               available_gucs: CaseInsensitiveSet) -> Optional[Any]:
+    """Validate *value* of GUC *name* for Postgres *version* using defined *validators* and *available_gucs*.
+
+    :param validators: a dictionary of all GUCs across all Postgres versions. Each key is the name of a Postgres GUC,
+        and the corresponding value is a variable length tuple of :class:`_Transformable`. Each item is a validation
+        rule for the GUC for a given range of Postgres versions. Should either contain recovery GUCs or general GUCs,
+        not both.
+    :param version: Postgres version to validate the GUC against.
+    :param name: name of the Postgres GUC.
+    :param value: value of the Postgres GUC.
+    :param available_gucs: a set of all GUCs available in Postgres *version*. Each item is the name of a Postgres
+        GUC. Used for a couple purposes:
+        * Disallow writing GUCs to ``postgresql.conf`` (or ``recovery.conf``) that does not exist in Postgres *version*;
+        * Avoid ignoring GUC *name* if it does not have a validator in *validators*, but is a valid GUC in Postgres
+            *version*.
+
+    :returns: the return value may be one among:
+        * *value* transformed to the expected format for GUC *name* in Postgres *version*, if *name* is present in
+            *available_gucs* and has a validator in *validators* for the corresponding Postgres *version*; or
+        * The own *value* if *name* is present in *available_gucs* but not in *validators*; or
+        * ``None`` if *name* is not present in *available_gucs*.
+    """
+    if name in available_gucs:
+        for validator in validators.get(name, ()) or ():
             if version >= validator.version_from and\
                     (validator.version_till is None or version < validator.version_till):
                 return validator.transform(name, value)
+        # Ideally we should have a validator in *validators*. However, if none is available, we will not discard a
+        # setting that exists in Postgres *version*, but rather allow the value with no validation.
+        return value
     logger.warning('Removing unexpected parameter=%s value=%s from the config', name, value)
 
 
-def transform_postgresql_parameter_value(version: int, name: str, value: Any) -> Optional[Any]:
-    if '.' in name:
+def transform_postgresql_parameter_value(version: int, name: str, value: Any,
+                                         available_gucs: CaseInsensitiveSet) -> Optional[Any]:
+    """Validate *value* of GUC *name* for Postgres *version* using ``parameters`` and *available_gucs*.
+
+    :param version: Postgres version to validate the GUC against.
+    :param name: name of the Postgres GUC.
+    :param value: value of the Postgres GUC.
+    :param available_gucs: a set of all GUCs available in Postgres *version*. Each item is the name of a Postgres
+        GUC. Used for a couple purposes:
+        * Disallow writing GUCs to ``postgresql.conf`` that does not exist in Postgres *version*;
+        * Avoid ignoring GUC *name* if it does not have a validator in ``parameters``, but is a valid GUC in Postgres
+            *version*.
+
+    :returns: The return value may be one among
+        * The original *value* if *name* seems to be an extension GUC (contains a period '.'); or
+        * ``None`` if **name** is a recovery GUC; or
+        * *value* transformed to the expected format for GUC *name* in Postgres *version* using validators defined in
+            ``parameters``. Can also return ``None``. See :func:`_transform_parameter_value`.
+    """
+    if '.' in name and name not in parameters:
+        # likely an extension GUC, so just return as it is. Otherwise, if `name` is in `parameters`, it's likely a
+        # namespaced GUC from a custom Postgres build, so we treat that over the usual validation means.
         return value
     if name in recovery_parameters:
         return None
-    return _transform_parameter_value(parameters, version, name, value)
+    return _transform_parameter_value(parameters, version, name, value, available_gucs)
 
 
-def transform_recovery_parameter_value(version: int, name: str, value: Any) -> Optional[Any]:
-    return _transform_parameter_value(recovery_parameters, version, name, value)
+def transform_recovery_parameter_value(version: int, name: str, value: Any,
+                                       available_gucs: CaseInsensitiveSet) -> Optional[Any]:
+    """Validate *value* of GUC *name* for Postgres *version* using ``recovery_parameters`` and *available_gucs*.
+
+    :param version: Postgres version to validate the recovery GUC against.
+    :param name: name of the Postgres recovery GUC.
+    :param value: value of the Postgres recovery GUC.
+    :param available_gucs: a set of all GUCs available in Postgres *version*. Each item is the name of a Postgres
+        GUC. Used for a couple purposes:
+        * Disallow writing GUCs to ``recovery.conf`` (or ``postgresql.conf`` depending on *version*), that does not
+        exist in Postgres *version*;
+        * Avoid ignoring recovery GUC *name* if it does not have a validator in ``recovery_parameters``, but is a valid
+            GUC in Postgres *version*.
+
+    :returns: *value* transformed to the expected format for recovery GUC *name* in Postgres *version* using validators
+        defined in ``recovery_parameters``. It can also return ``None``. See :func:`_transform_parameter_value`.
+    """
+    # Recovery settings are not present in ``postgres --describe-config`` output of Postgres <= 11. In that case we
+    # just pass down the list of settings defined in Patroni validators so :func:`_transform_parameter_value` will not
+    # discard the recovery GUCs when running Postgres <= 11.
+    # NOTE: At the moment this change was done Postgres 11 was almost EOL, and had been likely extensively used with
+    # Patroni, so we should be able to rely solely on Patroni validators as the source of truth.
+    return _transform_parameter_value(
+        recovery_parameters, version, name, value,
+        available_gucs if version >= 120000 else CaseInsensitiveSet(recovery_parameters.keys()))

patroni/version.py

@@ -2,4 +2,4 @@
 
 :var __version__: the current Patroni version.
 """
-__version__ = '3.0.2'
+__version__ = '3.0.3'

release.sh

@@ -1,11 +1,12 @@
 #!/bin/bash
 
 # Release process:
-# 1. Open a PR that updates release notes and Patroni version
-# 2. Merge it
-# 3. Run release.sh
-# 4. After the new tag is pushed, the .github/workflows/release.yaml will run tests and upload the new package to test.pypi.org
-# 5. Once the release is created, the .github/workflows/release.yaml will run tests and upload the new package to pypi.org
+# 1. Open a PR that updates release notes, Patroni version and pyright version in the tests workflow.
+# 2. Resolve possible typing issues.
+# 3. Merge the PR.
+# 4. Run release.sh
+# 5. After the new tag is pushed, the .github/workflows/release.yaml will run tests and upload the new package to test.pypi.org
+# 6. Once the release is created, the .github/workflows/release.yaml will run tests and upload the new package to pypi.org
 
 ## Bail out on any non-zero exitcode from the called processes
 set -xe

setup.py

@@ -88,7 +88,7 @@ class Flake8(_Command):
                 yield package_directory
 
     def targets(self):
-        return [package for package in self.package_files()] + ['tests', 'setup.py']
+        return [package for package in self.package_files()] + ['tests', 'features', 'setup.py']
 
     def run(self):
         from flake8.main.cli import main
@@ -116,7 +116,7 @@ class PyTest(_Command):
 
 
 def read(fname):
-    with open(os.path.join(__location__, fname)) as fd:
+    with open(os.path.join(__location__, fname), encoding='utf-8') as fd:
         return fd.read()
 
 
@@ -157,7 +157,10 @@ def setup_package(version):
         long_description=read('README.rst'),
         classifiers=CLASSIFIERS,
         packages=find_packages(exclude=['tests', 'tests.*']),
-        package_data={MAIN_PACKAGE: ["*.json"]},
+        package_data={MAIN_PACKAGE: [
+            "postgresql/available_parameters/*.yml",
+            "postgresql/available_parameters/*.yaml",
+        ]},
         install_requires=install_requires,
         extras_require=EXTRAS_REQUIRE,
         cmdclass=cmdclass,

tests/__init__.py

@@ -3,7 +3,7 @@ import os
 import shutil
 import unittest
 
-from mock import Mock, patch
+from mock import Mock, PropertyMock, patch
 
 import urllib3
 
@@ -19,6 +19,15 @@ class SleepException(Exception):
     pass
 
 
+mock_available_gucs = PropertyMock(return_value={
+    'cluster_name', 'constraint_exclusion', 'force_parallel_mode', 'hot_standby', 'listen_addresses', 'max_connections',
+    'max_locks_per_transaction', 'max_prepared_transactions', 'max_replication_slots', 'max_stack_depth',
+    'max_wal_senders', 'max_worker_processes', 'port', 'search_path', 'shared_preload_libraries',
+    'stats_temp_directory', 'synchronous_standby_names', 'track_commit_timestamp', 'unix_socket_directories',
+    'vacuum_cost_delay', 'vacuum_cost_limit', 'wal_keep_size', 'wal_level', 'wal_log_hints', 'zero_damaged_pages',
+})
+
+
 class MockResponse(object):
 
     def __init__(self, status_code=200):

tests/test_api.py

@@ -180,7 +180,6 @@ class MockRestApiServer(RestApiServer):
 
 @patch('ssl.SSLContext.load_cert_chain', Mock())
 @patch('ssl.SSLContext.wrap_socket', Mock(return_value=0))
-@patch('ssl.SSLContext.load_verify_locations', Mock(return_value=[Mock()]))
 @patch.object(HTTPServer, '__init__', Mock())
 class TestRestApiHandler(unittest.TestCase):
 
@@ -590,7 +589,6 @@ class TestRestApiServer(unittest.TestCase):
     @patch('ssl.SSLContext.load_cert_chain', Mock())
     @patch('ssl.SSLContext.set_ciphers', Mock())
     @patch('ssl.SSLContext.wrap_socket', Mock(return_value=0))
-    @patch('ssl.SSLContext.load_verify_locations', Mock(return_value=[Mock()]))
     @patch.object(HTTPServer, '__init__', Mock())
     def setUp(self):
         self.srv = MockRestApiServer(Mock(), '', {'listen': '*:8008', 'certfile': 'a', 'verify_client': 'required',
@@ -653,10 +651,9 @@ class TestRestApiServer(unittest.TestCase):
         mock_get_request.return_value = (self.__create_socket(), ('127.0.0.1', 55555))
         self.srv._handle_request_noblock()
 
-    @patch('ssl.SSLContext.load_verify_locations', Mock(return_value=[Mock()]))
+    @patch('ssl._ssl._test_decode_cert', Mock())
     def test_reload_local_certificate(self):
         self.assertTrue(self.srv.reload_local_certificate())
 
-    @patch('ssl.SSLContext.load_verify_locations', Mock(side_effect=Exception))
     def test_get_certificate_serial_number(self):
         self.assertIsNone(self.srv.get_certificate_serial_number())

tests/test_bootstrap.py

@@ -9,12 +9,13 @@ from patroni.postgresql.bootstrap import Bootstrap
 from patroni.postgresql.cancellable import CancellableSubprocess
 from patroni.postgresql.config import ConfigHandler
 
-from . import psycopg_connect, BaseTestPostgresql
+from . import psycopg_connect, BaseTestPostgresql, mock_available_gucs
 
 
 @patch('subprocess.call', Mock(return_value=0))
 @patch('patroni.psycopg.connect', psycopg_connect)
 @patch('os.rename', Mock())
+@patch.object(Postgresql, 'available_gucs', mock_available_gucs)
 class TestBootstrap(BaseTestPostgresql):
 
     @patch('patroni.postgresql.CallbackExecutor', Mock())

tests/test_citus.py

@@ -1,3 +1,4 @@
+import time
 from mock import Mock, patch
 from patroni.postgresql.citus import CitusHandler
 
@@ -16,7 +17,7 @@ class TestCitus(BaseTestPostgresql):
         self.cluster = get_cluster_initialized_with_leader()
         self.cluster.workers[1] = self.cluster
 
-    @patch('time.time', Mock(side_effect=[100, 130, 160, 190, 220, 250, 280, 310]))
+    @patch('time.time', Mock(side_effect=[100, 130, 160, 190, 220, 250, 280, 310, 340, 370]))
     @patch('patroni.postgresql.citus.logger.exception', Mock(side_effect=SleepException))
     @patch('patroni.postgresql.citus.logger.warning')
     @patch('patroni.postgresql.citus.PgDistNode.wait', Mock())
@@ -66,11 +67,14 @@ class TestCitus(BaseTestPostgresql):
             mock_logger.assert_called_once()
             self.assertTrue(mock_logger.call_args[0][0].startswith('Overriding existing task:'))
 
-        # add_task called from sync_pg_dist_node should not override already scheduled or in flight task
+        # add_task called from sync_pg_dist_node should not override already scheduled or in flight task until deadline
         self.assertIsNotNone(self.c.add_task('after_promote', 1, 'postgres://host:5432/postgres', 30))
         self.assertIsNone(self.c.add_task('after_promote', 1, 'postgres://host:5432/postgres'))
         self.c._in_flight = self.c._tasks.pop()
+        self.c._in_flight.deadline = self.c._in_flight.timeout + time.time()
         self.assertIsNone(self.c.add_task('after_promote', 1, 'postgres://host:5432/postgres'))
+        self.c._in_flight.deadline = 0
+        self.assertIsNotNone(self.c.add_task('after_promote', 1, 'postgres://host:5432/postgres'))
 
         # If there is no transaction in progress and cached pg_dist_node matching desired state task should not be added
         self.c._schedule_load_pg_dist_node = False

tests/test_config.py

@@ -70,7 +70,8 @@ class TestConfig(unittest.TestCase):
             'PATRONI_REPLICATION_USERNAME': 'replicator',
             'PATRONI_REPLICATION_PASSWORD': 'rep-pass',
             'PATRONI_admin_PASSWORD': 'admin',
-            'PATRONI_admin_OPTIONS': 'createrole,createdb'
+            'PATRONI_admin_OPTIONS': 'createrole,createdb',
+            'PATRONI_POSTGRESQL_BIN_POSTGRES': 'sergtsop'
         })
         config = Config('postgres0.yml')
         with patch.object(Config, '_load_config_file', Mock(return_value={'restapi': {}})):

tests/test_patroni.py

@@ -67,6 +67,7 @@ class TestPatroni(unittest.TestCase):
     @patch.object(etcd.Client, 'read', etcd_read)
     @patch.object(Thread, 'start', Mock())
     @patch.object(AbstractEtcdClientWithFailover, '_get_machines_list', Mock(return_value=['http://remotehost:2379']))
+    @patch.object(Postgresql, '_get_gucs', Mock(return_value={'foo': True, 'bar': True}))
     def setUp(self):
         self._handlers = logging.getLogger().handlers[:]
         RestApiServer._BaseServer__is_shut_down = Mock()
@@ -90,6 +91,7 @@ class TestPatroni(unittest.TestCase):
     @patch.object(etcd.Client, 'delete', Mock())
     @patch.object(AbstractEtcdClientWithFailover, '_get_machines_list', Mock(return_value=['http://remotehost:2379']))
     @patch.object(Thread, 'join', Mock())
+    @patch.object(Postgresql, '_get_gucs', Mock(return_value={'foo': True, 'bar': True}))
     def test_patroni_patroni_main(self):
         with patch('subprocess.call', Mock(return_value=1)):
             with patch.object(Patroni, 'run', Mock(side_effect=SleepException)):

tests/test_postgresql.py

@@ -10,6 +10,7 @@ from mock import Mock, MagicMock, PropertyMock, patch, mock_open
 import patroni.psycopg as psycopg
 
 from patroni.async_executor import CriticalTask
+from patroni.collections import CaseInsensitiveSet
 from patroni.config import GlobalConfig
 from patroni.dcs import RemoteMember
 from patroni.exceptions import PostgresConnectionException, PatroniException
@@ -17,10 +18,14 @@ from patroni.postgresql import Postgresql, STATE_REJECT, STATE_NO_RESPONSE
 from patroni.postgresql.bootstrap import Bootstrap
 from patroni.postgresql.callback_executor import CallbackAction
 from patroni.postgresql.postmaster import PostmasterProcess
+from patroni.postgresql.validator import (ValidatorFactoryNoType, ValidatorFactoryInvalidType,
+                                          ValidatorFactoryInvalidSpec, ValidatorFactory, InvalidGucValidatorsFile,
+                                          _get_postgres_guc_validators, _read_postgres_gucs_validators_file,
+                                          _load_postgres_gucs_validators, Bool, Integer, Real, Enum, EnumBool, String)
 from patroni.utils import RetryFailedError
 from threading import Thread, current_thread
 
-from . import BaseTestPostgresql, MockCursor, MockPostmaster, psycopg_connect
+from . import BaseTestPostgresql, MockCursor, MockPostmaster, psycopg_connect, mock_available_gucs
 
 
 mtime_ret = {}
@@ -91,6 +96,7 @@ Data page checksum version:           0
 
 @patch('subprocess.call', Mock(return_value=0))
 @patch('patroni.psycopg.connect', psycopg_connect)
+@patch.object(Postgresql, 'available_gucs', mock_available_gucs)
 class TestPostgresql(BaseTestPostgresql):
 
     @patch('subprocess.call', Mock(return_value=0))
@@ -98,6 +104,7 @@ class TestPostgresql(BaseTestPostgresql):
     @patch('patroni.postgresql.CallbackExecutor', Mock())
     @patch.object(Postgresql, 'get_major_version', Mock(return_value=140000))
     @patch.object(Postgresql, 'is_running', Mock(return_value=True))
+    @patch.object(Postgresql, 'available_gucs', mock_available_gucs)
     def setUp(self):
         super(TestPostgresql, self).setUp()
         self.p.config.write_postgresql_conf()
@@ -739,3 +746,212 @@ class TestPostgresql(BaseTestPostgresql):
     @patch.object(Postgresql, '_cluster_info_state_get', Mock(return_value=True))
     def test_handle_parameter_change(self):
         self.p.handle_parameter_change()
+
+    def test_validator_factory(self):
+        # validator with no type
+        validator = {
+            'version_from': 90300,
+            'version_till': None,
+        }
+        with self.assertRaises(ValidatorFactoryNoType) as e:
+            ValidatorFactory(validator)
+        self.assertEqual(str(e.exception), 'Validator contains no type.')
+
+        # validator with invalid type
+        validator = {
+            'type': 'Random',
+            'version_from': 90300,
+            'version_till': None,
+        }
+        with self.assertRaises(ValidatorFactoryInvalidType) as e:
+            ValidatorFactory(validator)
+        self.assertEqual(str(e.exception), f'Unexpected validator type: `{validator["type"]}`.')
+
+        # validator with missing attributes
+        validator = {
+            'type': 'Integer',
+            'version_from': 90300,
+            'min_val': 0,
+        }
+        with self.assertRaises(ValidatorFactoryInvalidSpec) as e:
+            ValidatorFactory(validator)
+        type_ = validator.pop('type')
+        self.assertRegex(
+            str(e.exception),
+            rf"Failed to parse `{type_}` validator \(`{validator}`\): `(Number\.)?__init__\(\) missing 1 "
+            "required keyword-only argument: 'max_val'`."
+        )
+
+        # valid validators
+        # Bool
+        validator = {
+            'type': 'Bool',
+            'version_from': 90300,
+            'version_till': None,
+        }
+        ret = ValidatorFactory(validator)
+        self.assertIsInstance(ret, Bool)
+        self.assertEqual(
+            ret.__dict__,
+            Bool(version_from=validator['version_from'], version_till=validator['version_till']).__dict__,
+        )
+
+        # Integer
+        validator = {
+            'type': 'Integer',
+            'version_from': 90300,
+            'version_till': None,
+            'min_val': 1,
+            'max_val': 100,
+            'unit': None,
+        }
+        ret = ValidatorFactory(validator)
+        self.assertIsInstance(ret, Integer)
+        self.assertEqual(
+            ret.__dict__,
+            Integer(version_from=validator['version_from'], version_till=validator['version_till'],
+                    min_val=validator['min_val'], max_val=validator['max_val'], unit=validator['unit']).__dict__,
+        )
+
+        # Real
+        validator = {
+            'type': 'Real',
+            'version_from': 90300,
+            'version_till': None,
+            'min_val': 1.0,
+            'max_val': 100.0,
+            'unit': None,
+        }
+        ret = ValidatorFactory(validator)
+        self.assertIsInstance(ret, Real)
+        self.assertEqual(
+            ret.__dict__,
+            Real(version_from=validator['version_from'], version_till=validator['version_till'],
+                 min_val=validator['min_val'], max_val=validator['max_val'], unit=validator['unit']).__dict__,
+        )
+
+        # Enum
+        validator = {
+            'type': 'Enum',
+            'version_from': 90300,
+            'version_till': None,
+            'possible_values': ('abc', 'def'),
+        }
+        ret = ValidatorFactory(validator)
+        self.assertIsInstance(ret, Enum)
+        self.assertEqual(
+            ret.__dict__,
+            Enum(version_from=validator['version_from'], version_till=validator['version_till'],
+                 possible_values=validator['possible_values']).__dict__,
+        )
+
+        # EnumBool
+        validator = {
+            'type': 'EnumBool',
+            'version_from': 90300,
+            'version_till': None,
+            'possible_values': ('abc', 'def'),
+        }
+        ret = ValidatorFactory(validator)
+        self.assertIsInstance(ret, EnumBool)
+        self.assertEqual(
+            ret.__dict__,
+            EnumBool(version_from=validator['version_from'], version_till=validator['version_till'],
+                     possible_values=validator['possible_values']).__dict__,
+        )
+
+        # String
+        validator = {
+            'type': 'String',
+            'version_from': 90300,
+            'version_till': None,
+        }
+        ret = ValidatorFactory(validator)
+        self.assertIsInstance(ret, String)
+        self.assertEqual(
+            ret.__dict__,
+            String(version_from=validator['version_from'], version_till=validator['version_till']).__dict__,
+        )
+
+    def test__get_postgres_guc_validators(self):
+        # normal run
+        parameter = 'my_parameter'
+
+        config = {
+            parameter: [{
+                'type': 'Bool',
+                'version_from': 90300,
+                'version_till': 90500,
+            }, {
+                'type': 'EnumBool',
+                'version_from': 90500,
+                'version_till': 90600,
+                'possible_values': [
+                    'always',
+                ],
+            }]
+        }
+        ret = _get_postgres_guc_validators(config, parameter)
+        self.assertIsInstance(ret, tuple)
+        self.assertEqual(len(ret), 2)
+        self.assertIsInstance(ret[0], Bool)
+        self.assertIsInstance(ret[1], EnumBool)
+
+        # log exceptions
+        del config[parameter][0]['type']
+
+        with patch('patroni.postgresql.validator.logger.warning') as mock_logger:
+            ret = _get_postgres_guc_validators(config, parameter)
+            self.assertIsInstance(ret, tuple)
+            self.assertEqual(len(ret), 1)
+            self.assertIsInstance(ret[0], EnumBool)
+
+            mock_logger.assert_called_once()
+            mock_call = mock_logger.call_args[0]
+            self.assertEqual(mock_call[0], 'Faced an issue while parsing a validator for parameter `%s`: `%r`')
+            self.assertEqual(mock_call[1], parameter)
+            self.assertIsInstance(mock_call[2], ValidatorFactoryNoType)
+
+    def test__read_postgres_gucs_validators_file(self):
+        # raise exception
+        with self.assertRaises(InvalidGucValidatorsFile) as exc:
+            _read_postgres_gucs_validators_file('random_file.yaml')
+        self.assertEqual(
+            str(exc.exception),
+            "Unexpected issue while reading parameters file `random_file.yaml`: `[Errno 2] No such file or directory: "
+            "'random_file.yaml'`."
+        )
+
+    def test__load_postgres_gucs_validators(self):
+        # log messages
+        with patch('os.walk', Mock(return_value=iter([('.', [], ['file.txt', 'random.yaml'])]))), \
+             patch('patroni.postgresql.validator.logger.info') as mock_info, \
+             patch('patroni.postgresql.validator.logger.warning') as mock_warning:
+            _load_postgres_gucs_validators()
+            mock_info.assert_called_once_with('Ignored a non-YAML file found under `available_parameters` directory: '
+                                              '`%s`.', os.path.join('.', 'file.txt'))
+            mock_warning.assert_called_once()
+            self.assertIn(
+                "Unexpected issue while reading parameters file `{0}`: `[Errno 2] No such file or "
+                "directory:".format(os.path.join('.', 'random.yaml')),
+                mock_warning.call_args[0][0]
+            )
+
+
+@patch('subprocess.call', Mock(return_value=0))
+@patch('patroni.psycopg.connect', psycopg_connect)
+class TestPostgresql2(BaseTestPostgresql):
+
+    @patch('subprocess.call', Mock(return_value=0))
+    @patch('os.rename', Mock())
+    @patch('patroni.postgresql.CallbackExecutor', Mock())
+    @patch.object(Postgresql, 'get_major_version', Mock(return_value=140000))
+    @patch.object(Postgresql, 'is_running', Mock(return_value=True))
+    def setUp(self):
+        super(TestPostgresql2, self).setUp()
+
+    @patch('subprocess.check_output', Mock(return_value='\n'.join(mock_available_gucs.return_value).encode('utf-8')))
+    def test_available_gucs(self):
+        gucs = self.p.available_gucs
+        self.assertIsInstance(gucs, CaseInsensitiveSet)
+        self.assertEqual(gucs, mock_available_gucs.return_value)

tests/test_sync.py

@@ -7,11 +7,12 @@ from patroni.config import GlobalConfig
 from patroni.dcs import Cluster, SyncState
 from patroni.postgresql import Postgresql
 
-from . import BaseTestPostgresql, psycopg_connect
+from . import BaseTestPostgresql, psycopg_connect, mock_available_gucs
 
 
 @patch('subprocess.call', Mock(return_value=0))
 @patch('patroni.psycopg.connect', psycopg_connect)
+@patch.object(Postgresql, 'available_gucs', mock_available_gucs)
 class TestSync(BaseTestPostgresql):
 
     @patch('subprocess.call', Mock(return_value=0))
@@ -19,6 +20,7 @@ class TestSync(BaseTestPostgresql):
     @patch('patroni.postgresql.CallbackExecutor', Mock())
     @patch.object(Postgresql, 'get_major_version', Mock(return_value=140000))
     @patch.object(Postgresql, 'is_running', Mock(return_value=True))
+    @patch.object(Postgresql, 'available_gucs', mock_available_gucs)
     def setUp(self):
         super(TestSync, self).setUp()
         self.p.config.write_postgresql_conf()