Release v2.1.5 (#2462)

* bump version
* update release notes
* run some behave tests on v15
* automate release process by building/pushing packages on tag creation and release publication

.github/workflows/install_deps.py

@@ -46,10 +46,14 @@ def install_packages(what):
     packages['exhibitor'] = packages['zookeeper']
     packages = packages.get(what, [])
     ver = versions.get(what)
+    subprocess.call(['sudo', 'apt-get', 'update', '-y'])
+    subprocess.call(['sudo', 'apt-get', 'install', '-y', 'wget', 'ca-certificates', 'gnupg', 'expect-dev'])
+    subprocess.call(['sudo', 'sh', '-c', "wget -qO - https://www.postgresql.org/media/keys/ACCC4CF8.asc"
+                     " | gpg --dearmor > /etc/apt/trusted.gpg.d/apt.postgresql.org.gpg"])
     subprocess.call(['sudo', 'sed', '-i', 's/pgdg main.*$/pgdg main {0}/'.format(ver),
                      '/etc/apt/sources.list.d/pgdg.list'])
     subprocess.call(['sudo', 'apt-get', 'update', '-y'])
-    return subprocess.call(['sudo', 'apt-get', 'install', '-y', 'postgresql-' + ver, 'expect-dev', 'wget'] + packages)
+    return subprocess.call(['sudo', 'apt-get', 'install', '-y', 'postgresql-' + ver] + packages)
 
 
 def get_file(url, name):

.github/workflows/mapping.py

@@ -1 +1 @@
-versions = {'etcd': '9.6', 'etcd3': '14', 'consul': '13', 'exhibitor': '12', 'raft': '11', 'kubernetes': '14'}
+versions = {'etcd': '9.6', 'etcd3': '14', 'consul': '13', 'exhibitor': '12', 'raft': '11', 'kubernetes': '15'}

.github/workflows/release.yaml

@@ -0,0 +1,41 @@
+name: Publish Patroni distributions to PyPI and TestPyPI
+on:
+  push:
+    tags:
+      - 'v[0-9]+.[0-9]+.[0-9]+'
+  release:
+    types:
+      - published
+jobs:
+  build-n-publish:
+    name: Build and publish Patroni distributions to PyPI and TestPyPI
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@master
+
+    - name: Set up Python 3.9
+      uses: actions/setup-python@v4
+      with:
+        python-version: 3.9
+
+    - name: Install dependencies
+      run: python .github/workflows/install_deps.py
+
+    - name: Run tests and flake8
+      run: python .github/workflows/run_tests.py
+
+    - name: Build a binary wheel and a source tarball
+      run: python setup.py sdist bdist_wheel
+
+    - name: Publish distribution to Test PyPI
+      if: github.event_name == 'push'
+      uses: pypa/[email protected]
+      with:
+        password: ${{ secrets.TEST_PYPI_API_TOKEN }}
+        repository_url: https://test.pypi.org/legacy/
+
+    - name: Publish distribution to PyPI
+      if: github.event_name == 'release'
+      uses: pypa/[email protected]
+      with:
+        password: ${{ secrets.PYPI_API_TOKEN }}

Dockerfile

@@ -1,6 +1,6 @@
 ## This Dockerfile is meant to aid in the building and debugging patroni whilst developing on your local machine
 ## It has all the necessary components to play/debug with a single node appliance, running etcd
-ARG PG_MAJOR=14
+ARG PG_MAJOR=15
 ARG COMPRESS=false
 ARG PGHOME=/home/postgres
 ARG PGDATA=$PGHOME/data
@@ -117,7 +117,7 @@ RUN if [ "$COMPRESS" = "true" ]; then \
 FROM scratch
 COPY --from=builder / /
 
-LABEL maintainer="Alexander Kukushkin <alexander.kukushkin@zalando.de>"
+LABEL maintainer="Alexander Kukushkin <akukushkin@microsoft.com>"
 
 ARG PG_MAJOR
 ARG COMPRESS
@@ -155,4 +155,4 @@ RUN sed -i 's/env python/&3/' /patroni*.py \
 
 USER postgres
 
-ENTRYPOINT ["/bin/sh", "/entrypoint.sh"]
+ENTRYPOINT ["/bin/sh", "/entrypoint.sh"]

README.rst

@@ -12,7 +12,7 @@ Patroni is a template for you to create your own customized, high-availability s
 
 We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely.
 
-Currently supported PostgreSQL versions: 9.3 to 14.
+Currently supported PostgreSQL versions: 9.3 to 15.
 
 **Note to Kubernetes users**: Patroni can run natively on top of Kubernetes. Take a look at the `Kubernetes <https://github.com/zalando/patroni/blob/master/docs/kubernetes.rst>`__ chapter of the Patroni documentation.
 

docs/index.rst

@@ -10,7 +10,7 @@ Patroni is a template for you to create your own customized, high-availability s
 
 We call Patroni a "template" because it is far from being a one-size-fits-all or plug-and-play replication system. It will have its own caveats. Use wisely. There are many ways to run high availability with PostgreSQL; for a list, see the `PostgreSQL Documentation <https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling>`__.
 
-Currently supported PostgreSQL versions: 9.3 to 14.
+Currently supported PostgreSQL versions: 9.3 to 15.
 
 **Note to Kubernetes users**: Patroni can run natively on top of Kubernetes. Take a look at the :ref:`Kubernetes <kubernetes>` chapter of the Patroni documentation.
 

docs/releases.rst

@@ -3,6 +3,104 @@
 Release notes
 =============
 
+Version 2.1.5
+-------------
+
+This version enhances compatibility with PostgreSQL 15 and declares Etcd v3 support as production ready. The Patroni on Raft remains in Beta.
+
+**New features**
+
+- Improve ``patroni --validate-config`` (Denis Laxalde)
+
+  Exit with code 1 if config is invalid and print errors to stderr.
+
+- Don't drop replication slots in pause (Alexander Kukushkin)
+
+  Patroni is automatically creating/removing physical replication slots when members are joining/leaving the cluster. In pause slots will no longer be removed.
+
+- Support the ``HEAD`` request method for monitoring endpoints (Robert Cutajar)
+
+  If used instead of ``GET`` Patroni will return only the HTTP Status Code.
+
+- Support behave tests on Windows (Alexander)
+
+  Emulate graceful Patroni shutdown (``SIGTERM``) on Windows by introduce the new REST API endpoint ``POST /sigterm``.
+
+- Introduce ``postgresql.proxy_address`` (Alexander)
+
+  It will be written to the member key in DCS as the ``proxy_url`` and could be used/useful for service discovery.
+
+
+**Stability improvements**
+
+- Call ``pg_replication_slot_advance()`` from a thread (Alexander)
+
+  On busy clusters with many logical replication slots the ``pg_replication_slot_advance()`` call was affecting the main HA loop and could result in the member key expiration.
+
+- Archive possibly missing WALs before calling ``pg_rewind`` on the old primary (Polina Bungina)
+
+  If the primary crashed and was down during considerable time, some WAL files could be missing from archive and from the new primary. There is a chance that ``pg_rewind`` could remove these WAL files from the old primary making it impossible to start it as a standby. By archiving ``ready`` WAL files we not only mitigate this problem but in general improving continues archiving experience.
+
+- Ignore ``403`` errors when trying to create Kubernetes Service (Nick Hudson, Polina)
+
+  Patroni was spamming logs by unsuccessful attempts to create the service, which in fact could already exist.
+
+- Improve liveness probe (Alexander)
+
+  The liveness problem will start failing if the heartbeat loop is running longer than `ttl` on the primary or `2*ttl` on the replica. That will allow us to use it as an alternative for :ref:`watchdog <watchdog>` on Kubernetes.
+
+- Make sure only sync node tries to grab the lock when switchover (Alexander, Polina)
+
+  Previously there was a slim chance that up-to-date async member could become the leader if the manual switchover was performed without specifying the target.
+
+- Avoid cloning while bootstrap is running (Ants Aasma)
+
+  Do not allow a create replica method that does not require a leader to be triggered while the cluster bootstrap is running.
+
+- Compatibility with kazoo-2.9.0 (Alexander)
+
+  Depending on python version the ``SequentialThreadingHandler.select()`` method may raise ``TypeError`` and ``IOError`` exceptions if ``select()`` is called on the closed socket.
+
+- Explicitly shut down SSL connection before socket shutdown (Alexander)
+
+  Not doing it resulted in ``unexpected eof while reading`` errors with OpenSSL 3.0.
+
+- Compatibility with `prettytable>=2.2.0` (Alexander)
+
+  Due to the internal API changes the cluster name header was shown on the incorrect line.
+
+
+**Bugfixes**
+
+- Handle expired token for Etcd lease_grant (monsterxx03)
+
+  In case of error get the new token and retry request.
+
+- Fix bug in the ``GET /read-only-sync`` endpoint (Alexander)
+
+  It was introduced in previous release and effectively never worked.
+
+- Handle the case when data dir storage disappeared (Alexander)
+
+  Patroni is periodically checking that the PGDATA is there and not empty, but in case of issues with storage the ``os.listdir()`` is raising the ``OSError`` exception, breaking the heart-beat loop.
+
+- Apply ``master_stop_timeout`` when waiting for user backends to close (Alexander)
+
+  Something that looks like user backend could be in fact a background worker (e.g., Citus Maintenance Daemon) that is failing to stop.
+
+- Accept ``*:<port>`` for ``postgresql.listen`` (Denis)
+
+  The ``patroni --validate-config`` was complaining about it being invalid.
+
+- Timeouts fixes in Raft (Alexander)
+
+  When Patroni or patronictl are starting they try to get Raft cluster topology from known members. These calls were made without proper timeouts.
+
+- Forcefully update consul service if token was changed (John A. Lotoski)
+
+  Not doing so results in errors "rpc error making call: rpc error making call: ACL not found".
+
+
 Version 2.1.4
 -------------
 

kubernetes/Dockerfile

@@ -1,5 +1,5 @@
-FROM postgres:11
-MAINTAINER Alexander Kukushkin <alexander.kukushkin@zalando.de>
+FROM postgres:15
+LABEL maintainer="Alexander Kukushkin <akukushkin@microsoft.com>"
 
 RUN export DEBIAN_FRONTEND=noninteractive \
     && echo 'APT::Install-Recommends "0";\nAPT::Install-Suggests "0";' > /etc/apt/apt.conf.d/01norecommend \

patroni/version.py

@@ -1 +1 @@
-__version__ = '2.1.4'
+__version__ = '2.1.5'

release.sh

@@ -4,6 +4,8 @@
 # 1. Open a PR that updates release notes and Patroni version
 # 2. Merge it
 # 3. Run release.sh
+# 4. After the new tag is pushed, the .github/workflows/release.yaml will run tests and upload the new package to test.pypi.org
+# 5. Once the release is created, the .github/workflows/release.yaml will run tests and upload the new package to pypi.org
 
 ## Bail out on any non-zero exitcode from the called processes
 set -xe
@@ -22,7 +24,5 @@ python setup.py clean
 python setup.py test
 python setup.py flake8
 
-python setup.py sdist bdist_wheel upload
-
 git tag "v$version"
 git push --tags