Commit Graph
2065 Commits
Author SHA1 Message Date
Alexander KukushkinandGitHub ed47224540 Improve behaviour of the insecure option (#2476)
It didn't worked correctly when client certificates are used for REST API requests.
2022-12-06 17:24:57 +01:00
Alexander KukushkinandGitHub c7a925a238 Switch from localkube to kind and/or k3d (#2465)
The only advantage of localkube was being a low weight. Anything else started creating only problems:
1. It is not properly maintained for many years.
2. It effectively worked only on Linux, but stopped on modern version due to changes in iptables.

Instead, we will use widely adoped tools like kind or k3s. The "kind-kind" is the default K8s context (see ~/.kube/config), but it could be overriden using `PATRONI_KUBERNETES_CONTEXT` environment variable. When executed from GH actions the context is set to k3d-k3s-default, because K3s is much faster to start.
2022-12-06 13:15:56 +01:00
Alexander KukushkinandGitHub 26244634ce Fix annoying exceptions on ssl socket shutdown (#2468)
The HAProxy is closing connections as soon as it got the HTTP Status code leaving no time for Patroni to properly shutdown SSL connection.

Close https://github.com/zalando/patroni/issues/2466
2022-12-06 11:57:12 +01:00
Alexander KukushkinandGitHub b47c50a788 Stick to the ubuntu-20.04 (#2472)
the ubuntu-latest is switching to 22.04 and discontinued support of python 2.7 and 3.6
2022-12-05 12:00:31 +01:00
Denis LaxaldeandGitHub 2bf7872d64 Declare proxy_address as optional (#2464)
It's effectively non-required when used in patroni/postgresql/config.py. This breaks most configuration files in the wild, starting from the examples:

    $ patroni --validate-config postgres0.yml
    restapi.connect_address 127.0.0.1:8008 didn't pass validation: 'must not contain "127.0.0.1", "0.0.0.0", "*", "::1", "localhost"'
    etcd.host 127.0.0.1:2379 didn't pass validation: '127.0.0.1:2379 is not reachable'
    postgresql.connect_address 127.0.0.1:5432 didn't pass validation: 'must not contain "127.0.0.1", "0.0.0.0", "*", "::1", "localhost"'
    postgresql.proxy_address  is not defined.
2022-11-28 14:02:20 +01:00
Alexander KukushkinandGitHub 412d508023 Bugfix: raise ConfigParseError (#2463)
instead of returning error string
v2.1.5
2022-11-28 11:44:06 +01:00
Alexander KukushkinandGitHub 53f89faaab Release v2.1.5 (#2462)
* bump version
* update release notes
* run some behave tests on v15
* automate release process by building/pushing packages on tag creation and release publication
2022-11-28 10:45:04 +01:00
John A. LotoskiandGitHub 2ed1793bbd fix: update service on consul token rotation (#2450)
Close #2449
2022-11-18 08:10:32 +01:00
Alexander KukushkinandGitHub 1b6e23ab6a Add Polina to maintainers (#2451)
and remove some old names
2022-11-10 10:22:42 +01:00
ef2922fe37 Bump actions, install wheel (#2446)
* Bump actions/checkout and actions/setup-python versions
* Install wheel to silence some warnings

Co-authored-by: Alexander Kukushkin <[email protected]>
2022-11-01 13:55:15 +01:00
Alexander KukushkinandGitHub bda2bedf48 Make sure self.__retry_timeout is set (#2440)
The default value was None, which is the same as no timeout.
2022-10-25 14:14:16 +02:00
Alexander KukushkinandGitHub 5a21ffa3e4 Fix a little bug in check_logical_slots_readiness() (#2439)
If there is no `catalog_xmin` on physical slot on the primary the only further check that makes sense is whether the `hot_standby_feedback` is enabled.

Close https://github.com/zalando/patroni/issues/2438
2022-10-24 15:24:36 +02:00
Alexander KukushkinandGitHub 4ecaf445fa Introduce configurable timeout in TcpUtility (#2435)
the value is calculated based on the number of nodes and retry_timeout.

Close https://github.com/zalando/patroni/issues/2431
2022-10-24 10:30:28 +02:00
Alexander KukushkinandGitHub a293b77d25 Compatibility with prettytable 2.2.0+ (#2436)
`patronictl list` and `patronictl topology` didn't worked correctly starting from prettytable 2.2.0 (https://github.com/jazzband/prettytable/pull/104).
Example of wrong output:
```
+----------+-----------+---------+---------+----+-----------+
| Member   | Host      | Role    | State   | TL | Lag in MB |
+ Cluster: demo-cluster-1 (7155048679605235377) +-----------+
| node-2   | 10.2.0.21 | Leader  | running |  1 |           |
| + node-1 | 10.2.0.6  | Replica | running |  1 |         0 |
| + node-3 | 10.2.0.25 | Replica | running |  1 |         0 |
+----------+-----------+---------+---------+----+-----------+
```
2022-10-24 10:23:21 +02:00
Alexander KukushkinandGitHub 8f8e9c9b81 Inptroduce postgresql.proxy_address (#2437)
It will be written to member key in DCS as the `proxy_url` and could be used/useful for service discovery.
2022-10-24 10:23:06 +02:00
Alexander KukushkinandGitHub 580530b30f Behave tests on Windows (#2432)
Windows doesn't support `SIGTERM`, but our behave tests in majority of cases relying on Patroni graceful shutdown.
In order to emulate the behaviour we introduced the new REST API endpoint `POST /sigterm`. The endpoint works only on Windows and when `BEHAVE_DEBUG` environment variable is set.
Besides that some minor adjustments in behave tests were done. Mainly related to backslash-slash handling.

In addition to that improve test coverage on Windows by properly mocking access to filesystem and avoiding calling
 `subprocess.call()`. Specifically, symlink creation on Windows requires Admin privileges and there is no `true.exe`.
2022-10-21 12:24:24 +02:00
Alexander KukushkinandGitHub f4ae55b92a Remove 'enable_group_by_reordering' from GUC validator (#2426)
The feature was recently reverted.
2022-10-13 10:53:29 +02:00
Alexander KukushkinandGitHub 816b66311b A small fix in unit tests (#2427)
Not all external resources were properly mocked
2022-10-13 10:53:13 +02:00
Alexander KukushkinandGitHub 8a227aa743 Explicitly shut down SSL connection before socket (#2425)
This is handled by calling the unwrap() method on SSLSocket.
In addition to that simplify code that handles deferred handshakes.

Close https://github.com/zalando/patroni/issues/2424
2022-10-13 10:34:43 +02:00
Alexander KukushkinandGitHub 531063f676 Compatibility with kazoo-2.9.0 (#2428)
Now the select() method may raise `TypeError` and `IOError` exceptions if the socket is closed.
2022-10-13 09:18:06 +02:00
Ants AasmaandGitHub db9b5962ec Avoid cloning while bootstrap is running (#2419)
If cluster has a create replica method that does not require a leader it can get triggered while bootstrap is running. If that method comes up with an accessible cluster faster than the bootstrap completes it will get promoted as the leader, only to lose leader lock when bootstrap completes. To fix this we only consider leaderless create replica methods if sysid is non-empty, i.e. there is no bootstrap running.
2022-09-29 13:39:51 +02:00
Polina BunginaandGitHub 3dcdb16d2a Fix exception handling in create_config_service (#2423) 2022-09-29 09:37:01 +02:00
Jim Chanco JrandGitHub 84dc72b031 docs: Change term "Master" to "primary" or "leader" (#2417) 2022-09-29 08:48:49 +02:00
Alexander KukushkinandGitHub 7102346f87 Update release.sh (#2421)
in order to reflect the current release process
2022-09-27 09:43:38 +02:00
Alexander KukushkinandGitHub 6d8d1a2556 Make sure only sync node tries to grab the lock when switchover (#2406)
- We use `failover.leader` check as an indicator of switchover. The check on `failover.candidate` was unnecessary and incorrect.
- Add more unit-tests for switchover in sync mode

Close https://github.com/zalando/patroni/issues/2405
Co-authored-by: Polina Bungina <[email protected]>
2022-09-19 11:22:35 +02:00
Alexander KukushkinandGitHub 88db6018ac Improve liveness probe (#2395)
it will start failing if the heartbeat loop isn't running longer than `ttl` on the primary or `2*ttl` on the replica.

Close https://github.com/zalando/patroni/issues/2388
2022-09-01 11:34:42 +02:00
Denis LaxaldeandGitHub cea1fa869b Accept '*:<port>' for postgresql.listen (#2398)
We catch this special value when validating configuration and check that
it's alone in the hosts list.

Fixes #2397.
2022-08-26 07:49:27 +02:00
Alexander KukushkinandGitHub 4a854a71c0 Call pg_replication_slot_advance() from a thread (#2391)
On busy clusters with many logical replication slots the pg_replication_slot_advance () call affects the main HA loop and could result in the member key expiration.
The only way to solve it is a dedicated thread response for moving slots forward.

The thread is started only when there are logical slots to be advanced.

Will help to solve #2388
Close #2239
2022-08-24 13:43:09 +02:00
Nick HudsonandGitHub a2ef950e08 Ignore 403s when trying to create Kubernetes Service (#2390)
Close #1132
2022-08-24 13:22:53 +02:00
Robert CutajarandGitHub f92d975e7b #2021 add HEAD support - minimal (#2360) 2022-08-19 13:27:08 +02:00
Polina BunginaandGitHub 2ee09d0a66 Check if .ready file exists in _archive_ready_wals (#2387) 2022-08-18 09:07:21 +02:00
ae0ede6944 Archive possibly missing WALs before rewind (#2384)
There is currently a risk to lose some WAL segments entirely in case
archive_mode was set to 'on' before a promotion and there are some WALs
with .ready files on the former leader we are trying to rewind. It happens
because of the pg_rewind's modus operandi: it simply syncs the content of
pg_wal directory of the old leader with the new leader's one. Including
deletion of all WALs that are not present on the current leader, regardless
their archive status on the former one. Thus, in case the new leader has
already recycled such files, we just remove them entirely.

In case archive_mode was set to 'always' and the .ready WALs are acrually
present in archive, it is for end user who writes the archive_command to
avoid overwritting and to properly test it.

Co-authored-by: Alexander Kukushkin <[email protected]>
2022-08-17 09:54:30 +02:00
Alexander KukushkinandGitHub b6f057850a Apply timeout when waiting for user backends to close (#2382)
Close #2365
2022-08-17 09:23:07 +02:00
Alexander KukushkinandGitHub a0b32379e5 Handle the case when data dir storage disappeared (#2381)
The `os.listdir()` is raising the OSError exception, breaking the heart-beat loop.

Close https://github.com/zalando/patroni/issues/2380
2022-08-15 15:11:59 +02:00
Alexander KukushkinandGitHub 2d08e88c3e Don't drop replication slots in pause (#2383)
If replication slots are enabled Patroni automatically creates them for any cluster member that is supposed to stream from a given node and for any permanent slot defined in the global configuration. If the member disappears from the DCS Patroni automatically removes the replication slot for it. The same behavior was in the maintenance mode (pause).

This commit disables removal of any replication slots that don't match Patroni's expectations in pause.

Close https://github.com/zalando/patroni/issues/2314
2022-08-15 15:11:27 +02:00
Polina BunginaandGitHub ea2b7d2368 Disable the option to open an empty issue (#2377)
Make @CyberDem0n happier
2022-08-04 17:14:12 +02:00
Michael BanckandGitHub f65efecac9 Clarify standby cluster documentation. (#2369)
This adds a paragraph to the Standby Cluster section clarifying that the standby cluster is independent of the primary cluster and not visible from the primary cluster's Patroni interface.

Close #2090
2022-08-02 10:14:37 +02:00
Nikolay SamokhvalovandGitHub a8b73ef021 systemd service options: restart patroni service if it crashed (#2372)
It makes little sense to have ` for the Patroni service – if it goes down and we don't notice it (I doubt that patroni service uptime is well monitored in most cases), then we lose autofailover for Postgres, implying bigger risks of downtime.

`Restart=on-failure` makes more sense in this case.
2022-08-01 14:55:45 +02:00
Alexander KukushkinandGitHub d8d634125c Compatibility with the latest flake8 (#2373)
Require flake8 at least 3.0.0 and just call main()
2022-08-01 12:25:04 +02:00
Alexander KukushkinandGitHub ead798d9ac Speed up behave tests by always using loop_wait=2 (#2361)
run time is reduced from ~5m30s to ~5m
2022-07-18 15:23:55 +02:00
Alexander KukushkinandGitHub cd5d20fa53 Fix bug with GET /read-only-sync endpoint (#2350)
effectively it never worked
2022-07-14 08:07:45 +02:00
Alexander KukushkinandGitHub 4c5cce5efd Automatically skip some behave tests on legacy Postgres (#2358)
previously behave had to be started with `--tags=-skip` argument.
2022-07-13 12:13:36 +02:00
Alexander KukushkinandGitHub 5b1fd23776 Always return checkpoint location as integer (#2349)
before it was also returning a str in some cases
2022-06-30 10:52:28 +02:00
Denis LaxaldeandGitHub 741243695a Improvements to 'patroni --validate-config' (#2344)
* Let `patroni --validate-config` exit 1 when config is invalid
* Print configuration errors to stderr

Close #2345
2022-06-30 10:51:47 +02:00
Victor SudakovandGitHub 8d7828b079 Location of postgresql.conf on the remote master. (#2343)
Close #2337
2022-06-30 10:50:38 +02:00
sahapasciandGitHub 8d773be533 Update SETTINGS.rst (#2339)
consul.service_check_interval defaults to 5 seconds
2022-06-30 10:49:56 +02:00
Lev KozlovandGitHub b8a6387236 Bump Postgres version in Dockerfile to 14 (#2333) 2022-06-13 15:26:01 +02:00
monsterxx03andGitHub c7ee5f008d Handle expired token for etcd lease_grant (#2331) (#2332)
Close #2331
2022-06-13 14:58:11 +02:00
Michael BanckandGitHub a77fbb1912 Fix markup - the -status is part of the command (#2323) 2022-06-13 14:57:28 +02:00
Alexander KukushkinandGitHub fb06af9adb Release 2.1.4 (#2322)
- bump version
- update release notes
- implement missing unit-tests
v2.1.4
2022-06-01 16:00:56 +02:00