- fix --validate-config not to error out if bin_dir is an empty string in the yaml config
- mention bin_dir optionality in the docs
- validate bin_dir even if it is not in the yaml config (add optional
default value for Optional config params in validator)
- make rewind user optional
Make it return the new `SyncState` object in order to avoid reading the new cluster state in the Ha.process_sync_replication().
Now it is a small optimization, but it will become very handy in the quorum commit feature.
- make parse_sync_standby_names() return NamedTuple instead of dict
- fix little issue in Postgresql.reset_cluster_info_state(), it should check global_config independently from cluster and cluster.config
- added pyrightconfig.json with typeCheckingMode=strict
- added type hints to all files except api.py
- added type stubs for dns, etcd, consul, kazoo, pysyncobj and other modules
- added type stubs for psycopg2 and urllib3 with some little fixes
- fixes most of the issues reported by pyright
- remaining issues will be addressed later, along with enabling CI linting task
* Further nested lists rendering fixes
* Remove a couple of sphinx warnings
* Fix bootstrap.users.password description
* Boto->boto3 in README's
* Split configuration docs and move some lines across files
* Fix a typo
The two cases we have in mind are:
* In spite of following all best practices client-side, logical replication connections can sometimes hang the Postgres shutdown sequence. We'd like to sigterm any misbehaving logical replication connections which remain after x seconds. These will inevitably get killed anyway on master stop timeout.
* remove "role=master" label on current primary when not using k8s as DCS. Waiting until after Postgres fully stops can sometimes be too long for this.
* Pause pgbouncer connections before switchover
Close#2596
Sometimes MacOS workers are so slow that Postgres shutdown might take more than 30s-40s, what breaks a test with replica reinit in parallel with the primary restart, because basebackup() does only two attempts and in a pause the replica remains running with empty PGDATA.
In addition to that increase timeouts in ignore_slots test.
Close#2637
Sets request queue size for TCP socket used by Patroni REST API. Once the queue is full, further requests get a "Connection denied" error. The default value is 5.
Previous to this commit the `parse_dcs` function would fail with a `PatroniCtlException` if the user ever passed an `etcd3` URL through `--dcs-url` command-line option.
As a consequence, the only way of using `etcd3` in `patronictl` was by using a configuration file passed through `-c` command-line option.
This commit fixes that issue and allows `etcd3` to be used in `--dcs-url`.
References: PAT-91
Close#2638
1. make `SyncHandler.current_state()` return `CaseInsensitiveSet` instead of `list` objects.
2. take `sync_node_count` and `sync_node_maxlag` from the `Postgresql._global_config` instead of passing them as arguments.
3. Make `AbstractDCS.write_sync_state()` accept any `Collection`-like objects.
For now it implements:
- CaseInsensitiveDict()
- CaseInsensitiveSet()
Update `patroni.postgresql.sync.parse_sync_standby_names()` to use `CaseInsensitiveSet()` instead of `CaseInsensitiveDict()`
* Ignore D401 in flake8-docstrings
* Fix newly reported flake8 issues, ignore the old W503 rule
* rely on concatenation of adjecent strings
* Format behave scripts
* Reformat ha.py according to new rules
Co-authored-by: Alexander Kukushkin <[email protected]>
despite being EOL the number of downloads from pypi for 3.6 is on the first place, hence we need to support it.
Fixed a couple of problems in tests:
1. pytest complains about `call` imported from mock, mock.call() is fine
2. one test of Citus intergration was broken
Previously it used to compare between the leader and sync_standbys, while in some cases (actually most of them) the leader should be excluded.
This commit makes `matches()` method flexible:
1. The leader will be included to comparison only if requested
2. checks will be performed as case insensitive (like PG does)
Besides that, everywhere in code start using `cluster.sync.matches()` instead of `name in cluster.sync.members`.
`patronictl edit-config` requires a pager to show the diff output back to the user. It used to be hard-coded to use either `less` or `more`.
When these tools were not available in the host that would cause `patronictl` to face an exception in `ydiff` module and to show the stack trace in the console.
This PR changes `patronictl edit-config` command to behave like this:
- If `PAGER` environment variable is set, attempt to find the corresponding executable.
- If `PAGER` is not set or is set with an invalid executable, then attempt to use either `less` or `more` as it used to do.
- If no executable is find at all then throw a `PatroniCtlException` to show an user friendly message
Unit tests in `tests/test_ctl.py` were modified accordingly.
References: PAT-21
Close#2604
It was possible to have it empty if the all cluster keys are missing in DCS. In this case the `Cluster` object was manually created with all values set to `None` or `[]` (including sync).
It already resulted in #2217, which is in fact wasn't a correct fix.
In order to solve it and reduce code duplication we introduce `Cluster.empty()` and `SyncState.empty()` methods, which will create corresponding empty objects and start using `Cluster.empty()` from all places where the empty `Cluster` object was manually created.
If communication with etcd nodes failed it is logical to start from scratch, from nodes that are listed in the config. But, it could happen that config is in fact outdated and all nodes in the real cluster were replaced.
Previously we used to track whether config file was changed, which turned out not to work in all possible cases.
The new strategy is a bit more different - if communication with all nodes failed we will continue keeping the last know topology and at the same time will try to figure out the new one by merging two lists together, the cached list and the list from the config file.
We made incorrect assumption that `citus_set_coordinator_host()` will trigger `pg_dist_node` sync. Instead we should also use `citus_update_node()` and call `citus_set_coordinator_host()` only during the bootstrap.
Adjust behave tests to verify that coordinator failover is visible on workers.
Configuration parameter is `kubernetes.retriable_http_codes` or `PATRONI_KUBERNETES_RETRIABLE_HTTP_CODES` environment variable.
These status codes are added to the default list of 500, 503, 504.
Close https://github.com/zalando/patroni/issues/2536
It could happen that Patroni is started up before PGDATA was mounted. In this case Patroni can't determine major Postgres version from PG_VERSION file. Later, when PGDATA is mounted, Patroni was trying to create the recovery.conf even if the actual Postgres major version is newver than 12.
To mitigate the problem we double check that the `Postgresql._major_version` is set before writing recovery configuration or starting postgres up.
Close https://github.com/zalando/patroni/issues/2434
Since a long time Patroni enforcing only one callback script running at a time. If the new callback is executed while the old one is still running, the old one is killed (including all child processes).
Such behavior is fine for all callbacks but on_reload, because the last one may accidentally cancel important ones, that for example updating DNS or assigning/removing Virtual IP.
To mitigate the problem we introduce a dedicated executor for on_reload callbacks, so that on_reload may only cancel another on_reload.
Ref: https://github.com/zalando/patroni/issues/2445
