Starting from 1.27 there is containerd process, which also uses k3s binary and being detected by pidof. Therefore we will search for "k3s server" string in the process list instead of just "k3s".
* Use YAML files to validate Postgres GUCs through Patroni.
Patroni used to have a static list of Postgres GUCs validators in
`patroni.postgresql.validator`.
One problem with that approach, for example, is that it would not
allow GUCs from custom Postgres builds to be validated/accepted.
The idea that we had to work around that issue was to move the
validators from the source code to an external and extendable source.
With that Patroni will start reading the current validators from that
external source plus whatever custom validators are found.
From this commit onwards Patroni will read and parse all YAML files
that are found under the `patroni/postgresql/available_parameters`
directory to build its Postgres GUCs validation rules.
All the details about how this work can be found in the docstring
of the introduced function `_load_postgres_gucs_validators`.
When using a custom Postgres distribution it may be the case that the Postgres binaries are compiled with different names other than the ones used by the community Postgres distribution.
With that in mind we implemented a new set of settings for Patroni, so the user is able to override the default binary names with custom binary names through the new section postgresql.bin_name in the local configuration.
References: PAT-17.
- abstract_main only creates Config object using the passed configfile
and instantiates the passed daemon class
- common args parser is extracted into a separate func that is called
from daemons' main funcs (specific args can be added afterwards)
Co-authored-by: Alexander Kukushkin <[email protected]>
it helps to get rid of recurring patterns:
```python
retry.deadline = retry.stoptime - time.time()
if retry.deadline < XXX:
raise Exception(...) or return False
```
Allows for running behave tests with an alternative base image
than the official postgres image.
Also provides a PG_USER/PG_GROUP should that be different to the
default `postgres`.
pass reference to a last known leader object in order to avoid obtaining it from the `AbstractDCS.cluster` cache.
This change is useful for Consul, Etcd3 and Zookeeper implementations.
It didn't took into account the fact that we can get a new lease after changing a TTL. In this case we have to update the exiting leader key with the new lease.
To solve it we introduce the on transaction 'failure' callback. The whole workflow looks like (schematically):
```python
txn(
compare=(old_value == self._name)),
success=put(self.leader_path, self._name, self._lease),
failure=txn(
compare=(create_revision == '0'),
success=put(self.leader_path, self._name, self._lease)
)
)
```
The problem was introduced in d98d6d0b02c9cc67464a7b1b31b1c5570c26e12d
- Always pass etcd3 key revision as a string
- Make sure the leader key isn't unconditionally overwritten. It may happen that the leader heart-beat loop didn't run properly and the session has expired. In this case the leader may create a new session and a new leader key. But, there are chances that the other node already created a leader key and we don't want to overwrite it.
- Try to sync HA loops between nodes by adding 0.5 seconds to timeout on non-leader nodes
The `__str__` method was calling `repr` over the exception message,
which was causing `print` calls to render not so nicely, e.g.:
```
$ patronictl show-config
Error: 'Can not find suitable configuration of distributed configuration store\nAvailable implementations: consul, etcd, etcd3, exhibitor, kubernetes, raft, zookeeper'
```
By removing the `__str__` method we get a better rendering, e.g.:
```
$ patronictl show-config
Error: Can not find suitable configuration of distributed configuration store
Available implementations: consul, etcd, etcd3, exhibitor, kubernetes, raft, zookeeper
```
References: PAT-107.
Signed-off-by: Israel Barth Rubio <[email protected]>
- fix --validate-config not to error out if bin_dir is an empty string in the yaml config
- mention bin_dir optionality in the docs
- validate bin_dir even if it is not in the yaml config (add optional
default value for Optional config params in validator)
- make rewind user optional
Make it return the new `SyncState` object in order to avoid reading the new cluster state in the Ha.process_sync_replication().
Now it is a small optimization, but it will become very handy in the quorum commit feature.
- make parse_sync_standby_names() return NamedTuple instead of dict
- fix little issue in Postgresql.reset_cluster_info_state(), it should check global_config independently from cluster and cluster.config
- added pyrightconfig.json with typeCheckingMode=strict
- added type hints to all files except api.py
- added type stubs for dns, etcd, consul, kazoo, pysyncobj and other modules
- added type stubs for psycopg2 and urllib3 with some little fixes
- fixes most of the issues reported by pyright
- remaining issues will be addressed later, along with enabling CI linting task
* Further nested lists rendering fixes
* Remove a couple of sphinx warnings
* Fix bootstrap.users.password description
* Boto->boto3 in README's
* Split configuration docs and move some lines across files
* Fix a typo
The two cases we have in mind are:
* In spite of following all best practices client-side, logical replication connections can sometimes hang the Postgres shutdown sequence. We'd like to sigterm any misbehaving logical replication connections which remain after x seconds. These will inevitably get killed anyway on master stop timeout.
* remove "role=master" label on current primary when not using k8s as DCS. Waiting until after Postgres fully stops can sometimes be too long for this.
* Pause pgbouncer connections before switchover
Close#2596
Sometimes MacOS workers are so slow that Postgres shutdown might take more than 30s-40s, what breaks a test with replica reinit in parallel with the primary restart, because basebackup() does only two attempts and in a pause the replica remains running with empty PGDATA.
In addition to that increase timeouts in ignore_slots test.
Close#2637
Sets request queue size for TCP socket used by Patroni REST API. Once the queue is full, further requests get a "Connection denied" error. The default value is 5.
Previous to this commit the `parse_dcs` function would fail with a `PatroniCtlException` if the user ever passed an `etcd3` URL through `--dcs-url` command-line option.
As a consequence, the only way of using `etcd3` in `patronictl` was by using a configuration file passed through `-c` command-line option.
This commit fixes that issue and allows `etcd3` to be used in `--dcs-url`.
References: PAT-91
Close#2638
1. make `SyncHandler.current_state()` return `CaseInsensitiveSet` instead of `list` objects.
2. take `sync_node_count` and `sync_node_maxlag` from the `Postgresql._global_config` instead of passing them as arguments.
3. Make `AbstractDCS.write_sync_state()` accept any `Collection`-like objects.
For now it implements:
- CaseInsensitiveDict()
- CaseInsensitiveSet()
Update `patroni.postgresql.sync.parse_sync_standby_names()` to use `CaseInsensitiveSet()` instead of `CaseInsensitiveDict()`
* Ignore D401 in flake8-docstrings
* Fix newly reported flake8 issues, ignore the old W503 rule
* rely on concatenation of adjecent strings
* Format behave scripts
* Reformat ha.py according to new rules
Co-authored-by: Alexander Kukushkin <[email protected]>
