1612 Commits
Author SHA1 Message Date
Alexander KukushkinandGitHub c6e70a9910 Release 1.5.5 (#979)
* Bump version
* Update release notes
v1.5.5
2019-02-15 16:14:39 +01:00
Alexander KukushkinandGitHub 0ec1760397 Don't write primary_conninfo into recovery.conf for wal only standby cluster (#971)
It is useless and makes postgres to generate a lot of errors
2019-02-15 13:35:34 +01:00
Alexander KukushkinandGitHub 317721991b Fix handling of PATRONI_*_PASSWORD environment variables (#970)
Bug was introduced in https://github.com/zalando/patroni/pull/947 and https://github.com/zalando/patroni/pull/500
2019-02-15 13:35:22 +01:00
Alexander KukushkinandGitHub 0c516de147 Create headless service associated with $SCOPE-config endpoint (#958)
if there is no service defined k8s assumes that endpoint is orphaned and removes it.
Patroni tries to create the service only in case if use_endpoints is enabled if the following cases:
1. Upon start
2. When it tries to (re-)create the config endpoint

If for some reason creation of the service has failed, Patroni will retry it on every cycle of HA loop. Usually it fails due to lack of permissions and if you don't want to give such permissions to the service account used by Patroni, you can create the service explicitly in the deployment manifest.
2019-02-15 13:35:04 +01:00
Michael BanckandAlexander Kukushkin 073074f83e Run coverage as python -m coverage (#968)
Depending on the platform the coverage binary might not always be available under the standard name.
2019-02-13 16:02:12 +01:00
Michael BanckandAlexander Kukushkin 345e6d3131 Copy away output directories of failed acceptance tests. (#967)
And dump logs on travis from only failed features
2019-02-13 16:00:15 +01:00
Michael BanckandAlexander Kukushkin d01a9bdcd5 Change base port for acceptance tests from 5440 to 5360 (#966) 2019-02-13 15:59:13 +01:00
Alexander KukushkinandGitHub 10bbf0c3c5 Always use replication=1, otherwise it is considered a logical walsender (#952)
Fixes: https://github.com/zalando/patroni/issues/894
Fixes: https://github.com/zalando/patroni/issues/951
2019-01-30 12:38:51 +01:00
Alexander KukushkinandGitHub 4304560ce2 Adjust read timeout for leader watch blocking query (#950)
According to the Consul documentation the actual response timeout is increased by a small random amount of additional wait time added to the supplied maximum wait time to spread out the wake up time of any concurrent requests. It adds up to wait / 16 additional time to the maximum duration.
In our case we will add wait/15 or 1 second depending on what is bigger.

Fixes: https://github.com/zalando/patroni/issues/945
2019-01-30 12:38:24 +01:00
Alexander KukushkinandGitHub 254ee2acfc Show information about timelines in patronictl list (#949)
This information will help to detect stale replicas.

In addition to that Host will include ':{port}' if the port value isn't default or more than one member running on the same host.

Fixes: https://github.com/zalando/patroni/issues/942
2019-01-30 12:37:51 +01:00
Alexander KukushkinandGitHub 739329b590 Make it possible to automatically reinit the former master (#948)
If the pg_rewind is disabled or can't be used, the former master could fail to start as a new replica due to diverged timelines. In this case, the only way to fix it is wiping the data directory and reinitializing.

So far Patroni was able to remove the data directory only after failed attempt to run pg_rewind. This commit fixes it.
If the `postgresql.remove_data_directory_on_diverged_timelines` is set, Patroni will wipe the data directory and reinitialize the former master automatically.

Fixes: https://github.com/zalando/patroni/issues/941
2019-01-30 12:37:21 +01:00
Étienne MandAlexander Kukushkin bd2c54581a Add ETCD_(PROTOCOL|USERNAME|PASSWORD) env variables (#947)
Fix #944
2019-01-30 12:36:50 +01:00
Maxim IvanovandAlexander Kukushkin f0b12b7e2e Document create_replicas_methods in standby_cluster section (#939)
Fixes https://github.com/zalando/patroni/issues/935
2019-01-30 12:36:24 +01:00
Étienne MandAlexander Kukushkin 93d157dea3 Document how to start Patroni with an existing data directory (#918) 2019-01-30 12:35:57 +01:00
Alexander KukushkinandGitHub 2c128520cf Python34 compatibility (#933)
and some other minor fixes.

Closes https://github.com/zalando/patroni/issues/932
2019-01-16 14:40:05 +01:00
Alexander KukushkinandGitHub 381a5b80d2 Release 1.5.4 (#931)
* Bump version
* Update release notes
* Make it possible to configure registration of Service in Consul via env variables
v1.5.4
2019-01-15 12:14:19 +01:00
Alexander KukushkinandGitHub 71dae6a905 Optionally consider node not healthy if it is not on the latest timeline (#892)
The latest timeline is calculated from the `/history` key in DCS. In case there is no such key or it contains some garbage we consider the node healthy.
Closes https://github.com/zalando/patroni/issues/890
2019-01-15 11:16:30 +01:00
Alexander KukushkinandGitHub cf34fb3934 Relax requirements on superuser credentials (#930)
libpq allows opening connection without explicitly specifying neither username nor password. Depending on situation it would rely either on `pgpass` file or trust authentication method in pg_hba.conf.

Since pg_rewind is also using libpq, it could work the same way.

Fixes https://github.com/zalando/patroni/issues/928
2019-01-15 11:15:35 +01:00
Alexander KukushkinandGitHub e080ded44b Make logging configurable via YAML file (#927)
It allows changing logging settings in runtime by updating config and doing reload or sending `SIGHUP` to the Patroni process.
Important! Environment configuration names related to logging were renamed and documentation accordingly updated. For compatibility reasons Patroni still accepts `PATRONI_LOGLEVEL` and `PATRONI_FORMAT`, but some other variables related to logging, which were introduced only
recently (between releases), will stop working. I think it is ok, since we didn't release the new version yet and therefore it is very unlikely that somebody is using them except authors of corresponding PRs.

Example of log section in the config file:
```yaml
log:
  dir: /where/to/write/patroni/logs  # if not specified, write logs to stderr
  file_size: 50000000  # 50MB
  file_num: 10  # keep history of 10 files
  dateformat: '%Y-%m-%d %H:%M:%S'
  loggers:  # increase log verbosity for etcd.client and urllib3
    etcd.client: DEBUG
    urllib3: DEBUG
```
2019-01-15 08:42:13 +01:00
jouirandAlexander Kukushkin dec3656f6e Redirect HTTPServer exceptions to logger (#900) (#925)
By default they were written to stdout
2019-01-15 08:37:06 +01:00
Alexander KukushkinandGitHub 1e2d89fa58 Apply 5 second backoff when loading global config up on start (#922)
It doesn't make much sense to hammer DCS when we just starting up.
Fixes https://github.com/zalando/patroni/issues/919
2019-01-14 14:55:56 +01:00
Alexander KukushkinandGitHub 994863c18d Refactor wait_for_user_backends_to_close method (#917)
1. Log only debug level messages on any kind of error
2. Update regexp for matching postgres aux processes to make it compatible with postgres 11

Fixes https://github.com/zalando/patroni/issues/914
2019-01-14 14:55:45 +01:00
Alexander KukushkinandGitHub 3fce982909 Set archive_mode to off during the custom bootstrap (#911)
We want to avoid archiving WALs and history files until the cluster is fully functional. It should really help if the custom bootstrap involves pg_upgrade.
2019-01-14 14:55:23 +01:00
Lucas CapistrantandAlexander Kukushkin d306092cbc Explicitly secure rw perms for recovery.conf at creation time (#910)
We don't want anybody except patroni/postgres user reading this file, it contains replication user and password.
2019-01-14 14:22:04 +01:00
Dmitry DolgovandAlexander Kukushkin 11f7ceb521 Do not check types of standby_cluster configuration (#924)
Simply allow valid keys
2019-01-14 14:16:15 +01:00
bradnicholsonandAlexander Kukushkin 05a13839aa Update replica_bootstrap.rst (#915)
Add some docs about replication slots for standby clusters
2019-01-14 12:57:21 +01:00
Étienne MandAlexander Kukushkin 04ac199fc8 Single quotes are mandatory around each host in PATRONI_ETCD_HOSTS (#926)
Otherwise YAML parser fails
2019-01-14 11:56:15 +01:00
anikin-aaandAlexander Kukushkin 0be8a9527b possibility to set logdatefmt via env. (#904)
The value could be configured via `PATRONI_LOGDATEFMT` environment variable. The default value is `%Y-%m-%d %H:%M:%S`
2019-01-14 08:46:38 +01:00
Pavel KirillovandAlexander Kukushkin 929ff08bfd Service deregister timeout must be in Go time format (#893) 2018-12-21 15:42:15 +01:00
Cody CoonsandAlexander Kukushkin 7bc8d0aac9 Removed stderr pipe to stdout on pg_ctl process (#896)
Inheriting stderr from the main Patroni process allows all Postgres logs to be seen along with all patroni logs. This is very useful in a container environment as Patroni and Postgres logs may be consumed using standard tools (docker logs, kubectl, etc).

In addition to that, this change fixes a bug with Patroni not being able to catch postmaster pid when postgres writing some warnings into stderr
2018-12-21 15:41:34 +01:00
Lucas CapistrantandAlexander Kukushkin f3da6de129 Add ability to configure app logs to be written to a file (#903)
It gives users the option to send Patroni application logs to a File instead of Standard Out. There are three environment variables that can be set to enable and configure file logging.
1. `PATRONI_FILE_LOG_DIR`: Path to a directory that is writeable by the executing user. Having this variable set is what activates file logging.
2. `PATRONI_FILE_LOG_NUM`: This is a rolling file logger. This variable dictates how many log files are retained.
3. `PATRONI_FILE_LOG_SIZE`: This variable dictates the size at which the logs will roll.

If `PATRONI_FILE_LOG_DIR` is not set than Patroni will log to stderr (default behavior does not change)

Closes https://github.com/zalando/patroni/issues/902
2018-12-21 15:38:29 +01:00
Alexander KukushkinandGitHub 491f230711 Release 1.5.3 (#889)
* Bump version
* Update release notes
v1.5.3
2018-12-03 17:12:53 +01:00
Alexander KukushkinandGitHub f1d7ccf36e Make sure we refresh session at least once per HA loop (#880)
Fixes https://github.com/zalando/patroni/issues/879
2018-12-03 16:35:14 +01:00
Kostiantyn NemchenkoandAlexander Kukushkin 96ea01bee4 Fix kubernetes demo files (#885)
- Update postgres docker image to the latest 11 version.

- Remove empty lines inside the `RUN` command to make the Dockerfile compatible with future docker versions.

- Set the `PATRONI_KUBERNETES_POD_IP` environment variable, which is required when _use_endpoints_ is enabled. Otherwise, the `KeyError` is raised [here](https://github.com/zalando/patroni/blob/master/patroni/dcs/kubernetes.py#L95).

- Set `EDITOR` environment variable to make configuration changes via `patronictl edit-config`.
2018-12-03 15:46:25 +01:00
Alexander KukushkinandGitHub e684ca66e5 Compatibility with postgres 9.3 (#882)
use replication=1 instead of replication='database' when opening replication connection, 9.3 doesn't understand it
2018-11-30 15:46:05 +01:00
Alexander KukushkinandGitHub 1a0876e5ca Refactor acceptance tests to improve stability (#884)
Hope it will crash less often when executed on travis against k8s
2018-11-30 12:40:56 +01:00
Alexander KukushkinandGitHub 9bf074acfb Compatibility with python3 (#883)
Change of `loop_wait` was causing Patroni to disconnect from zookeeper and never reconnect back. The error was happening only with python3 due to a difference in implementation of `select.select` function.
2018-11-30 11:40:34 +01:00
Alexander KukushkinandGitHub f8f928420d Release 1.5.2 (#875)
* Update release notes
* Bump version
v1.5.2
2018-11-26 10:31:14 +01:00
Shea StewartandAlexander Kukushkin 6519a192b1 add openshift customizations, templates, and test (#871)
- It modifies the Dockerfile and entrypoint slightly to allow for OpenShift SCCs to operate correctly
- It adds 2 template examples that can be easily modified by changing parameters

Fixes #572
2018-11-21 18:01:39 +01:00
Alexander KukushkinandGitHub 8c7e1892ee touch_member method should not raise exceptions (#859)
Fixes https://github.com/zalando/patroni/issues/853
2018-11-21 12:51:52 +01:00
Lardière SébastienandAlexander Kukushkin a1ba2cdca7 Add confd template for pgbouncer (#844) 2018-11-21 12:03:22 +01:00
Kostiantyn NemchenkoandAlexander Kukushkin ce9c7bdadc Describe Consul registration parameters (#870)
Mention Consul related parameters introduced by #802
2018-11-21 12:00:40 +01:00
Alexander KukushkinandGitHub fb01aaebc5 Compatibility with kazoo-2.6.0 (#872)
Recently 2.6.0 was release which changes the way how create_connection method is called. Before it was passing two arguments, and in the new version all argument names are specified explicitly.
2018-11-19 14:26:20 +01:00
alago197andAlexander Kukushkin a13cc8b847 Update SETTINGS.rst with connect_address clarifications (#858) 2018-11-12 16:56:51 +01:00
lwsboxandAlexander Kukushkin bd9f30372e fix bug: change pip to pip3 (#851)
should use python3 to install in postgres10 image.
2018-11-05 14:17:22 +01:00
Josh BerkusandAlexander Kukushkin d247b5ae17 Fix several build problems with the Docker image for testing Kubernet… (#735)
1. Update to Postgres 10
2. Install most of the python modules from deb packages
3. Do *not* upgrade pip
2018-11-04 08:31:11 +01:00
Alexander KukushkinandGitHub b4f35ecca0 Release 1.5.1 (#846)
* Bump version
* Update release notes
v1.5.1
2018-11-01 16:18:41 +01:00
Alexander KukushkinandGitHub 0f666e69f3 Prefix system tables, views and functions with pg_catalog (#845)
and implement missing unit tests
2018-11-01 16:17:40 +01:00
Alexander KukushkinandGitHub 2efd97baab Permanent replication slots (#819)
Permanent replication slots are preserved on failover/switchover, that is Patroni on the new primary will create configured replication slots right after doing promote.

Slots could be configured with the help of `patronictl edit-config`.
The initial configuration could be also done in the `bootstrap.dcs`

```yaml
slots:
  permanent_physical_1:
    type: physical
  permanent_logical_1:
    type: logical
    database: foo
    plugin: pgoutput
```

It is the responsibility of the operator to make sure that there are no clashes in names between replication slots automatically created by Patroni for members and permanent replication slots.

Closes https://github.com/zalando/patroni/issues/656
2018-10-31 11:37:42 +01:00
Alexander KukushkinandGitHub c65c4f1ffe Round-robing across all masters in pause mode if DCS is not accessible (#842)
Regression was introduced in https://github.com/zalando/patroni/commit/90cf930036a9d5249265af15d2b787ec7517cf57

Fixes https://github.com/zalando/patroni/issues/841
2018-10-30 16:48:53 +01:00