A Patroni configuration should be enough for Patronictl
The previous dcs_api url style is still supported.
To remove duplicate magic variables a DCS_DEFAULTS was introduced.
Some behaviour has changed: If you do not specify a DCS at all
(not in configuration, not on commandline, not in environment), it will
not default to etcd://localhost:4001
More test coverage for patronictl
Previously we explicitly injected a replication record into pg_hba.conf.
This doesn't allow users to explicitly write their configurations.
This change will just write the lines specified by the user.
Environment variables available to the postmaster can be examined
by non-superusers (e.g. plperl is trusted, yet you can show the environment).
In some environments secrets may be written in environment variables.
Commit 9744019341 introduced the ability to
specify the full Patroni configuration as an environment variable,
PATRONI_CONFIGURATION.
PATRONI_CONFIGURATION will by definition contain secrets, the passwords for superuser,
replcation user etc.
We therefore only retain a small subset of the environment for pg_ctl start, to ensure
no leakage of these values are possible.
Previously we explicitly injected a replication record into pg_hba.conf.
This doesn't allow users to explicitly write their configurations.
This change will just write the lines specified by the user.
Scheduled failover allows scheduling of a failover in the future. It does this by writing
a failover key in the DCS which contains the scheduled failover time.
The reason to allow a scheduled failover, is that it does not require one to use a
scheduler (e.g. cron) to schedule such a failover. One of the issues with using
a scheduler is that it may need to authenticate itself. With scheduled failover the authentication
takes place during the scheduling, not during the actual failover.
To allow the time of failover to be expressed, the failover key has changed its format; the old format however
can still be used. The new format expects the failover key to be a json-document with relevant
keys set. We need the timestamp specified to be time zone aware and to be expressed unambigiously, e.g. ISO 8601.
Our current master branch doesn't pass the code coverage test, due to behaviour changes
in upstream python-etcd.
As a bandaid, fix the version for now.
Reference build fail:
https://travis-ci.org/zalando/patroni/jobs/98470121
For easier development using Docker the $HOSTNAME variable will be used to
name the running Patroni. Bumped some _segments postgresql settings to ensure
WAL files are not removed very quickly.
Increased the timeout for the post request for Patroni, as some operations
(failover) may take considerable time to complete.
The failover to a specific member was broken in patronictl as it used a wrong
key to specify the member to failover to.
Pretty printing fix for xlog lag, to prevent false negatives to show up and have
good alignment.
Some refactoring to reuse some codepaths.
A dsn option is now added, it is useful in scripts like so:
psql -d "$(patronictl dsn alpha)"
Restarting has been extended to allow restarting based on role.
Previously, the leader key was watched for changes after a failover. This resulted in a delay
of up to 10 seconds to report a healthy failover back to the client.
With this patch, we are not relying on the role of a member registered in the dcs anymore.
For managing Patroni clusters, the Patroni api can be used. For many tasks, a command line interface for
this api would be a useful addition. This commit adds patroncli (The name is still under debate).
The command line interface needs access to the DCS; this is required for any operation. For some tasks it is required
to have access to the Patroni api.
A small summary of the additions to get the cli/ctl started:
* Updated Docker image to use 'true' as the archive_command, to ensure disk not filling up during failover
testing.
* The cli currently can list members, failover a master and remove a given cluster from DCS.
* The cli can be configured with a command, for repeated access to the same DCS
* Added some simple tests for the cli, code coverage is very low
As the Dockerfile is there mainly to support developers, we want to build the Dockerfile using the current working
directory instead of a previously released version.
To help in developing features, the Dockerfile and its entrypoint have been extended.
The README.md explains stuff in detail, in short:
- you can now run a Patroni cluster with a single command
Due to the renaming of Governor to Patroni some old references needed to be updated.
Also some python packages need to be added.
Added entrypoint.sh as a script, to ensure Patroni will have PID = 1 when the container is run.
The Dockerfile now only builds upon the publicly available Ubuntu docker image, thereby reducing the size of the image.
etcd release 2.0.12 was released today as well, updated that dependency.
Removed aws reference from Dockerfile, remove unnecessary steps.
Connect using libpq parameters, therefore providing nice application name for Governor.
For the postgresql helper, hardened the code to get "the" cursor of the postgresql instance.
For the statuspage, a small status json is returned.
To find out what status a PostgreSQL cluster is in we use the cursor (instead of the provided query() function), as the query function does some retrying etc. For the healthcheck we want to simple provide an answer to a simple query, if we have to reconnect, we are not *that* healthy anyway.
Dropped catching exceptions in the do_GET block, as the HTTPServer will do that nicely for us anyway.
The StatusPage uses a BaseHTTPServer to server 3 simple http endpoints:
- pg_master, returns 200 if PostgreSQL is running as master
- pg_slave, returns 200 if PostgreSQL is running as slave
- pg_status, returns some information about the cluster
It does this by relying on the PostgreSQL connection of the Governor.
Feike Steenbergen