For managing Patroni clusters, the Patroni api can be used. For many tasks, a command line interface for
this api would be a useful addition. This commit adds patroncli (The name is still under debate).
The command line interface needs access to the DCS; this is required for any operation. For some tasks it is required
to have access to the Patroni api.
A small summary of the additions to get the cli/ctl started:
* Updated Docker image to use 'true' as the archive_command, to ensure disk not filling up during failover
testing.
* The cli currently can list members, failover a master and remove a given cluster from DCS.
* The cli can be configured with a command, for repeated access to the same DCS
* Added some simple tests for the cli, code coverage is very low
The initialize key was checked against the value
of the node name before removal, but it was changed
recently to contain either an empty string, or cluster
sysid. To fix this, the check for the previous value
was simply removed: we can guarantee that the code path
that removes the key is the one that created it.
If the namespace is not specified in a config file /service/ would be
used.
Also it's possible to use just '/' as a namespace. It means we would
have following structure:
/scope1
/scope2
...
Such situation could happen if we replaced all etcd nodes except one
which was used by patroni. After replacing the last node patroni will
try to execute request on all other nodes from machines_cache but non of
them are available. Michines cache would became empty and patroni will
stick to the latest node which was available in the machines_cache and
will never try to refresh machines_cache from dns for example.
Currently machines cache is refreshed only when one request to the etcd
cluster has failed, but probably it should be done periodically, for
example every minute...
One can use pgpass configuration parameter in the postgres
subsection of Patroni. By default pgpass is written in ~/.
Mock actual writes to pgpass in the tests.
I accidentially removed the call when moving the backup functions
to the external script. It is intended to save the configuration,
so that at the restore phase one can just copy backup files.
Its primary intention was to save configuration files in the WAL-E
case (WAL-E just omits everything with .conf), but it is also
useful in the pg_basebackup case, which omits all symlinks, leaving
the cluster with .conf files symlinked in the broken state.
If the value is empty and leader didn't changed, this probably means
that failover failed.
After 15 seconds timeout we will consider failover status = unknown
curl -XPOST --data '{"leader": "leader_name", "member": "member_name"}' http://127.0.0.1:8008/failover
It will execute some preliminary checks and write failover key into DCS.
Afterward it will wait until new leader key will appear in a DCS.
It's better to execute this request on the master node. It will send a
signal to the main HA loop which makes possible to release leader key
immidiately even if you are working with etcd.
Feike Steenbergen