where it is not necessary (test_ha, test_ctl, etc...)
It will simplyfy further refactoring and make it possible to install
implementations of AbstractDCS independant of each other.
According to https://github.com/zalando/patroni/issues/48 'clonefrom'
tag should be boolean and it should be used to mark node as a suitable
for creation of a new replica from. If there are more then one such node
in the cluster (with tag clonefrom=true), one of them will be chosed
randomly.
Instead of starting it up in "read-only", it wil wait 2 seconds, to give
a time to somebody to prompte and after it will execute normal `recover`
procedure.
In addition to that rename confusing `Etcd.client` and
`ZooKeeper.client` into `_client`. This attribute is available from
AbstractDCS and people had wrong impression that it provides the same
interface for different DCS implementations, which is obviously not the
case. For Etcd it has type etcd.Client and for ZooKeeper - KazooClient.
Previously, "without_leader" suffix was used in the name of methods
and functions that initialize a replica without an active replication
connection, and leader was part of the name for parameters and messages
that require an active replication conneciton. Since we support init
from the members other than the leader, those conventions have to be
changed.
At the moment we just replace the master with the
node at the 'clonefrom' tag if it's present. Master
should be available anyway, otherwise, it will not
even try to do cloning.
Acceptance tests:
https://github.com/zalando/patroni/pull/144/commits
The replication method should have no_master flag set and
support getting the base backup from some external storage
(i.e. S3). At the moment we only support initialization of
replicas in the already existing cluster when no master is
present, since there is no 'one fits all' way to decide whether
to run initidb or wait for the replica data storage to become
available when dealing with the new cluster.
Rename the follow_the_leader to just follow, since the node to
be followed is not necessary a leader anymore. Extend the code
that manages replication slots to the non-master nodes if they
are mentioned in at least one replicatefrom tag.
Add the 3rd configuration in order to be able to run cascading
replicas.
Call normal follow the leader method from HA even during recovery.
This provides a single place that changes recovery.conf, making
it easier to plug in a cascading replica in the future.
Remove an obsolete demote function from PostreSQL module, modified
the tests.
Previously, the leader key was watched for changes after a failover. This resulted in a delay
of up to 10 seconds to report a healthy failover back to the client.
With this patch, we are not relying on the role of a member registered in the dcs anymore.
For managing Patroni clusters, the Patroni api can be used. For many tasks, a command line interface for
this api would be a useful addition. This commit adds patroncli (The name is still under debate).
The command line interface needs access to the DCS; this is required for any operation. For some tasks it is required
to have access to the Patroni api.
A small summary of the additions to get the cli/ctl started:
* Updated Docker image to use 'true' as the archive_command, to ensure disk not filling up during failover
testing.
* The cli currently can list members, failover a master and remove a given cluster from DCS.
* The cli can be configured with a command, for repeated access to the same DCS
* Added some simple tests for the cli, code coverage is very low
Tags are labels assigned to individual members in order
to alter its default behavior, i.e. exclude from the
leader election or indicate a possibility to create base
backups from the member.
This commit only adds support for setting tags in the
configuration file, exposes the tags to DCS /member subkey
and returns the tags in a response of the API request. At
the moment the tag names are not validated, nor they are
interpreted in any way.
Support for setting tags via the API is also in the scope
of further work.
1. run touch_member from the main loop
2. move code which takes care about long tasks into separate class
3. change format of data stored in a DCS: use json instead of url
4. change Member class: from now it deserialize everything into data property
5. rework API: from now it takes into account state of the current node in a dcs
Implementation is done on top of feature/is-healthiest-via-api and
feature/api branches.
In order to trigger manual failover one has to create 'failover' key in
a configuration store with the value in following format:
'leader_name:member_name'
leader_name can be empty or should match with the name of current leader
member_name can be empty or should match with the name one of cluster
nodes
Leader always checks that either desired member (if specified) or one of
the memners is accessible and healthy before demote.
After leader has deomted himself other nodes are performig checks that
desired node is healthy. If it is not they are participating in a leader
race. In some cases (when accidently there is no healthy nodes) former
leader can also participate in a leader race.
Current implementation does not provide REST API endpoint for a manual
failover.
Alexander Kukushkin