By default, haproxy sens an OPTION request, which we didn't
handle until now. In addition, all haproxy requests that doesn't
examine the request body close the connection as soon as the status
code is obtained. Such behavior breaks BaseHTTPRequestHandler,
namely handle_one_request, which doesn't check for connection reset
by peer and throw this error on a higher level, but since we don't
call this function directly, there is no place in the code to catch
it, therefore, we have to patch this function in the base class.
In addition, patch the StreamRequestHandler finish() function in
order to handle the connection reset error.
Re-read the cluster from DCS right after the failover to supply
the correct new values to the API thread. Fix a typo.
The initialize key was checked against the value
of the node name before removal, but it was changed
recently to contain either an empty string, or cluster
sysid. To fix this, the check for the previous value
was simply removed: we can guarantee that the code path
that removes the key is the one that created it.
One can use pgpass configuration parameter in the postgres
subsection of Patroni. By default pgpass is written in ~/.
Mock actual writes to pgpass in the tests.
I accidentially removed the call when moving the backup functions
to the external script. It is intended to save the configuration,
so that at the restore phase one can just copy backup files.
Its primary intention was to save configuration files in the WAL-E
case (WAL-E just omits everything with .conf), but it is also
useful in the pg_basebackup case, which omits all symlinks, leaving
the cluster with .conf files symlinked in the broken state.
After the PostgreSQL crash (i.e. with kill -9), the backend
patroni connects to may still exist. In this case, patroni
will get stale postgres role from this backend, preventing
a restarted node with a leader lock from being promoted.
Easily reproducible and also observed in a staging environment
after the postgres crash due to out of disk space.
- check the link before checking the file when deciding to remove it,
as isfile follows symlinks and, therefore, may return True on them.
- Remove append mode from write_pgpass, as it is always written anew
before it is used.
- make pg_controldata return an empty hash in case of an error, and
check for the empty value return by this function before using it.
some other minior fixed and test updates.
If patroni detects the former master was killed, it runs
it first in a single-user mode and then shuts down normally,
to make sure pg_rewind will see a normal shut down status
in pg_controldata.
Add a flag need_rewind, since the point where it is detected
that rewind might be necessary is moved out the code that
runs rewind.
Prevent treating the incompletely-initialized PostgreSQL cluster
as a valid on restart by forcefully moving the data directory.
I don't want to remove it altogether, since a DBA might decide
to analyze the failed PG cluster in order to resolve the init
issue.
If initializer node suddenly dies before the initialization is complete,
other nodes should try to take over.
Fix some unittests for etcd and zookeeper and add couple of new ones.
Cleanup the initialize flag if the initializing node fails
to bootstrap its PostgreSQL database.
Rename dcs.race to initialize, since we only call it for the
initialize flag. Factored out PostgreSQL bootstrapping code
into a separate function.
In addition, restart is now performed as stop/start,
which would allow it to benefit from the shutdown speedup.
The hooks in start/stop are modified in order not to run
when called as a part of restart.
- Defer SIGCHLD processing until the later stage in order to allow
subprocess to run its own reaping of children and return the correct
exit code from the process called.
- When changing the password at startup, do not assume the user postgres.
- add unit tests for the restore module
- additional dependencies in requirement
- harden the code that calls restore callbacks
- remove WAL-E related code from postgresql.py
- make sure all parameters are taken from the env variables.
- make sure WAL-E code is not involved if the image is not configured for it
- pick up pg_basebackup as an alternative if WAL-E fails.
The WAL-E support will be switched on in subsequent commits.
For 'on_stop' or 'on_restart' callbacks we cannot
call is_leader, because the cluster would be stopped.
Instead, call is_leader before running a callback.
In addition, make the call_nowait more robust by handling
the exceptions related to the DB cluster unavailability.
Functionality provided by that module will be achieved via callbacks.
- fix a typo in aws.py
- fix some mixups of old code and new callbacks in postgresql.py
Oleksii Kliukin