gitops/patroni
1
0
Fork 0
mirror of https://github.com/outbackdingo/patroni.git synced 2026-08-25 14:53:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
577 Commits 15 Branches 66 Tags
35641ac0727f04f9527333a45a76e7ded1f55734
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Oleksii Kliukin 35641ac072 Use distinct paths for pgpass from test nodes. 2015-10-20 11:40:52 +02:00
Oleksii Kliukin 92fe6a1de9 Make pgpass location configurable. 
One can use pgpass configuration parameter in the postgres
subsection of Patroni. By default pgpass is written in ~/.
Mock actual writes to pgpass in the tests.
 2015-10-20 11:28:26 +02:00
Oleksii Kliukin 5c86b60cd2 Fix an exception in the (rather unusual) case of attaching Patroni to an existing running replica. 2015-10-14 17:05:09 +02:00
Oleksii Kliukin c7246e48d9 Work around the differences in pg_controldata names. 2015-10-14 09:46:20 +02:00
Oleksii Kliukin 953ea749bf Make sure patroni is not using stale connections. 
After the PostgreSQL crash (i.e. with kill -9), the backend
patroni connects to may still exist. In this case, patroni
will get stale postgres role from this backend, preventing
a restarted node with a leader lock from being promoted.

Easily reproducible and also observed in a staging environment
after the postgres crash due to out of disk space.
 2015-10-13 15:00:16 +02:00
Oleksii Kliukin 101082fa3b more tests. 2015-10-13 09:08:27 +02:00
Oleksii Kliukin 94aa6873f4 Add more tests for the new postgresql methods. 2015-10-13 08:19:44 +02:00
Oleksii Kliukin 46f4788c28 Do not try to run postgres -D during unit tests. 2015-10-12 17:06:13 +02:00
Oleksii Kliukin d7988384d3 Address the code review by Alex Kukushkin: 
- check the link before checking the file when deciding to remove it,
  as isfile follows symlinks and, therefore, may return True on them.
- Remove append mode from write_pgpass, as it is always written anew
  before it is used.
- make pg_controldata return an empty hash in case of an error, and
  check for the empty value return by this function before using it.
 some other minior fixed and test updates.
 2015-10-12 16:24:02 +02:00
Oleksii Kliukin ce7169f61d Add new tests ha and postgresql. 2015-10-12 15:29:47 +02:00
Oleksii Kliukin 42137e689b Merge remote-tracking branch 'origin/master' into feature/pg_rewind_unclean_shutdown 
Modify HA tests to reflect the changes in HA::recover.
 2015-10-12 09:22:38 +02:00
Oleksii Kliukin b629e0852f Call pg_rewind in case of the master's unclean shutdown. 
If patroni detects the former master was killed, it runs
it first in a single-user mode and then shuts down normally,
to make sure pg_rewind will see a normal shut down status
in pg_controldata.

Add a flag need_rewind, since the point where it is detected
that rewind might be necessary is moved out the code that
runs rewind.
 2015-10-12 08:34:08 +02:00
Oleksii Kliukin bad37a5a21 Always check that cluster is configured correctly right before running pg_rewind. 2015-10-02 10:57:49 +02:00
Oleksii Kliukin ea910a8987 Make sure pgpass file name is also passed in the PGPASSFILE environment variable. 2015-09-30 18:02:07 +02:00
Oleksii Kliukin b223319183 use the PATH to get the python interpreter path for the scripts. 2015-09-30 18:00:28 +02:00
Oleksii Kliukin d7403f999d Merge branch 'feature/pg_rewind' of https://github.com/zalando/patroni into feature/pg_rewind 2015-09-30 17:13:16 +02:00
Oleksii Kliukin 316e4f8d74 Merge branch 'master' into feature/pg_rewind 2015-09-30 17:12:03 +02:00
Oleksii Kliukin b4a2ed874f Merge branch 'master' of https://github.com/zalando/patroni 2015-09-30 16:47:07 +02:00
Oleksii Kliukin 182a90973d Make sure tests work with the new pytest. 2015-09-30 16:46:53 +02:00
Oleksii Kliukin a500781b6d Mock remove_data_directory in the pg_rewind test. 2015-09-30 16:32:56 +02:00
Oleksii Kliukin e39d318732 Eliminate os.system call. 2015-09-28 12:04:06 +02:00
Oleksii Kliukin d44a54628a remove the data directory on an unsuccessfull rewind attempt. 2015-09-25 16:00:24 +02:00
Oleksii Kliukin b62d72a130 Merge branch 'master' into feature/pg_rewind 2015-09-25 13:08:42 +02:00
Oleksii Kliukin d6c8df45e1 Write the pg_rewind password in pgpass instead of passing it in the command line. 2015-09-25 13:08:12 +02:00
Oleksii Kliukin 027bcd39ce Move pg_rewind call into a separate sub. 
Add a Postgresql method to call pg_rewind.
Improve the test coverage.
 2015-09-24 12:46:36 +02:00
Oleksii Kliukin c8108f221e Check the exit code of the postgres start when determining whether to run the on_role_change callback. 2015-09-24 11:34:28 +02:00
Oleksii Kliukin 793325cb60 add support for pg_rewind. 2015-09-23 18:38:17 +02:00
Oleksii Kliukin 51eacc5042 Handle the case when initialize flag is not set and leader is present. 2015-09-14 12:36:28 +02:00
Oleksii Kliukin be110c4ba0 Do not try to stop postgres twice if initialization had failed. 2015-09-14 09:20:45 +02:00
Oleksii Kliukin 15cd10669d Change an outdated comment. 2015-09-10 18:04:47 +02:00
Oleksii Kliukin cd312de252 Fix a flake8 warning. 
Improve some unit tests by expecting specific exceptions.
 2015-09-10 17:15:43 +02:00
Oleksii Kliukin 2377c417e4 Fix etcd and zookeper interactions with initialize key. 
Fix unittests as well.
 2015-09-10 16:05:10 +02:00
Oleksii Kliukin 938b946e55 Merge branch 'master' into feature/cleanup_on_failed_initialization 2015-09-10 15:43:31 +02:00
Oleksii Kliukin 30a9e0f7f5 Move PostgreSQL data directory if init had failed. 
Prevent treating the incompletely-initialized PostgreSQL cluster
as a valid on restart by forcefully moving the data directory.
I don't want to remove it altogether, since a DBA might decide
to analyze the failed PG cluster in order to resolve the init
issue.
 2015-09-10 15:34:29 +02:00
Oleksii Kliukin 1c61280d70 Fix path to scripts subdirectory in configuration files. 2015-09-08 16:10:42 +02:00
Oleksii Kliukin ff499604f0 Act on removal of initialization flag. 
If initializer node suddenly dies before the initialization is complete,
other nodes should try to take over.

Fix some unittests for etcd and zookeeper and add couple of new ones.
 2015-09-08 16:04:54 +02:00
Oleksii Kliukin 92647b7aad Merge branch 'master' of https://github.com/zalando/patroni into feature/cleanup_on_failed_initialization 2015-09-08 14:54:52 +02:00
Oleksii Kliukin b842ed478b Make sure initialize flag is reset on failure. 
Cleanup the initialize flag if the initializing node fails
to bootstrap its PostgreSQL database.

Rename dcs.race to initialize, since we only call it for the
initialize flag. Factored out PostgreSQL bootstrapping code
into a separate function.
 2015-09-08 12:03:34 +02:00
Oleksii Kliukin da132ca1fc Do not rely on env vars for superuser role name. 
Environment variables may not be set in Docker.
Instead, obtain the superuser role from the database.
 2015-09-02 15:20:16 +02:00
Oleksii Kliukin b21bc89bd7 Merge branch 'master' into feature/faster_shutdown 2015-09-02 14:16:04 +02:00
Oleksii Kliukin 80f92b1dee Run CHECKPOINT before calling shutdown. 
In addition, restart is now performed as stop/start,
which would allow it to benefit from the shutdown speedup.

The hooks in start/stop are modified in order not to run
when called as a part of restart.
 2015-09-02 14:14:16 +02:00
Oleksii Kliukin a5f2ab6f88 Better SIGCHLD handling, use correct user name. 
- Defer SIGCHLD processing until the later stage in order to allow
  subprocess to run its own reaping of children and return the correct
  exit code from the process called.

- When changing the password at startup, do not assume the user postgres.
 2015-09-02 12:56:23 +02:00
Oleksii Kliukin 3e4d2df33b put a real restore/replication command inside the test postgres.yaml files 2015-09-02 12:04:39 +02:00
Oleksii Kliukin af10bb8ab8 make sure replica-creation scripts are also available to patroni docker image. 2015-08-25 18:22:47 +02:00
Oleksii Kliukin 6b60be09a1 include restore command in the entyrpoint 2015-08-25 16:19:22 +02:00
Oleksii Kliukin 824819bbff Make script compatible with boot2docker on OS X by exporting the correct LC_ variable and running docker with -P, exposing the ports to the underlying OS. 2015-08-25 16:18:59 +02:00
Oleksii Kliukin 29434e24ea Allow relative paths to external scripts. 2015-08-24 16:53:57 +02:00
Oleksii Kliukin 164b5760a4 remove line too long warnings where possible. 2015-08-07 16:35:00 +02:00
Oleksii Kliukin 5d1d59a42b Add mock-based unit tests for the restore module. 
- add unit tests for the restore module
- additional dependencies in requirement
- harden the code that calls restore callbacks
- remove WAL-E related code from postgresql.py
 2015-08-07 16:05:17 +02:00
Oleksii Kliukin 74a1daad09 fix a typo in the named tuple definition. 2015-08-04 17:13:36 +02:00
Oleksii Kliukin 8bed553d30 Use new-style classes. 2015-08-04 16:52:03 +02:00
Oleksii Kliukin e69f015369 switch to using WAL-E by default. 2015-08-04 16:22:46 +02:00
Oleksii Kliukin 67da1b3a78 Make parse_connstring staticmethod less 'selfish' 2015-08-04 15:26:12 +02:00
Oleksii Kliukin f17b651107 Adoprt WAL-E related code for a stand-alone script. 
- make sure all parameters are taken from the env variables.
- make sure WAL-E code is not involved if the image is not configured for it
- pick up pg_basebackup as an alternative if WAL-E fails.

The WAL-E support will be switched on in subsequent commits.
 2015-08-04 14:10:11 +02:00
Oleksii Kliukin fd884a53a9 Multiple small fixes: 
- Do delete trigger file in the patroni itself.
- Fix the typo in the datadir member.
- Add #! at the top of the python script.
 2015-08-03 15:51:40 +02:00
Oleksii Kliukin 2089a123c2 fix line too long warning. 2015-08-03 11:48:02 +02:00
Oleksii Kliukin 7f2dcc0a95 Move replica creation code into a separte file. So far, only the class that makes replicas with pg_basebackup is called. 2015-08-03 11:44:11 +02:00
Oleksii Kliukin a0fdd6398f Cosmetic changes: avoid magic strings in action names, name the parameter passed to the callback 'scope' for consistency. 2015-07-30 10:17:17 +02:00
Oleksii Kliukin 71a1200f64 Fix a mistake in checking for the number of parameters. 2015-07-29 12:09:17 +02:00
Oleksii Kliukin 5171964758 Merge branch 'master' into feature/callbacks 2015-07-27 16:19:03 +02:00
Oleksii Kliukin 5c5bae69ea make sure we can import from scripts. 2015-07-27 16:11:58 +02:00
Oleksii Kliukin 27bda5b4e4 document the new callbacks option. 2015-07-27 13:12:08 +02:00
Oleksii Kliukin 7d1bc32ad9 potentially run the doctests for the scripts. 2015-07-27 13:06:44 +02:00
Oleksii Kliukin d85451efae Increase test coverage for aws script. 2015-07-27 13:04:45 +02:00
Oleksii Kliukin b92bfb34a5 Futher test for postgresql module (includes callbacks) 2015-07-27 13:04:01 +02:00
Oleksii Kliukin 56983d867d Run coverage test for the data in scripts as well as helpers. 2015-07-27 13:03:12 +02:00
Oleksii Kliukin 115ba1d9ae Make sure postgresql.stop also return True if pg_ctl stop returned 0. 2015-07-27 11:23:08 +02:00
Oleksii Kliukin 201f9bf4b2 Make sure role check during callbacks does not accidentially modify the cluster state (is_promoted flag or trigger file). 2015-07-27 11:08:19 +02:00
Oleksii Kliukin 859da0db0a Make unit tets run again. 2015-07-27 10:10:29 +02:00
Oleksii Kliukin dac3d7997f Change a typo. 2015-07-24 10:47:26 +02:00
Oleksii Kliukin 35fe536e8b Set +x bit on aws.py 2015-07-23 17:09:39 +02:00
Oleksii Kliukin c9c678fb5b Pass scope, the cluster name, instead of the name, which is the instance name, to the callback. 2015-07-23 16:16:22 +02:00
Oleksii Kliukin f1d76611b6 Use the executable bit on AWS callback. 2015-07-17 17:26:52 +02:00
Oleksii Kliukin e9934be9aa Do not terminate on callback failure. 
Show the error if a callback command fails and move on.
Set the executable bit for the aws callback.
 2015-07-17 17:26:28 +02:00
Oleksii Kliukin a11d94aff5 Pass text names of cluster roles to the script. 
Seems better than passing a bool flag and
converting it from/to string.
 2015-07-17 15:39:34 +02:00
Oleksii Kliukin 33b5e1b22e Make sure is_leader is False does not trigger the leader re-check. 2015-07-17 15:22:21 +02:00
Oleksii Kliukin 11004f8c48 Be more robust on callbacks when cluster is down. 
For 'on_stop' or 'on_restart' callbacks we cannot
call is_leader, because the cluster would be stopped.

Instead, call is_leader before running a callback.
In addition, make the call_nowait more robust by handling
the exceptions related to the DB cluster unavailability.
 2015-07-17 11:19:05 +02:00
Oleksii Kliukin d8aa1ff849 fix a typo. 2015-07-14 09:22:49 +02:00
Oleksii Kliukin 15f94f4224 Use actual role name and not boolean flag if it is a master to tag AWS objects (EBS volumes and EC2 instances). 2015-07-13 11:47:55 +02:00
Oleksii Kliukin f734272f3f Process the command paths relative to the patroni dir correctly, fix the error of not supplying a callback action to the actual command. 2015-07-13 11:45:25 +02:00
Oleksii Kliukin 6d0a9bbdc0 create a new directory for callback scripts and move aws volume tags there. 2015-07-13 10:25:42 +02:00
Oleksii Kliukin c0cc590938 Remove usage of AWS.py from Patroni 
Functionality provided by that module will be achieved via callbacks.
- fix a typo in aws.py
- fix some mixups of old code and new callbacks in postgresql.py
 2015-07-10 12:58:37 +02:00
Oleksii Kliukin 78f0b16543 Enable aws.py to run as a stand-alone program. 2015-07-10 10:57:24 +02:00
Oleksii Kliukin 2c1627c2f3 get rid of yaml module, since AWS instance-identity natively sends JSON. 2015-07-10 10:33:07 +02:00
Oleksii Kliukin aa5a72fb37 Add on_start, on_stop, on_role_change, on_restart and on_reload callbacks. 2015-07-09 15:45:10 +02:00
Oleksii Kliukin 1fc3582e9b Merge branch 'feature/awstags' of https://github.com/zalando/governor into feature/awstags 2015-06-05 17:07:20 +02:00
Oleksii Kliukin 16a068f795 use instance-identity/document URL part to get instanceid and region at once. 2015-06-05 17:07:10 +02:00
Oleksii Kliukin a6fcd66819 line up dependency alphabetically. 2015-06-05 10:07:24 +02:00
Oleksii Kliukin 8ab7717f13 increase test coverage for the changes in postgresql.py 2015-06-05 10:03:42 +02:00
Oleksii Kliukin cdc7526216 Merge branch 'master' into feature/awstags 2015-06-05 09:50:17 +02:00
Oleksii Kliukin 20c785b63a python3 compatibility: use isinstance instead of type() 2015-06-05 09:44:47 +02:00
Oleksii Kliukin 553b26f385 call on_change_callback only if it has been set. 2015-06-05 09:01:42 +02:00
Oleksii Kliukin e2de4c9045 add ok flag to the MockResponse object in order to be compatible with the AWS code, which examines this flag in a callback. 2015-06-05 08:59:32 +02:00
Oleksii Kliukin 1ae19f758b update requirements.txt with boto. 2015-06-05 08:48:57 +02:00
Oleksii Kliukin 3d123b41df add unittests. 2015-06-04 17:19:33 +02:00
Oleksii Kliukin ac46600816 more bugfixes. 2015-06-03 16:26:38 +02:00
Oleksii Kliukin fb3bf799e9 Autodetect the host role during start from the existence of recovery.conf. Fix some typos. 2015-06-03 16:18:47 +02:00
Oleksii Kliukin e06cd49deb autodetect the host role during start from the existence of recovery.conf. 2015-06-03 16:13:03 +02:00
Oleksii Kliukin 1d3a946ffb Fix wrong number of arguments for the EBS instance. 2015-06-03 15:52:35 +02:00
Oleksii Kliukin c68beda75c make sure on role change is called during the initial start of the server. 2015-06-03 15:49:38 +02:00