Commit Graph
79 Commits
Author SHA1 Message Date
Oleksii KliukinandGitHub 3f7fa4b41f Avoid retries when syncing replication slots. (#282)
* Avoid retries when syncing replication slots.

Do not retry postgres queries that fetch, create and drop slots at the end of
the HA cycle. The complete run_cycle routine executes with the async_executor
lock. This lock is also used with scheduling operations like reinit or restart
in different threads. Looks like CPython threading class has fairness issues
when multiple threads try to acquire the same lock and one of them executes
long-running actions while holding it: the others have little chances of
acquiring the lock in order. To get around this issue, the long action (i.e.
retrying the query) is removed.

Investigation by Ants Aasma and Alexander Kukushkin.
2016-09-02 17:00:37 +02:00
Oleksii KliukinandGitHub b0beecffea Merge pull request #276 from zalando/docs/pause_mode_description
Describe the decisions made for the pause mode.
2016-09-02 11:55:39 +02:00
Oleksii KliukinandGitHub 5de0790e89 Merge pull request #284 from zalando/feature/reload_callbacks
Callbacks should be loaded not only on init.
2016-09-02 11:55:21 +02:00
Oleksii KliukinandGitHub 46f1c5b690 Merge pull request #269 from zalando/feature/replica-info
Return replication information on the api
2016-08-31 13:58:19 +02:00
Oleksii KliukinandGitHub adbea3929c Merge pull request #277 from ants/feature/translate-slot-names
Replace invalid characters in member names for replication slot names
2016-08-31 13:56:53 +02:00
Oleksii KliukinandGitHub 030ba4c898 Address the review
- fix the false claim that Patroni in a paused mode may promote a master.
- merge all allowed API actions together.
- describe the payload of the API pause request as JSON.
2016-08-30 10:58:20 +02:00
Oleksii KliukinandGitHub 64e3e2fb2d Formatting fix 2016-08-29 18:15:16 +02:00
Oleksii KliukinandGitHub de6cba6e04 Rephrase the leader lock section, mention reinit 2016-08-29 18:13:45 +02:00
Oleksii KliukinandGitHub 2743dc0f7a Spellcheck minor stylistic fixes 2016-08-29 15:35:57 +02:00
Oleksii KliukinandGitHub b498b22584 Minor formatting fix. 2016-08-29 15:23:59 +02:00
Oleksii KliukinandGitHub ebbc8bad4c Create pause.rst 2016-08-29 15:19:18 +02:00
Oleksii KliukinandGitHub 52806bb04d Merge pull request #266 from zalando/feature/do_not_drop_active_slots
Do not drop active replication slots.
2016-08-24 11:50:57 +02:00
Oleksii KliukinandGitHub 5b9411b9da Merge pull request #252 from zalando/feature/ctl_scaffolding
Add patronictl scaffold command.
2016-08-10 12:21:12 +02:00
Oleksii KliukinandGitHub 0a2de034e8 Merge branch 'master' into feature/ctl_scaffolding 2016-08-01 16:28:55 +02:00
Oleksii KliukinandGitHub b8a69f11d9 Merge branch 'master' into travis/python 2016-08-01 12:18:28 +02:00
Oleksii KliukinandGitHub af5441eb82 Merge pull request #250 from zalando/delete_old_zappr_conf
Delete .zappr.yml
2016-08-01 12:18:11 +02:00
Oleksii KliukinandGitHub b66fad81cc Delete .zappr.yml 2016-08-01 12:16:33 +02:00
Oleksii Kliukin 6eb2e21144 Add .zappr.yaml 2016-07-18 17:03:17 +02:00
Oleksii KliukinandGitHub 4f7a0e014b Merge pull request #227 from zalando/feature/scheduled_restarts
Implement scheduled restarts for Patroni via the API. 

Even normal (immediate) restarts will take advantage of additional modifiers you can supply to the restart endpoint:

- restart_pending: restart only if the pending restart flag is set (because of the configuration change)
- role: restart if the Postgres role is set to a certain value
- postgres_version (x.y.z) - restart if the current Postgres version is less than the one specified.

For the scheduled restart, the schedule parameter can be used the same way as it is currently used for the scheduled failovers. Particularly, we don't allow restarts in the past, and always require the timezone to be present in the request in order to avoid client/server TZ difference issues.

Unify the code that evaluates the schedule for the  scheduled restarts and scheduled failovers.
Use the RLock instead of Lock in the async_executor to avoid hanging if the thread takes the lock multiple times (mostly for the with blocks in the api).
2016-07-18 13:30:18 +02:00
Oleksii KliukinandGitHub 405dbb1cbe Merge pull request #242 from zalando/feature/avoid_initial_wait_for_bootstrap
Set the initialize flag to None and not False initially.
2016-07-11 16:23:58 +02:00
Oleksii KliukinandGitHub 5ffc8206b0 Merge pull request #217 from zalando/LappleApple-patch-2
Updated README to include note to Kubernetes users
2016-06-14 15:15:53 +02:00
Oleksii KliukinandGitHub 148562786d Some editorializing and fixing of typos. 2016-06-13 15:48:23 +02:00
Oleksii KliukinandGitHub 16b8afab20 Patroni is capitalized, Postgres is used consistently. 2016-06-13 15:41:41 +02:00
Oleksii Kliukin fb29b8426a Remove the obsolete diagram. 2016-05-30 16:22:00 +02:00
Oleksii Kliukin 677bd05369 Add badges back, remove the reference to the outdated flow diagram. 2016-05-30 16:20:58 +02:00
Oleksii Kliukin 1c6113eb97 Remove the quote 2016-05-30 16:18:00 +02:00
Oleksii Kliukin 455b7951b3 Fix the description of the zookeeper and initdb. 2016-05-30 16:17:17 +02:00
Oleksii Kliukin 3508b33f8d Remove the done items 2016-05-30 15:55:55 +02:00
Oleksii Kliukin 7d7cd2b4e2 Merge pull request #207 from zalando/LappleApple-patch-1
Edited README + added new SETTINGS.rst file.
2016-05-30 15:37:32 +02:00
Oleksii Kliukin ea29760bd2 Clarify our relationship with the Governor. 2016-05-30 09:52:03 +02:00
Oleksii Kliukin 539a23041e Move some parameters around, add new special ones
- name is moved to globals, maximum_lag_on_failover to the bootstrap
- add new parameters that cannot be changed (or those the values of which cannot be lower than default) into the postgresql parameters section.
2016-05-24 14:48:00 +02:00
Oleksii Kliukin 622e110f0a More formatting fixes 2016-05-11 09:57:40 +02:00
Oleksii Kliukin 688a09396d Fix list formatting 2016-05-11 09:55:39 +02:00
Oleksii Kliukin c1140ed7a8 Change the default options to bootstrap 2016-05-11 09:53:18 +02:00
Oleksii Kliukin 708f45568b Merge pull request #168 from zalando/feature/patroni_config_from_environment
Read Patroni configuration from the environment.
2016-04-04 17:27:32 +02:00
Oleksii Kliukin f132e2b4ad Merge pull request #153 from zalando/feature/acceptance_tests_behave
Feature/acceptance tests behave
2016-03-14 15:24:52 +01:00
Oleksii Kliukin e802bba5f9 Merge pull request #151 from zalando/feature/base_backup_from_the_replica
First implementation of cloning from the replica.
2016-03-11 16:57:06 +01:00
Oleksii Kliukin 25106e97f3 Merge pull request #146 from zalando/feature/remove-unused-code
Feature/remove unused code
2016-03-11 16:47:23 +01:00
Oleksii Kliukin 18f3284bf2 Merge pull request #143 from zalando/bugfix/zookeeper-support-for-ctl
Add support for ZooKeeper/Exhibitor DCS URI in patronictl ... -d
2016-03-11 16:45:25 +01:00
Oleksii Kliukin 8b96741e8e Merge pull request #129 from zalando/feature/xlog_lag_interval
Include timestamp of last replayed location in api call.
2016-03-11 16:42:16 +01:00
Oleksii Kliukin da53452a53 Merge pull request #152 from zalando/feature/acceptance_tests_travis
Run acceptance tests on travis
2016-03-11 09:32:05 +01:00
Oleksii Kliukin d65d47b517 Merge pull request #147 from zalando/features/acceptance_test_related_fixes
Features/acceptance test related fixes
2016-03-07 11:05:54 +01:00
Oleksii Kliukin 6ebfbd241a Merge pull request #141 from jankeirse/master
sample systemd service file
2016-02-23 12:35:37 +01:00
Oleksii Kliukin cde61187c2 Merge pull request #134 from zalando/feature/scheduled_failover_squashed
Feature/scheduled failover squashed
2016-02-22 16:40:17 +01:00
Oleksii Kliukin 9d4a85e461 Merge pull request #131 from zalando/bugfix/superuser_username
Bugfix/superuser username
2016-02-10 10:48:03 +01:00
Oleksii Kliukin d426a795c3 Merge pull request #122 from zalando/feature/replica_without_the_master
Run replicas without the master
2016-02-04 21:32:17 +01:00
Oleksii Kliukin 70bae1b267 Note multiple PostgreSQL listen addresses. 2016-01-08 16:51:24 +01:00
Oleksii Kliukin 146d9a688e Merge pull request #111 from zalando/bugfix/initdb_without_options
Fix for the crash due to a missing initidb section in the configurati…
2015-12-11 10:54:27 +01:00
Oleksii Kliukin 17250a20aa Merge pull request #108 from zalando/pg_rewind_manual_failover
Call a checkpoint on master before pg_rewind.
2015-12-09 14:13:43 +01:00
Oleksii KliukinandOleksii Kliukin 28f1d51cb6 Merge pull request #103 from zalando/pgexperts-restore/movebasebackup
Replica creation methods are configured via the 'create_replica_method'
parameter in the patroni configuration file. The value should contain a list
of acceptable replica creation method names. Patroni attempts to call them one
by one, until it finds one that returns 0. If there is a configuration section
with the same name as the replica creation method, the values in this section
are converted to the command-line options and appended to the command name.
The command name is constructued either from the 'command' key in the method
name configuration section, or by taking the replica creation method name
verbatim.

The wal_e replica creation method is included with Patroni (wale_restore.py
script), in addition to the 'basebackup', which is a special method: it's taken
by default, if no create_replica_method parameter is specified, or if it is
empty. This merge also reverses the decision to move the basebackup repli
method ouside of the core patroni.

Original pull request by Josh Berkus (@jberkus), with fixes and improvements
from Alexander Kukushkin (CyberDem0n) and Oleksii Kliukin (alexeyklyukin).

Per discussion at https://github.com/zalando/patroni/issues/57
2015-11-25 15:04:23 +01:00
Oleksii Kliukin ce1ff95f4c Merge pull request #103 from zalando/pgexperts-restore/movebasebackup
Replica creation methods are configured via the 'create_replica_method'
parameter in the patroni configuration file. The value should contain a list
of acceptable replica creation method names. Patroni attempts to call them one
by one, until it finds one that returns 0. If there is a configuration section
with the same name as the replica creation method, the values in this section
are converted to the command-line options and appended to the command name.
The command name is constructued either from the 'command' key in the method
name configuration section, or by taking the replica creation method name
verbatim.

The wal_e replica creation method is included with Patroni (wale_restore.py
script), in addition to the 'basebackup', which is a special method: it's taken
by default, if no create_replica_method parameter is specified, or if it is
empty. This merge also reverses the decision to move the basebackup repli
method ouside of the core patroni.

Original pull request by Josh Berkus, with fixes and inprovements
from Alexander Kukushkin and Oleksii Kliukin.

Per discussion at https://github.com/zalando/patroni/issues/57
2015-11-25 15:00:56 +01:00
Oleksii Kliukin 6c769554a3 Add -p 1 to an example restore_command in order to limit WAL-E to only a single thread
With out tests, the multi-threaded WAL fetch didn't work reliably with S3, resulting in stuck WAL-E processes.
2015-11-23 17:42:08 +01:00
Oleksii Kliukin ec1d1c827b Merge pull request #102 from zalando/bugfix/silenceapithread
Avoid excessive log output in the API thread.
2015-11-23 15:41:50 +01:00
Oleksii Kliukin c003af294a Merge pull request #82 from zalando/feature/patroni_cli_or_ctl_tbd
Feature/patroni cli or ctl tbd
2015-11-18 16:17:02 +01:00
Oleksii Kliukin 0e7adb38fa Merge pull request #100 from zalando/bugfix/etcd_exceptions
Handle unexpected exceptions in etcd.
2015-11-18 15:08:59 +01:00
Oleksii Kliukin 9f0ee544cf Merge pull request #95 from LappleApple/patch-1
Documentation improvements by @LappleApple
2015-11-12 12:18:13 +01:00
Oleksii Kliukin 194aa92fd2 Merge pull request #89 from zalando/bugfix/cleanup
Remove the useless code line and fix a typo in the error message. Per suggestion by @antban
2015-11-10 18:18:46 +01:00
Oleksii Kliukin 5f9276f5dc Merge pull request #77 from zalando/bugfix/remove_initialize_key
Fix removal of keys on failed initialization. Fixes the recent problem reported at #17
2015-10-23 15:42:48 +02:00
Oleksii Kliukin c61ac39e8f Merge pull request #71 from zalando/feature/configurable_pgpass
Feature/configurable pgpass
2015-10-21 14:45:02 +02:00
Oleksii Kliukin 3c3694dc7c Merge pull request #73 from zalando/bugfix/update-machines-cache
Bugfix/update machines cache
2015-10-21 12:11:11 +02:00
Oleksii Kliukin e2261f531b Merge pull request #72 from drnic/override-patroni-scope
allow $PATRONI_SCOPE to be set via 'docker run -e PATRONI_SCOPE=ironman'
2015-10-21 12:10:46 +02:00
Oleksii Kliukin 2d7909e357 Merge pull request #69 from zalando/feature/clusterid
Add clusterid to patroni in order to avoid connections from already existing nodes belonging to another cluster (master - replica system).
2015-10-21 12:09:55 +02:00
Oleksii Kliukin b922b8ad0e Merge pull request #70 from zalando/bugfix/save_configuration_files
Add a missing call to restore_configuration_files.
2015-10-21 11:45:18 +02:00
Oleksii Kliukin 75e5901360 Merge pull request #63 from zalando/bugfix/leader_crash_readonly
Make sure patroni is not using stale connections.
2015-10-16 09:40:55 +02:00
Oleksii Kliukin 5747f561c0 Merge pull request #64 from zalando/bugfix/pg_controldata_changes
Work around the differences in pg_controldata names.
2015-10-14 12:14:07 +02:00
Oleksii Kliukin 7b079067b9 Merge pull request #59 from zalando/feature/pg_rewind_unclean_shutdown
Feature/pg rewind unclean shutdown. Allow pg_rewind to rewind the former master if it was uncleanly shutdown or just lost the connection and advanced past the point where the new master had promoted. Add configuration block pg_rewind with username and password as mandatory parameters inside this block.
2015-10-13 09:15:25 +02:00
Oleksii Kliukin 52c4826569 Reflect the renaming of os-registry.stups.zalan.do to registry.opensource.zalan.do 2015-10-08 12:40:21 +02:00
Oleksii Kliukin 36101974cf Merge pull request #49 from zalando/feature/bootstrap-from-main-loop
Run initial cluster bootstrap from the main loop
2015-09-24 15:04:52 +02:00
Oleksii Kliukin 75be235d49 Merge pull request #40 from zalando/feature/ignore-second-sigterm
During shutdown sigterm is sent to all processes every second
2015-09-18 17:06:02 +02:00
Oleksii Kliukin bf5737614d Merge pull request #30 from zalando/feature/cleanup_on_failed_initialization
Make sure initialize flag is reset on failure.
2015-09-14 12:57:39 +02:00
Oleksii Kliukin d69403ab6f Merge pull request #37 from zalando/feature/zookeeper-fetch-initialize
Build Cluster object for ZooKeeper the same way as for Etcd
2015-09-14 12:55:10 +02:00
Oleksii Kliukin 9d79be56df Merge pull request #8 from zalando/feature/external_backup
Move the functionality of restoring a replica from backup into an external script
2015-08-24 18:12:38 +02:00
Oleksii Kliukin 6fdd0304f6 Merge pull request #7 from zalando/feature/callbacks
Implement callback actions for patroni in order to call external script on different events in the server lifecycle, supported are on start, stop, restart, reload and role change (which covers both promote and demote). A callback should receive 3 arguments: action, role and cluster name (in this order), and can do things like tagging the EC2 instances associated with PostgreSQL, or notifying the DBA when the master had been demoted.

A sample callback that assigns tags to AWS instances is provided.
2015-07-30 10:29:08 +02:00
Oleksii Kliukin dba2ced963 Merge pull request #23 from zalando/feature/awstags
Set tags on EC2 instances and EBS storage
2015-06-08 10:04:47 +02:00
Oleksii Kliukin 9ae5998a25 Merge pull request #20 from zalando/features/refactoring
Try to avoid of double promote
2015-06-01 14:43:20 +02:00
Oleksii Kliukin 6072a8a07b Merge pull request #15 from zalando/features/refactoring
More accurate exceptions handling
2015-05-26 10:36:57 +02:00
Oleksii Kliukin eec61cbb83 Merge pull request #12 from zalando/s3_backups
S3 backups
2015-05-20 17:11:49 +02:00
Oleksii Kliukin 363b3c7122 Merge pull request #11 from CyberDem0n/master
some simple tests
2015-05-20 16:55:27 +02:00
Oleksii Kliukin fce99b6ec3 Merge pull request #4 from zalando/handle_sigchld
Handle sigchld
2015-05-13 14:58:31 +02:00