gitops/patroni
1
0
Fork 0
mirror of https://github.com/outbackdingo/patroni.git synced 2026-08-25 14:53:37 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,318 Commits 15 Branches 66 Tags
0b1bfeca5b8aaf36bcf1e3f22bd65eb545bab3b8
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexander KukushkinandGitHub 0b1bfeca5b Make sure that we are running and testing latest versions of everything (#303) 2016-09-19 13:32:53 +02:00
Alexander KukushkinandGitHub 5c8399e4fa Make sure data directory is empty before trying to restore backup (#307) 
We are doing number of attempts when trying to initialize replica using
different methods. Any of this attemp may create and put something into
data directory, what causes next attempts fail.
In addition to that improve logging when creating replica.
 2016-09-19 13:32:27 +02:00
Alexander KukushkinandGitHub 540ee2b3c7 Bugfix/fast recover (#300) 
* reap children before and after running HA loop

When the Patroni is running in a docker container with the pid=1 it is
also responsible for reaping of all dead processes. We can't call
os.waitpid immediately after receiving SIGCHLD because it breaks
subprocess module. It simply stops receiving exit codes of the processes
it executes because these processes. That's why we just registering the
fact of receiving SIGCHLD and reaping children only after execution of
HA loop.
If the postmaster was dying for some reason, Patroni was able to detect
this fact only on the next iteration of HA loop, because zombie
processes where still there and it was possible to send 0 signal to it.
To avoid such situation we should also reap all dead processes before
executing HA loop.

* Don't rely on _cursor_holder when closing connection

it could happen that connection has been opened but not cursor...

* Don't "retry" when fetching current xlog location and it fails

On every iteration of HA loop we are updaing member key in DCS and among
other data there is current xlog location stored in the value.
If the postgres has died for some reason it is not possible to fetch
xlog position and we are just wasting retry_timeout/2 = 5 seconds there.
If this information will be missing from DCS during period of one HA
loop nothing should break. Patroni is not relying on this information
anyway. When it is doing manual or automatic failover it aways
communicates with other nodes directly to get the most fresh
infomation.

* Don't try to update leader optime when postgres is not 100% healthy

`update_lock` method is not only doing update of the leader lock but
also writes the most recent value of xlog position into optime/leader
key. If you know that postgres can be not 100% healthy because it is in
process of restart or recover we should not try to fetch current xlog
position and update 'optime/leader'. Previously we were using
`AsyncExecutor.busy` property for avoiding of such action, but I think
we should be more excpilicit and do the update only if we know that
postgres is 100% healty.
 2016-09-14 15:13:01 +02:00
Alexander KukushkinandGitHub 41d1724705 Merge pull request #274 from zalando/feature/disable-automatic-failover 
Feature/disable automatic failover
 2016-09-07 14:48:19 +02:00
Alexander KukushkinandGitHub 53bcc5c9bb Merge pull request #290 from zalando/feature/pyinstaller 
Binary build with PyInstaller
 2016-09-05 14:52:13 +02:00
Alexander KukushkinandGitHub dff79395a9 Merge pull request #258 from alkersan/master 
Binary build with PyInstaller
 2016-09-05 13:57:02 +02:00
Alexander KukushkinandGitHub 39d16fe2f9 Merge pull request #281 from CartoDB/feature/add_custom_conf_location 
Add configuration parameter to specify a path to a custom postgresql.base.conf and disable its backup
 2016-09-05 13:53:04 +02:00
Alexander KukushkinandGitHub 19c80df442 Try to mitigate EtcdEventIndexCleared exception (#287) 
This error is send by etcd when Patroni is doing "watch" on leader key
which is never updated after creation and etcd cluster receives a lot of
updates, what cleans history of events.

Instead of doing watch on modifiedIndex + 1 we will do watch on X-Etcd-Index,
which is probably still available...
 2016-09-02 13:44:47 +02:00
Alexander KukushkinandGitHub 05bcbe90eb close connection on start instead of stop (#272) 2016-08-25 10:43:50 +02:00
Alexander KukushkinandGitHub 688736c6b1 Merge branch 'master' into feature/do_not_drop_active_slots 2016-08-24 09:47:12 +02:00
Alexander KukushkinandGitHub 96da6340a9 Calculate future restart time dynamically (#268) 
`do_POST_restart` was ramdomly showing not 100% coverage after 2016-08-20 due to hardcoded timestamps.
 2016-08-24 09:46:56 +02:00
Alexander KukushkinandGitHub ef0b3c2296 Bring all configs to the new format (#265) 
The v1.0 has been released more than one month ago and the new version
is coming. It doesn't make a lot of sense to keep configuration files in
the old format anymore.
In addition to that I've also commented out all the lines enabling and
configuring "archiving" to avoid incidents like here:
https://github.com/zalando/patroni/issues/264
 2016-08-23 11:46:16 +02:00
Alexander KukushkinandGitHub fa7aa71092 Always call on_start callback when starting Patroni (#262) 
When Patroni was "joining" already running postgres it was not calling
callbacks, what in some cases causing issues (callback could be used to
change routing/load-balancer or assign/remove floating (service) ip.

In addition to that we should `start` postgres instead of `restart`-ing
it when doing recovery, because in this case 'on_start' callback should
be called, instead of 'on_restart'
 2016-08-18 09:35:13 +02:00
Alexander KukushkinandGitHub 8ef7178ddf Refactor code dealing with database connection string/params (#255) 
In the original code we were parsing/deparsing url-style connection
strings back and forth. That was not really resource greedy but rather
annoying. Also it was not really obvious how to switch all local
connections to unix-sockets (preferably).

This commit isolates different use-cases of working with connection
strings and minimizes amount of code parsing and deparsing them. Also it
introduces one new helper method in the `Member` object - `conn_kwargs`.
This method can accept as a parameter dict object with credentials
(username and password). As a result it returns dict object which could
be used by `psycopg2.connect` or for building connection urls for
pg_rewind, pg_basebackup or some other replica creation methods.

Params for local connection are builded in the `_local_connect_kwargs`
method and could be changed to unix-socket later easily.
 2016-08-10 10:19:52 +02:00
Alexander KukushkinandGitHub 413a84836b Update etcd topology only after original request succeed (#254) 
There is no point to try to update topology until original request is
not performed. Also for us it is more important to execute original
request rather then keep topology of etcd cluster in sync.

In addition to that implement the same retry-timeout logic in the
`machines` property which already is used in `api_execute` method.
 2016-08-10 10:17:37 +02:00
Alexander KukushkinandGitHub 702ab261a2 Use pgkutil to find dcs modules (#253) 2016-08-10 10:15:55 +02:00
Alexander KukushkinandGitHub 5fe74bec3b Make different kazoo timeouts depend on loop_wait (#243) 
* Make different kazoo timeouts dependant on loop_wait

ping timeout ~ 1/2 * loop_wait
connect_timeout ~ 1/2 * loop_wait

Originally these values were calculated from negotiated session timeout
and didn't worked very well, because it was taking significant time to
figure out that connection is dead and reconnect (up to session timeout)
and not giving us time to retry.

* Address the code review
 2016-08-10 10:15:09 +02:00
Alexander KukushkinandGitHub 13a9198013 Merge pull request #249 from zalando/travis/python 
Travis changed python version to 3.5.2
 2016-08-01 12:19:27 +02:00
Alexander KukushkinandGitHub 58ad8399cf Merge pull request #238 from zalando/bugfix/pg_rewind 
One more try to fix pg_rewind
 2016-07-05 16:25:03 +02:00
Alexander KukushkinandGitHub a19dbfaddf Merge pull request #232 from zalando/bugfix/pg_rewind 
Start readonly when holding leader lock
 2016-07-04 13:11:35 +02:00
Alexander KukushkinandGitHub 40d48eae76 Merge pull request #237 from zalando/bugfix/etcd-retry 
Calculate numer of retries and timeout
 2016-07-04 13:11:25 +02:00
Alexander KukushkinandGitHub f9298d30ca Merge pull request #231 from zalando/bugfix/etcd-retry 
Fix retry logic in etcd.py
 2016-07-04 10:37:55 +02:00
Alexander KukushkinandGitHub e025f335dc Merge pull request #233 from zalando/bugfix/initialize 
Initialize key can be present but empty
 2016-07-04 09:48:09 +02:00
Alexander KukushkinandGitHub a6d840d6be Merge pull request #234 from zalando/feature/zookeeper-connect-strategy 
Implement different connect strategy for zookeeper
 2016-07-04 09:47:55 +02:00
Alexander KukushkinandGitHub cd752fea14 Merge pull request #235 from zalando/bugfix/docker-volumes 
Volume could be passed to the docker only with absolute path
 2016-07-04 09:47:35 +02:00
Alexander KukushkinandGitHub 3f1d06da32 Merge pull request #236 from zalando/bugfix/strtol 
bugfix: strtol didn't worked correctly with 1 digit numbers
 2016-07-04 09:47:09 +02:00
Alexander KukushkinandGitHub dc27a30800 Merge pull request #230 from zalando/bugfix/pg_rewind 
Try to cover as much as possible pg_rewind corner-cases
 2016-06-30 12:09:10 +02:00
Alexander KukushkinandGitHub 489ed20331 Merge pull request #229 from zalando/bugfix/env-zookeeper-hosts 
Document that every single zookeeper host:port MUST be quoted
 2016-06-30 12:09:03 +02:00
Alexander KukushkinandGitHub 0c6a1307da Merge pull request #228 from zalando/bugfix/always-expose-role 
bugfix: api must report role=master during pg_ctl stop
 2016-06-28 15:24:16 +02:00
Alexander KukushkinandGitHub cae3b8ec1e Merge pull request #226 from zalando/bugfix/sigterm 
Setup signal handler before creating dcs
 2016-06-28 15:24:05 +02:00
Alexander KukushkinandGitHub 6fdd01a8eb Merge pull request #225 from zalando/feature/confd 
sample haproxy templates for confd
 2016-06-28 15:23:54 +02:00
Alexander KukushkinandGitHub fe0bc2f3eb Merge pull request #223 from zalando/feature/patronictl-at 
Acceptance tests for patronictl
 2016-06-28 15:23:44 +02:00
Alexander KukushkinandGitHub 26b475753e Merge pull request #222 from zalando/bugfix/sigterm 
SystemExit exception was swallowed in in thread
 2016-06-17 11:06:15 +02:00
Alexander KukushkinandGitHub 8aa6e6050a Merge pull request #219 from zalando/bugfix/docker_fixe 
Bugfix/docker fixe
 2016-06-17 11:06:01 +02:00
Alexander KukushkinandGitHub 17f317665f Merge pull request #221 from zalando/feature/patronictl-auth 
patronictl will send authorization header if it is configured
 2016-06-16 12:57:14 +02:00
Alexander KukushkinandGitHub 010a2961cb Merge pull request #220 from zalando/feature/patronictl-newconf 
Feature/patronictl newconf
 2016-06-16 12:56:47 +02:00
Alexander KukushkinandGitHub e29bfaf5cc Merge pull request #218 from zalando/feature/do-not-expose-repl-passwd 
Don't expose replication user/passwd in DCS
 2016-06-16 12:56:35 +02:00
Alexander KukushkinandGitHub dc6ae7a380 Merge pull request #216 from zalando/feature/dynamic-configuration-docs 
Update dynamic_configuration.rst
 2016-06-14 10:10:40 +02:00
Alexander KukushkinandGitHub 0a32005cc8 Merge pull request #202 from zalando/feature/dynamic-configuration 
Feature/dynamic configuration
 2016-06-14 10:10:26 +02:00
Alexander KukushkinandGitHub f2980b13fb Merge pull request #211 from zalando/feature/environment-configuration 
Implement possibility to configure Patroni via environment
 2016-06-14 10:10:09 +02:00
Alexander KukushkinandGitHub 8d38887ca1 Update dynamic_configuration.rst 2016-06-13 14:20:20 +02:00
Alexander Kukushkin c51039a5e3 Merge pull request #210 from misja/dbname 
make dbname and config file configurable
 2016-06-09 10:14:28 +02:00
Alexander Kukushkin 96bb2312cb Merge pull request #212 from zalando/feature/split-exhibitor 
Split ZooKeeper and Exhibitor
 2016-06-09 10:14:20 +02:00
Alexander Kukushkin 5372604170 Merge pull request #213 from zalando/feature/pg_ctl_status 
Reimplement pg_ctl status in python
 2016-06-09 10:14:13 +02:00
Alexander Kukushkin b65dc9a827 Update ENVIRONMENT.rst 2016-06-08 10:29:37 +02:00
Alexander Kukushkin 10e4e8354b Merge pull request #204 from zalando/bugfix/password-visibility 
Remove password from backup connection string
 2016-06-06 10:49:29 +02:00
Alexander Kukushkin 789ce297e9 Merge pull request #206 from zalando/bugfix/demote-follow 
BUGFIX: demote is already running in a thread
 2016-06-06 10:49:19 +02:00
Alexander Kukushkin 619699269b Merge pull request #203 from zalando/bugfix/pg_rewind 
Bugfix: pg_rewind can work only with master
 2016-05-30 11:01:49 +02:00
Alexander Kukushkin 082b6f818a Merge pull request #201 from zalando/feature/dcs-package 
Find and load dcs class implementations dynamically
 2016-05-19 14:25:06 +02:00
Alexander Kukushkin 73cd9951af Merge pull request #200 from zalando/feature/tests-refactoring 
Mininize number of references to dcs implementations from tests
 2016-05-19 10:38:17 +02:00
Alexander Kukushkin 6a9fb4fcec Merge pull request #196 from zalando/feature/write-options-to-postgresql.conf 
Write postgres options to postgresql.conf (except listen_addresses and port)
 2016-05-13 13:29:23 +02:00
Alexander Kukushkin 8a60c915cf Merge pull request #195 from zalando/bugfix/do-not-remove-data 
Remove data directory only if replica creation failed
 2016-05-11 12:12:41 +02:00
Alexander Kukushkin 14527c69d1 Merge pull request #194 from zalando/feature/api-request-handler-finish 
finish method does not have any arguments
 2016-05-10 13:10:32 +02:00
Alexander Kukushkin 5f33cf32f4 Merge pull request #193 from zalando/feature/primary_conninfo-application_name 
Set application_name to node name in primary_conninfo
 2016-05-10 13:10:19 +02:00
Alexander Kukushkin a615339b48 Merge pull request #192 from zalando/feature/api-sysid-no-errors 
Reset _sysid and don't call pg_controldata when restore of backup in progress
 2016-05-10 13:10:07 +02:00
Alexander Kukushkin 1a0aefeb63 Merge pull request #191 from zalando/feature/api-tests-request-encode 
Encode request body only once in a MockRequest
 2016-05-10 13:09:31 +02:00
Alexander Kukushkin ec7779fed6 Merge pull request #188 from zalando/bugfix/python351 
urllib3.exceptions.HTTPError fixes for python 3.5.1
 2016-04-26 11:55:19 +02:00
Alexander Kukushkin 0c6d259a0a Merge pull request #187 from zalando/feature/zookeeper-rename-variable 
Rename me->member (variable name was too confusing)
 2016-04-22 16:10:32 +02:00
Alexander Kukushkin e275a34f6a Merge pull request #183 from zalando/feature/api-refactoring 
Refactor api.py
 2016-04-22 13:47:14 +02:00
Alexander Kukushkin 3e32c226c0 Merge pull request #185 from zalando/feature/noloadbalance 
Implement noloadbalance support
 2016-04-22 10:31:20 +02:00
Alexander Kukushkin 578bd606cc Merge pull request #179 from zalando/bugfix/recovery-conf 
Append values from postgresql.recovery_conf to recovery.conf
 2016-04-20 16:45:56 +02:00
Alexander Kukushkin 2ba1b5e206 Merge pull request #171 from zalando/feature/pg_rewind_unconditionally 
Remove restrictions on running pg_rewind.
 2016-04-20 11:22:39 +02:00
Alexander Kukushkin 56f9366aa5 Merge pull request #176 from zalando/feature/get_dcs 
Refactor patroni/ctl.py
 2016-04-20 10:29:05 +02:00
Alexander Kukushkin bd27669bec Merge pull request #173 from zalando/feature/travis-build-matrix 
Disable sudo, reshuffle travis tasks and introduce caching
 2016-04-15 09:35:04 +02:00
Alexander Kukushkin c6cc731bf0 Merge pull request #166 from zalando/feature/clonefrom 
Correct implementation of 'clonefrom' feature
 2016-04-10 10:33:18 +02:00
Alexander Kukushkin 2c3f38f522 Merge pull request #170 from zalando/bugfix/patronictl-output_members 
In some cases output_members function was showing cluster=None
 2016-04-10 10:32:59 +02:00
Alexander Kukushkin 836cf3a1b8 Merge pull request #169 from jankeirse/master 
Add KillMode to avoid issues when restarting
 2016-04-05 17:04:09 +02:00
Alexander Kukushkin 727021306d Merge pull request #164 from zalando/feature/fast-reattach-former-leader 
Speed up reattach of former leader to the cluster
 2016-03-31 21:57:24 +02:00
Alexander Kukushkin e76f820cb5 Merge pull request #165 from zalando/feature/abc-metaclass 
Set metaclass of AbstractDCS with `six` to be python 2&3 compatible
 2016-03-31 21:57:12 +02:00
Alexander Kukushkin 454c9dcb33 Merge pull request #163 from zalando/bugfix/zookeeper 
Bugfix: In some cases Ha.cluster can be None after calling `get_cluster`
 2016-03-24 12:36:25 +01:00
Alexander Kukushkin 786dbcca24 Merge pull request #161 from zalando/feature/remove-unused-code 
Remove unused code from unit tests
 2016-03-24 08:57:34 +01:00
Alexander Kukushkin 6c0c1b1341 Merge pull request #160 from zalando/feature/failover-candidate 
Rename ambiguous `Failover.member` to candidate
 2016-03-21 18:00:08 +01:00
Alexander Kukushkin ba1dc49294 Merge pull request #159 from zalando/bugfix/scheduled-failover 
Ha loop was writing into scheduled_at during "manual_failover"
 2016-03-18 13:38:57 +01:00
Alexander Kukushkin 92394598c7 Merge pull request #156 from zalando/feature/delete-cluster-iface 
Implement `delete_cluster` interface in for all available dcs
 2016-03-18 08:54:09 +01:00
Alexander Kukushkin 780c2bc594 Merge pull request #155 from zalando/bugfix/python-etcd-0.4.3 
Migrate to python-etcd 0.4.3
 2016-03-14 10:56:19 +01:00
Alexander Kukushkin 70742e6aba Merge pull request #139 from zalando/codequality 
remove unused code
 2016-02-23 13:24:53 +01:00
Alexander Kukushkin 61feb47da8 Merge pull request #137 from LappleApple/patch-2 
Rename LICENCE to LICENSE
 2016-02-17 13:33:05 +01:00
Alexander Kukushkin f131961d8b Merge pull request #136 from zalando/codequality 
Improve quality of code by resolving issues found by quantifiedcode and codacy
 2016-02-17 09:26:21 +01:00
Alexander Kukushkin 50512fef45 Merge pull request #132 from zalando/bugfix/superuser_username 
Create superuser with the name specified in config file
 2016-02-10 14:10:28 +01:00
Alexander Kukushkin 57f19fb149 Merge pull request #80 from zalando/feature/nofailover 
Feature/nofailover
 2015-11-16 10:21:56 +01:00
Alexander Kukushkin 96dcb54429 Merge pull request #96 from zalando/bugfix/haproxy_errors 
Handle haproxy requests. Improve failover status code.
 2015-11-16 10:19:19 +01:00
Alexander Kukushkin 490a54f922 Merge pull request #94 from zalando/feature/initdb 
Make it possible to specify custom options for initdb
 2015-11-16 10:17:28 +01:00
Alexander Kukushkin 7d59a757fb Merge pull request #93 from zalando/bugfix/parameters 
Do not fail when there is no postgresql/parameters section in a config file
 2015-11-16 10:13:52 +01:00
Alexander Kukushkin 5290792dff Merge pull request #92 from zalando/feature/api 
Call touch_member right after demote and promote.
 2015-11-16 10:13:24 +01:00
Alexander Kukushkin 2e5efc6cc9 Merge pull request #75 from zalando/feature/custom-namespace 
Make it possible to override default namespace (/service/) from a config file
 2015-10-23 16:21:19 +02:00
Alexander Kukushkin 71275fc303 Merge pull request #67 from zalando/feature/api 
Manual failover via REST API
 2015-10-21 10:55:28 +02:00
Alexander Kukushkin 57ace2009c Merge pull request #66 from zalando/fix/tests 
Fix/tests
 2015-10-16 10:30:32 +02:00
Alexander Kukushkin 2c863ce9a6 Merge pull request #56 from zalando/feature/manual-failover 
Feature: manual failover
 2015-10-09 16:23:45 +02:00
Alexander Kukushkin 2fdf02e0be Merge pull request #55 from pgexperts/scripts/init-scripts 
Added Upstart job for running patroni under Ubuntu 14.04 or 12.04.
 2015-09-30 16:39:07 +02:00
Alexander Kukushkin 83c5416c82 Merge pull request #45 from zalando/feature/query-retry 
Refactor Postgresql.query method to use common retry mechanism
 2015-09-22 16:53:31 +02:00
Alexander Kukushkin 8a0af6985f Merge pull request #35 from zalando/feature/basic-auth 
Basic-auth and SSL support
 2015-09-17 16:22:23 +02:00
Alexander Kukushkin 918ac6a87d Merge pull request #39 from zalando/feature/reschedule-next-run 
Feature/reschedule next run of ha cycle depending on return value of `watch`
 2015-09-17 16:22:12 +02:00
Alexander Kukushkin b9ccd411ed Merge pull request #19 from zalando/feature/faster_shutdown 
Feature/faster shutdown
 2015-09-17 16:21:59 +02:00
Alexander Kukushkin a8305079c3 Merge pull request #38 from zalando/bugfix/zookeeper 
Bugfix/zookeeper
 2015-09-15 09:29:18 +02:00
Alexander Kukushkin 30a7d50a56 Merge pull request #34 from zalando/feature/constants-for-key-names 
Define initialize, leader, optime and members string constansts in Ab…
 2015-09-10 15:33:08 +02:00
Alexander Kukushkin c409ee4d37 Fix formatting in a README.rst 2015-09-09 12:39:48 +02:00
Alexander Kukushkin e90b14cd3b Merge pull request #31 from zalando/bugfix/script_paths 
Fix path to scripts subdirectory in configuration files.
 2015-09-08 16:25:52 +02:00
Alexander Kukushkin 9894a7ce2a Merge pull request #20 from zalando/feature/retry 
Retry requests to configuration store
 2015-09-04 16:24:43 +02:00
Alexander Kukushkin df06860bde Merge pull request #23 from pgexperts/93compat 
93compat
 2015-09-04 08:14:13 +02:00
Alexander Kukushkin ead0b7444f Merge pull request #10 from zalando/feature/python-etcd 
python-etcd
 2015-09-01 10:04:20 +02:00