Commit Graph
100 Commits
Author SHA1 Message Date
Alexander Kukushkin c49580d6a7 Rename governor into patroni 2015-07-08 10:37:35 +02:00
Alexander Kukushkin 259c3ac341 Merge branch 'multidcs' of github.com:CyberDem0n/governor into feature/multidcs 2015-07-08 08:54:02 +02:00
Alexander Kukushkin 00aa99b38f Revert restapi port to 8009 2015-07-08 08:53:51 +02:00
Alexander Kukushkin a1f11fe2fe Remove unneeded class CurrentLeaderError 2015-07-08 08:53:08 +02:00
Alexander Kukushkin 54400c6dec Merge branch 'multidcs' of github.com:CyberDem0n/governor into feature/multidcs
Conflicts:
	governor.py
	tests/test_governor.py
	tests/test_postgresql.py
2015-07-08 08:47:26 +02:00
Alexander Kukushkin 43b12af3a7 Implement possibility to work against ZooKeeper
This implementation is using the same interface (AbstractDCS) as Etcd
class. It means that there should be no problem to implement another
plugin to work agains Consul for example.
2015-07-07 12:45:14 +02:00
Alexander Kukushkin 7445590546 Keep node name in AbstractDCS class
It eliminates need to pass this name into most of the methods
2015-07-07 12:26:52 +02:00
Alexander Kukushkin c5edcb589b return result of last_operation as a string 2015-07-07 12:24:02 +02:00
Alexander Kukushkin 1531304baa return result of last_operation as a string 2015-07-07 12:23:06 +02:00
Alexander Kukushkin ae689a2f83 Rename ha.etcd into ha.dcs 2015-07-02 15:33:21 +02:00
Alexander Kukushkin cc71906009 Inherit Etcd from abstract class 2015-07-02 15:15:07 +02:00
Alexander Kukushkin 6079c8359f refactoring: preparing to support ZooKeeper
move some common classes info separate file in preparation to support distributed configuration store other then Etcd
2015-07-02 12:06:07 +02:00
Alexander Kukushkin 687fb7da10 Remove haproxy_status.sh and change haproxy.cfg to work directly with the governor 2015-06-25 14:17:51 +02:00
Alexander Kukushkin a723dbbed4 Merge branch 'master' of github.com:compose/governor 2015-06-25 14:15:52 +02:00
Alexander Kukushkin 9732106e5d BUGFIX: keep list of replication slots on slaves empty
Otherwise slaves accumulate wal files...
2015-06-25 14:01:06 +02:00
Alexander Kukushkin ffdd3b3d8e Merge branch 'master' of github.com:CyberDem0n/governor 2015-06-24 16:42:08 +02:00
Alexander Kukushkin a02f9d497f BUGFIX: it can be that /members request to etcd returns somesing strange with status_code 200 2015-06-24 16:41:37 +02:00
Alexander Kukushkin f7be52f63a real_ttl method will return -1 when expiration is not defined 2015-06-23 17:15:37 +02:00
Alexander Kukushkin b7c73fdba8 Rename hostname to name in a Member 2015-06-23 14:07:00 +02:00
Alexander Kukushkin 744026b4bb Do not update member TTL when it is far from being expired 2015-06-23 14:02:32 +02:00
Alexander Kukushkin 44e06b4cff Add expiration field into Member 2015-06-10 14:22:11 +02:00
Alexander Kukushkin 275e18e58a Refactor tests 2015-06-10 14:21:23 +02:00
Alexander Kukushkin 72b980bd97 Merge branch 'etcd-cluster-support' of github.com:CyberDem0n/governor 2015-06-10 11:58:48 +02:00
Alexander Kukushkin 971bc3b6e8 Increase verbosity of failed PUT and DELETE requests. Hope will help to debug some etcd related issues 2015-06-10 11:56:16 +02:00
Alexander Kukushkin 3f47e9e362 Merge branch 'etcd-cluster-support' of github.com:CyberDem0n/governor 2015-06-09 15:47:06 +02:00
Alexander Kukushkin a64dad267e Bugfix: target field of SRV object should be casted to string 2015-06-09 15:46:34 +02:00
Alexander Kukushkin 4fa9dbcf03 Merge branch 'etcd-cluster-support' of github.com:zalando/governor into etcd-cluster-support 2015-06-09 12:31:52 +02:00
Alexander Kukushkin de7630c65c Merge branch 'etcd-cluster-support' of github.com:CyberDem0n/governor into etcd-cluster-support 2015-06-09 12:31:09 +02:00
Alexander Kukushkin 57914821d0 Randomize list of etcd cluster members 2015-06-09 12:30:58 +02:00
Alexander Kukushkin a2988361c5 Retry with host from config when it failed to resolve SRV 2015-06-09 12:30:16 +02:00
Alexander Kukushkin 19b0d4701b Merge branch 'etcd-cluster-support' of github.com:CyberDem0n/governor into etcd-cluster-support
Conflicts:
	requirements-py2.txt
	tests/test_governor.py
2015-06-09 11:34:14 +02:00
Alexander Kukushkin 261fbf2721 Replace logging with logger in ha.py 2015-06-09 10:27:09 +02:00
Alexander Kukushkin da4f9e8445 Make logging of http requests more verbose 2015-06-09 10:25:33 +02:00
Alexander Kukushkin 5b7a60657f Fix syntax in .travis.yml 2015-06-09 10:04:07 +02:00
Alexander Kukushkin c79fba7656 Support work with etcd as a cluster
In case if one member of a cluster is not available it will retry with
another one and fetch the new cluster configuration. Default timeout for
all requests to etcd is 5 seconds.
Initial cluster configuration can be resolved through:
1) /v2/members call on one of the cluster members on a client port
2) when it is possible to resolve hostname into multiple ip's  it will
iterate through list and try to perform action from 1)
3) If there is discovery_srv defined in etcd section of config file it
will resolve peer addresses of all cluster members and will fetch
cluster configuration with using peer protocol by doing /members call on
a peer port
2015-06-09 09:51:53 +02:00
Alexander Kukushkin fad2fbeae0 Import ThreadingMixIn for python3 2015-06-03 16:39:29 +02:00
Alexander Kukushkin aa1fc315ae Process http requests in a threads with ThreadingMixIn 2015-06-03 16:22:05 +02:00
Alexander Kukushkin f9fbe8980f Implement lsn_to_bytes bytes_to_lsn functions 2015-06-02 13:36:36 +02:00
Alexander Kukushkin 5cceac7621 Merge branch 'master' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	tests/test_postgresql.py
2015-06-02 12:38:45 +02:00
Alexander Kukushkin d9fbbf3ae9 Make pg_hba section more compact 2015-06-02 12:37:26 +02:00
Alexander Kukushkin 88631e14fa Merge branch 'master' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	helpers/postgresql.py
	tests/test_postgresql.py
2015-06-02 10:28:28 +02:00
Alexander Kukushkin fca987deda set value of api_url from application_name parameter of conn_url. Reset parameters of conn_url 2015-06-02 09:45:37 +02:00
Alexander Kukushkin f9ce29d49f rename address to conn_url in a Member obj
and introduce new field: api_url
2015-06-02 08:52:34 +02:00
Alexander Kukushkin bacd05d99e Determine preferable local address to connect through.
If listen contains '*' or 0.0.0.0 - connect via localhost
In all other cases pick the first one.
2015-06-01 17:00:15 +02:00
Alexander Kukushkin efccd777b2 rename pid_path to postmaster_pid 2015-06-01 16:47:56 +02:00
Alexander Kukushkin 1bcc2b5fa6 Extend postgres?.yml with pg_hba section to give possibility to customize pg_hba.conf 2015-06-01 16:05:35 +02:00
Alexander Kukushkin ccdff5ea99 Merge branch 'master' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	helpers/postgresql.py
	tests/test_postgresql.py
2015-06-01 14:14:53 +02:00
Alexander Kukushkin adcc7ac256 Try to avoid "double" promotion.
Also check presence of trigger_file on master after promotion when
pg_is_in_recovery() = false and if it is there - remove it.
Plus check presence of trigger_file on a new slave (after running
pg_basebackup) and if it is there - also remove it.
2015-06-01 13:48:44 +02:00
Alexander Kukushkin 8c4e547d7a Handle KeyboardInterrupt in the main loop 2015-06-01 12:02:46 +02:00
Alexander Kukushkin 939254021e Code cleanup: do not modify os.environ but pass copy of it to subprocess.call 2015-06-01 11:03:06 +02:00
Alexander Kukushkin f8c3582715 Refactor api:
Call is_running only when it's not possible to execute query against
governed postgres.
2015-06-01 10:48:16 +02:00
Alexander Kukushkin b85262637d Bugfix: do not create replication slot for master 2015-06-01 09:20:24 +02:00
Alexander Kukushkin 5fcb19b817 Merge branch 'sleep-sigchld' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	tests/test_governor.py
2015-05-28 15:55:03 +02:00
Alexander Kukushkin aa486194ae Wrap time.sleep into wrapper function
In case if it was interrupted by SIGCHLD but scheduled awake time is not
reached it will continue sleeping. For all other signals behaviuor is
not changed.
2015-05-28 15:47:51 +02:00
Alexander Kukushkin 9cd7000d7f Merge branch 'master' of github.com:CyberDem0n/governor 2015-05-28 13:04:49 +02:00
Alexander Kukushkin cd5e589620 Revert previous commit (restore sigchld_handler) 2015-05-28 13:04:21 +02:00
Alexander Kukushkin f76f77e964 Ignore SIGCHLD instead of processing it.
This way time.sleep wont be interrupted when some external process is
finished and terminated childs would be reaped automatically.
2015-05-28 12:25:49 +02:00
Alexander Kukushkin 9971d214fe Merge branch 'restapi' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	governor.py
2015-05-27 14:54:08 +02:00
Alexander Kukushkin bdc4374ae3 connect address for rest api could be different from listen address 2015-05-27 14:51:41 +02:00
Alexander Kukushkin 5a817fad0d Store in etcd endpoint of restapi
In order to have backward compatibility connection url of restapi is
passed as application_name parameter in connection url of database.
Value in etcd will look like:
postgres://user:passwd@host:5432/postgres?application_name=http://host:8080/governor
Sure, this is a hack, but I guess it worth it.
2015-05-27 14:46:35 +02:00
Alexander Kukushkin 77911144c7 Improve test of governor against real config files 2015-05-27 14:44:52 +02:00
Alexander Kukushkin b163a8896b Merge branch 'restapi' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	helpers/postgresql.py
2015-05-27 11:20:51 +02:00
Alexander Kukushkin 56a9f52a15 Replace os.system with subprocess.call 2015-05-27 11:16:16 +02:00
Alexander Kukushkin 0cd81a1ecf Merge branch 'restapi' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	tests/test_postgresql.py
2015-05-27 11:15:15 +02:00
Alexander Kukushkin 9968a66048 helpers/postgresql.py
Replace os.system with subprocess.call
2015-05-27 11:08:06 +02:00
Alexander Kukushkin 56daec0d6c Api: Bugfix for current_xlog_location in slaves. 2015-05-26 15:53:57 +02:00
Alexander Kukushkin 8cc0aa90b1 Fix typos when passing connection parameters to psycopg2. 2015-05-26 10:39:16 +02:00
Alexander Kukushkin bbf1110518 Merge branch 'restapi' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	governor.py
	helpers/postgresql.py
	postgres0.yml
	postgres1.yml
	tests/test_postgresql.py
2015-05-26 09:56:35 +02:00
Alexander Kukushkin 518ef68a53 Use connect_timeout=3s and statement_timeout=2s everywhere (even for local connections) 2015-05-26 09:19:20 +02:00
Alexander Kukushkin 45c4ba85b4 port for http servert must be integer 2015-05-26 09:17:58 +02:00
Alexander Kukushkin cca5bdef8f Configure listen address for restapi via a yaml file 2015-05-26 09:07:39 +02:00
Alexander Kukushkin 3e3acc3d3a Do not start listening port during test 2015-05-26 08:59:42 +02:00
Alexander Kukushkin 7f9c10e33f Merge branch 'master' of github.com:CyberDem0n/governor into features/refactoring 2015-05-22 12:27:02 +02:00
Alexander Kukushkin b00846affc Merge branch 'master' of github.com:CyberDem0n/governor into features/refactoring
Conflicts:
	helpers/postgresql.py
2015-05-22 12:26:55 +02:00
Alexander Kukushkin 34f9b666cf Demote master when etcd is not accessible 2015-05-22 12:26:19 +02:00
Alexander Kukushkin 09163ac454 If there is no connect_address defined in yml file use address from listen as a fallback 2015-05-22 12:03:43 +02:00
Alexander Kukushkin 62d17d68de Use "member_ttl" with default value=3600 for members 2015-05-22 12:00:07 +02:00
Alexander Kukushkin 096e6e4556 Eventually member will expire in etcd 2015-05-22 11:55:20 +02:00
Alexander Kukushkin bb884bac00 Move is_unlocked method from Ha into Cluster class 2015-05-22 11:45:09 +02:00
Alexander Kukushkin 4a4d9b85e5 Make sure that there is the new line at the end of pg_hba.con 2015-05-22 11:23:11 +02:00
Alexander Kukushkin d56b8772e5 Update unit tests 2015-05-20 18:21:08 +02:00
Alexander Kukushkin 1fb7b2cbe1 Compatibility with python3 2015-05-20 18:19:23 +02:00
Alexander Kukushkin 5d9a3d46a2 Format according to pep8 2015-05-20 18:18:45 +02:00
Alexander Kukushkin 86408f8dc4 Merge branch 'master' of github.com:zalando/governor 2015-05-20 17:59:56 +02:00
Alexander Kukushkin 3533b790cb Update list of success return codes for PUT and DELETE 2015-05-20 12:03:12 +02:00
Alexander Kukushkin 8378315b6d Remove unneeded stuff from setup.py 2015-05-19 14:14:52 +02:00
Alexander Kukushkin 8d76889744 Merge branch 'master' of github.com:CyberDem0n/governor 2015-05-19 14:02:44 +02:00
Alexander Kukushkin 0d5bb48ead Close connection when querying other members of cluster 2015-05-19 14:01:30 +02:00
Alexander Kukushkin 16974ad752 do not test against 2.6 2015-05-19 13:32:35 +02:00
Alexander Kukushkin c8ee7b1c6d test against 2.6 and 3.3 2015-05-19 13:05:31 +02:00
Alexander Kukushkin 004cbf16c6 Merge branch 'master' of github.com:CyberDem0n/governor into testing 2015-05-19 13:01:15 +02:00
Alexander Kukushkin 805b3a04fc trigger rebuild 2015-05-19 12:59:37 +02:00
Alexander Kukushkin ebb8bce0f6 some simple tests 2015-05-19 12:49:53 +02:00
Alexander Kukushkin 3e8855799f Add requirements.txt 2015-05-19 09:14:13 +02:00
Alexander Kukushkin f7b6c8df40 Merge branch 'master' of github.com:CyberDem0n/governor
Conflicts:
	helpers/postgresql.py
2015-05-18 15:58:38 +02:00
Alexander Kukushkin e9ffcf9efe Small refactoring of initdb call 2015-05-18 15:57:24 +02:00
Alexander Kukushkin df855bb8de Merge branch 'master' of github.com:CyberDem0n/governor into features/refactoring 2015-05-18 14:39:39 +02:00
Alexander Kukushkin 520de12232 Remove debug print 2015-05-18 14:39:14 +02:00
Alexander Kukushkin a9b50b4e2e Merge branch 'master' of github.com:CyberDem0n/governor 2015-05-18 14:34:17 +02:00
Alexander Kukushkin ce936ceee2 Merge branch 'master' of github.com:compose/governor
Conflicts:
	governor.py
	helpers/ha.py
	helpers/postgresql.py
2015-05-18 14:33:27 +02:00