Commit Graph
134 Commits
Author SHA1 Message Date
Alexander Kukushkin 8a0af6985f Merge pull request #35 from zalando/feature/basic-auth
Basic-auth and SSL support
2015-09-17 16:22:23 +02:00
Alexander Kukushkin 918ac6a87d Merge pull request #39 from zalando/feature/reschedule-next-run
Feature/reschedule next run of ha cycle depending on return value of `watch`
2015-09-17 16:22:12 +02:00
Alexander Kukushkin 0b753d25e1 Get rid from is_promoted flag. use role == 'master' instead 2015-09-17 13:57:29 +02:00
Alexander Kukushkin dbcc5aff9b Track postgresql role in a Postgresql class 2015-09-16 16:22:10 +02:00
Alexander Kukushkin 7f8e95b334 Next run of ha cycle is rescheduled depending on return value of watch
Current etcd implementation does not yet support timeout option when
`wait=true`: https://github.com/coreos/etcd/issues/2468

Originaly I've implemented `watch` method for `Etcd` class in a
following manner: if the leader key was updated just because master
needs to update ttl and watch timeout is not yet expired, I was
recalculating timeout and starting `watch` call once again.
Usually after "restart" we were getting urllib3.exceptions.TimeoutError.
The only possible way to recover after such exception - close socket and
establish a new connection. With pure http it's relatively cheap, but
with https and some kind of authorization on etcd side it would became
rather expensive and should be avoided.
2015-09-16 10:38:34 +02:00
Alexander Kukushkin 01efcc8638 Merge branch 'master' of github.com:zalando/patroni into feature/basic-auth
Conflicts:
	tests/test_patroni.py
2015-09-15 09:33:18 +02:00
Alexander Kukushkin 90cfcf0c14 Make zookeeper module compatible with python3 2015-09-14 17:14:39 +02:00
Oleksii Kliukin d69403ab6f Merge pull request #37 from zalando/feature/zookeeper-fetch-initialize
Build Cluster object for ZooKeeper the same way as for Etcd
2015-09-14 12:55:10 +02:00
Oleksii Kliukin 51eacc5042 Handle the case when initialize flag is not set and leader is present. 2015-09-14 12:36:28 +02:00
Alexander Kukushkin 209c985420 get_node and get_children should catch only NoNodeError exception.
All other exceptions are needed to have retry functionality working
correctly.
2015-09-14 11:45:00 +02:00
Alexander Kukushkin f494d2ce64 Build Cluster object for ZooKeeper the same way as for Etcd
Previous implementation was always setting Cluster.initialize to True.
Also it was throwing ZooKeeperError when there were no members in a
cluster.

Plus BUGFIX of a bug introduced with
https://github.com/zalando/patroni/pull/34 in a `load_members` method.
- data = self.get_node(self.member_path)
+ data = self.get_node(self.members_path + member)
It was always fetching the same node for all cluster members.
Fortunately Etcd doesn't have such problem because we are fetching the
whole cluster directory with one recursive API call.
2015-09-14 11:19:46 +02:00
Oleksii Kliukin cd312de252 Fix a flake8 warning.
Improve some unit tests by expecting specific exceptions.
2015-09-10 17:15:43 +02:00
Alexander Kukushkin f292398e94 Merge branches with ssl and basic-auth support together 2015-09-10 16:49:55 +02:00
Oleksii Kliukin 2377c417e4 Fix etcd and zookeper interactions with initialize key.
Fix unittests as well.
2015-09-10 16:05:10 +02:00
Oleksii Kliukin 938b946e55 Merge branch 'master' into feature/cleanup_on_failed_initialization 2015-09-10 15:43:31 +02:00
Alexander Kukushkin 5d304e95ed Merge branch 'master' of github.com:zalando/patroni into feature/api-https 2015-09-10 15:34:43 +02:00
Alexander Kukushkin d830e8de7c Merge branch 'master' of github.com:zalando/patroni into feature/basic-auth 2015-09-10 15:33:31 +02:00
Alexander Kukushkin abcaf2b94a Possibility to protect some endpoints with basic-auth
user:passwd pair should be configured in restapi section of main
configuration file in following format:
restapi:
  auth: 'username:password'

Plus implemented some simple routing mechanisms:
GET /foo => do_GET_foo()
POST /bar => do_POST_bar()
2015-09-10 15:25:08 +02:00
Alexander Kukushkin 36cbd34ffc Fix zookeeper test coverage 2015-09-09 15:59:02 +02:00
Alexander Kukushkin 5bdb18761b Define initialize, leader, optime and members string constansts in AbstractDCS
Also define following properties:
* initialize_path
* members_path
* member_path
* leader_path
* leader_optime_path

And replace any occurrences of these strings or client_path calls in a
etcd and zookeeper implementations with given constants and properties.
2015-09-09 15:10:45 +02:00
Alexander Kukushkin b5a5ea2a75 Add SSL wrapper around restapi server socket
If config['restapi']['certfile'] is specified and not empty http server
would be wrapped into SSL and api connection string changed accordingly:
http:// => https://
2015-09-09 12:15:52 +02:00
Oleksii Kliukin ff499604f0 Act on removal of initialization flag.
If initializer node suddenly dies before the initialization is complete,
other nodes should try to take over.

Fix some unittests for etcd and zookeeper and add couple of new ones.
2015-09-08 16:04:54 +02:00
Oleksii Kliukin 92647b7aad Merge branch 'master' of https://github.com/zalando/patroni into feature/cleanup_on_failed_initialization 2015-09-08 14:54:52 +02:00
Oleksii Kliukin b842ed478b Make sure initialize flag is reset on failure.
Cleanup the initialize flag if the initializing node fails
to bootstrap its PostgreSQL database.

Rename dcs.race to initialize, since we only call it for the
initialize flag. Factored out PostgreSQL bootstrapping code
into a separate function.
2015-09-08 12:03:34 +02:00
Alexander Kukushkin 1774d6e31a Merge branches watch-leader-key and package-refactoring 2015-09-05 16:09:28 +02:00
Alexander Kukushkin 3a4a2dbb94 Merge branch 'master' of github.com:zalando/patroni into feature/watch-leader-key 2015-09-05 15:31:37 +02:00
Alexander Kukushkin ac2740eeb9 Drop unused variable server_version 2015-09-05 15:22:38 +02:00
Alexander Kukushkin 7cce02ae95 Always use pg_xlog_location_diff to calculate bytes written to xlog
Drop unused functionality (lsn_to_bytes, bytes_to_lsn)
Revert some changes.
2015-09-05 15:19:17 +02:00
Alexander Kukushkin 650e244904 Refactor directory structure in preparation for building pypi-package 2015-09-04 16:06:44 +02:00
Alexander Kukushkin c913c8ad92 Calculate xlog bytes with using lsn_to_bytes method
This method behaves differently depending on server version and will
allow to use patroni with postgres older than 9.3
2015-09-04 12:11:55 +02:00
Alexander Kukushkin 8b9e99090f Move lsn_to_bytes and bytes_to_lsn into Postgresql class
and make their behavior version specific
2015-09-04 10:41:59 +02:00
Alexander Kukushkin 9361732722 Fix pep8 formatting and unit-tests 2015-09-04 08:29:16 +02:00
Alexander Kukushkin df06860bde Merge pull request #23 from pgexperts/93compat
93compat
2015-09-04 08:14:13 +02:00
Josh Berkus f5627a498e Fixed test_postgresql.py to include use_slots. 2015-09-03 18:41:30 -07:00
Alexander Kukushkin 50420771ce Retry when executing take_leader method. 2015-09-03 16:53:15 +02:00
Alexander Kukushkin 2f6399de27 More unit-tests for etcd 2015-09-03 16:35:58 +02:00
Alexander Kukushkin 58410db9dd Implemented Retry class inspired by KazooRetry 2015-09-03 11:47:08 +02:00
Alexander Kukushkin 7456662eef Merge branch 'master' of github.com:zalando/patroni into feature/watch-leader-key 2015-09-03 10:33:26 +02:00
Alexander Kukushkin a670b598f4 Implement unit test for reap_children function 2015-09-02 16:48:40 +02:00
Alexander Kukushkin c6abf85740 Bugfix: sleep method was renamed to watch 2015-09-02 08:42:47 +02:00
Alexander Kukushkin eb85caa3bd Remove spurious print 2015-09-01 17:16:15 +02:00
Alexander Kukushkin 10c95a23e4 Rename sleep to watch in a AbstractDCS
This method suppose to watch for changes of leader key if current node
is not leader and also it could watch for changes in a members list if
current conde is the leader.
2015-09-01 09:59:37 +02:00
Alexander Kukushkin fab321c6b0 Watch for changes of leader key 2015-09-01 09:54:57 +02:00
Alexander Kukushkin 3b1efff53e Refactor Cluster object
`Cluster.leader` is not reference to `Member` anymore, but to `Leader`
`Leader` class contains field `index` (update index). This field is very
useful for watching for events which changing leader key. Also `Leader`
contains `member` field, which should reference real member.
2015-08-27 10:53:22 +02:00
Alexander Kukushkin 353fc58bee Do not inherit rest api port on fork 2015-08-26 15:14:44 +02:00
Alexander Kukushkin 420176aa75 Merge branch 'master' of github.com:zalando/patroni into feature/python-etcd 2015-08-25 10:11:29 +02:00
Alexander Kukushkin befd33555d Refactor helpers/etcd.py
Work with etcd cluster via high-level python-etcd module.
Plus change all unit tests accordingly.
2015-08-24 16:58:08 +02:00
Oleksii Kliukin 29434e24ea Allow relative paths to external scripts. 2015-08-24 16:53:57 +02:00
Feike Steenbergen b0707c8ddd Merge branch 'master' of github.com:zalando/patroni into feature/external_backup
Conflicts:
	requirements-py2.txt
2015-08-11 11:27:53 +02:00
Feike Steenbergen e51ea4c1ca Merge pull request #4 from zalando/feature/lost-dcs-data
Run is_healthiest_node against last know healthy configuration of cluster
2015-08-11 11:22:46 +02:00