diff --git a/README.md b/README.md deleted file mode 100644 index 2582eb37..00000000 --- a/README.md +++ /dev/null @@ -1,129 +0,0 @@ -[![Build Status](https://travis-ci.org/zalando/patroni.svg?branch=master)](https://travis-ci.org/zalando/patroni) -[![Coverage Status](https://coveralls.io/repos/zalando/patroni/badge.svg?branch=master)](https://coveralls.io/r/zalando/patroni?branch=master) -# Patroni: A Template for PostgreSQL HA with ZooKeeper or etcd - -Patroni was previously known as Governor. - -*There are many ways to run high availability with PostgreSQL; here we present a template for you to create your own custom fit high availability solution using python and distributed configuration store (like ZooKeeper or etcd) for maximum accessibility.* - -## Getting Started -To get started, do the following from different terminals: - -``` -> etcd --data-dir=data/etcd -> ./patroni.py postgres0.yml -> ./patroni.py postgres1.yml -``` - -From there, you will see a high-availability cluster start up. Test -different settings in the YAML files to see how behavior changes. Kill -some of the different components to see how the system behaves. - -Add more `postgres*.yml` files to create an even larger cluster. - -We provide a haproxy configuration, which will give your application a single endpoint for connecting to the cluster's leader. To configure, run: - -``` -> haproxy -f haproxy.cfg -``` - -``` -> psql --host 127.0.0.1 --port 5000 postgres -``` - -## How Patroni works - -For a diagram of the high availability decision loop, see the included a PDF: [postgres-ha.pdf](https://github.com/zalando/patroni/blob/master/postgres-ha.pdf) - -## YAML Configuration - -For an example file, see `postgres0.yml`. Below is an explanation of settings: - -* *ttl*: the TTL to acquire the leader lock. Think of it as the length of time before automatic failover process is initiated. -* *loop_wait*: the number of seconds the loop will sleep - -* *restapi* - * *listen*: ip address + port that Patroni will listen to provide health-check information for haproxy. - * *connect_address*: ip address + port through which restapi is accessible. - -* *etcd* - * *scope*: the relative path used on etcd's http api for this deployment, thus you can run multiple HA deployments from a single etcd - * *ttl*: the TTL to acquire the leader lock. Think of it as the length of time before automatic failover process is initiated. - * *host*: the host:port for the etcd endpoint - -* *zookeeper* - * *scope*: the relative path used on etcd's http api for this deployment, thus you can run multiple HA deployments from a single etcd - * *session_timeout*: the TTL to acquire the leader lock. Think of it as the length of time before automatic failover process is initiated. - * *reconnects_timeout*: how long we should try to reconnect to ZooKeeper after connection loss. After this timeout we assume that we don't have lock anymore and will restart in read-only mode. - * *hosts*: list of ZooKeeper cluster members in format: [ 'host1:port1', 'host2:port2', 'etc...'] - * *exhibitor*: if you are running ZooKeeper cluster under Exhibitor supervisory the following section could be interesting for you - * *poll_interval*: how often list of ZooKeeper and Exhibitor nodes should be updated from Exhibitor - * *port*: Exhibitor port - * *hosts*: initial list of Exhibitor (ZooKeeper) nodes in format: [ 'host1', 'host2', 'etc...' ]. This list would be updated automatically when Exhibitor (ZooKeeper) cluster topology changes. - -* *postgresql* - * *name*: the name of the Postgres host, must be unique for the cluster - * *listen*: ip address + port that Postgres listening. Must be accessible from other nodes in the cluster if using streaming replication. - * *connect_address*: ip address + port through which Postgres is accessible from other nodes and applications. - * *data_dir*: file path to initialize and store Postgres data files - * *maximum_lag_on_failover*: the maximum bytes a follower may lag before it is not eligible become leader - * *pg_hba*: list of lines which should be added to pg_hba.conf - * *- host all all 0.0.0.0/0 md5* - * *replication* - * *username*: replication username, user will be created during initialization - * *password*: replication password, user will be created during initialization - * *network*: network setting for replication in pg_hba.conf - * *callbacks* callback scripts to run on certain actions. Patroni will pass current action, role and cluster name. See scripts/aws.py as an example on how to write them. - * *on_start*: a script to run when the cluster starts - * *on_stop*: a script to run when the cluster stops - * *on_restart*: a script to run when the cluster restarts - * *on_reload*: a script to run when configuration reload is triggered - * *on_role_change*: a script to run when the cluster is being promoted or demoted - * *superuser* - * *password*: password for postgres user. It would be set during initialization - * *admin*: - * *username*: admin username, user will be created during initialization. It would have CREATEDB and CREATEROLE privileges - * *password*: admin password, user will be created during initialization. - * *recovery_conf*: configuration settings written to recovery.conf when configuring follower - * *parameters*: list of configuration settings for Postgres - -## Replication choices - -Patroni uses Postgres' streaming replication. By default, this replication is asynchronous. For more information, see the [Postgres documentation on streaming replication](http://www.postgresql.org/docs/current/static/warm-standby.html#STREAMING-REPLICATION). - -Patroni's asynchronous replication configuration allows for `maximum_lag_on_failover` settings. This setting ensures failover will not occur if a follower is more than a certain number of bytes behind the follower. This setting should be increased or decreased based on business requirements. - -When asynchronous replication is not best for your use-case, investigate how Postgres's [synchronous replication](http://www.postgresql.org/docs/current/static/warm-standby.html#SYNCHRONOUS-REPLICATION) works. Synchronous replication ensures consistency across a cluster by confirming that writes are written to a secondary before returning to the connecting client with a success. The cost of synchronous replication will be reduced throughput on writes. This throughput will be entirely based on network performance. In hosted datacenter environments (like AWS, Rackspace, or any network you do not control), synchrous replication increases the variability of write performance significantly. If followers become inaccessible from the leader, the leader will becomes effectively readonly. - -To enable a simple synchronous replication test, add the follow lines to the `parameters` section of your YAML configuration files. - -```YAML - synchronous_commit: "on" - synchronous_standby_names: "*" -``` - -When using synchronous replication, use at least a 3-Postgres data nodes to ensure write availability if one host fails. - -Choosing your replication schema is dependent on the many business decisions. Investigate both async and sync replication, as well as other HA solutions, to determine which solution is best for you. - -## Applications should not use superusers - -When connecting from an application, always use a non-superuser. Patroni requires access to the database to function properly. By using a superuser from application, you can potentially use the entire connection pool, including the connections reserved for superusers with the `superuser_reserved_connections` setting. If Patroni cannot access the Primary, because the connection pool is full, behavior will be undesireable. - -## Requirements on a Mac - -Run the following on a Mac to install requirements: - -``` -brew install postgresql etcd haproxy libyaml python -pip install psycopg2 pyyaml -``` - -## Notice - -There are many different ways to do HA with PostgreSQL, see [the -PostgreSQL documentation](https://wiki.postgresql.org/wiki/Replication,_Clustering,_and_Connection_Pooling) for a complete list. - -We call this project a "template" because it is far from a one-size fits -all, or a plug-and-play replication system. It will have it's own -caveats. Use wisely. diff --git a/README.rst b/README.rst new file mode 100644 index 00000000..c8c5eb7c --- /dev/null +++ b/README.rst @@ -0,0 +1,220 @@ +|Build Status| |Coverage Status| # Patroni: A Template for PostgreSQL HA +with ZooKeeper or etcd + +Patroni was previously known as Governor. + +*There are many ways to run high availability with PostgreSQL; here we +present a template for you to create your own custom fit high +availability solution using python and distributed configuration store +(like ZooKeeper or etcd) for maximum accessibility.* + +Getting Started +--------------- + +To get started, do the following from different terminals: + +:: + + > etcd --data-dir=data/etcd + > ./patroni.py postgres0.yml + > ./patroni.py postgres1.yml + +From there, you will see a high-availability cluster start up. Test +different settings in the YAML files to see how behavior changes. Kill +some of the different components to see how the system behaves. + +Add more ``postgres*.yml`` files to create an even larger cluster. + +We provide a haproxy configuration, which will give your application a +single endpoint for connecting to the cluster's leader. To configure, +run: + +:: + + > haproxy -f haproxy.cfg + +:: + + > psql --host 127.0.0.1 --port 5000 postgres + +How Patroni works +----------------- + +For a diagram of the high availability decision loop, see the included a +PDF: +`postgres-ha.pdf `__ + +YAML Configuration +------------------ + +For an example file, see ``postgres0.yml``. Below is an explanation of +settings: + +- *ttl*: the TTL to acquire the leader lock. Think of it as the length + of time before automatic failover process is initiated. +- *loop\_wait*: the number of seconds the loop will sleep + +- *restapi* +- *listen*: ip address + port that Patroni will listen to provide + health-check information for haproxy. +- *connect\_address*: ip address + port through which restapi is + accessible. + +- *etcd* +- *scope*: the relative path used on etcd's http api for this + deployment, thus you can run multiple HA deployments from a single + etcd +- *ttl*: the TTL to acquire the leader lock. Think of it as the length + of time before automatic failover process is initiated. +- *host*: the host:port for the etcd endpoint + +- *zookeeper* +- *scope*: the relative path used on etcd's http api for this + deployment, thus you can run multiple HA deployments from a single + etcd +- *session\_timeout*: the TTL to acquire the leader lock. Think of it + as the length of time before automatic failover process is initiated. +- *reconnects\_timeout*: how long we should try to reconnect to + ZooKeeper after connection loss. After this timeout we assume that we + don't have lock anymore and will restart in read-only mode. +- *hosts*: list of ZooKeeper cluster members in format: [ + 'host1:port1', 'host2:port2', 'etc...'] +- *exhibitor*: if you are running ZooKeeper cluster under Exhibitor + supervisory the following section could be interesting for you + + - *poll\_interval*: how often list of ZooKeeper and Exhibitor nodes + should be updated from Exhibitor + - *port*: Exhibitor port + - *hosts*: initial list of Exhibitor (ZooKeeper) nodes in format: [ + 'host1', 'host2', 'etc...' ]. This list would be updated + automatically when Exhibitor (ZooKeeper) cluster topology changes. + +- *postgresql* +- *name*: the name of the Postgres host, must be unique for the cluster +- *listen*: ip address + port that Postgres listening. Must be + accessible from other nodes in the cluster if using streaming + replication. +- *connect\_address*: ip address + port through which Postgres is + accessible from other nodes and applications. +- *data\_dir*: file path to initialize and store Postgres data files +- *maximum\_lag\_on\_failover*: the maximum bytes a follower may lag + before it is not eligible become leader +- *pg\_hba*: list of lines which should be added to pg\_hba.conf + + - *- host all all 0.0.0.0/0 md5* + +- *replication* + + - *username*: replication username, user will be created during + initialization + - *password*: replication password, user will be created during + initialization + - *network*: network setting for replication in pg\_hba.conf + +- *callbacks* callback scripts to run on certain actions. Patroni will + pass current action, role and cluster name. See scripts/aws.py as an + example on how to write them. + + - *on\_start*: a script to run when the cluster starts + - *on\_stop*: a script to run when the cluster stops + - *on\_restart*: a script to run when the cluster restarts + - *on\_reload*: a script to run when configuration reload is + triggered + - *on\_role\_change*: a script to run when the cluster is being + promoted or demoted + +- *superuser* + + - *password*: password for postgres user. It would be set during + initialization + +- *admin*: + + - *username*: admin username, user will be created during + initialization. It would have CREATEDB and CREATEROLE privileges + - *password*: admin password, user will be created during + initialization. + +- *recovery\_conf*: configuration settings written to recovery.conf + when configuring follower +- *parameters*: list of configuration settings for Postgres + +Replication choices +------------------- + +Patroni uses Postgres' streaming replication. By default, this +replication is asynchronous. For more information, see the `Postgres +documentation on streaming +replication `__. + +Patroni's asynchronous replication configuration allows for +``maximum_lag_on_failover`` settings. This setting ensures failover will +not occur if a follower is more than a certain number of bytes behind +the follower. This setting should be increased or decreased based on +business requirements. + +When asynchronous replication is not best for your use-case, investigate +how Postgres's `synchronous +replication `__ +works. Synchronous replication ensures consistency across a cluster by +confirming that writes are written to a secondary before returning to +the connecting client with a success. The cost of synchronous +replication will be reduced throughput on writes. This throughput will +be entirely based on network performance. In hosted datacenter +environments (like AWS, Rackspace, or any network you do not control), +synchrous replication increases the variability of write performance +significantly. If followers become inaccessible from the leader, the +leader will becomes effectively readonly. + +To enable a simple synchronous replication test, add the follow lines to +the ``parameters`` section of your YAML configuration files. + +.. code:: YAML + + synchronous_commit: "on" + synchronous_standby_names: "*" + +When using synchronous replication, use at least a 3-Postgres data nodes +to ensure write availability if one host fails. + +Choosing your replication schema is dependent on the many business +decisions. Investigate both async and sync replication, as well as other +HA solutions, to determine which solution is best for you. + +Applications should not use superusers +-------------------------------------- + +When connecting from an application, always use a non-superuser. Patroni +requires access to the database to function properly. By using a +superuser from application, you can potentially use the entire +connection pool, including the connections reserved for superusers with +the ``superuser_reserved_connections`` setting. If Patroni cannot access +the Primary, because the connection pool is full, behavior will be +undesireable. + +Requirements on a Mac +--------------------- + +Run the following on a Mac to install requirements: + +:: + + brew install postgresql etcd haproxy libyaml python + pip install psycopg2 pyyaml + +Notice +------ + +There are many different ways to do HA with PostgreSQL, see `the +PostgreSQL +documentation `__ +for a complete list. + +We call this project a "template" because it is far from a one-size fits +all, or a plug-and-play replication system. It will have it's own +caveats. Use wisely. + +.. |Build Status| image:: https://travis-ci.org/zalando/patroni.svg?branch=master + :target: https://travis-ci.org/zalando/patroni +.. |Coverage Status| image:: https://coveralls.io/repos/zalando/patroni/badge.svg?branch=master + :target: https://coveralls.io/r/zalando/patroni?branch=master diff --git a/patroni/__init__.py b/patroni/__init__.py new file mode 100644 index 00000000..a3e6290d --- /dev/null +++ b/patroni/__init__.py @@ -0,0 +1 @@ +__version__ = '0.15' diff --git a/patroni/__main__.py b/patroni/__main__.py new file mode 100644 index 00000000..7390c1f1 --- /dev/null +++ b/patroni/__main__.py @@ -0,0 +1,4 @@ +import patroni + +if __name__ == '__main__': + patroni.main() diff --git a/helpers/__init__.py b/patroni/helpers/__init__.py similarity index 100% rename from helpers/__init__.py rename to patroni/helpers/__init__.py diff --git a/helpers/api.py b/patroni/helpers/api.py similarity index 100% rename from helpers/api.py rename to patroni/helpers/api.py diff --git a/helpers/dcs.py b/patroni/helpers/dcs.py similarity index 99% rename from helpers/dcs.py rename to patroni/helpers/dcs.py index c7140c22..69ccdebd 100644 --- a/helpers/dcs.py +++ b/patroni/helpers/dcs.py @@ -1,7 +1,7 @@ import abc from collections import namedtuple -from helpers.utils import calculate_ttl, sleep +from patroni.helpers.utils import calculate_ttl, sleep from six.moves.urllib_parse import urlparse, urlunparse, parse_qsl diff --git a/helpers/etcd.py b/patroni/helpers/etcd.py similarity index 98% rename from helpers/etcd.py rename to patroni/helpers/etcd.py index add736c5..c58c1a78 100644 --- a/helpers/etcd.py +++ b/patroni/helpers/etcd.py @@ -8,8 +8,8 @@ import socket from dns.exception import DNSException from dns import resolver -from helpers.dcs import AbstractDCS, Cluster, DCSError, Member, parse_connection_string -from helpers.utils import sleep +from patroni.helpers.dcs import AbstractDCS, Cluster, DCSError, Member, parse_connection_string +from patroni.helpers.utils import sleep from requests.exceptions import RequestException logger = logging.getLogger(__name__) diff --git a/helpers/ha.py b/patroni/helpers/ha.py similarity index 99% rename from helpers/ha.py rename to patroni/helpers/ha.py index 8e283c55..0496a094 100644 --- a/helpers/ha.py +++ b/patroni/helpers/ha.py @@ -1,6 +1,6 @@ import logging -from helpers.dcs import DCSError +from patroni.helpers.dcs import DCSError from psycopg2 import InterfaceError, OperationalError logger = logging.getLogger(__name__) diff --git a/helpers/postgresql.py b/patroni/helpers/postgresql.py similarity index 99% rename from helpers/postgresql.py rename to patroni/helpers/postgresql.py index ce8ca18f..8593fb52 100644 --- a/helpers/postgresql.py +++ b/patroni/helpers/postgresql.py @@ -6,7 +6,7 @@ import shutil import subprocess import six -from helpers.utils import sleep +from patroni.helpers.utils import sleep from six.moves.urllib_parse import urlparse if six.PY3: diff --git a/helpers/utils.py b/patroni/helpers/utils.py similarity index 100% rename from helpers/utils.py rename to patroni/helpers/utils.py diff --git a/helpers/zookeeper.py b/patroni/helpers/zookeeper.py similarity index 98% rename from helpers/zookeeper.py rename to patroni/helpers/zookeeper.py index cb2918cd..fd5f4f5b 100644 --- a/helpers/zookeeper.py +++ b/patroni/helpers/zookeeper.py @@ -3,8 +3,8 @@ import random import requests import time -from helpers.dcs import AbstractDCS, Cluster, DCSError, Member, parse_connection_string -from helpers.utils import sleep +from patroni.helpers.dcs import AbstractDCS, Cluster, DCSError, Member, parse_connection_string +from patroni.helpers.utils import sleep from kazoo.client import KazooClient, KazooState from kazoo.exceptions import NoNodeError, NodeExistsError from requests.exceptions import RequestException diff --git a/patroni.py b/patroni/patroni.py similarity index 93% rename from patroni.py rename to patroni/patroni.py index e26a4059..dd7bd52e 100755 --- a/patroni.py +++ b/patroni/patroni.py @@ -5,12 +5,12 @@ import sys import time import yaml -from helpers.api import RestApiServer -from helpers.etcd import Etcd -from helpers.ha import Ha -from helpers.postgresql import Postgresql -from helpers.utils import setup_signal_handlers, sleep -from helpers.zookeeper import ZooKeeper +from patroni.helpers.api import RestApiServer +from patroni.helpers.etcd import Etcd +from patroni.helpers.ha import Ha +from patroni.helpers.postgresql import Postgresql +from patroni.helpers.utils import setup_signal_handlers, sleep +from patroni.helpers.zookeeper import ZooKeeper logger = logging.getLogger(__name__) @@ -117,7 +117,3 @@ def main(): patroni.touch_member(patroni.shutdown_member_ttl) # schedule member removal patroni.postgresql.stop() patroni.ha.dcs.delete_leader() - - -if __name__ == '__main__': - main() diff --git a/scripts/__init__.py b/patroni/scripts/__init__.py similarity index 100% rename from scripts/__init__.py rename to patroni/scripts/__init__.py diff --git a/scripts/aws.py b/patroni/scripts/aws.py similarity index 100% rename from scripts/aws.py rename to patroni/scripts/aws.py diff --git a/scripts/restore.py b/patroni/scripts/restore.py similarity index 100% rename from scripts/restore.py rename to patroni/scripts/restore.py diff --git a/release.sh b/release.sh new file mode 100755 index 00000000..5f97ba08 --- /dev/null +++ b/release.sh @@ -0,0 +1,28 @@ +#!/bin/sh + +if [ $# -ne 1 ]; then + >&2 echo "usage: $0 " + exit 1 +fi + +set -xe + +python3 --version +git --version + +version=$1 + +sed -i "s/__version__ = .*/__version__ = '${version}'/" __init__.py +python3 setup.py clean +python3 setup.py test +python3 setup.py flake8 + +git add __init__.py + +git commit -m "Bumped version to $version" +git push + +python3 setup.py sdist bdist_wheel upload + +git tag ${version} +git push --tags diff --git a/requirements-py3.txt b/requirements-py3.txt index 0fd9dfb3..544577ad 100644 --- a/requirements-py3.txt +++ b/requirements-py3.txt @@ -5,5 +5,5 @@ psycopg2 PyYAML requests six -kazoo>=2.2.1 -python-etcd>=0.4.1 +kazoo +python-etcd diff --git a/setup.py b/setup.py index 62d05a69..82ac8de8 100644 --- a/setup.py +++ b/setup.py @@ -18,14 +18,23 @@ if sys.version_info < (2, 7, 0): __location__ = os.path.join(os.getcwd(), os.path.dirname(inspect.getfile(inspect.currentframe()))) +def read_version(package): + data = {} + with open(os.path.join(package, '__init__.py'), 'r') as fd: + exec(fd.read(), data) + return data['__version__'] NAME = 'patroni' -MAIN_PACKAGE = 'patroni.py' +MAIN_PACKAGE = 'patroni' HELPERS = 'helpers' SCRIPTS = 'scripts' -VERSION = '0.1' -DESCRIPTION = 'A Template for PostgreSQL HA with etcd' +VERSION = read_version(MAIN_PACKAGE) +DESCRIPTION = 'PostgreSQL High-Available orchestrator and CLI' LICENSE = 'The MIT License' +URL = 'https://github.com/zalando/patroni' +AUTHOR = 'Alexander Kukushkin, Alexey Klyukin, Feike Steenbergen' +AUTHOR_EMAIL = 'alexander.kukushkin@zalando.de, oleksii.kliukin@zalando.de, feike.steenbergen@zalando.de' +KEYWORDS = 'etcd governor patroni postgresql postgres ha zookeeper streaming replication' COVERAGE_XML = True COVERAGE_HTML = False @@ -38,7 +47,7 @@ CLASSIFIERS = [ 'Environment :: Console', 'Intended Audience :: Developers', 'Intended Audience :: System Administrators', - 'License :: OSI Approved :: The MIT License', + 'License :: OSI Approved :: MIT License', 'Operating System :: POSIX :: Linux', 'Programming Language :: Python', 'Programming Language :: Python :: 2.7', @@ -82,7 +91,8 @@ class PyTest(TestCommand): params['plugins'] = ['cov'] if self.junitxml: params['args'] += self.junitxml - params['args'] += ['--doctest-modules', HELPERS, '--doctest-modules', SCRIPTS, '-s'] + #params['args'] += ['--doctest-modules', MAIN_PACKAGE, '--doctest-modules', HELPERS, '--doctest-modules', SCRIPTS, '-s'] + params['args'] += ['--doctest-modules', MAIN_PACKAGE, '-s', '-vv'] errno = pytest.main(**params) sys.exit(errno) @@ -118,10 +128,13 @@ def setup_package(): setup( name=NAME, version=version, + url=URL, + author=AUTHOR, + author_email=AUTHOR_EMAIL, description=DESCRIPTION, license=LICENSE, - keywords='etcd governor patroni postgresql postgres ha zookeeper', - long_description=read('README.md'), + keywords=KEYWORDS, + long_description=read('README.rst'), classifiers=CLASSIFIERS, test_suite='tests', packages=setuptools.find_packages(exclude=['tests', 'tests.*']), diff --git a/tests/test_api.py b/tests/test_api.py index 91b36943..12d7798b 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -1,7 +1,7 @@ import psycopg2 import unittest -from helpers.api import RestApiHandler, RestApiServer +from patroni.helpers.api import RestApiHandler, RestApiServer from six import BytesIO as IO from test_postgresql import psycopg2_connect diff --git a/tests/test_aws.py b/tests/test_aws.py index 84d495fe..09c357f4 100644 --- a/tests/test_aws.py +++ b/tests/test_aws.py @@ -2,7 +2,7 @@ import unittest import requests import boto.ec2 from collections import namedtuple -from scripts.aws import AWSConnection +from patroni.scripts.aws import AWSConnection from requests.exceptions import RequestException diff --git a/tests/test_etcd.py b/tests/test_etcd.py index 38692c52..3f09caa5 100644 --- a/tests/test_etcd.py +++ b/tests/test_etcd.py @@ -8,8 +8,8 @@ import time import unittest from dns.exception import DNSException -from helpers.dcs import Cluster, DCSError, Member -from helpers.etcd import Client, Etcd +from patroni.helpers.dcs import Cluster, DCSError, Member +from patroni.helpers.etcd import Client, Etcd from mock import Mock, patch diff --git a/tests/test_ha.py b/tests/test_ha.py index abfc4ac8..8f9a9c49 100644 --- a/tests/test_ha.py +++ b/tests/test_ha.py @@ -1,8 +1,8 @@ import unittest -from helpers.dcs import Cluster, DCSError -from helpers.etcd import Client, Etcd -from helpers.ha import Ha +from patroni.helpers.dcs import Cluster, DCSError +from patroni.helpers.etcd import Client, Etcd +from patroni.helpers.ha import Ha from mock import Mock, patch from test_etcd import etcd_read, etcd_write diff --git a/tests/test_patroni.py b/tests/test_patroni.py index 67ac1ffd..fd2e605d 100644 --- a/tests/test_patroni.py +++ b/tests/test_patroni.py @@ -1,5 +1,5 @@ import datetime -import helpers.zookeeper +import patroni.helpers.zookeeper import psycopg2 import subprocess import sys @@ -7,12 +7,12 @@ import time import unittest import yaml -from helpers.api import RestApiServer -from helpers.dcs import Cluster, Member -from helpers.etcd import Etcd -from helpers.zookeeper import ZooKeeper +from patroni.helpers.api import RestApiServer +from patroni.helpers.dcs import Cluster, Member +from patroni.helpers.etcd import Etcd +from patroni.helpers.zookeeper import ZooKeeper from mock import Mock, patch -from patroni import Patroni, main +from patroni.patroni import Patroni, main from six.moves import BaseHTTPServer from test_etcd import Client, etcd_read, etcd_write from test_ha import true, false @@ -70,7 +70,7 @@ class TestPatroni(unittest.TestCase): Postgresql.write_recovery_conf = self.write_recovery_conf def test_get_dcs(self): - helpers.zookeeper.KazooClient = MockKazooClient + patroni.helpers.zookeeper.KazooClient = MockKazooClient self.assertIsInstance(self.p.get_dcs('', {'zookeeper': {'scope': '', 'hosts': ''}}), ZooKeeper) self.assertRaises(Exception, self.p.get_dcs, '', {}) diff --git a/tests/test_postgresql.py b/tests/test_postgresql.py index 040d1c66..f79a1438 100644 --- a/tests/test_postgresql.py +++ b/tests/test_postgresql.py @@ -4,8 +4,8 @@ import shutil import subprocess import unittest -from helpers.dcs import Cluster, Member -from helpers.postgresql import Postgresql +from patroni.helpers.dcs import Cluster, Member +from patroni.helpers.postgresql import Postgresql def nop(*args, **kwargs): diff --git a/tests/test_restore.py b/tests/test_restore.py index 38b34c6a..2ffd8a58 100644 --- a/tests/test_restore.py +++ b/tests/test_restore.py @@ -1,7 +1,7 @@ import unittest from mock import MagicMock, patch import os -from scripts.restore import Restore, WALERestore +from patroni.scripts.restore import Restore, WALERestore def fake_cursor_fetchone(*args, **kwargs): diff --git a/tests/test_utils.py b/tests/test_utils.py index 76dbff66..3bb238b0 100644 --- a/tests/test_utils.py +++ b/tests/test_utils.py @@ -2,7 +2,7 @@ import os import time import unittest -from helpers.utils import sigchld_handler, sigterm_handler, sleep +from patroni.helpers.utils import sigchld_handler, sigterm_handler, sleep def nop(*args, **kwargs): diff --git a/tests/test_zookeeper.py b/tests/test_zookeeper.py index 53f5ab82..1333897a 100644 --- a/tests/test_zookeeper.py +++ b/tests/test_zookeeper.py @@ -1,8 +1,8 @@ -import helpers.zookeeper +import patroni.helpers.zookeeper import requests import unittest -from helpers.zookeeper import ExhibitorEnsembleProvider, ZooKeeper, ZooKeeperError +from patroni.helpers.zookeeper import ExhibitorEnsembleProvider, ZooKeeper, ZooKeeperError from kazoo.client import KazooState from kazoo.exceptions import NoNodeError, NodeExistsError from kazoo.protocol.states import ZnodeStat @@ -105,7 +105,7 @@ class TestExhibitorEnsembleProvider(unittest.TestCase): def set_up(self): requests.get = requests_get - helpers.zookeeper.sleep = exhibitor_sleep + patroni.helpers.zookeeper.sleep = exhibitor_sleep def test_init(self): self.assertRaises(Exception, ExhibitorEnsembleProvider, ['localhost'], 8181) @@ -119,7 +119,7 @@ class TestZooKeeper(unittest.TestCase): def set_up(self): requests.get = requests_get - helpers.zookeeper.KazooClient = MockKazooClient + patroni.helpers.zookeeper.KazooClient = MockKazooClient self.zk = ZooKeeper('foo', {'exhibitor': {'hosts': ['localhost', 'exhibitor'], 'port': 8181}, 'scope': 'test'}) def test_session_listener(self):