mirror of
https://github.com/outbackdingo/patroni.git
synced 2026-08-25 14:53:37 +00:00
Merge pull request #211 from zalando/feature/environment-configuration
Implement possibility to configure Patroni via environment
This commit is contained in:
Alexander Kukushkin
+7
-1
@@ -75,7 +75,13 @@ run:
|
||||
YAML Configuration
|
||||
===============
|
||||
|
||||
Go `here <https://github.com/zalando/patroni/blob/master/SETTINGS.rst>`__ for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/zalando/patroni/blob/master/postgres0.yml>`__.
|
||||
Go `here <https://github.com/zalando/patroni/blob/master/docs/SETTINGS.rst>`__ for comprehensive information about settings for etcd, consul, and ZooKeeper. And for an example, see `postgres0.yml <https://github.com/zalando/patroni/blob/master/postgres0.yml>`__.
|
||||
|
||||
=========================
|
||||
Environment Configuration
|
||||
=========================
|
||||
|
||||
Go `here <https://github.com/zalando/patroni/blob/master/docs/ENVIRONMENT.rst>`__ for comprehensive information about configuring(overriding) settings via environment variables.
|
||||
|
||||
===============
|
||||
Replication Choices
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
==================================
|
||||
Environment Configuration Settings
|
||||
==================================
|
||||
|
||||
It is possible to override some of the configuration parameters defined in the Patroni configuration file using the system environment variables. This document lists all environment variables handled by Patroni. The values set via those variables always take precedence over the ones set in the Patroni configuration file.
|
||||
|
||||
Global/Universal
|
||||
----------------
|
||||
- **PATRONI\_CONFIGURATION**: it is possible to set the entire configuration for the Patroni via ``PATRONI_CONFIGURATION`` environment variable. In this case any other environment variables will not be considered!
|
||||
- **PATRONI\_NAME**: name of the node where the current instance of Patroni is running. Must be unique for the cluster.
|
||||
- **PATRONI\_NAMESPACE**: path within the configuration store where Patroni will keep information about the cluster. Default value: "/service"
|
||||
- **PATRONI\_SCOPE**: cluster name
|
||||
|
||||
Bootstrap configuration
|
||||
-----------------------
|
||||
It is possible to create new database users right after the successful initialization of a new cluster. This process is defined by the following variables:
|
||||
|
||||
- **PATRONI\_<username>\_PASSWORD='<password>'**
|
||||
- **PATRONI\_<username>\_OPTIONS='list,of,options'**
|
||||
|
||||
Example: defining ``PATRONI_admin_PASSWORD=strongpasswd`` and ``PATRONI_admin_OPTIONS='createrole,createdb'`` will cause creation of the user **admin** with the password **strongpasswd** that is allowed to create other users and databases.
|
||||
|
||||
Consul
|
||||
------
|
||||
- **PATRONI\_CONSUL\_HOST**: the host:port for the Consul endpoint.
|
||||
|
||||
Etcd
|
||||
----
|
||||
- **PATRONI\_ETCD\_HOST**: the host:port for the etcd endpoint.
|
||||
|
||||
Exhibitor
|
||||
---------
|
||||
- **PATRONI\_EXHIBITOR\_HOSTS**: initial list of Exhibitor (ZooKeeper) nodes in format: ['host1', 'host2', 'etc...' ]. This list updates automatically whenever the Exhibitor (ZooKeeper) cluster topology changes.
|
||||
- **PATRONI\_EXHIBITOR\_PORT**: Exhibitor port.
|
||||
|
||||
PostgreSQL
|
||||
----------
|
||||
- **PATRONI\_POSTGRESQL\_LISTEN**: IP address + port that Postgres listens to. Multiple comma-separated addresses are permitted, as long as the port component is appended after to the last one with a colon, i.e. ``listen: 127.0.0.1,127.0.0.2:5432``. Patroni will use the first address from this list to establish local connections to the PostgreSQL node.
|
||||
- **PATRONI\_POSTGRESQL\_CONNECT\_ADDRESS**: IP address + port through which Postgres is accessible from other nodes and applications.
|
||||
- **PATRONI\_POSTGRESQL\_DATA\_DIR**: The location of the Postgres data directory, either existing or to be initialized by Patroni.
|
||||
- **PATRONI\_POSTGRESQL\_PGPASS**: path to the `.pgpass <https://www.postgresql.org/docs/current/static/libpq-pgpass.html>`__ password file. Patroni creates this file before executing pg\_basebackup and under some other circumstances. The location must be writable by Patroni.
|
||||
- **PATRONI\_REPLICATION\_USERNAME**: replication username; the user will be created during initialization. Replicas will use this user to access master via streaming replication
|
||||
- **PATRONI\_REPLICATION\_PASSWORD**: replication password; the user will be created during initialization.
|
||||
- **PATRONI\_SUPERUSER\_USERNAME**: name for the superuser, set during initialization (initdb) and later used by Patroni to connect to the postgres. Also this user is used by pg_rewind.
|
||||
- **PATRONI\_SUPERUSER\_PASSWORD**: password for the superuser, set during initialization (initdb).
|
||||
|
||||
REST API
|
||||
--------
|
||||
- **PATRONI\_RESTAPI\_CONNECT\_ADDRESS**: IP address and port to access the REST API.
|
||||
- **PATRONI\_RESTAPI\_LISTEN**: IP address and port that Patroni will listen to, to provide health-check information for HAProxy.
|
||||
- **PATRONI\_RESTAPI\_USERNAME**: Basic-auth username to protect unsafe REST API endpoints.
|
||||
- **PATRONI\_RESTAPI\_PASSWORD**: Basic-auth password to protect unsafe REST API endpoints.
|
||||
- **PATRONI\_RESTAPI\_CERTFILE**: Specifies the file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.
|
||||
- **PATRONI\_RESTAPI\_KEYFILE**: Specifies the file with the secret key in the PEM format.
|
||||
|
||||
ZooKeeper
|
||||
---------
|
||||
- **PATRONI\_ZOOKEEPER\_HOSTS**: comma separated list of ZooKeeper cluster members: 'host1:port1,host2:port2,etc...'
|
||||
@@ -5,7 +5,7 @@ YAML Configuration Settings
|
||||
Global/Universal
|
||||
----------------
|
||||
- **name**: the name of the host. Must be unique for the cluster.
|
||||
- **namespace**: path within configuration store where Patroni will keep information about cluster. Default value: "/service"
|
||||
- **namespace**: path within the configuration store where Patroni will keep information about the cluster. Default value: "/service"
|
||||
- **scope**: cluster name
|
||||
|
||||
Bootstrap configuration
|
||||
@@ -54,8 +54,8 @@ PostgreSQL
|
||||
- **username**: name for the superuser, set during initialization (initdb) and later used by Patroni to connect to the postgres.
|
||||
- **password**: password for the superuser, set during initialization (initdb).
|
||||
- **replication**:
|
||||
- **username**: replication username; user will be created during initialization.
|
||||
- **password**: replication password; user will be created during initialization.
|
||||
- **username**: replication username; the user will be created during initialization. Replicas will use this user to access master via streaming replication
|
||||
- **password**: replication password; the user will be created during initialization.
|
||||
- **callbacks**: callback scripts to run on certain actions. Patroni will pass the action, role and cluster name. (See scripts/aws.py as an example of how to write them.)
|
||||
- **on\_reload**: run this script when configuration reload is triggered.
|
||||
- **on\_restart**: run this script when the cluster restarts.
|
||||
@@ -64,20 +64,24 @@ PostgreSQL
|
||||
- **on\_stop**: run this script when the cluster stops.
|
||||
- **connect\_address**: IP address + port through which Postgres is accessible from other nodes and applications.
|
||||
- **create\_replica\_methods**: an ordered list of the create methods for turning a Patroni node into a new replica. "basebackup" is the default method; other methods are assumed to refer to scripts, each of which is configured as its own config item.
|
||||
- **data\_dir**: file path to initialize and store Postgres data files.
|
||||
- **data\_dir**: The location of the Postgres data directory, either existing or to be initialized by Patroni.
|
||||
- **listen**: IP address + port that Postgres listens to; must be accessible from other nodes in the cluster, if you're using streaming replication. Multiple comma-separated addresses are permitted, as long as the port component is appended after to the last one with a colon, i.e. ``listen: 127.0.0.1,127.0.0.2:5432``. Patroni will use the first address from this list to establish local connections to the PostgreSQL node.
|
||||
- **pgpass**: path to the `.pgpass <https://www.postgresql.org/docs/current/static/libpq-pgpass.html>`__ password file. Patroni creates this file before executing pg\_basebackup and under some other circumstances. The location must be writable by Patroni.
|
||||
- **recovery\_conf**: additional configuration settings written to recovery.conf when configuring follower.
|
||||
- **parameters**: list of configuration settings for Postgres. Many of these are required for replication to work.
|
||||
- **replica\_method** for each create_replica_method other than basebackup, you would add a configuration section of the same name. At a minimum, this should include "command" with a full path to the actual script to be executed. Other configuration parameters will be passed along to the script in the form "parameter=value".
|
||||
|
||||
REST API
|
||||
--------
|
||||
- **connect\_address**: IP address and port through which restapi is accessible.
|
||||
- **connect\_address**: IP address and port to access the REST API.
|
||||
- **listen**: IP address and port that Patroni will listen to, to provide health-check information for HAProxy.
|
||||
- **Optional**:
|
||||
- **auth**: 'username:password' to protect dangerous REST API endpoints.
|
||||
- **certfile**: Specifies a file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.
|
||||
- **keyfile**: Specifies a file with the secret key in the PEM format.
|
||||
- **authentication**:
|
||||
- **username**: Basic-auth username to protect unsafe REST API endpoints.
|
||||
- **password**: Basic-auth password to protect unsafe REST API endpoints.
|
||||
|
||||
- **certfile**: Specifies the file with the certificate in the PEM format. If the certfile is not specified or is left empty, the API server will work without SSL.
|
||||
- **keyfile**: Specifies the file with the secret key in the PEM format.
|
||||
|
||||
ZooKeeper
|
||||
----------
|
||||
+3
-17
@@ -1,7 +1,5 @@
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
|
||||
from patroni.api import RestApiServer
|
||||
@@ -17,11 +15,10 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Patroni(object):
|
||||
PATRONI_CONFIG_VARIABLE = 'PATRONI_CONFIGURATION'
|
||||
|
||||
def __init__(self, config_file=None, config_env=None):
|
||||
def __init__(self):
|
||||
self.version = __version__
|
||||
self.config = Config(config_file=config_file, config_env=config_env)
|
||||
self.config = Config()
|
||||
self.dcs = get_dcs(self.config)
|
||||
self.load_dynamic_configuration()
|
||||
|
||||
@@ -116,18 +113,7 @@ def main():
|
||||
logging.getLogger('requests').setLevel(logging.WARNING)
|
||||
setup_signal_handlers()
|
||||
|
||||
# Patroni reads the configuration from the command-line argument if it exists, and from the environment otherwise.
|
||||
config_env = False
|
||||
config_file = len(sys.argv) >= 2 and os.path.isfile(sys.argv[1]) and sys.argv[1]
|
||||
if not config_file:
|
||||
config_env = os.environ.get(Patroni.PATRONI_CONFIG_VARIABLE)
|
||||
if config_env is None:
|
||||
print('Usage: {0} config.yml'.format(sys.argv[0]))
|
||||
print('\tPatroni may also read the configuration from the {} environment variable'.
|
||||
format(Patroni.PATRONI_CONFIG_VARIABLE))
|
||||
return
|
||||
|
||||
patroni = Patroni(config_file, config_env)
|
||||
patroni = Patroni()
|
||||
try:
|
||||
patroni.run()
|
||||
except KeyboardInterrupt:
|
||||
|
||||
+7
-22
@@ -9,7 +9,7 @@ import datetime
|
||||
import pytz
|
||||
|
||||
from patroni.exceptions import PostgresConnectionException
|
||||
from patroni.utils import deep_compare, Retry, RetryFailedError
|
||||
from patroni.utils import deep_compare, patch_config, Retry, RetryFailedError
|
||||
from six.moves.BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
|
||||
from six.moves.socketserver import ThreadingMixIn
|
||||
from threading import Thread
|
||||
@@ -106,26 +106,11 @@ class RestApiHandler(BaseHTTPRequestHandler):
|
||||
self._write_status_response(200, response)
|
||||
|
||||
def do_GET_config(self):
|
||||
self._write_json_response(200, self.server.patroni.config.dynamic_configuration)
|
||||
|
||||
@staticmethod
|
||||
def _patch_config(config, data):
|
||||
is_changed = False
|
||||
for name, value in data.items():
|
||||
if value is None:
|
||||
if config.pop(name, None) is not None:
|
||||
is_changed = True
|
||||
elif name in config:
|
||||
if isinstance(value, dict):
|
||||
if RestApiHandler._patch_config(config[name], value):
|
||||
is_changed = True
|
||||
elif str(config[name]) != str(value):
|
||||
config[name] = value
|
||||
is_changed = True
|
||||
else:
|
||||
config[name] = value
|
||||
is_changed = True
|
||||
return is_changed
|
||||
cluster = self.server.patroni.ha.dcs.cluster or self.server.patroni.ha.dcs.get_cluster()
|
||||
if cluster.config:
|
||||
self._write_json_response(200, cluster.config.data)
|
||||
else:
|
||||
self.send_error(502)
|
||||
|
||||
def _read_json_content(self):
|
||||
if 'content-length' not in self.headers:
|
||||
@@ -145,7 +130,7 @@ class RestApiHandler(BaseHTTPRequestHandler):
|
||||
if request:
|
||||
cluster = self.server.patroni.ha.dcs.get_cluster()
|
||||
data = cluster.config.data.copy()
|
||||
if RestApiHandler._patch_config(data, request):
|
||||
if patch_config(data, request):
|
||||
value = json.dumps(data, separators=(',', ':'))
|
||||
if not self.server.patroni.ha.dcs.set_config_value(value, cluster.config.index):
|
||||
return self.send_error(409)
|
||||
|
||||
+114
-6
@@ -1,13 +1,15 @@
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import tempfile
|
||||
import yaml
|
||||
|
||||
from collections import defaultdict
|
||||
from copy import deepcopy
|
||||
from patroni.dcs import ClusterConfig
|
||||
from patroni.postgresql import Postgresql
|
||||
from patroni.utils import deep_compare
|
||||
from patroni.utils import deep_compare, parse_int, patch_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -32,6 +34,9 @@ class Config(object):
|
||||
to work with it as with the old `config` object.
|
||||
"""
|
||||
|
||||
PATRONI_ENV_PREFIX = 'PATRONI_'
|
||||
PATRONI_CONFIG_VARIABLE = PATRONI_ENV_PREFIX + 'CONFIGURATION'
|
||||
|
||||
__CACHE_FILENAME = 'patroni.dynamic.json'
|
||||
__DEFAULT_CONFIG = {
|
||||
'ttl': 30, 'loop_wait': 10, 'retry_timeout': 10,
|
||||
@@ -42,11 +47,25 @@ class Config(object):
|
||||
}
|
||||
}
|
||||
|
||||
def __init__(self, config_file=None, config_env=None):
|
||||
self._config_file = None if config_env else config_file
|
||||
def __init__(self):
|
||||
self._modify_index = -1
|
||||
self._dynamic_configuration = {}
|
||||
self._local_configuration = yaml.safe_load(config_env) if config_env else self._load_config_file()
|
||||
|
||||
self.__environment_configuration = self._build_environment_configuration()
|
||||
|
||||
# Patroni reads the configuration from the command-line argument if it exists, otherwise from the environment
|
||||
self._config_file = len(sys.argv) >= 2 and os.path.isfile(sys.argv[1]) and sys.argv[1]
|
||||
if self._config_file:
|
||||
self._local_configuration = self._load_config_file()
|
||||
else:
|
||||
config_env = os.environ.pop(self.PATRONI_CONFIG_VARIABLE, None)
|
||||
self._local_configuration = config_env and yaml.safe_load(config_env) or self.__environment_configuration
|
||||
if not self._local_configuration:
|
||||
print('Usage: {0} config.yml'.format(sys.argv[0]))
|
||||
print('\tPatroni may also read the configuration from the {0} environment variable'.
|
||||
format(self.PATRONI_CONFIG_VARIABLE))
|
||||
exit(1)
|
||||
|
||||
self.__effective_configuration = self._build_effective_configuration(self._dynamic_configuration,
|
||||
self._local_configuration)
|
||||
self._data_dir = self.__effective_configuration['postgresql']['data_dir']
|
||||
@@ -63,8 +82,11 @@ class Config(object):
|
||||
return deepcopy(self._dynamic_configuration)
|
||||
|
||||
def _load_config_file(self):
|
||||
"""Loads config.yaml from filesystem and applies some values which were set via ENV"""
|
||||
with open(self._config_file) as f:
|
||||
return yaml.safe_load(f)
|
||||
config = yaml.safe_load(f)
|
||||
patch_config(config, self.__environment_configuration)
|
||||
return config
|
||||
|
||||
def _load_cache(self):
|
||||
if os.path.isfile(self._cache_file):
|
||||
@@ -149,10 +171,90 @@ class Config(object):
|
||||
config['postgresql'][name].update(self._process_postgresql_parameters(value))
|
||||
elif name not in ('connect_address', 'listen', 'data_dir', 'pgpass', 'authentication'):
|
||||
config['postgresql'][name] = deepcopy(value)
|
||||
elif name in config:
|
||||
elif name in config: # only variables present in __DEFAULT_CONFIG allowed to be overriden from DCS
|
||||
config[name] = int(value)
|
||||
return config
|
||||
|
||||
@staticmethod
|
||||
def _build_environment_configuration():
|
||||
ret = defaultdict(dict)
|
||||
|
||||
def _popenv(name):
|
||||
return os.environ.pop(Config.PATRONI_ENV_PREFIX + name.upper(), None)
|
||||
|
||||
for param in ('name', 'namespace', 'scope'):
|
||||
value = _popenv(param)
|
||||
if value:
|
||||
ret[param] = value
|
||||
|
||||
def _set_section_values(section, params):
|
||||
for param in params:
|
||||
value = _popenv(section + '_' + param)
|
||||
if value:
|
||||
ret[section][param] = value
|
||||
|
||||
_set_section_values('restapi', ['listen', 'connect_address', 'certfile', 'keyfile'])
|
||||
_set_section_values('postgresql', ['listen', 'connect_address', 'data_dir', 'pgpass'])
|
||||
|
||||
def _get_auth(name):
|
||||
ret = {}
|
||||
for param in ('username', 'password'):
|
||||
value = _popenv(name + '_' + param)
|
||||
if value:
|
||||
ret[param] = value
|
||||
return len(ret) == 2 and ret or None
|
||||
|
||||
restapi_auth = _get_auth('restapi')
|
||||
if restapi_auth:
|
||||
ret['restapi']['authentication'] = restapi_auth
|
||||
|
||||
authentication = {}
|
||||
for user_type in ('replication', 'superuser'):
|
||||
entry = _get_auth(user_type)
|
||||
if entry:
|
||||
authentication[user_type] = entry
|
||||
|
||||
if authentication:
|
||||
ret['postgresql']['authentication'] = authentication
|
||||
|
||||
users = {}
|
||||
|
||||
def _parse_list(value):
|
||||
if not (value.strip().startswith('-') or '[' in value):
|
||||
value = '[{0}]'.format(value)
|
||||
try:
|
||||
return yaml.safe_load(value)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
for param in list(os.environ.keys()):
|
||||
if param.startswith(Config.PATRONI_ENV_PREFIX):
|
||||
name, suffix = (param[8:].rsplit('_', 1) + [''])[:2]
|
||||
if name and suffix:
|
||||
# PATRONI_(ETCD|CONSUL|ZOOKEEPER|EXHIBITOR|...)_(HOSTS?|PORT)
|
||||
if suffix in ('HOST', 'HOSTS', 'PORT') and '_' not in name:
|
||||
value = os.environ.pop(param)
|
||||
if suffix == 'PORT':
|
||||
value = value and parse_int(value)
|
||||
elif suffix == 'HOSTS':
|
||||
value = value and _parse_list(value)
|
||||
if value:
|
||||
ret[name.lower()][suffix.lower()] = value
|
||||
# PATRONI_<username>_PASSWORD=<password>, PATRONI_<username>_OPTIONS=<option1,option2,...>
|
||||
# CREATE USER "<username>" WITH <OPTIONS> PASSWORD '<password>'
|
||||
elif suffix == 'PASSWORD':
|
||||
password = os.environ.pop(param)
|
||||
if password:
|
||||
users[name] = {'password': password}
|
||||
options = os.environ.pop(param[:-9] + '_OPTIONS', None)
|
||||
options = options and _parse_list(options)
|
||||
if options:
|
||||
users[name]['options'] = options
|
||||
if users:
|
||||
ret['bootstrap']['users'] = users
|
||||
|
||||
return ret
|
||||
|
||||
def _build_effective_configuration(self, dynamic_configuration, local_configuration):
|
||||
config = self._safe_copy_dynamic_configuration(dynamic_configuration)
|
||||
for name, value in local_configuration.items():
|
||||
@@ -165,6 +267,12 @@ class Config(object):
|
||||
elif name not in config:
|
||||
config[name] = deepcopy(value) if value else {}
|
||||
|
||||
# restapi server expects to get restapi.auth = 'username:password'
|
||||
if 'authentication' in config['restapi']:
|
||||
restapi = config['restapi']
|
||||
auth = restapi['authentication']
|
||||
restapi['auth'] = '{0}:{1}'.format(auth['username'], auth['password'])
|
||||
|
||||
# special treatment for old config
|
||||
|
||||
# 'exhibitor' inside 'zookeeper':
|
||||
|
||||
@@ -60,6 +60,31 @@ def deep_compare(obj1, obj2):
|
||||
return True
|
||||
|
||||
|
||||
def patch_config(config, data):
|
||||
"""recursively 'patch' `config` with `data`
|
||||
:returns: `!True` if the `config` was changed"""
|
||||
is_changed = False
|
||||
for name, value in data.items():
|
||||
if value is None:
|
||||
if config.pop(name, None) is not None:
|
||||
is_changed = True
|
||||
elif name in config:
|
||||
if isinstance(value, dict):
|
||||
if isinstance(config[name], dict):
|
||||
if patch_config(config[name], value):
|
||||
is_changed = True
|
||||
else:
|
||||
config[name] = value
|
||||
is_changed = True
|
||||
elif str(config[name]) != str(value):
|
||||
config[name] = value
|
||||
is_changed = True
|
||||
else:
|
||||
config[name] = value
|
||||
is_changed = True
|
||||
return is_changed
|
||||
|
||||
|
||||
def parse_bool(value):
|
||||
"""
|
||||
>>> parse_bool(1)
|
||||
|
||||
@@ -4,6 +4,9 @@ name: postgresql0
|
||||
|
||||
restapi:
|
||||
listen: 127.0.0.1:8008
|
||||
# authentication:
|
||||
# username: username
|
||||
# password: password
|
||||
connect_address: 127.0.0.1:8008
|
||||
|
||||
etcd:
|
||||
|
||||
+6
-4
@@ -118,9 +118,11 @@ class TestRestApiHandler(unittest.TestCase):
|
||||
self.assertIsNotNone(MockRestApiServer(RestApiHandler, 'POST /restart HTTP/1.0'))
|
||||
MockRestApiServer(RestApiHandler, 'POST /restart HTTP/1.0\nAuthorization:')
|
||||
|
||||
@patch.object(MockPatroni, 'config')
|
||||
def test_do_GET_config(self, mock_config):
|
||||
mock_config.dynamic_configuration = {}
|
||||
@patch.object(MockHa, 'dcs')
|
||||
def test_do_GET_config(self, mock_dcs):
|
||||
mock_dcs.cluster.config.data = {}
|
||||
self.assertIsNotNone(MockRestApiServer(RestApiHandler, 'GET /config'))
|
||||
mock_dcs.cluster.config = None
|
||||
self.assertIsNotNone(MockRestApiServer(RestApiHandler, 'GET /config'))
|
||||
|
||||
@patch.object(MockHa, 'dcs')
|
||||
@@ -132,7 +134,7 @@ class TestRestApiHandler(unittest.TestCase):
|
||||
request += '\nContent-Length: '
|
||||
self.assertIsNotNone(MockRestApiServer(RestApiHandler, request + '34\n\n{"postgresql":{"use_slots":false}}'))
|
||||
config['ttl'] = 5
|
||||
config['postgresql'].update({'use_slots': True, "parameters": None})
|
||||
config['postgresql'].update({'use_slots': {'foo': True}, "parameters": None})
|
||||
config = json.dumps(config)
|
||||
request += str(len(config)) + '\n\n' + config
|
||||
MockRestApiServer(RestApiHandler, request)
|
||||
|
||||
+38
-3
@@ -1,4 +1,6 @@
|
||||
import os
|
||||
import unittest
|
||||
import sys
|
||||
|
||||
from mock import MagicMock, Mock, patch
|
||||
from patroni.config import Config
|
||||
@@ -11,15 +13,48 @@ class TestConfig(unittest.TestCase):
|
||||
@patch('json.load', Mock(side_effect=Exception))
|
||||
@patch.object(builtins, 'open', MagicMock())
|
||||
def setUp(self):
|
||||
self.config = Config(config_env='postgresql: {data_dir: foo}')
|
||||
sys.argv = ['patroni.py']
|
||||
os.environ[Config.PATRONI_CONFIG_VARIABLE] = 'restapi: {}\npostgresql: {data_dir: foo}'
|
||||
self.config = Config()
|
||||
|
||||
def test_no_config(self):
|
||||
self.assertRaises(SystemExit, Config)
|
||||
|
||||
@patch.object(Config, '_build_effective_configuration', Mock(side_effect=Exception))
|
||||
def test_set_dynamic_configuration(self):
|
||||
self.assertIsNone(self.config.set_dynamic_configuration({'foo': 'bar'}))
|
||||
|
||||
def test_reload_local_configuration(self):
|
||||
config = Config(config_file='postgres0.yml')
|
||||
with patch.object(Config, '_load_config_file', Mock(return_value={})):
|
||||
os.environ.update({
|
||||
'PATRONI_NAME': 'postgres0',
|
||||
'PATRONI_NAMESPACE': '/patroni/',
|
||||
'PATRONI_SCOPE': 'batman2',
|
||||
'PATRONI_RESTAPI_USERNAME': 'username',
|
||||
'PATRONI_RESTAPI_PASSWORD': 'password',
|
||||
'PATRONI_RESTAPI_LISTEN': '0.0.0.0:8008',
|
||||
'PATRONI_RESTAPI_CONNECT_ADDRESS': '127.0.0.1:8008',
|
||||
'PATRONI_RESTAPI_CERTFILE': '/certfile',
|
||||
'PATRONI_RESTAPI_KEYFILE': '/keyfile',
|
||||
'PATRONI_POSTGRESQL_LISTEN': '0.0.0.0:5432',
|
||||
'PATRONI_POSTGRESQL_CONNECT_ADDRESS': '127.0.0.1:5432',
|
||||
'PATRONI_POSTGRESQL_DATA_DIR': 'data/postgres0',
|
||||
'PATRONI_POSTGRESQL_PGPASS': '/tmp/pgpass0',
|
||||
'PATRONI_ETCD_HOST': '127.0.0.1:2379',
|
||||
'PATRONI_CONSUL_HOST': '127.0.0.1:8500',
|
||||
'PATRONI_ZOOKEEPER_HOSTS': 'host1,host2',
|
||||
'PATRONI_EXHIBITOR_HOSTS': 'host1,host2',
|
||||
'PATRONI_EXHIBITOR_PORT': '8181',
|
||||
'PATRONI_foo_HOSTS': '[host1,host2', # Exception in parse_list
|
||||
'PATRONI_SUPERUSER_USERNAME': 'postgres',
|
||||
'PATRONI_SUPERUSER_PASSWORD': 'zalando',
|
||||
'PATRONI_REPLICATION_USERNAME': 'replicator',
|
||||
'PATRONI_REPLICATION_PASSWORD': 'rep-pass',
|
||||
'PATRONI_admin_PASSWORD': 'admin',
|
||||
'PATRONI_admin_OPTIONS': 'createrole,createdb'
|
||||
})
|
||||
sys.argv = ['patroni.py', 'postgres0.yml']
|
||||
config = Config()
|
||||
with patch.object(Config, '_load_config_file', Mock(return_value={'restapi': {}})):
|
||||
with patch.object(Config, '_build_effective_configuration', Mock(side_effect=Exception)):
|
||||
self.assertRaises(Exception, config.reload_local_configuration, True)
|
||||
self.assertTrue(config.reload_local_configuration(True))
|
||||
|
||||
+8
-4
@@ -1,7 +1,8 @@
|
||||
import etcd
|
||||
import unittest
|
||||
import datetime
|
||||
import etcd
|
||||
import os
|
||||
import pytz
|
||||
import unittest
|
||||
|
||||
from mock import Mock, MagicMock, patch
|
||||
from patroni.config import Config
|
||||
@@ -49,7 +50,9 @@ def get_cluster_initialized_with_only_leader(failover=None):
|
||||
class MockPatroni(object):
|
||||
|
||||
def __init__(self, p, d):
|
||||
self.config = Config(config_env="""
|
||||
os.environ[Config.PATRONI_CONFIG_VARIABLE] = """
|
||||
restapi:
|
||||
listen: 0.0.0.0:8008
|
||||
bootstrap:
|
||||
users:
|
||||
replicator:
|
||||
@@ -66,7 +69,8 @@ zookeeper:
|
||||
exhibitor:
|
||||
hosts: [localhost]
|
||||
port: 8181
|
||||
""")
|
||||
"""
|
||||
self.config = Config()
|
||||
self.postgresql = p
|
||||
self.dcs = d
|
||||
self.api = Mock()
|
||||
|
||||
+2
-11
@@ -1,5 +1,4 @@
|
||||
import etcd
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
import unittest
|
||||
@@ -34,7 +33,8 @@ class TestPatroni(unittest.TestCase):
|
||||
RestApiServer.socket = 0
|
||||
with patch.object(etcd.Client, 'machines') as mock_machines:
|
||||
mock_machines.__get__ = Mock(return_value=['http://remotehost:2379'])
|
||||
self.p = Patroni('postgres0.yml')
|
||||
sys.argv = ['patroni.py', 'postgres0.yml']
|
||||
self.p = Patroni()
|
||||
|
||||
@patch('patroni.dcs.AbstractDCS.get_cluster', Mock(side_effect=[None, DCSError('foo'), None]))
|
||||
def test_load_dynamic_configuration(self):
|
||||
@@ -47,7 +47,6 @@ class TestPatroni(unittest.TestCase):
|
||||
@patch.object(etcd.Client, 'machines')
|
||||
def test_patroni_main(self, mock_machines):
|
||||
with patch('subprocess.call', Mock(return_value=1)):
|
||||
_main()
|
||||
sys.argv = ['patroni.py', 'postgres0.yml']
|
||||
|
||||
mock_machines.__get__ = Mock(return_value=['http://remotehost:2379'])
|
||||
@@ -55,14 +54,6 @@ class TestPatroni(unittest.TestCase):
|
||||
self.assertRaises(SleepException, _main)
|
||||
with patch.object(Patroni, 'run', Mock(side_effect=KeyboardInterrupt())):
|
||||
_main()
|
||||
sys.argv = ['patroni.py']
|
||||
# read the content of the yaml configuration file into the environment variable
|
||||
# in order to test how does patroni handle the configuration passed from the environment.
|
||||
with open('postgres0.yml', 'r') as f:
|
||||
os.environ[Patroni.PATRONI_CONFIG_VARIABLE] = f.read()
|
||||
with patch.object(Patroni, 'run', Mock(side_effect=SleepException())):
|
||||
self.assertRaises(SleepException, _main)
|
||||
del os.environ[Patroni.PATRONI_CONFIG_VARIABLE]
|
||||
|
||||
@patch('patroni.config.Config.save_cache', Mock())
|
||||
@patch('patroni.config.Config.reload_local_configuration', Mock(return_value=True))
|
||||
|
||||
Reference in New Issue
Block a user