mirror of
https://github.com/outbackdingo/patroni.git
synced 2026-08-25 14:53:37 +00:00
Merge branch 'master' of github.com:zalando/patroni into feature/disable-automatic-failover
This commit is contained in:
Alexander Kukushkin
+42
-6
@@ -1,12 +1,48 @@
|
||||
data/*
|
||||
*.pyc
|
||||
*.egg/
|
||||
*.egg-info/
|
||||
*.py[cod]
|
||||
|
||||
# vi(m) swap files:
|
||||
*.sw?
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Packages
|
||||
.cache/
|
||||
*.egg
|
||||
*.eggs
|
||||
*.egg-info
|
||||
dist
|
||||
build
|
||||
eggs
|
||||
parts
|
||||
bin
|
||||
var
|
||||
sdist
|
||||
develop-eggs
|
||||
.installed.cfg
|
||||
lib
|
||||
lib64
|
||||
|
||||
# Installer logs
|
||||
pip-log.txt
|
||||
|
||||
# Unit test / coverage reports
|
||||
.coverage
|
||||
.eggs/
|
||||
build/
|
||||
.tox
|
||||
nosetests.xml
|
||||
coverage.xml
|
||||
htmlcov
|
||||
junit.xml
|
||||
features/output
|
||||
dummy
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
|
||||
# Mr Developer
|
||||
.mr.developer.cfg
|
||||
.project
|
||||
.pydevproject
|
||||
|
||||
pgpass
|
||||
scm-source.json
|
||||
|
||||
@@ -69,6 +69,7 @@ PostgreSQL
|
||||
- **listen**: IP address + port that Postgres listens to; must be accessible from other nodes in the cluster, if you're using streaming replication. Multiple comma-separated addresses are permitted, as long as the port component is appended after to the last one with a colon, i.e. ``listen: 127.0.0.1,127.0.0.2:5432``. Patroni will use the first address from this list to establish local connections to the PostgreSQL node.
|
||||
- **pgpass**: path to the `.pgpass <https://www.postgresql.org/docs/current/static/libpq-pgpass.html>`__ password file. Patroni creates this file before executing pg\_basebackup and under some other circumstances. The location must be writable by Patroni.
|
||||
- **recovery\_conf**: additional configuration settings written to recovery.conf when configuring follower.
|
||||
- **custom_conf** : path to an optional custom ``postgresql.conf`` file, that will be used in place of ``postgresql.base.conf``. The file must exist on all cluster nodes, be readable by PostgreSQL and will be included from its location on the real ``postgresql.conf``. Note that Patroni will not monitor this file for changes, nor backup it. However, its settings can still be overriden by Patroni's own configuration facilities - see `dynamic configuration <https://github.com/zalando/patroni/blob/master/docs/dynamic_configuration.rst>`__ for details.
|
||||
- **parameters**: list of configuration settings for Postgres. Many of these are required for replication to work.
|
||||
- **pg\_ctl\_timeout**: How long should pg_ctl wait when doing ``start``, ``stop`` or ``restart``. Default value is 60 seconds.
|
||||
- **use\_pg\_rewind**: try to use pg\_rewind on the former leader when it joins cluster as a replica.
|
||||
|
||||
@@ -48,23 +48,24 @@ To be on the safe side parameters from the above lists are not written into ``po
|
||||
|
||||
When applying the local or dynamic configuration options, the following actions are taken:
|
||||
|
||||
- The node first checks if there is a postgresql.base.conf.
|
||||
- If it exists, it contains the renamed "original" configuration.
|
||||
- If it doesn't, the original postgresql.conf is taken and renamed to postgresql.base.conf.
|
||||
- The node first checks if there is a postgresql.base.conf or if the ``custom_conf`` parameter is set.
|
||||
- If the `custom_conf` parameter is set, it will take the file specified on it as a base configuration, ignoring `postgresql.base.conf` and `postgresql.conf`.
|
||||
- If the `custom_conf` parameter is not set and `postgresql.base.conf` exists, it contains the renamed "original" configuration and it will be used as a base configuration.
|
||||
- If there is no `custom_conf` nor `postgresql.base.conf`, the original postgresql.conf is taken and renamed to postgresql.base.conf.
|
||||
- The dynamic options (with the exceptions above) are dumped into the postgresql.conf and an include is set in
|
||||
postgresql.conf to postgresql.base.conf. Therefore, we would be able to apply new options without re-reading the configuration file to check if the include is present not.
|
||||
postgresql.conf to the used base configuration (either postgresql.base.conf or what is on ``custom_conf``). Therefore, we would be able to apply new options without re-reading the configuration file to check if the include is present not.
|
||||
- Some parameters that are essential for Patroni to manage the cluster are overridden using the command line.
|
||||
- If some of the options that require restart are changed (we should look at the context in pg_settings and at the actual
|
||||
values of those options), a pending_restart flag of a given node is set. This flag is reset on any restart.
|
||||
|
||||
The parameters would be applied in the following order (run-time are given the highest priority):
|
||||
|
||||
1. load parameters from file `postgresql.base.conf`
|
||||
1. load parameters from file `postgresql.base.conf` (or from a `custom_conf` file, if set)
|
||||
2. load parameters from file `postgresql.conf`
|
||||
3. load parameters from file `postgresql.auto.conf`
|
||||
4. run-time parameter using `-o --name=value`
|
||||
|
||||
This allows configuration for all the nodes (2), configuration for a specific node using `ALTER SYSTEM` (3) and ensures that parameters essential to the running of Patroni are enforced. (4)
|
||||
This allows configuration for all the nodes (2), configuration for a specific node using `ALTER SYSTEM` (3) and ensures that parameters essential to the running of Patroni are enforced (4), as well as leaves room for configuration tools that manage `postgresql.conf` directly without involving Patroni (1).
|
||||
|
||||
|
||||
Also, the following Patroni configuration options can be changed only dynamically:
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
Pause/Resume mode for the cluster
|
||||
=================================
|
||||
|
||||
The goal
|
||||
--------
|
||||
|
||||
Under certain circumstances Patroni needs to temporary step down from managing the cluster, while still retaining the cluster state in DCS. Possible use cases are uncommon activities on the cluster, such as major version upgrades or corruption recovery. During those activities nodes are often started and stopped for the reason unknown to Patroni, some nodes can be even temporary promoted, violating the assumption of running only one master. Therefore, Patroni needs to be able to "detach" from the running cluster, implementing an equivalent of the maintenance mode in Pacemaker.
|
||||
|
||||
|
||||
|
||||
The implementation
|
||||
------------------
|
||||
|
||||
When Patroni runs in a paused mode, it does not change the state of PostgreSQL, except for the following cases:
|
||||
|
||||
- For each node, the member key in DCS is updated with the current information about the cluster. This causes Patroni to run read-only queries on a member node if the member is running.
|
||||
|
||||
- For the Postgres master with the leader lock Patroni updates the lock. If the node with the leader lock stops being the master (i.e. is demoted manually), Patroni will release the lock instead of promoting the node back.
|
||||
|
||||
- Manual unscheduled restart, reinitialize and manual failover are allowed. Manual failover is only allowed if the node to failover to is specified. In the paused mode, manual failover does not require a running master node.
|
||||
|
||||
- If 'parallel' masters are detected by Patroni, it emits a warning, but does not demote the masters without the leader lock.
|
||||
|
||||
- If there is no leader lock in the cluster, the running master acquires the lock. If there is more than one master node, then the first master to acquire the lock wins. If there are no masters altogether, Patroni does not try to promote any replicas. There is an exception in this rule: if there is no leader lock because the old master has demoted itself due to the manual promotion, then only the candidate node mentioned in the promotion request may take the leader lock. When the new leader lock is granted (i.e. after promoting a replica manually), Patroni makes sure the replicas that were streaming from the previous leader will switch to the new one.
|
||||
|
||||
- When Postgres is stopped, Patroni does not try to start it. When Patroni is stopped, it does not to stop Postgres instance it is managing.
|
||||
|
||||
User guide
|
||||
----------
|
||||
|
||||
``patronictl`` supports ``pause`` and ``resume`` commands.
|
||||
|
||||
One can also issue a ``PATCH`` request to the ``{namespace}/{cluster}/config`` key with ``{"pause": true/false/null}``
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
pip install --ignore-installed setuptools==19.2 pyinstaller
|
||||
pyinstaller --clean --onefile patroni.spec
|
||||
@@ -0,0 +1,39 @@
|
||||
# -*- mode: python -*-
|
||||
|
||||
block_cipher = None
|
||||
|
||||
|
||||
def hiddenimports():
|
||||
import sys
|
||||
sys.path.insert(0, '.')
|
||||
try:
|
||||
import patroni.dcs
|
||||
return patroni.dcs.dcs_modules()
|
||||
finally:
|
||||
sys.path.pop(0)
|
||||
|
||||
|
||||
a = Analysis(['patroni/__main__.py'],
|
||||
pathex=[],
|
||||
binaries=None,
|
||||
datas=None,
|
||||
hiddenimports=hiddenimports(),
|
||||
hookspath=[],
|
||||
runtime_hooks=[],
|
||||
excludes=[],
|
||||
win_no_prefer_redirects=False,
|
||||
win_private_assemblies=False,
|
||||
cipher=block_cipher)
|
||||
|
||||
pyz = PYZ(a.pure, a.zipped_data, cipher=block_cipher)
|
||||
|
||||
exe = EXE(pyz,
|
||||
a.scripts,
|
||||
a.binaries,
|
||||
a.zipfiles,
|
||||
a.datas,
|
||||
name='patroni',
|
||||
debug=False,
|
||||
strip=False,
|
||||
upx=True,
|
||||
console=True)
|
||||
@@ -86,6 +86,10 @@ class Patroni(object):
|
||||
nap_time = self.next_run - current_time
|
||||
if nap_time <= 0:
|
||||
self.next_run = current_time
|
||||
# Release the GIL so we don't starve anyone waiting on async_executor lock
|
||||
time.sleep(0.001)
|
||||
# Warn user that Patroni is not keeping up
|
||||
logger.warning("Loop time exceeded, rescheduling immediately.")
|
||||
elif self.dcs.watch(nap_time):
|
||||
self.next_run = time.time()
|
||||
|
||||
|
||||
+1
-1
@@ -65,7 +65,7 @@ class Config(object):
|
||||
print('Usage: {0} config.yml'.format(sys.argv[0]))
|
||||
print('\tPatroni may also read the configuration from the {0} environment variable'.
|
||||
format(self.PATRONI_CONFIG_VARIABLE))
|
||||
exit(1)
|
||||
sys.exit(1)
|
||||
|
||||
self.__effective_configuration = self._build_effective_configuration({}, self._local_configuration)
|
||||
self._data_dir = self.__effective_configuration['postgresql']['data_dir']
|
||||
|
||||
+30
-14
@@ -6,6 +6,7 @@ import json
|
||||
import os
|
||||
import pkgutil
|
||||
import six
|
||||
import sys
|
||||
|
||||
from collections import namedtuple
|
||||
from patroni.exceptions import PatroniException
|
||||
@@ -31,22 +32,37 @@ def parse_connection_string(value):
|
||||
return conn_url, api_url
|
||||
|
||||
|
||||
def dcs_modules():
|
||||
"""Get names of DCS modules, depending on execution environment. If being packaged with PyInstaller,
|
||||
modules aren't discoverable dynamically by scanning source directory because `FrozenImporter` doesn't
|
||||
implement `iter_modules` method. But it is still possible to find all potential DCS modules by
|
||||
iterating through `toc`, which contains list of all "frozen" resources."""
|
||||
|
||||
dcs_dirname = os.path.dirname(__file__)
|
||||
module_prefix = __package__ + '.'
|
||||
|
||||
if getattr(sys, 'frozen', False):
|
||||
importer = pkgutil.get_importer(dcs_dirname)
|
||||
return [module for module in list(importer.toc) if module.startswith(module_prefix) and module.count('.') == 2]
|
||||
else:
|
||||
return [module_prefix + name for _, name, is_pkg in pkgutil.iter_modules([dcs_dirname]) if not is_pkg]
|
||||
|
||||
|
||||
def get_dcs(config):
|
||||
available_implementations = set()
|
||||
for _, module_name, is_pkg in pkgutil.iter_modules([os.path.dirname(__file__)]):
|
||||
if not is_pkg:
|
||||
module = importlib.import_module(__package__ + '.' + module_name)
|
||||
for name in filter(lambda name: not name.startswith('__'), dir(module)): # iterate through module content
|
||||
value = getattr(module, name)
|
||||
name = name.lower()
|
||||
# try to find implementation of AbstractDCS interface, class name must match with module_name
|
||||
if inspect.isclass(value) and issubclass(value, AbstractDCS) and name == module_name:
|
||||
available_implementations.add(name)
|
||||
if name in config: # which has configuration section in the config file
|
||||
# propagate some parameters
|
||||
config[name].update({p: config[p] for p in ('namespace', 'name', 'scope',
|
||||
'loop_wait', 'ttl', 'retry_timeout') if p in config})
|
||||
return value(config[name])
|
||||
for module_name in dcs_modules():
|
||||
module = importlib.import_module(module_name)
|
||||
for name in filter(lambda name: not name.startswith('__'), dir(module)): # iterate through module content
|
||||
value = getattr(module, name)
|
||||
name = name.lower()
|
||||
# try to find implementation of AbstractDCS interface, class name must match with module_name
|
||||
if inspect.isclass(value) and issubclass(value, AbstractDCS) and __package__ + '.' + name == module_name:
|
||||
available_implementations.add(name)
|
||||
if name in config: # which has configuration section in the config file
|
||||
# propagate some parameters
|
||||
config[name].update({p: config[p] for p in ('namespace', 'name', 'scope',
|
||||
'loop_wait', 'ttl', 'retry_timeout') if p in config})
|
||||
return value(config[name])
|
||||
raise PatroniException("""Can not find suitable configuration of distributed configuration store
|
||||
Available implementations: """ + ', '.join(available_implementations))
|
||||
|
||||
|
||||
+3
-2
@@ -292,7 +292,8 @@ class Etcd(AbstractDCS):
|
||||
if leader:
|
||||
member = Member(-1, leader.value, None, {})
|
||||
member = ([m for m in members if m.name == leader.value] or [member])[0]
|
||||
leader = Leader(leader.modifiedIndex, leader.ttl, member)
|
||||
index = result.etcd_index if result.etcd_index > leader.modifiedIndex else leader.modifiedIndex + 1
|
||||
leader = Leader(index, leader.ttl, member)
|
||||
|
||||
# failover key
|
||||
failover = nodes.get(self._FAILOVER)
|
||||
@@ -371,7 +372,7 @@ class Etcd(AbstractDCS):
|
||||
|
||||
while timeout >= 1: # when timeout is too small urllib3 doesn't have enough time to connect
|
||||
try:
|
||||
self._client.watch(self.leader_path, index=cluster.leader.index + 1, timeout=timeout + 0.5)
|
||||
self._client.watch(self.leader_path, index=cluster.leader.index, timeout=timeout + 0.5)
|
||||
# Synchronous work of all cluster members with etcd is less expensive
|
||||
# than reestablishing http connection every time from every replica.
|
||||
return True
|
||||
|
||||
+1
-3
@@ -625,9 +625,7 @@ class Ha(object):
|
||||
return 'postgres is not running'
|
||||
|
||||
# try to start dead postgres
|
||||
msg = self.recover()
|
||||
if msg is not None:
|
||||
return msg
|
||||
return self.recover()
|
||||
|
||||
try:
|
||||
if self.cluster.is_unlocked():
|
||||
|
||||
+25
-14
@@ -94,15 +94,12 @@ class Postgresql(object):
|
||||
self._schedule_load_slots = self.use_slots
|
||||
|
||||
self._pgpass = config.get('pgpass') or os.path.join(os.path.expanduser('~'), 'pgpass')
|
||||
self.callback = config.get('callbacks') or {}
|
||||
self.__cb_called = False
|
||||
config_base_name = config.get('config_base_name', 'postgresql')
|
||||
self._postgresql_conf = os.path.join(self._data_dir, config_base_name + '.conf')
|
||||
self._postgresql_base_conf_name = config_base_name + '.base.conf'
|
||||
self._postgresql_base_conf = os.path.join(self._data_dir, self._postgresql_base_conf_name)
|
||||
self._recovery_conf = os.path.join(self._data_dir, 'recovery.conf')
|
||||
self._configuration_to_save = (self._postgresql_conf, self._postgresql_base_conf,
|
||||
os.path.join(self._data_dir, 'pg_hba.conf'))
|
||||
self._postmaster_pid = os.path.join(self._data_dir, 'postmaster.pid')
|
||||
self._trigger_file = config.get('recovery_conf', {}).get('trigger_file') or 'promote'
|
||||
self._trigger_file = os.path.abspath(os.path.join(self._data_dir, self._trigger_file))
|
||||
@@ -124,10 +121,23 @@ class Postgresql(object):
|
||||
self.set_role('master' if self.is_leader() else 'replica')
|
||||
self._write_postgresql_conf() # we are "joining" already running postgres
|
||||
|
||||
@property
|
||||
def _configuration_to_save(self):
|
||||
configuration = [self._postgresql_conf]
|
||||
if 'custom_conf' not in self.config:
|
||||
configuration.append(self._postgresql_base_conf)
|
||||
if not self.config['parameters'].get('hba_file'):
|
||||
configuration.append(os.path.join(self._data_dir, 'pg_hba.conf'))
|
||||
return configuration
|
||||
|
||||
@property
|
||||
def use_slots(self):
|
||||
return self._use_slots and self._major_version >= 9.4
|
||||
|
||||
@property
|
||||
def callback(self):
|
||||
return self.config.get('callbacks') or {}
|
||||
|
||||
def _version_file_exists(self):
|
||||
return not self.data_directory_empty() and os.path.isfile(self._version_file)
|
||||
|
||||
@@ -416,7 +426,7 @@ class Postgresql(object):
|
||||
# If there is no configuration key, or no value is specified, use basebackup
|
||||
replica_methods = self.config.get('create_replica_method') or ['basebackup']
|
||||
|
||||
if clone_member:
|
||||
if clone_member and clone_member.conn_url:
|
||||
r = clone_member.conn_kwargs(self._replication)
|
||||
connstring = 'postgres://{user}@{host}:{port}/{database}'.format(**r)
|
||||
# add the credentials to connect to the replica origin to pgpass.
|
||||
@@ -622,12 +632,12 @@ class Postgresql(object):
|
||||
|
||||
def _write_postgresql_conf(self):
|
||||
# rename the original configuration if it is necessary
|
||||
if not os.path.exists(self._postgresql_base_conf):
|
||||
if 'custom_conf' not in self.config and not os.path.exists(self._postgresql_base_conf):
|
||||
os.rename(self._postgresql_conf, self._postgresql_base_conf)
|
||||
|
||||
with open(self._postgresql_conf, 'w') as f:
|
||||
f.write('# Do not edit this file manually!\n# It will be overwritten by Patroni!\n')
|
||||
f.write("include '{0}'\n\n".format(self._postgresql_base_conf_name))
|
||||
f.write("include '{0}'\n\n".format(self.config.get('custom_conf') or self._postgresql_base_conf_name))
|
||||
for name, value in sorted(self._server_parameters.items()):
|
||||
if name not in self.CMDLINE_OPTIONS:
|
||||
f.write("{0} = '{1}'\n".format(name, value))
|
||||
@@ -886,7 +896,7 @@ $$""".format(name, ' '.join(options)), name, password, password)
|
||||
|
||||
def load_replication_slots(self):
|
||||
if self.use_slots and self._schedule_load_slots:
|
||||
cursor = self.query("SELECT slot_name FROM pg_replication_slots WHERE slot_type='physical'")
|
||||
cursor = self._query("SELECT slot_name FROM pg_replication_slots WHERE slot_type='physical'")
|
||||
self._replication_slots = [r[0] for r in cursor]
|
||||
self._schedule_load_slots = False
|
||||
|
||||
@@ -926,19 +936,20 @@ $$""".format(name, ' '.join(options)), name, password, password)
|
||||
|
||||
# drop unused slots
|
||||
for slot in set(self._replication_slots) - slots:
|
||||
self.query("""SELECT pg_drop_replication_slot(%s)
|
||||
WHERE EXISTS(SELECT 1 FROM pg_replication_slots
|
||||
WHERE slot_name = %s AND NOT active)""", slot, slot)
|
||||
self._query("""SELECT pg_drop_replication_slot(%s)
|
||||
WHERE EXISTS(SELECT 1 FROM pg_replication_slots
|
||||
WHERE slot_name = %s AND NOT active)""", slot, slot)
|
||||
|
||||
# create new slots
|
||||
for slot in slots - set(self._replication_slots):
|
||||
self.query("""SELECT pg_create_physical_replication_slot(%s)
|
||||
WHERE NOT EXISTS (SELECT 1 FROM pg_replication_slots
|
||||
WHERE slot_name = %s)""", slot, slot)
|
||||
self._query("""SELECT pg_create_physical_replication_slot(%s)
|
||||
WHERE NOT EXISTS (SELECT 1 FROM pg_replication_slots
|
||||
WHERE slot_name = %s)""", slot, slot)
|
||||
|
||||
self._replication_slots = slots
|
||||
except psycopg2.Error:
|
||||
except Exception:
|
||||
logger.exception('Exception when changing replication slots')
|
||||
self._schedule_load_slots = True
|
||||
|
||||
def last_operation(self):
|
||||
return str(self.xlog_position())
|
||||
|
||||
@@ -73,7 +73,7 @@ class WALERestore(object):
|
||||
# base_00000001000000000000007F_00000040 2015-05-18T10:13:25.000Z
|
||||
# 20310671 00000001000000000000007F 00000040
|
||||
# 00000001000000000000007F 00000240
|
||||
backup_strings = latest_backup.splitlines() if latest_backup else ()
|
||||
backup_strings = latest_backup.decode('utf-8').splitlines() if latest_backup else ()
|
||||
if len(backup_strings) != 2:
|
||||
return False
|
||||
|
||||
|
||||
+3
-1
@@ -103,7 +103,9 @@ def etcd_read(self, key, **kwargs):
|
||||
"expiration": "2015-05-15T09:11:09.611860899Z", "ttl": 30,
|
||||
"modifiedIndex": 20730, "createdIndex": 20730}],
|
||||
"modifiedIndex": 1581, "createdIndex": 1581}], "modifiedIndex": 1581, "createdIndex": 1581}}
|
||||
return etcd.EtcdResult(**response)
|
||||
result = etcd.EtcdResult(**response)
|
||||
result.etcd_index = 0
|
||||
return result
|
||||
|
||||
|
||||
class SleepException(Exception):
|
||||
|
||||
@@ -14,6 +14,11 @@ from test_etcd import SleepException, etcd_read, etcd_write
|
||||
from test_postgresql import Postgresql, psycopg2_connect
|
||||
|
||||
|
||||
class MockFrozenImporter(object):
|
||||
|
||||
toc = set(['patroni.dcs.etcd'])
|
||||
|
||||
|
||||
@patch('time.sleep', Mock())
|
||||
@patch('subprocess.call', Mock(return_value=0))
|
||||
@patch('psycopg2.connect', psycopg2_connect)
|
||||
@@ -27,6 +32,8 @@ from test_postgresql import Postgresql, psycopg2_connect
|
||||
@patch.object(etcd.Client, 'read', etcd_read)
|
||||
class TestPatroni(unittest.TestCase):
|
||||
|
||||
@patch('pkgutil.get_importer', Mock(return_value=MockFrozenImporter()))
|
||||
@patch('sys.frozen', Mock(return_value=True), create=True)
|
||||
@patch.object(etcd.Client, 'read', etcd_read)
|
||||
def setUp(self):
|
||||
RestApiServer._BaseServer__is_shut_down = Mock()
|
||||
|
||||
@@ -311,11 +311,9 @@ class TestPostgresql(unittest.TestCase):
|
||||
def test_sync_replication_slots(self):
|
||||
self.p.start()
|
||||
cluster = Cluster(True, None, self.leader, 0, [self.me, self.other, self.leadermem], None)
|
||||
with mock.patch('patroni.postgresql.Postgresql._query', Mock(side_effect=psycopg2.OperationalError)):
|
||||
self.p.sync_replication_slots(cluster)
|
||||
self.p.sync_replication_slots(cluster)
|
||||
self.p.query = Mock(side_effect=psycopg2.OperationalError)
|
||||
self.p.schedule_load_slots = True
|
||||
self.p.sync_replication_slots(cluster)
|
||||
self.p.schedule_load_slots = False
|
||||
with mock.patch('patroni.postgresql.Postgresql.role', new_callable=PropertyMock(return_value='replica')):
|
||||
self.p.sync_replication_slots(cluster)
|
||||
with mock.patch('patroni.postgresql.logger.error', new_callable=Mock()) as errorlog_mock:
|
||||
|
||||
+10
-28
@@ -6,30 +6,10 @@ from mock import MagicMock, patch, PropertyMock
|
||||
from patroni.scripts.wale_restore import WALERestore, main as _main
|
||||
|
||||
|
||||
def fake_backup_data(self, *args, **kwargs):
|
||||
""" return the fake result of WAL-E backup-list"""
|
||||
return """name last_modified expanded_size_bytes wal_segment_backup_start wal_segment_offset_backup_start wal_segment_backup_stop wal_segment_offset_backup_stop
|
||||
base_00000001000000000000007F_00000040 2015-05-18T10:13:25.000Z 167772160 00000001000000000000007F 00000040 00000001000000000000007F 00000240
|
||||
"""
|
||||
|
||||
|
||||
def fake_backup_data_2(self, *args, **kwargs):
|
||||
""" return the fake result of WAL-E backup-list"""
|
||||
return """name last_modified expanded_size_bytes wal_segment_backup_start wal_segment_offset_backup_start wal_segment_backup_stop wal_segment_offset_backup_stop """
|
||||
|
||||
|
||||
def fake_backup_data_3(self, *args, **kwargs):
|
||||
""" return the fake result of WAL-E backup-list"""
|
||||
return """name last_modified expanded_size_bytes wal_segment_backup_start wal_segment_offset_backup_start wal_segment_backup_stop
|
||||
base_00000001000000000000007F_00000040 2015-05-18T10:13:25.000Z 167772160 00000001000000000000007F 00000040 00000001000000000000007F 00000240
|
||||
"""
|
||||
|
||||
|
||||
def fake_backup_data_4(self, *args, **kwargs):
|
||||
""" return the fake result of WAL-E backup-list"""
|
||||
return """name last_modified expanded_size_foo wal_segment_backup_start wal_segment_offset_backup_start wal_segment_backup_stop wal_segment_offset_backup_stop
|
||||
base_00000001000000000000007F_00000040 2015-05-18T10:13:25.000Z 167772160 00000001000000000000007F 00000040 00000001000000000000007F 00000240
|
||||
"""
|
||||
wale_output = b'name last_modified expanded_size_bytes wal_segment_backup_start ' +\
|
||||
b'wal_segment_offset_backup_start wal_segment_backup_stop wal_segment_offset_backup_stop\n' +\
|
||||
b'base_00000001000000000000007F_00000040 2015-05-18T10:13:25.000Z 167772160 ' +\
|
||||
b'00000001000000000000007F 00000040 00000001000000000000007F 00000240\n'
|
||||
|
||||
|
||||
@patch('os.access', MagicMock(return_value=True))
|
||||
@@ -39,7 +19,7 @@ base_00000001000000000000007F_00000040 2015-05-18T10:13:25.000Z 167772160 000
|
||||
@patch('psycopg2.extensions.cursor', MagicMock(autospec=True))
|
||||
@patch('psycopg2.extensions.connection', MagicMock(autospec=True))
|
||||
@patch('psycopg2.connect', MagicMock(autospec=True))
|
||||
@patch('subprocess.check_output', MagicMock(side_effect=fake_backup_data))
|
||||
@patch('subprocess.check_output', MagicMock(return_value=wale_output))
|
||||
class TestWALERestore(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
@@ -50,11 +30,13 @@ class TestWALERestore(unittest.TestCase):
|
||||
self.assertFalse(self.wale_restore.should_use_s3_to_create_replica())
|
||||
with patch('subprocess.check_output', MagicMock(side_effect=subprocess.CalledProcessError(1, "cmd", "foo"))):
|
||||
self.assertFalse(self.wale_restore.should_use_s3_to_create_replica())
|
||||
with patch('subprocess.check_output', MagicMock(side_effect=fake_backup_data_2)):
|
||||
with patch('subprocess.check_output', MagicMock(return_value=wale_output.split(b'\n')[0])):
|
||||
self.assertFalse(self.wale_restore.should_use_s3_to_create_replica())
|
||||
with patch('subprocess.check_output', MagicMock(side_effect=fake_backup_data_3)):
|
||||
with patch('subprocess.check_output',
|
||||
MagicMock(return_value=wale_output.replace(b' wal_segment_offset_backup_stop', b''))):
|
||||
self.assertFalse(self.wale_restore.should_use_s3_to_create_replica())
|
||||
with patch('subprocess.check_output', MagicMock(side_effect=fake_backup_data_4)):
|
||||
with patch('subprocess.check_output',
|
||||
MagicMock(return_value=wale_output.replace(b'expanded_size_bytes', b'expanded_size_foo'))):
|
||||
self.assertFalse(self.wale_restore.should_use_s3_to_create_replica())
|
||||
|
||||
self.wale_restore.should_use_s3_to_create_replica()
|
||||
|
||||
Reference in New Issue
Block a user