diff --git a/docs/ENVIRONMENT.rst b/docs/ENVIRONMENT.rst index c2579f0e..e98f49c6 100644 --- a/docs/ENVIRONMENT.rst +++ b/docs/ENVIRONMENT.rst @@ -54,7 +54,8 @@ Consul - **PATRONI\_CONSUL\_DC**: (optional) Datacenter to communicate with. By default the datacenter of the host is used. - **PATRONI\_CONSUL\_CONSISTENCY**: (optional) Select consul consistency mode. Possible values are ``default``, ``consistent``, or ``stale`` (more details in `consul API reference `__) - **PATRONI\_CONSUL\_CHECKS**: (optional) list of Consul health checks used for the session. By default an empty list is used. -- **PATRONI\_CONSUL\_REGISTER\_SERVICE**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, replica or standby-leader depending on the node's role. Defaults to **false** +- **PATRONI\_CONSUL\_REGISTER\_SERVICE**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, primary, replica, or standby-leader depending on the node's role. Defaults to **false** +- **PATRONI\_CONSUL\_SERVICE\_TAGS**: (optional) additional static tags to add to the Consul service apart from the role (``master``/``primary``/``replica``/``standby-leader``). By default an empty list is used. - **PATRONI\_CONSUL\_SERVICE\_CHECK\_INTERVAL**: (optional) how often to perform health check against registered url - **PATRONI\_CONSUL\_SERVICE\_CHECK\_TLS\_SERVER\_NAME**: (optional) overide SNI host when connecting via TLS, see also `consul agent check API reference `__. diff --git a/docs/SETTINGS.rst b/docs/SETTINGS.rst index e5d26937..14e8cc0d 100644 --- a/docs/SETTINGS.rst +++ b/docs/SETTINGS.rst @@ -17,8 +17,8 @@ Dynamic configuration is stored in the DCS (Distributed Configuration Store) and - **maximum\_lag\_on\_failover**: the maximum bytes a follower may lag to be able to participate in leader election. - **maximum\_lag\_on\_syncnode**: the maximum bytes a synchronous follower may lag before it is considered as an unhealthy candidate and swapped by healthy asynchronous follower. Patroni utilize the max replica lsn if there is more than one follower, otherwise it will use leader's current wal lsn. Default is -1, Patroni will not take action to swap synchronous unhealthy follower when the value is set to 0 or below. Please set the value high enough so Patroni won't swap synchrounous follower fequently during high transaction volume. - **max\_timelines\_history**: maximum number of timeline history items kept in DCS. Default value: 0. When set to 0, it keeps the full history in DCS. -- **master\_start\_timeout**: the amount of time a primary is allowed to recover from failures before failover is triggered (in seconds). Default is 300 seconds. When set to 0 failover is done immediately after a crash is detected if possible. When using asynchronous replication a failover can cause lost transactions. Worst case failover time for primary failure is: loop\_wait + master\_start\_timeout + loop\_wait, unless master\_start\_timeout is zero, in which case it's just loop\_wait. Set the value according to your durability/availability tradeoff. -- **master\_stop\_timeout**: The number of seconds Patroni is allowed to wait when stopping Postgres and effective only when synchronous_mode is enabled. When set to > 0 and the synchronous_mode is enabled, Patroni sends SIGKILL to the postmaster if the stop operation is running for more than the value set by master_stop_timeout. Set the value according to your durability/availability tradeoff. If the parameter is not set or set <= 0, master_stop_timeout does not apply. +- **primary\_start\_timeout**: the amount of time a primary is allowed to recover from failures before failover is triggered (in seconds). Default is 300 seconds. When set to 0 failover is done immediately after a crash is detected if possible. When using asynchronous replication a failover can cause lost transactions. Worst case failover time for primary failure is: loop\_wait + primary\_start\_timeout + loop\_wait, unless primary\_start\_timeout is zero, in which case it's just loop\_wait. Set the value according to your durability/availability tradeoff. +- **primary\_stop\_timeout**: The number of seconds Patroni is allowed to wait when stopping Postgres and effective only when synchronous_mode is enabled. When set to > 0 and the synchronous_mode is enabled, Patroni sends SIGKILL to the postmaster if the stop operation is running for more than the value set by primary\_stop\_timeout. Set the value according to your durability/availability tradeoff. If the parameter is not set or set <= 0, primary\_stop\_timeout does not apply. - **synchronous\_mode**: turns on synchronous replication mode. In this mode a replica will be chosen as synchronous and only the latest leader and synchronous replica are able to participate in leader election. Synchronous mode makes sure that successfully committed transactions will not be lost at failover, at the cost of losing availability for writes when Patroni cannot ensure transaction durability. See :ref:`replication modes documentation ` for details. - **synchronous\_mode\_strict**: prevents disabling synchronous replication if no synchronous replicas are available, blocking all client writes to the primary. See :ref:`replication modes documentation ` for details. - **failsafe\_mode**: Enables :ref:`DCS Failsafe Mode `. Defaults to `false`. @@ -139,8 +139,8 @@ Most of the parameters are optional, but you have to specify one of the **host** - **dc**: (optional) Datacenter to communicate with. By default the datacenter of the host is used. - **consistency**: (optional) Select consul consistency mode. Possible values are ``default``, ``consistent``, or ``stale`` (more details in `consul API reference `__) - **checks**: (optional) list of Consul health checks used for the session. By default an empty list is used. -- **register\_service**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, replica or standby-leader depending on the node's role. Defaults to **false**. -- **service\_tags**: (optional) additional static tags to add to the Consul service apart from the role (``master``/``replica``/``standby-leader``). By default an empty list is used. +- **register\_service**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, primary, replica, or standby-leader depending on the node's role. Defaults to **false**. +- **service\_tags**: (optional) additional static tags to add to the Consul service apart from the role (``master``/``primary``/``replica``/``standby-leader``). By default an empty list is used. - **service\_check\_interval**: (optional) how often to perform health check against registered url. Defaults to '5s'. - **service\_check\_tls\_server\_name**: (optional) overide SNI host when connecting via TLS, see also `consul agent check API reference `__. diff --git a/docs/citus.rst b/docs/citus.rst index a48df30c..0c349bb7 100644 --- a/docs/citus.rst +++ b/docs/citus.rst @@ -127,7 +127,7 @@ An example of ``patronictl switchover`` on the worker cluster:: | 2 | work2-2 | 172.27.0.7 | Leader | running | 1 | | +-------+---------+-------------+--------------+---------+----+-----------+ Citus group: 2 - Master [work2-2]: + Primary [work2-2]: Candidate ['work2-1'] []: When should the switchover take place (e.g. 2022-12-22T08:02 ) [now]: Current cluster topology @@ -137,7 +137,7 @@ An example of ``patronictl switchover`` on the worker cluster:: | work2-1 | 172.27.0.5 | Sync Standby | running | 1 | 0 | | work2-2 | 172.27.0.7 | Leader | running | 1 | | +---------+------------+--------------+---------+----+-----------+ - Are you sure you want to switchover cluster demo, demoting current master work2-2? [y/N]: y + Are you sure you want to switchover cluster demo, demoting current primary work2-2? [y/N]: y 2022-12-22 07:02:40.33003 Successfully switched over to "work2-1" + Citus cluster: demo (group: 2, 7179854924063375386) ------+ | Member | Host | Role | State | TL | Lag in MB | diff --git a/docs/kubernetes.rst b/docs/kubernetes.rst index c0c24190..699321fa 100644 --- a/docs/kubernetes.rst +++ b/docs/kubernetes.rst @@ -23,10 +23,7 @@ Use ConfigMaps In this mode, Patroni will create ConfigMaps instead of Endpoints and store keys inside meta-data of those ConfigMaps. Changing the leader takes at least two updates, one to the leader ConfigMap and another to the respective Endpoint. -There are two ways to direct the traffic to the Postgres leader: - -- use the `callback script `_ provided by Patroni -- configure the Kubernetes Postgres service to use the label selector with the `role_label` (configured in patroni configuration). +To direct the traffic to the Postgres leader you need to configure the Kubernetes Postgres service to use the label selector with the `role_label` (configured in patroni configuration). Note that in some cases, for instance, when running on OpenShift, there is no alternative to using ConfigMaps. @@ -39,7 +36,7 @@ Examples -------- - The `kubernetes `__ folder of the Patroni repository contains - examples of the Docker image, the Kubernetes manifest and the callback script in order to test Patroni Kubernetes setup. + examples of the Docker image, and the Kubernetes manifest to test Patroni Kubernetes setup. Note that in the current state it will not be able to use PersistentVolumes because of permission issues. - You can find the full-featured Docker image that can use Persistent Volumes in the diff --git a/docs/replica_bootstrap.rst b/docs/replica_bootstrap.rst index 7b17cd85..42a7d3b0 100644 --- a/docs/replica_bootstrap.rst +++ b/docs/replica_bootstrap.rst @@ -77,7 +77,7 @@ scripts to clone a new replica. Those are configured in the ``postgresql`` confi command: keep_data: True no_params: True - no_master: 1 + no_leader: 1 example: wal_e @@ -89,7 +89,7 @@ example: wal_e - basebackup wal_e: command: patroni_wale_restore - no_master: 1 + no_leader: 1 envdir: {{WALE_ENV_DIR}} use_iam: 1 basebackup: @@ -126,7 +126,7 @@ to execute and any custom parameters that should be passed to that command. All Connection string to connect to the cluster member to clone from (primary or other replica). The user in the connection string can execute SQL and replication protocol commands. -A special ``no_master`` parameter, if defined, allows Patroni to call the replica creation method even if there is no +A special ``no_leader`` parameter, if defined, allows Patroni to call the replica creation method even if there is no running leader or replicas. In that case, an empty string will be passed in a connection string. This is useful for restoring the formerly running cluster from the binary backup. diff --git a/docs/rest_api.rst b/docs/rest_api.rst index e6f9ecb3..88aa6ca5 100644 --- a/docs/rest_api.rst +++ b/docs/rest_api.rst @@ -12,7 +12,6 @@ For all health check ``GET`` requests Patroni returns a JSON document with the s - The following requests to Patroni REST API will return HTTP status code **200** only when the Patroni node is running as the primary with leader lock: - ``GET /`` - - ``GET /master`` - ``GET /primary`` - ``GET /read-write`` @@ -33,7 +32,6 @@ For all health check ``GET`` requests Patroni returns a JSON document with the s In the following requests, since we are checking for the leader or standby-leader status, Patroni doesn't apply any of the user defined tags and they will be ignored. - ``GET /?tag_key1=value1&tag_key2=value2`` - - ``GET /master?tag_key1=value1&tag_key2=value2`` - ``GET /leader?tag_key1=value1&tag_key2=value2`` - ``GET /primary?tag_key1=value1&tag_key2=value2`` - ``GET /read-write?tag_key1=value1&tag_key2=value2`` @@ -368,7 +366,7 @@ Restart endpoint - **restart_pending**: boolean, if set to ``true`` Patroni will restart PostgreSQL only when restart is pending in order to apply some changes in the PostgreSQL config. - **role**: perform restart only if the current role of the node matches with the role from the POST request. - **postgres_version**: perform restart only if the current version of postgres is smaller than specified in the POST request. - - **timeout**: how long we should wait before PostgreSQL starts accepting connections. Overrides ``master_start_timeout``. + - **timeout**: how long we should wait before PostgreSQL starts accepting connections. Overrides ``primary_start_timeout``. - **schedule**: timestamp with time zone, schedule the restart somewhere in the future. - ``DELETE /restart``: delete the scheduled restart diff --git a/extras/confd/templates/haproxy.tmpl b/extras/confd/templates/haproxy.tmpl index 31f88194..edbf224c 100644 --- a/extras/confd/templates/haproxy.tmpl +++ b/extras/confd/templates/haproxy.tmpl @@ -16,9 +16,9 @@ listen stats stats enable stats uri / -listen master +listen primary bind *:5000 - option httpchk HEAD /master + option httpchk HEAD /primary http-check expect status 200 default-server inter 3s fall 3 rise 2 on-marked-down shutdown-sessions {{range gets "/members/*"}} server {{base .Key}} {{$data := json .Value}}{{base (replace (index (split $data.conn_url "/") 2) "@" "/" -1)}} maxconn 100 check port {{index (split (index (split $data.api_url "/") 2) ":") 1}} diff --git a/features/basic_replication.feature b/features/basic_replication.feature index 96ad5a04..6acffe5c 100644 --- a/features/basic_replication.feature +++ b/features/basic_replication.feature @@ -76,7 +76,7 @@ Feature: basic replication Then postgres1 is a leader after 10 seconds And postgres1 role is the primary after 10 seconds - Scenario: check rejoin of the former master with pg_rewind + Scenario: check rejoin of the former primary with pg_rewind Given I add the table splitbrain to postgres0 And I start postgres0 Then postgres0 role is the secondary after 20 seconds diff --git a/features/dcs_failsafe_mode.feature b/features/dcs_failsafe_mode.feature index 10f103d6..4a34b754 100644 --- a/features/dcs_failsafe_mode.feature +++ b/features/dcs_failsafe_mode.feature @@ -27,7 +27,7 @@ Feature: dcs failsafe mode Given DCS is up When I do a backup of postgres0 And I shut down postgres0 - When I start postgres1 in a cluster batman from backup with no_master + When I start postgres1 in a cluster batman from backup with no_leader And I sleep for 2 seconds Then postgres1 role is the replica after 12 seconds @@ -55,7 +55,7 @@ Feature: dcs failsafe mode And logical slot dcs_slot_0 is in sync between postgres0 and postgres1 after 20 seconds @dcs-failsafe - Scenario: check master is demoted when one replica is shut down and DCS is down + Scenario: check primary is demoted when one replica is shut down and DCS is down Given DCS is down And I kill postgres1 And I kill postmaster on postgres1 diff --git a/features/environment.py b/features/environment.py index 0c1e2c1f..5b05887d 100644 --- a/features/environment.py +++ b/features/environment.py @@ -905,7 +905,7 @@ class PatroniPoolController(object): } self.start(name, custom_config=custom_config) - def bootstrap_from_backup_no_master(self, name, cluster_name): + def bootstrap_from_backup_no_leader(self, name, cluster_name): custom_config = { 'scope': cluster_name, 'postgresql': { @@ -914,11 +914,11 @@ class PatroniPoolController(object): '--dirname {} --filename %f --pathname %p').format( os.path.join(self.patroni_path, 'data', 'wal_archive').replace('\\', '/')) }, - 'create_replica_methods': ['no_master_bootstrap'], - 'no_master_bootstrap': { + 'create_replica_methods': ['no_leader_bootstrap'], + 'no_leader_bootstrap': { 'command': (self.BACKUP_RESTORE_SCRIPT + ' --sourcedir=' + os.path.join(self.patroni_path, 'data', 'basebackup').replace('\\', '/')), - 'no_master': '1' + 'no_leader': '1' } } } diff --git a/features/ignored_slots.feature b/features/ignored_slots.feature index abaaef74..cb8a4c70 100644 --- a/features/ignored_slots.feature +++ b/features/ignored_slots.feature @@ -52,7 +52,7 @@ Feature: ignored slots And postgres1 has a logical replication slot named unmanaged_slot_3 with the test_decoding plugin And postgres1 does not have a logical replication slot named dummy_slot - # 3. After a failover the server (now a master) still has the slot. + # 3. After a failover the server (now a primary) still has the slot. When I shut down postgres0 Then "members/postgres1" key in DCS has role=master after 3 seconds And postgres1 has a logical replication slot named unmanaged_slot_0 with the test_decoding plugin diff --git a/features/patroni_api.feature b/features/patroni_api.feature index 58112959..7c04872e 100644 --- a/features/patroni_api.feature +++ b/features/patroni_api.feature @@ -94,11 +94,11 @@ Scenario: check the switchover via the API in the pause mode And postgres0 role is the secondary after 10 seconds And replication works from postgres1 to postgres0 after 20 seconds And "members/postgres0" key in DCS has state=running after 10 seconds - When I issue a GET request to http://127.0.0.1:8008/master + When I issue a GET request to http://127.0.0.1:8008/primary Then I receive a response code 503 When I issue a GET request to http://127.0.0.1:8008/replica Then I receive a response code 200 - When I issue a GET request to http://127.0.0.1:8009/master + When I issue a GET request to http://127.0.0.1:8009/primary Then I receive a response code 200 When I issue a GET request to http://127.0.0.1:8009/replica Then I receive a response code 503 @@ -116,11 +116,11 @@ Scenario: check the scheduled switchover And postgres1 role is the secondary after 10 seconds And replication works from postgres0 to postgres1 after 25 seconds And "members/postgres1" key in DCS has state=running after 10 seconds - When I issue a GET request to http://127.0.0.1:8008/master + When I issue a GET request to http://127.0.0.1:8008/primary Then I receive a response code 200 When I issue a GET request to http://127.0.0.1:8008/replica Then I receive a response code 503 - When I issue a GET request to http://127.0.0.1:8009/master + When I issue a GET request to http://127.0.0.1:8009/primary Then I receive a response code 503 When I issue a GET request to http://127.0.0.1:8009/replica Then I receive a response code 200 diff --git a/features/standby_cluster.feature b/features/standby_cluster.feature index 5333b5d6..4e3bd5f0 100644 --- a/features/standby_cluster.feature +++ b/features/standby_cluster.feature @@ -35,7 +35,7 @@ Feature: standby cluster When I add the table foo to postgres0 Then table foo is present on postgres1 after 20 seconds And I sleep for 3 seconds - When I issue a GET request to http://127.0.0.1:8009/master + When I issue a GET request to http://127.0.0.1:8009/primary Then I receive a response code 503 When I issue a GET request to http://127.0.0.1:8009/standby_leader Then I receive a response code 200 @@ -50,7 +50,7 @@ Feature: standby cluster When I kill postgres1 And I kill postmaster on postgres1 Then postgres2 is replicating from postgres0 after 32 seconds - When I issue a GET request to http://127.0.0.1:8010/master + When I issue a GET request to http://127.0.0.1:8010/primary Then I receive a response code 503 And I sleep for 3 seconds When I issue a GET request to http://127.0.0.1:8010/standby_leader diff --git a/features/steps/basic_replication.py b/features/steps/basic_replication.py index a8f619f8..664288ac 100644 --- a/features/steps/basic_replication.py +++ b/features/steps/basic_replication.py @@ -83,10 +83,10 @@ def check_role(context, pg_name, pg_role, max_promotion_timeout): "{0} role didn't change to {1} after {2} seconds".format(pg_name, pg_role, max_promotion_timeout) -@step('replication works from {master:w} to {replica:w} after {time_limit:d} seconds') -@then('replication works from {master:w} to {replica:w} after {time_limit:d} seconds') -def replication_works(context, master, replica, time_limit): +@step('replication works from {primary:w} to {replica:w} after {time_limit:d} seconds') +@then('replication works from {primary:w} to {replica:w} after {time_limit:d} seconds') +def replication_works(context, primary, replica, time_limit): context.execute_steps(u""" When I add the table test_{0} to {1} Then table test_{0} is present on {2} after {3} seconds - """.format(int(time()), master, replica, time_limit)) + """.format(int(time()), primary, replica, time_limit)) diff --git a/features/steps/dcs_failsafe_mode.py b/features/steps/dcs_failsafe_mode.py index 613d698f..8273e996 100644 --- a/features/steps/dcs_failsafe_mode.py +++ b/features/steps/dcs_failsafe_mode.py @@ -11,6 +11,6 @@ def stop_dcs_outage(context): context.dcs_ctl.stop_outage() -@step('I start {name:w} in a cluster {cluster_name:w} from backup with no_master') -def start_cluster_from_backup_no_master(context, name, cluster_name): - context.pctl.bootstrap_from_backup_no_master(name, cluster_name) +@step('I start {name:w} in a cluster {cluster_name:w} from backup with no_leader') +def start_cluster_from_backup_no_leader(context, name, cluster_name): + context.pctl.bootstrap_from_backup_no_leader(name, cluster_name) diff --git a/patroni/api.py b/patroni/api.py index 038b55d7..2be0777e 100644 --- a/patroni/api.py +++ b/patroni/api.py @@ -97,7 +97,7 @@ class RestApiHandler(BaseHTTPRequestHandler): def do_GET(self, write_status_code_only=False): """Default method for processing all GET requests which can not be routed to other methods""" - path = '/master' if self.path == '/' else self.path + path = '/primary' if self.path == '/' else self.path response = self.get_postgresql_status() patroni = self.server.patroni @@ -114,8 +114,8 @@ class RestApiHandler(BaseHTTPRequestHandler): response.get('role') == 'replica' and response.get('state') == 'running' else 503 if not cluster and patroni.ha.is_paused(): - leader_status_code = 200 if response.get('role') in ('master', 'standby_leader') else 503 - primary_status_code = 200 if response.get('role') == 'master' else 503 + leader_status_code = 200 if response.get('role') in ('master', 'primary', 'standby_leader') else 503 + primary_status_code = 200 if response.get('role') in ('master', 'primary') else 503 standby_leader_status_code = 200 if response.get('role') == 'standby_leader' else 503 elif patroni.ha.is_leader(): leader_status_code = 200 @@ -191,7 +191,7 @@ class RestApiHandler(BaseHTTPRequestHandler): def do_GET_liveness(self): patroni = self.server.patroni - is_primary = patroni.postgresql.role == 'master' and patroni.postgresql.is_running() + is_primary = patroni.postgresql.role in ('master', 'primary') and patroni.postgresql.is_running() # We can tolerate Patroni problems longer on the replica. # On the primary the liveness probe most likely will start failing only after the leader key expired. # It should not be a big problem because replicas will see that the primary is still alive via REST API call. @@ -255,7 +255,11 @@ class RestApiHandler(BaseHTTPRequestHandler): metrics.append("# HELP patroni_master Value is 1 if this node is the leader, 0 otherwise.") metrics.append("# TYPE patroni_master gauge") - metrics.append("patroni_master{0} {1}".format(scope_label, int(postgres['role'] == 'master'))) + metrics.append("patroni_master{0} {1}".format(scope_label, int(postgres['role'] in ('master', 'primary')))) + + metrics.append("# HELP patroni_primary Value is 1 if this node is the leader, 0 otherwise.") + metrics.append("# TYPE patroni_primary gauge") + metrics.append("patroni_primary{0} {1}".format(scope_label, int(postgres['role'] in ('master', 'primary')))) metrics.append("# HELP patroni_xlog_location Current location of the Postgres" " transaction log, 0 if this node is not the leader.") @@ -443,9 +447,9 @@ class RestApiHandler(BaseHTTPRequestHandler): status_code = _ break elif k == 'role': - if request[k] not in ('master', 'replica'): + if request[k] not in ('master', 'primary', 'replica'): status_code = 400 - data = "PostgreSQL role should be either master or replica" + data = "PostgreSQL role should be either primary or replica" break elif k == 'postgres_version': try: diff --git a/patroni/config.py b/patroni/config.py index 2d25f918..91b14186 100644 --- a/patroni/config.py +++ b/patroni/config.py @@ -59,13 +59,17 @@ class Config(object): PATRONI_CONFIG_VARIABLE = PATRONI_ENV_PREFIX + 'CONFIGURATION' __CACHE_FILENAME = 'patroni.dynamic.json' + __REMAP_KEYS = { + 'master_start_timeout': 'primary_start_timeout', + 'master_stop_timeout': 'primary_stop_timeout' + } __DEFAULT_CONFIG = { 'ttl': 30, 'loop_wait': 10, 'retry_timeout': 10, 'maximum_lag_on_failover': 1048576, 'maximum_lag_on_syncnode': -1, 'check_timeline': False, - 'master_start_timeout': 300, - 'master_stop_timeout': 0, + 'primary_start_timeout': 300, + 'primary_stop_timeout': 0, 'synchronous_mode': False, 'synchronous_mode_strict': False, 'synchronous_node_count': 1, @@ -225,6 +229,9 @@ class Config(object): config = deepcopy(self.__DEFAULT_CONFIG) for name, value in dynamic_configuration.items(): + # allow copying master_start_timeout->primary_start_timeout when the latter isn't in dynamic_configuration + if name in self.__REMAP_KEYS and self.__REMAP_KEYS[name] not in dynamic_configuration: + name = self.__REMAP_KEYS[name] if name == 'postgresql': for name, value in (value or {}).items(): if name == 'parameters': @@ -355,8 +362,8 @@ class Config(object): if suffix in ('HOST', 'HOSTS', 'PORT', 'USE_PROXIES', 'PROTOCOL', 'SRV', 'SRV_SUFFIX', 'URL', 'PROXY', 'CACERT', 'CERT', 'KEY', 'VERIFY', 'TOKEN', 'CHECKS', 'DC', 'CONSISTENCY', 'REGISTER_SERVICE', 'SERVICE_CHECK_INTERVAL', 'SERVICE_CHECK_TLS_SERVER_NAME', - 'NAMESPACE', 'CONTEXT', 'USE_ENDPOINTS', 'SCOPE_LABEL', 'ROLE_LABEL', 'POD_IP', - 'PORTS', 'LABELS', 'BYPASS_API_SERVICE', 'KEY_PASSWORD', 'USE_SSL', 'SET_ACLS', + 'SERVICE_TAGS', 'NAMESPACE', 'CONTEXT', 'USE_ENDPOINTS', 'SCOPE_LABEL', 'ROLE_LABEL', + 'POD_IP', 'PORTS', 'LABELS', 'BYPASS_API_SERVICE', 'KEY_PASSWORD', 'USE_SSL', 'SET_ACLS', 'GROUP', 'DATABASE') and name: value = os.environ.pop(param) if name == 'CITUS': @@ -366,7 +373,7 @@ class Config(object): continue elif suffix == 'PORT': value = value and parse_int(value) - elif suffix in ('HOSTS', 'PORTS', 'CHECKS'): + elif suffix in ('HOSTS', 'PORTS', 'CHECKS', 'SERVICE_TAGS'): value = value and _parse_list(value) elif suffix in ('LABELS', 'SET_ACLS'): value = _parse_dict(value) diff --git a/patroni/ctl.py b/patroni/ctl.py index fcdf2f3f..1e1824f3 100644 --- a/patroni/ctl.py +++ b/patroni/ctl.py @@ -36,7 +36,7 @@ from .dcs import get_dcs as _get_dcs from .exceptions import PatroniException from .postgresql import Postgresql from .postgresql.misc import postgres_version_to_int -from .utils import cluster_as_json, find_executable, patch_config, polling_loop +from .utils import cluster_as_json, find_executable, patch_config, polling_loop, is_standby_cluster from .request import PatroniRequest from .version import __version__ @@ -141,6 +141,7 @@ option_default_citus_group = click.option('--group', required=False, type=int, h default=lambda: click.get_current_context().obj.get('citus', {}).get('group')) option_citus_group = click.option('--group', required=False, type=int, help='Citus group') option_insecure = click.option('-k', '--insecure', is_flag=True, help='Allow connections to SSL sites without certs') +role_choice = click.Choice(['leader', 'primary', 'standby-leader', 'replica', 'standby', 'any', 'master']) @click.group() @@ -241,24 +242,28 @@ def watching(w, watch, max_count=None, clear=True): yield 0 -def get_all_members(obj, cluster, group, role='master'): +def get_all_members(obj, cluster, group, role='leader'): clusters = {0: cluster} if obj.get('citus') and group is None: clusters.update(cluster.workers) - if role == 'master': + if role in ('leader', 'master', 'primary', 'standby-leader'): + role = {'primary': 'master', 'standby-leader': 'standby_leader'}.get(role, role) for cluster in clusters.values(): - if cluster.leader is not None and cluster.leader.name: + if cluster.leader is not None and cluster.leader.name and\ + (role == 'leader' or + cluster.leader.data.get('role') != 'master' and role == 'standby_leader' or + cluster.leader.data.get('role') != 'standby_leader' and role == 'master'): yield cluster.leader.member return for cluster in clusters.values(): leader_name = (cluster.leader.member.name if cluster.leader else None) for m in cluster.members: - if role == 'any' or role == 'replica' and m.name != leader_name: + if role == 'any' or role in ('replica', 'standby') and m.name != leader_name: yield m -def get_any_member(obj, cluster, group, role='master', member=None): +def get_any_member(obj, cluster, group, role='leader', member=None): for m in get_all_members(obj, cluster, group, role): if member is None or m.name == member: return m @@ -273,7 +278,7 @@ def get_all_members_leader_first(cluster): yield member -def get_cursor(obj, cluster, group, connect_parameters, role='master', member=None): +def get_cursor(obj, cluster, group, connect_parameters, role='leader', member=None): member = get_any_member(obj, cluster, group, role=role, member=member) if member is None: return None @@ -288,13 +293,14 @@ def get_cursor(obj, cluster, group, connect_parameters, role='master', member=No from . import psycopg conn = psycopg.connect(**params) cursor = conn.cursor() - if role == 'any': + if role in ('any', 'leader'): return cursor cursor.execute('SELECT pg_catalog.pg_is_in_recovery()') in_recovery = cursor.fetchone()[0] - if in_recovery and role == 'replica' or not in_recovery and role == 'master': + if in_recovery and role in ('replica', 'standby', 'standby-leader')\ + or not in_recovery and role in ('master', 'primary'): return cursor conn.close() @@ -347,9 +353,8 @@ def confirm_members_action(members, force, action, scheduled_at=None): raise PatroniCtlException('Aborted {0}'.format(action)) -@ctl.command('dsn', help='Generate a dsn for the provided member, defaults to a dsn of the master') -@click.option('--role', '-r', help='Give a dsn of any member with this role', type=click.Choice(['master', 'replica', - 'any']), default=None) +@ctl.command('dsn', help='Generate a dsn for the provided member, defaults to a dsn of the leader') +@click.option('--role', '-r', help='Give a dsn of any member with this role', type=role_choice, default=None) @click.option('--member', '-m', help='Generate a dsn for this member', type=str) @arg_cluster_name @option_citus_group @@ -360,7 +365,7 @@ def dsn(obj, cluster_name, group, role, member): raise PatroniCtlException('--role and --member are mutually exclusive options') role = 'any' if member is None and role is None: - role = 'master' + role = 'leader' cluster = get_dcs(obj, cluster_name, group).get_cluster() m = get_any_member(obj, cluster, group, role=role, member=member) @@ -380,8 +385,7 @@ def dsn(obj, cluster_name, group, role, member): @click.option('-U', '--username', help='database user name', type=str) @option_watch @option_watchrefresh -@click.option('--role', '-r', help='The role of the query', type=click.Choice(['master', 'replica', 'any']), - default=None) +@click.option('--role', '-r', help='The role of the query', type=role_choice, default=None) @click.option('--member', '-m', help='Query a specific member', type=str) @click.option('--delimiter', help='The column delimiter', default='\t') @click.option('--command', '-c', help='The SQL commands to execute') @@ -408,7 +412,7 @@ def query( raise PatroniCtlException('--role and --member are mutually exclusive options') role = 'any' if member is None and role is None: - role = 'master' + role = 'leader' if p_file is not None and command is not None: raise PatroniCtlException('--file and --command are mutually exclusive options') @@ -488,9 +492,9 @@ def remove(obj, cluster_name, group, fmt): raise PatroniCtlException('You did not exactly type "{0}"'.format(message)) if cluster.leader and cluster.leader.name: - confirm = click.prompt('This cluster currently is healthy. Please specify the master name to continue') + confirm = click.prompt('This cluster currently is healthy. Please specify the leader name to continue') if confirm != cluster.leader.name: - raise PatroniCtlException('You did not specify the current master of the cluster') + raise PatroniCtlException('You did not specify the current leader of the cluster') dcs.delete_cluster() @@ -524,8 +528,7 @@ def parse_scheduled(scheduled): @click.argument('cluster_name') @click.argument('member_names', nargs=-1) @option_citus_group -@click.option('--role', '-r', help='Reload only members with this role', default='any', - type=click.Choice(['master', 'replica', 'any'])) +@click.option('--role', '-r', help='Reload only members with this role', type=role_choice, default='any') @option_force @click.pass_obj def reload(obj, cluster_name, member_names, group, force, role): @@ -551,8 +554,7 @@ def reload(obj, cluster_name, member_names, group, force, role): @click.argument('cluster_name') @click.argument('member_names', nargs=-1) @option_citus_group -@click.option('--role', '-r', help='Restart only members with this role', default='any', - type=click.Choice(['master', 'replica', 'any'])) +@click.option('--role', '-r', help='Restart only members with this role', type=role_choice, default='any') @click.option('--any', 'p_any', help='Restart a single member only', is_flag=True) @click.option('--scheduled', help='Timestamp of a scheduled restart in unambiguous format (e.g. ISO 8601)', default=None) @@ -662,11 +664,11 @@ def reinit(obj, cluster_name, group, member_names, force, wait): wait_on_members.remove(member) -def _do_failover_or_switchover(obj, action, cluster_name, group, master, candidate, force, scheduled=None): +def _do_failover_or_switchover(obj, action, cluster_name, group, leader, candidate, force, scheduled=None): """ We want to trigger a failover or switchover for the specified cluster name. - We verify that the cluster name, master name and candidate name are correct. + We verify that the cluster name, leader name and candidate name are correct. If so, we trigger an action and keep the client up to date. """ @@ -684,19 +686,20 @@ def _do_failover_or_switchover(obj, action, cluster_name, group, master, candida cluster = dcs.get_cluster() if action == 'switchover' and (cluster.leader is None or not cluster.leader.name): - raise PatroniCtlException('This cluster has no master') + raise PatroniCtlException('This cluster has no leader') - if master is None: + if leader is None: if force or action == 'failover': - master = cluster.leader and cluster.leader.name + leader = cluster.leader and cluster.leader.name else: - master = click.prompt('Master', type=str, default=cluster.leader.member.name) + prompt = 'Standby Leader' if is_standby_cluster(cluster.config) else 'Primary' + leader = click.prompt(prompt, type=str, default=cluster.leader.member.name) - if master is not None and cluster.leader and cluster.leader.member.name != master: - raise PatroniCtlException('Member {0} is not the leader of cluster {1}'.format(master, cluster_name)) + if leader is not None and cluster.leader and cluster.leader.member.name != leader: + raise PatroniCtlException('Member {0} is not the leader of cluster {1}'.format(leader, cluster_name)) # excluding members with nofailover tag - candidate_names = [str(m.name) for m in cluster.members if m.name != master and not m.nofailover] + candidate_names = [str(m.name) for m in cluster.members if m.name != leader and not m.nofailover] # We sort the names for consistent output to the client candidate_names.sort() @@ -709,7 +712,7 @@ def _do_failover_or_switchover(obj, action, cluster_name, group, master, candida if action == 'failover' and not candidate: raise PatroniCtlException('Failover could be performed only to a specific candidate') - if candidate == master: + if candidate == leader: raise PatroniCtlException(action.title() + ' target and source are the same.') if candidate and candidate not in candidate_names: @@ -730,13 +733,13 @@ def _do_failover_or_switchover(obj, action, cluster_name, group, master, candida raise PatroniCtlException("Can't schedule switchover in the paused state") scheduled_at_str = scheduled_at.isoformat() - failover_value = {'leader': master, 'candidate': candidate, 'scheduled_at': scheduled_at_str} + failover_value = {'leader': leader, 'candidate': candidate, 'scheduled_at': scheduled_at_str} logging.debug(failover_value) # By now we have established that the leader exists and the candidate exists if not force: - demote_msg = ', demoting current master ' + master if master else '' + demote_msg = ', demoting current leader ' + leader if leader else '' if scheduled_at_str: if not click.confirm('Are you sure you want to schedule {0} of cluster {1} at {2}{3}?' .format(action, cluster_name, scheduled_at_str, demote_msg)): @@ -768,7 +771,7 @@ def _do_failover_or_switchover(obj, action, cluster_name, group, master, candida logging.exception(r) logging.warning('Failing over to DCS') click.echo('{0} Could not {1} using Patroni api, falling back to DCS'.format(timestamp(), action)) - dcs.manual_failover(master, candidate, scheduled_at=scheduled_at) + dcs.manual_failover(leader, candidate, scheduled_at=scheduled_at) output_members(obj, cluster, cluster_name, group=group) @@ -776,26 +779,26 @@ def _do_failover_or_switchover(obj, action, cluster_name, group, master, candida @ctl.command('failover', help='Failover to a replica') @arg_cluster_name @option_citus_group -@click.option('--master', help='The name of the current master', default=None) +@click.option('--leader', '--primary', '--master', 'leader', help='The name of the current leader', default=None) @click.option('--candidate', help='The name of the candidate', default=None) @option_force @click.pass_obj -def failover(obj, cluster_name, group, master, candidate, force): - action = 'switchover' if master else 'failover' - _do_failover_or_switchover(obj, action, cluster_name, group, master, candidate, force) +def failover(obj, cluster_name, group, leader, candidate, force): + action = 'switchover' if leader else 'failover' + _do_failover_or_switchover(obj, action, cluster_name, group, leader, candidate, force) @ctl.command('switchover', help='Switchover to a replica') @arg_cluster_name @option_citus_group -@click.option('--master', help='The name of the current master', default=None) +@click.option('--leader', '--primary', '--master', 'leader', help='The name of the current leader', default=None) @click.option('--candidate', help='The name of the candidate', default=None) @click.option('--scheduled', help='Timestamp of a scheduled switchover in unambiguous format (e.g. ISO 8601)', default=None) @option_force @click.pass_obj -def switchover(obj, cluster_name, group, master, candidate, force, scheduled): - _do_failover_or_switchover(obj, 'switchover', cluster_name, group, master, candidate, force, scheduled) +def switchover(obj, cluster_name, group, leader, candidate, force, scheduled): + _do_failover_or_switchover(obj, 'switchover', cluster_name, group, leader, candidate, force, scheduled) def generate_topology(level, member, topology): @@ -1007,8 +1010,7 @@ def scaffold(obj, cluster_name, group, sysid): @option_citus_group @click.argument('member_names', nargs=-1) @click.argument('target', type=click.Choice(['restart', 'switchover'])) -@click.option('--role', '-r', help='Flush only members with this role', default='any', - type=click.Choice(['master', 'replica', 'any'])) +@click.option('--role', '-r', help='Flush only members with this role', type=role_choice, default='any') @option_force @click.pass_obj def flush(obj, cluster_name, group, member_names, force, role, target): diff --git a/patroni/dcs/__init__.py b/patroni/dcs/__init__.py index c1684787..d14a73c1 100644 --- a/patroni/dcs/__init__.py +++ b/patroni/dcs/__init__.py @@ -229,8 +229,7 @@ class Member(namedtuple('Member', 'index,name,session,data')): class RemoteMember(Member): - """ Represents a remote master for a standby cluster - """ + """Represents a remote member (typically a primary) for a standby cluster""" def __new__(cls, name, data): return super(RemoteMember, cls).__new__(cls, None, name, None, data) @@ -283,7 +282,7 @@ class Leader(namedtuple('Leader', 'index,session,member')): version = self.member.version # 1.5.6 is the last version which doesn't expose checkpoint_after_promote: false if version and version > (1, 5, 6): - return self.data.get('role') == 'master' and 'checkpoint_after_promote' not in self.data + return self.data.get('role') in ('master', 'primary') and 'checkpoint_after_promote' not in self.data class Failover(namedtuple('Failover', 'index,leader,candidate,scheduled_at')): @@ -520,16 +519,16 @@ class Cluster(namedtuple('Cluster', 'initialize,config,leader,last_lsn,members,' def get_replication_slots(self, my_name, role, nofailover, major_version, show_error=False): # if the replicatefrom tag is set on the member - we should not create the replication slot for it on - # the current master, because that member would replicate from elsewhere. We still create the slot if + # the current primary, because that member would replicate from elsewhere. We still create the slot if # the replicatefrom destination member is currently not a member of the cluster (fallback to the - # master), or if replicatefrom destination member happens to be the current master + # primary), or if replicatefrom destination member happens to be the current primary use_slots = self.use_slots - if role in ('master', 'standby_leader'): + if role in ('master', 'primary', 'standby_leader'): slot_members = [m.name for m in self.members if use_slots and m.name != my_name and (m.replicatefrom is None or m.replicatefrom == my_name or not self.has_member(m.replicatefrom))] permanent_slots = self.__permanent_slots if use_slots and \ - role == 'master' else self.__permanent_physical_slots + role in ('master', 'primary') else self.__permanent_physical_slots else: # only manage slots for replicas that replicate from this one, except for the leader among them slot_members = [m.name for m in self.members if use_slots and @@ -794,7 +793,7 @@ class AbstractDCS(object): :param path: the path in DCS where to load Cluster(s) from. :param loader: one of `_cluster_loader` or `_citus_cluster_loader` :raise: `~DCSError` in case of communication problems with DCS. - If the current node was running as a master and exception + If the current node was running as a primary and exception raised, instance would be demoted.""" def _bypass_caches(self): @@ -926,7 +925,7 @@ class AbstractDCS(object): """Attempt to acquire leader lock This method should create `/leader` key with value=`~self._name` :param permanent: if set to `!True`, the leader key will never expire. - Used in patronictl for the external master + Used in patronictl for the external primary :returns: `!True` if key has been created successfully. Key must be created atomically. In case if key already exists it should not be @@ -964,7 +963,7 @@ class AbstractDCS(object): :param data: information about instance (including connection strings) :param ttl: ttl for member key, optional parameter. If it is None `~self.member_ttl will be used` :param permanent: if set to `!True`, the member key will never expire. - Used in patronictl for the external master. + Used in patronictl for the external primary :returns: `!True` on success otherwise `!False` """ @@ -1031,7 +1030,7 @@ class AbstractDCS(object): """""" def watch(self, leader_index, timeout): - """If the current node is a master it should just sleep. + """If the current node is a leader it should just sleep. Any other node should watch for changes of leader key with a given timeout :param leader_index: index of a leader key diff --git a/patroni/dcs/consul.py b/patroni/dcs/consul.py index 86809c10..310553e3 100644 --- a/patroni/dcs/consul.py +++ b/patroni/dcs/consul.py @@ -478,6 +478,10 @@ class Consul(AbstractDCS): check['TLSServerName'] = self._service_check_tls_server_name tags = self._service_tags[:] tags.append(role) + if role == 'master': + tags.append('primary') + elif role == 'primary': + tags.append('master') self._previous_loop_service_tags = self._service_tags self._previous_loop_token = self._client.token @@ -495,7 +499,7 @@ class Consul(AbstractDCS): return self.deregister_service(params['service_id']) self._previous_loop_register_service = self._register_service - if role in ['master', 'replica', 'standby-leader']: + if role in ['master', 'primary', 'replica', 'standby-leader']: if state != 'running': return return self.register_service(service_name, **params) diff --git a/patroni/dcs/etcd3.py b/patroni/dcs/etcd3.py index 21e705eb..11e5763e 100644 --- a/patroni/dcs/etcd3.py +++ b/patroni/dcs/etcd3.py @@ -92,7 +92,7 @@ class Unavailable(Etcd3ClientError): code = GRPCCode.Unavailable -# https://github.com/etcd-io/etcd/blob/master/etcdserver/api/v3rpc/rpctypes/error.go +# https://github.com/etcd-io/etcd/commits/main/api/v3rpc/rpctypes/error.go class LeaseNotFound(NotFound): error = "etcdserver: requested lease not found" diff --git a/patroni/dcs/exhibitor.py b/patroni/dcs/exhibitor.py index 8b7b5914..c4e0f429 100644 --- a/patroni/dcs/exhibitor.py +++ b/patroni/dcs/exhibitor.py @@ -19,7 +19,7 @@ class ExhibitorEnsembleProvider(object): self._uri_path = uri_path self._poll_interval = poll_interval self._exhibitors = hosts - self._master_exhibitors = hosts + self._boot_exhibitors = hosts self._zookeeper_hosts = '' self._next_poll = None while not self.poll(): @@ -32,7 +32,7 @@ class ExhibitorEnsembleProvider(object): json = self._query_exhibitors(self._exhibitors) if not json: - json = self._query_exhibitors(self._master_exhibitors) + json = self._query_exhibitors(self._boot_exhibitors) if isinstance(json, dict) and 'servers' in json and 'port' in json: self._next_poll = time.time() + self._poll_interval diff --git a/patroni/dcs/kubernetes.py b/patroni/dcs/kubernetes.py index 5e49a5b8..03505cde 100644 --- a/patroni/dcs/kubernetes.py +++ b/patroni/dcs/kubernetes.py @@ -242,7 +242,7 @@ class K8sClient(object): def set_base_uri(self, value): logger.info('Selected new K8s API server endpoint %s', value) - # We will connect by IP of the master node which is not listed as alternative name + # We will connect by IP of the K8s master node which is not listed as alternative name self.pool_manager.connection_pool_kw['assert_hostname'] = False self._base_uri = value @@ -1190,7 +1190,7 @@ class Kubernetes(AbstractDCS): cluster = self.cluster if cluster and cluster.leader and cluster.leader.name == self._name: role = 'master' - elif data['state'] == 'running' and data['role'] != 'master': + elif data['state'] == 'running' and data['role'] not in ('master', 'primary'): role = data['role'] else: role = None diff --git a/patroni/ha.py b/patroni/ha.py index 422a3a71..54a4667f 100644 --- a/patroni/ha.py +++ b/patroni/ha.py @@ -173,9 +173,9 @@ class Ha(object): else: return self.patroni.config.check_mode(mode) - def master_stop_timeout(self): - """ Master stop timeout """ - ret = parse_int(self.patroni.config['master_stop_timeout']) + def primary_stop_timeout(self): + """ Primary stop timeout """ + ret = parse_int(self.patroni.config['primary_stop_timeout']) return ret if ret and ret > 0 and self.is_synchronous_mode() else None def is_paused(self): @@ -341,10 +341,10 @@ class Ha(object): ret = self.dcs.touch_member(data) if ret: - if self._last_state != (data['state'], data['role'])\ - and (data['state'], data['role']) == ('running', 'master'): + new_state = (data['state'], {'master': 'primary'}.get(data['role'], data['role'])) + if self._last_state != new_state and new_state == ('running', 'primary'): self.notify_citus_coordinator('after_promote') - self._last_state = (data['state'], data['role']) + self._last_state = new_state return ret def clone(self, clone_member=None, msg='(without leader)'): @@ -369,7 +369,7 @@ class Ha(object): ret = self._async_executor.try_run_async('bootstrap {0}'.format(msg), self.clone, args=(clone_member, msg)) return ret or 'trying to bootstrap {0}'.format(msg) - # no initialize key and node is allowed to be master and has 'bootstrap' section in a configuration file + # no initialize key and node is allowed to be primary and has 'bootstrap' section in a configuration file elif self.cluster.initialize is None and not self.patroni.nofailover and 'bootstrap' in self.patroni.config: if self.dcs.initialize(create_new=True): # race for initialization self.state_handler.bootstrapping = True @@ -397,11 +397,11 @@ class Ha(object): def bootstrap_standby_leader(self): """ If we found 'standby' key in the configuration, we need to bootstrap - not a real master, but a 'standby leader', that will take base backup - from a remote master and start follow it. + not a real primary, but a 'standby leader', that will take base backup + from a remote member and start follow it. """ - clone_source = self.get_remote_master() - msg = 'clone from remote master {0}'.format(clone_source.conn_url) + clone_source = self.get_remote_member() + msg = 'clone from remote member {0}'.format(clone_source.conn_url) result = self.clone(clone_source, msg) with self._async_response: # pretend that post_bootstrap was already executed self._async_response.complete(result) @@ -418,7 +418,7 @@ class Ha(object): return self._async_executor.try_run_async(msg, self._rewind.ensure_clean_shutdown) or msg def _handle_rewind_or_reinitialize(self): - leader = self.get_remote_master() if self.is_standby_cluster() else self.cluster.leader + leader = self.get_remote_member() if self.is_standby_cluster() else self.cluster.leader if not self._rewind.rewind_or_reinitialize_needed_and_possible(leader): return None @@ -442,12 +442,12 @@ class Ha(object): self.watchdog.disable() if self.has_lock() and self.update_lock(): - timeout = self.patroni.config['master_start_timeout'] + timeout = self.patroni.config['primary_start_timeout'] if timeout == 0: - # We are requested to prefer failing over to restarting master. But see first if there + # We are requested to prefer failing over to restarting primary. But see first if there # is anyone to fail over to. if self.is_failover_possible(self.cluster.members): - logger.info("Master crashed. Failing over.") + logger.info("Primary crashed. Failing over.") self.demote('immediate') return 'stopped PostgreSQL to fail over after a crash' else: @@ -475,8 +475,8 @@ class Ha(object): role = 'standby_leader' node_to_follow = self._get_node_to_follow(self.cluster) elif self.is_standby_cluster() and self.cluster.is_unlocked(): - msg = "trying to follow a remote master because standby cluster is unhealthy" - node_to_follow = self.get_remote_master() + msg = "trying to follow a remote member because standby cluster is unhealthy" + node_to_follow = self.get_remote_member() else: msg = "starting as a secondary" node_to_follow = self._get_node_to_follow(self.cluster) @@ -498,7 +498,7 @@ class Ha(object): standby_config = self.get_standby_cluster_config() is_standby_cluster = _is_standby_cluster(standby_config) if is_standby_cluster and (self.cluster.is_unlocked() or self.has_lock(False)): - node_to_follow = self.get_remote_master() + node_to_follow = self.get_remote_member() elif self.patroni.replicatefrom and self.patroni.replicatefrom != self.state_handler.name: node_to_follow = cluster.get_member(self.patroni.replicatefrom) else: @@ -529,7 +529,7 @@ class Ha(object): or self.cluster.is_unlocked(): if is_leader: self.state_handler.set_role('master') - return 'continue to run as master without lock' + return 'continue to run as primary without lock' elif self.state_handler.role != 'standby_leader': self.state_handler.set_role('replica') @@ -584,7 +584,7 @@ class Ha(object): """Process synchronous standby beahvior. Synchronous standbys are registered in two places postgresql.conf and DCS. The order of updating them must - be right. The invariant that should be kept is that if a node is master and sync_standby is set in DCS, + be right. The invariant that should be kept is that if a node is primary and sync_standby is set in DCS, then that node must have synchronous_standby set to that value. Or more simple, first set in postgresql.conf and then in DCS. When removing, first remove in DCS, then in postgresql.conf. This is so we only consider promoting standbys that were guaranteed to be replicating synchronously. @@ -650,7 +650,7 @@ class Ha(object): If the connection to DCS fails we run the action anyway, as this is only a hint. - There is a small race window where this function runs between a master picking us the sync standby and + There is a small race window where this function runs between a primary picking us the sync standby and publishing it to the DCS. As the window is rather tiny consequences are holding up commits for one cycle period we don't worry about it here.""" @@ -661,7 +661,7 @@ class Ha(object): self._disable_sync += 1 try: if self.touch_member(): - # Master should notice the updated value during the next cycle. We will wait double that, if master + # Primary should notice the updated value during the next cycle. We will wait double that, if primary # hasn't noticed the value by then not disabling sync replication is not likely to matter. for _ in polling_loop(timeout=self.dcs.loop_wait*2, interval=2): try: @@ -670,7 +670,7 @@ class Ha(object): except DCSError: logger.warning("Could not get cluster state, skipping synchronous standby disable") break - logger.info("Waiting for master to release us from synchronous standby") + logger.info("Waiting for primary to release us from synchronous standby") else: logger.warning("Updating member state failed, skipping synchronous standby disable") @@ -680,14 +680,14 @@ class Ha(object): self._disable_sync -= 1 def update_cluster_history(self): - master_timeline = self.state_handler.get_master_timeline() + primary_timeline = self.state_handler.get_primary_timeline() cluster_history = self.cluster.history and self.cluster.history.lines - if master_timeline == 1: + if primary_timeline == 1: if cluster_history: self.dcs.set_history_value('[]') - elif not cluster_history or cluster_history[-1][0] != master_timeline - 1 or len(cluster_history[-1]) != 5: + elif not cluster_history or cluster_history[-1][0] != primary_timeline - 1 or len(cluster_history[-1]) != 5: cluster_history = {line[0]: line for line in cluster_history or []} - history = self.state_handler.get_history(master_timeline) + history = self.state_handler.get_history(primary_timeline) if history and self.cluster.config: history = history[-self.cluster.config.max_timelines_history:] for line in history: @@ -700,14 +700,14 @@ class Ha(object): line.append(cluster_history[line[0]][4]) self.dcs.set_history_value(json.dumps(history, separators=(',', ':'))) - def enforce_follow_remote_master(self, message): - demote_reason = 'cannot be a real master in standby cluster' + def enforce_follow_remote_member(self, message): + demote_reason = 'cannot be a real primary in standby cluster' return self.follow(demote_reason, message) - def enforce_master_role(self, message, promote_message): + def enforce_primary_role(self, message, promote_message): """ Ensure the node that has won the race for the leader key meets criteria - for promoting its PG server to the 'master' role. + for promoting its PG server to the 'primary' role. """ if not self.is_paused(): if not self.watchdog.is_running and not self.watchdog.activate(): @@ -728,14 +728,14 @@ class Ha(object): return 'Promotion cancelled because the pre-promote script failed' if self.state_handler.is_leader(): - # Inform the state handler about its master role. + # Inform the state handler about its primary role. # It may be unaware of it if postgres is promoted manually. self.state_handler.set_role('master') self.process_sync_replication() self.update_cluster_history() self.state_handler.citus_handler.sync_pg_dist_node(self.cluster) return message - elif self.state_handler.role in ('master', 'promoted'): + elif self.state_handler.role in ('master', 'promoted', 'primary'): self.process_sync_replication() return message else: @@ -748,7 +748,7 @@ class Ha(object): return 'Postponing promotion because synchronous replication state was updated by somebody else' self.state_handler.sync_handler.set_synchronous_standby_names( ['*'] if self.is_synchronous_mode_strict() else []) - if self.state_handler.role not in ('master', 'promoted'): + if self.state_handler.role not in ('master', 'promoted', 'primary'): def on_success(): self._rewind.reset_state() logger.info("cleared rewind state after becoming the leader") @@ -785,7 +785,7 @@ class Ha(object): return results def update_failsafe(self, data): - if self.state_handler.state == 'running' and self.state_handler.role == 'master': + if self.state_handler.state == 'running' and self.state_handler.role in ('master', 'primary'): return 'Running as a leader' self._failsafe.update(data) @@ -842,7 +842,7 @@ class Ha(object): my_wal_position = self.state_handler.last_operation() if check_replication_lag and self.is_lagging(my_wal_position): logger.info('My wal position exceeds maximum replication lag') - return False # Too far behind last reported wal position on master + return False # Too far behind last reported wal position on primary if not self.is_standby_cluster() and self.check_timeline(): cluster_timeline = self.cluster.timeline @@ -858,7 +858,7 @@ class Ha(object): for st in self.fetch_nodes_statuses(members): if st.failover_limitation() is None: if not st.in_recovery: - logger.warning('Master (%s) is still alive', st.member.name) + logger.warning('Primary (%s) is still alive', st.member.name) return False if my_wal_position < st.wal_position: logger.info('Wal position of %s is ahead of my wal position', st.member.name) @@ -901,7 +901,7 @@ class Ha(object): return True elif self.is_paused(): # Remove failover key if the node to failover has terminated to avoid waiting for it indefinitely - # In order to avoid attempts to delete this key from all nodes only the master is allowed to do it. + # In order to avoid attempts to delete this key from all nodes only the primary is allowed to do it. if (not self.cluster.get_member(failover.candidate, fallback_to_leader=False) and self.state_handler.is_leader()): logger.warning("manual failover: removing failover key because failover candidate is not running") @@ -954,7 +954,7 @@ class Ha(object): if self.is_paused() and not self.patroni.nofailover and \ self.cluster.failover and not self.cluster.failover.scheduled_at: ret = self.manual_failover_process_no_leader() - if ret is not None: # continue if we just deleted the stale failover key as a master + if ret is not None: # continue if we just deleted the stale failover key as a leader return ret if self.state_handler.is_starting(): # postgresql still starting up is unhealthy @@ -994,7 +994,7 @@ class Ha(object): all_known_members += [RemoteMember(name, {'api_url': url}) for name, url in failsafe_members.items()] all_known_members += self.cluster.members - # When in sync mode, only last known master and sync standby are allowed to promote automatically. + # When in sync mode, only last known primary and sync standby are allowed to promote automatically. if self.is_synchronous_mode() and self.cluster.sync and self.cluster.sync.leader: if not self.cluster.sync.matches(self.state_handler.name): return False @@ -1017,14 +1017,14 @@ class Ha(object): logger.info("Leader key released") def demote(self, mode): - """Demote PostgreSQL running as master. + """Demote PostgreSQL running as primary. :param mode: One of offline, graceful or immediate. offline is used when connection to DCS is not available. graceful is used when failing over to another node due to user request. May only be called running async. - immediate is used when we determine that we are not suitable for master and want to failover quickly + immediate is used when we determine that we are not suitable for primary and want to failover quickly without regard for data durability. May only be called synchronously. - immediate-nolock is used when find out that we have lost the lock to be master. Need to bring down + immediate-nolock is used when find out that we have lost the lock to be primary. Need to bring down PostgreSQL as quickly as possible without regard for data durability. May only be called synchronously. """ mode_control = { @@ -1061,7 +1061,7 @@ class Ha(object): on_safepoint=self.watchdog.disable if self.watchdog.is_running else None, on_shutdown=on_shutdown if mode_control['release'] else None, before_shutdown=before_shutdown if mode == 'graceful' else None, - stop_timeout=self.master_stop_timeout()) + stop_timeout=self.primary_stop_timeout()) self.state_handler.set_role('demoted') self.set_is_leader(False) @@ -1189,11 +1189,11 @@ class Ha(object): if self.is_standby_cluster(): # standby leader disappeared, and this is the healthiest # replica, so it should become a new standby leader. - # This implies we need to start following a remote master + # This implies we need to start following a remote member msg = 'promoted self to a standby leader by acquiring session lock' - return self.enforce_follow_remote_master(msg) + return self.enforce_follow_remote_member(msg) else: - return self.enforce_master_role( + return self.enforce_primary_role( 'acquired session lock as a leader', 'promoted self to leader by acquiring session lock' ) @@ -1208,7 +1208,7 @@ class Ha(object): time.sleep(2) # Give a time to somebody to take the leader lock if self.patroni.nofailover: - return self.follow('demoting self because I am not allowed to become master', + return self.follow('demoting self because I am not allowed to become primary', 'following a different leader because I am not allowed to promote') return self.follow('demoting self because i am not the healthiest node', 'following a different leader because i am not the healthiest node') @@ -1217,11 +1217,11 @@ class Ha(object): if self.has_lock(): if self.is_paused() and not self.state_handler.is_leader(): if self.cluster.failover and self.cluster.failover.candidate == self.state_handler.name: - return 'waiting to become master after promote...' + return 'waiting to become primary after promote...' if not self.is_standby_cluster(): self._delete_leader() - return 'removed leader lock because postgres is not running as master' + return 'removed leader lock because postgres is not running as primary' if self.update_lock(True): msg = self.process_manual_failover_from_leader() @@ -1233,14 +1233,14 @@ class Ha(object): if self.is_standby_cluster(): # in case of standby cluster we don't really need to - # enforce anything, since the leader is not a master. + # enforce anything, since the leader is not a primary # So just remind the role. msg = 'no action. I am ({0}), the standby leader with the lock'.format(self.state_handler.name) \ if self.state_handler.role == 'standby_leader' else \ 'promoted self to a standby leader because i had the session lock' - return self.enforce_follow_remote_master(msg) + return self.enforce_follow_remote_member(msg) else: - return self.enforce_master_role( + return self.enforce_primary_role( 'no action. I am ({0}), the leader with the lock'.format(self.state_handler.name), 'promoted self to leader because I had the session lock' ) @@ -1249,7 +1249,7 @@ class Ha(object): logger.error('failed to update leader lock') if self.state_handler.is_leader(): if self.is_paused(): - return 'continue to run as master after failing to update leader lock in DCS' + return 'continue to run as primary after failing to update leader lock in DCS' self.demote('immediate-nolock') return 'demoted self because failed to update leader lock in DCS' else: @@ -1356,7 +1356,7 @@ class Ha(object): # Now that restart is scheduled we can set timeout for startup, it will get reset # once async executor runs and main loop notices PostgreSQL as up. - timeout = restart_data.get('timeout', self.patroni.config['master_start_timeout']) + timeout = restart_data.get('timeout', self.patroni.config['primary_start_timeout']) self.set_start_timeout(timeout) def before_shutdown(): @@ -1419,7 +1419,7 @@ class Ha(object): """ if self.has_lock() and self.update_lock(): if self._async_executor.scheduled_action == 'doing crash recovery in a single user mode': - time_left = self.patroni.config['master_start_timeout'] - (time.time() - self._crash_recovery_started) + time_left = self.patroni.config['primary_start_timeout'] - (time.time() - self._crash_recovery_started) if time_left <= 0 and self.is_failover_possible(self.cluster.members): logger.info("Demoting self because crash recovery is taking too long") self.state_handler.cancellable.cancel(True) @@ -1428,7 +1428,7 @@ class Ha(object): return 'updated leader lock during ' + self._async_executor.scheduled_action elif not self.state_handler.bootstrapping and not self.is_paused(): - # Don't have lock, make sure we are not promoting or starting up a master in the background + # Don't have lock, make sure we are not promoting or starting up a primary in the background if self._async_executor.scheduled_action == 'promote': with self._async_response: cancel = self._async_response.cancel() @@ -1436,8 +1436,8 @@ class Ha(object): self.state_handler.cancellable.cancel() return 'lost leader before promote' - if self.state_handler.role == 'master': - logger.info("Demoting master during " + self._async_executor.scheduled_action) + if self.state_handler.role in ('master', 'primary'): + logger.info("Demoting primary during " + self._async_executor.scheduled_action) if self._async_executor.scheduled_action == 'restart': # Restart needs a special interlocking cancel because postmaster may be just started in a # background thread and has not even written a pid file yet. @@ -1462,7 +1462,7 @@ class Ha(object): if not self.state_handler.is_running(): self.watchdog.disable() if self.has_lock(): - if self.state_handler.role in ('master', 'standby_leader'): + if self.state_handler.role in ('master', 'primary', 'standby_leader'): self.state_handler.set_role('demoted') self._delete_leader() return 'removed leader key after trying and failing to start postgres' @@ -1525,16 +1525,16 @@ class Ha(object): self.demote('immediate-nolock') return 'stopped PostgreSQL while starting up because leader key was lost' - timeout = self._start_timeout or self.patroni.config['master_start_timeout'] + timeout = self._start_timeout or self.patroni.config['primary_start_timeout'] time_left = timeout - self.state_handler.time_in_state() if time_left <= 0: if self.is_failover_possible(self.cluster.members): - logger.info("Demoting self because master startup is taking too long") + logger.info("Demoting self because primary startup is taking too long") self.demote('immediate') return 'stopped PostgreSQL because of startup timeout' else: - return 'master start has timed out, but continuing to wait because failover is not possible' + return 'primary start has timed out, but continuing to wait because failover is not possible' else: msg = self.process_manual_failover_from_leader() if msg is not None: @@ -1547,7 +1547,7 @@ class Ha(object): return None def set_start_timeout(self, value): - """Sets timeout for starting as master before eligible for failover. + """Sets timeout for starting as primary before eligible for failover. Must be called when async_executor is busy or in the main thread.""" self._start_timeout = value @@ -1622,7 +1622,7 @@ class Ha(object): if not data_directory_is_accessible or data_directory_is_empty: self.state_handler.set_role('uninitialized') self.state_handler.stop('immediate', stop_timeout=self.patroni.config['retry_timeout']) - # In case datadir went away while we were master. + # In case datadir went away while we were primary self.watchdog.disable() # is this instance the leader? @@ -1662,7 +1662,7 @@ class Ha(object): and not self.state_handler.is_leader(): self._join_aborted = True logger.error('No initialize key in DCS and PostgreSQL is running as replica, aborting start') - logger.error('Please first start Patroni on the node running as master') + logger.error('Please first start Patroni on the node running as primary') sys.exit(1) self.dcs.initialize(create_new=(self.cluster.initialize is None), sysid=data_sysid) @@ -1694,7 +1694,7 @@ class Ha(object): # we might not have a valid PostgreSQL connection here if another thread # stops PostgreSQL, therefore, we only reload replication slots if no - # asynchronous processes are running (should be always the case for the master) + # asynchronous processes are running (should be always the case for the primary) if not self._async_executor.busy and not self.state_handler.is_starting(): create_slots = self._sync_replication_slots(False) if not self.state_handler.cb_called: @@ -1809,7 +1809,7 @@ class Ha(object): self.while_not_sync_standby(lambda: self.state_handler.stop(checkpoint=False, on_safepoint=disable_wd, on_shutdown=on_shutdown, before_shutdown=before_shutdown, - stop_timeout=self.master_stop_timeout())) + stop_timeout=self.primary_stop_timeout())) if not self.state_handler.is_running(): if self.is_leader() and not status['deleted']: checkpoint_location = self.state_handler.latest_checkpoint_location() @@ -1834,18 +1834,18 @@ class Ha(object): def wakeup(self): """Call of this method will trigger the next run of HA loop if there is no "active" leader watch request in progress. - This usually happens on the master or if the node is running async action""" + This usually happens on the leader or if the node is running async action""" self.dcs.event.set() def get_remote_member(self, member=None): """ In case of standby cluster this will tel us from which remote - master to stream. Config can be both patroni config or + member to stream. Config can be both patroni config or cluster.config.data """ cluster_params = self.get_standby_cluster_config() if cluster_params: - name = member.name if member else 'remote_master:{}'.format(uuid.uuid1()) + name = member.name if member else 'remote_member:{}'.format(uuid.uuid1()) data = {k: v for k, v in cluster_params.items() if k in RemoteMember.allowed_keys()} data['no_replication_slot'] = 'primary_slot_name' not in cluster_params @@ -1855,6 +1855,3 @@ class Ha(object): data['conn_kwargs'] = conn_kwargs return RemoteMember(name, data) - - def get_remote_master(self): - return self.get_remote_member() diff --git a/patroni/postgresql/__init__.py b/patroni/postgresql/__init__.py index 54ab4005..2356110c 100644 --- a/patroni/postgresql/__init__.py +++ b/patroni/postgresql/__init__.py @@ -56,7 +56,7 @@ class Postgresql(object): POSTMASTER_START_TIME = "pg_catalog.pg_postmaster_start_time()" TL_LSN = ("CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 " "ELSE ('x' || pg_catalog.substr(pg_catalog.pg_{0}file_name(" - "pg_catalog.pg_current_{0}_{1}()), 1, 8))::bit(32)::int END, " # master timeline + "pg_catalog.pg_current_{0}_{1}()), 1, 8))::bit(32)::int END, " # primary timeline "CASE WHEN pg_catalog.pg_is_in_recovery() THEN 0 " "ELSE pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_current_{0}_{1}(), '0/0')::bigint END, " # write_lsn "pg_catalog.pg_{0}_{1}_diff(pg_catalog.pg_last_{0}_replay_{1}(), '0/0')::bigint, " @@ -126,7 +126,7 @@ class Postgresql(object): ident_saved = self.config.replace_pg_ident() if hba_saved or ident_saved: self.reload() - elif self.role == 'master': + elif self.role in ('master', 'primary'): self.set_role('demoted') @property @@ -186,7 +186,7 @@ class Postgresql(object): "FROM pg_catalog.pg_stat_get_wal_senders() w," + " pg_catalog.pg_stat_get_activity(w.pid)" + " WHERE w.state = 'streaming') r)").format(self.wal_name, self.lsn_name) - if self._is_synchronous_mode and self.role == 'master' else "'on', '', NULL") + if self._is_synchronous_mode and self.role in ('master', 'primary') else "'on', '', NULL") if self._major_version >= 90600: extra = ("(SELECT pg_catalog.json_agg(s.*) FROM (SELECT slot_name, slot_type as type, datoid::bigint, " + @@ -331,7 +331,8 @@ class Postgresql(object): return deepcopy(self.config.get(method, {})) def replica_method_can_work_without_replication_connection(self, method): - return method != 'basebackup' and self.replica_method_options(method).get('no_master') + return method != 'basebackup' and (self.replica_method_options(method).get('no_master') or + self.replica_method_options(method).get('no_leader')) def can_create_replica_without_replication_connection(self, replica_methods=None): """ go through the replication methods to see if there are ones @@ -421,7 +422,7 @@ class Postgresql(object): return bool(self._cluster_info_state_get('timeline')) except PostgresConnectionException: logger.warning('Failed to determine PostgreSQL state from the connection, falling back to cached role') - return bool(self.is_running() and self.role == 'master') + return bool(self.is_running() and self.role in ('master', 'primary')) def replay_paused(self): return self._cluster_info_state_get('replay_paused') @@ -905,12 +906,13 @@ class Postgresql(object): except Exception: logger.exception('Can not fetch local timeline and lsn from replication connection') - def replica_cached_timeline(self, master_timeline): - if not self._cached_replica_timeline or not master_timeline or self._cached_replica_timeline != master_timeline: + def replica_cached_timeline(self, primary_timeline): + if not self._cached_replica_timeline or not primary_timeline\ + or self._cached_replica_timeline != primary_timeline: self._cached_replica_timeline = self.get_replica_timeline() return self._cached_replica_timeline - def get_master_timeline(self): + def get_primary_timeline(self): return self._cluster_info_state_get('timeline') def get_history(self, timeline): @@ -933,11 +935,11 @@ class Postgresql(object): recovery_params = self.config.build_recovery_params(member) self.config.write_recovery_conf(recovery_params) - # When we demoting the master or standby_leader to replica or promoting replica to a standby_leader + # When we demoting the primary or standby_leader to replica or promoting replica to a standby_leader # and we know for sure that postgres was already running before, we will only execute on_role_change # callback and prevent execution of on_restart/on_start callback. # If the role remains the same (replica or standby_leader), we will execute on_start or on_restart - change_role = self.cb_called and (self.role in ('master', 'demoted') or + change_role = self.cb_called and (self.role in ('master', 'primary', 'demoted') or not {'standby_leader', 'replica'} - {self.role, role}) if change_role: self.__cb_pending = ACTION_NOOP @@ -982,7 +984,7 @@ class Postgresql(object): return ret == 0 def promote(self, wait_seconds, task, before_promote=None, on_success=None): - if self.role in ('promoted', 'master'): + if self.role in ('promoted', 'master', 'primary'): return True ret = self._pre_promote() diff --git a/patroni/postgresql/bootstrap.py b/patroni/postgresql/bootstrap.py index 12208c14..89f9d377 100644 --- a/patroni/postgresql/bootstrap.py +++ b/patroni/postgresql/bootstrap.py @@ -155,12 +155,12 @@ class Bootstrap(object): self._postgresql.set_state('creating replica') self._postgresql.schedule_sanity_checks_after_pause() - is_remote_master = isinstance(clone_member, RemoteMember) + is_remote_member = isinstance(clone_member, RemoteMember) # get list of replica methods either from clone member or from # the config. If there is no configuration key, or no value is # specified, use basebackup - replica_methods = (clone_member.create_replica_methods if is_remote_master + replica_methods = (clone_member.create_replica_methods if is_remote_member else self._postgresql.create_replica_methods) or ['basebackup'] if clone_member and clone_member.conn_url: @@ -212,7 +212,7 @@ class Bootstrap(object): "datadir": self._postgresql.data_dir, "connstring": connstring}) else: - for param in ('no_params', 'no_master', 'keep_data'): + for param in ('no_params', 'no_master', 'no_leader', 'keep_data'): method_config.pop(param, None) params = ["--{0}={1}".format(arg, val) for arg, val in method_config.items()] try: @@ -269,7 +269,7 @@ class Bootstrap(object): def clone(self, clone_member): """ - - initialize the replica from an existing member (master or replica) + - initialize the replica from an existing member (primary or replica) - initialize the replica using the replica creation method that works without the replication connection (i.e. restore from on-disk base backup) diff --git a/patroni/postgresql/citus.py b/patroni/postgresql/citus.py index 5f27ec9e..0866eb9d 100644 --- a/patroni/postgresql/citus.py +++ b/patroni/postgresql/citus.py @@ -153,7 +153,7 @@ class CitusHandler(Thread): for group, worker in cluster.workers.items(): leader = worker.leader if leader and leader.conn_url\ - and leader.data.get('role') == 'master' and leader.data.get('state') == 'running': + and leader.data.get('role') in ('master', 'primary') and leader.data.get('state') == 'running': self.add_task('after_promote', group, leader.conn_url) def find_task_by_group(self, group): diff --git a/patroni/postgresql/config.py b/patroni/postgresql/config.py index e14f1d5a..27fd4cf6 100644 --- a/patroni/postgresql/config.py +++ b/patroni/postgresql/config.py @@ -529,24 +529,24 @@ class ConfigHandler(object): recovery_params.update({'recovery_target': '', 'recovery_target_name': '', 'recovery_target_time': '', 'recovery_target_xid': '', 'recovery_target_lsn': ''}) - is_remote_master = isinstance(member, RemoteMember) + is_remote_member = isinstance(member, RemoteMember) primary_conninfo = self.primary_conninfo_params(member) if primary_conninfo: use_slots = self.get('use_slots', True) and self._postgresql.major_version >= 90400 - if use_slots and not (is_remote_master and member.no_replication_slot): - primary_slot_name = member.primary_slot_name if is_remote_master else self._postgresql.name + if use_slots and not (is_remote_member and member.no_replication_slot): + primary_slot_name = member.primary_slot_name if is_remote_member else self._postgresql.name recovery_params['primary_slot_name'] = slot_name_from_member_name(primary_slot_name) # We are a standby leader and are using a replication slot. Make sure we connect to # the leader of the main cluster (in case more than one host is specified in the # connstr) by adding 'target_session_attrs=read-write' to primary_conninfo. - if is_remote_master and 'target_sesions_attrs' not in primary_conninfo and\ + if is_remote_member and 'target_sesions_attrs' not in primary_conninfo and\ self._postgresql.major_version >= 100000: primary_conninfo['target_session_attrs'] = 'read-write' recovery_params['primary_conninfo'] = primary_conninfo # standby_cluster config might have different parameters, we want to override them standby_cluster_params = ['restore_command', 'archive_cleanup_command']\ - + (['recovery_min_apply_delay'] if is_remote_master else []) + + (['recovery_min_apply_delay'] if is_remote_member else []) recovery_params.update({p: member.data.get(p) for p in standby_cluster_params if member and member.data.get(p)}) return recovery_params @@ -1065,7 +1065,7 @@ class ConfigHandler(object): As a workaround we will start it with the values from controldata and set `pending_restart` to true as an indicator that current values of parameters are not matching expectations.""" - if self._postgresql.role == 'master': + if self._postgresql.role in ('master', 'primary'): return self._server_parameters options_mapping = { diff --git a/patroni/postgresql/rewind.py b/patroni/postgresql/rewind.py index 76e6a54f..06c11a62 100644 --- a/patroni/postgresql/rewind.py +++ b/patroni/postgresql/rewind.py @@ -126,7 +126,7 @@ class Rewind(object): in_recovery = True lsn = data.get('Minimum recovery ending location') timeline = int(data.get("Min recovery ending loc's timeline")) - if lsn == '0/0' or timeline == 0: # it was a master when it crashed + if lsn == '0/0' or timeline == 0: # it was a primary when it crashed data['Database cluster state'] = 'shut down' if data.get('Database cluster state') == 'shut down': in_recovery = False @@ -157,7 +157,7 @@ class Rewind(object): return in_recovery, timeline, lsn @staticmethod - def _log_master_history(history, i): + def _log_primary_history(history, i): start = max(0, i - 3) end = None if i + 4 >= len(history) else i + 2 history_show = [] @@ -172,7 +172,7 @@ class Rewind(object): history_show.append('...') history_show.append(format_history_line(history[-1])) - logger.info('master: history=%s', '\n'.join(history_show)) + logger.info('primary: history=%s', '\n'.join(history_show)) def _conn_kwargs(self, member, auth): ret = member.conn_kwargs(auth) @@ -189,7 +189,7 @@ class Rewind(object): if local_timeline is None or local_lsn is None: return - if isinstance(leader, Leader) and leader.member.data.get('role') != 'master': + if isinstance(leader, Leader) and leader.member.data.get('role') not in ('master', 'primary'): return # We want to use replication credentials when connecting to the "postgres" database in case if @@ -206,20 +206,20 @@ class Rewind(object): try: with self._postgresql.get_replication_connection_cursor(**leader.conn_kwargs()) as cur: cur.execute('IDENTIFY_SYSTEM') - master_timeline = cur.fetchone()[1] - logger.info('master_timeline=%s', master_timeline) - if local_timeline > master_timeline: # Not always supported by pg_rewind + primary_timeline = cur.fetchone()[1] + logger.info('primary_timeline=%s', primary_timeline) + if local_timeline > primary_timeline: # Not always supported by pg_rewind need_rewind = True - elif local_timeline == master_timeline: + elif local_timeline == primary_timeline: need_rewind = False - elif master_timeline > 1: - cur.execute('TIMELINE_HISTORY {0}'.format(master_timeline)) + elif primary_timeline > 1: + cur.execute('TIMELINE_HISTORY {0}'.format(primary_timeline)) history = cur.fetchone()[1] if not isinstance(history, six.string_types): history = bytes(history).decode('utf-8') - logger.debug('master: history=%s', history) + logger.debug('primary: history=%s', history) except Exception: - return logger.exception('Exception when working with master via replication connection') + return logger.exception('Exception when working with primary via replication connection') if history is not None: history = list(parse_history(history)) @@ -240,7 +240,7 @@ class Rewind(object): break else: need_rewind = True - self._log_master_history(history, i) + self._log_primary_history(history, i) self._state = need_rewind and REWIND_STATUS.NEED or REWIND_STATUS.NOT_NEED @@ -270,7 +270,7 @@ class Rewind(object): if self._checkpoint_task.result is not None: self._state = REWIND_STATUS.CHECKPOINT self._checkpoint_task = None - elif self._postgresql.get_master_timeline() == self._postgresql.pg_control_timeline(): + elif self._postgresql.get_primary_timeline() == self._postgresql.pg_control_timeline(): self._state = REWIND_STATUS.CHECKPOINT else: self._checkpoint_task = CriticalTask() @@ -431,12 +431,12 @@ class Rewind(object): # prepare pg_rewind connection r = self._conn_kwargs(leader, self._postgresql.config.rewind_credentials) - # 1. make sure that we are really trying to rewind from the master + # 1. make sure that we are really trying to rewind from the primary # 2. make sure that pg_control contains the new timeline by: # running a checkpoint or - # waiting until Patroni on the master will expose checkpoint_after_promote=True + # waiting until Patroni on the primary will expose checkpoint_after_promote=True checkpoint_status = leader.checkpoint_after_promote if isinstance(leader, Leader) else None - if checkpoint_status is None: # we are the standby-cluster leader or master still runs the old Patroni + if checkpoint_status is None: # we are the standby-cluster leader or primary still runs the old Patroni # superuser credentials match rewind_credentials if the latter are not provided or we run 10 or older if self._postgresql.config.superuser == self._postgresql.config.rewind_credentials: leader_status = self._postgresql.checkpoint( @@ -455,11 +455,11 @@ class Rewind(object): self._state = REWIND_STATUS.SUCCESS else: if not self.check_leader_is_not_in_recovery(r): - logger.warning('Failed to rewind because master %s become unreachable', leader.name) + logger.warning('Failed to rewind because primary %s become unreachable', leader.name) if not self.can_rewind: # It is possible that the previous attempt damaged pg_control file! self._state = REWIND_STATUS.FAILED else: - logger.error('Failed to rewind from healty master: %s', leader.name) + logger.error('Failed to rewind from healty primary: %s', leader.name) self._state = REWIND_STATUS.FAILED if self.failed: diff --git a/patroni/postgresql/sync.py b/patroni/postgresql/sync.py index bbc30044..578224c6 100644 --- a/patroni/postgresql/sync.py +++ b/patroni/postgresql/sync.py @@ -254,5 +254,5 @@ class SyncHandler(object): self._postgresql.reset_cluster_info_state(None) # timeline == 0 -- indicates that this is the replica, shoudn't ever happen - if self._postgresql.get_master_timeline() > 0: + if self._postgresql.get_primary_timeline() > 0: self._handle_synchronous_standby_names_change() diff --git a/patroni/scripts/wale_restore.py b/patroni/scripts/wale_restore.py index 189eb273..7e6de976 100755 --- a/patroni/scripts/wale_restore.py +++ b/patroni/scripts/wale_restore.py @@ -11,7 +11,7 @@ # arguments are: # - cluster scope # - cluster role -# - master connection string +# - leader connection string # - number of retries # - envdir for the WALE env # - WALE_BACKUP_THRESHOLD_MEGABYTES if WAL amount is above that - use pg_basebackup @@ -104,11 +104,11 @@ WALEConfig = namedtuple( class WALERestore(object): def __init__(self, scope, datadir, connstring, env_dir, threshold_mb, - threshold_pct, use_iam, no_master, retries): + threshold_pct, use_iam, no_leader, retries): self.scope = scope - self.master_connection = connstring + self.leader_connection = connstring self.data_dir = datadir - self.no_master = no_master + self.no_leader = no_leader wale_cmd = [ 'envdir', @@ -213,11 +213,11 @@ class WALERestore(object): diff_in_bytes = backup_size attempts_no = 0 while True: - if self.master_connection: + if self.leader_connection: con = None try: # get the difference in bytes between the current WAL location and the backup start offset - con = psycopg.connect(self.master_connection) + con = psycopg.connect(self.leader_connection) if con.server_version >= 100000: wal_name = 'wal' lsn_name = 'lsn' @@ -235,22 +235,22 @@ class WALERestore(object): diff_in_bytes = int(cur.fetchone()[0]) except psycopg.Error: - logger.exception('could not determine difference with the master location') + logger.exception('could not determine difference with the leader location') if attempts_no < self.retries: # retry in case of a temporarily connection issue attempts_no = attempts_no + 1 time.sleep(RETRY_SLEEP_INTERVAL) continue else: - if not self.no_master: + if not self.no_leader: return False # do no more retries on the outer level - logger.info("continue with base backup from S3 since master is not available") + logger.info("continue with base backup from S3 since leader is not available") diff_in_bytes = 0 break finally: if con: con.close() else: - # always try to use WAL-E if master connection string is not available + # always try to use WAL-E if leader connection string is not available diff_in_bytes = 0 break @@ -346,22 +346,22 @@ def main(): parser.add_argument('--threshold_megabytes', type=int, default=10240) parser.add_argument('--threshold_backup_size_percentage', type=int, default=30) parser.add_argument('--use_iam', type=int, default=0) - parser.add_argument('--no_master', type=int, default=0) + parser.add_argument('--no_leader', '--no_master', type=int, default=0) args = parser.parse_args() exit_code = None assert args.retries >= 0 - # Retry cloning in a loop. We do separate retries for the master + # Retry cloning in a loop. We do separate retries for the leader # connection attempt inside should_use_s3_to_create_replica, # because we need to differentiate between the last attempt and # the rest and make a decision when the last attempt fails on - # whether to use WAL-E or not depending on the no_master flag. + # whether to use WAL-E or not depending on the no_leader flag. for _ in range(0, args.retries + 1): restore = WALERestore(scope=args.scope, datadir=args.datadir, connstring=args.connstring, env_dir=args.envdir, threshold_mb=args.threshold_megabytes, threshold_pct=args.threshold_backup_size_percentage, use_iam=args.use_iam, - no_master=args.no_master, retries=args.retries) + no_leader=args.no_leader, retries=args.retries) exit_code = restore.run() if not exit_code == ExitCode.RETRY_LATER: # only WAL-E failures lead to the retry logger.debug('exit_code is %r, not retrying', exit_code) diff --git a/postgres0.yml b/postgres0.yml index 5b373c2c..7aed19b9 100644 --- a/postgres0.yml +++ b/postgres0.yml @@ -51,7 +51,7 @@ bootstrap: loop_wait: 10 retry_timeout: 10 maximum_lag_on_failover: 1048576 -# master_start_timeout: 300 +# primary_start_timeout: 300 # synchronous_mode: false #standby_cluster: #host: 127.0.0.1 diff --git a/tests/__init__.py b/tests/__init__.py index d8b5ece8..2778a846 100644 --- a/tests/__init__.py +++ b/tests/__init__.py @@ -67,9 +67,8 @@ def requests_get(url, method='GET', endpoint=None, data='', **kwargs): class MockPostmaster(object): - def __init__(self, is_running=True, is_single_master=False): - self.is_running = Mock(return_value=is_running) - self.is_single_master = Mock(return_value=is_single_master) + def __init__(self, pid=1): + self.is_running = Mock(return_value=self) self.wait_for_user_backends_to_close = Mock() self.signal_stop = Mock(return_value=None) self.wait = Mock() @@ -191,7 +190,7 @@ class PostgresInit(unittest.TestCase): @patch.object(ConfigHandler, 'write_postgresql_conf', Mock()) @patch.object(ConfigHandler, 'replace_pg_hba', Mock()) @patch.object(ConfigHandler, 'replace_pg_ident', Mock()) - @patch.object(Postgresql, 'get_postgres_role_from_data_directory', Mock(return_value='master')) + @patch.object(Postgresql, 'get_postgres_role_from_data_directory', Mock(return_value='primary')) def setUp(self): data_dir = os.path.join('data', 'test0') self.p = Postgresql({'name': 'postgresql0', 'scope': 'batman', 'data_dir': data_dir, diff --git a/tests/test_api.py b/tests/test_api.py index ffe85c7e..6cce5baf 100644 --- a/tests/test_api.py +++ b/tests/test_api.py @@ -25,7 +25,7 @@ class MockPostgresql(object): name = 'test' state = 'running' - role = 'master' + role = 'primary' server_version = '999999' sysid = 'dummysysid' scope = 'dummy' @@ -189,7 +189,7 @@ class TestRestApiHandler(unittest.TestCase): MockRestApiServer(RestApiHandler, 'GET /read-only') with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={})): MockRestApiServer(RestApiHandler, 'GET /replica') - with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'master'})): + with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'primary'})): MockRestApiServer(RestApiHandler, 'GET /replica') with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'state': 'running'})): MockRestApiServer(RestApiHandler, 'GET /health') @@ -208,11 +208,11 @@ class TestRestApiHandler(unittest.TestCase): with patch.object(MockHa, 'is_standby_cluster', Mock(return_value=True)): MockRestApiServer(RestApiHandler, 'GET /standby_leader') MockPatroni.dcs.cluster = None - with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'master'})): - MockRestApiServer(RestApiHandler, 'GET /master') + with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'primary'})): + MockRestApiServer(RestApiHandler, 'GET /primary') with patch.object(MockHa, 'restart_scheduled', Mock(return_value=True)): - MockRestApiServer(RestApiHandler, 'GET /master') - self.assertIsNotNone(MockRestApiServer(RestApiHandler, 'GET /master')) + MockRestApiServer(RestApiHandler, 'GET /primary') + self.assertIsNotNone(MockRestApiServer(RestApiHandler, 'GET /primary')) with patch.object(RestApiServer, 'query', Mock(return_value=[('', 1, '', '', '', '', False, '')])): self.assertIsNotNone(MockRestApiServer(RestApiHandler, 'GET /patroni')) with patch.object(MockHa, 'is_standby_cluster', Mock(return_value=True)): @@ -220,30 +220,30 @@ class TestRestApiHandler(unittest.TestCase): # test tags # - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag') - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=False&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag') - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1.0&tag_key4=1.4&tag_key5=RandomTag') - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag&tag_key6=RandomTag2') # - with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'master'})): - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'primary'})): + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag') - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=False&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag') - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1.0&tag_key4=1.4&tag_key5=RandomTag') - MockRestApiServer(RestApiHandler, 'GET /master?lag=1M&' + MockRestApiServer(RestApiHandler, 'GET /primary?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag&tag_key6=RandomTag2') # @@ -274,7 +274,7 @@ class TestRestApiHandler(unittest.TestCase): 'tag_key1=true&tag_key2=false&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag&tag_key6=RandomTag2') # - with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'master'})): + with patch.object(RestApiHandler, 'get_postgresql_status', Mock(return_value={'role': 'primary'})): MockRestApiServer(RestApiHandler, 'GET /replica?lag=1M&' 'tag_key1=true&tag_key2=false&' 'tag_key3=1&tag_key4=1.4&tag_key5=RandomTag') @@ -428,27 +428,27 @@ class TestRestApiHandler(unittest.TestCase): request = make_request(schedule=future_restart_time.isoformat(), role='unknown', postgres_version='9.5.3') MockRestApiServer(RestApiHandler, request) # wrong version - request = make_request(schedule=future_restart_time.isoformat(), role='master', postgres_version='9.5.3.1') + request = make_request(schedule=future_restart_time.isoformat(), role='primary', postgres_version='9.5.3.1') MockRestApiServer(RestApiHandler, request) # unknown filter request = make_request(schedule=future_restart_time.isoformat(), batman='lives') MockRestApiServer(RestApiHandler, request) # incorrect schedule - request = make_request(schedule='2016-08-42 12:45TZ+1', role='master') + request = make_request(schedule='2016-08-42 12:45TZ+1', role='primary') MockRestApiServer(RestApiHandler, request) # everything fine, but the schedule is missing - request = make_request(role='master', postgres_version='9.5.2') + request = make_request(role='primary', postgres_version='9.5.2') MockRestApiServer(RestApiHandler, request) for retval in (True, False): with patch.object(MockHa, 'schedule_future_restart', Mock(return_value=retval)): request = make_request(schedule=future_restart_time.isoformat()) MockRestApiServer(RestApiHandler, request) with patch.object(MockHa, 'restart', Mock(return_value=(retval, "foo"))): - request = make_request(role='master', postgres_version='9.5.2') + request = make_request(role='primary', postgres_version='9.5.2') MockRestApiServer(RestApiHandler, request) mock_dcs.get_cluster.return_value.is_paused.return_value = True - MockRestApiServer(RestApiHandler, make_request(schedule='2016-08-42 12:45TZ+1', role='master')) + MockRestApiServer(RestApiHandler, make_request(schedule='2016-08-42 12:45TZ+1', role='primary')) # Valid timeout MockRestApiServer(RestApiHandler, make_request(timeout='60s')) # Invalid timeout diff --git a/tests/test_aws.py b/tests/test_aws.py index 4f9762a9..91345f33 100644 --- a/tests/test_aws.py +++ b/tests/test_aws.py @@ -37,15 +37,15 @@ class TestAWSConnection(unittest.TestCase): self.conn = AWSConnection('test') def test_on_role_change(self): - self.assertTrue(self.conn.on_role_change('master')) + self.assertTrue(self.conn.on_role_change('primary')) with patch.object(MockVolumes, 'filter', Mock(return_value=[])): self.conn._retry.max_tries = 1 - self.assertFalse(self.conn.on_role_change('master')) + self.assertFalse(self.conn.on_role_change('primary')) @patch('patroni.scripts.aws.requests_get', Mock(side_effect=Exception('foo'))) def test_non_aws(self): conn = AWSConnection('test') - self.assertFalse(conn.on_role_change("master")) + self.assertFalse(conn.on_role_change("primary")) @patch('patroni.scripts.aws.requests_get', Mock(return_value=urllib3.HTTPResponse(status=200, body=b'foo'))) def test_aws_bizare_response(self): diff --git a/tests/test_config.py b/tests/test_config.py index 9406ea49..6b607c1f 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -21,7 +21,9 @@ class TestConfig(unittest.TestCase): def test_set_dynamic_configuration(self): with patch.object(Config, '_build_effective_configuration', Mock(side_effect=Exception)): self.assertIsNone(self.config.set_dynamic_configuration({'foo': 'bar'})) - self.assertTrue(self.config.set_dynamic_configuration({'synchronous_mode': True, 'standby_cluster': {}})) + self.assertTrue(self.config.set_dynamic_configuration({'synchronous_mode': True, + 'standby_cluster': {}, 'master_start_timeout': 1})) + self.assertEqual(self.config.get('primary_start_timeout'), 1) def test_reload_local_configuration(self): os.environ.update({ diff --git a/tests/test_consul.py b/tests/test_consul.py index 64d75dda..7934551c 100644 --- a/tests/test_consul.py +++ b/tests/test_consul.py @@ -229,7 +229,7 @@ class TestConsul(unittest.TestCase): def test_set_history_value(self): self.assertTrue(self.c.set_history_value('{}')) - @patch.object(consul.Consul.Agent.Service, 'register', Mock(side_effect=(False, True))) + @patch.object(consul.Consul.Agent.Service, 'register', Mock(side_effect=(False, True, True, True))) @patch.object(consul.Consul.Agent.Service, 'deregister', Mock(return_value=True)) def test_update_service(self): d = {'role': 'replica', 'api_url': 'http://a/t', 'conn_url': 'pg://c:1', 'state': 'running'} @@ -244,6 +244,9 @@ class TestConsul(unittest.TestCase): d['state'] = 'running' d['role'] = 'bla' self.assertIsNone(self.c.update_service({}, d)) + for role in ('master', 'primary'): + d['role'] = role + self.assertTrue(self.c.update_service({}, d)) @patch.object(consul.Consul.KV, 'put', Mock(side_effect=ConsulException)) def test_reload_config(self): diff --git a/tests/test_ctl.py b/tests/test_ctl.py index 67ae1fe7..decd2779 100644 --- a/tests/test_ctl.py +++ b/tests/test_ctl.py @@ -25,6 +25,7 @@ from .test_ha import get_cluster_initialized_without_leader, get_cluster_initial 'etcd': {'host': 'localhost:2379'}, 'citus': {'database': 'citus', 'group': 0}, 'postgresql': {'data_dir': '.', 'pgpass': './pgpass', 'parameters': {}, 'retry_timeout': 5}})) class TestCtl(unittest.TestCase): + TEST_ROLES = ('master', 'primary', 'leader') @patch('socket.getaddrinfo', socket_getaddrinfo) def setUp(self): @@ -58,9 +59,9 @@ class TestCtl(unittest.TestCase): @patch('patroni.psycopg.connect', psycopg_connect) def test_get_cursor(self): - self.assertIsNone(get_cursor({}, get_cluster_initialized_without_leader(), None, {}, role='master')) - - self.assertIsNotNone(get_cursor({}, get_cluster_initialized_with_leader(), None, {}, role='master')) + for role in self.TEST_ROLES: + self.assertIsNone(get_cursor({}, get_cluster_initialized_without_leader(), None, {}, role=role)) + self.assertIsNotNone(get_cursor({}, get_cluster_initialized_with_leader(), None, {}, role=role)) # MockCursor returns pg_is_in_recovery as false self.assertIsNone(get_cursor({}, get_cluster_initialized_with_leader(), None, {}, role='replica')) @@ -160,7 +161,7 @@ class TestCtl(unittest.TestCase): result = self.runner.invoke(ctl, ['switchover', 'dummy', '--group', '0'], input='leader\nother\n\ny') assert result.exit_code == 1 - # No master available + # No leader available mock_get_dcs.return_value.get_cluster = get_cluster_initialized_without_leader result = self.runner.invoke(ctl, ['switchover', 'dummy', '--group', '0'], input='leader\nother\n\ny') assert result.exit_code == 1 @@ -187,8 +188,9 @@ class TestCtl(unittest.TestCase): def test_query(self, mock_get_dcs): mock_get_dcs.return_value = self.e # Mutually exclusive - result = self.runner.invoke(ctl, ['query', 'alpha', '--member', 'abc', '--role', 'master']) - assert result.exit_code == 1 + for role in self.TEST_ROLES: + result = self.runner.invoke(ctl, ['query', 'alpha', '--member', 'abc', '--role', role]) + assert result.exit_code == 1 with self.runner.isolated_filesystem(): with open('dummy', 'w') as dummy_file: @@ -216,8 +218,9 @@ class TestCtl(unittest.TestCase): def test_query_member(self): with patch('patroni.ctl.get_cursor', Mock(return_value=MockConnect().cursor())): - rows = query_member({}, None, None, None, None, 'master', 'SELECT pg_catalog.pg_is_in_recovery()', {}) - self.assertTrue('False' in str(rows)) + for role in self.TEST_ROLES: + rows = query_member({}, None, None, None, None, role, 'SELECT pg_catalog.pg_is_in_recovery()', {}) + self.assertTrue('False' in str(rows)) with patch.object(MockCursor, 'execute', Mock(side_effect=OperationalError('bla'))): rows = query_member({}, None, None, None, None, 'replica', 'SELECT pg_catalog.pg_is_in_recovery()', {}) @@ -239,8 +242,9 @@ class TestCtl(unittest.TestCase): assert 'host=127.0.0.1 port=5435' in result.output # Mutually exclusive options - result = self.runner.invoke(ctl, ['dsn', 'alpha', '--role', 'master', '--member', 'dummy']) - assert result.exit_code == 1 + for role in self.TEST_ROLES: + result = self.runner.invoke(ctl, ['dsn', 'alpha', '--role', role, '--member', 'dummy']) + assert result.exit_code == 1 # Non-existing member result = self.runner.invoke(ctl, ['dsn', 'alpha', '--member', 'dummy']) @@ -345,14 +349,14 @@ class TestCtl(unittest.TestCase): mock_get_dcs.return_value.get_cluster = get_cluster_initialized_with_leader result = self.runner.invoke(ctl, ['remove', 'dummy'], input='\n') assert 'For Citus clusters the --group must me specified' in result.output - result = self.runner.invoke(ctl, ['-k', 'remove', 'alpha', '--group', '0'], input='alpha\nslave') + result = self.runner.invoke(ctl, ['-k', 'remove', 'alpha', '--group', '0'], input='alpha\nstandby') assert 'Please confirm' in result.output assert 'You are about to remove all' in result.output # Not typing an exact confirmation assert result.exit_code == 1 - # master specified does not match master of cluster - result = self.runner.invoke(ctl, ['remove', 'alpha', '--group', '0'], input='alpha\nYes I am aware\nslave') + # leader specified does not match leader of cluster + result = self.runner.invoke(ctl, ['remove', 'alpha', '--group', '0'], input='alpha\nYes I am aware\nstandby') assert result.exit_code == 1 # cluster specified on cmdline does not match verification prompt @@ -369,17 +373,19 @@ class TestCtl(unittest.TestCase): assert 'Usage:' in result.output def test_get_any_member(self): - self.assertIsNone(get_any_member({}, get_cluster_initialized_without_leader(), None, role='master')) + for role in self.TEST_ROLES: + self.assertIsNone(get_any_member({}, get_cluster_initialized_without_leader(), None, role=role)) - m = get_any_member({}, get_cluster_initialized_with_leader(), None, role='master') - self.assertEqual(m.name, 'leader') + m = get_any_member({}, get_cluster_initialized_with_leader(), None, role=role) + self.assertEqual(m.name, 'leader') def test_get_all_members(self): - self.assertEqual(list(get_all_members({}, get_cluster_initialized_without_leader(), None, role='master')), []) + for role in self.TEST_ROLES: + self.assertEqual(list(get_all_members({}, get_cluster_initialized_without_leader(), None, role=role)), []) - r = list(get_all_members({}, get_cluster_initialized_with_leader(), None, role='master')) - self.assertEqual(len(r), 1) - self.assertEqual(r[0].name, 'leader') + r = list(get_all_members({}, get_cluster_initialized_with_leader(), None, role=role)) + self.assertEqual(len(r), 1) + self.assertEqual(r[0].name, 'leader') r = list(get_all_members({}, get_cluster_initialized_with_leader(), None, role='replica')) self.assertEqual(len(r), 1) @@ -467,8 +473,9 @@ class TestCtl(unittest.TestCase): mock_get_dcs.return_value = self.e mock_get_dcs.return_value.get_cluster = get_cluster_initialized_with_leader - result = self.runner.invoke(ctl, ['flush', 'dummy', 'restart', '-r', 'master'], input='y') - assert 'No scheduled restart' in result.output + for role in self.TEST_ROLES: + result = self.runner.invoke(ctl, ['flush', 'dummy', 'restart', '-r', role], input='y') + assert 'No scheduled restart' in result.output result = self.runner.invoke(ctl, ['flush', 'dummy', 'restart', '--force']) assert 'Success: flush scheduled restart' in result.output diff --git a/tests/test_ha.py b/tests/test_ha.py index 38b4ca47..65a3460c 100644 --- a/tests/test_ha.py +++ b/tests/test_ha.py @@ -53,7 +53,7 @@ def get_cluster_bootstrapping_without_leader(cluster_config=None): def get_cluster_initialized_without_leader(leader=False, failover=None, sync=None, cluster_config=None, failsafe=False): m1 = Member(0, 'leader', 28, {'conn_url': 'postgres://replicator:rep-pass@127.0.0.1:5435/postgres', 'api_url': 'http://127.0.0.1:8008/patroni', 'xlog_location': 4, - 'role': 'master', 'state': 'running'}) + 'role': 'primary', 'state': 'running'}) leader = Leader(0, 0, m1 if leader else Member(0, '', 28, {})) m2 = Member(0, 'other', 28, {'conn_url': 'postgres://replicator:rep-pass@127.0.0.1:5436/postgres', 'api_url': 'http://127.0.0.1:8011/patroni', @@ -181,7 +181,7 @@ def run_async(self, func, args=()): @patch.object(Postgresql, 'checkpoint', Mock()) @patch.object(CancellableSubprocess, 'call', Mock(return_value=0)) @patch.object(Postgresql, 'get_replica_timeline', Mock(return_value=2)) -@patch.object(Postgresql, 'get_master_timeline', Mock(return_value=2)) +@patch.object(Postgresql, 'get_primary_timeline', Mock(return_value=2)) @patch.object(ConfigHandler, 'restore_configuration_files', Mock()) @patch.object(etcd.Client, 'write', etcd_write) @patch.object(etcd.Client, 'read', etcd_read) @@ -229,7 +229,7 @@ class TestHa(PostgresInit): self.p.timeline_wal_position = Mock(return_value=(0, 1, 1)) self.p.set_role('standby_leader') self.ha.touch_member() - self.p.set_role('master') + self.p.set_role('primary') self.ha.dcs.touch_member = true self.ha.touch_member() @@ -280,11 +280,11 @@ class TestHa(PostgresInit): self.ha.dcs.__class__.__name__ = 'Raft' self.assertEqual(self.ha.run_cycle(), 'started as a secondary') - def test_recover_former_master(self): + def test_recover_former_primary(self): self.p.follow = false self.p.is_running = false self.p.name = 'leader' - self.p.set_role('master') + self.p.set_role('primary') self.p.controldata = lambda: {'Database cluster state': 'shut down', 'Database system identifier': SYSID} self.ha.cluster = get_cluster_initialized_with_leader() self.assertEqual(self.ha.run_cycle(), 'starting as readonly because i had the session lock') @@ -322,7 +322,7 @@ class TestHa(PostgresInit): def test_recover_with_rewind(self): self.p.is_running = false self.ha.cluster = get_cluster_initialized_with_leader() - self.ha.cluster.leader.member.data.update(version='2.0.2', role='master') + self.ha.cluster.leader.member.data.update(version='2.0.2', role='primary') self.ha._rewind.pg_rewind = true self.ha._rewind.check_leader_is_not_in_recovery = true with patch.object(Rewind, 'rewind_or_reinitialize_needed_and_possible', Mock(return_value=True)): @@ -359,7 +359,7 @@ class TestHa(PostgresInit): self.assertEqual(self.ha.run_cycle(), 'promoted self to leader because I had the session lock') @patch('patroni.psycopg.connect', psycopg_connect) - def test_acquire_lock_as_master(self): + def test_acquire_lock_as_primary(self): self.assertEqual(self.ha.run_cycle(), 'acquired session lock as a leader') def test_promoted_by_acquiring_lock(self): @@ -384,7 +384,7 @@ class TestHa(PostgresInit): self.ha.cluster.is_unlocked = false self.ha.has_lock = true self.p.is_leader = false - self.p.set_role('master') + self.p.set_role('primary') self.assertEqual(self.ha.run_cycle(), 'no action. I am (postgresql0), the leader with the lock') def test_demote_after_failing_to_obtain_lock(self): @@ -468,7 +468,7 @@ class TestHa(PostgresInit): def test_follow_in_pause(self): self.ha.cluster.is_unlocked = false self.ha.is_paused = true - self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as master without lock') + self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as primary without lock') self.p.is_leader = false self.assertEqual(self.ha.run_cycle(), 'PAUSE: no action. I am (postgresql0)') @@ -480,7 +480,7 @@ class TestHa(PostgresInit): self.ha.cluster = get_cluster_initialized_with_leader() self.assertEqual(self.ha.run_cycle(), 'running pg_rewind from leader') - def test_no_dcs_connection_master_demote(self): + def test_no_dcs_connection_primary_demote(self): self.ha.load_cluster_from_dcs = Mock(side_effect=DCSError('Etcd is not responding properly')) self.assertEqual(self.ha.run_cycle(), 'demoting self because DCS is not accessible and I was a leader') self.ha._async_executor.schedule('dummy') @@ -539,7 +539,7 @@ class TestHa(PostgresInit): def test_update_failsafe(self): self.assertRaises(Exception, self.ha.update_failsafe, {}) - self.p.set_role('master') + self.p.set_role('primary') self.assertEqual(self.ha.update_failsafe({}), 'Running as a leader') @patch('time.sleep', Mock()) @@ -655,7 +655,7 @@ class TestHa(PostgresInit): self.assertEqual(self.ha.run_cycle(), 'updated leader lock during restart') self.ha.update_lock = false - self.p.set_role('master') + self.p.set_role('primary') with patch('patroni.async_executor.CriticalTask.cancel', Mock(return_value=False)): with patch('patroni.postgresql.Postgresql.terminate_starting_postmaster') as mock_terminate: self.assertEqual(self.ha.run_cycle(), 'lost leader lock during restart') @@ -813,9 +813,9 @@ class TestHa(PostgresInit): def test_manual_failover_process_no_leader_in_pause(self): self.ha.is_paused = true self.ha.cluster = get_cluster_initialized_without_leader(failover=Failover(0, '', 'other', None)) - self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as master without lock') + self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as primary without lock') self.ha.cluster = get_cluster_initialized_without_leader(failover=Failover(0, 'leader', '', None)) - self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as master without lock') + self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as primary without lock') self.ha.cluster = get_cluster_initialized_without_leader(failover=Failover(0, 'leader', 'blabla', None)) self.assertEqual('PAUSE: acquired session lock as a leader', self.ha.run_cycle()) self.p.is_leader = false @@ -874,11 +874,11 @@ class TestHa(PostgresInit): def test_post_recover(self): self.p.is_running = false self.ha.has_lock = true - self.p.set_role('master') + self.p.set_role('primary') self.assertEqual(self.ha.post_recover(), 'removed leader key after trying and failing to start postgres') self.ha.has_lock = false self.assertEqual(self.ha.post_recover(), 'failed to start postgres') - leader = Leader(0, 0, Member(0, 'l', 2, {"version": "1.6", "conn_url": "postgres://a", "role": "master"})) + leader = Leader(0, 0, Member(0, 'l', 2, {"version": "1.6", "conn_url": "postgres://a", "role": "primary"})) self.ha._rewind.execute(leader) self.p.is_running = true self.assertIsNone(self.ha.post_recover()) @@ -925,7 +925,7 @@ class TestHa(PostgresInit): self.p._role = 'replica' self.p._connection.server_version = 90500 self.p._pending_restart = True - self.assertFalse(self.ha.restart_matches("master", "9.5.0", True)) + self.assertFalse(self.ha.restart_matches("primary", "9.5.0", True)) self.assertFalse(self.ha.restart_matches("replica", "9.4.3", True)) self.p._pending_restart = False self.assertFalse(self.ha.restart_matches("replica", "9.5.2", True)) @@ -936,9 +936,9 @@ class TestHa(PostgresInit): self.ha.is_paused = true self.p.name = 'leader' self.ha.cluster = get_cluster_initialized_with_leader() - self.assertEqual(self.ha.run_cycle(), 'PAUSE: removed leader lock because postgres is not running as master') + self.assertEqual(self.ha.run_cycle(), 'PAUSE: removed leader lock because postgres is not running as primary') self.ha.cluster = get_cluster_initialized_with_leader(Failover(0, '', self.p.name, None)) - self.assertEqual(self.ha.run_cycle(), 'PAUSE: waiting to become master after promote...') + self.assertEqual(self.ha.run_cycle(), 'PAUSE: waiting to become primary after promote...') @patch('patroni.postgresql.mtime', Mock(return_value=1588316884)) @patch.object(builtins, 'open', mock_open(read_data='1\t0/40159C0\tno recovery target specified\n')) @@ -979,7 +979,7 @@ class TestHa(PostgresInit): self.p.name = 'replica' self.ha.cluster = get_standby_cluster_initialized_with_only_leader() self.ha.is_unlocked = true - self.assertTrue(self.ha.run_cycle().startswith('running pg_rewind from remote_master:')) + self.assertTrue(self.ha.run_cycle().startswith('running pg_rewind from remote_member:')) def test_recover_unhealthy_leader_in_standby_cluster(self): self.p.is_leader = false @@ -997,7 +997,7 @@ class TestHa(PostgresInit): self.ha.cluster = get_standby_cluster_initialized_with_only_leader() self.ha.cluster.is_unlocked = true self.ha.has_lock = false - self.assertEqual(self.ha.run_cycle(), 'trying to follow a remote master because standby cluster is unhealthy') + self.assertEqual(self.ha.run_cycle(), 'trying to follow a remote member because standby cluster is unhealthy') def test_failed_to_update_lock_in_pause(self): self.ha.update_lock = false @@ -1005,7 +1005,7 @@ class TestHa(PostgresInit): self.p.name = 'leader' self.ha.cluster = get_cluster_initialized_with_leader() self.assertEqual(self.ha.run_cycle(), - 'PAUSE: continue to run as master after failing to update leader lock in DCS') + 'PAUSE: continue to run as primary after failing to update leader lock in DCS') def test_postgres_unhealthy_in_pause(self): self.ha.is_paused = true @@ -1039,7 +1039,7 @@ class TestHa(PostgresInit): self.p.time_in_state = lambda: 350 self.ha.fetch_node_status = get_node_status(reachable=False) # inaccessible, in_recovery self.assertEqual(self.ha.run_cycle(), - 'master start has timed out, but continuing to wait because failover is not possible') + 'primary start has timed out, but continuing to wait because failover is not possible') check_calls([(update_lock, True), (demote, False)]) self.ha.fetch_node_status = get_node_status() # accessible, in_recovery @@ -1065,27 +1065,27 @@ class TestHa(PostgresInit): self.assertEqual(self.ha.run_cycle(), 'manual failover: demoting myself') @patch('patroni.ha.Ha.demote') - def test_failover_immediately_on_zero_master_start_timeout(self, demote): + def test_failover_immediately_on_zero_primary_start_timeout(self, demote): self.p.is_running = false self.ha.cluster = get_cluster_initialized_with_leader(sync=(self.p.name, 'other')) self.ha.cluster.config.data['synchronous_mode'] = True - self.ha.patroni.config.set_dynamic_configuration({'master_start_timeout': 0}) + self.ha.patroni.config.set_dynamic_configuration({'primary_start_timeout': 0}) self.ha.has_lock = true self.ha.update_lock = true self.ha.fetch_node_status = get_node_status() # accessible, in_recovery self.assertEqual(self.ha.run_cycle(), 'stopped PostgreSQL to fail over after a crash') demote.assert_called_once() - def test_master_stop_timeout(self): - self.assertEqual(self.ha.master_stop_timeout(), None) - self.ha.patroni.config.set_dynamic_configuration({'master_stop_timeout': 30}) + def test_primary_stop_timeout(self): + self.assertEqual(self.ha.primary_stop_timeout(), None) + self.ha.patroni.config.set_dynamic_configuration({'primary_stop_timeout': 30}) with patch.object(Ha, 'is_synchronous_mode', Mock(return_value=True)): - self.assertEqual(self.ha.master_stop_timeout(), 30) - self.ha.patroni.config.set_dynamic_configuration({'master_stop_timeout': 30}) + self.assertEqual(self.ha.primary_stop_timeout(), 30) + self.ha.patroni.config.set_dynamic_configuration({'primary_stop_timeout': 30}) with patch.object(Ha, 'is_synchronous_mode', Mock(return_value=False)): - self.assertEqual(self.ha.master_stop_timeout(), None) - self.ha.patroni.config.set_dynamic_configuration({'master_stop_timeout': None}) - self.assertEqual(self.ha.master_stop_timeout(), None) + self.assertEqual(self.ha.primary_stop_timeout(), None) + self.ha.patroni.config.set_dynamic_configuration({'primary_stop_timeout': None}) + self.assertEqual(self.ha.primary_stop_timeout(), None) @patch('patroni.postgresql.Postgresql.follow') def test_demote_immediate(self, follow): @@ -1177,7 +1177,7 @@ class TestHa(PostgresInit): self.ha.run_cycle() mock_set_sync.assert_called_once_with(['*']) - def test_sync_replication_become_master(self): + def test_sync_replication_become_primary(self): self.ha.is_synchronous_mode = true mock_set_sync = self.p.sync_handler.set_synchronous_standby_names = Mock() @@ -1188,17 +1188,17 @@ class TestHa(PostgresInit): self.p.name = 'leader' self.ha.cluster = get_cluster_initialized_with_leader(sync=('other', None)) - # When we just became master nobody is sync - self.assertEqual(self.ha.enforce_master_role('msg', 'promote msg'), 'promote msg') + # When we just became primary nobody is sync + self.assertEqual(self.ha.enforce_primary_role('msg', 'promote msg'), 'promote msg') mock_set_sync.assert_called_once_with([]) mock_write_sync.assert_called_once_with('leader', None, index=0) mock_set_sync.reset_mock() - # When we just became master nobody is sync + # When we just became primary nobody is sync self.p.set_role('replica') mock_write_sync.return_value = False - self.assertTrue(self.ha.enforce_master_role('msg', 'promote msg') != 'promote msg') + self.assertTrue(self.ha.enforce_primary_role('msg', 'promote msg') != 'promote msg') mock_set_sync.assert_not_called() def test_unhealthy_sync_mode(self): @@ -1332,7 +1332,7 @@ class TestHa(PostgresInit): self.ha.has_lock = true self.ha.cluster.is_unlocked = false for tl in (1, 3): - self.p.get_master_timeline = Mock(return_value=tl) + self.p.get_primary_timeline = Mock(return_value=tl) self.assertEqual(self.ha.run_cycle(), 'no action. I am (postgresql0), the leader with the lock') @patch('sys.exit', return_value=1) @@ -1377,7 +1377,7 @@ class TestHa(PostgresInit): def test_sysid_no_match_in_pause(self): self.ha.is_paused = true self.p.controldata = lambda: {'Database cluster state': 'in recovery', 'Database system identifier': '123'} - self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as master without lock') + self.assertEqual(self.ha.run_cycle(), 'PAUSE: continue to run as primary without lock') self.ha.has_lock = true self.assertEqual(self.ha.run_cycle(), 'PAUSE: released leader key voluntarily due to the system ID mismatch') diff --git a/tests/test_kubernetes.py b/tests/test_kubernetes.py index b0a9dcdd..7e13210f 100644 --- a/tests/test_kubernetes.py +++ b/tests/test_kubernetes.py @@ -293,7 +293,7 @@ class TestKubernetesConfigMaps(BaseTestKubernetes): self.k.touch_member({'role': 'replica'}) self.k._name = 'p-1' self.k.touch_member({'state': 'running', 'role': 'replica'}) - self.k.touch_member({'state': 'stopped', 'role': 'master'}) + self.k.touch_member({'state': 'stopped', 'role': 'primary'}) def test_initialize(self): self.k.initialize() diff --git a/tests/test_postgresql.py b/tests/test_postgresql.py index b8725218..40411208 100644 --- a/tests/test_postgresql.py +++ b/tests/test_postgresql.py @@ -144,7 +144,8 @@ class TestPostgresql(BaseTestPostgresql): @patch('patroni.postgresql.polling_loop', Mock(return_value=range(1))) def test_wait_for_port_open(self, mock_pg_isready): mock_pg_isready.return_value = STATE_NO_RESPONSE - mock_postmaster = MockPostmaster(is_running=False) + mock_postmaster = MockPostmaster() + mock_postmaster.is_running.return_value = None # No pid file and postmaster death self.assertFalse(self.p.wait_for_port_open(mock_postmaster, 1)) @@ -502,13 +503,13 @@ class TestPostgresql(BaseTestPostgresql): self.p.config._config['create_replica_method'] = [] self.assertFalse(self.p.can_create_replica_without_replication_connection()) self.p.config._config['create_replica_method'] = ['wale', 'basebackup'] - self.p.config._config['wale'] = {'command': 'foo', 'no_master': 1} + self.p.config._config['wale'] = {'command': 'foo', 'no_leader': 1} self.assertTrue(self.p.can_create_replica_without_replication_connection()) def test_replica_method_can_work_without_replication_connection(self): self.assertFalse(self.p.replica_method_can_work_without_replication_connection('basebackup')) self.assertFalse(self.p.replica_method_can_work_without_replication_connection('foobar')) - self.p.config._config['foo'] = {'command': 'bar', 'no_master': 1} + self.p.config._config['foo'] = {'command': 'bar', 'no_leader': 1} self.assertTrue(self.p.replica_method_can_work_without_replication_connection('foo')) self.p.config._config['foo'] = {'command': 'bar'} self.assertFalse(self.p.replica_method_can_work_without_replication_connection('foo')) @@ -669,8 +670,8 @@ class TestPostgresql(BaseTestPostgresql): def test_replica_cached_timeline(self): self.assertEqual(self.p.replica_cached_timeline(2), 3) - def test_get_master_timeline(self): - self.assertEqual(self.p.get_master_timeline(), 1) + def test_get_primary_timeline(self): + self.assertEqual(self.p.get_primary_timeline(), 1) @patch.object(Postgresql, 'get_postgres_role_from_data_directory', Mock(return_value='replica')) @patch.object(Bootstrap, 'running_custom_bootstrap', PropertyMock(return_value=True)) diff --git a/tests/test_rewind.py b/tests/test_rewind.py index 9da500e9..63010bb2 100644 --- a/tests/test_rewind.py +++ b/tests/test_rewind.py @@ -117,7 +117,7 @@ class TestRewind(BaseTestPostgresql): self.r.trigger_check_diverged_lsn() self.r.execute(self.leader) - self.leader.member.data.update(version='1.5.7', checkpoint_after_promote=False, role='master') + self.leader.member.data.update(version='1.5.7', checkpoint_after_promote=False, role='primary') self.assertIsNone(self.r.execute(self.leader)) del self.leader.member.data['checkpoint_after_promote'] @@ -128,9 +128,9 @@ class TestRewind(BaseTestPostgresql): self.r.execute(self.leader) @patch('patroni.postgresql.rewind.logger.info') - def test__log_master_history(self, mock_logger): + def test__log_primary_history(self, mock_logger): history = [[n, n, ''] for n in range(1, 10)] - self.r._log_master_history(history, 1) + self.r._log_primary_history(history, 1) expected = '\n'.join(['{0}\t0/{0}\t'.format(n) for n in range(1, 4)] + ['...', '9\t0/9\t']) self.assertEqual(mock_logger.call_args[0][1], expected) @@ -298,14 +298,14 @@ class TestRewind(BaseTestPostgresql): @patch('patroni.postgresql.rewind.Thread', MockThread) @patch.object(Postgresql, 'controldata') @patch.object(Postgresql, 'checkpoint') - @patch.object(Postgresql, 'get_master_timeline') - def test_ensure_checkpoint_after_promote(self, mock_get_master_timeline, mock_checkpoint, mock_controldata): + @patch.object(Postgresql, 'get_primary_timeline') + def test_ensure_checkpoint_after_promote(self, mock_get_primary_timeline, mock_checkpoint, mock_controldata): mock_controldata.return_value = {"Latest checkpoint's TimeLineID": 1} - mock_get_master_timeline.return_value = 1 + mock_get_primary_timeline.return_value = 1 self.r.ensure_checkpoint_after_promote(Mock()) self.r.reset_state() - mock_get_master_timeline.return_value = 2 + mock_get_primary_timeline.return_value = 2 mock_checkpoint.return_value = 0 self.r.ensure_checkpoint_after_promote(Mock()) self.r.ensure_checkpoint_after_promote(Mock()) diff --git a/tests/test_slots.py b/tests/test_slots.py index 6fe1691c..95e55851 100644 --- a/tests/test_slots.py +++ b/tests/test_slots.py @@ -52,7 +52,7 @@ class TestSlotsHandler(BaseTestPostgresql): patch.object(SlotsHandler, 'drop_replication_slot') as mock_drop: self.s.sync_replication_slots(cluster, False, paused=True) mock_drop.assert_not_called() - self.p.set_role('master') + self.p.set_role('primary') with mock.patch('patroni.postgresql.Postgresql.role', new_callable=PropertyMock(return_value='replica')): self.s.sync_replication_slots(cluster, False) with patch('patroni.dcs.logger.error', new_callable=Mock()) as errorlog_mock: diff --git a/tests/test_wale_restore.py b/tests/test_wale_restore.py index fc5b4e08..2aff0e09 100644 --- a/tests/test_wale_restore.py +++ b/tests/test_wale_restore.py @@ -59,22 +59,22 @@ class TestWALERestore(unittest.TestCase): self.assertFalse(self.wale_restore.should_use_s3_to_create_replica()) with patch('patroni.psycopg.connect', Mock(side_effect=psycopg.Error("foo"))): - save_no_master = self.wale_restore.no_master - save_master_connection = self.wale_restore.master_connection + save_no_leader = self.wale_restore.no_leader + save_leader_connection = self.wale_restore.leader_connection self.assertFalse(self.wale_restore.should_use_s3_to_create_replica()) with patch('time.sleep', mock_sleep): - self.wale_restore.no_master = 1 + self.wale_restore.no_leader = 1 self.assertTrue(self.wale_restore.should_use_s3_to_create_replica()) # verify retries self.assertEqual(sleeps[0], WALE_TEST_RETRIES) - self.wale_restore.master_connection = '' + self.wale_restore.leader_connection = '' self.assertTrue(self.wale_restore.should_use_s3_to_create_replica()) - self.wale_restore.no_master = save_no_master - self.wale_restore.master_connection = save_master_connection + self.wale_restore.no_leader = save_no_leader + self.wale_restore.leader_connection = save_leader_connection with patch('subprocess.check_output', Mock(side_effect=subprocess.CalledProcessError(1, "cmd", "foo"))): self.assertFalse(self.wale_restore.should_use_s3_to_create_replica())