Release 1.5.4 (#931)

* Bump version
* Update release notes
* Make it possible to configure registration of Service in Consul via env variables

docs/ENVIRONMENT.rst

@@ -41,6 +41,8 @@ Consul
 -  **PATRONI\_CONSUL\_KEY**: (optional) File with the client key. Can be empty if the key is part of certificate.
 -  **PATRONI\_CONSUL\_DC**: (optional) Datacenter to communicate with. By default the datacenter of the host is used.
 -  **PATRONI\_CONSUL\_CHECKS**: (optional) list of Consul health checks used for the session. If not specified Consul will use "serfHealth" in additional to the TTL based check created by Patroni. Additional checks, in particular the "serfHealth", may cause the leader lock to expire faster than in `ttl` seconds when the leader instance becomes unavailable.
+-  **PATRONI\_CONSUL\_REGISTER\_SERVICE**: (optional) whether or not to register a service with the name defined by the scope parameter and the tag master, replica or standby-leader depending on the node's role. Defaults to **false**
+-  **PATRONI\_CONSUL\_SERVICE\_CHECK\_INTERVAL**: (optional) how often to perform health check against registered url
 
 Etcd
 ----

docs/releases.rst

@@ -3,6 +3,65 @@
 Release notes
 =============
 
+Version 1.5.4
+-------------
+
+This version implements flexible logging and fixes a number of bugs.
+
+**New features**
+
+- Improvements in logging infrastructure (Alexander Kukushkin, Lucas Capistrant, Alexander Anikin)
+
+  Logging configuration could be configured not only from environment variables but also from Patroni config file. It makes it possible to change logging configuration in runtime by updating config and doing reload or sending SIGHUP to the Patroni process. By default Patroni writes logs to stderr, but now it becomes possible to write logs directly into the file and rotate when it reaches a certain size. In addition to that added support of custom dateformat and the possibility to fine-tune log level for each python module.
+
+- Make it possible to take into account the current timeline during leader elections (Alexander Kukushkin)
+
+  It could happen that the node is considering itself as a healthiest one although it is currently not on the latest known timeline. In some cases we want to avoid promoting of such node, which could be achieved by setting `check_timeline` parameter to `true` (default behavior remains unchanged).
+
+- Relaxed requirements on superuser credentials
+
+  Libpq allows opening connections without explicitly specifying neither username nor password. Depending on situation it relies either on pgpass file or trust authentication method in pg_hba.conf. Since pg_rewind is also using libpq, it will work the same way.
+
+- Implemented possibility to configure Consul Service registration and check interval via environment variables (Alexander Kukushkin)
+
+  Registration of service in Consul was added in the 1.5.0, but so far it was only possible to turn it on via patroni.yaml.
+
+**Stability Improvements**
+
+- Set archive_mode to off during the custom bootstrap (Alexander Kukushkin)
+
+  We want to avoid archiving wals and history files until the cluster is fully functional.  It really helps if the custom bootstrap involves pg_upgrade.
+
+- Apply five seconds backoff when loading global config on start (Alexander Kukushkin)
+
+  It helps to avoid hammering DCS when Patroni just starting up.
+
+- Reduce amount of error messages generated on shutdown (Alexander Kukushkin)
+
+  They were harmless but rather annoying and sometimes scary.
+
+- Explicitly secure rw perms for recovery.conf at creation time (Lucas)
+
+  We don't want anybody except patroni/postgres user reading this file, because it contains replication user and password.
+
+- Redirect HTTPServer exceptions to logger (Julien Riou)
+
+  By default, such exceptions were logged on standard output messing with regular logs.
+
+**Bug fixes**
+
+- Removed stderr pipe to stdout on pg_ctl process (Cody Coons)
+
+  Inheriting stderr from the main Patroni process allows all Postgres logs to be seen along with all patroni logs. This is very useful in a container environment as Patroni and Postgres logs may be consumed using standard tools (docker logs, kubectl, etc). In addition to that, this change fixes a bug with Patroni not being able to catch postmaster pid when postgres writing some warnings into stderr.
+
+- Set Consul service check deregister timeout in Go time format (Pavel Kirillov)
+
+  Without explicitly mentioned time unit registration was failing.
+
+- Relax checks of standby_cluster cluster configuration (Dmitry Dolgov, Alexander Kukushkin)
+
+  It was accepting only strings as valid values and therefore it was not possible to specify the port as integer and create_replica_methods as a list.
+
 Version 1.5.3
 -------------
 

features/environment.py

@@ -372,6 +372,7 @@ class ConsulController(AbstractDcsController):
     def __init__(self, context):
         super(ConsulController, self).__init__(context)
         os.environ['PATRONI_CONSUL_HOST'] = 'localhost:8500'
+        os.environ['PATRONI_CONSUL_REGISTER_SERVICE'] = 'on'
         self._client = consul.Consul()
         self._config_file = None
 

patroni/config.py

@@ -288,9 +288,9 @@ class Config(object):
                 name, suffix = (param[8:].split('_', 1) + [''])[:2]
                 if name and suffix:
                     # PATRONI_(ETCD|CONSUL|ZOOKEEPER|EXHIBITOR|...)_(HOSTS?|PORT|..)
-                    if suffix in ('HOST', 'HOSTS', 'PORT', 'SRV', 'URL', 'PROXY', 'CACERT', 'CERT',
-                                  'KEY', 'VERIFY', 'TOKEN', 'CHECKS', 'DC', 'NAMESPACE', 'CONTEXT',
-                                  'USE_ENDPOINTS', 'SCOPE_LABEL', 'ROLE_LABEL', 'POD_IP', 'PORTS', 'LABELS'):
+                    if suffix in ('HOST', 'HOSTS', 'PORT', 'SRV', 'URL', 'PROXY', 'CACERT', 'CERT', 'KEY', 'VERIFY',
+                                  'TOKEN', 'CHECKS', 'DC', 'REGISTER_SERVICE', 'SERVICE_CHECK_INTERVAL', 'NAMESPACE',
+                                  'CONTEXT', 'USE_ENDPOINTS', 'SCOPE_LABEL', 'ROLE_LABEL', 'POD_IP', 'PORTS', 'LABELS'):
                         value = os.environ.pop(param)
                         if suffix == 'PORT':
                             value = value and parse_int(value)
@@ -298,6 +298,8 @@ class Config(object):
                             value = value and _parse_list(value)
                         elif suffix == 'LABELS':
                             value = _parse_dict(value)
+                        elif suffix == 'REGISTER_SERVICE':
+                            value = parse_bool(value)
                         if value:
                             ret[name.lower()][suffix.lower()] = value
                     # PATRONI_<username>_PASSWORD=<password>, PATRONI_<username>_OPTIONS=<option1,option2,...>

patroni/version.py

@@ -1 +1 @@
-__version__ = '1.5.3'
+__version__ = '1.5.4'

tests/test_api.py

@@ -401,3 +401,6 @@ class TestRestApiServer(unittest.TestCase):
         self.assertRaises(ValueError, srv.reload_config, bad_config)
         self.assertRaises(ValueError, srv.reload_config, {})
         srv.reload_config({'listen': '127.0.0.2:8008'})
+
+    def test_handle_error(self):
+        self.assertIsNone(MockRestApiServer.handle_error(None, ('127.0.0.1', 55555)))

tests/test_config.py

@@ -51,6 +51,7 @@ class TestConfig(unittest.TestCase):
             'PATRONI_ETCD_CERT': '/cert',
             'PATRONI_ETCD_KEY': '/key',
             'PATRONI_CONSUL_HOST': '127.0.0.1:8500',
+            'PATRONI_CONSUL_REGISTER_SERVICE': 'on',
             'PATRONI_KUBERNETES_LABELS': 'a:b:c',
             'PATRONI_KUBERNETES_SCOPE_LABEL': 'a',
             'PATRONI_KUBERNETES_PORTS': '[{"name": "postgresql"}]',

tests/test_postgresql.py

@@ -651,6 +651,7 @@ class TestPostgresql(unittest.TestCase):
 
     @patch('time.sleep', Mock())
     @patch('os.unlink', Mock())
+    @patch('os.path.isfile', Mock(return_value=True))
     @patch.object(Postgresql, 'run_bootstrap_post_init', Mock(return_value=True))
     @patch.object(Postgresql, '_custom_bootstrap', Mock(return_value=True))
     @patch.object(Postgresql, 'start', Mock(return_value=True))