gitops/patroni
1
0
Fork 0
mirror of https://github.com/outbackdingo/patroni.git synced 2026-08-29 08:59:32 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,755 Commits 15 Branches 66 Tags
6dcaa697c0cb046460f29a9d355af7edefbb507b
Commit Graph
1755 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Alexander Kukushkin 6dcaa697c0 Replace prettytable with terminaltables 
It is a more advanced library and can deal with multi-line strings, what
allows us to present tags in a nice format. It also could nicely show
the table header, so we get rid of redundant Cluster column.

Debian/Ubuntu already have the module packaged as `python3-terminaltables`,
so it shouldn't be a problem for package maintainers.

Example output:
```bash
$ patronictl list
┌ Cluster: batman (6792870884189948744) ┬─────────┬────┬───────────┬─────────────────┬────────────────────────────┐
│ Member      │ Host           │ Role   │ State   │ TL │ Lag in MB │ Pending restart │ Tags                       │
├─────────────┼────────────────┼────────┼─────────┼────┼───────────┼─────────────────┼────────────────────────────┤
│ postgresql0 │ 127.0.0.1:5432 │ Leader │ running │  2 │           │                 │                            │
├─────────────┼────────────────┼────────┼─────────┼────┼───────────┼─────────────────┼────────────────────────────┤
│ postgresql1 │ 127.0.0.1:5433 │        │ running │  2 │         0 │                 │ clonefrom: true            │
│             │                │        │         │    │           │                 │ nofailover: true           │
│             │                │        │         │    │           │                 │ noloadbalance: true        │
│             │                │        │         │    │           │                 │ replicatefrom: postgresql0 │
├─────────────┼────────────────┼────────┼─────────┼────┼───────────┼─────────────────┼────────────────────────────┤
│ postgresql2 │ 127.0.0.1:5434 │        │ running │  2 │         0 │ *               │ replicatefrom: postgres1   │
└─────────────┴────────────────┴────────┴─────────┴────┴───────────┴─────────────────┴────────────────────────────┘

$ patronictl list badclustername
┌ Cluster: badclustername (uninitialized) ──────┐
│ Member │ Host │ Role │ State │ TL │ Lag in MB │
└────────┴──────┴──────┴───────┴────┴───────────┘

$ patronictl history
┌────┬──────────┬──────────────────────────────┬───────────────────────────┐
│ TL │      LSN │ Reason                       │ Timestamp                 │
├────┼──────────┼──────────────────────────────┼───────────────────────────┤
│  1 │ 25657792 │ no recovery target specified │ 2020-02-13T11:26:27+01:00 │
│  2 │ 25690088 │ no recovery target specified │ 2020-02-13T11:29:53+01:00 │
└────┴──────────┴──────────────────────────────┴───────────────────────────┘

$ patronictl query -c "SELECT a, repeat('x', a) from generate_series(1,3) a"
┌───┬────────┐
│ a │ repeat │
├───┼────────┤
│ 1 │ x      │
│ 2 │ xx     │
│ 3 │ xxx    │
└───┴────────┘
```
 2020-02-14 12:38:35 +01:00
Paul VossandGitHub 7e17092809 Fix permissions for openshift block devices (#1361) 
OpenShift enforces securityContext.fsGroups for block devices and sets group stickybits for volumeMounts.

This leads to patroni pods failing to start after the first restart:
> 2020-01-13 14:46:13.695 UTC [143] FATAL:  data directory "/home/postgres/pgdata/pgroot/data" has invalid permissions
2020-01-13 14:46:13.695 UTC [143] DETAIL:  Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).

A initContainer which fixes the OpenShift tampering solves the issue. I stole the solution from the stable postgres helm chart:
https://github.com/helm/charts/pull/14540/files

Tested on OpenShift v3.11

Note: This error does not occur when using shared filesystems (like NFS)
 2020-02-13 15:07:56 +01:00
Kostiantyn NemchenkoandGitHub bc948ce551 Show tags for member via patronictl (#1383) 
Add one more field to patronictl output to check tags defined for a member.
Closes #1375
 2020-02-13 15:07:05 +01:00
Michael BanckandGitHub f419d73465 Set postgresql.pgpass to ./pgpass (#1386) 
This avoids test failures if $HOME is not available (fixes: #1385).
 2020-02-13 15:06:36 +01:00
Alexander KukushkinandGitHub 4737af48bf Get rid of dependency on tzlocal module (#1390) 
It was used only to add the local timezone to the datetime specified in the patronictl for scheduled switchover or restart.
The `dateutil.tz.tzlocal()` does the same job equally well.
 2020-02-13 14:49:57 +01:00
Alexander KukushkinandGitHub dc1966e3bc Release 1.6.4 (#1380) 
* Bump version
* Update release notes
v1.6.4 		2020-01-27 14:15:21 +01:00
Alexander KukushkinandGitHub 6aa3f809d4 Configure keepalive for connections to K8s API (#1366) 
In case if we got nothing from the socket after the TTL seconds it should be considered dead.
 2020-01-27 09:25:08 +01:00
Alexander KukushkinandGitHub 902411239f More compatibility with windows (#1367) 
* unix-domain sockets are not yet supported
* signal.SIGQUIT doesn't exists
 2020-01-24 12:52:55 +01:00
Alexander KukushkinandGitHub 0eb1e0568b Compatibility with python 3.7.6 (#1368) 
The urlparse function has changed.

Old versions:
```python
>>> urlparse('localhost:8500')
ParseResult(scheme='', netloc='', path='localhost:8500', params='', query='', fragment='')
```

3.7.6:
```python
>>> urlparse('localhost:8500')
ParseResult(scheme='localhost', netloc='', path='8500', params='', query='', fragment='')
```
 2020-01-24 12:52:34 +01:00
Alexander KukushkinandGitHub 27ff9dfda3 Avoid logging passwords on user creation (#1370) 
Fixes https://github.com/zalando/patroni/issues/1365
 2020-01-24 10:52:23 +01:00
Igor YanchenkoandAlexander Kukushkin 16fe180ed6 implemented stop signal using pg_ctl for non posix systems (#1342) 
Using pg_ctl to send stop signal for non posix os.
 2020-01-16 14:35:47 +01:00
Alexander KukushkinandGitHub 1c4d395d5a Handle exception from Ha.shutdown (#1351) 
During the shutdown Patroni is trying to update its status in the DCS.
If the DCS is inaccessible an exception might be raised. Lack of exception handling prevents logger thread from stopping.

Fixes https://github.com/zalando/patroni/issues/1344
 2020-01-16 14:34:58 +01:00
Alexander KukushkinandGitHub c5fc6fd936 Special handling of parameters with period (#1349) 
It might be that they are defined by the extension and therefore the unit is not necessarily is the string.
It also could be that change of the value requires a restart (for example pg_stat_statements.max).
 2020-01-16 14:34:31 +01:00
Alexander KukushkinandGitHub 1461d7d4b8 Allow certain recovery parameters be defined in the custom_conf (#1335) 
Fixes https://github.com/zalando/patroni/issues/1333
 2020-01-15 12:41:07 +01:00
Igor YanchenkoandAlexander Kukushkin ea76a40845 Make sure postgresql.pgpass is a file or it does not exist (#1337) 
Also make sure that it is located in the writable directory.
 2020-01-15 12:40:41 +01:00
Alexander KukushkinandGitHub 102a12ea5a Catch all exceptions when calling socket.getaddrinfo (#1355) 
it serves two purposes:
1. We don't want accidentally break the thread
2. During the shutdown socket.gaierror become unresolvable and nasty exceptions are raised

Close: https://github.com/zalando/patroni/issues/1353
 2020-01-15 12:38:34 +01:00
Alexander KukushkinandGitHub 9e8ae95bce Finally fix a problem with case sensitivity of sync standby names (#1359) 
Close https://github.com/zalando/patroni/issues/1358
 2020-01-15 12:38:05 +01:00
Kostiantyn NemchenkoandAlexander Kukushkin a2a5cc2f71 Disable serfHealth Consul check (#1364) 
Fixes #1362 and #1363.
 2020-01-15 12:37:35 +01:00
Alexander KukushkinandGitHub 16d1ffdde7 Update timeline on standby cluster (#1332) 
Fixes https://github.com/zalando/patroni/issues/1031
 2019-12-20 12:56:00 +01:00
Alexander KukushkinandGitHub a675fa18dc Handle case when replication password is set to null (#1330) 
Fixes https://github.com/zalando/patroni/issues/1231
 2019-12-20 12:06:18 +01:00
Igor YanchenkoandAlexander Kukushkin 26b6e00575 wait option for patronictl reinit implemented (#1339) 
Wait to finish `reinit` if `--wait` option is used.
Every 2 seconds it pulls the status from Patroni REST API and reports to console.
 2019-12-20 12:05:39 +01:00
Alexander KukushkinandGitHub d941f6bc5e Use restore_command from standby_cluster config on cascading replicas (#1341) 
The standby_leader was already doing it from the beginning feature existed. Not doing the same on replicas might prevent them from catching up with standby leader due to WALs being recycled.

In addition to that apply the same strategy to archive_cleanup_command.
 2019-12-20 12:03:25 +01:00
Igor YanchenkoandAlexander Kukushkin 7ff27d9e10 Make sure unix_socket_directories and stats_temp_directory exist (#1293) 
Upon the start of Patroni and Postgres make sure that unix_socket_directories and stats_temp_directory exist or try to create them. Patroni will exit if failed to create them.

Close https://github.com/zalando/patroni/issues/863
 2019-12-11 12:26:17 +01:00
Igor YanchenkoandAlexander Kukushkin 2174d66f97 Rewriten shell scripts in python to make them compatible with windows (#1326) 2019-12-11 12:07:05 +01:00
Pavlo GolubandAlexander Kukushkin 919e9c54d2 Make dest argument default value of backup() cross platform (#1324) 
Fixes #1325
 2019-12-11 11:25:41 +01:00
Alexander KukushkinandGitHub 08d6e5e50e BUGFIX: don't leak password when running pg_rewind (#1321) 
In addition to that:
* enforce security settings from `postgresql.authention`
* update release notes
* bump version
* close https://github.com/zalando/patroni/issues/1320
v1.6.3 		2019-12-05 18:19:38 +01:00
Alexander KukushkinandGitHub b542e4b5f0 Release 1.6.2 (#1319) 
* update release notes
* bump version
v1.6.2 		2019-12-05 11:36:17 +01:00
Alexander KukushkinandGitHub 0693fe7dd0 Housekeeping (#1315) 
* Reduce memory usage by patroni init process
* More cleanup in setup.py
* Implement missing tests
 2019-12-04 11:28:46 +01:00
Igor YanchenkoandAlexander Kukushkin 49d3968c23 Make it possible to configure log level for exception tracebacks (#1311) 
If you set `log.traceback_level=DEBUG`, the tracebacks will be visible only when `log.level=DEBUG`. The default behavior remains the same.
 2019-12-03 15:13:42 +01:00
Alexander KukushkinandGitHub f1819443ef Avoid spawning semaphore tracker process (#1299) 
We are not using semaphores, therefore we don't need to track them.
 2019-12-02 12:16:18 +01:00
Igor YanchenkoandAlexander Kukushkin cf0c0f8e7c Make the error more helpful if restapi cannot bind (#1300) 
Giving the user a hint if we couldn't start the restapi service.
 2019-12-02 12:15:45 +01:00
Alexander KukushkinandGitHub e1d569ad75 Inherit CaseInsensitiveDict from urllib3 HTTPHeaderDict (#1302) 
It might look like a hack, but the API is stable enough and didn't change in the past 3+ years.
 2019-12-02 12:14:59 +01:00
Igor YanchenkoandAlexander Kukushkin 726ee46111 Implemented patroni --version (#1291) 
That required a refactoring of `Config` and `Patroni` classes. Now one has to explicitely create the instance of `Config` before creating `Patroni`.

The Config file can optionally call the validate function.
 2019-12-02 12:14:19 +01:00
Alexander KukushkinandGitHub cc0df4900b Set User-Agent for all http requests (#1312) 
Example: `Patroni/1.6.1 Python/3.6.8 Linux`
 2019-12-02 10:46:20 +01:00
Igor YanchenkoandAlexander Kukushkin 638aa63023 Don't make user to choose from an empty list (#1305) 
If a user provides a wrong cluster name, we will raise an exception rather than ask to choose a member from an empty list.
 2019-12-02 10:38:35 +01:00
Alexander KukushkinandGitHub a5ff38a034 Improve behave tests (#1313) 
Hopefully, make them less flaky
 2019-12-02 10:33:44 +01:00
Alexander KukushkinandGitHub a3be2958a7 Tidy up setup.py (#1308) 
1. Stop using `setuptools.command.test`, it is being deprecated
2. Remove junit integration
 2019-11-27 15:56:50 +01:00
Alexander KukushkinandGitHub 85341ff78b Use passfile in primary_conninfo only on 10+ (#1301) 
somehow passfile happened to work on ubuntu with older postgres versions, but it is not always the case for other distros.
 2019-11-27 14:58:23 +01:00
Alexander KukushkinandGitHub 7793887ea7 Fix tests on windows (#1303) 
and disable junit, it produces a deprecation warning
 2019-11-27 14:57:33 +01:00
Alexander KukushkinandGitHub 525a26fab5 Solve the problem of cyclic imports (#1306) 
Move `PATRONI_ENV_PREFIX` into the `patroni/__init__.py`
 2019-11-26 17:03:34 +01:00
Alexander KukushkinandGitHub 90a4208390 Get rid from requests module (#1296) 
It wasn't used for anything critical anyway, so it doesn't make a lot of sense to keep it as an explicit dependency.
 2019-11-22 15:31:55 +01:00
Alexander KukushkinandGitHub f03b85f5b0 Fix calculation of wal_buffes (#1297) 
Close https://github.com/zalando/patroni/issues/1288
 2019-11-22 15:30:28 +01:00
Alexander KukushkinandGitHub 474ac3cc11 Move multiprocessing.set_start_method() back to main (#1295) 
It is not possible to call it from forked process
 2019-11-21 17:28:14 +01:00
Alexander KukushkinandGitHub 412c720d3a Avoid importing all DCS modules (#1286) 
We will try to import only the module which has a configuration section.
I.e. if there is only zookeeper section in the config, Patroni will try to import only `patroni.dcs.zookeeper` and skip `etcd`, `consul`, and `kubernetes`.
This approach has two benefits:
1. When there are no dependencies installed Patroni was showing INFO messages `Failed to import smth`, which looks scary.
2. It reduces memory usage, because sometimes dependencies are heavy.
 2019-11-21 14:39:37 +01:00
Igor YanchenkoandAlexander Kukushkin cd96a10dd2 Provide an example of using multiple etcd endpoints in yaml files (#1289) 2019-11-21 13:29:05 +01:00
Alexander KukushkinandGitHub 183adb7848 Housekeeping (#1284) 
* Implement proper tests for `multiprocessing.set_start_method()`
* Exclude some watchdog code from coverage (it is used only for behave tests)
* properly use os.path.join for windows compatibility
* import DCS modules in `features/environment.py` on demand. It allows to run behave tests against chosen DCS without installing all dependencies.
* remove some unused behave code
* fix some minor issues in the dcs.kubernetes module
 2019-11-21 13:27:55 +01:00
Igor YanchenkoandAlexander Kukushkin 8b26733f6a striping extra spaces if etcd.hosts is written as comma separated string (#1290) 
Patroni was failing to connect to the etcd server if config looks like following:
```yaml
etcd:
    hosts: host1:port1, host2:port2
```
 2019-11-21 10:50:14 +01:00
Alexander KukushkinandGitHub 35a2ccf8a8 A couple of small fixes in docs (#1285) 
* fix formatting in release notes
* fix patronictl reinit command name
 2019-11-21 10:39:28 +01:00
Alexander KukushkinandGitHub 2f9a48fae4 Release 1.6.1 (#1281) 
* Bump version to 1.6.1
* Update release notes
v1.6.1 		2019-11-15 12:48:00 +01:00
Maciej KowalczykandAlexander Kukushkin efcd05ace2 Use "spawn" multiprocessing start method (#1279) 
workaround https://bugs.python.org/issue6721

Fixes #1278
 2019-11-15 10:56:18 +01:00