diff --git a/docs/dynamic_configuration.rst b/docs/dynamic_configuration.rst new file mode 100644 index 00000000..ffcf299f --- /dev/null +++ b/docs/dynamic_configuration.rst @@ -0,0 +1,67 @@ +Patroni configuration reload +============================ + +Patroni configuration should be stored in the DCS. There will be 3 types of configuration: + +- bootstrap configuration set in Patroni + That should be applied during the initialization time and written to etcd. + +- startup configuration (also in patroni.yml). + They should be applied during the initialization time. Unlike other options, they are not written in etcd and + any attempts to change them dynamically are blocked. + +- dynamic configuration. + Those options can be set in etcd at any time. If the options changed are not part of the startup configuration, + they are applied asynchronously (upon the next wake up cycle) to every node, which gets subsequently reloaded. + If the node requires a restart to apply the configuration (for options with context postmaster or internal, if + their values have changed), a special flag indicating this should be set in the members.data JSON. A new API + endpoint should return whether the given node requires a restart. Additionally, the node status should also + indicate this. + +Some options may not be increased on the master independently of the replicas (master-dependent options): + +- max_connections +- max_locks_per_transactions +- max_worker_processes +- max_prepared_transactions + +Regarding the options that can be set, the following restrictions apply: + +- dynamic configuration options that are also listed in the startup configuration will not be changed, + except for the case of master-dependent options. + +When applying the startup or dynamic configuration options, the following actions should be taken: + +- The node should first check if there is a postgresql.base.conf. +- If it exists, it contains the renamed "original" configuration. +- If it doesn't, the original postgresql.conf is taken and renamed to postgresql.base.conf. +- The dynamic options (with the exceptions above) are dumped into the postgresql.conf and an include is set in + postgresql.conf to postgresql.base.conf. Therefore, we would be able to apply new options without re-reading the configuration file to check if the include is present not. +- Some parameters that are essential for Patroni to manage the cluster are overridden using the command line, these + include at least `port`, `listen_addresses`, `wal_level` +- If some of the options that require restart are changed (we should look at the context in pg_settings and at the actual + values of those options), a restart_pending flag of a given node should be set. This flag is reset on any restart. + +Parameters would be applied in the following order (run-time are given the highest priority): + +1. load parameters from file `postgresql.base.conf` +2. load parameters from file `postgresql.conf` +3. load parameters from file `postgresql.auto.conf` +4. run-time parameter using `-o --name=value` + +This allows configuration for all the nodes (2), configuration for a specific node using `ALTER SYSTEM` (3) and ensures that +parameters essential to the running of Patroni are enforced. (4) + + +Also, the following patroni configuration options can be changed dynamically: + +- ttl +- loop_wait +- retry_timeouts (to be defined first in patroni.yaml) + +Upon changing those options, Patroni should read the relevant section of the configuration stored in DCS and change their +run-time values. + +Patroni nodes should dump the state of the DCS options to disk on startup and upon every change of the configuration. +Only master is allowed to restore those options from the on-disk dump if those are completely absent from the DCS or invalid. +