Make it easier to develop locally (#1748)

Previously the only documentation for how to run tests was the
implementation in the Travis configuration file. Here we add
instructions as well as move development dependencies to an easily used
and shared (with Travis config) separate requirements.dev.txt file.

.travis.yml

@@ -7,7 +7,7 @@ addons:
       - expect-dev # for unbuffer
 env:
   global:
-  - ETCDVERSION=3.3.13 ZKVERSION=3.4.14 CONSULVERSION=0.7.4
+  - ETCDVERSION=3.3.13 ZKVERSION=3.6.2 CONSULVERSION=0.7.4
   - PYVERSIONS="2.7 3.6"
   - BOTO_CONFIG=/doesnotexist
 matrix:
@@ -93,14 +93,15 @@ install:
         }
 
         function get_exhibitor() {
-            ZC=~/mycache/zookeeper-${ZKVERSION}
-            if [[ ! -d $ZC ]]; then
-                curl -L http://www.apache.org/dist/zookeeper/zookeeper-${ZKVERSION}/zookeeper-${ZKVERSION}.tar.gz | tar xz
+            ZK_PACKAGE=apache-zookeeper-${ZKVERSION}-bin
+            ZK=~/mycache/$ZK_PACKAGE
+            if [[ ! -d $ZK ]]; then
+                curl -L https://downloads.apache.org/zookeeper/zookeeper-${ZKVERSION}/${ZK_PACKAGE}.tar.gz | tar xz
                 [[ ${PIPESTATUS[0]} == 0 ]] || return 1
-                mv zookeeper-${ZKVERSION}/conf/zoo_sample.cfg zookeeper-${ZKVERSION}/conf/zoo.cfg
-                mv zookeeper-${ZKVERSION} $ZC
+                mv $ZK_PACKAGE/conf/zoo_sample.cfg $ZK_PACKAGE/conf/zoo.cfg
+                mv $ZK_PACKAGE $ZK
             fi
-            $ZC/bin/zkServer.sh start
+            $ZK/bin/zkServer.sh start
             # following lines are 'emulating' exhibitor REST API
             while true; do
                 echo -e 'HTTP/1.0 200 OK\nContent-Type: application/json\n\n{"servers":["127.0.0.1"],"port":2181}' \
@@ -128,7 +129,7 @@ install:
         fi
         source ~/virtualenv/python${pv}/bin/activate
         # explicitly install all needed python modules to cache them
-        for p in '-r requirements.txt' 'psycopg2-binary behave codacy-coverage coverage coveralls flake8 mock pytest-cov pytest setuptools'; do
+        for p in '-r requirements.txt' '-r requirements.dev.txt' 'codacy-coverage coveralls'; do
             pip install $p --upgrade
         done
     done

docs/CONTRIBUTING.rst

@@ -10,6 +10,39 @@ Chatting
 
 Just want to chat with other Patroni users?  Looking for interactive troubleshooting help? Join us on channel #patroni in the `PostgreSQL Slack <https://postgres-slack.herokuapp.com/>`__.
 
+Running tests
+-------------
+
+Requirements for running behave tests:
+
+1. PostgreSQL packages need to be installed.
+2. PostgreSQL binaries must be available in your `PATH`. You may need to add them to the path with something like `PATH=/usr/lib/postgresql/11/bin:$PATH python -m behave`.
+3. If you'd like to test with external DCSs (e.g., Etcd, Consul, and Zookeeper) you'll need the packages installed and respective services running and accepting unencrypted/unprotected connections on localhost and default port. In the case of Etcd or Consul, the behave test suite could start them up if binaries are available in the `PATH`. See the `get_<dcs>` functions in the scripting in [the Travis configuration](.travis.yml) for more details.
+
+ Install dependencies:
+
+.. code-block:: bash
+
+    # You may want to use Virtualenv or specify pip3.
+    pip install -r requirements.txt
+    pip install -r requirements.dev.txt
+
+After you have all dependencies installed, you can run the various test suites:
+
+.. code-block:: bash
+
+    # You may want to use Virtualenv or specify python3.
+
+    # Run flake8 to check syntax and formatting:
+    python setup.py flake8
+
+    # Run the pytest suite in tests/:
+    python setup.py test
+
+    # Run the behave (https://behave.readthedocs.io/en/latest/) test suite in features/;
+    # modify DCS as desired (raft has no dependencies so is the easiest to start with):
+    DCS=raft python -m behave
+
 Reporting issues
 ----------------
 

requirements.dev.txt

@@ -0,0 +1,8 @@
+psycopg2-binary
+behave
+coverage
+flake8
+mock
+pytest-cov
+pytest
+setuptools