End 2 end tests #134

maxbeatty · 2016-07-12T23:24:06Z

Now that the app is getting more use and bugs are popping up outside of what unit tests will easily catch, it's time to add some end to end tests (full browser automation).

This uses a new Docker Compose configuration of a Selenium Grid Hub and Selenium Grid Nodes (Firefox because that's built-in) along with the existing Docker Compose containers and lab test framework to run tests locally.

I also updated some of the documentation and steps to get up and running. In order for these to run on Travis, we'll need to modify the setup so it's automated. This PR is already big enough and is going to create merge conflicts for others so may save that for another more manageable branch.

maxbeatty · 2016-07-12T23:24:34Z

.dockerignore

@@ -3,11 +3,10 @@ node_modules
 .dockerignore
 .editorconfig
 .env
-.eslintignore
-.eslintrc


want these to be included in the e2e_runner container so lab can lint e2e test files

maxbeatty · 2016-07-12T23:25:02Z

.eslintignore

@@ -1,2 +1,3 @@
 node_modules
 public/_js
+tmp*


the custom firefox profile creates a temporary directory with JS that lab tries to lint so ignoring that

maxbeatty · 2016-07-12T23:26:29Z

Dockerfile

 WORKDIR /code
+
+COPY ./package.json /code/package.json


npm install wasn't being cached for me when using ADD. Docker now encourages COPY instead of ADD

maxbeatty · 2016-07-12T23:29:05Z

Dockerfile-e2e-runner

@@ -0,0 +1,13 @@
+FROM mhart/alpine-node:5.11.1


this alpine image is 95 MB compared to the main node image which is 749 MB. should consider using in another branch where change is more isolated

maxbeatty · 2016-07-12T23:29:48Z

docker-compose.yml

@@ -12,12 +12,12 @@ web:
    SERVICE_3000_CHECK_HTTP: '/health'
    SERVICE_3000_CHECK_INTERVAL: '1s'
 db:
-  image: mysql
+  image: mysql:5.7


locking down version

maxbeatty · 2016-07-12T23:30:40Z

docker-compose.yml

  volumes_from:
    - data-jsperf-mysql
  env_file: .env
  environment:
-    MYSQL_RANDOM_ROOT_PASSWORD: 'True'
+    MYSQL_RANDOM_ROOT_PASSWORD: 'yes'


image docs say to use yes. couldn't find reference to True

maxbeatty · 2016-07-12T23:31:12Z

package.json

@@ -6,10 +6,9 @@
  "scripts": {
    "clean": "rm npm-debug.log lcov.info coverage.html",
    "lint": "lab -d -L",
-    "setup": "node setup",
-    "shrinkwrap": "npm-shrinkwrap",


npm@3 solved a lot of the problems npm-shrinkwrap was solving

maxbeatty · 2016-07-12T23:31:59Z

package.json

@@ -38,7 +37,7 @@
    "good": "^5.1.2",
    "good-console": "^4.1.0",
    "handlebars": "^3.0.0",
-    "hapi": "^8.2.0",
+    "hapi": "8.6.1",


at one point, hapi@8.8.x was installed and shot (used for request injection during testing) broke. locking this down to prevent bad installs

maxbeatty · 2016-07-12T23:33:57Z

package.json

+    "eslint-config-standard": "5.3.5",
+    "eslint-plugin-promise": "1.3.2",
+    "eslint-plugin-standard": "1.3.3",
+    "lab": "10.9.0",


originally upgraded this to see if my problem with building a web driver in beforeEach would be fixed (it wasn't) but when other lab changes weren't needed, I kept it. upgrades eslint configs should give us more ES2015 goodness

maxbeatty · 2016-07-12T23:35:12Z

package.json

    "prompt": "^0.2.14",
    "proxyquire": "^1.4.0",
+    "selenium-webdriver": "2.47.0",


chose this version based on Sauce Labs docs

maxbeatty · 2016-07-12T23:35:31Z

server/lib/regex.js

@@ -85,5 +85,5 @@ module.exports = {
    // resource path
    '(?:/\\S*)?' + '$',
  slug: '[A-Za-z0-9](?:-?[A-Za-z0-9])*',
-  script: '(<script[^>]*?>)([\\s\\S]*?)(<\/script>)'
+  script: '(<script[^>]*?>)([\\s\\S]*?)(</script>)'


updated eslint config caught unneeded escape

maxbeatty · 2016-07-12T23:36:34Z

server/web/errors/index.js

@@ -15,7 +15,7 @@ exports.register = function (server, options, next) {
      return reply.view('errors/' + statusCode).code(statusCode);
    } else {
      debug(request.response);
-      return reply.view('errors/general');
+      return reply.view('errors/general').code(500);


our general error wasn't actually reporting as being unhealthy so even if mysql wasn't reachable we were reporting to consul that it was still healthy causing the node to stay in nginx's rotation. now, it'll report a 500 so consul will take it out of rotation and nginx will send 502 if none available.

maxbeatty · 2016-07-12T23:37:25Z

test/e2e/faq.js

+const lab = exports.lab = Lab.script();
+
+lab.experiment('FAQ page', () => {
+  // FIXME: cannot figure out why this doesn't work


would love to know why this doesn't work. always times out for me

maxbeatty · 2016-07-12T23:38:46Z

test/unit/config.js

@@ -1,7 +1,7 @@
 var Lab = require('lab');
 var Code = require('code');
 var Proxyquire = require('proxyquire');
-var Config = Proxyquire('../config', {
+var Config = Proxyquire('../../config', {


here begins all of the path changes caused by restructuring the tests to be under test/unit. this allows lab to not grep e2e tests when only caring about unit tests

maxbeatty · 2016-07-12T23:40:05Z

test/unit/server/web/edit/index.js

@@ -228,7 +228,7 @@ lab.experiment('GET', function () {
    });
    server.inject('/setsession', function (res) {
      var header = res.headers['set-cookie'];
-      var cookie = header[0].match(/(?:[^\x00-\x20\(\)<>@\,;\:\\'\/\[\]\?\=\{\}\x7F]+)\s*=\s*(?:([^\x00-\x20\'\,\;\\\x7F]*))/);
+      var cookie = header[0].match(/(?:[^\x00-\x20\(\)<>@\,;\:\\'\/\[\]\?\=\{\}\x7F]+)\s*=\s*(?:([^\x00-\x20\'\,\;\\\x7F]*))/); // eslint-disable-line no-control-regex, no-useless-escape


didn't have enough regex foo to make eslint happy so ignored individual lines. might be nice to move this in a more managable branch to a reusable module so it's not repeated so often

maxbeatty added 3 commits Jul 12, 2016

move unit test to unit direcotry

6dcd86f

fix failing health check responding 200 instead of 500

b1f6b81

add end to end test structure

Loading status checks…

56d7f1d