This implements the configuration mechanism I described in #2621. The heavy lifting is delegated to [node-config](https://github.com/lorenwest/node-config) with a minor assist from [dotenv](https://github.com/motdotla/dotenv).
`private/secret.json` has been replaced with environment variables and/or `config/local.yml`. See `doc/server-secrets.md`.
Because `server.js` was long a monolith, there are a bunch of shims in place to facilitate unit testing. A few of the test suites share port 1111 which means if one of them fails to set up, the port won't be freed and other unrelated tests will fail. Some of the tests which trigger server setup include timeouts which were added to give setup code time to run. In one the test suites, we actually modify `process.argv`, which seems completely gross.
This implements a few changes which improve this:
1. Separate the server from the server startup script, splitting out `lib/server.js`.
2. Inject config into the server and validate the config schema.
3. Inject config into the service test runner.
4. Use `portfinder`, a popular utility for grabbing open ports during testing.
5. Switch more of the setup code from callbacks to async-await.
Overall it leaves everything acting more reliably and looking rather cleaner, if in a few places more verbose.
It also fixes the root cause of #1455, a `setTimeout` in `rate-limit`. Off and on during development of this changeset, Mocha would decide not to exit, and that turned out to be the culprit.
Fix#1455
- Add tests to request-handler to prepare for some tweaks to caching for #820
- Clean up code in request-handler: renames, DRY, arrows, imports
- Allow for clean shutdown of `setInterval` code. This requires the ability to cancel autosaving.
- Upgrade to Mocha 4, and clean up so the process exits on its own (see mochajs/mocha#3044)
- Better encapsulate analytics
Reorg of the tests: move them just alongside their code. The principle relates to grouping by coupling, not by function and is established in best-practice documents (e.g. https://github.com/focusaurus/express_code_structure#underlying-principles-and-motivations), despite its break from the tradition of a separate `test/` tree. All of today's tools can handle tests spread through the repository.
There are some good, if subtle consequences of this change:
- Since files are close at hand, friction is reduced at development time, which encourages that new tests are written to cover new behaviors.
- It's easier to find the tests that cover a particular piece of functionality.
- It's easier to see which code has tests and which doesn't.
