Use the endpoint on img.shields.io added in #1114 to fetch the PR title.
This sidesteps the authentication requirement, and helps with #979 – except for PRs that need to run the Github service tests, which will need a separate solution.
The intended behavior of the bracketed [github], [bower], [discord] service names in the pull request title is to trigger the designated service tests. That way, affected services can be proven working during code review, without needing to run tests on a dev machine, nor running all the slow (and flaky) service tests.
Example pull request titles:
- [Travis] Fix timeout issues
- [Travis Sonar] Support user token authentication
- [CRAN CPAN CTAN] Add test coverage
The observed behavior is that, whenever bracketed service names are provided, all of the service tests run.
This is due to a Mocha limitation, which is that exclusive tests (it.only and describe.only) can only be applied synchronously. In other words, if we try to fetch the PR title and then add exclusive tests in the callback, all the tests will run anyway. This is true even when using _mocha --delay, as we are, and is true whether I use request or node-fetch.
Undoubtedly this could be fixed, though it's not worth it. The problem is obscure and therefore low priority for Mocha, which is quite backlogged.
And, there is an easy workaround, which is to generate the list of services to test in a separate process.
The pull request script test:services:pr is now split into two parts. First the :prepare script infers the pull request context, fetches the PR title, and writes the list of affected services to a file. Then the :run script reads the list of affected services and runs the appropriate tests.
In addition to sidestepping the Mocha bug, this setup makes it easier to reason about and debug these two steps of the test runner on a dev machine, and since I can't get pipefail to work – and want to be able to run the steps separately – I'm not using Node's built in pre scripts.
Overall, separating these concerns this makes the test runner easier to reason about.
Paul MelnikowandGitHub