The GitHub service family is the largest, and as yet untouched by our service rewrite. I thought I would start the process by tackling one service.
This pull request has a few things going on:
1. Rename pull-request-status to pull-request-check-state. We have another badge called pull request status. It seems like the checks are called one thing in the UI and another thing in the API, which is unfortunate. If other folks have strong feelings about the name, I’ll defer.
2. Move its tests and tighten up the syntax.
3. Move its badge examples including the doc string.
4. Add a new helper `errorMessagesFor` to use in the new services in place of `githubCheckErrorResponse`. It seems like we didn’t really use the `errorMessages` parameter to `githubCheckErrorResponse`, so I pared this down. I’m not sure if this is the function we’ll ultimately want, but it seems like a good place to start.
5. Pull fetch code I _know_ we use in other places into `github-common-fetch`. As in the PR I just opened for azure-devops, this takes a functional approach to the shared code, which is more direct, nimble, and easy to reason about than inheritance.
6. Create `GithubAuthService` which functions identically to BaseJsonService, except for one thing, which is that it uses the token pool. I accomplished this by adding a `_requestFetcher` property to BaseService, which is initialized to `sendAndCacheRequest` in the constructor, and can be overridden in subclasses. Since we weren’t using `_sendAndCacheRequest` directly except in BaseService and tests, I removed that property. I like this approach to patching in the GitHub auth because it’s very simple and creates no new API exposure. However, the way we’re doing the dependency injection feels a bit odd. Maybe the eventual refactor of request-handler would be a godo time to revisit this.
The GitHub requests go through many, many layers of indirection at this point. Later on it would be good to shave some of these off, perhaps once the legacy GitHub services have been converted, or when all the services are done and we can take another look at the base service hierarchy. The work in #2021 and #1205 is also related.
The term “url” is overloaded in services, to refer to the Shields route and also the API URL. Calling the Shields URL a “route” is on the whole more descriptive, and makes it clearer and more obvious which one of these we’re talking about. It’s a small thing, though seems like an improvement.
We have a few functions called `buildUrl`. I’ve renamed them to `buildRoute` when they refer to routes, and left them as `buildUrl` when they refer to API URLs.
I included a minor style tweak and some formatting cleanup in `TUTORIAL.md`.
I went down a rabbit hole while trying to untangle the bug in the dockbit and bitrise examples https://github.com/badges/shields/pull/2234#pullrequestreview-169997546.
The URL generation code is spaghetti-like, with functions, many of which I wrote, with opaque names, doing similar but not identical things, and making slightly incompatible assumptions about the way query strings are handled.
I got a bit lost and need to take a step back.
Meanwhile, this is a small piece of work I did that’s worth keeping. It doesn’t scratch the surface of the tangle, but it does remove a bit of duplication.
It also makes a minor stylistic ES6 change in the handling of default arguments.
Ref: #2027
This is consistent with what we're pretty much already doing, and saves us from making the request during code review.
These were all autofixed and most of them seem easier to read. Some in the legacy services should be rewritten in more legible forms during refactor (ie using intermediate variables, or using request’s qs option). There are some in helper functions and elsewhere that should get rewritten separately. I don't want to change them in this PR because the changes will get lost in this diff, though we could identify them here and fix them before or just after.
all-badge-examples is a common cause of merge conflicts. It’s difficult to adjust the badge categorization in that file – or to understand the diff – because it requires moving a block from one point to another. It’s much easier to edit a badge’s category in one place.
This starts the process of breaking up what’s left of that file, following up on the work from #1931. New-style services can only be in one category, which means legacy service examples have to be split along category lines. I split out separate legacy service classes where I could do so easily, leaving behind the ones which require more work, for one reason or another.
I've rewritten the deprecated services using the `deprecatedService` helper from #1922. I added a test for `Deprecated`, and for `enforceDeprecation`, which isn't being used right now, but is there for future use.
This also makes it possible to write services using BaseService which do not have any named parameters (with a test).
Ref: #1358
This continues a consistency update we’ve been making to standardize on URL based on a recommendation from WHATWG: https://url.spec.whatwg.org/#goals
This also helps with copying and pasting between all-badge-examples and new-style services, where it’s otherwise easy to make a mistake.
Ref: #1322#1341
* allow service classes to define a static example
* define static example for some services
(apm, appveyor, cdnjs, clojars, gem, librariesio, npm, uptimerobot)
* add/update tests
This allows us to show an example without making an API call to a live service for better performance.
We can now specify 3 fields in the example definition:
* urlPattern for the version with placeholders e.g: /npm/dw/:package.svg
* ExampleUrl/Uri for the concrete example e.g: /npm/dw/localeval.svg
* PreviewUrl/Uri for the static (or live) image we will actually show
- Log the response when using `test:services:trace`
- Fully log large nested objects
Since the `badgeData` is in a different format from the JSON response, and also doesn't include the title, including this output is helpful. It makes it clearer what the Joi matchers are trying to match.
Sometimes when there's a deep nested structure, it's helpful or necessary to see the entire thing.
* InvalidParameter: New error type
* Return inaccessible for 5xx errors from services
* Add test for Inaccessible on 5xx
* Add tests for named error types
I ran into this while working on #1867, where ultimately I couldn't solve my problem by injecting config. This will make it easier in the future, though.
When JSON responses come back, they are sometimes not in the format expected by the API. As a result we have a lot of defensive coding (expressions like `(data || {}).someProperty`) to avoid exceptions being thrown in badge processing. Often we rely on the `try` blocks that wrap so much of the badge-processing code, which catch all JavaScript exceptions and return some error result, usually **invalid**. The problem with this is that these `try` blocks catch all sorts of programmer errors too, so when we see **invalid** we don't know whether the API returned something unexpected, or we've made a mistake. We also spend a lot of time writing defensive tests around malformed responses, and creating and maintaining the defensive coding.
A better solution is to validate the API responses using declarative contracts. Here the programmer says exactly what they expect from the API. That way, if the response isn't what we expect we can just say it's an **invalid json response**. And if our code then throws an exception, well that's our mistake; when we catch that we can call it a **shields internal error**. It's also less code and less error-prone. Over time we may be confident enough in the contracts that we won't need so many tests of malformed responses. The contract doesn't need to describe the entire response, only the part that's needed. Unknown keys can simply be dropped, preventing unvalidated parts of the response from creeping into the code. Checking what's in our response before calling values on it also makes our code more secure.
I used Joi here, since we're already using it for testing. There may be another contracts library that's a better fit, though I think we could look at that later.
Those changes are in base.js.
The rest is a rewrite of the remaining NPM badges, including the extraction of an NpmBase class. Inspired by @chris48s's work in #1740, this class splits the service concerns into fetching, validation, transformation, and rendering. This is treated as a design pattern. See the PR discussion for more. There are two URL patterns, one which allows specifying a tag (used by e.g. the version badge `https://img.shields.io/npm/v/npm/next.svg`), and the other which does not accept a tag (e.g. the license badge `https://img.shields.io/npm/l/express.svg`). Subclasses like NpmLicense and NpmTypeDefinitions can specify the URL fragment, examples, the validation schema for the chunk of the package data they use, and a render function. The NpmVersion subclass uses a different endpoint, so it overrides the `handle` implementation from NpmBase.
The remaining services using BaseJsonService are shimmed, so they will keep working after the changes.
* allow services to export >1 classes
This change to loadServiceClasses() allows us to define
services which either export a single service class e.g:
module.exports = class Cdnjs extends BaseService {
//...
}
or more than one. e.g:
module.exports = {
GemVersion,
GemDownloads,
GemOwner,
GemRank,
}
* refactor ruby gem badges
- move badge code to service classes
- throw exceptions for errors
- use let and const
- change tests to expect 'downloads' label for error badges
- general tidying
* fix typo in tests
* Don't always use class name in example label
This allows (for example) GemVersion and GemDownloads
both to use the example label 'Gem'
Make a clear distinction between programmer errors ("internal errors") and runtime errors, and allow configuring the server to let the programmer errors bubble up in development and unit testing. This saves a huge amount of time because it generates ordinary stack traces when things go wrong. And, if these errors occur in production, we'll catch them, and display **shields | internal error** which is the equivalent of a 500 error.
Instead of centralizing examples, specify them from within a service.
* Avoid duplication in service loading + refactor
* Avoid duplication in URLs, rename uri -> url in BaseService
This merges the `node-8` branch. The heavy lift was by @Daniel15 with refactoring from me and a patch by @RedSparr0w.
* New API for registering services (#963)
* Disable Node 6 tests on node-8 branch (#1423)
* BaseService: Factor out methods _regex and _namedParamsForMatch (#1425)
- Adjust test grouping
- Rename data -> queryParams, text -> message
* BaseService tests: Use Chai (#1450)
* BaseService: make serviceData and badgeData explicit and declarative (#1451)
* fix isValidStyle test (#1544)
* Run tests in Node 9, not Node 6 (#1543)
