* Refactor existing metrics support into MetricHelper
This completes the refactor done at https://github.com/badges/shields/pull/3662#issuecomment-509011229 in anticipation of adding more metrics support, such as response size of an upstream service, or response time.
* Clean up
* Renames
* Add response time metrics
This adds around 30 new metrics to cover response times at a fairly granular level. We may be able to shrink the number of buckets with time, though I think using 30 metrics is probably okay given that I think may become our most important metric.
* Fix
* add base class for Graphql APIs
* add GithubAuthV4Service + updates to GH token pool
* update github forks to use GithubAuthV4Service
* rename GithubAuthService to GithubAuthV3Service
Use of this feature [has been discouraged for a long time](https://nodejs.org/api/domain.html).
Since most of our code is now bubbling through async, we aren't really getting these "vendor errors" anymore.
Errors that _do_ bubble up through the services have been reported to Sentry since #3706, though they seem to be missing a bunch of their stack traces. Sentry also seems to be combining unrelated internal errors. (https://github.com/badges/shields/issues/3709#issuecomment-514299441) Maybe this will help.
Adding TypeScript to the frontend and a `.d.ts` file to `core` (see #3742) has multiplied out the different combinations of lint rules. ESLint has support for file-pattern-based overrides, which we've used in some places, but we've also maintained a separate eslintrc for `frontend/`.
This merges the config together, with the strategy of putting all the rules at the top level except where they conflict, and applying settings to exactly the files where they should apply.
This introduces a few new errors in the server but they are true positives – hoisting and lowercase class names – things we don't really need to be doing).
This is a reworking of #3410 based on some feedback @calebcartwright left on that PR.
The goals of injecting the secrets are threefold:
1. Simplify testing
2. Be consistent with all of the other config (which is injected)
3. Encapsulate the sensitive auth-related code in one place so it can be studied and tested thoroughly
- Rather than add more code to BaseService to handle authorization logic, it delegates that to an AuthHelper class.
- When the server starts, it fetches the credentials from `config` and injects them into `BaseService.register()` which passes them to `invoke()`.
- In `invoke()` the service's auth configuration is checked (`static get auth()`, much like `static get route()`).
- If the auth config is present, an AuthHelper instance is created and attached to the new instance.
- Then within the service, the password, basic auth config, or bearer authentication can be accessed via e.g. `this.authHelper.basicAuth` and passed to `this._requestJson()` and friends.
- Everything is being done very explicitly, so it should be very clear where and how the configured secrets are being used.
- Testing different configurations of services can now be done by injecting the config into `invoke()` in `.spec` files instead of mocking global state in the service tests as was done before. See the new Jira spec files for a good example of this.
Ref #3393
I find having these in a consistent order makes the services much faster to read.
This is the order I’ve generally been using:
1. Category
2. Route
3. Examples
4. Rendering
5. Other helpers (`fetch()`, `transform()`)
6. `handle()`
Attacking this in two pieces for ease of review. The legacy implementation for coverage is still there, though I disabled it via the route. That whole file will be removed in the next PR.
Ref #2863
* Modernised JSON format and removed _shields_test style
* Added logoWidth and labelColor fields to JSON response
* Reinstated and updated comment
* Extended expectBadge to accept Joi schemas for all fields
* Label all deprecated services as such
This will change
service_requests_total{category="other",family="cocoapods",service="cocoapods_apps"} 76
to
service_requests_total{category="other",family="cocoapods",service="deprecated_cocoapods_apps"} 76
* Fix tests
This picks up #2068 by adding per-badge stats as discussed in #966.
It ensures every service has a unique `name` property. By default this comes from the class name, and is overridden in all the various places where the class names are duplicated. (Some of those don't seem that useful, like the various download interval services, though those need to be refactored down into a single service anyway.) Tests enforce the names are unique. These are the names used by the service-test runner, so it's a good idea to make them unique anyway. (It was sort of strange before that you had to specify `nuget` instead of e.g. `resharper`.)
I've added validation to `deprecatedService` and `redirector`, and required that every `route` has a `base`, even if it's an empty string.
The name is used to generate unique metric labels, generating metrics like these:
```
service_requests_total{category="activity",family="eclipse-marketplace",service="eclipse_marketplace_update"} 2
service_requests_total{category="activity",family="npm",service="npm_collaborators"} 3
service_requests_total{category="activity",family="steam",service="steam_file_release_date"} 2
service_requests_total{category="analysis",family="ansible",service="ansible_galaxy_content_quality_score"} 2
service_requests_total{category="analysis",family="cii-best-practices",service="cii_best_practices_service"} 4
service_requests_total{category="analysis",family="cocoapods",service="cocoapods_docs"} 2
service_requests_total{category="analysis",family="codacy",service="codacy_grade"} 3
service_requests_total{category="analysis",family="coverity",service="coverity_scan"} 2
service_requests_total{category="analysis",family="coverity",service="deprecated_coverity_ondemand"} 2
service_requests_total{category="analysis",family="dependabot",service="dependabot_semver_compatibility"} 3
service_requests_total{category="analysis",family="lgtm",service="lgtm_alerts"} 2
service_requests_total{category="analysis",family="lgtm",service="lgtm_grade"} 3
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_git_hub"} 4
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_npm"} 5
service_requests_total{category="analysis",family="symfony",service="sensiolabs_i_redirector"} 1
service_requests_total{category="analysis",family="symfony",service="symfony_insight_grade"} 1
service_requests_total{category="build",family="appveyor",service="app_veyor_ci"} 3
service_requests_total{category="build",family="appveyor",service="app_veyor_tests"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_build"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_release"} 5
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_tests"} 6
service_requests_total{category="build",family="azure-devops",service="vso_build_redirector"} 2
service_requests_total{category="build",family="azure-devops",service="vso_release_redirector"} 1
service_requests_total{category="build",family="bitbucket",service="bitbucket_pipelines"} 5
service_requests_total{category="build",family="circleci",service="circle_ci"} 5
```
This is predicated on being able to use Prometheus's [`rate()`](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) function to visualize a counter's rate of change, as mentioned at https://github.com/badges/shields/issues/2068#issuecomment-466696561. Otherwise the stats will be disrupted every time a server restarts.
The metrics only appear on new-style services.
This will definitely save time, and ensure more uniformity.
It moves the `createServiceTester()` calls to a different place from where I'd like them, though I'm happy to have them checked by the linter.
Closes#2701
This is a mid-sized PR that adds query param validation to BaseService and updates most of the services which use query param validation to use it. There are a couple minor tweaks I made along the way.
Fix#2676
chris48sandrepo-ranger[bot]
Paul Melnikowandrepo-ranger[bot]
dependabot-preview[bot]andPaul Melnikow
dependabot-preview[bot]andCaleb Cartwright
Paul MelnikowandPierre-Yves B
mbarkhauandPaul Melnikow
