* Make it easier to benchmark and profile the code
* Remove unnecessary escape
* Clarify that the backend server is started without the frontend
* Add missing NODE_CONFIG_ENV environment variable
* Add error message when user has not included console.time statements
* Fix lint issue
* Handle multiple console.time statements
* Switch NODE_CONFIG_ENV to test
* Switch to const as variable never re-assigned
* Response size metric for all services
* Unused code removed
* Test for service response size metric
* All buckes of the service_response_bytes in a comment
* Register parameter in PrometheusMetrics is optional
* service response size metric enabled for dynamic badges
* Better test name
* JSDoc removed
* One import from one file
* Gather metrics in the background
* Revert saving response time metrics in the background
* Changed configuration validation to allow using a named pipe for port.
* Update core/server/server.js
Co-Authored-By: Caleb Cartwright <calebcartwright@users.noreply.github.com>
* Applied prettier.
* Update core/server/server.spec.js
Co-Authored-By: Caleb Cartwright <calebcartwright@users.noreply.github.com>
* Updated spec for port setting to allow only numbers or formatted named pipes.
* Try enable universal cors
* Move handle from registerErrorHandlers because it isn't and error handler
* Add test for cors headers
Also add link to issue
* add /twitch/status/:user badge
* update comments
* use a proper schema for the Twitch API calls
* use a token to make Twitch api calls
* fix handling of rate-limit error and bad token error
* [twitch] get a token as soon as creating a Twitch service
* [twitch] start both requests to users and stream before awaiting
* [twitch] set a timeout to replace the token before it expires
* [twitch] use authHelper
* [twitch] skip tests when no credentials
* [twitch] add one more status test
* twitch: do not check whether a user exists
* Refactor existing metrics support into MetricHelper
This completes the refactor done at https://github.com/badges/shields/pull/3662#issuecomment-509011229 in anticipation of adding more metrics support, such as response size of an upstream service, or response time.
* Clean up
* Renames
* Add response time metrics
This adds around 30 new metrics to cover response times at a fairly granular level. We may be able to shrink the number of buckets with time, though I think using 30 metrics is probably okay given that I think may become our most important metric.
* Fix
Adding TypeScript to the frontend and a `.d.ts` file to `core` (see #3742) has multiplied out the different combinations of lint rules. ESLint has support for file-pattern-based overrides, which we've used in some places, but we've also maintained a separate eslintrc for `frontend/`.
This merges the config together, with the strategy of putting all the rules at the top level except where they conflict, and applying settings to exactly the files where they should apply.
This introduces a few new errors in the server but they are true positives – hoisting and lowercase class names – things we don't really need to be doing).
This is a reworking of #3410 based on some feedback @calebcartwright left on that PR.
The goals of injecting the secrets are threefold:
1. Simplify testing
2. Be consistent with all of the other config (which is injected)
3. Encapsulate the sensitive auth-related code in one place so it can be studied and tested thoroughly
- Rather than add more code to BaseService to handle authorization logic, it delegates that to an AuthHelper class.
- When the server starts, it fetches the credentials from `config` and injects them into `BaseService.register()` which passes them to `invoke()`.
- In `invoke()` the service's auth configuration is checked (`static get auth()`, much like `static get route()`).
- If the auth config is present, an AuthHelper instance is created and attached to the new instance.
- Then within the service, the password, basic auth config, or bearer authentication can be accessed via e.g. `this.authHelper.basicAuth` and passed to `this._requestJson()` and friends.
- Everything is being done very explicitly, so it should be very clear where and how the configured secrets are being used.
- Testing different configurations of services can now be done by injecting the config into `invoke()` in `.spec` files instead of mocking global state in the service tests as was done before. See the new Jira spec files for a good example of this.
Ref #3393
Fixes https://github.com/badges/shields/issues/3260
Problem happens when a value of a color in an old PNG static badge is a number: http://localhost:8080/my-label/my-message.png?color=1. In this case `color` in `queryParams` is a number.
0a0b5b3f03/core/server/server.js (L203-L212)
Surprisingly service test listed below is passing currently on master - value `1` is represented in `queryParams` as a String (only in test).
`services/static-badge/static-badge.tester.js`
```js
t.create('Old static badge with a number as a color')
.get('/foo/bar.png?color=1', { followRedirect: false })
.expectStatus(301)
.expectHeader('Location', '/badge/foo-bar-1.png')
```
Moreover I added some code + description allowing to debug server.
This picks up #2068 by adding per-badge stats as discussed in #966.
It ensures every service has a unique `name` property. By default this comes from the class name, and is overridden in all the various places where the class names are duplicated. (Some of those don't seem that useful, like the various download interval services, though those need to be refactored down into a single service anyway.) Tests enforce the names are unique. These are the names used by the service-test runner, so it's a good idea to make them unique anyway. (It was sort of strange before that you had to specify `nuget` instead of e.g. `resharper`.)
I've added validation to `deprecatedService` and `redirector`, and required that every `route` has a `base`, even if it's an empty string.
The name is used to generate unique metric labels, generating metrics like these:
```
service_requests_total{category="activity",family="eclipse-marketplace",service="eclipse_marketplace_update"} 2
service_requests_total{category="activity",family="npm",service="npm_collaborators"} 3
service_requests_total{category="activity",family="steam",service="steam_file_release_date"} 2
service_requests_total{category="analysis",family="ansible",service="ansible_galaxy_content_quality_score"} 2
service_requests_total{category="analysis",family="cii-best-practices",service="cii_best_practices_service"} 4
service_requests_total{category="analysis",family="cocoapods",service="cocoapods_docs"} 2
service_requests_total{category="analysis",family="codacy",service="codacy_grade"} 3
service_requests_total{category="analysis",family="coverity",service="coverity_scan"} 2
service_requests_total{category="analysis",family="coverity",service="deprecated_coverity_ondemand"} 2
service_requests_total{category="analysis",family="dependabot",service="dependabot_semver_compatibility"} 3
service_requests_total{category="analysis",family="lgtm",service="lgtm_alerts"} 2
service_requests_total{category="analysis",family="lgtm",service="lgtm_grade"} 3
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_git_hub"} 4
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_npm"} 5
service_requests_total{category="analysis",family="symfony",service="sensiolabs_i_redirector"} 1
service_requests_total{category="analysis",family="symfony",service="symfony_insight_grade"} 1
service_requests_total{category="build",family="appveyor",service="app_veyor_ci"} 3
service_requests_total{category="build",family="appveyor",service="app_veyor_tests"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_build"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_release"} 5
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_tests"} 6
service_requests_total{category="build",family="azure-devops",service="vso_build_redirector"} 2
service_requests_total{category="build",family="azure-devops",service="vso_release_redirector"} 1
service_requests_total{category="build",family="bitbucket",service="bitbucket_pipelines"} 5
service_requests_total{category="build",family="circleci",service="circle_ci"} 5
```
This is predicated on being able to use Prometheus's [`rate()`](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) function to visualize a counter's rate of change, as mentioned at https://github.com/badges/shields/issues/2068#issuecomment-466696561. Otherwise the stats will be disrupted every time a server restarts.
The metrics only appear on new-style services.
This will definitely save time, and ensure more uniformity.
It moves the `createServiceTester()` calls to a different place from where I'd like them, though I'm happy to have them checked by the linter.
Closes#2701
While Next.js can handle static sites, we've had a few issues with it, notably a performance hit at runtime and some bugginess around routing and SSR. Gatsby being fully intended for high-performance static sites makes it a great technical fit for the Shields frontend. The `createPages()` API should be a really nice way to add a page for each service family, for example.
This migrates the frontend from Next.js to Gatsby. Gatsby is a powerful tool, which has a bit of downside as there's a lot to dig through. Overall I found configuration easier than Next.js. There are a lot of plugins and for the most part they worked out of the box. The documentation is good.
Links are cleaner now: there is no #. This will break old links though perhaps we could add some redirection to help with that. The only one I’m really concerned about `/#/endpoint`. I’m not sure if folks are deep-linking to the category pages.
There are a lot of enhancements we could add, in order to speed up the site even more. In particular we could think about inlining the SVGs rather than making separate requests for each one.
While Gatsby recommends GraphQL, it's not required. To keep things simple and reduce the learning curve, I did not use it here.
Close#1943Fix#2837Fix#2616
The suggest code was an exception to our usual organization pattern. There was a service test, but it's not a service. The code would sometimes regress because it wasn't being tested all the time.
This makes them no longer run as service tests, which is good because they run as part of every build. Some of them are smaller-bracket tests which is good too, because it will make them easier to test, especially as this code grows.
I'd have liked to keep using frisby for the ones that make requests to the server, though I ran into some issues with sequencing of setup that I think will require upstream changes.
In #2698 we decided to put legacy helper functions in `core/legacy`. I think that’s a fine idea, though if we’re going to have a bunch of badge helper functions in there, it seems like it is probably better to keep these two important but esoteric helper functions with the core code to which they are most coupled. So I added `legacy-` to the name, and put them in `core/base-service`.
This test is being weirdly flaky in #2809. The problem seems to be in the test helper code, so I rewrote this using Joi.
I imagine the change has to do with a change to the test ordering. It's a bit puzzling.
However, the new test seems fine (and the endpoint is rarely used; not critical to begin with).
