Commit Graph

93 Commits

Author SHA1 Message Date
Paul Melnikow
cfbd2c30df SVG by default (#3717)
Make cleaner badge URLs by omitting the `.svg` extension.

Closes #2674
2019-07-24 12:57:39 -05:00
Paul Melnikow
3964eb5cb0 Remove use of deprecated Domain API (#3762)
Use of this feature [has been discouraged for a long time](https://nodejs.org/api/domain.html).

Since most of our code is now bubbling through async, we aren't really getting these "vendor errors" anymore.

Errors that _do_ bubble up through the services have been reported to Sentry since #3706, though they seem to be missing a bunch of their stack traces. Sentry also seems to be combining unrelated internal errors. (https://github.com/badges/shields/issues/3709#issuecomment-514299441) Maybe this will help.
2019-07-23 16:43:09 -05:00
Paul Melnikow
28b8836595 Unify the ESLint config (#3743)
Adding TypeScript to the frontend and a `.d.ts` file to `core` (see #3742) has multiplied out the different combinations of lint rules. ESLint has support for file-pattern-based overrides, which we've used in some places, but we've also maintained a separate eslintrc for `frontend/`.

This merges the config together, with the strategy of putting all the rules at the top level except where they conflict, and applying settings to exactly the files where they should apply.

This introduces a few new errors in the server but they are true positives – hoisting and lowercase class names – things we don't really need to be doing).
2019-07-22 15:06:38 -05:00
Paul Melnikow
ec0264dfb7 Fix Redis prod issue for Node 9 (#3707)
Encountered while testing #3332
2019-07-12 19:51:05 -04:00
Paul Melnikow
153732756c Log internal errors to Sentry (#3706)
Fix #3688
2019-07-12 19:40:46 -04:00
Paul Melnikow
65afff40d4 Fix up Redis config (#3705)
- Move to private
- Fix validation
- Don't log the URL

Ref #3332
2019-07-12 19:12:49 -04:00
Paul Melnikow
84063f8ae3 Switch to ioredis, and update RedisTokenPersistence to use a set instead of a list (#3668)
Also add a script for importing a batch of tokens.

Ref #3332
2019-07-12 18:30:24 -04:00
Paul Melnikow
5d20a371d1 Update serverSecrets reference in legacy monitor (#3697)
Ref #3652 #3393
2019-07-11 19:41:22 -04:00
Paul Melnikow
e2608a6570 Refactor GitHub OAuth credential handling (#3693)
Continues the work of #3652.
2019-07-11 18:04:53 -04:00
Paul Melnikow
b2e5aa9208 Cache raster redirects longer (#3690)
Haven't seen error reports on these, so let's cache these a bit longer to reduce our server load.
2019-07-11 16:57:21 -04:00
chris48s
38cdc0033f adopt JSDoc, eslint-plugin-jsdoc (#3645)
eslint-plugin-jsdoc:
- install eslint-plugin-jsdoc
- config file
- fix lint/style errors

JSDoc:
- add JSDoc as a dev dependency
- get everything rendering nicely with JSDoc
- config, build command + ignores
2019-07-11 20:14:47 +01:00
Paul Melnikow
ce0ddf93fc Inject secrets into the services (#3652)
This is a reworking of #3410 based on some feedback @calebcartwright left on that PR.

The goals of injecting the secrets are threefold:

1. Simplify testing
2. Be consistent with all of the other config (which is injected)
3. Encapsulate the sensitive auth-related code in one place so it can be studied and tested thoroughly

- Rather than add more code to BaseService to handle authorization logic, it delegates that to an AuthHelper class.
- When the server starts, it fetches the credentials from `config` and injects them into `BaseService.register()` which passes them to `invoke()`.
- In `invoke()` the service's auth configuration is checked (`static get auth()`, much like `static get route()`).
- If the auth config is present, an AuthHelper instance is created and attached to the new instance.
- Then within the service, the password, basic auth config, or bearer authentication can be accessed via e.g. `this.authHelper.basicAuth` and passed to `this._requestJson()` and friends.
- Everything is being done very explicitly, so it should be very clear where and how the configured secrets are being used.
- Testing different configurations of services can now be done by injecting the config into `invoke()` in `.spec` files instead of mocking global state in the service tests as was done before. See the new Jira spec files for a good example of this.

Ref #3393
2019-07-09 23:14:36 -04:00
Paul Melnikow
aa135d370f Auto-annotate live and mock tests (#3661)
Close #2555
2019-07-07 17:06:01 -04:00
Paul Melnikow
eec904c262 Fix raster redirect and drop the legacy static .png route (#3659) 2019-07-07 11:49:21 -04:00
Paul Melnikow
66c7f13e38 Drop gif + png, and redirect png to raster.shields.io (#3644)
1. Remove rasterization support from the server. This responsibility is delegated to a raster server which proxies the SVG badges and renders them.
2. When a raster server URL is configured, 301 redirect all .png badges to the identical URL on the raster server.
    `https://img.shields.io/npm/v/express.png?style=flat-square` ↪️`https://raster.shields.io/npm/v/express.png?style=flat-square`
3. For configured redirects, redirect to the canonical URL on the raster server.
    `https://img.shields.io/vso/build/totodem/8cf3ec0e-d0c2-4fcd-8206-ad204f254a96/2.png?style=flat-square`
    ↪️`https://img.shields.io/azure-devops/build/totodem/8cf3ec0e-d0c2-4fcd-8206-ad204f254a96/2.png?style=flat-square`
4. Redirect the "legacy badge old version" to the appropriate URL on the raster server.
5. When no raster server is configured (e.g. PRs), render an SVG containing **404 | raster badges not available** for all `.png` badges. (Note that the raster server can be self-hosted; however, this is deferred to a later PR.)
5. Drop support for jpg and gif which are very infrequently used (see #3112). Render an SVG containing **410 | jpg no longer available**.
7. ~~Remove raster dependencies.~~ Remove the raster cache (which is only used in the CLI, and therefore pointless).
8. Move the LRUCache code out of the npm package.
8. A wee bit of refactoring in `server.js`.

Ref #3112
Close #3631
2019-07-06 16:41:46 -04:00
Paul Melnikow
093d9cd368 Minor refactor: expose custom promisify function from legacy request handler (#3650)
Cherry-picked from #3410; should simplify reworking it.
2019-07-05 21:26:05 -04:00
chris48s
0d3266a0ad JSDoc comments for service test runner (#3630) 2019-07-02 12:41:28 -04:00
chris48s
f29da0abdd JSDoc comments for token pool (#3632) 2019-07-01 17:55:08 -04:00
chris48s
d78edaf5b5 add/improve some JSDoc comments on core API (#3595) 2019-06-25 22:42:01 +01:00
dependabot-preview[bot]
95b6e48bff Build(deps-dev): bump nock from 11.0.0-beta.14 to 11.0.0-beta.19 (#3552)
* Build(deps-dev): bump nock from 11.0.0-beta.14 to 11.0.0-beta.19

Bumps [nock](https://github.com/nock/nock) from 11.0.0-beta.14 to 11.0.0-beta.19.
- [Release notes](https://github.com/nock/nock/releases)
- [Changelog](https://github.com/nock/nock/blob/beta/CHANGELOG.md)
- [Commits](https://github.com/nock/nock/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix one test

* Update for breaking change in nock api

* Always use got without retries in tests
2019-06-21 10:45:34 -04:00
dependabot-preview[bot]
d732b937c9 Build(deps-dev): bump prettier from 1.17.1 to 1.18.2 (#3580)
* Build(deps-dev): bump prettier from 1.17.1 to 1.18.2

Bumps [prettier](https://github.com/prettier/prettier) from 1.17.1 to 1.18.2.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/1.17.1...1.18.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* chore: prettier
2019-06-17 19:15:44 -05:00
chris48s
8621fe42d7 Upgrade Joi (#3505)
* upgrade joi
* find & replace ALL THE THINGS
* update related deps
2019-06-02 21:59:55 +01:00
Marcin Mielnicki
283601423f Redirect an old png badge with a number as a color; test on [static] (#3412)
Fixes https://github.com/badges/shields/issues/3260

Problem happens when a value of a color in an old PNG static badge is a number: http://localhost:8080/my-label/my-message.png?color=1. In this case `color` in `queryParams` is a number. 
0a0b5b3f03/core/server/server.js (L203-L212)

Surprisingly service test listed below is passing currently on master - value `1` is represented in `queryParams` as a String (only in test). 
`services/static-badge/static-badge.tester.js`
```js
t.create('Old static badge with a number as a color')
  .get('/foo/bar.png?color=1', { followRedirect: false })
  .expectStatus(301)
  .expectHeader('Location', '/badge/foo-bar-1.png')
```

Moreover I added some code + description allowing to debug server.
2019-05-08 12:33:43 -04:00
Paul Melnikow
786bac21ce Convert remaining uses of node-fetch to got (#3417) 2019-05-06 10:10:07 +01:00
Paul Melnikow
001ade3704 Move checkErrorResponse from lib/ to core/ [chrome] (#3408) 2019-05-02 18:07:18 -04:00
Paul Melnikow
a0da978886 Bury the loader fixtures with the code that uses them (#3407)
One less thing to have in the root of the project!
2019-05-02 12:36:00 -04:00
Paul Melnikow
ae2499bf53 Generate JSON badges without using a template [GithubSearch] (#3395)
Warm up for #2428.
2019-04-29 23:26:52 -04:00
Paul Melnikow
b2e21da34d Remove legacy helper makeBadgeData (#3392)
Close #3369
2019-04-29 19:44:25 -04:00
Pierre-Yves B
9a869e24ba Added link test expectations (#3376) 2019-04-29 18:52:58 +01:00
Paul Melnikow
a0492c5283 Remove unused callback-based error helpers (#3371) 2019-04-29 13:35:57 -04:00
Pierre-Yves B
2806eb8a00 Legacy services clean up (#3367)
* Deleted remaining legacy service bits

* Removed badge from README

* Removed no longer needed try/catch

* Deleted refactoring script

* Switched to const

* Reinstated doc

* Ran Prettier
2019-04-27 19:25:01 +01:00
Paul Melnikow
0088a9d0da Unify remaining service order and enforce (#3359)
Ref #3353
2019-04-24 16:53:33 -04:00
Paul Melnikow
051e7d04c1 Rewrite [travisphpversion]; improve error message in expectBadge (#3352)
The change in `expectBadge` prints a more helpful error when `message` is empty.

Ref #2863
2019-04-23 23:03:21 -04:00
Paul Melnikow
1cdcaabd38 Unify order of properties and methods in services (#3353)
I find having these in a consistent order makes the services much faster to read.

This is the order I’ve generally been using:

1. Category
2. Route
3. Examples
4. Rendering
5. Other helpers (`fetch()`, `transform()`)
6. `handle()`
2019-04-23 21:36:04 -04:00
Paul Melnikow
5bedbbd9c1 [security] npm audit fixes; ignore [security] in PR title (#3322) 2019-04-17 11:30:34 -04:00
Paul Melnikow
91d6dd6643 Rewrite [codeclimate] coverage (#3316)
Attacking this in two pieces for ease of review. The legacy implementation for coverage is still there, though I disabled it via the route. That whole file will be removed in the next PR.

Ref #2863
2019-04-15 23:47:25 -04:00
Pierre-Yves B
25f8541e5b JSON format modernisation and _shields_test removal (#3272)
* Modernised JSON format and removed _shields_test style

* Added logoWidth and labelColor fields to JSON response

* Reinstated and updated comment

* Extended expectBadge to accept Joi schemas for all fields
2019-04-07 18:57:55 +01:00
Paul Melnikow
3733de6232 Rewrite GitHub commit status (#3186)
* WIP

* Parse the error response

* Clarify

* Restore one test

* Add a schema
2019-03-10 18:43:37 -05:00
Paul Melnikow
612831a2d8 Remove legacy analytics (#3179)
We're getting good results from #3093, so there's no reason to keep maintaining this code.

Ref #1848 #2068
2019-03-08 00:05:37 -05:00
Paul Melnikow
a6d02239f7 Document queryParams + rename example module (#3170)
This file does more than transform, and `examples` seems is a bit more consistent with e.g. `trace.js` and `route.js`.
2019-03-06 21:04:08 -06:00
Paul Melnikow
a12cbb76fc Prevent queryParams from reaching handle() when no schema is defined (#3171)
I believe I’ve added all the schemas to the new-style services in #3164, so this should be purely preventive.
2019-03-06 21:26:09 -05:00
Paul Melnikow
388b0eefbb Add query param validation to remaining new-style services [azuredevops appveyor npm] (#3164)
Remove now-obsolete code.

Close #2675
2019-03-06 18:13:36 -05:00
Paul Melnikow
4f411755da Add a stats category for static badges (#3168)
Right now they're showing up in "other," though I expect they make up
most of that category.

https://github.com/badges/shields/issues/966#issuecomment-469851361
2019-03-05 21:23:20 -06:00
Paul Melnikow
e159d90516 Reorganize a couple of the legacy helpers [npm node jenkinsplugin dynamic] (#3163) 2019-03-05 20:43:33 -06:00
Caleb Cartwright
726830f067 Add ability to transform query params in redirct service, run [Endpoint GitHubReleaseRedirect SensioLabs VSO] (#3125)
* feat(redirector): added transformQueryParams to redirector

* refactor: renamed to transformPath in redirector

* Rename targetUrl to targetPath

Co-Authored-By: calebcartwright <calebcartwright@users.noreply.github.com>

* feat(redirector): handle param conflicts
2019-03-04 18:11:37 -06:00
Pierre-Yves B
96dfa5b085 Allowed expectBadge to be called with null color (#3137) 2019-03-02 23:26:10 +00:00
Paul Melnikow
56e71d7c76 Label all deprecated services as such (#3120)
* Label all deprecated services as such

This will change

service_requests_total{category="other",family="cocoapods",service="cocoapods_apps"} 76

to

service_requests_total{category="other",family="cocoapods",service="deprecated_cocoapods_apps"} 76

* Fix tests
2019-02-28 18:22:28 -06:00
Paul Melnikow
97358c1399 Rewrite [GitHubStars] (#3119)
* Refactor [GithubStars]

* Add test of link array
2019-02-28 18:18:32 -06:00
Paul Melnikow
fafb22efee Move "good" badge helpers from lib/ to services/ (#3101)
This moves a few helpers from `lib/` to `services/`:

build-status.js
build-status.spec.js
color-formatters.js
color-formatters.spec.js
contributor-count.js
licenses.js
licenses.spec.js
php-version.js
php-version.spec.js
text-formatters.js
text-formatters.spec.js
version.js
version.spec.js

And one from `lib/` to `core/`:

unhandled-rejection.spec.js

The diff is long, but the changes are straightforward.

Ref #2832
2019-02-27 20:47:46 -05:00
Paul Melnikow
dfc95ddd1a Add per-badge metrics for BaseService (#3093)
This picks up #2068 by adding per-badge stats as discussed in #966.

It ensures every service has a unique `name` property. By default this comes from the class name, and is overridden in all the various places where the class names are duplicated. (Some of those don't seem that useful, like the various download interval services, though those need to be refactored down into a single service anyway.) Tests enforce the names are unique. These are the names used by the service-test runner, so it's a good idea to make them unique anyway. (It was sort of strange before that you had to specify `nuget` instead of e.g. `resharper`.)

I've added validation to `deprecatedService` and `redirector`, and required that every `route` has a `base`, even if it's an empty string.

The name is used to generate unique metric labels, generating metrics like these:

```
service_requests_total{category="activity",family="eclipse-marketplace",service="eclipse_marketplace_update"} 2
service_requests_total{category="activity",family="npm",service="npm_collaborators"} 3
service_requests_total{category="activity",family="steam",service="steam_file_release_date"} 2
service_requests_total{category="analysis",family="ansible",service="ansible_galaxy_content_quality_score"} 2
service_requests_total{category="analysis",family="cii-best-practices",service="cii_best_practices_service"} 4
service_requests_total{category="analysis",family="cocoapods",service="cocoapods_docs"} 2
service_requests_total{category="analysis",family="codacy",service="codacy_grade"} 3
service_requests_total{category="analysis",family="coverity",service="coverity_scan"} 2
service_requests_total{category="analysis",family="coverity",service="deprecated_coverity_ondemand"} 2
service_requests_total{category="analysis",family="dependabot",service="dependabot_semver_compatibility"} 3
service_requests_total{category="analysis",family="lgtm",service="lgtm_alerts"} 2
service_requests_total{category="analysis",family="lgtm",service="lgtm_grade"} 3
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_git_hub"} 4
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_npm"} 5
service_requests_total{category="analysis",family="symfony",service="sensiolabs_i_redirector"} 1
service_requests_total{category="analysis",family="symfony",service="symfony_insight_grade"} 1
service_requests_total{category="build",family="appveyor",service="app_veyor_ci"} 3
service_requests_total{category="build",family="appveyor",service="app_veyor_tests"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_build"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_release"} 5
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_tests"} 6
service_requests_total{category="build",family="azure-devops",service="vso_build_redirector"} 2
service_requests_total{category="build",family="azure-devops",service="vso_release_redirector"} 1
service_requests_total{category="build",family="bitbucket",service="bitbucket_pipelines"} 5
service_requests_total{category="build",family="circleci",service="circle_ci"} 5
```

This is predicated on being able to use Prometheus's [`rate()`](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) function to visualize a counter's rate of change, as mentioned at https://github.com/badges/shields/issues/2068#issuecomment-466696561. Otherwise the stats will be disrupted every time a server restarts.

The metrics only appear on new-style services.
2019-02-27 18:58:59 -05:00