Commit Graph

101 Commits

Author SHA1 Message Date
Paul Melnikow
926e837457 Exclude more in-use GitHub IPs from rate limiting and add metrics (#3950)
Fix issue where badges loaded through GitHub intermittently generated 502's on camo.

Ref https://github.com/badges/shields/issues/3874#issuecomment-527904731
2019-09-04 11:30:32 -04:00
Paul Melnikow
b7a29f20ef Add a response-time metric (#3948)
* Refactor existing metrics support into MetricHelper

This completes the refactor done at https://github.com/badges/shields/pull/3662#issuecomment-509011229 in anticipation of adding more metrics support, such as response size of an upstream service, or response time.

* Clean up

* Renames

* Add response time metrics

This adds around 30 new metrics to cover response times at a fairly granular level. We may be able to shrink the number of buckets with time, though I think using 30 metrics is probably okay given that I think may become our most important metric.

* Fix
2019-09-03 22:19:24 +00:00
Paul Melnikow
b36d01ba50 More TypeScript in the frontend (#3742) 2019-08-16 18:29:05 +03:00
chris48s
02e1baf690 improve docs for base service classes (#3850)
improve docs for base service classes
2019-08-14 21:54:46 +01:00
chris48s
75ee413178 Add BaseGraphqlService, support [github] V4 API (#3763)
* add base class for Graphql APIs
* add GithubAuthV4Service + updates to GH token pool
* update github forks to use GithubAuthV4Service
* rename GithubAuthService to GithubAuthV3Service
2019-07-29 21:42:03 +01:00
Paul Melnikow
e33cb30a9a Fix regex match in notfound route (#3790)
Fixes #3789
2019-07-25 17:18:02 -05:00
Paul Melnikow
3f891e4a1f Fix timeout logic (#3774)
Close #3773
2019-07-24 22:51:25 -05:00
Paul Melnikow
a120e5df9b Upgrade to new Sentry SDKs; fix error reporting detail (#3764) 2019-07-24 18:58:22 -05:00
Paul Melnikow
cfbd2c30df SVG by default (#3717)
Make cleaner badge URLs by omitting the `.svg` extension.

Closes #2674
2019-07-24 12:57:39 -05:00
Paul Melnikow
3964eb5cb0 Remove use of deprecated Domain API (#3762)
Use of this feature [has been discouraged for a long time](https://nodejs.org/api/domain.html).

Since most of our code is now bubbling through async, we aren't really getting these "vendor errors" anymore.

Errors that _do_ bubble up through the services have been reported to Sentry since #3706, though they seem to be missing a bunch of their stack traces. Sentry also seems to be combining unrelated internal errors. (https://github.com/badges/shields/issues/3709#issuecomment-514299441) Maybe this will help.
2019-07-23 16:43:09 -05:00
Paul Melnikow
28b8836595 Unify the ESLint config (#3743)
Adding TypeScript to the frontend and a `.d.ts` file to `core` (see #3742) has multiplied out the different combinations of lint rules. ESLint has support for file-pattern-based overrides, which we've used in some places, but we've also maintained a separate eslintrc for `frontend/`.

This merges the config together, with the strategy of putting all the rules at the top level except where they conflict, and applying settings to exactly the files where they should apply.

This introduces a few new errors in the server but they are true positives – hoisting and lowercase class names – things we don't really need to be doing).
2019-07-22 15:06:38 -05:00
Paul Melnikow
ec0264dfb7 Fix Redis prod issue for Node 9 (#3707)
Encountered while testing #3332
2019-07-12 19:51:05 -04:00
Paul Melnikow
153732756c Log internal errors to Sentry (#3706)
Fix #3688
2019-07-12 19:40:46 -04:00
Paul Melnikow
65afff40d4 Fix up Redis config (#3705)
- Move to private
- Fix validation
- Don't log the URL

Ref #3332
2019-07-12 19:12:49 -04:00
Paul Melnikow
84063f8ae3 Switch to ioredis, and update RedisTokenPersistence to use a set instead of a list (#3668)
Also add a script for importing a batch of tokens.

Ref #3332
2019-07-12 18:30:24 -04:00
Paul Melnikow
5d20a371d1 Update serverSecrets reference in legacy monitor (#3697)
Ref #3652 #3393
2019-07-11 19:41:22 -04:00
Paul Melnikow
e2608a6570 Refactor GitHub OAuth credential handling (#3693)
Continues the work of #3652.
2019-07-11 18:04:53 -04:00
Paul Melnikow
b2e5aa9208 Cache raster redirects longer (#3690)
Haven't seen error reports on these, so let's cache these a bit longer to reduce our server load.
2019-07-11 16:57:21 -04:00
chris48s
38cdc0033f adopt JSDoc, eslint-plugin-jsdoc (#3645)
eslint-plugin-jsdoc:
- install eslint-plugin-jsdoc
- config file
- fix lint/style errors

JSDoc:
- add JSDoc as a dev dependency
- get everything rendering nicely with JSDoc
- config, build command + ignores
2019-07-11 20:14:47 +01:00
Paul Melnikow
ce0ddf93fc Inject secrets into the services (#3652)
This is a reworking of #3410 based on some feedback @calebcartwright left on that PR.

The goals of injecting the secrets are threefold:

1. Simplify testing
2. Be consistent with all of the other config (which is injected)
3. Encapsulate the sensitive auth-related code in one place so it can be studied and tested thoroughly

- Rather than add more code to BaseService to handle authorization logic, it delegates that to an AuthHelper class.
- When the server starts, it fetches the credentials from `config` and injects them into `BaseService.register()` which passes them to `invoke()`.
- In `invoke()` the service's auth configuration is checked (`static get auth()`, much like `static get route()`).
- If the auth config is present, an AuthHelper instance is created and attached to the new instance.
- Then within the service, the password, basic auth config, or bearer authentication can be accessed via e.g. `this.authHelper.basicAuth` and passed to `this._requestJson()` and friends.
- Everything is being done very explicitly, so it should be very clear where and how the configured secrets are being used.
- Testing different configurations of services can now be done by injecting the config into `invoke()` in `.spec` files instead of mocking global state in the service tests as was done before. See the new Jira spec files for a good example of this.

Ref #3393
2019-07-09 23:14:36 -04:00
Paul Melnikow
aa135d370f Auto-annotate live and mock tests (#3661)
Close #2555
2019-07-07 17:06:01 -04:00
Paul Melnikow
eec904c262 Fix raster redirect and drop the legacy static .png route (#3659) 2019-07-07 11:49:21 -04:00
Paul Melnikow
66c7f13e38 Drop gif + png, and redirect png to raster.shields.io (#3644)
1. Remove rasterization support from the server. This responsibility is delegated to a raster server which proxies the SVG badges and renders them.
2. When a raster server URL is configured, 301 redirect all .png badges to the identical URL on the raster server.
    `https://img.shields.io/npm/v/express.png?style=flat-square` ↪️`https://raster.shields.io/npm/v/express.png?style=flat-square`
3. For configured redirects, redirect to the canonical URL on the raster server.
    `https://img.shields.io/vso/build/totodem/8cf3ec0e-d0c2-4fcd-8206-ad204f254a96/2.png?style=flat-square`
    ↪️`https://img.shields.io/azure-devops/build/totodem/8cf3ec0e-d0c2-4fcd-8206-ad204f254a96/2.png?style=flat-square`
4. Redirect the "legacy badge old version" to the appropriate URL on the raster server.
5. When no raster server is configured (e.g. PRs), render an SVG containing **404 | raster badges not available** for all `.png` badges. (Note that the raster server can be self-hosted; however, this is deferred to a later PR.)
5. Drop support for jpg and gif which are very infrequently used (see #3112). Render an SVG containing **410 | jpg no longer available**.
7. ~~Remove raster dependencies.~~ Remove the raster cache (which is only used in the CLI, and therefore pointless).
8. Move the LRUCache code out of the npm package.
8. A wee bit of refactoring in `server.js`.

Ref #3112
Close #3631
2019-07-06 16:41:46 -04:00
Paul Melnikow
093d9cd368 Minor refactor: expose custom promisify function from legacy request handler (#3650)
Cherry-picked from #3410; should simplify reworking it.
2019-07-05 21:26:05 -04:00
chris48s
0d3266a0ad JSDoc comments for service test runner (#3630) 2019-07-02 12:41:28 -04:00
chris48s
f29da0abdd JSDoc comments for token pool (#3632) 2019-07-01 17:55:08 -04:00
chris48s
d78edaf5b5 add/improve some JSDoc comments on core API (#3595) 2019-06-25 22:42:01 +01:00
dependabot-preview[bot]
95b6e48bff Build(deps-dev): bump nock from 11.0.0-beta.14 to 11.0.0-beta.19 (#3552)
* Build(deps-dev): bump nock from 11.0.0-beta.14 to 11.0.0-beta.19

Bumps [nock](https://github.com/nock/nock) from 11.0.0-beta.14 to 11.0.0-beta.19.
- [Release notes](https://github.com/nock/nock/releases)
- [Changelog](https://github.com/nock/nock/blob/beta/CHANGELOG.md)
- [Commits](https://github.com/nock/nock/commits)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* Fix one test

* Update for breaking change in nock api

* Always use got without retries in tests
2019-06-21 10:45:34 -04:00
dependabot-preview[bot]
d732b937c9 Build(deps-dev): bump prettier from 1.17.1 to 1.18.2 (#3580)
* Build(deps-dev): bump prettier from 1.17.1 to 1.18.2

Bumps [prettier](https://github.com/prettier/prettier) from 1.17.1 to 1.18.2.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/master/CHANGELOG.md)
- [Commits](https://github.com/prettier/prettier/compare/1.17.1...1.18.2)

Signed-off-by: dependabot-preview[bot] <support@dependabot.com>

* chore: prettier
2019-06-17 19:15:44 -05:00
chris48s
8621fe42d7 Upgrade Joi (#3505)
* upgrade joi
* find & replace ALL THE THINGS
* update related deps
2019-06-02 21:59:55 +01:00
Marcin Mielnicki
283601423f Redirect an old png badge with a number as a color; test on [static] (#3412)
Fixes https://github.com/badges/shields/issues/3260

Problem happens when a value of a color in an old PNG static badge is a number: http://localhost:8080/my-label/my-message.png?color=1. In this case `color` in `queryParams` is a number. 
0a0b5b3f03/core/server/server.js (L203-L212)

Surprisingly service test listed below is passing currently on master - value `1` is represented in `queryParams` as a String (only in test). 
`services/static-badge/static-badge.tester.js`
```js
t.create('Old static badge with a number as a color')
  .get('/foo/bar.png?color=1', { followRedirect: false })
  .expectStatus(301)
  .expectHeader('Location', '/badge/foo-bar-1.png')
```

Moreover I added some code + description allowing to debug server.
2019-05-08 12:33:43 -04:00
Paul Melnikow
786bac21ce Convert remaining uses of node-fetch to got (#3417) 2019-05-06 10:10:07 +01:00
Paul Melnikow
001ade3704 Move checkErrorResponse from lib/ to core/ [chrome] (#3408) 2019-05-02 18:07:18 -04:00
Paul Melnikow
a0da978886 Bury the loader fixtures with the code that uses them (#3407)
One less thing to have in the root of the project!
2019-05-02 12:36:00 -04:00
Paul Melnikow
ae2499bf53 Generate JSON badges without using a template [GithubSearch] (#3395)
Warm up for #2428.
2019-04-29 23:26:52 -04:00
Paul Melnikow
b2e21da34d Remove legacy helper makeBadgeData (#3392)
Close #3369
2019-04-29 19:44:25 -04:00
Pierre-Yves B
9a869e24ba Added link test expectations (#3376) 2019-04-29 18:52:58 +01:00
Paul Melnikow
a0492c5283 Remove unused callback-based error helpers (#3371) 2019-04-29 13:35:57 -04:00
Pierre-Yves B
2806eb8a00 Legacy services clean up (#3367)
* Deleted remaining legacy service bits

* Removed badge from README

* Removed no longer needed try/catch

* Deleted refactoring script

* Switched to const

* Reinstated doc

* Ran Prettier
2019-04-27 19:25:01 +01:00
Paul Melnikow
0088a9d0da Unify remaining service order and enforce (#3359)
Ref #3353
2019-04-24 16:53:33 -04:00
Paul Melnikow
051e7d04c1 Rewrite [travisphpversion]; improve error message in expectBadge (#3352)
The change in `expectBadge` prints a more helpful error when `message` is empty.

Ref #2863
2019-04-23 23:03:21 -04:00
Paul Melnikow
1cdcaabd38 Unify order of properties and methods in services (#3353)
I find having these in a consistent order makes the services much faster to read.

This is the order I’ve generally been using:

1. Category
2. Route
3. Examples
4. Rendering
5. Other helpers (`fetch()`, `transform()`)
6. `handle()`
2019-04-23 21:36:04 -04:00
Paul Melnikow
5bedbbd9c1 [security] npm audit fixes; ignore [security] in PR title (#3322) 2019-04-17 11:30:34 -04:00
Paul Melnikow
91d6dd6643 Rewrite [codeclimate] coverage (#3316)
Attacking this in two pieces for ease of review. The legacy implementation for coverage is still there, though I disabled it via the route. That whole file will be removed in the next PR.

Ref #2863
2019-04-15 23:47:25 -04:00
Pierre-Yves B
25f8541e5b JSON format modernisation and _shields_test removal (#3272)
* Modernised JSON format and removed _shields_test style

* Added logoWidth and labelColor fields to JSON response

* Reinstated and updated comment

* Extended expectBadge to accept Joi schemas for all fields
2019-04-07 18:57:55 +01:00
Paul Melnikow
3733de6232 Rewrite GitHub commit status (#3186)
* WIP

* Parse the error response

* Clarify

* Restore one test

* Add a schema
2019-03-10 18:43:37 -05:00
Paul Melnikow
612831a2d8 Remove legacy analytics (#3179)
We're getting good results from #3093, so there's no reason to keep maintaining this code.

Ref #1848 #2068
2019-03-08 00:05:37 -05:00
Paul Melnikow
a6d02239f7 Document queryParams + rename example module (#3170)
This file does more than transform, and `examples` seems is a bit more consistent with e.g. `trace.js` and `route.js`.
2019-03-06 21:04:08 -06:00
Paul Melnikow
a12cbb76fc Prevent queryParams from reaching handle() when no schema is defined (#3171)
I believe I’ve added all the schemas to the new-style services in #3164, so this should be purely preventive.
2019-03-06 21:26:09 -05:00
Paul Melnikow
388b0eefbb Add query param validation to remaining new-style services [azuredevops appveyor npm] (#3164)
Remove now-obsolete code.

Close #2675
2019-03-06 18:13:36 -05:00