* Migrate [Discord] implementation to use bot token
* Rework authorization field creation
* Revert "Rework authorization field creation"
This reverts commit caf65bde5d.
* Add LGTM exclusion for hardcoded credentials
* Validate input to BadgeFactory.create() (#3875)
* validate input to create()
* remove deprecated properties (#3881)
* remove BadgeFactory class (#3884)
* Template literal templates (#4459)
- Remove use of the doT template library and move to generating SVG output using javascript template literals.
- Drop SVGO and mostly manually implement the optimisations.
- Add a bunch more tests
Co-authored-by: Paul Melnikow <github@paulmelnikow.com>
* drop raster support in package CLI (#4523)
* drop raster support in package CLI
* update docs
* rename gh-badges package to badge-maker
* rename gh-badges dir to badge-maker
* update relative imports and other refs to in parent dir
'gh-badges' --> 'badge-maker'
* update snyk service tests
This change is only tangentially related
We've used the shields repo as an example for these tests so
moving files around in our repo has a knock-on effect on them
* add missing type hints to dev style page
* write the changelog/migration guide for v3
* use extension in README CLI example
* update CLI help
whoops - missed this in #4523
* bump version
* update for self-hosting users
* README updates
* drop .format param from CLI, always output SVG
* Change text[] to label and message, Remove JSON output
- Change text[] to label and message
- Fix message only badge
- Remove JSON output format
- Update the docs
* update package-lock
* rename 'template' to 'style'
* handle invalid styles in coalesceBadge
* ensure makeBadge is passed a string for template in coalesceBadge()
issue #4925
* fix (logo/no label text/label color specified) case
issue #4926
* add example of (logo/no label text/label color specified) to style debug page
* update type defs
* padding fix for FTB style
Co-authored-by: Paul Melnikow <github@paulmelnikow.com>
* prom-client JSON to InfluxDB line protocol converter
* Converts a metric with separate names
* prom-client JSON to InfluxDB line protocol (version 2) converter
* Server has instance id
* Read the instance id from an environment variable
* More unit tests for instance-metadata
* Log instance id
* Push influx metrics
* INSTANCE_ID with dyno metadata
* Prepare influx metrics in one place
* Influx metrics endpoint should return metrics
* More readable tests
* Env added to instance metadata
* hostname as an instance label value
* HEROKU_DYNO_ID as an instance id for heroku
* Instance env can be set by env variable
* HEROKU_APP_NAME as an instance env
* Log instance metadata as a JSON
* Typo fix
* Code refactoring in tests
* wait-for-expect dev dependency added
* Test for pushing metrics
* Test for pushing metrics
* Use basic authentication for pushing metrics
* intervalSeconds=2 for development env
* Using existing methods
* TODOs removed
* Schema for influx credentials
* Influx config removed from config files
* Require username and password when influx metrics are enabled
* Unused args removed
* pushing component should log errors
* Speed up tests
* should log error responses
* InstanceMetadata class replaces by simple object
* Influx metrics can be configuredd by env variables
* Use application label name instead of service
* Unused code removed
* Integration test for prom-client and converter
* metrics.influx.enabled configuration option added
* Improved influx configuration schema
* instanceMetadata validation
* Typo fix
* Default value for env
* metrics.infux.hostnameAsAInstanceId added
* should add hostname as an instance label when hostnameAsAInstanceId is enabled
* Default values for influx configuration
* flatMap is not available in Node.js 9.4
* Env vars removed from Procfile
* Better instance metadata values in tests
* Typo fix
* lodash.groupby added to prod dependencies
* Allow other keys in private config
* Missing test - should allow other private keys when influx metrics are enabled
* Missing test - should require private metrics config when influx configuration is enabled
* log.error instead of console.log
* metrics.influx.uri -> metrics.influx.url
* Unused arguments removed
* async removed
* promisify sendMetrics
* Allow to disable prometheus metrics
* Create test server with custom config
* 'metrics-influx' resource removed
* 'metrics-influx' resource removed
* Private config schema flattened out
* Extra code removed in Prometheus tests
* promisify moved outside of the class
* Do not throw errors from got in a specific test
* hostnameAliases added
* instanceIdFrom added
* instanceIdEnvVarName added
* envLabel added to schema
* instanceMetadata is not used by InfluxMetrics
* Instance metadata removed
* hostnameAsAnInstanceId removed
* A comment added
* waitForExpect removed
* Unused code removed
* Make it easier to benchmark and profile the code
* Remove unnecessary escape
* Clarify that the backend server is started without the frontend
* Add missing NODE_CONFIG_ENV environment variable
* Add error message when user has not included console.time statements
* Fix lint issue
* Handle multiple console.time statements
* Switch NODE_CONFIG_ENV to test
* Switch to const as variable never re-assigned
* Response size metric for all services
* Unused code removed
* Test for service response size metric
* All buckes of the service_response_bytes in a comment
* Register parameter in PrometheusMetrics is optional
* service response size metric enabled for dynamic badges
* Better test name
* JSDoc removed
* One import from one file
* Gather metrics in the background
* Revert saving response time metrics in the background
* Changed configuration validation to allow using a named pipe for port.
* Update core/server/server.js
Co-Authored-By: Caleb Cartwright <calebcartwright@users.noreply.github.com>
* Applied prettier.
* Update core/server/server.spec.js
Co-Authored-By: Caleb Cartwright <calebcartwright@users.noreply.github.com>
* Updated spec for port setting to allow only numbers or formatted named pipes.
* Try enable universal cors
* Move handle from registerErrorHandlers because it isn't and error handler
* Add test for cors headers
Also add link to issue
* add /twitch/status/:user badge
* update comments
* use a proper schema for the Twitch API calls
* use a token to make Twitch api calls
* fix handling of rate-limit error and bad token error
* [twitch] get a token as soon as creating a Twitch service
* [twitch] start both requests to users and stream before awaiting
* [twitch] set a timeout to replace the token before it expires
* [twitch] use authHelper
* [twitch] skip tests when no credentials
* [twitch] add one more status test
* twitch: do not check whether a user exists
* Refactor existing metrics support into MetricHelper
This completes the refactor done at https://github.com/badges/shields/pull/3662#issuecomment-509011229 in anticipation of adding more metrics support, such as response size of an upstream service, or response time.
* Clean up
* Renames
* Add response time metrics
This adds around 30 new metrics to cover response times at a fairly granular level. We may be able to shrink the number of buckets with time, though I think using 30 metrics is probably okay given that I think may become our most important metric.
* Fix
Adding TypeScript to the frontend and a `.d.ts` file to `core` (see #3742) has multiplied out the different combinations of lint rules. ESLint has support for file-pattern-based overrides, which we've used in some places, but we've also maintained a separate eslintrc for `frontend/`.
This merges the config together, with the strategy of putting all the rules at the top level except where they conflict, and applying settings to exactly the files where they should apply.
This introduces a few new errors in the server but they are true positives – hoisting and lowercase class names – things we don't really need to be doing).
This is a reworking of #3410 based on some feedback @calebcartwright left on that PR.
The goals of injecting the secrets are threefold:
1. Simplify testing
2. Be consistent with all of the other config (which is injected)
3. Encapsulate the sensitive auth-related code in one place so it can be studied and tested thoroughly
- Rather than add more code to BaseService to handle authorization logic, it delegates that to an AuthHelper class.
- When the server starts, it fetches the credentials from `config` and injects them into `BaseService.register()` which passes them to `invoke()`.
- In `invoke()` the service's auth configuration is checked (`static get auth()`, much like `static get route()`).
- If the auth config is present, an AuthHelper instance is created and attached to the new instance.
- Then within the service, the password, basic auth config, or bearer authentication can be accessed via e.g. `this.authHelper.basicAuth` and passed to `this._requestJson()` and friends.
- Everything is being done very explicitly, so it should be very clear where and how the configured secrets are being used.
- Testing different configurations of services can now be done by injecting the config into `invoke()` in `.spec` files instead of mocking global state in the service tests as was done before. See the new Jira spec files for a good example of this.
Ref #3393
Fixes https://github.com/badges/shields/issues/3260
Problem happens when a value of a color in an old PNG static badge is a number: http://localhost:8080/my-label/my-message.png?color=1. In this case `color` in `queryParams` is a number.
0a0b5b3f03/core/server/server.js (L203-L212)
Surprisingly service test listed below is passing currently on master - value `1` is represented in `queryParams` as a String (only in test).
`services/static-badge/static-badge.tester.js`
```js
t.create('Old static badge with a number as a color')
.get('/foo/bar.png?color=1', { followRedirect: false })
.expectStatus(301)
.expectHeader('Location', '/badge/foo-bar-1.png')
```
Moreover I added some code + description allowing to debug server.
This picks up #2068 by adding per-badge stats as discussed in #966.
It ensures every service has a unique `name` property. By default this comes from the class name, and is overridden in all the various places where the class names are duplicated. (Some of those don't seem that useful, like the various download interval services, though those need to be refactored down into a single service anyway.) Tests enforce the names are unique. These are the names used by the service-test runner, so it's a good idea to make them unique anyway. (It was sort of strange before that you had to specify `nuget` instead of e.g. `resharper`.)
I've added validation to `deprecatedService` and `redirector`, and required that every `route` has a `base`, even if it's an empty string.
The name is used to generate unique metric labels, generating metrics like these:
```
service_requests_total{category="activity",family="eclipse-marketplace",service="eclipse_marketplace_update"} 2
service_requests_total{category="activity",family="npm",service="npm_collaborators"} 3
service_requests_total{category="activity",family="steam",service="steam_file_release_date"} 2
service_requests_total{category="analysis",family="ansible",service="ansible_galaxy_content_quality_score"} 2
service_requests_total{category="analysis",family="cii-best-practices",service="cii_best_practices_service"} 4
service_requests_total{category="analysis",family="cocoapods",service="cocoapods_docs"} 2
service_requests_total{category="analysis",family="codacy",service="codacy_grade"} 3
service_requests_total{category="analysis",family="coverity",service="coverity_scan"} 2
service_requests_total{category="analysis",family="coverity",service="deprecated_coverity_ondemand"} 2
service_requests_total{category="analysis",family="dependabot",service="dependabot_semver_compatibility"} 3
service_requests_total{category="analysis",family="lgtm",service="lgtm_alerts"} 2
service_requests_total{category="analysis",family="lgtm",service="lgtm_grade"} 3
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_git_hub"} 4
service_requests_total{category="analysis",family="snyk",service="snyk_vulnerability_npm"} 5
service_requests_total{category="analysis",family="symfony",service="sensiolabs_i_redirector"} 1
service_requests_total{category="analysis",family="symfony",service="symfony_insight_grade"} 1
service_requests_total{category="build",family="appveyor",service="app_veyor_ci"} 3
service_requests_total{category="build",family="appveyor",service="app_veyor_tests"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_build"} 6
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_release"} 5
service_requests_total{category="build",family="azure-devops",service="azure_dev_ops_tests"} 6
service_requests_total{category="build",family="azure-devops",service="vso_build_redirector"} 2
service_requests_total{category="build",family="azure-devops",service="vso_release_redirector"} 1
service_requests_total{category="build",family="bitbucket",service="bitbucket_pipelines"} 5
service_requests_total{category="build",family="circleci",service="circle_ci"} 5
```
This is predicated on being able to use Prometheus's [`rate()`](https://prometheus.io/docs/prometheus/latest/querying/functions/#rate) function to visualize a counter's rate of change, as mentioned at https://github.com/badges/shields/issues/2068#issuecomment-466696561. Otherwise the stats will be disrupted every time a server restarts.
The metrics only appear on new-style services.
This will definitely save time, and ensure more uniformity.
It moves the `createServiceTester()` calls to a different place from where I'd like them, though I'm happy to have them checked by the linter.
Closes#2701
While Next.js can handle static sites, we've had a few issues with it, notably a performance hit at runtime and some bugginess around routing and SSR. Gatsby being fully intended for high-performance static sites makes it a great technical fit for the Shields frontend. The `createPages()` API should be a really nice way to add a page for each service family, for example.
This migrates the frontend from Next.js to Gatsby. Gatsby is a powerful tool, which has a bit of downside as there's a lot to dig through. Overall I found configuration easier than Next.js. There are a lot of plugins and for the most part they worked out of the box. The documentation is good.
Links are cleaner now: there is no #. This will break old links though perhaps we could add some redirection to help with that. The only one I’m really concerned about `/#/endpoint`. I’m not sure if folks are deep-linking to the category pages.
There are a lot of enhancements we could add, in order to speed up the site even more. In particular we could think about inlining the SVGs rather than making separate requests for each one.
While Gatsby recommends GraphQL, it's not required. To keep things simple and reduce the learning curve, I did not use it here.
Close#1943Fix#2837Fix#2616
The suggest code was an exception to our usual organization pattern. There was a service test, but it's not a service. The code would sometimes regress because it wasn't being tested all the time.
This makes them no longer run as service tests, which is good because they run as part of every build. Some of them are smaller-bracket tests which is good too, because it will make them easier to test, especially as this code grows.
I'd have liked to keep using frisby for the ones that make requests to the server, though I ran into some issues with sequencing of setup that I think will require upstream changes.
In #2698 we decided to put legacy helper functions in `core/legacy`. I think that’s a fine idea, though if we’re going to have a bunch of badge helper functions in there, it seems like it is probably better to keep these two important but esoteric helper functions with the core code to which they are most coupled. So I added `legacy-` to the name, and put them in `core/base-service`.
