Commit Graph
108 Commits
Author SHA1 Message Date
chris48sandGitHub e294dc4edc refactor [circleci] integration (#1927)
* refactor [circleci] integration
2018-08-28 15:49:43 +01:00
Y0hy0handPaul Melnikow edea36dd10 [elm-package] Fix elm package service (#1986) 2018-08-28 10:25:58 -04:00
Paul MelnikowandGitHub 64d325b002 Reduce boilerplate in a few more service testers [cdnjs continuousphp node shippable wercker gem] (#1957)
Follow up to #1934.
2018-08-27 16:36:35 -04:00
Paul MelnikowandGitHub bedba47d77 Move legacy services from server.js into services/ (#1958)
This builds on the work of #1931 by moving the legacy services into `services/`.
2018-08-27 13:29:54 -04:00
Paul MelnikowandGitHub 302c8606ff Rewrite [pypi]; affects [npm] (#1922) 2018-08-27 07:46:06 -04:00
Nicco KunzmannandPyves 504b68ef13 Make wercker preview badge static (#1974) 2018-08-26 15:40:24 +01:00
PyvesandGitHub ddb3bf4256 Fixed libraries.io error handling (#1964) 2018-08-24 18:32:28 +01:00
chris48sandGitHub ae190c5f07 generate static examples without api call [apm appveyor cdnjs clojars gem npm uptimerobot] (#1740)
* allow service classes to define a static example
* define static example for some services
  (apm, appveyor, cdnjs, clojars, gem, librariesio, npm, uptimerobot)
* add/update tests


This allows us to show an example without making an API call to a live service for better performance.

We can now specify 3 fields in the example definition:

* urlPattern for the version with placeholders e.g: /npm/dw/:package.svg
* ExampleUrl/Uri for the concrete example e.g: /npm/dw/localeval.svg
* PreviewUrl/Uri for the static (or live) image we will actually show
2018-08-23 20:22:24 +01:00
Paul MelnikowandGitHub a0c43ed219 Reduce boilerplate for creating new testers [npm appveyor gem uptimerobot clojars] (#1934)
This is a bit of sugar that reduces the boilerplate for creating testers in what I expect will become the standard case: a service in `foo/foo-thing.service.js` with its tests in `foo/foo-thing.tester.js`.

This makes a small stylistic change, which is to name the service by its CamelCase class name rather than an invented snake-case ID. Whereas before the name was specified in `class FooThing extends Base[Json]Service` and a second time in the tester, it now only needs to be specified once.
2018-08-22 18:07:51 -04:00
Paul MelnikowandGitHub 452affba15 Sort some more badges (#1948)
- Move the Docker Pulls badge from **Other** to **Downloads**.
- Create **Platform & Version Support** for the following:
    - django version support
    - python version support
    - node version support
    - pypi - wheel (waiting to avoid conflict with #1922)
    - pipi - format (waiting to avoid conflict with #1922)
    - pipi - implementation (waiting to avoid conflict with #1922)
    - hhvm
    - cocoapods platform
    - conda
    - php version badges
2018-08-22 15:05:11 -04:00
Paul MelnikowandGitHub 942466aa92 Allow migrating legacy services to services/ (#1931) 2018-08-21 23:38:12 -04:00
PyvesandGitHub 82bc90cec1 Deprecated VersionEye (#1953) 2018-08-20 20:04:11 +01:00
Paul MelnikowandGitHub 656326d693 Rewrite [appveyor] tests badge (#1940) 2018-08-20 14:49:59 -04:00
Paul MelnikowandGitHub 77b54c8c9a Sort some badges (#1945)
This implements proposals 1, 2, and 4 from #1905:

* Move gem rank from ratings to downloads; tweak title

* Move docker build from other to build

* Move bundlephobia, github file, github repo, imagelayers, and microbadger size + layers badges into new category

* Fill in a couple missing titles
2018-08-19 17:55:20 -04:00
Paul MelnikowandGitHub a16d436602 Optionally persist [Github] tokens in Redis (#1939)
This is a fairly simple addition of a Redis-backed TokenPersistence. When GithubConstellation is initialized, it will create a FsTokenPersistence or a RedisTokenPersistence based on configuration. Have added tests of the Redis backend as an integration test, and ensured the server starts up correctly when a `REDIS_URL` is configured.

Ref: #1848
2018-08-19 10:27:23 -04:00
Paul MelnikowandGitHub edb7d82500 Change format of [npm] type definitions badge (#1897)
Based on feedback at chalk/chalk#286.
2018-08-19 09:27:23 -04:00
Paul MelnikowandGitHub b10a6a4aa7 Refactor [github] token persistence, again (#1906)
Instead of saving tokens on a timer, save them when they change. Use EventEmitter to keep the components loosely coupled.

This is easier to reason about, much easier to test, and better supports adapting to backends which support atomic operations.

This replaces json-autosave, which was a bit difficult to read and also hard to test, with fsos, the lower-level utility it’s built on.

Ref: #1848
2018-08-18 23:54:53 -04:00
Paul MelnikowandGitHub a252239018 Fix and test trace logging in BaseJsonService (#1944)
This fixes a regression introduced in #1929 which was caught by service tests post-merge.
2018-08-18 17:45:46 -04:00
Paul MelnikowandGitHub 12be1bd747 Service logging tweaks (#1929)
- Log the response when using `test:services:trace`
- Fully log large nested objects

Since the `badgeData` is in a different format from the JSON response, and also doesn't include the title, including this output is helpful. It makes it clearer what the Joi matchers are trying to match.

Sometimes when there's a deep nested structure, it's helpful or necessary to see the entire thing.
2018-08-18 11:25:40 -04:00
PyvesandGitHub 886cde72d5 Rewrite [Wercker] service (#1920) 2018-08-16 22:03:15 +01:00
Paul MelnikowandGitHub 9119c235a9 Prevent merging exclusive service tests (#1926)
* Remove an .only() left behind previously
2018-08-16 11:06:08 -07:00
PyvesandGitHub b0070ff861 [GitHub] Error message customisation for all status codes (#1888) 2018-08-14 18:57:24 +01:00
Paul MelnikowandGitHub 0c703c11b5 Rewrite [uptimerobot] (#1891) 2018-08-12 21:47:30 -07:00
Paul MelnikowandGitHub 66d444aa40 Clean up our callback style and enforce no exclusive tests and remove an exclusive test (#1900)
We use arrow functions in most places; this enforces it.

Passing arrow functions to Mocha is discouraged: https://mochajs.org/#arrow-functions

This was a mix of autofixes and hand adjustments.
2018-08-12 20:45:43 -04:00
Paul MelnikowandGitHub 0a7c833452 Fix an edge case in NpmBase (#1904)
I noticed this when I copied this code while I was working on #1895.
2018-08-12 19:13:31 -04:00
chris48sandGitHub 0f534cefc0 use fetch/render pattern in [apm appveyor cdnjs clojars gem] (#1901)
* remove tests we don't need
* use fetch/render pattern in [apm appveyor cdnjs clojars gem]
2018-08-12 18:27:47 +01:00
Paul MelnikowandGitHub 67193ce09b Add [librariesio-sourcerank] (#1898)
It seems like 30+ is great; under 10 pretty bad. I picked some values for a color scheme.

Closes #1532
2018-08-12 12:58:03 -04:00
Paul MelnikowandGitHub 1a71acfa2e Add badge for dependents and dependent repos using Libraries.io (#1895)
See #1845
2018-08-12 11:05:43 -04:00
Paul MelnikowandGitHub 2a035d9111 ServiceTester: Minor readability tweak (#1893)
Was debugging this just now, and think this is easier to read and understand than the previous version.
2018-08-12 07:43:12 -04:00
DanialandGitHub b2af8f7a04 [GitHub] support issue tags with "/" character (#1902)
* support tags with "/" character
2018-08-12 15:20:30 +12:00
Paul MelnikowandGitHub 9007658fd0 Refactor and test [github] token persistence (#1863)
Ref #1848 #1205
2018-08-11 20:13:40 -04:00
Paul MelnikowandGitHub 39d393028d Fix and test the github admin route (#1886)
Fix regressions in the github admin and token acceptor endpoints, introduced in #1813.
2018-08-11 19:48:43 -04:00
Paul Melnikowandchris48s cd6c38a616 Add InvalidParameter runtime error and return inaccessible for 5xx errors (#1890)
* InvalidParameter: New error type
* Return inaccessible for 5xx errors from services
* Add test for Inaccessible on 5xx
* Add tests for named error types
2018-08-11 21:05:56 +01:00
Paul MelnikowandGitHub db4bffb300 Split BaseService and BaseJsonService into separate modules (#1889)
There’s a lot of behavior here, and going to be even more, so I think it makes sense to split these up as I’ve done with the tests.
2018-08-11 10:43:05 -04:00
Paul MelnikowandGitHub ed81dcbb36 npm downloads: Better experience for new projects (#1892)
When a new project is created, the download badge shows “project not found” as reported in #1882. This gives a more informative message in that case.
2018-08-11 10:18:41 -04:00
Paul MelnikowandGitHub 09be682963 BaseService: Make it easier to add more config options (#1884)
I ran into this while working on #1867, where ultimately I couldn't solve my problem by injecting config. This will make it easier in the future, though.
2018-08-10 16:02:58 -04:00
chris48sandGitHub f05a3496ee Validate response json in [apm appveyor cdnjs clojars gem] (#1883)
* split gem service into multiple files
* add validation to apm, appveyor, cdnjs, clojars & gem services
* fix the apm examples
2018-08-10 20:52:09 +01:00
Paul MelnikowandGitHub 0db88d33e0 Add debug logging to BaseService and BaseJsonService (#1867)
Invoke this using `npm run test:services:trace`.
2018-08-10 15:22:13 -04:00
Paul MelnikowandGitHub ada1dec815 Lengthen timeout on a flaky service test (#1885) 2018-08-10 15:10:31 -04:00
Paul MelnikowandGitHub 8af48c0a21 Minor tweaks in [github] auth code (#1862)
* Move / rework is-valid-token

* TokenPool: Add allValidTokenIds method

* Minor cleanup in server.js
2018-08-09 07:22:54 -04:00
Paul MelnikowandGitHub c11d97a192 Refactor [github] initialization (#1861)
This creates a new convenience class which consolidates all the Github initialization. It supports dependency injection and facilitates refactoring the persistence along the lines of #1205.

Also ref #1848
2018-08-09 07:20:57 -04:00
Paul MelnikowandGitHub 7a664ca3e8 Run prettier (#1866)
Merging this separately so the commit with the tooling change is readable. This is a follow-on to #1167 which turned prettier on.
2018-08-08 17:57:14 -04:00
Paul MelnikowandGitHub e3b100504d Add Joi-based request validation to BaseJsonService and rewrite [NPM] and [node] badges (#1743)
When JSON responses come back, they are sometimes not in the format expected by the API. As a result we have a lot of defensive coding (expressions like `(data || {}).someProperty`) to avoid exceptions being thrown in badge processing. Often we rely on the `try` blocks that wrap so much of the badge-processing code, which catch all JavaScript exceptions and return some error result, usually **invalid**. The problem with this is that these `try` blocks catch all sorts of programmer errors too, so when we see **invalid** we don't know whether the API returned something unexpected, or we've made a mistake. We also spend a lot of time writing defensive tests around malformed responses, and creating and maintaining the defensive coding.

A better solution is to validate the API responses using declarative contracts. Here the programmer says exactly what they expect from the API. That way, if the response isn't what we expect we can just say it's an **invalid json response**. And if our code then throws an exception, well that's our mistake; when we catch that we can call it a **shields internal error**. It's also less code and less error-prone. Over time we may be confident enough in the contracts that we won't need so many tests of malformed responses. The contract doesn't need to describe the entire response, only the part that's needed. Unknown keys can simply be dropped, preventing unvalidated parts of the response from creeping into the code. Checking what's in our response before calling values on it also makes our code more secure.

I used Joi here, since we're already using it for testing. There may be another contracts library that's a better fit, though I think we could look at that later.

Those changes are in base.js.

The rest is a rewrite of the remaining NPM badges, including the extraction of an NpmBase class. Inspired by @chris48s's work in #1740, this class splits the service concerns into fetching, validation, transformation, and rendering. This is treated as a design pattern. See the PR discussion for more. There are two URL patterns, one which allows specifying a tag (used by e.g. the version badge `https://img.shields.io/npm/v/npm/next.svg`), and the other which does not accept a tag (e.g. the license badge `https://img.shields.io/npm/l/express.svg`). Subclasses like NpmLicense and NpmTypeDefinitions can specify the URL fragment, examples, the validation schema for the chunk of the package data they use, and a render function. The NpmVersion subclass uses a different endpoint, so it overrides the `handle` implementation from NpmBase.

The remaining services using BaseJsonService are shimmed, so they will keep working after the changes.
2018-08-08 17:08:16 -04:00
PyvesandGitHub e1affea266 [GitHub] Fix undefined badges (#1816) 2018-08-08 20:57:57 +01:00
Paul MelnikowandGitHub 56fcb2e5ba GithubApiProvider: injectible interface for code that calls github (#1812) 2018-08-07 16:46:12 -04:00
Paul MelnikowandGitHub 2ff71dab6a Start cleaning up github auth routes (#1813)
- Move github token debug route to separate module
- Use crypto.timingSafeEqual
- Rename getTokenDebugInfo -> serializeDebugInfo
2018-08-01 21:58:11 -04:00
Paul MelnikowandGitHub 7e04f25fc5 Tweak layout of services to conveniently handle large service families 2018-08-01 16:49:07 -04:00
chris48sandGitHub 7b10ba2f11 update dependabot tests (#1820) 2018-08-01 21:23:36 +01:00
PyvesandGitHub 9335f607aa [conda] Fixed wrong variable name (#1817) 2018-07-31 22:27:06 +01:00
chris48sandGitHub 241cac7100 Fix [circleci] badge (#1815)
require all jobs in the build to pass in order for the status to be 'passing'
Closes #1792
2018-07-30 17:14:08 +01:00