diff --git a/doc/rewriting-services.md b/doc/rewriting-services.md new file mode 100644 index 0000000000..1046d8e059 --- /dev/null +++ b/doc/rewriting-services.md @@ -0,0 +1,239 @@ +# Tips for rewriting legacy services + +## First, write some tests + +If service tests don’t exist for the legacy service, stop and write them first. +It’s recommended to PR these separately. If there’s some test coverage, it’s +probably fine to move right ahead and add more in the process. Make sure the +tests are passing, though. + +## Organization + +1. When there’s a single legacy service that handles lots of different things +(e.g. version, license, and downloads), it should be split into three separate +service classes and placed in three separate files, e.g.: + +- `example-version.service.js` +- `example-license.service.js` +- `example-downloads.service.js` + +2. When a badge offers different variants of basically the same thing, it’s okay +to put them in the same service class. For example, daily/weekly/monthly/total +downloads can go in one badge, and star rating vs point rating vs rating count +can go in one badge, and same with various kinds of detail about a pull request. +The hard limit (as of now anyway) is *one category per service class*. + +3. If the tests haven’t been split up, split them up too and make sure they +still pass. + +## Get the route working + +1. Disable the legacy service by adding a `return` at the top of +`registerLegacyRouteHandler()`. + +2. Set up the route for one of the badges. First determine if you can express +the route using a `pattern`. A `pattern` (e.g. `pattern: ':param1/:param2'`) is +the simplest way to declare the route, also the most readable, and will be +useful for displaying a badge builder with fields in the front end and +generating badge URLs programmatically. One limitation to keep in mind is that, +at present, the trailing parameter of a pattern can't be optional. If the last +part of a route is optional, like a branch, you will need to use a `format` +regex string (e.g. `format: '([^/]+/[^/]+)'`). + +3. When creating the initial route, you can stub out the service. A minimal +service extends BaseJsonService (or BaseService, or one of the others), and +defines `route()` and `handle()`. `defaultBadgeData` is optional but suggested: + +```js +const BaseJsonService = require('../base-json') + +class ExampleDownloads extends BaseJsonService { + static get route() { + return { + base: 'example/d', + pattern: ':param1/:param2', + } + } + + static defaultBadgeData() { + return { label: 'downloads' } // or whatever + } + + async handle({ param1, param2 }) { + return { message: 'hello' } + } +} +``` + +4. We don’t have really good tools for debugging matches, so the best you can do +is run a subset of your tests. To run a single service test, add `.only()` +somewhere in the chain, and run `npm run test:services:trace — —only=example`. + +```js +t.create('build status') + .get('/pip.json') + .only() // Prevent this ServiceTester from running its other tests. + .expectJSONTypes( + Joi.object().keys({ + name: 'docs', + value: Joi.alternatives().try(isBuildStatus, Joi.equal('unknown')), + }) + ) +``` + +5. Presumably the test will fail, though by examining the copious output, you +can confirm the route was matched and the named parameters mapped successfully. +Since you'll have just run the tests on the old code (right?) you'll know you +haven't inadvertently changed the route (an easy mistake to make). + +6. If the legacy service had a base URL and you've changed it, you’ll need to +update the tests _and_ the examples. Take care to do that. + +## Implement `render()` and `handle()` + +Once the route is working, fill out `render()` and `handle()`. + +1. If there’s a single service, you can implement fetch as a method or a +function at the top of the file. If there are more than one service which share +fetching code, you can put the fetch function in `example-common.js` like this +one for github: + +
+ +```js +const Joi = require('joi') +const { errorMessagesFor } = require('./github-helpers') + +const issueSchema = Joi.object({ + head: Joi.object({ + sha: Joi.string().required(), + }).required(), +}).required() + +async function fetchIssue(serviceInstance, { user, repo, number }) { + return serviceInstance._requestJson({ + schema: issueSchema, + url: `/repos/${user}/${repo}/pulls/${number}`, + errorMessages: errorMessagesFor('pull request or repo not found'), + }) +} + +module.exports = { + fetchIssue, +} +``` + +
+ +or create an abstract superclass like **PypiBase**: + +
+ +```js +const Joi = require('joi') +const BaseJsonService = require('../base-json') + +const schema = Joi.object({ + info: Joi.object({ + ... + }).required() +}).required() + +module.exports = class PypiBase extends BaseJsonService { + static buildRoute(base) { + return { + base, + pattern: ':egg*', + } + } + + async fetch({ egg }) { + return this._requestJson({ + schema, + url: `https://pypi.org/pypi/${egg}/json`, + errorMessages: { 404: 'package or version not found' }, + }) + } +} +``` + +
+ +2. Validation should be handled using Joi. Save this for last. While you're +getting things working, you can use `const schema = Joi.any()`, which matches +anything. + +3. Substitution of default values should also be handled by Joi, using +`.default()`. + +5. To keep with the design pattern of `render()`, formatting concerns, including +concatenation and color computation, should be dealt with inside `render()`. +This helps avoid static examples falling out of sync with the implementation. + +## Error handling + +BaseService includes built-in runtime error handling. Error classes are defined +in `services/errors.js`. Request code and validation code will throw a runtime +error, which will then bubble up to BaseService, which then renders an error +badge. The cases covered by built-in error handling need not be tested in each +service, and existing tests should be removed. + +1. If an external server can't be reached or returns a 5xx status code, +`_requestJson()` along with code in `lib/error-helper.js` will bubble up an +**Inaccessible** error. + +2. If a response does not match the schema, `validate()` will bubble up an +**InvalidResponse** error which will display **invalid response data**. + +Error handling can also be customized by the service. Alternate messages +corresponding to HTTP status codes can be specified in the `errorMessages` +parameter to `_requestJson()` etc. + +For the not found case, a service test should establish that the API is doing +what we expect. If the API returns a 404 error, code in `lib/error-helper.js` +will automatically throw a **NotFound** error. The error message can, and +generally should be customized to display something more specific like +**package not found** or **room not found**. + +Not all services return a 404 response in the not found case. Sometimes a +different status code is returned. + +Sometimes a 200 response must be examined to distinguish the not found case from a success case. This can be handled in either of two ways: + +- Write a schema which accommodates both the success and error cases. +- Write the schema for the success case. Pass `schema: Joi.any()` to + `_requestJson()`. Manually check for the error case, then invoke + `_validate()` with the success-case schema. + +In either case, the service should throw e.g +`new NotFound({ prettyMessage: 'package not found' })`. + +## Convert the examples + +1. Convert all the examples to `pattern`, `namedParams`, and `staticExample`. In some cases you can use the `pattern` inherited from `route`, though in other cases you may need to specify a pattern in the example. For example, when showing download badges for several periods, you may want to render the example with an explicit `dt` instead of `:which`. You will also need to specify a pattern for badges that use a `format` regex in the route. + +2. Open the frontend and check that the static preview badges look good. +Remember, none of them are live. + +3. Open up the prepared example URLs in their own tabs, and make sure they work correctly. + +## Validation + +When it's time to add the schema, refer to the Joi API docs: +https://github.com/hapijs/joi/blob/master/API.md + +## Housekeeping + +Switch to `createServiceTester`: + +```js +const t = require('../create-service-tester')() +``` + +This may require updating the URLs, which will be relative to the service's base +URL. When using `createServiceTester`, services need to be specified using +the non-case-sensitive service class name, or a trailing substring (e.g. +`AppveyorTests` or `appveyor`). + +Do this last. Since it involves changing test URLs, and you don't want to +accidentally change them.