Review bot? #82

Closed
opened 2025-11-06 15:05:29 -06:00 by GiteaMirror · 14 comments
Owner

Originally created by @egeerardyn on GitHub (Jun 9, 2016).

After seeing https://github.com/sindresorhus/awesome/pull/682#issuecomment-224618088 (and quite a bit of similar PRs ), I think the only solution we have left to reduce people from inconsiderately submitting a PR, is by automating a part of the review process.

In principle, it should not be hard to write a small bot that runs e.g. on Travis CI to check:

  • that all commit titles in a PR are not just "Update readme.md",
  • that the repository is added at the bottom of a category,
  • that a new repository:
    • is e.g. older than 2 weeks,
    • contains the awesome badge,
    • contains a "CONTRIBUTING.md", and
    • contains a "LICENSE.md" or similar file.

Those are all mechanic checks that such a bot could carry out instead of a human being. Probably we can make this bot even react to the GitHub PR with an error message and even close the PR: personally, I would be quite ruthless when it comes to handling such inconsiderate PRs: if it is too hard to read some instructions, I am pretty sure it is too hard to maintain an awesome list.

The danger of this idea are false positives: people that are of good faith who get their PR closed because of this.

Originally created by @egeerardyn on GitHub (Jun 9, 2016). After seeing https://github.com/sindresorhus/awesome/pull/682#issuecomment-224618088 (and quite a bit of similar PRs ), I think the only solution we have left to reduce people from inconsiderately submitting a PR, is by automating a part of the review process. In principle, it should not be hard to write a small bot that runs e.g. on Travis CI to check: - that all commit titles in a PR are not just "Update readme.md", - that the repository is added at the bottom of a category, - that a new repository: - is e.g. older than 2 weeks, - contains the awesome badge, - contains a "CONTRIBUTING.md", and - contains a "LICENSE.md" or similar file. Those are all mechanic checks that such a bot could carry out instead of a human being. Probably we can make this bot even react to the GitHub PR with an error message and even close the PR: personally, I would be quite ruthless when it comes to handling such inconsiderate PRs: if it is too hard to read some instructions, I am pretty sure it is too hard to maintain an awesome list. The danger of this idea are false positives: people that are of good faith who get their PR closed because of this.
Author
Owner

@JesseTG commented on GitHub (Jun 9, 2016):

Then they can reopen their PRs.

@JesseTG commented on GitHub (Jun 9, 2016): Then they can reopen their PRs.
Author
Owner

@sindresorhus commented on GitHub (Jun 9, 2016):

Would be nice, though I'm sure people will find many other ingenious ways to do it wrong that the bot won't catch.

@sindresorhus commented on GitHub (Jun 9, 2016): Would be nice, though I'm sure people will find many other ingenious ways to do it wrong that the bot won't catch.
Author
Owner

@egeerardyn commented on GitHub (Jun 9, 2016):

@sindresorhus I am sure that people will find both

  • ways around such a bot (I can think of a few myself), and
  • ways to mangle their list such that a bot cannot decide whether a list is good or bad.

However, I don't think that would be a problem. I see such a bot as a spam filter. Even if it doesn't get rid of 100% of the problem, it still is an improvement: it could take over a few simple repetitive tasks such that we free some time and energy for the human review process.

@egeerardyn commented on GitHub (Jun 9, 2016): @sindresorhus I am sure that people will find both - ways around such a bot (I can think of a few myself), and - ways to mangle their list such that a bot cannot decide whether a list is good or bad. However, I don't think that would be a problem. I see such a bot as a spam filter. Even if it doesn't get rid of 100% of the problem, it still is an improvement: it could take over a few simple repetitive tasks such that we free some time and energy for the human review process.
Author
Owner

@JesseTG commented on GitHub (Jun 10, 2016):

ways to mangle their list such that a bot cannot decide whether a list is good or bad.

Then the list is probably too mangled to be useful for awesome anyway. :)

ways around such a bot (I can think of a few myself)

Cool, let's hear them.

@JesseTG commented on GitHub (Jun 10, 2016): > ways to mangle their list such that a bot cannot decide whether a list is good or bad. Then the list is probably too mangled to be useful for `awesome` anyway. :) > ways around such a bot (I can think of a few myself) Cool, let's hear them.
Author
Owner

@sindresorhus commented on GitHub (Jun 10, 2016):

Don't get me wrong though. I'm all for a good bot.

@sindresorhus commented on GitHub (Jun 10, 2016): Don't get me wrong though. I'm all for a good bot.
Author
Owner

@sobolevn commented on GitHub (Jun 23, 2016):

Any progress yet? I am very interested also. And can help for sure.

@sobolevn commented on GitHub (Jun 23, 2016): Any progress yet? I am very interested also. And can help for sure.
Author
Owner

@egeerardyn commented on GitHub (Jun 23, 2016):

@sobolevn Haven't started yet: I currently have bigger priorities for work.

@egeerardyn commented on GitHub (Jun 23, 2016): @sobolevn Haven't started yet: I currently have bigger priorities for work.
Author
Owner

@jacc commented on GitHub (Nov 24, 2017):

I feel extremely bad for necroposting on a issue over a year old, but opening another issue didn't seem practical. I could look into creating a CLI app that the users run before contributing to check if it has the correct files / badges, etc.

@jacc commented on GitHub (Nov 24, 2017): I feel extremely bad for necroposting on a issue over a year old, but opening another issue didn't seem practical. I could look into creating a CLI app that the users run before contributing to check if it has the correct files / badges, etc.
Author
Owner

@ntwb commented on GitHub (Dec 3, 2017):

Sounds like an app for https://probot.github.io/ would be ideal, could be used by all awesome repos

@ntwb commented on GitHub (Dec 3, 2017): Sounds like an app for https://probot.github.io/ would be ideal, could be used by _all_ awesome repos
Author
Owner

@jacc commented on GitHub (Dec 3, 2017):

@ntwb I could definately work on something like that. Probot's API seems super simple. All I have to do is just get the repo information to see if it has the Awesome badge in it.

@jacc commented on GitHub (Dec 3, 2017): @ntwb I could definately work on something like that. Probot's API seems super simple. All I have to do is just get the repo information to see if it has the Awesome badge in it.
Author
Owner

@jacc commented on GitHub (Dec 3, 2017):

@ntwb @sindresorhus I've started development of the bot at https://github.com/jacc/awesome-bot. Haven't committed in a while but I will soon with the new features.

@jacc commented on GitHub (Dec 3, 2017): @ntwb @sindresorhus I've started development of the bot at https://github.com/jacc/awesome-bot. Haven't committed in a while but I will soon with the new features.
Author
Owner

@ntwb commented on GitHub (Dec 4, 2017):

Great @jacc 👍


As an FYI, I came across this issue in my search for an issue that was looking to standarize pull requests, my thought was and is that if PRs were standardized across awesome repos then it would make my life easier as a watcher of ~12 awesome repos.

Primarily I'd love to see a consistent format of URLs in PRs so when I see a new notification in my email the link is easily clickable so that I can checkout the latest link being added to a repo without having to juggle copy and paste, one click, done 😬

@ntwb commented on GitHub (Dec 4, 2017): Great @jacc 👍 ---- As an FYI, I came across this issue in my search for an issue that was looking to _standarize_ pull requests, my thought was and is that if PRs were standardized across _awesome repos_ then it would make my life easier as a _watcher_ of ~12 awesome repos. Primarily I'd love to see a consistent format of URLs in PRs so when I see a new notification in my email the link is easily clickable so that I can checkout the _latest_ link being added to a repo without having to juggle copy and paste, one click, done 😬
Author
Owner

@fkromer commented on GitHub (Oct 8, 2018):

There is also Danger to validate PRs with awesome_bot.

@fkromer commented on GitHub (Oct 8, 2018): There is also [`Danger` to validate PRs with `awesome_bot`](https://github.com/dkhamsing/awesome_bot#danger).
Author
Owner

@sindresorhus commented on GitHub (Feb 26, 2019):

Use https://github.com/sindresorhus/awesome-lint in your Awesome lists.

We plan to add it here too: https://github.com/sindresorhus/awesome/pull/1394

@sindresorhus commented on GitHub (Feb 26, 2019): Use https://github.com/sindresorhus/awesome-lint in your Awesome lists. We plan to add it here too: https://github.com/sindresorhus/awesome/pull/1394
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/awesome-sindresorhus#82