Convention for monorepos #57

Closed
opened 2026-02-17 11:40:11 -06:00 by GiteaMirror · 16 comments
Owner

Originally created by @tunnckoCore on GitHub (Jan 25, 2019).

Heya.

I just ended up with a need to define a list of "changed packages".

https://github.com/tunnckoCoreLabs/parse-commit-message/issues/56

I'm opening this so we can come up with some convention together.

For now I'm thinking about

feat: foo bar baz

The changed packages will be collected through `jest-changed-files`
and added as a list below (including the workspace directory name
in front of the changed package name).

packages:
- packages/recommended-bump
- @tunnckocore/execa
- packages/git-commits-since

Signed-off-by: Charlike Mike Reagent

Based on that commit, later I can extract the package names and bump only their versions to their next minor.

Originally created by @tunnckoCore on GitHub (Jan 25, 2019). Heya. I just ended up with a need to define a list of "changed packages". https://github.com/tunnckoCoreLabs/parse-commit-message/issues/56 I'm opening this so we can come up with some convention together. For now I'm thinking about ``` feat: foo bar baz The changed packages will be collected through `jest-changed-files` and added as a list below (including the workspace directory name in front of the changed package name). packages: - packages/recommended-bump - @tunnckocore/execa - packages/git-commits-since Signed-off-by: Charlike Mike Reagent ``` Based on that commit, later I can extract the package names and bump only their versions to their next `minor`.
Author
Owner

@damianopetrungaro commented on GitHub (Jan 26, 2019):

Mmm di you want to communicate a change in different packages?

N different commits will work if this is your need :)

@damianopetrungaro commented on GitHub (Jan 26, 2019): Mmm di you want to communicate a change in different packages? N different commits will work if this is your need :)
Author
Owner

@tunnckoCore commented on GitHub (Jan 26, 2019):

Usually it will. In monorepo setups probably not always. You may need one commit which changes few packages.

It's not for me only, i'm building app/tool that will be used by other devs.

@tunnckoCore commented on GitHub (Jan 26, 2019): Usually it will. In monorepo setups probably not always. You may need one commit which changes few packages. It's not for me only, i'm building app/tool that will be used by other devs.
Author
Owner

@hutson commented on GitHub (Feb 10, 2019):

A similar question came up within the company I work for with respect to mono-repositories.

I've been trying to put together use cases where commit messages need to include package information, but I haven't come up with any yet.

The closest will be a mono-repository where I will house a library package right next to a CLI package, which consumes the library.

If I modify a single package, do I need to indicate in my commit message which package was modified? @tunnckoCore brings up the need to know which package was modified so that the package can have its version updated. The ability to update versions in individual packages in a mono-repository is definitely a requirement of mine too.

However, the mono-repository tool Lerna, along with the conventional-changelog packages, are able to extract commits that only touched files within a given package. That's how Lerna knows if it needs to create a new release for a package.

@tunnckoCore could you leverage git to retrieve only commits that touch a file path within a particular package, and then scan those commits to see if the package needs to be updated with a new version?

Another scenario that came to mind was what happens when I need to release a feature in the library, and also expose it in the CLI? In that particular scenario, does it make sense that submitting a single commit for two packages would mean I would be using the same scope for both?

@hutson commented on GitHub (Feb 10, 2019): A similar question came up within the company I work for with respect to mono-repositories. I've been trying to put together use cases where commit messages need to include package information, but I haven't come up with any yet. The closest will be a mono-repository where I will house a library package right next to a CLI package, which consumes the library. If I modify a single package, do I need to indicate in my commit message which package was modified? @tunnckoCore brings up the need to know which package was modified so that the package can have its version updated. The ability to update versions in individual packages in a mono-repository is definitely a requirement of mine too. However, the mono-repository tool Lerna, along with the `conventional-changelog` packages, are able to extract commits that only touched files within a given package. That's how Lerna knows if it needs to create a new release for a package. @tunnckoCore could you leverage `git` to retrieve only commits that touch a file path within a particular package, and then scan those commits to see if the package needs to be updated with a new version? Another scenario that came to mind was what happens when I need to release a feature in the library, and also expose it in the CLI? In that particular scenario, does it make sense that submitting a single commit for two packages would mean I would be using the same `scope` for both?
Author
Owner

@tunnckoCore commented on GitHub (Feb 10, 2019):

If I modify a single package, do I need to indicate in my commit message which package was modified?

In that case, no you won't need.

The closest will be a mono-repository where I will house a library package right next to a CLI package, which consumes the library.

This also is an easy case, and probably a candidate for synced versioning.

My use case is that I want to move all my packages scoped and non-scoped ones to a single repository, independently versioned. Every package on my profile will be on the packages/ folder on the root. And there also will be folders on the root for each scope, for example @tunnckocore and @hela - https://github.com/tunnckoCore/monorepo. Also somewhere heard about nested monorepos, for me, meaning that @hela could be synced versioning monorepo, inside the root. 🤔 Crazy huh? Everything in packages/ and @tunnckocore would be independently versioned, but packages inside @hela folder would be sync-versioned. 😱

I kinda trying Lerna and still learning it. And from what I seeing I can't trust it, because I don't feel it's bumping versions correctly. For some reason, it bumps versions of some packages that are not touched and even not used by the other (changed) ones. I should try from-package too. Still playing.

I already built an ecosystem of tools around the automation of publishing and releasing, and this proposed convention will probably allow me to drop Lerna, not sure yet.

would be using the same scope for both?

Yea, I thought for that too. It's not that great but is an acceptable compromise. 🤔

I came to that proposal after I was thinking that we can put change package names on the scope, but that would be ugly for > 1 changed packages :D So, better would be listing them on the commit message.

This discussion/proposal is only for the syntax, not the way how changed packages will be collected and listed in the commit message - one may use lerna changed, other something similar to jest-changed-files. Jest is working great in monorepo environments, so it could be used for everything https://twitter.com/tunnckoCore/status/1091478498831667205.

@tunnckoCore commented on GitHub (Feb 10, 2019): > If I modify a single package, do I need to indicate in my commit message which package was modified? In that case, no you won't need. > The closest will be a mono-repository where I will house a library package right next to a CLI package, which consumes the library. This also is an easy case, and probably a candidate for synced versioning. My use case is that I want to move all my packages scoped and non-scoped ones to a single repository, independently versioned. Every package on my profile will be on the `packages/` folder on the root. And there also will be folders on the root for each scope, for example `@tunnckocore` and `@hela` - https://github.com/tunnckoCore/monorepo. Also somewhere heard about nested monorepos, for me, meaning that `@hela` could be synced versioning monorepo, inside the root. :thinking: Crazy huh? Everything in `packages/` and `@tunnckocore` would be independently versioned, but packages inside `@hela` folder would be sync-versioned. :scream: I kinda trying Lerna and still learning it. And from what I seeing I can't trust it, because I don't feel it's bumping versions correctly. For some reason, it bumps versions of some packages that are not touched and even not used by the other (changed) ones. I should try `from-package` too. Still playing. I already built an ecosystem of tools around the automation of publishing and releasing, and this proposed convention will probably allow me to drop Lerna, not sure yet. > would be using the same `scope` for both? Yea, I thought for that too. It's not that great but is an acceptable compromise. :thinking: I came to that proposal after I was thinking that we can put change package names on the scope, but that would be ugly for > 1 changed packages :D So, better would be listing them on the commit message. This discussion/proposal is only for the syntax, not the way how changed packages will be collected and listed in the commit message - one may use `lerna changed`, other something similar to `jest-changed-files`. Jest is working great in monorepo environments, so it could be used for everything https://twitter.com/tunnckoCore/status/1091478498831667205.
Author
Owner

@bcoe commented on GitHub (Feb 18, 2019):

👋 hey @tunnckoCore, I put some work a while back to adding conventional commit support to lerna. Like @hutson indicates, it uses the path of the git message to essentially maintain multiple conventional commit logs in the same repo -- it works okay, but I don't think the approach works as well as regular git histories.

I'm not sure if we'd want to extend this spec to address a mono-repo use-case, my concern is it might add a lot of bullet points to the spec for a methodology only a portion of users apply... what I think might be cool, would be adding an additional section to the conventionalcommits.org website for "guides", which would touch on specific topics like mono-repos.

☝️ would love help from the community getting this off the ground.

I kinda trying Lerna and still learning it. And from what I seeing I can't trust it, because I don't feel it's bumping versions correctly. For some reason, it bumps versions of some packages that are not touched and even not used by the other (changed) ones. I should try from-package too. Still playing.

I think we should loop @evocateur in on issues like this, it feels like (conventional commit convention aside) there's significant room for improvement in how lerna handles conventional commit messages:

  • the CHANGELOG ends up having more cruft in it than non-mono-repos, because there are more point releases with no commit messages.
  • if you're right that some package versions are bumped that shouldn't be, we should dig into this.
@bcoe commented on GitHub (Feb 18, 2019): 👋 hey @tunnckoCore, I put some work a while back to adding conventional commit support to [lerna](https://github.com/lerna/lerna/blob/030de9d6285f2b364ed1460b9534bc2190acdf7a/commands/version/README.md#--conventional-commits). Like @hutson indicates, it uses the path of the git message to essentially maintain multiple conventional commit logs in the same repo -- it works okay, but I don't think the approach works as well as regular git histories. I'm not sure if we'd want to extend this spec to address a mono-repo use-case, my concern is it might add a lot of bullet points to the spec for a methodology only a portion of users apply... what I think might be cool, would be adding an additional section to the conventionalcommits.org website for "guides", which would touch on specific topics like mono-repos. ☝️ _would love help from the community getting this off the ground._ > I kinda trying Lerna and still learning it. And from what I seeing I can't trust it, because I don't feel it's bumping versions correctly. For some reason, it bumps versions of some packages that are not touched and even not used by the other (changed) ones. I should try `from-package` too. Still playing. I think we should loop @evocateur in on issues like this, it feels like (conventional commit convention aside) there's significant room for improvement in how lerna handles conventional commit messages: * the CHANGELOG ends up having more cruft in it than non-mono-repos, because there are more point releases with no commit messages. * if you're right that some package versions are bumped that shouldn't be, we should dig into this.
Author
Owner

@tunnckoCore commented on GitHub (Feb 18, 2019):

Heya 👋

The thing is that I proposing it, not because of Lerna, but because I need some sign in the commit message side, so I can extract it through the GitHub API so to be used in a GitHub App.

Even if nothing is done there and I and others use Lerna only for publishing, we will still need to have some sign for the GitHub App so to make it work with mono-repos.

What is just popped in my mind is that we actually have a list of changed/published packages inside a commit message when Lerna push the releases/tags and the e.g. chore: release message. So I might be able to allow users of the App to configure the App to understand mono-repo setups. So the app instead of listening on each commit, it will listen only on those chore: release commits and get the changed packages and their versions from there, and so get the commits between those "release" commits.

In any way, I will probably go the other path: just don't support mono-repo setups on the App side, because Lerna is doing great and recently added support for creating GitHub releases (not only tags). Also, even if it hasn't support for creating releases, it not make much sense to me to have GitHub Releases on a monorepo, it's a lot harder to follow, so CHANGELOG is best fit - the end user of a package in anyway is just opening the package root so he will see the CHANGELOG for that exact package, won't need to search through the Releases (btw, GitHub should add a search feature there).

Thanks anyway for the great discussion. It seems like when some time pass man is clearing his mind and everything is more easy. 🙏

@tunnckoCore commented on GitHub (Feb 18, 2019): Heya :wave: The thing is that I proposing it, not because of Lerna, but because I need some sign in the commit message side, so I can extract it through the GitHub API so to be used in a GitHub App. Even if nothing is done there and I and others use Lerna only for publishing, we will still need to have some sign for the GitHub App so to make it work with mono-repos. What is just popped in my mind is that we actually have a list of changed/published packages inside a commit message when Lerna push the releases/tags and the e.g. `chore: release` message. So I might be able to allow users of the App to configure the App to understand mono-repo setups. So the app instead of listening on each commit, it will listen only on those `chore: release` commits and get the changed packages and their versions from there, and so get the commits between those "release" commits. In any way, I will probably go the other path: just don't support mono-repo setups on the App side, because Lerna is doing great and recently added support for creating GitHub releases (not only tags). Also, even if it hasn't support for creating releases, it not make much sense to me to have GitHub Releases on a monorepo, it's a lot harder to follow, so CHANGELOG is best fit - the end user of a package in anyway is just opening the package root so he will see the CHANGELOG for that exact package, won't need to search through the Releases (btw, GitHub should add a search feature there). Thanks anyway for the great discussion. It seems like when some time pass man is clearing his mind and everything is more easy. :pray:
Author
Owner

@evocateur commented on GitHub (Feb 18, 2019):

I kinda trying Lerna and still learning it. And from what I seeing I can't trust it, because I don't feel it's bumping versions correctly. For some reason, it bumps versions of some packages that are not touched and even not used by the other (changed) ones.

lerna/lerna#707 goes into great detail why you shouldn't care about Lerna bumping transitive dependents of a changed module. If there is no relationship at all, it means your tags are getting mangled and Lerna defaults to publishing everything when it can't find the last annotated release tag (convoluted "release" branches are often the culprit in these cases).

@evocateur commented on GitHub (Feb 18, 2019): > I kinda trying Lerna and still learning it. And from what I seeing I can't trust it, because I don't feel it's bumping versions correctly. For some reason, it bumps versions of some packages that are not touched and even not used by the other (changed) ones. lerna/lerna#707 goes into great detail why you shouldn't care about Lerna bumping transitive dependents of a changed module. If there _is_ no relationship at all, it means your tags are getting mangled and Lerna defaults to publishing everything when it can't find the last annotated release tag (convoluted "release" branches are often the culprit in these cases).
Author
Owner

@ntwb commented on GitHub (Feb 19, 2019):

* the CHANGELOG ends up having more cruft in it than non-mono-repos, because there are more point releases with no commit messages.

@bcoe, so currently a single changelog file?

I'm hoping to be able to have a changelog for each Lerna package, and using scopes in some way to determine which changelog the commit is related to 🤔

@ntwb commented on GitHub (Feb 19, 2019): > * the CHANGELOG ends up having more cruft in it than non-mono-repos, because there are more point releases with no commit messages. @bcoe, so currently a single _changelog_ file? I'm hoping to be able to have a _changelog_ for each Lerna _package_, and using _scopes_ in some way to determine which _changelog_ the commit is related to 🤔
Author
Owner

@stevemao commented on GitHub (Feb 19, 2019):

(I haven't read the whole conversion here)
When I worked at Atlassian we used affects: <package-1>, <package-2>
See 06e18185c6

Then lerna-semantic-release can parse affected packages and bump them only.

cc @jpnelson

@stevemao commented on GitHub (Feb 19, 2019): (I haven't read the whole conversion here) When I worked at Atlassian we used `affects: <package-1>, <package-2>` See https://github.com/atlassian/lerna-semantic-release/commit/06e18185c6477085b7046248e7dd7eb59c06e0c9 Then lerna-semantic-release can parse affected packages and bump them only. cc @jpnelson
Author
Owner

@tunnckoCore commented on GitHub (Feb 19, 2019):

@evocateur thanks, that's what I want, I'll check out the thread to see why such a cool flag is removed at all.

@tunnckoCore commented on GitHub (Feb 19, 2019): @evocateur thanks, that's what I want, I'll check out the thread to see why such a cool flag is removed at all.
Author
Owner

@ntwb commented on GitHub (Feb 19, 2019):

p.s. Currently @WordPress is looking at a custom solution using a Git pre-commit hook

https://github.com/WordPress/gutenberg/pull/13594

@ntwb commented on GitHub (Feb 19, 2019): p.s. Currently @WordPress is looking at a custom solution using a Git pre-commit hook https://github.com/WordPress/gutenberg/pull/13594
Author
Owner

@bcoe commented on GitHub (Feb 19, 2019):

@bcoe, so currently a single changelog file?

@ntwb there is a CHANGELOG generated for each of the libraries in the packages/ folder. By cruft, I was referring to the fact that you end up with quite a few releases that are generated by the fact that a dependent package was updated. bar packages CHANGELOG will be updated with an entry, due to a change on its dependent package foo. It looks like this is currently handled in lerna by entries that look like the following:

<a name="2.0.2"></a>
## [2.0.2](https://github.com/istanbuljs/istanbuljs/compare/istanbul-lib-coverage@2.0.1...istanbul-lib-coverage@2.0.2) (2018-12-25)

**Note:** Version bump only for package istanbul-lib-coverage

☝️ this seem pretty reasonable, I believe the Note: was added since I last looked at this functionality.

When I worked at Atlassian we used affects: <package-1>, <package-2>

@stevemao/@tunnckoCore rather than extending on the commit messages themselves, lerna modifies the format of the generated tags:

istanbul-api@2.0.6
istanbul-api@2.0.7
istanbul-api@2.0.8
istanbul-lib-coverage@1.0.1
istanbul-lib-coverage@1.0.2
istanbul-lib-coverage@1.1.0
istanbul-lib-coverage@1.1.1
istanbul-lib-coverage@1.1.2

By prefixing them with the library that the tag was generated for ... this doesn't seem like a terrible approach, a lot of folks I believe are converging on a packages/project, packages/project2, packages/project3 folder layout for mono-repos (at least in JavaScript land).

@bcoe commented on GitHub (Feb 19, 2019): > @bcoe, so currently a single _changelog_ file? @ntwb there is a CHANGELOG generated for each of the libraries in the `packages/` folder. By cruft, I was referring to the fact that you end up with quite a few releases that are generated by the fact that a dependent package was updated. `bar` packages CHANGELOG will be updated with an entry, due to a change on its dependent package `foo`. It looks like this is currently handled in lerna by entries that look like the following: ```md <a name="2.0.2"></a> ## [2.0.2](https://github.com/istanbuljs/istanbuljs/compare/istanbul-lib-coverage@2.0.1...istanbul-lib-coverage@2.0.2) (2018-12-25) **Note:** Version bump only for package istanbul-lib-coverage ``` ☝️ this seem pretty reasonable, I believe the `Note:` was added since I last looked at this functionality. > When I worked at Atlassian we used `affects: <package-1>, <package-2>` @stevemao/@tunnckoCore rather than extending on the commit messages themselves, lerna modifies the format of the generated tags: ```shell istanbul-api@2.0.6 istanbul-api@2.0.7 istanbul-api@2.0.8 istanbul-lib-coverage@1.0.1 istanbul-lib-coverage@1.0.2 istanbul-lib-coverage@1.1.0 istanbul-lib-coverage@1.1.1 istanbul-lib-coverage@1.1.2 ``` By prefixing them with the library that the tag was generated for ... this doesn't seem like a terrible approach, a lot of folks I believe are converging on a `packages/project, packages/project2, packages/project3` folder layout for mono-repos (at least in JavaScript land).
Author
Owner

@evocateur commented on GitHub (Feb 20, 2019):

**Note:** Version bump only for package ${PACKAGE_NAME}

Not a huge fan of this personally, but it's the best we've got at the moment. I'd rather provide a rollup of what dependencies got bumped so it'd be a little more expressive. It would require some tricky refactoring of the --conventional-commits code in Lerna, however, and it's low on my priority list.

@evocateur commented on GitHub (Feb 20, 2019): > `**Note:** Version bump only for package ${PACKAGE_NAME}` Not a huge fan of this personally, but it's the best we've got at the moment. I'd rather provide a rollup of _what_ dependencies got bumped so it'd be a little more expressive. It would require some tricky refactoring of the `--conventional-commits` code in Lerna, however, and it's low on my priority list.
Author
Owner

@jasonkuhrt commented on GitHub (Jun 30, 2019):

fwiw we independently came to use the same tagging style as lerna and its working well for us.

@jasonkuhrt commented on GitHub (Jun 30, 2019): fwiw we independently came to use the same tagging style as lerna and its working well for us.
Author
Owner

@damianopetrungaro commented on GitHub (Jul 4, 2019):

@tunnckoCore feel free to re-open it if you still want to discuss it.

@damianopetrungaro commented on GitHub (Jul 4, 2019): @tunnckoCore feel free to re-open it if you still want to discuss it.
Author
Owner

@gaetansenn commented on GitHub (Mar 30, 2023):

Why not using the .versionrc scope option from types ?

Assume you want to filter by "ui" scope

{
  "types": [
    {"type": "feat", "section": "Features", "scope": "ui" },
    {"type": "fix", "section": "Bug Fixes", "scope": "ui" },
    {"type": "chore", "hidden": true, "scope": "ui" },
    {"type": "style", "hidden": true, "scope": "ui" },
    {"type": "refactor", "hidden": true, "scope": "ui"},
    {"type": "perf", "hidden": true, "scope": "ui"},
    {"type": "test", "hidden": true, "scope": "ui"}
  ]
}
@gaetansenn commented on GitHub (Mar 30, 2023): Why not using the .versionrc scope option from types ? Assume you want to filter by "ui" scope ``` { "types": [ {"type": "feat", "section": "Features", "scope": "ui" }, {"type": "fix", "section": "Bug Fixes", "scope": "ui" }, {"type": "chore", "hidden": true, "scope": "ui" }, {"type": "style", "hidden": true, "scope": "ui" }, {"type": "refactor", "hidden": true, "scope": "ui"}, {"type": "perf", "hidden": true, "scope": "ui"}, {"type": "test", "hidden": true, "scope": "ui"} ] } ```
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/conventionalcommits.org#57