github-starred/n8n
2
0
Fork 0
mirror of https://github.com/n8n-io/n8n.git synced 2025-12-05 19:27:26 -06:00
Code Issues 1.3k Packages Projects Releases 100 Wiki Activity

[PR #22058] feat(Aggregate Node): add group by functionality to aggregate node #22019

New Issue
Open
opened 2025-11-20 06:22:48 -06:00 by GiteaMirror · 0 comments
GiteaMirror commented 2025-11-20 06:22:48 -06:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/n8n-io/n8n/pull/22058
Author: @pemontto
Created: 11/19/2025
Status: 🔄 Open

Base: master ← Head: aggregate-group-by

📝 Commits (2)

  • c6ea2bf feat(Aggregate Node): add group by functionality to aggregate node
  • a93b338 feat(Aggregate): improve conflict detection for group by fields

📊 Changes

3 files changed (+1164 additions, -15 deletions)

View changed files

📝 packages/nodes-base/nodes/Transform/Aggregate/Aggregate.node.ts (+288 -15)
➕ packages/nodes-base/nodes/Transform/Aggregate/test/Aggregate.groupby.test.ts (+578 -0)
➕ packages/nodes-base/nodes/Transform/Aggregate/test/workflow.groupby.json (+298 -0)

📄 Description

Summary

Group By and Aggregate go together like peas and carrots 🥕 - Forrest Gump (probably)

This PR adds Group By support to the Aggregate node, enabling users to group items by field values before aggregating. This is a fundamental data manipulation pattern that brings n8n closer to standards like SQL GROUP BY, Pandas groupby(), or Excel Pivot Tables.

Features

✅ Group by fields: Split data into groups based on one or multiple fields (comma-separated).
✅ Flexible Modes: Supports both "Individual Fields" and "All Item Data" aggregation modes within the groups.
✅ Binary Support: Full support for including/grouping binary data.

Why Add This?

Group By is essential for structured data aggregation. It allows users to move from a global aggregation to specific buckets.

Before (One aggregation across all items):

// Output: 1 item
{
  emails: ["user1@yahoo.com", "user2@gmail.com", "user3@yahoo.com", "user4@hotmail.com"]
}

After (Group By 'domain'):

// Output: 3 items
[
  { domain: "yahoo.com", emails: ["user1@yahoo.com", "user3@yahoo.com"] },
  { domain: "gmail.com", emails: ["user2@gmail.com"] },
  { domain: "hotmail.com", emails: ["user4@hotmail.com"] }
]

Outstanding Questions

Item Linking (pairedItem): Initially, I mapped all constituent input items to the pairedItem array for the resulting group (which seemed semantically correct). However, I didn't observe any functional difference in the UI compared to just mapping the first item. I found just using the first item at least gives me a functional way to trace back to an originating item, i.e. allowing $('previous node').item to work.

Mapping all related items as an array would return the same as if no mapping took place:
image

Sample Workflow

image
Click to expand workflow JSON 
{
  "nodes": [
    {
      "parameters": {},
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        0,
        -80
      ],
      "id": "28bdc599-680f-4306-b43f-a972c5ac5ab4",
      "name": "When clicking ‘Execute workflow’"
    },
    {
      "parameters": {
        "aggregate": "aggregateAllItemData",
        "options": {
          "groupByFields": "domain",
          "includeBinaries": true
        }
      },
      "type": "n8n-nodes-base.aggregate",
      "typeVersion": 1,
      "position": [
        880,
        -176
      ],
      "id": "29086978-669e-49de-a153-8ab5cf423f3a",
      "name": "Aggregate All"
    },
    {
      "parameters": {
        "fieldsToAggregate": {
          "fieldToAggregate": [
            {
              "fieldToAggregate": "email"
            },
            {
              "fieldToAggregate": "confirmed",
              "renameField": true,
              "outputFieldName": "T/F"
            }
          ]
        },
        "options": {
          "groupByFields": "domain"
        }
      },
      "type": "n8n-nodes-base.aggregate",
      "typeVersion": 1,
      "position": [
        448,
        16
      ],
      "id": "68650cbd-3885-4e68-9398-57ac13b75d55",
      "name": "Aggregate Individual"
    },
    {
      "parameters": {
        "operation": "toText",
        "sourceProperty": "email",
        "options": {}
      },
      "type": "n8n-nodes-base.convertToFile",
      "typeVersion": 1.1,
      "position": [
        448,
        -176
      ],
      "id": "4facc738-c1ef-4923-a44b-24e687a98adf",
      "name": "Convert to File"
    },
    {
      "parameters": {
        "mode": "raw",
        "jsonOutput": "={{ \n$('Emails').item.json\n}}",
        "includeOtherFields": true,
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        656,
        -176
      ],
      "id": "c2ff68b1-7552-444a-93b5-dbe6b5f903a3",
      "name": "Add Fields"
    },
    {
      "parameters": {
        "jsCode": "return [\n  {\n    json: {\n      email: \"Grant.Goyette@hotmail.com\",\n      confirmed: true,\n      domain: 'hotmail.com'\n    }\n  },\n  {\n    json: {\n      email: \"Ian_Schroeder@yahoo.com\",\n      confirmed: false,\n      domain: 'yahoo.com'\n    }\n  },\n  {\n    json: {\n      email: \"Jerome83@yahoo.com\",\n      confirmed: false,\n      domain: 'yahoo.com'\n    }\n  },\n  {\n    json: {\n      email: \"Marie55@gmail.com\",\n      confirmed: false,\n      domain: 'gmail.com'\n    }\n  },\n  {\n    json: {\n      email: \"Rene78@yahoo.com\",\n      confirmed: true,\n      domain: 'yahoo.com'\n    }\n  },\n]"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "position": [
        224,
        -80
      ],
      "id": "c74ca043-1115-4d60-9834-f08cab7dfd04",
      "name": "Emails"
    },
    {
      "parameters": {
        "assignments": {
          "assignments": [
            {
              "id": "f5422421-b865-4488-b261-33a95f51b724",
              "name": "orig_data",
              "value": "={{ $('Emails').item.json }}",
              "type": "string"
            }
          ]
        },
        "includeOtherFields": true,
        "options": {}
      },
      "type": "n8n-nodes-base.set",
      "typeVersion": 3.4,
      "position": [
        1088,
        -176
      ],
      "id": "12963bce-e1c4-42f3-b86b-ca3e3af8ce7f",
      "name": "PairedItem"
    }
  ],
  "connections": {
    "When clicking ‘Execute workflow’": {
      "main": [
        [
          {
            "node": "Emails",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Aggregate All": {
      "main": [
        [
          {
            "node": "PairedItem",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Convert to File": {
      "main": [
        [
          {
            "node": "Add Fields",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Add Fields": {
      "main": [
        [
          {
            "node": "Aggregate All",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Emails": {
      "main": [
        [
          {
            "node": "Aggregate Individual",
            "type": "main",
            "index": 0
          },
          {
            "node": "Convert to File",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {}
}

Review / Merge checklist

  • PR title and summary are descriptive. (conventions)
    -->
  • Docs updated or follow-up ticket created.
  • Tests included.
  • PR Labeled with release/backport (if the PR is an urgent fix that needs to be backported)

Note

Adds configurable group-by to Aggregate with nested-field conflict checks, per-group aggregation for both modes, and binary handling, plus comprehensive tests and example workflow.

  • Aggregate node (packages/nodes-base/nodes/Transform/Aggregate/Aggregate.node.ts):
    • Feature: New options.groupByFields enabling grouping by one or more fields (dot-notation supported) with per-group outputs.
    • Aggregation: Supports grouping for both aggregateIndividualFields and aggregateAllItemData, re-aggregating within each group.
    • Validation:
      • Reject non-scalar group-by values.
      • Detect nested field path conflicts via hasNestedPathConflict between group-by fields and output/destination fields.
      • Enforce unique output field names; preserve existing missing/merge behaviors.
    • Binaries: Include binaries per group with optional uniqueness filtering.
    • Minor: options placeholder changed to Add Option.
  • Tests (packages/nodes-base/nodes/Transform/Aggregate/test/Aggregate.groupby.test.ts):
    • Cover conflicts (parent/child paths), duplicate outputs, empty fields, non-scalar grouping, success cases, and binary inclusion.
  • Example workflow (packages/nodes-base/nodes/Transform/Aggregate/test/workflow.groupby.json):
    • Demonstrates grouping by domain for both aggregation modes and field include/exclude variants.

Written by Cursor Bugbot for commit a93b338d0a. This will update automatically on new commits. Configure here.

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/n8n-io/n8n/pull/22058 **Author:** [@pemontto](https://github.com/pemontto) **Created:** 11/19/2025 **Status:** 🔄 Open **Base:** `master` ← **Head:** `aggregate-group-by` --- ### 📝 Commits (2) - [`c6ea2bf`](https://github.com/n8n-io/n8n/commit/c6ea2bfd23eeee0e602dca9df96174b5042ba0da) feat(Aggregate Node): add group by functionality to aggregate node - [`a93b338`](https://github.com/n8n-io/n8n/commit/a93b338d0ac916a1c1b0e8c295d4961092237709) feat(Aggregate): improve conflict detection for group by fields ### 📊 Changes **3 files changed** (+1164 additions, -15 deletions) <details> <summary>View changed files</summary> 📝 `packages/nodes-base/nodes/Transform/Aggregate/Aggregate.node.ts` (+288 -15) ➕ `packages/nodes-base/nodes/Transform/Aggregate/test/Aggregate.groupby.test.ts` (+578 -0) ➕ `packages/nodes-base/nodes/Transform/Aggregate/test/workflow.groupby.json` (+298 -0) </details> ### 📄 Description ## Summary _`Group By and Aggregate go together like peas and carrots 🥕` - Forrest Gump (probably)_ This PR adds Group By support to the Aggregate node, enabling users to group items by field values before aggregating. This is a fundamental data manipulation pattern that brings n8n closer to standards like SQL GROUP BY, Pandas groupby(), or Excel Pivot Tables. ### Features ✅ Group by fields: Split data into groups based on one or multiple fields (comma-separated). ✅ Flexible Modes: Supports both "Individual Fields" and "All Item Data" aggregation modes within the groups. ✅ Binary Support: Full support for including/grouping binary data. ### Why Add This? Group By is essential for structured data aggregation. It allows users to move from a global aggregation to specific buckets. Before (One aggregation across all items): ```js // Output: 1 item { emails: ["user1@yahoo.com", "user2@gmail.com", "user3@yahoo.com", "user4@hotmail.com"] } ``` After (Group By 'domain'): ```js // Output: 3 items [ { domain: "yahoo.com", emails: ["user1@yahoo.com", "user3@yahoo.com"] }, { domain: "gmail.com", emails: ["user2@gmail.com"] }, { domain: "hotmail.com", emails: ["user4@hotmail.com"] } ] ``` ### Outstanding Questions Item Linking (pairedItem): Initially, I mapped all constituent input items to the pairedItem array for the resulting group (which seemed semantically correct). However, I didn't observe any functional difference in the UI compared to just mapping the first item. I found just using the first item at least gives me a functional way to trace back to an originating item, i.e. allowing `$('previous node').item` to work. Mapping all related items as an array would return the same as if no mapping took place: <img width="746" height="152" alt="image" src="https://github.com/user-attachments/assets/0d4ba46e-1bba-4052-8713-27e855df0317" /> ### Sample Workflow <img width="1466" height="508" alt="image" src="https://github.com/user-attachments/assets/2e9f76b2-3848-4f2a-a628-a1fde9f8aae2" /> <details> <summary> Click to expand workflow JSON</summary> ```json { "nodes": [ { "parameters": {}, "type": "n8n-nodes-base.manualTrigger", "typeVersion": 1, "position": [ 0, -80 ], "id": "28bdc599-680f-4306-b43f-a972c5ac5ab4", "name": "When clicking ‘Execute workflow’" }, { "parameters": { "aggregate": "aggregateAllItemData", "options": { "groupByFields": "domain", "includeBinaries": true } }, "type": "n8n-nodes-base.aggregate", "typeVersion": 1, "position": [ 880, -176 ], "id": "29086978-669e-49de-a153-8ab5cf423f3a", "name": "Aggregate All" }, { "parameters": { "fieldsToAggregate": { "fieldToAggregate": [ { "fieldToAggregate": "email" }, { "fieldToAggregate": "confirmed", "renameField": true, "outputFieldName": "T/F" } ] }, "options": { "groupByFields": "domain" } }, "type": "n8n-nodes-base.aggregate", "typeVersion": 1, "position": [ 448, 16 ], "id": "68650cbd-3885-4e68-9398-57ac13b75d55", "name": "Aggregate Individual" }, { "parameters": { "operation": "toText", "sourceProperty": "email", "options": {} }, "type": "n8n-nodes-base.convertToFile", "typeVersion": 1.1, "position": [ 448, -176 ], "id": "4facc738-c1ef-4923-a44b-24e687a98adf", "name": "Convert to File" }, { "parameters": { "mode": "raw", "jsonOutput": "={{ \n$('Emails').item.json\n}}", "includeOtherFields": true, "options": {} }, "type": "n8n-nodes-base.set", "typeVersion": 3.4, "position": [ 656, -176 ], "id": "c2ff68b1-7552-444a-93b5-dbe6b5f903a3", "name": "Add Fields" }, { "parameters": { "jsCode": "return [\n {\n json: {\n email: \"Grant.Goyette@hotmail.com\",\n confirmed: true,\n domain: 'hotmail.com'\n }\n },\n {\n json: {\n email: \"Ian_Schroeder@yahoo.com\",\n confirmed: false,\n domain: 'yahoo.com'\n }\n },\n {\n json: {\n email: \"Jerome83@yahoo.com\",\n confirmed: false,\n domain: 'yahoo.com'\n }\n },\n {\n json: {\n email: \"Marie55@gmail.com\",\n confirmed: false,\n domain: 'gmail.com'\n }\n },\n {\n json: {\n email: \"Rene78@yahoo.com\",\n confirmed: true,\n domain: 'yahoo.com'\n }\n },\n]" }, "type": "n8n-nodes-base.code", "typeVersion": 2, "position": [ 224, -80 ], "id": "c74ca043-1115-4d60-9834-f08cab7dfd04", "name": "Emails" }, { "parameters": { "assignments": { "assignments": [ { "id": "f5422421-b865-4488-b261-33a95f51b724", "name": "orig_data", "value": "={{ $('Emails').item.json }}", "type": "string" } ] }, "includeOtherFields": true, "options": {} }, "type": "n8n-nodes-base.set", "typeVersion": 3.4, "position": [ 1088, -176 ], "id": "12963bce-e1c4-42f3-b86b-ca3e3af8ce7f", "name": "PairedItem" } ], "connections": { "When clicking ‘Execute workflow’": { "main": [ [ { "node": "Emails", "type": "main", "index": 0 } ] ] }, "Aggregate All": { "main": [ [ { "node": "PairedItem", "type": "main", "index": 0 } ] ] }, "Convert to File": { "main": [ [ { "node": "Add Fields", "type": "main", "index": 0 } ] ] }, "Add Fields": { "main": [ [ { "node": "Aggregate All", "type": "main", "index": 0 } ] ] }, "Emails": { "main": [ [ { "node": "Aggregate Individual", "type": "main", "index": 0 }, { "node": "Convert to File", "type": "main", "index": 0 } ] ] } }, "pinData": {} } ``` </details> ## Review / Merge checklist - [x] PR title and summary are descriptive. ([conventions](../blob/master/.github/pull_request_title_conventions.md)) --> - [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up ticket created. - [X] Tests included. <!-- A bug is not considered fixed, unless a test is added to prevent it from happening again. A feature is not complete without tests. --> - [ ] PR Labeled with `release/backport` (if the PR is an urgent fix that needs to be backported) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Adds configurable group-by to `Aggregate` with nested-field conflict checks, per-group aggregation for both modes, and binary handling, plus comprehensive tests and example workflow. > > - **Aggregate node (`packages/nodes-base/nodes/Transform/Aggregate/Aggregate.node.ts`)**: > - **Feature**: New `options.groupByFields` enabling grouping by one or more fields (dot-notation supported) with per-group outputs. > - **Aggregation**: Supports grouping for both `aggregateIndividualFields` and `aggregateAllItemData`, re-aggregating within each group. > - **Validation**: > - Reject non-scalar group-by values. > - Detect nested field path conflicts via `hasNestedPathConflict` between group-by fields and output/destination fields. > - Enforce unique output field names; preserve existing missing/merge behaviors. > - **Binaries**: Include binaries per group with optional uniqueness filtering. > - Minor: options placeholder changed to `Add Option`. > - **Tests** (`packages/nodes-base/nodes/Transform/Aggregate/test/Aggregate.groupby.test.ts`): > - Cover conflicts (parent/child paths), duplicate outputs, empty fields, non-scalar grouping, success cases, and binary inclusion. > - **Example workflow** (`packages/nodes-base/nodes/Transform/Aggregate/test/workflow.groupby.json`): > - Demonstrates grouping by `domain` for both aggregation modes and field include/exclude variants. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit a93b338d0ac916a1c1b0e8c295d4961092237709. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the
pull-request
 label 2025-11-20 06:22:48 -06:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
Backlog
bug-report
Bump
Bumped
closed:duplicate
closed:enhancement/feature
closed:incomplete-template
closed:info
closed:non-english
closed:support-issue
closed:working-as-expected
Duplicate
in linear
In Review
n8n team
Needs Feedback
Needs Feedback:Unable to reproduce
node/improvement
node/issue
node/new
pull-request
Mirrored from GitHub Pull Request
Released
Stale
status:complete
status:in-linear
status:pending-assignment
status:team-assigned
team:adore
team:ai
team:cats
team:nodes
team:payday
triage:complete
triage:in-progress
triage:needs-info
triage:needs-reproduction
triage:pending
triage:stalled
ui
Upcoming Release
No Label
pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/n8n#22019
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?