github-starred/actual
2
0
Fork 0
mirror of https://github.com/actualbudget/actual.git synced 2026-07-15 22:54:03 -05:00
Code Issues 3.0k Packages Projects Releases 58 Wiki Activity

[PR #6193] [MERGED] fix flaky reports tests #130484

New Issue
Closed
opened 2026-06-15 22:26:24 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-06-15 22:26:24 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/actualbudget/actual/pull/6193
Author: @matt-fidd
Created: 11/18/2025
Status: ✅ Merged
Merged: 11/18/2025
Merged by: @matt-fidd

Base: master ← Head: flaky-reports-test

📝 Commits (6)

  • 8ec14de add isTestEnv hook
  • 8364bde disable all recharts animations in test env
  • cbe966d note
  • 1bd0d9d drop all reports snapshots
  • a8f7a55 Update VRT screenshots
  • 6112bb3 disable animation instead of zeroing duration

📊 Changes

46 files changed (+109 additions, -31 deletions)

View changed files

📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Area-Graph-and-checks-the-visuals-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Area-Graph-and-checks-the-visuals-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Area-Graph-and-checks-the-visuals-3-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Bar-Graph-and-checks-the-visuals-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Bar-Graph-and-checks-the-visuals-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Bar-Graph-and-checks-the-visuals-3-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Data-Table-and-checks-the-visuals-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Data-Table-and-checks-the-visuals-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Data-Table-and-checks-the-visuals-3-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Donut-Graph-and-checks-the-visuals-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Donut-Graph-and-checks-the-visuals-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Donut-Graph-and-checks-the-visuals-3-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Line-Graph-and-checks-the-visuals-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Line-Graph-and-checks-the-visuals-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Line-Graph-and-checks-the-visuals-3-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-labels-button-shows-the-labels-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-labels-button-shows-the-labels-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-legend-button-shows-the-legend-side-bar-1-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-legend-button-shows-the-legend-side-bar-2-chromium-linux.png (+0 -0)
📝 packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-summary-button-shows-the-summary-1-chromium-linux.png (+0 -0)

...and 26 more files

📄 Description

Hopefully this will fix the flaky cash flow chart VRT. That chart's animation is set to run for 1000ms, so I imagine the issue is caused by the screenshot sometimes grabbing just before it's finished. To mitigate this in the future, I also just disabled all recharts animations in VRT in case we use some other really long animation in the future. I also swapped from animationDuration: 0 to disabling the animations completely, the reports page seems a bit quicker for that too.

Also refactored some other places we detect the playwright env to tidy up a bit, it's only a tiny hook but we may need to expand this in the future and it's easier to just change it in one place if we need.

Looks like the flake was introduced because the order that stacked bars animated was changed in https://github.com/recharts/recharts/pull/6593

There look to be lots of changes here because I regenerated all reports screenshots. Some were so close to the threshold that tiny normal inconsistencies were pushing them over the edge and causing flakes.

I've run the VRT about 5/6 times and no fails after this change

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/actualbudget/actual/pull/6193 **Author:** [@matt-fidd](https://github.com/matt-fidd) **Created:** 11/18/2025 **Status:** ✅ Merged **Merged:** 11/18/2025 **Merged by:** [@matt-fidd](https://github.com/matt-fidd) **Base:** `master` ← **Head:** `flaky-reports-test` --- ### 📝 Commits (6) - [`8ec14de`](https://github.com/actualbudget/actual/commit/8ec14dee3cac23c5677d55f1c2610b7f55ce7e61) add isTestEnv hook - [`8364bde`](https://github.com/actualbudget/actual/commit/8364bde26946d3c0b2c7199216a0315be1cb8e72) disable all recharts animations in test env - [`cbe966d`](https://github.com/actualbudget/actual/commit/cbe966d6aef10295970c8652144fa88a9f6ebe57) note - [`1bd0d9d`](https://github.com/actualbudget/actual/commit/1bd0d9d41b18b7b8035dd716b894345afb0ab800) drop all reports snapshots - [`a8f7a55`](https://github.com/actualbudget/actual/commit/a8f7a55d2c9ce32dc933fe8e29b6f2f85d3757ec) Update VRT screenshots - [`6112bb3`](https://github.com/actualbudget/actual/commit/6112bb3e204113597cf49b158b36cbd5eb62a88f) disable animation instead of zeroing duration ### 📊 Changes **46 files changed** (+109 additions, -31 deletions) <details> <summary>View changed files</summary> 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Area-Graph-and-checks-the-visuals-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Area-Graph-and-checks-the-visuals-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Area-Graph-and-checks-the-visuals-3-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Bar-Graph-and-checks-the-visuals-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Bar-Graph-and-checks-the-visuals-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Bar-Graph-and-checks-the-visuals-3-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Data-Table-and-checks-the-visuals-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Data-Table-and-checks-the-visuals-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Data-Table-and-checks-the-visuals-3-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Donut-Graph-and-checks-the-visuals-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Donut-Graph-and-checks-the-visuals-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Donut-Graph-and-checks-the-visuals-3-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Line-Graph-and-checks-the-visuals-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Line-Graph-and-checks-the-visuals-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Switches-to-Line-Graph-and-checks-the-visuals-3-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-labels-button-shows-the-labels-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-labels-button-shows-the-labels-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-legend-button-shows-the-legend-side-bar-1-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-legend-button-shows-the-legend-side-bar-2-chromium-linux.png` (+0 -0) 📝 `packages/desktop-client/e2e/reports.test.ts-snapshots/Reports-custom-reports-Validates-that-show-summary-button-shows-the-summary-1-chromium-linux.png` (+0 -0) _...and 26 more files_ </details> ### 📄 Description Hopefully this will fix the flaky cash flow chart VRT. That chart's animation is set to run for 1000ms, so I imagine the issue is caused by the screenshot sometimes grabbing just before it's finished. To mitigate this in the future, I also just disabled all recharts animations in VRT in case we use some other really long animation in the future. I also swapped from `animationDuration: 0` to disabling the animations completely, the reports page seems a bit quicker for that too. Also refactored some other places we detect the playwright env to tidy up a bit, it's only a tiny hook but we may need to expand this in the future and it's easier to just change it in one place if we need. Looks like the flake was introduced because the order that stacked bars animated was changed in https://github.com/recharts/recharts/pull/6593 There look to be lots of changes here because I regenerated all reports screenshots. Some were so close to the threshold that tiny normal inconsistencies were pushing them over the edge and causing flakes. I've run the VRT about 5/6 times and no fails after this change --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-06-15 22:26:24 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
AI generated
API
bank sync
budgeting
bug
can’t replicate
dependencies
docker
documentation
electron
experimental feature
feature
feedback
goal templates
good first issue
help wanted
importers
maintenance
needs info
needs testing
needs triage
needs votes
openid
payees
pull-request
Mirrored from GitHub Pull Request
 regression
reports
responsive
rules
schedules
server
✨ merged
split transactions
tech debt
tech-support
theme
transaction import
transaction reconciliation
transactions
translations
upstream
user interface
✅ approved
wontfix
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/actual#130484
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?