[Bug]: e2e tests have become flaky #1437

Closed
opened 2026-02-28 19:43:32 -06:00 by GiteaMirror · 7 comments
Owner

Originally created by @MatissJanis on GitHub (Sep 22, 2024).

Verified issue does not already exist?

  • I have searched and found no existing issue
  • I will be providing steps how to reproduce the bug (in most cases this will also mean uploading a demo budget file)

What happened?

The e2e tests have become increasingly more flaky in the past few days. Need to look into what's the issue and address it.

Where are you hosting Actual?

None

What browsers are you seeing the problem on?

No response

Operating System

None

Originally created by @MatissJanis on GitHub (Sep 22, 2024). ### Verified issue does not already exist? - [X] I have searched and found no existing issue - [X] I will be providing steps how to reproduce the bug (in most cases this will also mean uploading a demo budget file) ### What happened? The e2e tests have become increasingly more flaky in the past few days. Need to look into what's the issue and address it. ### Where are you hosting Actual? None ### What browsers are you seeing the problem on? _No response_ ### Operating System None
GiteaMirror added the regressionbug labels 2026-02-28 19:43:32 -06:00
Author
Owner

@joel-jeremy commented on GitHub (Sep 23, 2024):

One this I noticed is that the rules page is crashing on VRT but when you open the page in the browser it works just fine.
VRT:
image

Browser:
image

@joel-jeremy commented on GitHub (Sep 23, 2024): One this I noticed is that the rules page is crashing on VRT but when you open the page in the browser it works just fine. VRT: ![image](https://github.com/user-attachments/assets/22805ff3-d563-4984-aead-ff162f5def0a) Browser: ![image](https://github.com/user-attachments/assets/ed2b72c5-5acc-453d-a925-4aa18e671a31)
Author
Owner

@joel-jeremy commented on GitHub (Sep 23, 2024):

This change also appears randomly:
image

@joel-jeremy commented on GitHub (Sep 23, 2024): This change also appears randomly: ![image](https://github.com/user-attachments/assets/b48ca5f2-6bd5-42f2-a1cb-539e103c6528)
Author
Owner

@UnderKoen commented on GitHub (Sep 25, 2024):

One thing I noticed with https://github.com/actualbudget/actual/pull/3499 was that the css classes of glamour weren't applied with the same precedence as in the browser. This can be caused by the order of the css in the final css changing

@UnderKoen commented on GitHub (Sep 25, 2024): One thing I noticed with https://github.com/actualbudget/actual/pull/3499 was that the css classes of glamour weren't applied with the same precedence as in the browser. This can be caused by the order of the css in the final css changing
Author
Owner

@MatissJanis commented on GitHub (Sep 25, 2024):

Another issue I'm seeing semi-frequently: onboarding test.


Sample failing ci job: https://github.com/actualbudget/actual/actions/runs/11037117182/job/30657293317

Trace: trace.zip

View the trace here: https://trace.playwright.dev


Edit: it seems that there are a bunch of different redirects happening when clicking "start fresh". Some are redirecting to /budget and some to /accounts. Must be a race condition somewhere.

@MatissJanis commented on GitHub (Sep 25, 2024): Another issue I'm seeing semi-frequently: onboarding test. --- Sample failing ci job: https://github.com/actualbudget/actual/actions/runs/11037117182/job/30657293317 Trace: [trace.zip](https://github.com/user-attachments/files/17137709/trace.zip) View the trace here: https://trace.playwright.dev --- Edit: it seems that there are a bunch of different redirects happening when clicking "start fresh". Some are redirecting to `/budget` and some to `/accounts`. Must be a race condition somewhere.
Author
Owner

@MatissJanis commented on GitHub (Sep 25, 2024):

Both the rules & onboarding e2e should be more stable now. Haven't been able to reproduce the settings one yet. If anyone sees a failing CI job with it - please post it here.

Or if you also see other e2e ci jobs failing (that use the latest changes in master) - please post them here.

@MatissJanis commented on GitHub (Sep 25, 2024): Both the rules & onboarding e2e should be more stable now. Haven't been able to reproduce the settings one yet. If anyone sees a failing CI job with it - please post it here. Or if you also see other e2e ci jobs failing (that use the latest changes in master) - please post them here.
Author
Owner

@MatissJanis commented on GitHub (Oct 3, 2024):

Related https://github.com/actualbudget/actual/pull/3506

@MatissJanis commented on GitHub (Oct 3, 2024): Related https://github.com/actualbudget/actual/pull/3506
Author
Owner

@MatissJanis commented on GitHub (Oct 4, 2024):

e2e tests are stable in master again. Closing this as complete

@MatissJanis commented on GitHub (Oct 4, 2024): e2e tests are stable in `master` again. Closing this as complete
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/actual#1437