mirror of
https://github.com/actualbudget/actual.git
synced 2026-03-09 11:42:54 -05:00
[Bug]: Importing transactions from OFX deletes similar transactions from previous dates #961
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: github-starred/actual#961
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @keriati on GitHub (Mar 4, 2024).
Verified issue does not already exist?
Is this related to GoCardless, Simplefin or another bank-sync provider?
What happened?
When I import an OFX file with my transactions and it has similar transactions from previous days, some of the older transactions will be deleted.
What error did you receive?
Transactions are missing after ofx import.
Where are you hosting Actual?
Docker
What browsers are you seeing the problem on?
Firefox
Operating System
Mac OSX
@youngcw commented on GitHub (Mar 5, 2024):
where those older transactions imported via ofx?
@keriati commented on GitHub (Mar 5, 2024):
yes, i did not research this issue yet, but it happens when similar transactions are made on close dates
@MatissJanis commented on GitHub (Mar 15, 2024):
Interesting. Would you mind providing reproduction steps (perhaps an existing budget file + an example OFX import)?
Transactions should not be deleted on import. Only create & update operations should be happening. So I'm curious to see how this could be reproduced.
@britto89 commented on GitHub (Mar 22, 2024):
Hi @MatissJanis
I can confirm I experience a similar issue but with CSV imports where the original transaction was also imported by CSV. I find it happens when the transaction is for the same dollar value and description but different dates but have also seen it for a different date, different description and same dollar value. The one I notice the most is my loan payment and coffee shop purchases. If I had a loan payment for $500 on the 15/3/24 already imported and reconciled then when I import the next week which has a loan repayment on the 22/3/24 for the same amount the entry for the 15/3/24 disappears. I don't think it deletes but I think the date gets adjusted to the 22/3/24 and no transaction is imported meaning the balance is now off by $500 as you are missing a transaction. You can find out which ones have done this since I find they are not bolded like the other freshly imported transactions. Let me know if you need some sample csv's to replicate and I can pull some from my bank and set some fictitious descriptions.
@keriati commented on GitHub (Mar 22, 2024):
I can confirm that for me it is the same with ofx import, usually my same amount daily coffee transactions and lunch transactions are gone.
@MatissJanis commented on GitHub (Mar 23, 2024):
Missing reproduction steps.
@britto89 commented on GitHub (Mar 23, 2024):
Steps to reproduce.
This should show that all transactions get overwritten/updated even the coffee shop purchases that have different descriptions.
week1.csv
week2.csv
Client version: v24.3.0
Server version: v24.3.0
@MatissJanis commented on GitHub (Mar 23, 2024):
@britto89 that functionality is actually by design. The imports have de-dupe logic that attempts to match the new imported transactions to the old ones. And if they match - it updates their details. It matches by date and amount. Which in the week1/2 case doesn't work well because.. all the amounts are the same.
For your use-case my recommendation would be to perform imports with a months worth of data. And then to allow the de-dupe logic to take care of removing duplicates & creating new transactions. Instead of doing weekly exports.
However, I'm still missing reproduction steps for the root-issue reported here: transactions getting deleted. Please provide reproduction steps to delete transactions on import.
@danspam commented on GitHub (Mar 31, 2024):
Cant import anything, it has totally trashed my accounts.
@crzdg commented on GitHub (Apr 5, 2024):
I also experience similar issues.
When I import similar transactions with CSV it actual updates older ones with the newer date. For some this might occur as deleting, but for me the transactions note does not change.
Also, when you have locked transactions after reconcile, the same CSV is not updating old transactions but also not adding new ones.
@svoboda77 commented on GitHub (Apr 5, 2024):
It breaks my OFX imports the same way for me too. If this behavior is by design, then the de-dupe logic is broken. Two transactions with
1/ different dates
2/ different note/memo
3/ same payees
4/ same amounts
should not be considered duplicates.
@keriati commented on GitHub (Apr 5, 2024):
Well it might technically not be a delete, it probably just does not import similar transactions. I think all reproduction steps are already in this thread and it's getting clear what happens. I am importing monthly once and some transactions are so far always lost, usually the ones that are almost similar just like how other people described it.
We could also say that the de-duplication logic is rather harmful in the current implementation. Anyhow it should not de-duplicate anything as I import full months and there is in my case no overlap or anything that would need such a logic.
@keriati commented on GitHub (Apr 6, 2024):
@MatissJanis I think the needs info label can now be removed.
@MatissJanis commented on GitHub (Apr 6, 2024):
👋 I understand how the existing de-duplication logic can be quite frustrating under some circumstances. The dedupe logic has been unchanged basically since the very inception of open-source Actual and while it's not absolutely perfect, IMO it does handle the majority of the scenarios that occur when doing file imports.
Here's a feature request I have opened: the ability to disable the dedupe logic. Which would give you folks a solution. PRs with the implementation are welcome!
The dedupe logic works quite well if larger time-ranges are used. So my recommendation for now (until someone builds the feature) would be to import larger time ranges. The specific time-range depends on your use case, but here's the logic you can follow: "previous time range * 2" (i.e. if you usually import a month - try importing the current month + previous in the same go; if you usually import a week - try importing this week + previous week).
I will close this bug report in favour of the feature ticket.