[GH-ISSUE #921] bitwarden_rs sometimes sends invalid responses ("Data": null) #13442

Closed
opened 2026-04-23 05:02:13 -05:00 by GiteaMirror · 13 comments
Owner

Originally created by @champtar on GitHub (Mar 22, 2020).
Original GitHub issue: https://github.com/dani-garcia/vaultwarden/issues/921

Subject of the issue

bitwarden_rs in some cases sends invalid responses where "Data" is null, breaking Android sync

Your environment

  • Bitwarden_rs version: 1.4
  • Install method: docker image
  • Clients used: Android 2.3.1 / Firefox extension
  • Other relevant information:

Steps to reproduce

Not sure exactly, maybe doing a clone of an item using the Firefox extension, my wife ended up with broken sync on the Android App

Expected behaviour

Everything works

Actual behaviour

{
  "Ciphers": [
...
    {
      "Attachments": [],
      "CollectionIds": [],
      "Data": null,
      "Edit": true,
      "Favorite": false,
      "Fields": null,
      "FolderId": null,
      "Id": "e3c473c5-a001-48d6-b8a6-b9225a604725",
      "Login": null,
      "Name": "<redacted / encrypted>",
      "Notes": null,
      "Object": "cipher",
      "OrganizationId": null,
      "OrganizationUseTotp": true,
      "PasswordHistory": null,
      "RevisionDate": "2020-03-13T17:05:42.535886Z",
      "Type": 1
    },
...

And here the same item when using the webui json export

{
  "folders": [
...
  ],
  "items": [
...
    {
      "id": "e3c473c5-a001-48d6-b8a6-b9225a604725",
      "organizationId": null,
      "folderId": null,
      "type": 1,
      "name": "<redacted>",
      "notes": null,
      "favorite": false,
      "login": {
        "username": null,
        "password": null,
        "totp": null
      },
      "collectionIds": null
    },
...

Additional information:

How to reproduce the bug might be fixed by 2ee07ea1d8, but we might need to clean the database

This was also reported at https://github.com/bitwarden/mobile/issues/790, the closing message is that Data should not be null

Originally created by @champtar on GitHub (Mar 22, 2020). Original GitHub issue: https://github.com/dani-garcia/vaultwarden/issues/921 ### Subject of the issue bitwarden_rs in some cases sends invalid responses where "Data" is null, breaking Android sync ### Your environment * Bitwarden_rs version: 1.4 * Install method: docker image * Clients used: Android 2.3.1 / Firefox extension * Other relevant information: ### Steps to reproduce Not sure exactly, maybe doing a clone of an item using the Firefox extension, my wife ended up with broken sync on the Android App ### Expected behaviour Everything works ### Actual behaviour ``` { "Ciphers": [ ... { "Attachments": [], "CollectionIds": [], "Data": null, "Edit": true, "Favorite": false, "Fields": null, "FolderId": null, "Id": "e3c473c5-a001-48d6-b8a6-b9225a604725", "Login": null, "Name": "<redacted / encrypted>", "Notes": null, "Object": "cipher", "OrganizationId": null, "OrganizationUseTotp": true, "PasswordHistory": null, "RevisionDate": "2020-03-13T17:05:42.535886Z", "Type": 1 }, ... ``` And here the same item when using the webui json export ``` { "folders": [ ... ], "items": [ ... { "id": "e3c473c5-a001-48d6-b8a6-b9225a604725", "organizationId": null, "folderId": null, "type": 1, "name": "<redacted>", "notes": null, "favorite": false, "login": { "username": null, "password": null, "totp": null }, "collectionIds": null }, ... ``` ### Additional information: How to reproduce the bug might be fixed by https://github.com/dani-garcia/bitwarden_rs/commit/2ee07ea1d8a7e42ca6532f461f2aeeec4db629be, but we might need to clean the database This was also reported at https://github.com/bitwarden/mobile/issues/790, the closing message is that Data should not be null
Author
Owner

@deeztek commented on GitHub (Mar 30, 2020):

Try this.

Exported my vault to .json. Then I did a search for the following:

"username": null,
"password": null

Then in the web browser, I located the particular entry and deleted it.

Not sure which entry it was, but after deleting 27 entries, the android app started syncing again.

<!-- gh-comment-id:606215482 --> @deeztek commented on GitHub (Mar 30, 2020): Try this. Exported my vault to .json. Then I did a search for the following: "username": null, "password": null Then in the web browser, I located the particular entry and deleted it. Not sure which entry it was, but after deleting 27 entries, the android app started syncing again.
Author
Owner

@dani-garcia commented on GitHub (Mar 30, 2020):

This should be solved with adf47827c9. Hopefully that also fixes your issue @deeztek, without having to modify the export manually.

<!-- gh-comment-id:606227690 --> @dani-garcia commented on GitHub (Mar 30, 2020): This should be solved with https://github.com/dani-garcia/bitwarden_rs/commit/adf47827c98b09dc5cc0564f790f5f6c4a1906f8. Hopefully that also fixes your issue @deeztek, without having to modify the export manually.
Author
Owner

@deeztek commented on GitHub (Mar 30, 2020):

I'm glad there is a fix, too late for me since I already did the export but hopefully this will help other people.

<!-- gh-comment-id:606229797 --> @deeztek commented on GitHub (Mar 30, 2020): I'm glad there is a fix, too late for me since I already did the export but hopefully this will help other people.
Author
Owner

@alex-martel commented on GitHub (Mar 30, 2020):

@dani-garcia thanks for the fix!
I have a question though, with the fix then it will sync empty username/password, so users wont be able to know until they arrive at the moment they try to login to one of the faulty service. Should it be logged, or told somehow?

<!-- gh-comment-id:606264617 --> @alex-martel commented on GitHub (Mar 30, 2020): @dani-garcia thanks for the fix! I have a question though, with the fix then it will sync empty username/password, so users wont be able to know until they arrive at the moment they try to login to one of the faulty service. Should it be logged, or told somehow?
Author
Owner

@dani-garcia commented on GitHub (Mar 30, 2020):

There isn't much we can show the user, the data we return is encrpyted so we can't, say, modify the name to append (Corrupted) to it or something. We can log an entry in the console but that's easily missed unless you are looking at it.

<!-- gh-comment-id:606270484 --> @dani-garcia commented on GitHub (Mar 30, 2020): There isn't much we can show the user, the data we return is encrpyted so we can't, say, modify the name to append (Corrupted) to it or something. We can log an entry in the console but that's easily missed unless you are looking at it.
Author
Owner

@dani-garcia commented on GitHub (Apr 8, 2020):

Fixed already so closing.

<!-- gh-comment-id:610954088 --> @dani-garcia commented on GitHub (Apr 8, 2020): Fixed already so closing.
Author
Owner

@jjlin commented on GitHub (Apr 11, 2020):

I looked into this issue a bit more, and it's easy to reproduce. But to summarize:

  • If you're running 1.14.1 and use the web vault or browser extensions to delete any malformed entries as described further up, mobile sync should work fine.
  • If you're running 1.14.2 or later (which contains adf47827), mobile sync should be successful despite the presence of any empty cloned entries, and you can delete such entries whenever you get around to it.

To reproduce:

  1. Create a new bitwarden_rs instance using the bitwardenrs/server:1.14 image (which doesn't have the 2ee07ea1 cloning fix).
  2. Log in with a version of the mobile app that supports cloning entries.
  3. Create an entry and attempt to clone it. This returns an Organization id not provided error, but a partially-cloned empty/broken entry still ends up getting created.
  4. Attempt to sync the mobile app. This now returns Syncing failed.
  5. However, the empty cloned entry will show up in the web vault, browser extensions, or desktop client (any of the TypeScript-based clients). Once the empty cloned entry is deleted using one of those clients, the mobile app can sync again.

For reference, here's the vault export and corresponding cipher entries:

Vault export
{
  "folders": [],
  "items": [
    {
      "id": "88f11a65-3126-4b29-ad86-99460f75a2c9",
      "organizationId": null,
      "folderId": null,
      "type": 1,
      "name": "test",
      "notes": null,
      "favorite": false,
      "login": {
        "username": "test",
        "password": "test",
        "totp": null
      },
      "collectionIds": null
    },
    {
      "id": "645bcd1a-2d38-465e-81dc-6cdc3714904a",
      "organizationId": null,
      "folderId": null,
      "type": 1,
      "name": "test - Clone",
      "notes": null,
      "favorite": false,
      "login": {
        "username": null,
        "password": null,
        "totp": null
      },
      "collectionIds": null
    }
  ]
}
Cipher entries
sqlite> select * from ciphers;
uuid|created_at|updated_at|user_uuid|organization_uuid|atype|name|notes|fields|data|favorite|password_history
88f11a65-3126-4b29-ad86-99460f75a2c9|2020-04-11 07:57:03.409154538|2020-04-11 07:57:03.411865188|e9d4e3cb-15d4-4154-896b-a6082007e044||1|2.8OCDidGbG7hG3ZUJDafgtA==|5mGv5qwjmqufuNGJAI6SFw==|IjKkqx1ivXhP04n+1+FayswrIAcjX+py4NdvmeJeJ3g=|||{"Fields":null,"Name":"2.8OCDidGbG7hG3ZUJDafgtA==|5mGv5qwjmqufuNGJAI6SFw==|IjKkqx1ivXhP04n+1+FayswrIAcjX+py4NdvmeJeJ3g=","Notes":null,"Password":"2.tPm92jSmJo2M1ERxSrn3uA==|BncRMWxZ3R2RiX81WcTbnQ==|q985DioypbXdC9UiAJ73GV0JrKgrc08PnmMYRl6SOn0=","PasswordHistory":null,"PasswordRevisionDate":null,"Response":null,"Totp":null,"Uris":null,"Username":"2.rmM80Hd1Z1iS4FdAiJ8xsw==|AC3JxHWYdFjS375Vtn0J/g==|0XBvAMxZ2zZXpOIUXPZYuV2hf6Ohs92rVI8kjfoO3ZM="}|0|
645bcd1a-2d38-465e-81dc-6cdc3714904a|2020-04-11 08:04:43.097618261|2020-04-11 08:04:43.146774503|e9d4e3cb-15d4-4154-896b-a6082007e044||1|2.E0lIk7R7lhW8Sej/w2nixg==|LY1zXfP6Gewy6X1039AC0w==|QgM99SpdwSQwoRHNwvBBHaKmJS9Q30Xnotoavewp/4o=||||0|
<!-- gh-comment-id:612368933 --> @jjlin commented on GitHub (Apr 11, 2020): I looked into this issue a bit more, and it's easy to reproduce. But to summarize: * If you're running 1.14.1 and use the web vault or browser extensions to delete any malformed entries as described further up, mobile sync should work fine. * If you're running 1.14.2 or later (which contains adf47827), mobile sync should be successful despite the presence of any empty cloned entries, and you can delete such entries whenever you get around to it. To reproduce: 1. Create a new bitwarden_rs instance using the `bitwardenrs/server:1.14` image (which doesn't have the 2ee07ea1 cloning fix). 1. Log in with a version of the mobile app that supports cloning entries. 1. Create an entry and attempt to clone it. This returns an `Organization id not provided` error, but a partially-cloned empty/broken entry still ends up getting created. 1. Attempt to sync the mobile app. This now returns `Syncing failed.` 1. However, the empty cloned entry will show up in the web vault, browser extensions, or desktop client (any of the TypeScript-based clients). Once the empty cloned entry is deleted using one of those clients, the mobile app can sync again. For reference, here's the vault export and corresponding cipher entries: <details> <summary>Vault export</summary> ```json { "folders": [], "items": [ { "id": "88f11a65-3126-4b29-ad86-99460f75a2c9", "organizationId": null, "folderId": null, "type": 1, "name": "test", "notes": null, "favorite": false, "login": { "username": "test", "password": "test", "totp": null }, "collectionIds": null }, { "id": "645bcd1a-2d38-465e-81dc-6cdc3714904a", "organizationId": null, "folderId": null, "type": 1, "name": "test - Clone", "notes": null, "favorite": false, "login": { "username": null, "password": null, "totp": null }, "collectionIds": null } ] } ``` </details> <details> <summary>Cipher entries</summary> ``` sqlite> select * from ciphers; uuid|created_at|updated_at|user_uuid|organization_uuid|atype|name|notes|fields|data|favorite|password_history 88f11a65-3126-4b29-ad86-99460f75a2c9|2020-04-11 07:57:03.409154538|2020-04-11 07:57:03.411865188|e9d4e3cb-15d4-4154-896b-a6082007e044||1|2.8OCDidGbG7hG3ZUJDafgtA==|5mGv5qwjmqufuNGJAI6SFw==|IjKkqx1ivXhP04n+1+FayswrIAcjX+py4NdvmeJeJ3g=|||{"Fields":null,"Name":"2.8OCDidGbG7hG3ZUJDafgtA==|5mGv5qwjmqufuNGJAI6SFw==|IjKkqx1ivXhP04n+1+FayswrIAcjX+py4NdvmeJeJ3g=","Notes":null,"Password":"2.tPm92jSmJo2M1ERxSrn3uA==|BncRMWxZ3R2RiX81WcTbnQ==|q985DioypbXdC9UiAJ73GV0JrKgrc08PnmMYRl6SOn0=","PasswordHistory":null,"PasswordRevisionDate":null,"Response":null,"Totp":null,"Uris":null,"Username":"2.rmM80Hd1Z1iS4FdAiJ8xsw==|AC3JxHWYdFjS375Vtn0J/g==|0XBvAMxZ2zZXpOIUXPZYuV2hf6Ohs92rVI8kjfoO3ZM="}|0| 645bcd1a-2d38-465e-81dc-6cdc3714904a|2020-04-11 08:04:43.097618261|2020-04-11 08:04:43.146774503|e9d4e3cb-15d4-4154-896b-a6082007e044||1|2.E0lIk7R7lhW8Sej/w2nixg==|LY1zXfP6Gewy6X1039AC0w==|QgM99SpdwSQwoRHNwvBBHaKmJS9Q30Xnotoavewp/4o=||||0| ``` </details>
Author
Owner

@dani-garcia commented on GitHub (Apr 11, 2020):

Okay so I've tried debugging this and I can get it to reproduce, but once I upgrade to 1.14.1 with the fix in adf4782, it starts working in the phone apps for me again. The data field is empty in the broken cipher, and that fails to deserialize for me with the error Err(Error("EOF while parsing a value", line: 1, column: 0)), so it gets the value from the unwrap, which should be a valid value.

You can check with println!("{:?}", serde_json::from_str::<Value>(&self.data));

<!-- gh-comment-id:612433562 --> @dani-garcia commented on GitHub (Apr 11, 2020): Okay so I've tried debugging this and I can get it to reproduce, but once I upgrade to 1.14.1 with the fix in adf4782, it starts working in the phone apps for me again. The data field is empty in the broken cipher, and that fails to deserialize for me with the error `Err(Error("EOF while parsing a value", line: 1, column: 0))`, so it gets the value from the unwrap, which should be a valid value. You can check with `println!("{:?}", serde_json::from_str::<Value>(&self.data));`
Author
Owner

@jjlin commented on GitHub (Apr 11, 2020):

1.14.1 doesn't contain adf4782 though, or do you mean you created a build of 1.14.1 with that commit cherry-picked? After I did the initial reproduction with 1.14 here, I switched to the 1.14.1 tag with the same data files, and the sync still failed for me. I'll look more into it when I get a chance.

<!-- gh-comment-id:612477969 --> @jjlin commented on GitHub (Apr 11, 2020): 1.14.1 doesn't contain adf4782 though, or do you mean you created a build of 1.14.1 with that commit cherry-picked? After I did the initial reproduction with 1.14 here, I switched to the 1.14.1 tag with the same data files, and the sync still failed for me. I'll look more into it when I get a chance.
Author
Owner

@dani-garcia commented on GitHub (Apr 11, 2020):

Ah you are right, I just checked out that commit specifically, thought the 1.14.1 tag was after it, not before.

Yes, then version 1.14.1 shouldn't work but any commit after adf4782 should work. If you can confirm I'll tag a new version 1.14.2.

<!-- gh-comment-id:612478938 --> @dani-garcia commented on GitHub (Apr 11, 2020): Ah you are right, I just checked out that commit specifically, thought the 1.14.1 tag was after it, not before. Yes, then version 1.14.1 shouldn't work but any commit after adf4782 should work. If you can confirm I'll tag a new version 1.14.2.
Author
Owner

@jjlin commented on GitHub (Apr 11, 2020):

After 1.14.1 didn't work, I pulled and switched to the latest tag, which also didn't work. At that time, I verified that the /api/sync response was still returning "Data": null, which was why I said adf4782 wasn't working. But maybe I'm somehow mistaken, I'll check again later.

<!-- gh-comment-id:612480442 --> @jjlin commented on GitHub (Apr 11, 2020): After 1.14.1 didn't work, I pulled and switched to the `latest` tag, which also didn't work. At that time, I verified that the `/api/sync` response was still returning `"Data": null`, which was why I said adf4782 wasn't working. But maybe I'm somehow mistaken, I'll check again later.
Author
Owner

@jjlin commented on GitHub (Apr 11, 2020):

I tried the test again, and it worked this time with latest. I also added some print statements and it looks like everything is working as expected. Maybe it was a caching issue, or I was just up too late. Sorry about that, I'll update the previous comment to say it's fixed in 1.14.2.

<!-- gh-comment-id:612514779 --> @jjlin commented on GitHub (Apr 11, 2020): I tried the test again, and it worked this time with `latest`. I also added some print statements and it looks like everything is working as expected. Maybe it was a caching issue, or I was just up too late. Sorry about that, I'll update the previous comment to say it's fixed in 1.14.2.
Author
Owner

@jjlin commented on GitHub (Jul 12, 2020):

There's now a related fix upstream in the mobile client (bitwarden/mobile#1008), though I'm not sure that it handles this issue completely.

<!-- gh-comment-id:657255547 --> @jjlin commented on GitHub (Jul 12, 2020): There's now a related fix upstream in the mobile client (bitwarden/mobile#1008), though I'm not sure that it handles this issue completely.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/vaultwarden#13442