<!-- gh-comment-id:3128470668 -->
@tycho commented on GitHub (Jul 28, 2025):
Additionally, this line makes it hard to use other S3-compatible implementations like Garage:
https://github.com/dani-garcia/vaultwarden/blob/a0198d8d7cadc666643a4e32a2a8d4217038ea3a/src/config.rs#L1240
I wonder if we can look at the bucket config to determine if we should call these methods or not. We definitely want these options if it’s a real aws s3 bucket. If it’s not, then we can skip calling these methods.
I can look into this within the next few days and report back here on what I think we should do.
<!-- gh-comment-id:3128491479 -->
@txase commented on GitHub (Jul 28, 2025):
I wonder if we can look at the bucket config to determine if we should call these methods or not. We definitely want these options if it’s a real aws s3 bucket. If it’s not, then we can skip calling these methods.
I can look into this within the next few days and report back here on what I think we should do.
So maybe not worth to spend too many resources and effort on a software that will be not so popular overall anymore?
<!-- gh-comment-id:3130832753 -->
@rezzorix commented on GitHub (Jul 29, 2025):
Just wondering... isnt MiniO something to avoid or move away from anyway considering what the developers recently pulled?
Just one example; https://www.reddit.com/r/selfhosted/comments/1kva3pw/avoid_minio_developers_introduce_trojan_horse/
So maybe not worth to spend too many resources and effort on a software that will be not so popular overall anymore?
The aws s3 also support Use Path Style, so consider setting Use Path Style to true.
There are many S3-compatible implementations, such as Cloudflare R2, Cephfs, SeaweedFS, Garage, and minio.
<!-- gh-comment-id:3131967413 -->
@Asutorufa commented on GitHub (Jul 29, 2025):
The `aws s3` also support `Use Path Style`, so consider setting `Use Path Style` to true.
There are many S3-compatible implementations, such as `Cloudflare R2`, `Cephfs`, `SeaweedFS`, `Garage`, and `minio`.
AWS has said they are deprecating path style hosting. That said, it’s been so long now that it’s unclear if they really will do it. But it is clear that they suggest virtual host style hosting is the preferred way to operate.
<!-- gh-comment-id:3132530326 -->
@txase commented on GitHub (Jul 29, 2025):
AWS has said they are [deprecating path style hosting](https://aws.amazon.com/blogs/storage/update-to-amazon-s3-path-deprecation-plan/). That said, it’s been so long now that it’s unclear if they really will do it. But it is clear that they suggest virtual host style hosting is the preferred way to operate.
I have now reviewed s3 and a few compatible services, including minio and garage. I think we need an extensible solution for enabling s3 features, as each service supports different sets of capabilities. @tycho @BlackDex: Please respond with agreement or concerns with the following proposal. 🙏
Proposal
We currently support a URI for file paths for s3-compatible stores where the URI starts with s3://:
We can add support for querystring parameters for s3 features, which would be similar to how many database connection strings support parameters. For example, we could have the following for minio:
We could set a provider parameter: s3://my-minio-bucket?provider=minio to determine which s3 features to enable. However, I think it is likely easier to maintain per-feature params over time (e.g. if minio began supporting intelligent tiering tomorrow, what should the Vaultwarden provider=minio functionality do?). It also means each service provider must have its own hardcoded set of features, and/or users must map new service providers to other existing service providers already implemented in Vaultwarden that enable a compatible set of features.
Alternative Proposal 2: Provider Protocols
We could use the URI protocol to specify the provider type: minio://my-minio-bucket. But this has the same long-term maintenance issue as the above provider parameter alternative proposal.
Alternative Proposal 3: Dynamic Checks
When we first access an s3-compatible service we could check whether it supports various features. However, this may require multiple requests and the possibility of mutating the underlying s3 store. For example, there is no way to check whether intelligent tiering is available other than to attempt to create an object. This will also increase startup latency, which is a problem when running Vaultwarden in a serverless FaaS (Functions-as-a-Service) service like AWS Lambda.
<!-- gh-comment-id:3133134570 -->
@txase commented on GitHub (Jul 29, 2025):
I have now reviewed s3 and a few compatible services, including minio and garage. I think we need an extensible solution for enabling s3 features, as each service supports different sets of capabilities. @tycho @BlackDex: Please respond with agreement or concerns with the following proposal. 🙏
## Proposal
We currently support a URI for file paths for s3-compatible stores where the URI starts with `s3://`:
https://github.com/dani-garcia/vaultwarden/blob/5d84f17600e179280e44c391b92ee9eecc2b7cdc/src/config.rs#L1173
We can add support for querystring parameters for s3 features, which would be similar to how many database connection strings support parameters. For example, we could have the following for minio:
`s3://my-minio-bucket?storage-class=STANDARD`
And for garage:
`s3://my-garage-bucket?path-style-requests`
An example of this parameterization approach can be found in the [Go Cloud Development Kit](https://gocloud.dev/howto/blob/#s3).
### Alternative Proposal 1: Provider Parameters
We could set a provider parameter: `s3://my-minio-bucket?provider=minio` to determine which s3 features to enable. However, I think it is likely easier to maintain per-feature params over time (e.g. if minio began supporting intelligent tiering tomorrow, what should the Vaultwarden `provider=minio` functionality do?). It also means each service provider must have its own hardcoded set of features, and/or users must map new service providers to other existing service providers already implemented in Vaultwarden that enable a compatible set of features.
### Alternative Proposal 2: Provider Protocols
We could use the URI protocol to specify the provider type: `minio://my-minio-bucket`. But this has the same long-term maintenance issue as the above provider parameter alternative proposal.
### Alternative Proposal 3: Dynamic Checks
When we first access an s3-compatible service we could check whether it supports various features. However, this may require multiple requests and the possibility of mutating the underlying s3 store. For example, there is no way to check whether intelligent tiering is available other than to attempt to create an object. This will also increase startup latency, which is a problem when running Vaultwarden in a serverless FaaS (Functions-as-a-Service) service like AWS Lambda.
I think your first proposal looks pretty good and it also mirrors something that OpenDAL itself wants to implement in the future: https://github.com/apache/opendal/issues/5445. That seems like it would resolve this issue for us, but it's not implemented yet. I'd recommend to try to keep it similar to the OpenDAL config naming, so we can migrate over once that's released, so something like might work best:
<!-- gh-comment-id:3133176620 -->
@dani-garcia commented on GitHub (Jul 29, 2025):
I think your first proposal looks pretty good and it also mirrors something that OpenDAL itself wants to implement in the future: https://github.com/apache/opendal/issues/5445. That seems like it would resolve this issue for us, but it's not implemented yet. I'd recommend to try to keep it similar to the [OpenDAL config](https://github.com/apache/opendal/blob/main/core/src/services/s3/config.rs) naming, so we can migrate over once that's released, so something like might work best:
```
s3://my-minio-bucket?default_storage_class=STANDARD
s3://my-garage-bucket?enable_virtual_host_style=false
```
This reminds me that @Xuanwo chimed in to offer guidance on integrating OpenDAL in #5626. @Xuanwo: Can you confirm that this is the best approach, and that the specific params and values @dani-garcia provided above would be correct for future OpenDAL compatibility?
<!-- gh-comment-id:3133192783 -->
@txase commented on GitHub (Jul 29, 2025):
@dani-garcia Oh, good find!
This reminds me that @Xuanwo chimed in to offer guidance on integrating OpenDAL in #5626. @Xuanwo: Can you confirm that this is the best approach, and that the specific params and values @dani-garcia provided above would be correct for future OpenDAL compatibility?
Hi, I think the parameters proposed by @dani-garcia are correct and compatible.
<!-- gh-comment-id:3133207785 -->
@Xuanwo commented on GitHub (Jul 29, 2025):
Hi, I think the parameters proposed by @dani-garcia are correct and compatible.
Looks good to me too. And, it also mimics the same way as is done for Diesel's Database connections.
<!-- gh-comment-id:3133210080 -->
@BlackDex commented on GitHub (Jul 29, 2025):
Looks good to me too. And, it also mimics the same way as is done for Diesel's Database connections.
@tycho: Is this something you would have an interest in drafting a PR for to fix your issue? If you're not able (e.g. you're not familiar enough with Rust), I can probably get to it sometime in the next week. But I don't want to steal away an opportunity for you to have fun getting it working :).
<!-- gh-comment-id:3133223389 -->
@txase commented on GitHub (Jul 29, 2025):
Great!
@tycho: Is this something you would have an interest in drafting a PR for to fix your issue? If you're not able (e.g. you're not familiar enough with Rust), I can probably get to it sometime in the next week. But I don't want to steal away an opportunity for you to have fun getting it working :).
I might be able to, however I've just started a new job so I might not have time for a bit. If someone can get something done sooner, I'd really appreciate if they could work on it!
I think the best path forward is the initial proposal by @txase, with specific functionality overrides in query strings. We could customize things like:
virtual_host_style, 1 or 0, to enable use of virtual host-based vs. path-based respectively
storage_class, supporting any option described in the OpenDAL docs, though we can probably just pass it through directly as a string and let OpenDAL decide whether it's valid
access_key_id and secret_access_key, it might make sense to allow explicitly defining credentials in the connection string, instead of relying on the AWS CLI being configured, especially in a service account like Vaultwarden would typically use (that was a pain point when setting this up for me actually)
endpoint_uri, another useful one for people that don't have the AWS CLI configured -- when using an S3-compatible self-hosted option, you don't want to use the default https://s3.amazonaws.com endpoint, you want to use a custom one. Right now to do that you have to plug it into the AWS CLI config, so having an explicit option would be good!
So a complete S3 storage URI would probably look something like:
<!-- gh-comment-id:3133828252 -->
@tycho commented on GitHub (Jul 29, 2025):
I might be able to, **however** I've just started a new job so I might not have time for a bit. If someone can get something done sooner, I'd really appreciate if they could work on it!
I think the best path forward is the initial proposal by @txase, with specific functionality overrides in query strings. We could customize things like:
- `virtual_host_style`, 1 or 0, to enable use of virtual host-based vs. path-based respectively
- `storage_class`, supporting any option described in the [OpenDAL docs](https://opendal.apache.org/docs/rust/opendal/services/struct.S3.html#method.default_storage_class), though we can probably just pass it through directly as a string and let OpenDAL decide whether it's valid
- `access_key_id` and `secret_access_key`, it might make sense to allow explicitly defining credentials in the connection string, instead of relying on the AWS CLI being configured, especially in a service account like Vaultwarden would typically use (that was a pain point when setting this up for me actually)
- `endpoint_uri`, another useful one for people that don't have the AWS CLI configured -- when using an S3-compatible self-hosted option, you don't want to use the default `https://s3.amazonaws.com` endpoint, you want to use a custom one. Right now to do that you have to plug it into the AWS CLI config, so having an explicit option would be good!
So a complete S3 storage URI would probably look something like:
```
s3://bucket-name?endpoint_uri=https%3A%2F%2Fminio.example.com&access_key_id=SOME_ACCESS_KEY_ID&secret_access_key=SOME_SECRET_ACCESS_KEY&storage_class=STANDARD&virtual_host_style=0
```
access_key_id and secret_access_key, it might make sense to allow explicitly defining credentials in the connection string, instead of relying on the AWS CLI being configured, especially in a service account like Vaultwarden would typically use (that was a pain point when setting this up for me actually)
Hi, OpenDAL supports ENV, IMDSv2, and many other credential loaders, so usually you can just use IAM to configure them instead of static keys, which are insecure.
<!-- gh-comment-id:3134599321 -->
@Xuanwo commented on GitHub (Jul 30, 2025):
> * `access_key_id` and `secret_access_key`, it might make sense to allow explicitly defining credentials in the connection string, instead of relying on the AWS CLI being configured, especially in a service account like Vaultwarden would typically use (that was a pain point when setting this up for me actually)
Hi, OpenDAL supports ENV, IMDSv2, and many other credential loaders, so usually you can just use IAM to configure them instead of static keys, which are insecure.
PR is posted: #6127. Testing from others using non-AWS S3 services would be greatly appreciated :).
<!-- gh-comment-id:3148676607 -->
@txase commented on GitHub (Aug 3, 2025):
PR is posted: #6127. Testing from others using non-AWS S3 services would be greatly appreciated :).
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Originally created by @tycho on GitHub (Jul 28, 2025).
Original GitHub issue: https://github.com/dani-garcia/vaultwarden/issues/6112
This line:
https://github.com/dani-garcia/vaultwarden/blob/a0198d8d7cadc666643a4e32a2a8d4217038ea3a/src/config.rs#L1243
Causes problems if using Minio as the S3 endpoint:
Can that storage class configuration be made user-configurable or even optional, please?
@tycho commented on GitHub (Jul 28, 2025):
Additionally, this line makes it hard to use other S3-compatible implementations like Garage:
https://github.com/dani-garcia/vaultwarden/blob/a0198d8d7cadc666643a4e32a2a8d4217038ea3a/src/config.rs#L1240
@BlackDex commented on GitHub (Jul 28, 2025):
@txase, you might have some ideas here?
@txase commented on GitHub (Jul 28, 2025):
I wonder if we can look at the bucket config to determine if we should call these methods or not. We definitely want these options if it’s a real aws s3 bucket. If it’s not, then we can skip calling these methods.
I can look into this within the next few days and report back here on what I think we should do.
@rezzorix commented on GitHub (Jul 29, 2025):
Just wondering... isnt MiniO something to avoid or move away from anyway considering what the developers recently pulled?
Just one example; https://www.reddit.com/r/selfhosted/comments/1kva3pw/avoid_minio_developers_introduce_trojan_horse/
So maybe not worth to spend too many resources and effort on a software that will be not so popular overall anymore?
@Asutorufa commented on GitHub (Jul 29, 2025):
The
aws s3also supportUse Path Style, so consider settingUse Path Styleto true.There are many S3-compatible implementations, such as
Cloudflare R2,Cephfs,SeaweedFS,Garage, andminio.@txase commented on GitHub (Jul 29, 2025):
AWS has said they are deprecating path style hosting. That said, it’s been so long now that it’s unclear if they really will do it. But it is clear that they suggest virtual host style hosting is the preferred way to operate.
@txase commented on GitHub (Jul 29, 2025):
I have now reviewed s3 and a few compatible services, including minio and garage. I think we need an extensible solution for enabling s3 features, as each service supports different sets of capabilities. @tycho @BlackDex: Please respond with agreement or concerns with the following proposal. 🙏
Proposal
We currently support a URI for file paths for s3-compatible stores where the URI starts with
s3://:https://github.com/dani-garcia/vaultwarden/blob/5d84f17600e179280e44c391b92ee9eecc2b7cdc/src/config.rs#L1173
We can add support for querystring parameters for s3 features, which would be similar to how many database connection strings support parameters. For example, we could have the following for minio:
s3://my-minio-bucket?storage-class=STANDARDAnd for garage:
s3://my-garage-bucket?path-style-requestsAn example of this parameterization approach can be found in the Go Cloud Development Kit.
Alternative Proposal 1: Provider Parameters
We could set a provider parameter:
s3://my-minio-bucket?provider=minioto determine which s3 features to enable. However, I think it is likely easier to maintain per-feature params over time (e.g. if minio began supporting intelligent tiering tomorrow, what should the Vaultwardenprovider=miniofunctionality do?). It also means each service provider must have its own hardcoded set of features, and/or users must map new service providers to other existing service providers already implemented in Vaultwarden that enable a compatible set of features.Alternative Proposal 2: Provider Protocols
We could use the URI protocol to specify the provider type:
minio://my-minio-bucket. But this has the same long-term maintenance issue as the above provider parameter alternative proposal.Alternative Proposal 3: Dynamic Checks
When we first access an s3-compatible service we could check whether it supports various features. However, this may require multiple requests and the possibility of mutating the underlying s3 store. For example, there is no way to check whether intelligent tiering is available other than to attempt to create an object. This will also increase startup latency, which is a problem when running Vaultwarden in a serverless FaaS (Functions-as-a-Service) service like AWS Lambda.
@dani-garcia commented on GitHub (Jul 29, 2025):
I think your first proposal looks pretty good and it also mirrors something that OpenDAL itself wants to implement in the future: https://github.com/apache/opendal/issues/5445. That seems like it would resolve this issue for us, but it's not implemented yet. I'd recommend to try to keep it similar to the OpenDAL config naming, so we can migrate over once that's released, so something like might work best:
@txase commented on GitHub (Jul 29, 2025):
@dani-garcia Oh, good find!
This reminds me that @Xuanwo chimed in to offer guidance on integrating OpenDAL in #5626. @Xuanwo: Can you confirm that this is the best approach, and that the specific params and values @dani-garcia provided above would be correct for future OpenDAL compatibility?
@Xuanwo commented on GitHub (Jul 29, 2025):
Hi, I think the parameters proposed by @dani-garcia are correct and compatible.
@BlackDex commented on GitHub (Jul 29, 2025):
Looks good to me too. And, it also mimics the same way as is done for Diesel's Database connections.
@txase commented on GitHub (Jul 29, 2025):
Great!
@tycho: Is this something you would have an interest in drafting a PR for to fix your issue? If you're not able (e.g. you're not familiar enough with Rust), I can probably get to it sometime in the next week. But I don't want to steal away an opportunity for you to have fun getting it working :).
@tycho commented on GitHub (Jul 29, 2025):
I might be able to, however I've just started a new job so I might not have time for a bit. If someone can get something done sooner, I'd really appreciate if they could work on it!
I think the best path forward is the initial proposal by @txase, with specific functionality overrides in query strings. We could customize things like:
virtual_host_style, 1 or 0, to enable use of virtual host-based vs. path-based respectivelystorage_class, supporting any option described in the OpenDAL docs, though we can probably just pass it through directly as a string and let OpenDAL decide whether it's validaccess_key_idandsecret_access_key, it might make sense to allow explicitly defining credentials in the connection string, instead of relying on the AWS CLI being configured, especially in a service account like Vaultwarden would typically use (that was a pain point when setting this up for me actually)endpoint_uri, another useful one for people that don't have the AWS CLI configured -- when using an S3-compatible self-hosted option, you don't want to use the defaulthttps://s3.amazonaws.comendpoint, you want to use a custom one. Right now to do that you have to plug it into the AWS CLI config, so having an explicit option would be good!So a complete S3 storage URI would probably look something like:
@Xuanwo commented on GitHub (Jul 30, 2025):
Hi, OpenDAL supports ENV, IMDSv2, and many other credential loaders, so usually you can just use IAM to configure them instead of static keys, which are insecure.
@txase commented on GitHub (Aug 3, 2025):
PR is posted: #6127. Testing from others using non-AWS S3 services would be greatly appreciated :).