[PR #2690] [CLOSED] Add CloudFile-Mover to awesome-python #2003

New Issue

GiteaMirror · 2025-11-06T13:27:30-06:00

GiteaMirror commented

2025-11-06 13:27:30 -06:00

📋 Pull Request Information

Original PR: https://github.com/vinta/awesome-python/pull/2690
Author: @RaghavaCh440
Created: 4/27/2025
Status: ❌ Closed

Base: master ← Head: add-cloudfile-mover

📝 Commits (1)

baaa48b Add CloudFile-Mover to awesome-python

📊 Changes

1 file changed (+4 additions, -0 deletions)

View changed files

📝 README.md (+4 -0)

📄 Description

What is this Python project?

Project Description:
CloudFile-Mover is a Python-based cross-cloud large file transfer tool supporting Google Cloud Storage (GCS), Amazon S3, and Azure Blob Storage. It enables efficient and secure file migrations across cloud platforms.

GitHub Repo: https://github.com/RaghavaCh440/cloudfile-mover

Cross-Provider Transfers: Move files between S3, GCS, and Azure Blob Storage in any combination (e.g. S3 → GCS, Azure → S3, etc.).

High Performance via Multithreading: Utilizes multi-threading to download and upload file chunks in parallel, significantly speeding up transfers for large files.

Large File Segmentation: Automatically splits large files into chunks (e.g. using S3 multipart upload, GCS object composition, Azure block blobs) to support very large sizes and improve throughput.

Robustness with Retries: Implements retry logic for network hiccups or transient cloud errors on each chunk transfer.

Progress Bar: Displays a tqdm progress bar showing the transfer progress (with options to suppress it for non-interactive use).

Flexible Logging: Uses Python’s logging to output transfer information. Verbosity can be adjusted (quiet, info, debug) via a CLI flag.

True Move Semantics: Upon successful transfer to the destination, the source object is deleted, mimicking a “move” rather than “copy”.

Secure Authentication: Leverages standard cloud authentication methods (no credentials are ever hardcoded). Supports AWS IAM roles/keys, Google Application Default Credentials, and Azure DefaultAzureCredential for seamless auth.

Command-Line Interface (CLI): Includes a CLI script cloudfile-mover with options such as --threads, --verbose, --no-progress.

Importable Library: Can be used as a Python module with a move_file function for programmatic access.

PyPI-Ready Package: Provided with setup files, tests, and documentation for ease of installation and distribution.

What's the difference between this Python project and similar ones?

This project deals with large file transfers between different Clouds (GCP, AWS and Azure).
It supports multipart and Multithreading feature.
Flexibility in sending concurrent file transfers
We can assign resources like CPU and RAM based on our requirements.
It supports excellent network throughput

Anyone who agrees with this pull request could submit an Approve review to it.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/vinta/awesome-python/pull/2690 **Author:** [@RaghavaCh440](https://github.com/RaghavaCh440) **Created:** 4/27/2025 **Status:** ❌ Closed **Base:** `master` ← **Head:** `add-cloudfile-mover` --- ### 📝 Commits (1) - [`baaa48b`](https://github.com/vinta/awesome-python/commit/baaa48bc1ba4ace2e46bc49a54faca75352749ef) Add CloudFile-Mover to awesome-python ### 📊 Changes **1 file changed** (+4 additions, -0 deletions) <details> <summary>View changed files</summary> 📝 `README.md` (+4 -0) </details> ### 📄 Description ## What is this Python project? Project Description: CloudFile-Mover is a Python-based cross-cloud large file transfer tool supporting Google Cloud Storage (GCS), Amazon S3, and Azure Blob Storage. It enables efficient and secure file migrations across cloud platforms. GitHub Repo: https://github.com/RaghavaCh440/cloudfile-mover Cross-Provider Transfers: Move files between S3, GCS, and Azure Blob Storage in any combination (e.g. S3 → GCS, Azure → S3, etc.). High Performance via Multithreading: Utilizes multi-threading to download and upload file chunks in parallel, significantly speeding up transfers for large files. Large File Segmentation: Automatically splits large files into chunks (e.g. using S3 multipart upload, GCS object composition, Azure block blobs) to support very large sizes and improve throughput. Robustness with Retries: Implements retry logic for network hiccups or transient cloud errors on each chunk transfer. Progress Bar: Displays a tqdm progress bar showing the transfer progress (with options to suppress it for non-interactive use). Flexible Logging: Uses Python’s logging to output transfer information. Verbosity can be adjusted (quiet, info, debug) via a CLI flag. True Move Semantics: Upon successful transfer to the destination, the source object is deleted, mimicking a “move” rather than “copy”. Secure Authentication: Leverages standard cloud authentication methods (no credentials are ever hardcoded). Supports AWS IAM roles/keys, Google Application Default Credentials, and Azure DefaultAzureCredential for seamless auth. Command-Line Interface (CLI): Includes a CLI script cloudfile-mover with options such as --threads, --verbose, --no-progress. Importable Library: Can be used as a Python module with a move_file function for programmatic access. PyPI-Ready Package: Provided with setup files, tests, and documentation for ease of installation and distribution. ## What's the difference between this Python project and similar ones? 1. This project deals with large file transfers between different Clouds (GCP, AWS and Azure). 2. It supports multipart and Multithreading feature. 3. Flexibility in sending concurrent file transfers 4. We can assign resources like CPU and RAM based on our requirements. 5. It supports excellent network throughput Anyone who agrees with this pull request could submit an *Approve* review to it. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2025-11-06 13:27:30 -06:00

GiteaMirror closed this issue

2025-11-06 13:27:30 -06:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/awesome-python#2003