Dagster is a job orchestration framework that is used to build data assets. It is a great tool, works on multiple platforms, and is very easy to set it up. It has a lot of useful features, such as:
Managing data assets with code, building python-based data jobs to load, transfer, and transform data
Integrates with dbt and provides a user interface to run dbt models
Provides a UI to monitor jobs, debug runs, inspect assets, or even launch backfills
Multiplatform - works with Windows and Mac by just installing from pip (no other dependencies)
It has multiple executors to run tasks, including in-process, multi-process, Dask, Celery, Docker, and Kubernetes.
What's the difference between this Python project and similar ones?
Enumerate comparisons.
One great comparison for dagster is Airflow - but Dagster concentrates more on orchestrating data assets, including table materializations, etc
It is not just a scheduler, but a framework to build jobs using Dagster API
Anyone who agrees with this pull request could submit an Approve review to it.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/vinta/awesome-python/pull/2300
**Author:** [@metinsenturk](https://github.com/metinsenturk)
**Created:** 1/7/2023
**Status:** ❌ Closed
**Base:** `master` ← **Head:** `master`
---
### 📝 Commits (1)
- [`328766d`](https://github.com/vinta/awesome-python/commit/328766d9654e4a3dac54327c3cba4a7a8030260f) Add dagster
### 📊 Changes
**1 file changed** (+1 additions, -0 deletions)
<details>
<summary>View changed files</summary>
📝 `README.md` (+1 -0)
</details>
### 📄 Description
## What is this Python project?
Dagster is a job orchestration framework that is used to build data assets. It is a great tool, works on multiple platforms, and is very easy to set it up. It has a lot of useful features, such as:
- Managing data assets with code, building python-based data jobs to load, transfer, and transform data
- Integrates with [dbt](https://www.getdbt.com/) and provides a user interface to run dbt models
- Provides a UI to monitor jobs, debug runs, inspect assets, or even launch backfills
- Multiplatform - works with Windows and Mac by just installing from pip (no other dependencies)
- It has multiple executors to run tasks, including in-process, multi-process, Dask, Celery, Docker, and Kubernetes.
## What's the difference between this Python project and similar ones?
Enumerate comparisons.
- One great comparison for dagster is Airflow - but Dagster concentrates more on orchestrating data assets, including table materializations, etc
- It is not just a scheduler, but a framework to build jobs using Dagster API
Anyone who agrees with this pull request could submit an *Approve* review to it.
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/vinta/awesome-python/pull/2300
Author: @metinsenturk
Created: 1/7/2023
Status: ❌ Closed
Base:
master← Head:master📝 Commits (1)
328766dAdd dagster📊 Changes
1 file changed (+1 additions, -0 deletions)
View changed files
📝
README.md(+1 -0)📄 Description
What is this Python project?
Dagster is a job orchestration framework that is used to build data assets. It is a great tool, works on multiple platforms, and is very easy to set it up. It has a lot of useful features, such as:
What's the difference between this Python project and similar ones?
Enumerate comparisons.
Anyone who agrees with this pull request could submit an Approve review to it.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.