From 990411a90f0120f70fcbc896c5a2142d5e32cee7 Mon Sep 17 00:00:00 2001 From: Vinta Chen Date: Sun, 8 Nov 2026 22:35:51 +0800 Subject: [PATCH] docs: add temporalio to Job Schedulers intro The intro did not mention a README entry added after it was written, so readers got no advice on it. Co-Authored-By: Claude --- website/data/category_intros/job-schedulers.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/website/data/category_intros/job-schedulers.md b/website/data/category_intros/job-schedulers.md index f16650e8..4d2c81be 100644 --- a/website/data/category_intros/job-schedulers.md +++ b/website/data/category_intros/job-schedulers.md @@ -7,6 +7,7 @@ How to choose: - Your Python functions as pipelines, with tasks created at runtime: Prefect - Pipelines built around the data assets they produce, like tables and models: Dagster - A simple loop of periodic jobs in one script: schedule +- Long-running business processes that survive crashes, with retries: Temporal APScheduler [schedules your Python code to run later](https://github.com/agronholm/apscheduler/blob/3.x/README.rst), once or periodically. It runs inside your existing application, not as a service. Give each job [a trigger](https://apscheduler.readthedocs.io/en/latest/userguide.html#choosing-the-right-scheduler-job-store-s-executor-s-and-trigger-s): `date` runs it once, `interval` at fixed intervals, and `cron` at set times of day. Jobs [live in memory by default](https://apscheduler.readthedocs.io/en/latest/userguide.html#basic-concepts). When they must survive restarts and crashes, add a persistent job store. @@ -18,4 +19,6 @@ Prefect [turns your Python functions into data pipelines](https://docs.prefect.i Dagster is [a data orchestrator built for data engineers](https://docs.dagster.io/), with lineage and observability built in. You [declare data assets like tables, datasets, and ML models as Python functions](https://github.com/dagster-io/dagster). Dagster runs them at the right time to keep them up to date. If you're just starting out, its docs [strongly recommend assets rather than ops](https://docs.dagster.io/guides/build/ops). Run assets on [a cron schedule](https://docs.dagster.io/guides/automate/schedules). -In an orchestrator, make every task safe to run twice. Airflow [can retry a failed task](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#creating-a-task), so its docs say a task should produce the same outcome on every re-run. Prefect tasks are [retryable units of work](https://docs.prefect.io/v3/concepts/tasks) too. +Temporal [runs your application code as a Durable Execution](https://docs.temporal.io/evaluate/understanding-temporal#durable-execution). Once a workflow starts, it runs to completion, even when that takes a year. If its process crashes, another process picks it up where it stopped. With the Temporal Python SDK, [put `@activity.defn` on each function that does one action, like calling an API, and `@workflow.defn` on the class that orchestrates them](https://docs.temporal.io/develop/python/set-up-your-local-python#run-hello-world-test-your-installation). Keep anything that talks to the outside world in an activity, since Temporal [reruns workflow code to rebuild its state](https://docs.temporal.io/evaluate/understanding-temporal#workflows). A failed activity [retries with backoff](https://docs.temporal.io/evaluate/understanding-temporal#activities). [A worker on your own infrastructure](https://docs.temporal.io/evaluate/understanding-temporal#workers) runs your code and asks the Temporal Service for work. [The Temporal Service is the Temporal Server plus a database](https://docs.temporal.io/temporal#temporal-platform), and you run it yourself or use Temporal Cloud. + +In an orchestrator, make every task safe to run twice. Airflow [can retry a failed task](https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#creating-a-task), so its docs say a task should produce the same outcome on every re-run. Prefect tasks are [retryable units of work](https://docs.prefect.io/v3/concepts/tasks) too. Temporal [recommends idempotent activities](https://docs.temporal.io/activity-definition#idempotency), since a retried activity can run more than once.