[PR #609] [CLOSED] Add Airbyte template for data pipeline deployment #1949

New Issue

GiteaMirror · 2026-04-20T10:59:55-05:00

GiteaMirror commented

2026-04-20 10:59:55 -05:00

📋 Pull Request Information

Original PR: https://github.com/Dokploy/templates/pull/609
Author: @Copilot
Created: 12/20/2025
Status: ❌ Closed

Base: canary ← Head: copilot/add-airbyte-template

📝 Commits (7)

624f7fa Initial plan
4067b98 Add Airbyte template with complete deployment setup
1a81ad9 Make database and storage credentials configurable for security
52d2fc6 Make temporal database password configurable and document Docker socket requirement
9a350d8 Update docker-compose.yml
db877e7 Fix Temporal DB driver and add missing STORAGE_TYPE environment variable
bda2d4f Update docker-compose.yml

📊 Changes

4 files changed (+173 additions, -0 deletions)

View changed files

➕ blueprints/airbyte/airbyte.png (+0 -0)
➕ blueprints/airbyte/docker-compose.yml (+136 -0)
➕ blueprints/airbyte/template.toml (+20 -0)
📝 meta.json (+17 -0)

📄 Description

What is this PR about?

New PR of Airbyte - an open-source data integration platform for building ETL/ELT pipelines with 200+ connectors.

Template Structure:

Multi-service deployment: webapp (UI), server (API), worker (job executor), temporal (workflow engine), PostgreSQL databases, MinIO (S3-compatible storage)
Version: 1.7.8 (latest stable)
Dynamic credential generation via ${password:32} helper for all database and storage passwords
Domain routing configured for webapp on port 80

Key Configuration:

# template.toml
[variables]
main_domain = "${domain}"
db_password = "${password:32}"
minio_password = "${password:32}"
temporal_db_password = "${password:32}"

[config]
env = [
  "AIRBYTE_DB_PASSWORD=${db_password}",
  "MINIO_ROOT_PASSWORD=${minio_password}",
  "TEMPORAL_DB_PASSWORD=${temporal_db_password}"
]
# Note: The Airbyte worker service requires a Docker socket mount at /var/run/docker.sock.
# This mount is intentionally configured in docker-compose.yml rather than via config.mounts.

Architecture Notes:

Worker mounts /var/run/docker.sock (required for Airbyte to execute connector jobs in isolated containers) - documented in template.toml
Temporal orchestrates workflow execution across distributed services with correct postgres12 driver configuration
MinIO stores pipeline logs and state for job recovery with STORAGE_TYPE: MINIO configuration
No exposed ports; Dokploy handles reverse proxy routing

Security Features:

All database credentials (Airbyte DB, Temporal DB, MinIO) are dynamically generated using secure ${password:32} helper
No hardcoded passwords in the deployment configuration

Configuration Fixes:

Temporal database driver set to postgres12 (Temporal only supports: mysql8, postgres12, postgres12_pgx, cassandra)
STORAGE_TYPE: MINIO added to server and worker services for proper MinIO storage backend configuration

Checklist

Before submitting this PR, please make sure that:

I have read the suggestions in the README.md file https://github.com/Dokploy/templates?tab=readme-ov-file#general-requirements-when-creating-a-template
I have tested the template in my instance, so the maintainers don't spend time trying to figure out what's wrong.
I have added tests that demonstrate that my correction works or that my new feature works.

Closes #609

Screenshots or Videos

N/A - Template deployment validation performed via docker manifest inspect for all 7 images and docker compose config syntax validation. Deployment issues resolved and verified working.

Original prompt

This section details on the original issue you should resolve

<issue_title>Request: Airbyte</issue_title>
<issue_description>### Template Name

Airbyte

Template URL

https://github.com/airbytehq/airbyte

Docker Compose Link

https://docs.airbyte.com/platform/using-airbyte/getting-started/oss-quickstart

Resources

Website: https://airbyte.com/
Docs: https://docs.airbyte.com/

Template Description

The Open Standard for Data Movement.

Build data pipelines that work anywhere. Deploy on-prem, in the cloud, or hybrid - keeping your data sovereign and AI ready.

Will you send a PR to implement it?

No</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes Dokploy/templates#492

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/Dokploy/templates/pull/609 **Author:** [@Copilot](https://github.com/apps/copilot-swe-agent) **Created:** 12/20/2025 **Status:** ❌ Closed **Base:** `canary` ← **Head:** `copilot/add-airbyte-template` --- ### 📝 Commits (7) - [`624f7fa`](https://github.com/Dokploy/templates/commit/624f7fa56cdcea0e09c8ba735d2b59ed848e951d) Initial plan - [`4067b98`](https://github.com/Dokploy/templates/commit/4067b9803e662fb5d09ed33461a1912d5df24157) Add Airbyte template with complete deployment setup - [`1a81ad9`](https://github.com/Dokploy/templates/commit/1a81ad9688a245f63ec46728f7944cb00d41d322) Make database and storage credentials configurable for security - [`52d2fc6`](https://github.com/Dokploy/templates/commit/52d2fc6ff62a2db79ee4e5bb475d40e6f4e23707) Make temporal database password configurable and document Docker socket requirement - [`9a350d8`](https://github.com/Dokploy/templates/commit/9a350d8bf7cb50d8b7cf00e74a86c39731d9ba10) Update docker-compose.yml - [`db877e7`](https://github.com/Dokploy/templates/commit/db877e711f376a3efd3cce018ecb6938cbff2268) Fix Temporal DB driver and add missing STORAGE_TYPE environment variable - [`bda2d4f`](https://github.com/Dokploy/templates/commit/bda2d4f9e8ecb998178799d0f7da19b4b3eb76a7) Update docker-compose.yml ### 📊 Changes **4 files changed** (+173 additions, -0 deletions) <details> <summary>View changed files</summary> ➕ `blueprints/airbyte/airbyte.png` (+0 -0) ➕ `blueprints/airbyte/docker-compose.yml` (+136 -0) ➕ `blueprints/airbyte/template.toml` (+20 -0) 📝 `meta.json` (+17 -0) </details> ### 📄 Description ## What is this PR about? New PR of Airbyte - an open-source data integration platform for building ETL/ELT pipelines with 200+ connectors. **Template Structure:** - Multi-service deployment: webapp (UI), server (API), worker (job executor), temporal (workflow engine), PostgreSQL databases, MinIO (S3-compatible storage) - Version: 1.7.8 (latest stable) - Dynamic credential generation via `${password:32}` helper for all database and storage passwords - Domain routing configured for webapp on port 80 **Key Configuration:** ```toml # template.toml [variables] main_domain = "${domain}" db_password = "${password:32}" minio_password = "${password:32}" temporal_db_password = "${password:32}" [config] env = [ "AIRBYTE_DB_PASSWORD=${db_password}", "MINIO_ROOT_PASSWORD=${minio_password}", "TEMPORAL_DB_PASSWORD=${temporal_db_password}" ] # Note: The Airbyte worker service requires a Docker socket mount at /var/run/docker.sock. # This mount is intentionally configured in docker-compose.yml rather than via config.mounts. ``` **Architecture Notes:** - Worker mounts `/var/run/docker.sock` (required for Airbyte to execute connector jobs in isolated containers) - documented in template.toml - Temporal orchestrates workflow execution across distributed services with correct `postgres12` driver configuration - MinIO stores pipeline logs and state for job recovery with `STORAGE_TYPE: MINIO` configuration - No exposed ports; Dokploy handles reverse proxy routing **Security Features:** - All database credentials (Airbyte DB, Temporal DB, MinIO) are dynamically generated using secure `${password:32}` helper - No hardcoded passwords in the deployment configuration **Configuration Fixes:** - Temporal database driver set to `postgres12` (Temporal only supports: mysql8, postgres12, postgres12_pgx, cassandra) - `STORAGE_TYPE: MINIO` added to server and worker services for proper MinIO storage backend configuration ## Checklist Before submitting this PR, please make sure that: - [x] I have read the suggestions in the README.md file https://github.com/Dokploy/templates?tab=readme-ov-file#general-requirements-when-creating-a-template - [x] I have tested the template in my instance, so the maintainers don't spend time trying to figure out what's wrong. - [ ] I have added tests that demonstrate that my correction works or that my new feature works. ## Issues related (if applicable) Closes #609 ## Screenshots or Videos N/A - Template deployment validation performed via `docker manifest inspect` for all 7 images and `docker compose config` syntax validation. Deployment issues resolved and verified working.  <details> <summary>Original prompt</summary> > > ---- > > *This section details on the original issue you should resolve* > > <issue_title>Request: Airbyte</issue_title> > <issue_description>### Template Name > > Airbyte > > ### Template URL > > https://github.com/airbytehq/airbyte > > ### Docker Compose Link > > https://docs.airbyte.com/platform/using-airbyte/getting-started/oss-quickstart > > ### Resources > > Website: https://airbyte.com/ > Docs: https://docs.airbyte.com/ > > ### Template Description > > The Open Standard for Data Movement. > > Build data pipelines that work anywhere. Deploy on-prem, in the cloud, or hybrid - keeping your data sovereign and AI ready. > > ### Will you send a PR to implement it? > > No</issue_description> > > ## Comments on the Issue (you are @copilot in this section) > > <comments> > </comments> > </details>  - Fixes Dokploy/templates#492  --- 💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

GiteaMirror added the pull-request label 2026-04-20 10:59:55 -05:00

GiteaMirror closed this issue

2026-04-20 10:59:56 -05:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: github-starred/templates#1949