github-starred/android
2
0
Fork 0
mirror of https://github.com/bitwarden/android.git synced 2026-05-06 07:48:22 -05:00
Code Issues 1.1k Packages Projects Releases 122 Wiki Activity

[PR #6091] [CLOSED] (WIP) Optimize test workflows #54523

New Issue
Closed
opened 2026-05-01 20:06:13 -05:00 by GiteaMirror · 0 comments
GiteaMirror commented 2026-05-01 20:06:13 -05:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/bitwarden/android/pull/6091
Author: @SaintPatrck
Created: 10/29/2025
Status: ❌ Closed

Base: main ← Head: optimize-test-workflows

📝 Commits (6)

  • facfec1 Enable Gradle configuration cache for faster builds
  • d6a5f5b Restructure test workflow with parallel job execution (Tier 1)
  • d58de79 Fix coverage aggregation in parallel test workflow
  • 8b72b1e Fix ConcurrentModificationException in coverage report generation
  • 954f39d Extract coverage reports before uploading
  • 11d9537 Move coverage reports to root directory for Codecov upload

📊 Changes

2 files changed (+243 additions, -26 deletions)

View changed files

📝 .github/workflows/test.yml (+241 -26)
📝 gradle.properties (+2 -0)

📄 Description

🎟️ Tracking

📔 Objective

This PR restructures the test workflow to execute tests in parallel across multiple jobs, with the goal of significantly reducing CI runtime. The workflow now splits testing into 5 concurrent jobs instead of a single sequential job.

Workflow Structure

Before: Single job running all tasks sequentially
After: 5 parallel jobs with specialized responsibilities

┌─────────────────────────────────────────────────────────────────┐
│ OLD: Single Job (Sequential Execution)                          │
├─────────────────────────────────────────────────────────────────┤
│ detekt → lint → test all modules → coverage → upload            │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│ NEW: 5 Parallel Jobs (Concurrent Execution)                     │
├─────────────────────────────────────────────────────────────────┤
│ ┌───────────────────┐  ┌─────────────────┐                      │
│ │ Lint & Static     │  │ Test Libraries  │                      │
│ │ Analysis          │  │ (core, data,    │                      │
│ │                   │  │  network, ui,   │                      │
│ └───────────────────┘  │  etc.)          │                      │
│                        │                 │                      │
│ ┌───────────────────┐  └─────────────────┘                      │
│ │ Test App          │                                           │
│ │ (5,447 tests)     │  ┌─────────────────┐                      │
│ │                   │  │ Test            │                      │
│ └───────────────────┘  │ Authenticator   │                      │
│                        │                 │                      │
│                        └─────────────────┘                      │
│                                                                 │
│                        ┌─────────────────┐                      │
│                        │ Aggregate       │                      │
│                        │ Coverage        │                      │
│                        │                 │                      │
│                        └─────────────────┘                      │
└─────────────────────────────────────────────────────────────────┘

Jobs Breakdown

1. lint-and-static-analysis

  • Runs detekt for code quality checks
  • Executes lintStandardDebug and lintDebug
  • Provides early failure detection (typically fastest to fail)

2. test-libraries

  • Tests all library modules: :core, :data, :network, :ui, :authenticatorbridge, :cxf
  • Generates XML coverage reports for each module
  • Represents ~10% of total test suite (423 tests)

3. test-app

  • Tests :app module (5,447 tests - 88.5% of total)
  • Generates XML coverage report
  • Critical path bottleneck (longest-running job)

4. test-authenticator

  • Tests :authenticator module (281 tests - 4.6% of total)
  • Generates XML coverage report

5. aggregate-coverage

  • Downloads coverage reports from all test jobs
  • Uploads merged coverage to codecov.io
  • Depends on: test-libraries, test-app, test-authenticator

Technical Changes

1. Enable Gradle Configuration Cache

File: gradle.properties

 org.gradle.caching=true
+org.gradle.configuration-cache=true
+org.gradle.configuration-cache.problems=warn
 org.gradle.jvmargs=-Xmx4g -XX:+UseParallelGC -Dfile.encoding=UTF-8
 org.gradle.parallel=true

Benefits:

  • Faster Gradle configuration phase through caching
  • Reduced overhead across all jobs
  • Cumulative time savings with repeated builds

2. Restructure Test Workflow

File: .github/workflows/test.yml

Key Changes:

  • Split single test job into 5 specialized jobs
  • Each job has dedicated test and coverage generation
  • Coverage reports generated immediately after tests (same Gradle session)
  • Codecov automatically merges multiple XML reports

Coverage Strategy:

  • Each test job generates its own Kover XML report
  • Reports uploaded as artifacts (coverage-libraries, coverage-app, coverage-authenticator)
  • Aggregation job downloads all reports and uploads to codecov
  • No dependency on Fastlane for coverage merging (codecov handles it)

3. Combined Test + Coverage Commands

Tests and coverage generation run in a single Gradle invocation to ensure Kover has access to test execution data:

# App module
./gradlew :app:testStandardDebug :app:koverXmlReportStandardDebug

# Authenticator module
./gradlew :authenticator:testDebug :authenticator:koverXmlReportDebug

# Library modules (all in one command)
./gradlew \
  :core:testDebug :core:koverXmlReportDebug \
  :data:testDebug :data:koverXmlReportDebug \
  :network:testDebug :network:koverXmlReportDebug \
  :ui:testDebug :ui:koverXmlReportDebug \
  :authenticatorbridge:testDebug :authenticatorbridge:koverXmlReportDebug \
  :cxf:testDebug :cxf:koverXmlReportDebug

Running both tasks together in the same Gradle daemon session ensures the coverage report generation can access the test execution data produced by the test task.

Performance Impact

Expected Improvements

Metric Before After
Critical Path All tasks sequential Longest job (test-app) - better utilization
Failure Detection Late (after all tests) Early (lint fails first) - faster feedback
Resource Usage 1 runner 5 concurrent runners - better parallelization

Test Distribution

  • 6,151 total tests across all modules
  • 5,447 tests (88.5%) in :app module (still bottleneck)
  • 423 tests (6.9%) in library modules (completes quickly)
  • 281 tests (4.6%) in :authenticator module

Testing & Validation

Local Testing

  • ✅ Configuration cache enabled and tested locally
  • ✅ Test execution with coverage generation validated
  • ✅ All modules generate XML reports correctly

CI Validation Checklist

  • All 5 jobs run in parallel
  • Test failures reported correctly per-job
  • Coverage artifacts upload successfully
  • Codecov receives and merges coverage data
  • Total coverage % matches baseline
  • Performance improvement achieved

Future Optimization Opportunities

This PR implements Tier 1 (Quick Wins) optimizations. Additional improvements are possible:

Tier 2: App Module Sharding

  • Split :app module's 5,447 tests across 3-4 parallel jobs
  • Package-based sharding (auth, vault, platform, UI)
  • Further reduce critical path bottleneck

Tier 3: Strategic Investments

  • Remote Gradle build cache (Gradle Enterprise or self-hosted)
  • Predictive test selection (run only affected tests on PRs)
  • Feature module extraction (break up monolithic :app)

Risks & Mitigations

Risk: Coverage Reporting Changes

Mitigation: Codecov's XML merging is battle-tested and widely used. Coverage % should match previous implementation.

Risk: Workflow Complexity Increase

Mitigation: Clear job naming, comprehensive documentation, each job remains simple and focused.

Risk: More CI Runner Usage

Impact: Uses 5 concurrent runners instead of 1. Total runner-minutes may be similar or slightly higher, but wall-clock time is significantly reduced.

Risk: Configuration Cache Issues

Mitigation: Enabled with problems=warn to identify compatibility issues without failing builds. Can be disabled if problems arise.

Rollback Plan

If issues arise:

  1. Revert this PR to return to single-job workflow
  2. Disable configuration cache by reverting gradle.properties changes
  3. All changes are isolated to workflow and properties files

📸 Screenshots

Coming soon!

⏰ Reminders before review

  • Contributor guidelines followed
  • All formatters and local linters executed and passed
  • Written new unit and / or integration tests where applicable
  • Protected functional changes with optionality (feature flags)
  • Used internationalization (i18n) for all UI strings
  • CI builds passed
  • Communicated to DevOps any deployment requirements
  • Updated any necessary documentation (Confluence, contributing docs) or informed the documentation team

🦮 Reviewer guidelines

  • 👍 (:+1:) or similar for great changes
  • 📝 (:memo:) or ℹ️ (:information_source:) for notes or general info
  • ❓ (:question:) for questions
  • 🤔 (:thinking:) or 💭 (:thought_balloon:) for more open inquiry that's not quite a confirmed issue and could potentially benefit from discussion
  • 🎨 (:art:) for suggestions / improvements
  • ❌ (:x:) or ⚠️ (:warning:) for more significant problems or concerns needing attention
  • 🌱 (:seedling:) or ♻️ (:recycle:) for future improvements or indications of technical debt
  • ⛏ (:pick:) for minor or nitpick changes

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/bitwarden/android/pull/6091 **Author:** [@SaintPatrck](https://github.com/SaintPatrck) **Created:** 10/29/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `optimize-test-workflows` --- ### 📝 Commits (6) - [`facfec1`](https://github.com/bitwarden/android/commit/facfec128dcc79cff9a7d1f0bf1cf538a5d87342) Enable Gradle configuration cache for faster builds - [`d6a5f5b`](https://github.com/bitwarden/android/commit/d6a5f5bbc7e26db1b379a68b89dd642361c898be) Restructure test workflow with parallel job execution (Tier 1) - [`d58de79`](https://github.com/bitwarden/android/commit/d58de79ecadcf8d445b2d50a13679dd9bf795823) Fix coverage aggregation in parallel test workflow - [`8b72b1e`](https://github.com/bitwarden/android/commit/8b72b1e29ef937a695a8ca8b7bf3461cd3314c53) Fix ConcurrentModificationException in coverage report generation - [`954f39d`](https://github.com/bitwarden/android/commit/954f39d4323d0102e8bf5062f0d0d7b5df13fad7) Extract coverage reports before uploading - [`11d9537`](https://github.com/bitwarden/android/commit/11d9537b058b6fefebab78f4f28b20f6780201d2) Move coverage reports to root directory for Codecov upload ### 📊 Changes **2 files changed** (+243 additions, -26 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/test.yml` (+241 -26) 📝 `gradle.properties` (+2 -0) </details> ### 📄 Description ## 🎟️ Tracking <!-- Paste the link to the Jira or GitHub issue or otherwise describe / point to where this change is coming from. --> ## 📔 Objective This PR restructures the test workflow to execute tests in parallel across multiple jobs, with the goal of significantly reducing CI runtime. The workflow now splits testing into 5 concurrent jobs instead of a single sequential job. ### Workflow Structure **Before:** Single job running all tasks sequentially **After:** 5 parallel jobs with specialized responsibilities ``` ┌─────────────────────────────────────────────────────────────────┐ │ OLD: Single Job (Sequential Execution) │ ├─────────────────────────────────────────────────────────────────┤ │ detekt → lint → test all modules → coverage → upload │ └─────────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────────┐ │ NEW: 5 Parallel Jobs (Concurrent Execution) │ ├─────────────────────────────────────────────────────────────────┤ │ ┌───────────────────┐ ┌─────────────────┐ │ │ │ Lint & Static │ │ Test Libraries │ │ │ │ Analysis │ │ (core, data, │ │ │ │ │ │ network, ui, │ │ │ └───────────────────┘ │ etc.) │ │ │ │ │ │ │ ┌───────────────────┐ └─────────────────┘ │ │ │ Test App │ │ │ │ (5,447 tests) │ ┌─────────────────┐ │ │ │ │ │ Test │ │ │ └───────────────────┘ │ Authenticator │ │ │ │ │ │ │ └─────────────────┘ │ │ │ │ ┌─────────────────┐ │ │ │ Aggregate │ │ │ │ Coverage │ │ │ │ │ │ │ └─────────────────┘ │ └─────────────────────────────────────────────────────────────────┘ ``` ### Jobs Breakdown #### 1. `lint-and-static-analysis` - Runs `detekt` for code quality checks - Executes `lintStandardDebug` and `lintDebug` - Provides early failure detection (typically fastest to fail) #### 2. `test-libraries` - Tests all library modules: `:core`, `:data`, `:network`, `:ui`, `:authenticatorbridge`, `:cxf` - Generates XML coverage reports for each module - Represents ~10% of total test suite (423 tests) #### 3. `test-app` - Tests `:app` module (5,447 tests - 88.5% of total) - Generates XML coverage report - Critical path bottleneck (longest-running job) #### 4. `test-authenticator` - Tests `:authenticator` module (281 tests - 4.6% of total) - Generates XML coverage report #### 5. `aggregate-coverage` - Downloads coverage reports from all test jobs - Uploads merged coverage to codecov.io - Depends on: `test-libraries`, `test-app`, `test-authenticator` ## Technical Changes ### 1. Enable Gradle Configuration Cache **File:** `gradle.properties` ```diff org.gradle.caching=true +org.gradle.configuration-cache=true +org.gradle.configuration-cache.problems=warn org.gradle.jvmargs=-Xmx4g -XX:+UseParallelGC -Dfile.encoding=UTF-8 org.gradle.parallel=true ``` **Benefits:** - Faster Gradle configuration phase through caching - Reduced overhead across all jobs - Cumulative time savings with repeated builds ### 2. Restructure Test Workflow **File:** `.github/workflows/test.yml` **Key Changes:** - Split single `test` job into 5 specialized jobs - Each job has dedicated test and coverage generation - Coverage reports generated immediately after tests (same Gradle session) - Codecov automatically merges multiple XML reports **Coverage Strategy:** - Each test job generates its own Kover XML report - Reports uploaded as artifacts (`coverage-libraries`, `coverage-app`, `coverage-authenticator`) - Aggregation job downloads all reports and uploads to codecov - No dependency on Fastlane for coverage merging (codecov handles it) ### 3. Combined Test + Coverage Commands Tests and coverage generation run in a single Gradle invocation to ensure Kover has access to test execution data: ```yaml # App module ./gradlew :app:testStandardDebug :app:koverXmlReportStandardDebug # Authenticator module ./gradlew :authenticator:testDebug :authenticator:koverXmlReportDebug # Library modules (all in one command) ./gradlew \ :core:testDebug :core:koverXmlReportDebug \ :data:testDebug :data:koverXmlReportDebug \ :network:testDebug :network:koverXmlReportDebug \ :ui:testDebug :ui:koverXmlReportDebug \ :authenticatorbridge:testDebug :authenticatorbridge:koverXmlReportDebug \ :cxf:testDebug :cxf:koverXmlReportDebug ``` Running both tasks together in the same Gradle daemon session ensures the coverage report generation can access the test execution data produced by the test task. ## Performance Impact ### Expected Improvements | Metric | Before | After | |--------|--------|-------| | **Critical Path** | All tasks sequential | Longest job (test-app) - better utilization | | **Failure Detection** | Late (after all tests) | Early (lint fails first) - faster feedback | | **Resource Usage** | 1 runner | 5 concurrent runners - better parallelization | ### Test Distribution - **6,151 total tests** across all modules - **5,447 tests (88.5%)** in `:app` module (still bottleneck) - **423 tests (6.9%)** in library modules (completes quickly) - **281 tests (4.6%)** in `:authenticator` module ## Testing & Validation ### Local Testing - ✅ Configuration cache enabled and tested locally - ✅ Test execution with coverage generation validated - ✅ All modules generate XML reports correctly ### CI Validation Checklist - [x] All 5 jobs run in parallel - [x] Test failures reported correctly per-job - [x] Coverage artifacts upload successfully - [ ] Codecov receives and merges coverage data - [ ] Total coverage % matches baseline - [ ] Performance improvement achieved ## Future Optimization Opportunities This PR implements **Tier 1 (Quick Wins)** optimizations. Additional improvements are possible: ### Tier 2: App Module Sharding - Split `:app` module's 5,447 tests across 3-4 parallel jobs - Package-based sharding (auth, vault, platform, UI) - Further reduce critical path bottleneck ### Tier 3: Strategic Investments - Remote Gradle build cache (Gradle Enterprise or self-hosted) - Predictive test selection (run only affected tests on PRs) - Feature module extraction (break up monolithic `:app`) ## Risks & Mitigations ### Risk: Coverage Reporting Changes **Mitigation:** Codecov's XML merging is battle-tested and widely used. Coverage % should match previous implementation. ### Risk: Workflow Complexity Increase **Mitigation:** Clear job naming, comprehensive documentation, each job remains simple and focused. ### Risk: More CI Runner Usage **Impact:** Uses 5 concurrent runners instead of 1. Total runner-minutes may be similar or slightly higher, but wall-clock time is significantly reduced. ### Risk: Configuration Cache Issues **Mitigation:** Enabled with `problems=warn` to identify compatibility issues without failing builds. Can be disabled if problems arise. ## Rollback Plan If issues arise: 1. Revert this PR to return to single-job workflow 2. Disable configuration cache by reverting `gradle.properties` changes 3. All changes are isolated to workflow and properties files ## 📸 Screenshots Coming soon! ## ⏰ Reminders before review - Contributor guidelines followed - All formatters and local linters executed and passed - Written new unit and / or integration tests where applicable - Protected functional changes with optionality (feature flags) - Used internationalization (i18n) for all UI strings - CI builds passed - Communicated to DevOps any deployment requirements - Updated any necessary documentation (Confluence, contributing docs) or informed the documentation team ## 🦮 Reviewer guidelines <!-- Suggested interactions but feel free to use (or not) as you desire! --> - 👍 (`:+1:`) or similar for great changes - 📝 (`:memo:`) or ℹ️ (`:information_source:`) for notes or general info - ❓ (`:question:`) for questions - 🤔 (`:thinking:`) or 💭 (`:thought_balloon:`) for more open inquiry that's not quite a confirmed issue and could potentially benefit from discussion - 🎨 (`:art:`) for suggestions / improvements - ❌ (`:x:`) or ⚠️ (`:warning:`) for more significant problems or concerns needing attention - 🌱 (`:seedling:`) or ♻️ (`:recycle:`) for future improvements or indications of technical debt - ⛏ (`:pick:`) for minor or nitpick changes --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
GiteaMirror added the pull-request label 2026-05-01 20:06:13 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
app:authenticator
app:password-manager
bug
bug-passkey
customer-support
duplicate
enhancement
good first issue
help wanted
needs-reply
needs-response
pull-request
Mirrored from GitHub Pull Request
 question
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
GiteaMirror ninjasurge
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: github-starred/android#54523
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?