Skip to content

[Dont merge] Add R8 regression check workflow - #219

Open
AjeshRPai wants to merge 10 commits into
mainfrom
r8-modular-regression-gate
Open

AjeshRPai wants to merge 10 commits into
mainfrom
r8-modular-regression-gate

Conversation

@AjeshRPai

@AjeshRPai AjeshRPai commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Adds Automated presubmit (Git hooks) and CI (GitHub Actions) regression gates powered by android analyze to catch R8 configuration regressions, unshrunk libraries, and DEX size bloat before merge.


1. Prerequisites & Setup Requirements

To adopt this workflow in a repository, the following baseline file, secrets, build settings, and tools are required:

A. Initial R8 Baseline File (r8-baseline.json)

  • Why it is needed: check_app_r8_score.py compares every PR's release App Bundle (.aab) against a committed r8-baseline.json file in the repository root. This file stores the reference android analyze output (shrinkingRatio, optimizationRatio, obfuscationRatio, and uncompressedDexSizeBytes).
  • How to generate it the first time:
    # 1. Build the release App Bundle
    ./gradlew :app:bundleRelease
    
    # 2. Run android analyze and save the output to r8-baseline.json
    python3 scripts/check_app_r8_score.py \
      --aab app/build/outputs/bundle/release/app-release.aab \
      --baseline r8-baseline.json \
      --save-baseline
    
    # 3. Commit r8-baseline.json to your default branch
    git add r8-baseline.json && git commit -m "Add initial R8 baseline"
  • Ongoing updates: Once committed, the GitHub Actions workflow automatically refreshes and commits r8-baseline.json whenever a passing PR merges to main.

B. GitHub Actions Secrets & Permissions

Configure the following in Settings → Secrets and variables → Actions:

  1. GEMINI_API_KEY (Required for AI Diagnosis on Failure):
    • Used by the google-github-actions/run-gemini-cli step when either the App Bundle gate or Library Shrinking gate fails.
    • Allows Gemini CLI (equipped with the r8-analyzer skill) to inspect the PR diff (.gemini/pr_diff.patch) and comment with the exact keep rule or dependency that caused the regression.
  2. Workflow Token Permissions (GITHUB_TOKEN / Optional PAT):
    • The workflow declares contents: write, pull-requests: write, and issues: write permissions so it can post scorecard comments on PRs (gh pr comment) and auto-commit updated r8-baseline.json files on main.
    • Optional PAT Secret: If your main branch has branch protection rules that block github.token from pushing commits, add a Personal Access Token secret named PAT with repo / contents: write scope.
  3. Project Build Secrets (e.g., DEBUG_GOOGLE_SERVICES_JSON_BASE64):
    • Any secrets required to build :app:bundleRelease in CI (in Androidify, if DEBUG_GOOGLE_SERVICES_JSON_BASE64 is not set, the workflow falls back to copying test-google-services.json to app/google-services.json).

C. Android Gradle Plugin (AGP) & R8 Build Configuration

  1. AGP 9.0+ with Release Minification Enabled (isMinifyEnabled = true):
    • In app/build.gradle.kts, the release build type must have isMinifyEnabled = true.
    • AGP 9.0+ embeds BUNDLE-METADATA/com.android.tools/r8.json inside app-release.aab, which android analyze reads to calculate shrinking, optimization, and obfuscation scores.
  2. Library Module Consumer Rules (consumerProguardFiles):
    • For multi-module checks (check_library_shrinking.py --scan-libraries), library modules must build release AARs (./gradlew assembleRelease) so android analyze can inspect their packaged proguard.txt consumer rules and run isolated D8 vs. R8 shrinking comparisons.

D. Android CLI & Agent Skill

  • Android CLI (android):
    • Installed automatically in CI via https://dl.google.com/android/cli/latest/linux_x86_64/install.sh (or locally on macOS via darwin_arm64/install.sh), providing the android analyze --analysis=optimization command.
  • r8-analyzer Skill:
    • Installed in CI via android skills add --agent=gemini --project=. r8-analyzer, providing the domain instructions at .agents/skills/r8-analyzer/SKILL.md for root-cause keep-rule analysis.

2. Modular Check Scripts (scripts/)

Teams can adopt all three checks or pick only the ones that fit their workflow:

Script Target What It Checks Typical Stage
check_staged_r8_diff.py Staged git diff, *.pro, libs.versions.toml Blocks enableR8.fullMode=false, newly added -keep class <pkg>.** { *; } rules, and audits newly added Maven dependencies in ~/.gradle/caches Local pre-commit (<1s)
check_library_shrinking.py .aar & .jar modules/libraries Runs an isolated R8 dry-run ((1 - r8DexSize / d8DexSize) * 100) and flags consumer proguard.txt rules with package-wide wildcards Local pre-commit / CI
check_app_r8_score.py Release App Bundle (.aab) Compares shrinkingRatio, optimizationRatio, obfuscationRatio (0–100) and uncompressedDexSizeBytes against r8-baseline.json Local pre-push / CI

3. GitHub Actions Workflow (.github/workflows/r8_regression_check.yml)

Triggers & Inputs

  • pull_request / push (main): Runs automatically whenever ProGuard/R8 files (**/*.pro), Gradle build scripts (**/build.gradle.kts, gradle/libs.versions.toml, gradle.properties), r8-baseline.json, or scripts/check_*.py change.
  • workflow_dispatch: Supports manual execution against any PR or commit with configurable thresholds:
    • pr_number: Optional PR number to checkout and comment on.
    • max_score_drop (default 2.0): Maximum allowed drop in R8 shrinking, optimization, or obfuscation scores (percentage points).
    • max_dex_growth_pct (default 5.0): Maximum allowed uncompressed DEX size increase (%).
    • min_lib_shrinking_pct (default 90.0): Minimum required isolated DEX shrinking percentage for module AARs (%).

Workflow Pipeline Steps

  1. Prepare Review Context: Resolves the PR or commit diff range and extracts modified .pro, .gradle.kts, libs.versions.toml, and gradle.properties changes into .gemini/pr_diff.patch.
  2. Install Android CLI & Skills: Installs the latest android CLI (https://dl.google.com/android/cli/latest/linux_x86_64/install.sh) and adds the r8-analyzer skill (android skills add --agent=gemini --project=. r8-analyzer).
  3. Build Release Artifacts: Runs ./gradlew :app:bundleRelease assembleRelease to produce the release App Bundle (app-release.aab) and module AARs (*-release.aar).
  4. Check App Bundle R8 Scores & DEX Size (app_r8_gate): Runs check_app_r8_score.py to compare the release .aab against r8-baseline.json and writes the scorecard to .gemini/r8_summary.md and $GITHUB_STEP_SUMMARY.
  5. Check Module & Library Shrinking Percentage (lib_r8_gate): Runs check_library_shrinking.py (--scan-libraries --append-summary) across built module AARs to verify isolated R8 shrinking ratios and flag package-wide consumer keep rules.
  6. Auto-Ratify Baseline on main: On push to main when both gates pass, runs check_app_r8_score.py --save-baseline and auto-commits updated metrics to r8-baseline.json.
  7. AI-Assisted Diagnosis on Failure (gemini_review): If either gate fails, invokes google-github-actions/run-gemini-cli with the r8-analyzer skill to inspect .gemini/pr_diff.patch and pinpoint the exact keep rule or dependency causing the regression.
  8. PR Comment & HTML Report Upload: Posts the combined R8 scorecard and r8-analyzer remediation recommendations directly to the PR via gh pr comment, uploads r8-config-analyzer-report (r8-config-analyzer-release.html), and fails the check if any regression threshold was breached.

4. Presubmit Git Hooks (scripts/git-hooks/ & .pre-commit-config.yaml)

Comment thread scripts/check_app_r8_score.py Outdated
from typing import Any

DEFAULT_AAB = "app/build/outputs/bundle/release/app-release.aab"
FALLBACK_AAB = "app/app-release.aab"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the reason to have both a default and a fallback.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, there wasn't a good reason. The fallback was left over from a local demo setup. Removed it in be01dc1; there's now one default path.

Comment thread scripts/check_app_r8_score.py Outdated
parser.add_argument(
"--aab",
type=Path,
default=None,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should set default=Path(DEFAULT_AAB).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in be01dc1.

Comment thread scripts/check_app_r8_score.py Outdated
subprocess.run(["./gradlew", ":app:bundleRelease"], check=True)

aab_path = args.aab
if aab_path is None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you set a default, this can just be if not aab_path.exists(): and you can say "No AAB found at {aab_path}"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in be01dc1: it's now a single if not args.aab.exists() check that says "No AAB found at {path}".

Comment thread scripts/check_app_r8_score.py Outdated
)
return 2

print(f"{BOLD}==> Running `ANDROID_CLI_ANALYZE=1 android analyze` on {aab_path}...{RESET}")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ANDROID_CLI_ANALYZE=1 is no longer necessary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in be01dc1, here and in the other scripts and the workflow.

Comment thread scripts/check_library_shrinking.py Outdated
def run_android_analyze(artifact_path: Path) -> dict[str, Any]:
"""Invokes `android analyze --analysis=optimization --format=json` on an AAR/JAR."""
env = os.environ.copy()
env["ANDROID_CLI_ANALYZE"] = "1"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer necessary.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in be01dc1.

Comment thread scripts/check_library_shrinking.py Outdated


def _unwrap_optimization(data: dict[str, Any]) -> dict[str, Any]:
"""Supports both `{"results": [{"optimization": ...}]}` and `{"optimization": ...}`."""

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer necessary

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed in be01dc1.

Comment thread scripts/check_library_shrinking.py Outdated
}


def format_kb(num_bytes: int) -> str:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

function name is misleading, it formats bytes to either MB and KB

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to format_size in be01dc1.

Comment thread scripts/check_library_shrinking.py Outdated
r8_bytes = int(r8_compile.get("dexSizeBytes", 0))

keep_ratio_pct = (r8_bytes / d8_bytes * 100.0) if d8_bytes > 0 else 0.0
dex_shrinking_pct = 100.0 - keep_ratio_pct if d8_bytes > 0 else 100.0

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why both the keep % and shrinking %?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for both. In be01dc1 I dropped the keep ratio; only the shrinking % is reported.

Comment thread scripts/check_library_shrinking.py Outdated
fld_total, fld_kept, fld_shrunk_pct = _item_stats("fields")

validate = dex.get("validateConsumerKeepRulesResult", {})
used_pkg_wide = int(validate.get("usedPackageWideKeepRules", 0))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you want len() here and below instead to get the count of rules.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, thanks. Checking it against the real android analyze output turned up more problems in this function: the package-wide rule lists are under r8CompileResult.configuration, not validateConsumerKeepRulesResult; the kept counts are keptItemCount, not keptCount; and blocked rules use source, not value. All fixed in be01dc1. The watchface AAR now reports its 11 package-wide consumer rules (WARN) instead of 0.

Comment thread scripts/check_r8_regression.py Outdated
def run_android_analyze(artifact_path: Path) -> dict[str, Any]:
"""Invokes `android analyze --analysis=optimization --format=json` on an AAB/APK/AAR/JAR."""
env = os.environ.copy()
env["ANDROID_CLI_ANALYZE"] = "1"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the whole file in be01dc1. It was the old all-in-one script, replaced by the three split scripts.

- check_library_shrinking.py: read the keys `android analyze` actually
  outputs (keptItemCount, configuration.used/unusedPackageWideKeepRules,
  blockedKeepRules[].source) and count rules with len(). Package-wide
  consumer rules were always reported as 0 before (watchface has 11).
  Drop the redundant keep ratio and rename format_kb to format_size.
- check_app_r8_score.py: single default AAB path, drop --build and the
  old JSON shape, read the package name from --analysis=basic.
- Remove ANDROID_CLI_ANALYZE (no longer needed) and
  --no-configuration-cache from the scripts and workflow.
- Always run the git hooks, the pre-commit config hooks and the CI
  workflow instead of filtering by changed files.
- Post one PR comment and edit it on later runs.
- Remove check_r8_regression.py, superseded by the split scripts.
@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

🛡️ R8 Optimization & Shrinking Gate (android analyze)

📦 App Bundle R8 Scores & DEX Size

  • Package: com.android.developers.androidify
  • Compiler: AGP 9.3.2 / R8 9.3.16 (Full Mode: True)
  • Baseline: r8-baseline.json
Metric Baseline Current Delta Threshold Status
Shrinking Score 51.15% 51.64% +0.49% pts max -2.00% pts ✅ PASS
Optimization Score 50.81% 51.30% +0.49% pts max -2.00% pts ✅ PASS
Obfuscation Score 51.11% 51.61% +0.50% pts max -2.00% pts ✅ PASS
Uncompressed DEX Size 18.32 MB 17.53 MB -0.79 MB (-4.3%) max +5.0% / +500KB ✅ PASS

📚 Library Shrinking Percentage (Min Threshold: 90.0%)

Library (.aar / .jar) D8 DEX R8 Kept DEX DEX Shrinking % Classes Shrunk Methods Shrunk Pkg-Wide Rules Status
network-release.aar 64.35 KB 0.00 KB 100.00% 100.0% (49/49) 100.0% (250/250) 0 ✅ PASS
testing-release.aar 29.22 KB 0.00 KB 100.00% 100.0% (17/17) 100.0% (119/119) 0 ✅ PASS
theme-release.aar 209.49 KB 0.00 KB 100.00% 100.0% (51/51) 100.0% (405/405) 0 ✅ PASS
util-release.aar 44.71 KB 0.00 KB 100.00% 100.0% (24/24) 100.0% (111/111) 0 ✅ PASS
xr-release.aar 44.73 KB 0.00 KB 100.00% 100.0% (13/13) 100.0% (64/64) 0 ✅ PASS
data-release.aar 55.13 KB 0.00 KB 100.00% 100.0% (54/54) 100.0% (193/193) 0 ✅ PASS
camera-release.aar 289.75 KB 0.00 KB 100.00% 100.0% (93/93) 100.0% (550/550) 0 ✅ PASS
creation-release.aar 343.21 KB 0.00 KB 100.00% 100.0% (86/86) 100.0% (610/610) 0 ✅ PASS
home-release.aar 186.55 KB 0.00 KB 100.00% 100.0% (47/47) 100.0% (310/310) 0 ✅ PASS
results-release.aar 602.10 KB 0.00 KB 100.00% 100.0% (200/200) 100.0% (1127/1127) 0 ✅ PASS
watchface-release.aar 72.20 KB 2.73 KB 96.21% 95.7% (44/46) 96.1% (246/256) 11 ⚠️ WARN
common-release.aar 45.32 KB 0.00 KB 100.00% 100.0% (31/31) 100.0% (229/229) 0 ✅ PASS

On every PR, post a GitHub-style diff of the keep rules that changed
compared with main, and the classes they keep. It covers rules that come
with library upgrades and existing wildcard rules that start keeping new
classes, neither of which shows up in the PR's own diff.

It doesn't build anything. With AGP 9.3.2, every release build writes
the keep rule .pb file (app/build/outputs/mapping/release/configanalyzer.pb)
by default, so the gate uploads the file from the build it already runs.

- keep_rule_diff.yml: reusable workflow, given the .pb file's path. On
  pushes to main it caches the file per commit; on PRs it diffs against
  main's cached file, writes the job summary and keeps one PR comment
  up to date.
- r8_regression_check.yml: upload the .pb file after the release build
  and call keep_rule_diff.yml in a second job, which also runs when the
  gate fails. Optionally skip the diff when the R8 scores didn't change
  (KEEP_RULE_DIFF_ONLY_IF_SCORES_CHANGED).
- scripts/keep_rule_diff.py and scripts/schema_pb2.py: compare two keep
  rule .pb files and write the Markdown report.

Needs AGP 9.3 or newer to write the .pb file; until then the diff is
skipped.
zizmor's excessive-permissions audit flags workflow-level write
permissions once a workflow has more than one job, which the new
keep-rule-diff job made it. Move them to the r8-regression-gate job;
keep-rule-diff already sets its own (contents: read, pull-requests: write).
@android android deleted a comment from github-actions Bot Oct 29, 2026
main doesn't run the keep rule diff yet, so the cache has no keep rules
from main to compare this PR against. A temporary job builds the PR's
base commit once, with AGP 9.3.2 so that it writes the keep rule .pb file,
and caches it under the key the keep-rule-diff job looks for.

To be removed once the diff has run.
@github-actions

Copy link
Copy Markdown

🔍 R8 keep rule diff

+ and - lines are keep rules the PR adds or removes. The indented lines under a rule are the classes it keeps. A rule with no sign didn't change, but now keeps different classes.

@@ app/proguard-rules.pro @@
- -keep class com.firebase.** { *; }
- -keep interface com.firebase.** { *; }
- -keep class okhttp3.** { *; }
-     okhttp3.Address
-     okhttp3.Authenticator
-     okhttp3.Authenticator$Companion
-     okhttp3.Authenticator$Companion$AuthenticatorNone
-     okhttp3.Cache
-     okhttp3.Cache$CacheResponseBody
-     okhttp3.Cache$CacheResponseBody$1
-     okhttp3.Cache$Companion
-     okhttp3.Cache$Entry
-     okhttp3.Cache$Entry$Companion
-     okhttp3.Cache$RealCacheRequest
-     okhttp3.Cache$RealCacheRequest$1
-     okhttp3.Cache$urls$1
-     okhttp3.CacheControl
-     okhttp3.CacheControl$Builder
-     okhttp3.CacheControl$Companion
-     okhttp3.Call
-     okhttp3.Call$Factory
-     okhttp3.Callback
-     okhttp3.CertificatePinner
-     okhttp3.CertificatePinner$Builder
-     okhttp3.CertificatePinner$Companion
-     okhttp3.CertificatePinner$Pin
-     okhttp3.Challenge
-     okhttp3.CipherSuite
-     okhttp3.CipherSuite$Companion
-     okhttp3.CipherSuite$Companion$ORDER_BY_NAME$1
-     okhttp3.CompressionInterceptor
-     okhttp3.CompressionInterceptor$DecompressionAlgorithm
-     okhttp3.Connection
-     okhttp3.ConnectionPool
-     okhttp3.ConnectionSpec
-     okhttp3.ConnectionSpec$Builder
-     okhttp3.ConnectionSpec$Companion
-     okhttp3.Cookie
-     okhttp3.Cookie$Builder
-     okhttp3.Cookie$Companion
-     okhttp3.CookieJar
-     okhttp3.CookieJar$Companion
-     okhttp3.CookieJar$Companion$NoCookies
-     okhttp3.Credentials
-     okhttp3.Dispatcher
-     okhttp3.Dispatcher$promoteAndExecute$Effects
-     okhttp3.Dns
-     okhttp3.Dns$Companion
-     okhttp3.Dns$Companion$DnsSystem
-     okhttp3.EventListener
-     okhttp3.EventListener$AggregateEventListener
-     okhttp3.EventListener$Companion
-     okhttp3.EventListener$Companion$NONE$1
-     … and 323 more
- -keep interface okhttp3.** { *; }
-     okhttp3.Authenticator
-     okhttp3.Call
-     okhttp3.Call$Factory
-     okhttp3.Callback
-     okhttp3.CompressionInterceptor$DecompressionAlgorithm
-     okhttp3.Connection
-     okhttp3.CookieJar
-     okhttp3.Dns
-     okhttp3.EventListener$Factory
-     okhttp3.Interceptor
-     okhttp3.Interceptor$Chain
-     okhttp3.TrailersSource
-     okhttp3.WebSocket
-     okhttp3.WebSocket$Factory
-     okhttp3.internal.SuppressSignatureCheck
-     okhttp3.internal.cache.CacheRequest
-     okhttp3.internal.concurrent.Lockable
-     okhttp3.internal.concurrent.TaskRunner$Backend
-     okhttp3.internal.connection.BufferedSocket
-     okhttp3.internal.connection.ExchangeFinder
-     okhttp3.internal.connection.RoutePlanner
-     okhttp3.internal.connection.RoutePlanner$Plan
-     okhttp3.internal.http.ExchangeCodec
-     okhttp3.internal.http.ExchangeCodec$Carrier
-     okhttp3.internal.http2.FlowControlListener
-     okhttp3.internal.http2.Http2Reader$Handler
-     okhttp3.internal.http2.PushObserver
-     okhttp3.internal.platform.ContextAwarePlatform
-     okhttp3.internal.platform.android.DeferredSocketAdapter$Factory
-     okhttp3.internal.platform.android.SocketAdapter
-     okhttp3.internal.publicsuffix.PublicSuffixList
-     okhttp3.internal.sse.ServerSentEventReader$Callback
-     okhttp3.internal.tls.TrustRootIndex
-     okhttp3.internal.ws.WebSocketReader$FrameCallback
-     okhttp3.logging.HttpLoggingInterceptor$Logger
-     okhttp3.sse.EventSource
-     okhttp3.sse.EventSource$Factory

@cwoodwar6 cwoodwar6 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if having the keep rule diff is worth the added proto complexity. It is nice to have when a library changes a keep rule, but for anything changed within the project, those changes are already visible in the git diff.

Comment thread scripts/schema_pb2.py

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should avoid checking in this file. Do you have the .proto file? Adding a proto dependency seems a bit risky, I'd avoid it if possible.

Comment thread scripts/keep_rule_diff.py

python3 scripts/keep_rule_diff.py base.pb pr.pb -o keep_rule_diff.md

Reading .pb files needs `pip install 'protobuf>=6.33.4'` and schema_pb2.py next to this script.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This dependency should be installed in your workflow yaml.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants