Mastering Screenshot Testing in Jetpack Compose: A Step-by-Step Guide

As Android and Compose Multiplatform developers, ensuring the visual integrity of our user interfaces is critical. Jetpack Compose, Android’s modern UI toolkit, simplifies UI development, but how do we ensure our composables look as intended across updates? Enter screenshot testing — a powerful technique to catch visual regressions by comparing UI snapshots against baseline images. In this blog, we’ll explore the Compose Preview Screenshot Testing library, provide a step-by-step implementation guide, discuss its pros and cons, and compare it with other popular libraries like Paparazzi and Showkase.

What is Screenshot Testing?

Screenshot testing involves rendering a UI component, capturing its visual output as an image, and comparing it to a pre-approved baseline image. If the images differ, the test fails, indicating a potential visual regression. This is particularly useful in Jetpack Compose, where composables are declarative and highly customizable, making manual UI verification time-consuming.

The Compose Preview Screenshot Testing library, introduced at Google I/O 2024, leverages Compose’s @Preview annotations to simplify screenshot testing, integrating seamlessly with Android Studio and CI/CD pipelines.

Step-by-Step Implementation of Compose Preview Screenshot Testing

Let’s walk through setting up and using the Compose Preview Screenshot Testing library in an Android project. We’ll create a simple composable, write a screenshot test, and validate it.

Prerequisites

• Kotlin: Version 2.0.20 or newer
• Android Gradle Plugin: Version 8.5.0 or higher
• Jetpack Compose: Version 1.5.4 or later
• Android Studio: Latest stable version for preview support

Step 1: Set Up Your Project

Add the required dependencies and plugins to your project.

Update libs.versions.toml

In your project’s gradle/libs.versions.toml, define the versions:

[versions]
agp = "8.6.1"
kotlin = "2.0.20"
compose = "1.5.4"
composeScreenshot = "0.0.1-alpha08"

[libraries]
androidx-compose-ui-tooling = { group = "androidx.compose.ui", name = "ui-tooling", version.ref = "compose" }
screenshot-validation-api = { group = "com.android.tools.screenshot", name = "screenshot-validation-api", version.ref = "composeScreenshot" }

[plugins]
android-application = { id = "com.android.application", version.ref = "agp" }
kotlin-android = { id = "org.jetbrains.kotlin.android", version.ref = "kotlin" }
compose-screenshot = { id = "com.android.compose.screenshot", version.ref = "composeScreenshot" }

Update Module-Level build.gradle.kts

In your app module’s build.gradle.kts, apply the screenshot testing plugin and dependencies:

plugins {
    alias(libs.plugins.android.application)
    alias(libs.plugins.kotlin.android)
    alias(libs.plugins.compose.screenshot)
}

android {
    compileSdk = 34
    defaultConfig {
        minSdk = 24
        targetSdk = 34
        testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
    }
    composeOptions {
        kotlinCompilerExtensionVersion = libs.versions.compose.get()
    }
    experimentalProperties["android.experimental.enableScreenshotTest"] = true
}

dependencies {
    implementation(libs.androidx.compose.ui.tooling)
    screenshotTestImplementation(libs.screenshot.validation.api)
}

Enable Experimental Properties

In your project’s gradle.properties, enable the screenshot testing feature:

android.experimental.enableScreenshotTest=true

Sync your project to ensure all dependencies are resolved.

Step 2: Create a Composable to Test

Let’s create a simple PrimaryButton composable to test.

package com.example.myapp.ui

import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.Button
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.compose.ui.graphics.Color

@Composable
fun PrimaryButton(title: String) {
    Button(
        modifier = Modifier
            .fillMaxWidth()
            .padding(horizontal = 16.dp),
        onClick = {},
        shape = MaterialTheme.shapes.medium,
        colors = androidx.compose.material3.ButtonDefaults.buttonColors(
            containerColor = MaterialTheme.colorScheme.primary,
            contentColor = Color.White
        )
    ) {
        Text(title)
    }
}

Step 3: Create a Screenshot Test

Create a new source set for screenshot tests and write a test case.

Set Up the screenshotTest Source Set

In your app module, create a directory: app/src/screenshotTest/kotlin/com/example/myapp/. This separates screenshot tests from unit and integration tests.

Write the Screenshot Test

Create a file named PrimaryButtonScreenshotTest.kt:

package com.example.myapp

import androidx.compose.material3.MaterialTheme
import androidx.compose.runtime.Composable
import androidx.compose.ui.tooling.preview.Preview
import com.example.myapp.ui.PrimaryButton

@Preview(showBackground = true)
@Composable
fun PrimaryButtonPreview() {
    MaterialTheme {
        PrimaryButton("Login")
    }
}

The @Preview annotation allows the composable to be rendered in Android Studio and used for screenshot testing.

Step 4: Generate Baseline Screenshots

Run the following Gradle command to generate baseline screenshots:

./gradlew recordDebugScreenshotTest

This creates a reference directory under app/src/screenshotTest/ containing a PNG file (e.g., com.example.myapp.PrimaryButtonScreenshotTest_PrimaryButtonPreview_*.png).

Step 5: Validate Screenshots

To validate the composable against the baseline, run:

./gradlew validateDebugScreenshotTest

This compares the current UI rendering with the baseline image. If they match, the test passes. If not, an HTML report is generated at app/build/reports/screenshotTest/preview/debug/index.html, highlighting differences.

Step 6: Test a Visual Regression

Modify the PrimaryButton composable to introduce a change, e.g., update the padding and shape:

@Composable
fun PrimaryButton(title: String) {
    Button(
        modifier = Modifier
            .fillMaxWidth()
            .padding(horizontal = 20.dp), // Changed from 16.dp
        onClick = {},
        shape = RoundedCornerShape(30.dp), // Changed shape
        colors = androidx.compose.material3.ButtonDefaults.buttonColors(
            containerColor = MaterialTheme.colorScheme.primary,
            contentColor = Color.White
        )
    ) {
        Text(title)
    }
}

Run the validation command again:

./gradlew validateDebugScreenshotTest

The test will fail, and the HTML report will show the visual differences. To accept the new UI as the baseline, re-run:

./gradlew recordDebugScreenshotTest

Step 7: Integrate with CI/CD

To automate screenshot testing, integrate it into your CI/CD pipeline (e.g., GitHub Actions). Here’s an example workflow:

name: Screenshot Testing
on:
  push:
    branches: [ main ]
  pull_request:
jobs:
  screenshot-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Set Up JDK 17
        uses: actions/setup-java@v4
        with:
          java-version: 17
          distribution: temurin
      - name: Setup Gradle
        uses: gradle/actions/setup-gradle@v4
      - name: Run Screenshot Tests
        run: ./gradlew validateDebugScreenshotTest

This ensures visual regressions are caught before merging code.

Pros and Cons of Compose Preview Screenshot Testing

Pros

• Seamless Integration with Previews: Leverages existing @Preview annotations, reducing the need for additional test code.
• Minimal Setup: Requires only a few Gradle configurations, making it easy to adopt.
• Clean Organization: Uses a dedicated screenshotTest source set, keeping tests separate from other test types.
• CI/CD Friendly: Integrates well with Gradle and CI pipelines, with HTML reports for easy debugging.
• Flexible Configurations: Supports multiple preview configurations (e.g., light/dark themes, font scales) via @Preview parameters.

Cons

• Experimental Status: As of version 0.0.1-alpha08, the library is still experimental, with potential API changes and bugs.
• Cross-Platform Issues: Screenshots may vary across operating systems due to rendering differences, requiring threshold adjustments or server-side rendering.
• Limited to Previews: Only works with @Preview-annotated composables, limiting flexibility for dynamic UI states.
• Baseline Management: Maintaining baseline images for multiple configurations can be cumbersome, especially in large projects.

Comparison with Other Screenshot Testing Libraries

Let’s compare Compose Preview Screenshot Testing with two popular alternatives: Paparazzi and Showkase.

1. Paparazzi

• Overview: An open-source library by CashApp for screenshot testing, designed for both View-based and Compose UIs. It runs tests on the JVM without requiring a device or emulator.
• Setup: Requires adding the Paparazzi plugin and dependencies to your Gradle files. Tests are written in the test source set.

Pros:

• Device-agnostic, reducing test flakiness.
• Fast execution on JVM.
• Supports both Compose and View-based UIs.
• Open-source with active community maintenance.

Cons:

• More complex setup compared to Compose Preview Screenshot Testing.
• Requires writing custom test rules for each composable.
• Limited integration with Android Studio’s preview system.

Use Case: Ideal for projects needing device-independent testing or supporting legacy View-based UIs.

2. Showkase

• Overview: Developed by Airbnb, Showkase is a library that creates a UI showcase for @Preview-annotated composables and supports screenshot testing by integrating with other libraries (e.g., Facebook’s screenshot testing library).
• Setup: Requires integrating Showkase and a screenshot testing library, with tests written in the androidTest source set.

Pros:

• Provides a visual UI showcase for designers and QA, enhancing collaboration.
• Automatically turns @Preview composables into screenshot tests.
• Supports multiple configurations (e.g., dark mode, RTL).

Cons:

• Depends on external screenshot testing libraries, increasing complexity.
• Requires an emulator or device, which can introduce flakiness.
• Less native integration with Jetpack Compose compared to Google’s library.

Use Case: Best for teams needing a visual catalog of UI components alongside screenshot testing.

Comparison Table

Feature/Library Compose Preview Screenshot Testing Paparazzi Showkase Integration with @Preview Native Limited Strong Device/Emulator Required No No Yes Setup Complexity Low Medium High Cross-Platform Consistency Moderate (threshold needed) High Moderate CI/CD Integration Strong Strong Moderate Best For Compose-only projects Mixed UI projects UI showcase + testing

When to Choose Which?

• Compose Preview Screenshot Testing: Choose for Compose-only projects with minimal setup and tight integration with Android Studio previews.
• Paparazzi: Opt for projects requiring fast, device-agnostic testing, especially with mixed View and Compose UIs.
• Showkase: Use when you need a visual UI catalog for stakeholders alongside screenshot testing.

Best Practices for Screenshot Testing

1. Use Meaningful Preview Configurations: Test critical scenarios (e.g., light/dark modes, different font scales) using @Preview parameters.
2. Automate Baseline Updates: Use CI/CD to regenerate baselines for intentional UI changes, but review them carefully to avoid masking regressions.
3. Keep Tests Focused: Test critical UI components to avoid bloating your baseline image set.
4. Handle Cross-Platform Issues: Adjust tolerance thresholds or use server-side rendering to mitigate rendering differences.
5. Integrate with QA: Share screenshot reports with QA teams for visual validation, as supported by tools like Screenshotbot.io.

Conclusion

The Compose Preview Screenshot Testing library is a game-changer for Jetpack Compose developers, offering a streamlined way to catch visual regressions using familiar @Preview annotations. Its minimal setup and CI/CD integration make it a great choice for Compose-only projects, though its experimental status and cross-platform challenges require careful consideration. Compared to Paparazzi and Showkase, it strikes a balance between simplicity and functionality, but your choice depends on your project’s needs.

By following the steps outlined, you can implement screenshot testing in your Jetpack Compose projects and ensure a polished UI. Try it out, and let your UI shine without visual bugs!

Happy coding, and don’t forget to clap 👏 if you found this helpful! Follow me on Medium for more Android and Compose Multiplatform insights.

References:

• Android Developers: Compose Preview Screenshot Testing
• Paparazzi: GitHub Repository
• Showkase: GitHub Repository