How to Test Reports Generation on Android (Complete Guide)

Reports are often the final deliverable that users see after completing a workflow such as expense tracking, medical data aggregation, or financial reconciliation. When a report is missing, malformed,

Why Testing Reports Generation Matters on Android

Reports are often the final deliverable that users see after completing a workflow such as expense tracking, medical data aggregation, or financial reconciliation. When a report is missing, malformed, or contains incorrect data, user trust erodes quickly and regulatory compliance can be jeopardized. On Android, reports frequently involve file I/O, external storage permissions, third‑party libraries (PDF, CSV, Excel), and sharing intents that traverse process boundaries. A defect in any of these layers can surface only under specific device states—low storage, locale change, or background restriction—making exhaustive testing essential.

Common Failure Modes in Reports Generation

Understanding the typical ways reports break helps focus test effort. The most frequent failure categories observed in production are:

Each of these categories can be reproduced with a targeted test matrix, which we define next.

Building a Comprehensive Test Matrix

A test matrix captures the combinations of input conditions, device states, and expected outcomes that must be verified. Below is a master table that can be copied into a test‑management tool (e.g., TestRail, Zephyr). Each row represents a distinct test case; you can add or remove columns based on your project’s needs.

*Tip:* When you copy this matrix into a test‑case tool, add columns for Automation Status (Manual, Automated, Planned) and Owner to track progress.

Sub‑tables for Specific Domains

For brevity in daily stand‑ups you may extract focused tables:

Accessibility Sub‑matrix

Security/Privacy Sub‑matrix

Manual Testing Approach: Step‑by‑Step Guide

Even with automation, a disciplined manual pass catches nuances that scripts may miss, especially around user perception and device‑state interactions. Follow this procedure for each new report feature or after a major refactor.

1. Environment Setup

• Devices: Use a matrix of physical devices covering API levels 21‑34, different manufacturers (Samsung, Pixel, OnePlus), and varying screen densities.
• Emulators: Include at least one API 33 emulator with Play Store to test Google‑Play‑services‑dependent features (e.g., Firebase Crashlytics).
• Tooling: Install Android Studio, ADB, adb shell pm grant android.permission.WRITE_EXTERNAL_STORAGE for legacy tests, and uiautomatorviewer for UI inspection.
• Data: Prepare a static JSON fixture set (small, medium, large) that mimics real‑world payloads. Store it in src/androidTest/assets/ for easy push via adb push.

2. Preparing Test Data

• Load the fixture into the app via a debug menu or directly through the UI (e.g., “Load Sample Data”).
• Verify that the data appears correctly in the preview screen before report generation.
• For large‑data tests, use a generator script (Kotlin or Python) to create a 10 MB JSON file; push it to /sdcard/Download/ and load via file picker.

3. Executing Happy Path Tests

1. Navigate to the report screen.
2. Select the desired format (PDF, CSV, Excel).
3. Tap Generate.
4. Wait for the success toast or notification.
5. Open the file via a file manager or the built‑in preview.
6. Validate:

• File exists in expected directory.
• Size >0.
• Header/magic number matches format.
• Content matches source data (spot‑check a few rows).

1. Document the outcome with a screenshot and the file’s SHA‑256 hash (adb shell sha256sum ).

4. Executing Error Path Tests

• Storage Full: Follow the precondition in RPT‑03, then attempt generation. Observe the error dialog; confirm no file is left behind.
• Permission Denial: Disable the permission via Settings → Apps → → Permissions → Storage → Deny. Attempt generation; verify the fallback path or error message.
• Interrupted Write: Use adb shell am kill mid‑generation (you can monitor logs to know when the write starts). After relaunch, check that either no file exists or the file is valid and usable.

5. Logging and Evidence Collection

• Capture logcat with adb logcat -v time > logcat.txt before each test series.
• After each test, pull the generated report: adb pull /sdcard/Documents/AppName/reports/report_20240926.pdf ./artifacts/.
• Store artifacts in a folder named after the test‑case ID for traceability.
• If a failure occurs, annotate the logcat with timestamps and the exact UI state (screenshot from adb shell screencap).

6. Post‑Test Cleanup

• Delete any test files created in shared storage to avoid interfering with subsequent runs.
• Reset device locale, font size, and theme to default values.
• Revoke any temporarily granted permissions (adb shell pm revoke android.permission.WRITE_EXTERNAL_STORAGE).

Automated Testing on Android: Tools and Frameworks

Manual validation is essential but does not scale for regression. Automated checks provide fast feedback on CI pipelines and guard against re‑introducing known defects. Below we detail the layers of automation that work best for report generation on Android.

Unit Testing Report Generation Logic

If the report creation is encapsulated in a pure Kotlin/Java class (e.g., ReportGenerator), unit tests can verify data transformation without UI or file system involvement.

// ReportGeneratorTest.kt
class ReportGeneratorTest {

    private val generator = ReportGenerator()

    @Test
    fun `pdf generator writes correct header`() {
        val data = listOf(ReportRow("Alice", 28, "alice@example.com"))
        val output = generator.generatePdf(data) // returns ByteArray
        assertTrue(output.startsWith(byteArrayOf(0x25, 0x50, 0x44, 0x46))) // "%PDF"
    }

    @Test
    fun `csv generator respects locale delimiter`() {
        val data = listOf(ReportRow("Bob", 35, "bob@example.com"))
        val csv = generator.generateCsv(data, Locale.GERMANY)
        assertTrue(csv.contains(";")) // German locale uses semicolon
    }
}

Run these tests with ./gradlew testAndroidUnit. They execute on the JVM, providing sub‑second feedback.

Instrumented UI Tests with Espresso

Espresso validates that the UI triggers the generation flow and that the success state is observable.

// ReportGenerationTest.kt
@RunWith(AndroidJUnit4::class)
class ReportGenerationTest {

    @get:Rule
    val activityRule = ActivityScenarioRule(MainActivity::class)

    @Test
    fun generatePdf_showsSuccessToast_andCreatesFile() {
        // Load sample data via debug menu
        onView(withId(R.id.btnLoadSample)).perform(click())
        onView(withId(R.id.btnGeneratePdf)).perform(click())

        // Wait for toast
        onView(withText(containsString("Report saved")))
            .inRoot(IsPlatformToast())
            .check(matches(isDisplayed()))

        // Verify file exists using a helper that runs on a background thread
        val context = ApplicationProvider.getApplicationContext()
        val file = File(context.getExternalFilesDir(Environment.DIRECTORY_DOCUMENTS),
                       "reports/report_${System.currentTimeMillis()}.pdf")
        assertTrue("File should exist", file.exists())
        // Optional: check PDF header
        val header = FileInputStream(file).readNBytes(4)
        assertArrayEquals(byteArrayOf(0x25, 0x50, 0x44, 0x46), header)
    }
}

Key points:

• Use IdlingResource if the generation performs asynchronous work (e.g., coroutines).
• Leverage FileProvider to expose the generated file for assertions without needing READ_EXTERNAL_STORAGE in the test manifest (grant temporary permission via grantUriPermission).

UIAutomator for File System Checks

When you need to inspect the file system outside the app’s sandbox (e.g., verify that a report appears in the public Downloads folder), UIAutomator works well because it can run with broader permissions.

// ReportFileCheck.java
public class ReportFileCheck extends UiAutomatorTestCase {

    public void testPdfAppearsInDownloads() throws Exception {
        // Launch the app and trigger generation
        final String pkg = getTargetContext().getPackageName();
        UiDevice device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation());
        device.launchActivity(new Intent(Intent.ACTION_MAIN)
                .setPackage(pkg)
                .setCategory(Intent.CATEGORY_LAUNCHER));

        // Click generate button (replace with actual resource ID)
        UiObject2 genBtn = device.findObject(By.res(pkg, "id/generatePdf"));
        genBtn.click();

        // Wait for toast
        UiObject2 toast = device.findObject(By.text(contains("Report saved")));
        assertNotNull(toast);

        // Check file in Downloads
        File downloads = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS);
        File[] files = downloads.listFiles((dir, name) -> name.endsWith(".pdf"));
        assertTrue("No PDF found in Downloads", files.length > 0);
        // Validate first file’s header
        FileInputStream fis = new FileInputStream(files[0]);
        byte[] header = new byte[4];
        fis.read(header);
        assertArrayEquals(new byte[]{(byte)0x25, 0x50, 0x44, 0x46}, header);
        fis.close();
    }
}

Run with ./gradlew connectedAndroidTest. This test can be tagged (androidTestAnnotation) to run only on nightly builds if it is slower.

Leveraging SUSA Autonomous Exploration (Mention)

SUSA’s autonomous agent can be pointed at the APK or a internal test build. It will explore the reports flow using its built‑in personas (curious, impatient, novice, etc.) without any test scripts. During exploration it:

• Attempts to generate reports after every navigation change, uncovering race conditions that only appear when the user rapidly switches tabs.
• Triggers permission dialogs and observes how the app handles denial versus grant.
• Varies system settings (locale, font scale, dark mode) via its persona‑driven configuration matrix, surfacing UI layout bugs that Espresso may miss if the test script does not change those settings.
• Logs any crashes, ANRs, or file‑system exceptions and automatically creates a regression script in Appium (Android) and Playwright (Web) for future CI runs.

To invoke SUSA locally:

pip install susatest-agent
susatest run --apk path/to/app-debug.apk --goal "reports" --personas all --output ./susa-reports

The agent will produce a JSON summary with discovered issues, each tagged with the responsible persona (e.g., “impatient” found a double‑tap crash).

Integrating with CI/CD

• Unit tests: Run on every pull request (./gradlew testAndroidUnit).
• Instrumented tests: Execute on a device farm (Firebase Test Lab, AWS Device Farm) for each merge to main.
• Susa runs: Schedule a nightly job that pulls the latest artifact.apk from the build pipeline, executes the agent with a curated set of personas, and fails the build if any new crash or ANR is detected.
• Artifact archiving: Store generated reports and logs as build artifacts for audit purposes.

Code Snippets: Espresso Test for PDF Report Generation

Below is a self‑contained example that demonstrates how to verify PDF content using the PdfiumAndroid library, which can parse PDF pages and extract text for assertions.

// PdfReportVerificationTest.kt
@RunWith(AndroidJUnit4::class)
class PdfReportVerificationTest {

    @get:Rule
    val activityRule = ActivityScenarioRule(MainActivity::class)

    @Test
    fun generatedPdfContainsExpectedData() {
        // Preload known data
        onView(withId(R.id.btnLoadFixture)).perform(click())
        onView(withId(R.id.btnGeneratePdf)).perform(click())

        // Wait for generation completion
        onView(withText("Report saved"))
            .inRoot(IsPlatformToast())
            .check(matches(isDisplayed()))

        // Locate the newest PDF in app‑specific docs
        val context = ApplicationProvider.getApplicationContext()
        val docsDir = File(context.getExternalFilesDir(Environment.DIRECTORY_DOCUMENTS), "reports")
        val pdfFile = docsDir.listFiles()
                .filter { it.name.endsWith(".pdf") }
                .maxByOrNull { it.lastModified() }
                ?: error("No PDF found")

        // Load PDF with PdfiumAndroid
        val document = PdfDocument.openFile(pdfFile.absolutePath)
        val page = document.getPage(0)
        val text = page.text
        page.close()
        document.close()

        // Assert that known strings appear
        assertTrue(text.contains("Alice Smith"))
        assertTrue(text.contains("2024-09-26"))
        // Optional: check that no extra pages were added
        assertEquals(1, document.pagesCount)
    }
}

Explanation

1. The test loads a deterministic fixture so the expected output is known.
2. After generation, it waits for the success toast (a reliable UI signal).
3. It scans the app‑specific reports directory for the most‑recent PDF, avoiding race conditions with parallel test runs.
4. PdfiumAndroid extracts the text from the first page; assertions confirm that the data from the fixture is present.
5. Resources are closed promptly to avoid file‑handle leaks.

Code Snippets: UIAutomator to Validate Report Sharing Intent

Sharing a report often involves an Intent with a content:// URI. This UIAutomator test confirms that the intent is correctly formed and that the receiving app can read the file.

// ShareIntentVerification.java
public class ShareIntentVerification extends UiAutomatorTestCase {

    public void testShareIntentHasCorrectMimeAndUriPermission() throws Exception {
        String pkg = getTargetContext().getPackageName();
        UiDevice device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation());

        // Launch app and navigate to report screen
        device.launchActivity(new Intent(Intent.ACTION_MAIN)
                .setPackage(pkg)
                .setCategory(Intent.CATEGORY_LAUNCHER));
        device.wait(Until.hasObject(By.pkg(pkg).depth(0)), 5000);

        // Load sample data (assuming a debug button)
        device.findObject(By.res(pkg, "id/loadSample")).click();
        device.wait(Until.hasObject(By.text("Data loaded")), 3000);

        // Trigger share
        device.findObject(By.res(pkg, "id/sharePdf")).click();

        // Wait for the chooser to appear
        UiObject2 chooser = device.wait(Until.hasObject(By.text("Share via")), 5000);
        assertNotNull(chooser);

        // Retrieve the intent sent to the chooser via instrumentation
        Intent sentIntent = getInstrumentation().getTargetContext()
                .getSystemService(Context.class)
                .getSystemService(NotificationManager.class)
                .getActiveNotifications()[0]
                .notification
                .contentIntent
                .getIntent();

        assertNotNull(sentIntent);
        assertEquals(Intent.ACTION_SEND, sentIntent.getAction());
        assertEquals("application/pdf", sentIntent.getType());

        Uri uri = (Uri) sentIntent.getParcelableExtra(Intent.EXTRA_STREAM);
        assertNotNull(uri);
        // Verify that the URI grants read permission to the chooser
        int flag = sentIntent.getFlags();
        assertTrue((flag & Intent.FLAG_GRANT_READ_URI_PERMISSION) != 0);

        // Optionally, launch a known viewer (e.g., Chrome) and confirm it opens
        ComponentName chrome = new ComponentName("com.android.chrome", "com.google.android.apps.chrome.Main");
        device.executeShellCommand("am start -n " + chrome.flattenToString() + " -d " + uri.toString());
        device.wait(Until.hasObject(By.clazz("android.webkit.WebView")), 8000);
    }
}

What this test does

• Launches the app, loads data, and triggers the share flow.
• Captures the intent that the system sends to the chooser.
• Validates action, MIME type, and the presence of the FLAG_GRANT_READ_URI_PERMISSION flag.
• Optionally starts a viewer app to ensure the URI is readable.

Edge Cases that Only Appear in Production

Even with exhaustive matrices, certain conditions surface only after the app reaches a broad user base. Below are the most insidious ones, why they escape lab testing, and how to mitigate them.

Production‑level monitoring

• Integrate Firebase Crashlytics to capture native and Java exceptions that occur during report generation.
• Add custom keys: report_format, report_size_bytes, storage_free_mb to help triage.
• Use Firebase Performance Monitoring to trace the duration of the generation process; set alerts if the 95th‑percentile exceeds a threshold (e.g., 8 s on median device).

Checklist for Reports Generation Testing

Copy this list into your team’s wiki or Definition of Done (DoD) for any report‑related feature.

• [ ] Happy path verified for each supported format (PDF, CSV, Excel, HTML).
• [ ] Error paths: storage full, permission denial, interrupted write, network loss (if applicable).
• [ ] Locale matrix: at least en_US, fr_FR, ar_EG, ja_JP, zh_CN – verify numbers, dates, and currency symbols.
• [ ] Accessibility: font scale 100%–200%, TalkBack navigation, dark mode contrast ≥4.5:1.
• [ ] Security: file written to app‑specific directory or scoped storage; no world‑readable permissions; no PII in logs or Crashlytics custom keys.
• [ ] Privacy: opt‑out respected; if user disables analytics, report generation does not send telemetry.
• [ ] Sharing intent: correct MIME, FLAG_GRANT_READ_URI_PERMISSION granted, chooser appears, receiving app can open file.
• [ ] Background behavior: generation completes or is paused/resumed correctly under Doze, battery optimization, and foreground service restrictions.
• [ ] Multi‑window / multi‑display: UI remains usable, no overlapping controls, report preview scales.
• [ ] Concurrency: double‑tap or rapid successive generates do not corrupt files or crash the app.
• [ ] Performance: 95th‑percentile generation time < X s on median device (define X based on product spec).
• [ ] Regression: automated unit + instrumented tests pass on every PR; SUSA autonomous run reports no new crashes/ANRs on nightly builds.

Closing