How to Write Test Cases for Accessibility Settings (With Examples)

How to Write Test Cases for Accessibility Settings (With Examples)

How to Write Test Cases for Accessibility Settings (With Examples)

How to Write Test Cases for Accessibility Settings (With Examples) – Part 1: Test Case Anatomy

Writing a test case for accessibility settings starts with a clear structure that maps directly to requirements. Each case should contain a unique identifier, a short title, preconditions, detailed steps, and an expected result. The identifier often follows a pattern like ACC-SET-001 where the prefix denotes the feature area (accessibility settings) and the suffix is a sequential number. Traceability is achieved by linking the ID to a requirement document, such as WCAG 2.1 Success Criterion 1.4.3 (Contrast Minimum) or a internal spec that defines how a user can toggle high‑contrast mode.

Preconditions describe the state the device or application must be in before the test begins. For accessibility settings this might include: the device is unlocked, the Settings app is closed, no assistive services are running, and the user profile is a standard account without administrative restrictions. If the test depends on a specific locale or language, that should be noted here (e.g., device language set to English (United States)).

Steps are the atomic actions a tester performs. They should be written in imperative mood, numbered, and avoid ambiguous language. Example: “1. Open Settings → Accessibility → Display → High contrast text. 2. Toggle the switch to ON. 3. Verify that the switch appears highlighted.” Each step should be executable by a person with minimal domain knowledge and reproducible by an automation script.

Expected results define the observable outcome after the final step. They must be measurable: a UI element changes state, a system setting is persisted, an accessibility service announces a change, or a contrast ratio meets a threshold. Avoid vague phrasing like “the user should notice a difference”; instead state “the text color of all system UI elements changes from #FFFFFF to #000000, yielding a contrast ratio of 21:1 against the default background.”

Finally, include a pass/fail criterion that references the expected result. If any step fails to produce the expected outcome, the test is marked FAIL. Capture any error messages, logs, or screenshots as evidence. This anatomy forms the foundation for all subsequent categories of tests.

How to Write Test Cases for Accessibility Settings (With Examples) – Part 2: Positive/Negative/Edge Cases

Accessibility testing benefits from classifying cases into four buckets: positive, negative, edge, and boundary. Positive tests verify that a feature works as intended when used correctly. For a toggle like “Invert colors”, a positive case checks that enabling the switch actually inverts the display colors and that the setting survives a reboot.

Negative tests confirm that the system rejects invalid or disallowed actions. Examples include attempting to set a font size below the minimum allowed value, or trying to enable an accessibility service that requires a permission the user has denied. The expected result is usually an error message, a disabled control, or the setting reverting to its previous state.

Edge cases explore conditions at the extremes of normal operation but still within spec. For instance, changing the language while an accessibility shortcut is active, or rapidly toggling a switch ten times in succession. These tests often uncover race conditions or UI glitches that only appear under rapid interaction.

Boundary cases focus on the limits of input domains. If a setting accepts a numeric value (e.g., audio volume level for screen reader feedback), test the minimum, maximum, and just‑outside values. If the spec says volume can be set from 0 to 100 inclusive, test 0, 1, 99, 100, and also -1 and 101 to ensure proper clamping or rejection.

By populating each bucket with targeted cases you achieve a balanced suite that validates both typical usage and the robustness of the implementation under stress.

How to Write Test Cases for Accessibility Settings (With Examples) – Part 3: Worked Example Matrix

Below is a concrete test matrix covering common accessibility settings found on Android and iOS platforms. The table includes ID, preconditions, steps, and expected result. Feel free to copy‑paste into your test management tool and adapt the values to your specific product.

*Notes:*

• Adjust the exact menu paths to match your OS version; the logic remains identical.
• For each case, consider adding a negative variant (e.g., trying to set font size to 0 % or a non‑numeric value) and an edge variant (rapid toggling, changing locale while setting is active).

Data Setup and Environment Preparation

Reliable accessibility testing hinges on a consistent test lab. Begin by defining a device matrix that covers the OS versions you support. For Android, include at least one device running the latest API level, one running a two‑year‑old version, and one running a legacy version if your user base still contains it. For iOS, similarly cover the current major release, the previous release, and an older release still receiving security patches.

Prepare a base image for each device: factory reset, clear cache, disable developer options unless needed for logging, and set the locale to a known value (e.g., English (United States)). Install any required assistive technologies TalkBack, VoiceOver, Switch Control, or third‑party screen readers that your product supports.

If your tests rely on external data (e.g., a specific contact list for testing “Speak password”), script the provisioning using ADB for Android or idevice tools for iOS. Example ADB snippet to push a contacts XML file:

adb push contacts.xml /sdcard/Download/
adb shell am start -n com.android.providers.contacts/.ContactPickerActivity

For iOS, you can use ideviceinstaller to install a test app that populates the contacts database.

Capture logs systematically. On Android, enable logcat with a buffer size sufficient for accessibility events:

adb logcat -b main -b system -b events -v threadtime > accessibility_logcat.txt

On iOS, use the Console app or sysdiagnose to collect logs after each test run.

Finally, calibrate any measurement tools you intend to use for contrast or audio level verification. A colorimeter can confirm that the contrast ratio meets WCAG thresholds; a sound level meter can validate that mono audio sum matches the expected decibel level. Store calibration certificates with your test artifacts for auditability.

Prioritization and Risk-Based Selection

Not all test cases carry equal weight. Use a simple risk matrix that plots Impact (how severely a defect affects users with disabilities) against Likelihood (probability of the defect occurring given current code churn).

For accessibility settings, high‑impact items typically involve loss of core functionality (e.g., screen reader not announcing critical alerts, inability to toggle high contrast). Medium impact includes cosmetic issues like incorrect animation removal that may cause discomfort but not block usage. Low impact covers edge‑case visual glitches that rarely surface.

Likelihood is informed by recent code changes: if a touch‑handling module was refactored, any test that touches touch accommodation or switch access gains higher likelihood. Use your version‑control churn data to weight the matrix each sprint.

Select test cases for each build by taking all High‑Impact/Medium‑Likelihood and above items, plus a sampling of Medium‑Impact/Low‑Likelihood cases to maintain coverage. This approach yields a lean yet effective regression set that can be run on every commit, while the full matrix is executed nightly or before major releases.

Automating Accessibility Test Cases

Manual execution of the full matrix is time‑consuming, especially when you need to verify persistence across reboots or interaction with hardware accessories. Automation augments manual effort by covering repeatable steps and providing objective measurements.

Choosing the Right Framework

For native Android UI, Appium combined with UiAutomator2 offers reliable interaction with accessibility toggles. For iOS, XCUITest via Appium or direct XCTest works well. Web‑based settings (if you expose a web portal for accessibility preferences) can be automated with Playwright or Selenium.

Sample Automation Snippet (Appium + Java)

Below is a concise example that verifies the “Font size” setting persists after a reboot. It assumes you have configured an Appium server and have the desired capabilities for the target device.

import io.appium.java_client.AppiumDriver;
import io.appium.java_client.MobileBy;
import io.appium.java_client.MobileElement;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.remote.DesiredCapabilities;

import java.net.URL;
import java.util.concurrent.TimeUnit;

public class FontSizePersistenceTest {
    private AppiumDriver<MobileElement> driver;

    @Before
    public void setUp() throws Exception {
        DesiredCapabilities caps = new DesiredCapabilities();
        caps.setCapability("platformName", "Android");
        caps.setCapability("deviceName", "Pixel_4_API_33");
        caps.setCapability("automationName", "UiAutomator2");
        caps.setCapability("appPackage", "com.android.settings");
        caps.setCapability("appActivity", ".Settings");
        driver = new AppiumDriver<>(new URL("http://localhost:4789/wd/hub"), caps);
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
    }

    @Test
    public void testFontSizePersistsAfterReboot() {
        // Navigate to Font size setting
        driver.findElement(MobileBy.AccessibilityId("Accessibility")).click();
        driver.findElement(MobileBy.AndroidUIAutomator(
                "new UiScrollable(new UiSelector().scrollable(true)).scrollIntoView(" +
                "new UiSelector().textContains(\"Font size\"));")).click();

        // Select Largest option
        driver.findElement(MobileBy.AndroidUIAutomator(
                "new UiSelector().text(\"Largest\"));")).click();

        // Reboot device via ADB (requires privileged session)
        Runtime.getRuntime().exec("adb reboot");
        Thread.sleep(30000); // wait for device to come back online

        // Reconnect after reboot (Appium will handle reconnection if configured)
        driver = new AppiumDriver<>(new URL("http://localhost:4789/wd/hub"), caps);
        driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);

        // Re‑navigate to Font size and verify Largest is still selected
        driver.findElement(MobileBy.AccessibilityId("Accessibility")).click();
        driver.findElement(MobileBy.AndroidUIAutomator(
                "new UiScrollable(new UiSelector().scrollable(true)).scrollIntoView(" +
                "new UiSelector().textContains(\"Font size\"));")).click();

        MobileElement largestOption = driver.findElement(MobileBy.AndroidUIAutomator(
                "new UiSelector().text(\"Largest\").selected(true)"));
        assert largestOption != null : "Largest font size not persisted after reboot";
    }

    @After
    public void tearDown() {
        if (driver != null) {
            driver.quit();
        }
    }
}

Key points:

• Use accessibility IDs where possible; they are less prone to change than text labels.
• After a reboot, re‑establish the driver session; many cloud labs provide automatic reconnection.
• Validate the selected state via the selected(true) attribute rather than relying on UI text alone.

Sample Automation Snippet (Playwright for Web)

If your product exposes a web‑based accessibility settings panel, Playwright can assert ARIA attributes and computed styles.

const { test, expect } = require('@playwright/test');

test('high contrast toggle changes computed contrast ratio', async ({ page }) => {
  await page.goto('https://example.com/settings/accessibility');
  const toggle = page.locator('#high-contrast-toggle');
  await toggle.click();

  // Wait for CSS variable update
  await page.waitForFunction(() => {
    const root = document.documentElement;
    return getComputedStyle(root).getPropertyValue('--bg-color').trim() === '#000000';
  });

  // Compute contrast ratio using a tiny helper
  const contrast = await page.evaluate(() => {
    const bg = getComputedStyle(document.documentElement)
                    .getPropertyValue('--bg-color').trim();
    const fg = getComputedStyle(document.documentElement)
                    .getPropertyValue('--text-color').trim();
    // Simple sRGB conversion (assumes hex format)
    const hexToRgb = hex => {
      const shorthandRegex = /^#?([a-f\d])([a-f\d])([a-f\d])$/i;
      hex = hex.replace(shorthandRegex, (m, r, g, b) => r + r + g + g + b + b);
      const result = /^#?([a-f\d]{2})([a-f\d]{2})([a-f\d]{2})$/i.exec(hex);
      return result
        ? {
            r: parseInt(result[1], 16),
            g: parseInt(result[2], 16),
            b: parseInt(result[3], 16)
          }
        : null;
    };
    const rgbToHex = (r, g, b) => "#" + [r, g, b].map(x => x.toString(16).padStart(2, '0')).join('');
    const [r1, g1, b1] = hexToRgb(bg);
    const [r2, g2, b2] = hexToRgb(fg);
    const luminance = (r, g, b) => {
      const [rs, gs, bs] = [r, g, b].map(v => {
        const vv = v / 255;
        return vv <= 0.03928 ? vv / 12.92 : Math.pow((vv + 0.055) / 1.055, 2.4);
      });
      return 0.2126 * rs + 0.7152 * gs + 0.0722 * bs;
    };
    const L1 = luminance(r1, g1, b1) + 0.05;
    const L2 = luminance(r2, g2, b2) + 0.05;
    return Math.max(L1, L2) / Math.min(L1, L2);
  });

  expect(contrast).toBeGreaterThanOrEqual(4.5); // WCAG AA for normal text
});

This snippet clicks a toggle, waits for CSS custom properties to update, then computes the contrast ratio from the computed background and foreground colors. It asserts that the ratio meets WCAG AA (≥ 4.5:1 for normal text).

Leveraging SUSATest Autonomous Exploration

SUSATest can complement scripted tests by exploring accessibility settings without predefined steps. After uploading an APK or pointing the agent at a web URL, SUSATest creates a session that:

1. Discovers all reachable screens via automated taps, swipes, and text entry.
2. Applies each of its built‑in personas (e.g., “elderly”, “power user”, “accessibility”) which modify interaction speed, error tolerance, and assistive‑technology usage.
3. Records any changes to system settings, including toggles in the Accessibility pane, and flags anomalies such as a setting that fails to persist after a simulated reboot or a contrast ratio that drops below WCAG thresholds when a persona with reduced vision interacts.

Because the agent learns from prior runs, it avoids re‑exploring dead ends and focuses on newly added screens or altered UI hierarchies. This exploratory layer catches regressions that manual test case authors might overlook, such as a hidden setting that only appears after a specific locale is selected or a toggle that becomes disabled when a certain accessibility service is active.

You can feed the logs from SUSATest runs into your test management system as additional evidence, linking them to the relevant test case IDs (e.g., ACC-SET-007 for “Remove animations”). Over time, the autonomous exploration data informs prioritization: if a particular setting repeatedly shows failures across personas, raise its impact rating in the risk matrix.

Traceability, Reporting, and Continuous Improvement

Linking each test case to a requirement ensures that gaps are visible and that stakeholders understand the coverage. Use a traceability matrix that maps test IDs to WCAG success criteria, internal design specifications, and user stories. Example:

When a test fails, the report should include:

• Test ID and title
• Steps executed with screenshots or video
• Observed behavior vs. expected result
• Log excerpts (logcat, console, or system diagnostics)
• Link to the requirement(s) it validates
• Suggested severity based on the risk matrix

Integrate this reporting into your CI pipeline. If a test fails, the build can be marked unstable, and a ticket can be auto‑generated in Jira or Azure DevOps with the above details pre‑populated.

Continuous improvement comes from reviewing trends. Monthly, examine the failure rate per accessibility setting. If a specific toggle (e.g., “Speak password”) shows a high flakiness rate, investigate whether the underlying API is unstable or if the test needs better synchronization. Update the test case accordingly, add any missing preconditions, and consider promoting the setting to a higher priority if user impact data shows it is frequently used.

Finally, close the loop with SUSATest: after each autonomous exploration run, export the list of discovered accessibility-related issues and map them to existing test IDs. Any issue that does not match an existing case becomes a candidate for a new test script, expanding your coverage organically.

Checklist for Writing Accessibility Settings Test Cases

Use this concise list before finalizing a case. Tick each item to confirm completeness.

• [ ] ID & Traceability: Unique identifier, linked to WCAG criterion or internal spec.
• [ ] Title: Short, descriptive, includes the setting name and action (e.g., “Enable high contrast text”).
• [ ] Preconditions: Device state, OS version, locale, assistive‑technology status, any required accounts or permissions.
• [ ] Steps: Numbered, imperative, no ambiguous language, includes navigation paths and any required waits.
• [ ] Expected Result: Measurable outcome (UI state change, setting persistence, contrast ratio, audio metric, log entry).
• [ ] Pass/Fail Criterion: Directly ties to expected result; defines what constitutes a FAIL.
• [ ] Data Setup: Any test data (contacts, media files) required; how it is provisioned.
• [ ] Automation Feasibility: Indicates if the case can be automated, notes any special tools or privileges needed.
• [ ] Risk Rating: Impact × Likelihood from the risk matrix; used for test suite selection.
• [ ] Notes: Known flakiness, dependencies on hardware, or open questions for the dev team.

Run through the checklist for every new case and during periodic test‑suite reviews.

Closing Takeaways

Writing high‑signal test cases for accessibility settings demands a disciplined structure, clear traceability, and a balanced mix of positive, negative, edge, and boundary scenarios. Begin by defining a solid anatomy—ID, preconditions, steps, expected result—and tie each case to a requirement such as WCAG 2.1 AA. Populate your suite with examples that cover typical user interactions as well as stress conditions like rapid toggling, locale shifts, and hardware changes.

Prioritize using a simple risk‑based matrix so that your regression suite remains fast yet meaningful, and automate the repeatable steps with tools like Appium, XCTest, or Playwright. Complement scripted efforts with autonomous exploration platforms such as SUSATest, which can surface hidden regressions and provide persona‑driven insights that manual authors might miss.

Maintain traceability through a matrix that connects test IDs to specifications, user stories, and accessibility guidelines, and feed failures back into your issue tracker with rich diagnostics. Regularly review failure trends, update flaky tests, and adjust priorities based on real‑world usage data.

By following this guide, you will build a robust, maintainable set of accessibility settings test cases that not only verify compliance but also improve the experience for users with disabilities. The result is a product that respects inclusive design principles and reduces the likelihood of accessibility‑related escapes in production.

---

*Feel free to adapt the tables, snippets, and checklist to your specific technology stack