Search Functionality Testing Checklist (2026)
Search Functionality Testing Checklist (2026)
Search Functionality Testing Checklist (2026)
A search bar is often the first point of interaction for users trying to find content, products, or data. Verifying that it behaves correctly under normal use, abnormal input, and stress conditions prevents frustration, lost conversions, and security gaps. This guide gives you a concrete, check‑able matrix you can apply to any web or mobile product, with pass/fail criteria, real‑world examples, and notes on how much of the work can be done autonomously by a tool like SUSATest.
1. Happy Path Test Cases
The happy path confirms that the core search workflow works when everything is as expected. Each item below should be executable manually and, where practical, automated with a UI‑level script or API call.
1.1 Basic Query Execution
| Test ID | Description | Pass Criteria | Automation Level |
| HP‑01 | Enter a single‑word query that matches at least one record and press Enter or tap the search icon. | Results list contains at least one item whose title or snippet includes the exact query term (case‑insensitive). Highlighting of the term appears in the UI. | UI test (e.g., Playwright) |
| HP‑02 | Enter a multi‑word phrase that matches records containing all words in any order. | Returned set equals the intersection of individual word matches; no extraneous items. | UI test |
| HP‑03 | Submit an empty query (just press search with no text). | System either shows a placeholder message like “Type to search” or returns all items if that is the designed behavior; no error dialog appears. | UI test |
| HP‑04 | Use voice input (if supported) to speak a query and submit. | Transcribed text appears in the box and results match the transcribed query. | UI test with microphone mock |
1.2 Result Ranking and Relevance
| Test ID | Description | Pass Criteria | Automation Level |
| HP‑05 | Query a term that appears in title, description, and tags of different items. | Items with the term in the title rank higher than those with it only in description or tags. | API test + ranking verification |
| HP‑06 | Apply a known boost (e.g., promoting a product) and search for the boosted item. | Boosted item appears in the top three positions. | API test |
| HP‑07 | Search with a synonym that the backend maps to the same concept (e.g., “sneaker” → “running shoe”). | Results include items tagged with the synonym’s target term. | API test |
1.3 Pagination and Infinite Scroll
| Test ID | Description | Pass Criteria | Automation Level |
| HP‑08 | Scroll to the bottom of a paginated results page (page size 20). | Next page loads automatically, URL updates with correct page token, and no duplicate items appear. | UI test |
| HP‑09 | Jump directly to page 5 via a page selector. | Displayed items correspond to offset 80‑99 (assuming zero‑based) and total count matches backend. | UI test |
| HP‑10 | In infinite‑scroll mode, scroll rapidly past the 200th item. | No blank spots; each new batch arrives within 300 ms of the scroll event. | UI performance test |
1.4 Filters and Facets
| Test ID | Description | Pass Criteria | Automation Level |
| HP‑11 | Apply a single facet (e.g., category = “Electronics”). | Result set reduces to only items whose category matches; facet count updates correctly. | UI test |
| HP‑12 | Combine two facets (category and price range). | Result set equals intersection of both facet filters. | UI test |
| HP‑13 | Clear all facets via a “Reset” button. | UI returns to original unfiltered result set; query term remains unchanged. | UI test |
| HP‑14 | Select a facet that yields zero matches. | UI shows an empty state message and disables further facet selection until query changes. | UI test |
1.5 Sorting Options
| Test ID | Description | Pass Criteria | Automation Level |
| HP‑15 | Sort by price low‑to‑high. | Displayed prices are monotonic non‑decreasing; ties broken by secondary sort (e.g., name). | UI test |
| HP‑16 | Sort by relevance then date newest first. | First screen shows most relevant items; among equal relevance, dates descend. | UI test |
| HP‑17 | Change sort order while a filter is active. | New order respects both the active filter and the selected sort criterion. | UI test |
1.6 Persistence and State
| Test ID | Description | Pass Criteria | Automation Level |
| HP‑18 | Perform a search, navigate away to a product detail page, then press the browser back button. | Search box. | Search box repopulated | UI test |
| HP‑19 | Refresh the page after a box retains the query, results page shows same scroll position and pagination state. | UI test |
| HP‑19 | Share a search URL (e.g., ?q=laptop&page=2). | Opening that URL in a new tab shows the same query, page, and results. | API test + URL validation |
| HP‑20 | Lose network connectivity after results are loaded, then regain it. | UI shows an offline indicator but does not lose already rendered results; a retry restores fresh data. | UI test |
2. Error Handling and Invalid Input
A robust search must gracefully handle malformed, unexpected, or hostile input without crashing or leaking information.
2.1 Malformed Queries
| Test ID | Description | Pass Criteria | Automation Level |
| EH‑01 | Enter only special characters (!@#$%^&*()). | System treats them as literal characters if supported, otherwise shows “No results” or a helpful hint; no exception. | UI test |
| EH‑02 | Paste a string longer than the maximum allowed length (e.g., 5000 chars). | Input is either truncated to the limit with a subtle indicator or rejected with an inline validation message; no server‑side error. | UI test |
| EH‑03 | Enter leading/trailing whitespace only. | Whitespace is trimmed before submission; behavior matches empty query case. | UI test |
| EH‑04 | Input containing Unicode control characters (e.g., \u0000, \uFEFF). | Characters are stripped or replaced; search proceeds without crash. | UI test |
2.2 SQL‑Like Injection Attempts
| Test ID | Description | Pass Criteria | Automation Level |
| EH‑05 | Submit ' OR '1'='1. | Input is escaped or parameterized; result set equals that of a literal search for the string ' OR '1'='1 (typically zero matches). No error page revealing stack trace. | API test |
| EH‑06 | Submit ; DROP TABLE users;--. | Same as above; no modification to database; response time remains within normal bounds. | API test |
| EH‑07 | Submit <script>alert(1)</script> as query. | Characters are HTML‑encoded in the UI; script does not execute. | UI test + CSP verification |
2.3 Unexpected Data Types
| Test ID | Description | Pass Criteria | Automation Level |
| EH‑08 | Send a JSON payload with a number instead of a string for the q field via API. | API returns 400 Bad Request with a clear validation error (“q must be a string”). | API test |
| EH‑09 | Send an array as the query value. | Same as above; response includes error code and message. | API test |
| EH‑10 | Submit a search request with a missing q parameter. | API returns 400 and indicates required field missing. | API test |
2.4 User‑Facing Feedback
| Test ID | Description | Pass Criteria | Automation Level |
| EH‑11 | Invalid input triggers inline validation (e.g., red border, message “Please enter at least one character”). | Message appears within 200 ms of input loss of focus; does not obscure other UI elements. | UI test |
| EH‑12 | Server returns a 500 error due to internal failure. | UI shows a generic “Something went wrong, please try again” dialog; no technical details exposed. | UI test + monitoring check |
| EH‑13 | Rate‑limit exceeded (e.g., >10 requests/sec). | UI shows “Too many requests, try again later” and disables the search button for the cool‑down period. | UI test + mock server |
3. Edge and Boundary Cases
These tests target the limits of input size, value ranges, and uncommon linguistic phenomena that often slip through basic test suites.
3.1 Length Boundaries
| Test ID | Description | Pass Criteria | Automation Level |
| EB‑01 | Minimum allowed query length (e.g., 2 characters). | Query of exactly 2 characters returns results if matches exist; otherwise shows “No results”. | UI test |
| EB‑02 | Maximum allowed query length (e.g., 100 characters). | Query of exactly 100 characters is accepted; system does not truncate silently. | UI test |
| EB‑03 | One character beyond maximum (101 chars). | Input is rejected with validation message; no server call made. | UI test |
| EB‑04 | Query consisting solely of spaces up to the max length. | Treated as empty after trim; behaves like empty query case. | UI test |
3.2 Unicode and Internationalization
| Test ID | Description | Pass Criteria | Automation Level |
| EB‑05 | Enter a query in a right‑to‑left language (Arabic, Hebrew). | Text aligns correctly; cursor movement respects RTL direction; results display correctly. | UI test |
| EB‑06 | Use emojis (😀, 🚀) as part of the query. | If emojis are indexed, results include items containing those emojis; otherwise system treats them as regular characters and returns “No results”. | UI test |
| EB‑07 | Query with combining accents (e.g., é vs é). | Normalization (NFC/NFD) yields same result set; no missing matches due to different byte sequences. | API test |
| EB‑08 | Search for a CJK word without spaces (e.g., “スマートフォン”). | Tokenizer correctly splits or matches the term; results appear as expected. | UI test |
3.3 Numeric and Date Boundaries
| Test ID | Description | Pass Criteria | Automation Level |
| EB‑09 | Search for a negative number (-10) when the field stores only positive integers. | System returns zero results; no crash. | UI test |
| EB‑10 | Search for a date far in the future (2099-12-31). | Same as above; if date range filtering exists, UI shows no matches. | UI test |
| EB‑11 | Use a date format not supported by the parser (31/12/2020). | Validation message indicates unsupported format; no server error. | UI test |
| EB‑12 | Search with a fractional price (19.99) when the backend stores cents as integer. | System converts correctly or treats as string; results reflect correct rounding or no matches. | UI test |
3.4 Whitespace and Separator Variations
| Test ID | Description | Pass Criteria | Automation Level |
| EB‑13 | Query with multiple consecutive spaces (“hello world”). | Internal tokenizer collapses whitespace; results match “hello world”. | UI test |
| EB‑14 | Query containing tabs or newlines (“hello\tworld”). | Whitespace characters are stripped or treated as space; no crash. | UI test |
| EB‑15 | Leading/trailing punctuation (“!!hello!!”). | Punctuation is either ignored or searched literally based on spec; behavior is documented and consistent. | UI test |
| EB‑16 | Query with mixed language script (“привет hello”). | Each token is processed according to its language analyzer; results reflect both language matches. | UI test |
3.5 Result Set Extremes
| Test ID | Description | Pass Criteria | Automation Level |
| EB‑17 | Query that matches every record in a catalog of 10 M items. | System returns paginated results; first page loads within 2 s; total count is accurate. | Load test |
| EB‑18 | Query that matches zero records. | UI shows a clear “No results found” message with suggestions (e.g., try different spelling). No empty page layout breakage. | UI test |
| EB‑19 | Query that matches exactly one record. | UI highlights the single item; facets reflect that item’s attributes; no pagination controls shown. | UI test |
| EB‑20 | Query that returns a result set where the first item has a very long title (>500 chars). | Title truncates gracefully with ellipsis; layout does not overflow; tooltip shows full text on hover. | UI test |
4. Accessibility (WCAG) Checks
Ensuring search is usable by people with disabilities is not optional; it also improves SEO and overall quality.
4.1 Keyboard Navigation
| Test ID | Description | Pass Criteria | Automation Level |
| AC‑01 | Tab into the search input from any preceding element. | Input receives visible focus indicator (minimum 3 px contrast). | UI test |
| AC‑02 | Press Enter while focus is in the input to submit. | Search triggers without requiring mouse click. | UI test |
| AC‑03 | Use Arrow keys to navigate autocomplete suggestions (if present). | Focus moves among suggestions; Enter selects the highlighted suggestion. | UI test |
| AC‑04 | Escape key closes an open suggestion list and returns focus to the input. | UI state resets correctly. | UI test |
4.2 Screen Reader Support
| Test ID | Description | Pass Criteria | Automation Level |
| AC‑05 | Input has an associated <label> or aria-label that reads “Search”. | Screen reader announces the purpose when the input gains focus. | UI test + axe core |
| AC‑06 | Live region (aria-live="polite") announces the number of results returned. | After search completes, reader says “X results found”. | UI test |
| AC‑07 | Each result item has a unique, descriptive aria-label (e.g., product name + price). | Reader can differentiate items without relying on visual layout. | UI test |
| AC‑08 | Facet checkboxes have accessible names and state (aria-checked). | Reader announces “Filter, Electronics, checked” when toggled. | UI test |
4.3 Color and Contrast
| Test ID | Description | Pass Criteria | Automation Level |
| AC‑09 | Placeholder text meets WCAG AA contrast (≥4.5:1) against input background. | Verified with contrast checker. | Automated tool (axe) |
| AC‑10 | Active/focused input border has a contrast ratio ≥3:1 against surrounding surface. | Verified. | Automated tool |
| AC‑11 | Error or validation messages use a color contrast ≥4.5:1 and are also conveyed via text or icon. | Verified. | Automated tool + manual review |
4.4 Touch Target Size
| Test ID | Description | Pass Criteria | Automation Level |
| AC‑12 | Search button or icon has a minimum touch target of 48 × 48 dp (Android) or 44 × 44 pt (iOS). | Measured via UI inspector; no smaller tappable area. | UI test |
| AC‑13 | Sufficient spacing (≥8 dp) between search button and adjacent elements to avoid mis‑taps. | Verified. | UI test |
4.5 Reduced Motion and Animations
| Test ID | Description | Pass Criteria | Automation Level |
| AC‑14 | If autocomplete shows a fade‑in animation, respecting prefers-reduced-motion reduces or removes it. | Verified via devtools emulation. | UI test |
| AC‑15 | Infinite scroll does not trigger vestibular‑disorienting parallax effects when reduced motion is enabled. | Verified. | UI test |
5. Security and Privacy Considerations
Search can be an injection vector and may inadvertently expose sensitive data via logs, autocomplete suggestions, or caching.
5.1 Input Sanitization and Injection
| Test ID | Description | Pass Criteria | Automation Level |
| SE‑01 | Attempt stored XSS via a query that includes <img src=x onerror=alert(1)>. | Characters are escaped in the rendered results; no script execution. | UI test + CSP report |
| SE‑02 | Attempt reflected XSS via URL parameter (?q=<script>alert(1)</script>). | Response contains HTML‑escaped query; script does not run. | API test |
| SE‑03 | Attempt to inject newline characters to break JSON payload (q="\nadmin"). | Server treats the newline as part of the string; no parsing error. | API test |
| SE‑04 | Attempt to overflow a buffer with a very long Unicode string (e.g., 100 KB of \uFFFF). | Input is rejected or truncated; no crash or memory exhaustion observed. | Fuzz test (AFL/libFuzzer) |
5.2 Autocomplete Privacy
| Test ID | Description | Pass Criteria | Automation Level |
| SE‑05 | Autocomplete suggestions should not surface personally identifiable information (PII) from other users’ queries. | Verify that suggestions list contains only generic terms or popular queries, never email addresses, phone numbers, etc. | Manual audit + logs review |
| SE‑06 | If the system stores recent queries for the current session, they are cleared on logout or after a configurable timeout. | Verify storage is scoped and cleared appropriately. | UI test + devtools storage inspection |
| SE‑07 | Disable or obfuscate query logging in production environments unless strictly required for debugging, and ensure logs are encrypted at rest. | Confirm via logging configuration review. | Documentation check |
5.3 Rate Limiting and Abuse Prevention
| Test ID | Description | Pass Criteria | Automation Level |
| SE‑08 | Simulate a burst of 20 requests per second from a single IP. | Server responds with 429 Too Many Requests after the threshold; includes Retry-After header. | Load test (k6) |
| SE‑09 | Distributed attack simulation using 100 IPs each sending 5 rps. | System still enforces per‑IP or per‑account limits; overall latency remains within SLA. | Load test |
| SE‑10 | Captcha or challenge appears after a predefined number of failed attempts (e.g., 5 consecutive zero‑result queries). | UI presents challenge; search is blocked until passed. | UI test |
5.4 Secure Transmission
| Test ID | Description | Pass Criteria | Automation Level |
| SE‑11 | All search-related API calls use HTTPS with a valid certificate; no mixed content. | Verified via network tab; no http:// requests. | UI test |
| SE‑12 | Sensitive query parameters (e.g., containing a password reset token) are never sent via GET; they appear only in POST body if required. | Confirmed by inspecting requests. | Manual review |
6. Performance and Load Testing
Performance directly affects user satisfaction and conversion. This section defines measurable benchmarks and how to verify them.
6.1 Response Time Goals
| Metric | Target (95th percentile) | Measurement Tool | Notes |
| Full‑text search latency (API) | ≤250 ms | k6, Gatling, or JMeter | Includes network, DB query, ranking |
| First contentful paint (FCP) after search | ≤1.2 s | Lighthouse, Web Vitals | Measures UI rendering |
| Time to interactive (TTI) after results load | ≤2.0 s | Lighthouse | Ensures user can scroll/filter quickly |
| Autosuggest latency (per keystroke) | ≤100 ms | Synthetic monitoring | Keeps UI responsive |
| Infinite‑scroll batch load | ≤300 ms for 20 items | Custom script measuring scroll event to DOM update | Prevents jank |
6.2 Load Scenarios
| Scenario | Virtual Users | Ramp‑Up | Duration | Success Criteria |
| Light traffic (typical day) | 50 | 2 min | 10 min | 99 % of requests < 250 ms, zero 5xx |
| Peak traffic (sale event) | 500 | 5 min | 15 min | 95 % < 500 ms, error rate < 0.5 % |
| Stress test (breakpoint) | 2000 | 10 min | 20 min | Observe graceful degradation; no crash, error rate may rise but system stays up |
| Soak test (memory leak) | 100 | 5 min | 12 h | Memory growth < 5 % over period, no GC spikes |
6.3 Resource Utilization
| Metric | Target | Tool |
| CPU usage per search node | ≤60 % avg | Prometheus + node‑exporter |
| Memory usage per search node | ≤1 GB | Prometheus |
| Query cache hit rate | ≥80 % | Redis/Memcached stats |
| Database read IOPS | < 2000 | CloudWatch/Datadog |
6.4 Performance Test Script Example (k6)
import http from 'k6/http';
import { check, sleep } from 'k6';
export const options = {
stages: [
{ duration: '2m', target: 50 }, // ramp‑up
{ duration: '10m', target: 50 }, // steady
{ duration: '2m', target: 0 }, // ramp‑down
],
};
const BASE_URL = 'https://api.example.com/search';
export default function () {
const payload = JSON.stringify({
q: 'laptop',
page: 1,
pageSize: 20,
});
const params = {
headers: {
'Content-Type': 'application/json',
},
timeout: '10s',
};
const res = http.post(`${BASE_URL}`, payload, params);
check(res, {
'status is 200': (r) => r.status === 200,
'response time < 250ms': (r) => r.timings.duration < 250,
'has results': (r) => r.json().totalHits > 0,
});
sleep(1);
}
Run with: k6 run search-test.js
6.5 Monitoring Alerts (Prometheus)
- alert: SearchLatencyHigh
expr: histogram_quantile(0.95, sum(rate(search_request_duration_seconds_bucket[5m])) by (le)) > 0.25
for: 2m
labels:
severity: warning
annotations:
summary: "Search 95th‑latency > 250 ms"
description: "The search service is slower than expected on {{ $labels.instance }}."
7. Release Readiness and Regression
Before tagging a release, run through this checklist to guarantee that search does not introduce regressions and that all necessary documentation and monitoring are in place.
7.1 Feature Flag Validation
| Test ID | Description | Pass Criteria |
| RR‑01 | Deploy with the new search ranking algorithm behind a flag; verify that flag‑off yields legacy behavior. | Compare result sets for a sample of 100 queries; they must be identical when flag is off. |
| RR‑02 | Enable flag for 10 % of traffic via canary; monitor error rate and latency. | No statistically significant increase in errors or latency vs baseline. |
| RR‑03 | Ability to toggle flag at runtime without restarting service. | Verify via admin endpoint; change propagates within 30 s. |
7.2 Automated Regression Suite
| Test ID | Description | Pass Criteria |
| RR‑04 | Run the full happy‑path matrix (HP‑01‑HP‑20) on the staging build. | All tests pass; no new failures. |
| RR‑05 | Run error‑handling matrix (EH‑01‑EH‑13). | All tests pass; no new failures. |
| RR‑06 | Run accessibility automated scans (axe, WCAG) on the search page. | No new violations; existing violations unchanged. |
| RR‑07 | Execute performance k6 script at baseline load. | 95th‑percentile latency within acceptable limits; no significant increase versus previous release. |
| RR‑08 | Validate that generated Appium/Playwright scripts from SUSA (if used) still pass after code changes. | Script execution yields same PASS/FAIL outcomes as baseline. |
7.3 Documentation and Runbooks
| Item | Description | Pass Criteria |
| RR‑09 | Update API reference with any new query parameters, enumeration values, or response fields. | Reference renders correctly; examples are copy‑pasta‑able and produce expected results. |
| RR‑10 | Add or modify troubleshooting guide for common search issues (e.g., “No results despite correct spelling”). | Guide is searchable in internal wiki; includes steps and expected logs. |
| RR‑11 | Ensure alerting rules (see Section 6.5) are present in monitoring repo and pass linting. | No syntax errors; alerts fire in test environment when condition simulated. |
| RR‑12 | Verify that the search feature appears in the release notes with a clear user‑impact statement. | Notes are present, concise, and mention any behavior change (e.g., new ranking factor). |
7.4 Sign‑off Checklist (One‑Page)
[ ] All happy‑path tests pass (HP‑01‑HP‑20)
[ ] All error‑handling tests pass (EH‑01‑EH‑13)
[ ] No new WCAG violations (axe score ≥ 90)
[ ] 95th‑p latency ≤ 250 ms under load test
[ ] Feature flag toggles work as expected
[ ] No regression in autogenerated Appium/Playwright scripts
[ ] Documentation updated (API, troubleshooting, release notes)
[ ] Monitoring alerts in place and silent
[ ] Security review completed (injection, rate‑limit, PII)
[ ] Performance baseline recorded for future comparison
If any item is unchecked, the release is blocked until resolved.
8. How Autonomous Exploration Covers Most of This Checklist in One Pass
Modern autonomous QA platforms can exercise a large portion of the search checklist without writing a single test script. Below is a mapping of the checklist items to what a tool like SUSATest (the autonomous agent that explores an app or website) can discover on its own.
| Checklist Area | What SUSA Does Automatically | How It Maps to Manual Items |
| Happy path (HP‑01‑HP‑04) | Agent launches the app, locates the search bar via accessibility labels or placeholder text, types a random dictionary word, submits, and verifies that results change. | Confirms basic query execution, empty query handling, and voice input (if mic permission granted). |
| Ranking & relevance (HP‑05‑HP‑07) | By logging the order of returned titles for a set of predefined queries, the agent can detect regressions in ranking when a new model is deployed. | Validates that title matches outrank description matches and that boosted items appear top‑k. |
| Pagination & infinite scroll (HP‑08‑HP‑10) | Agent scrolls to the bottom of the results list, waits for network calls, and checks that new items appear without duplication. It also attempts to jump to a specific page using UI controls if present. | Checks next‑page loading, direct page jump, and rapid‑scroll performance. |
| Facets & filters (HP‑11‑HP‑14) | Agent identifies filter UI (checkboxes, sliders, toggles) by role and state, toggles each, and asserts that the result set shrinks accordingly. It also tries clearing all filters. | Validates single‑facet, multi‑facet, reset, and zero‑match states. |
| Sorting (HP‑15‑HP‑17) | Agent locates sort dropdown, selects each option, and captures the first few items to ensure monotonic order (price, date, relevance). | Confirms low‑to‑high, relevance‑then‑date, and sort‑under‑filter behavior. |
| State persistence (HP‑18‑HP‑20) | Agent navigates away from results, then uses back/button or deep link to return, comparing query box content and scroll offset. It also shares the current URL and opens it in a new tab/window. | Checks URL persistence, back‑button behavior, and offline‑then‑online resilience. |
| Error handling (EH‑01‑EH‑04) | Agent feeds the search bar with strings of special characters, very long strings, whitespace only, and Unicode control characters, then observes whether the app crashes or shows an error message. | Confirms graceful handling of malformed input. |
| Injection attempts (EH‑05‑EH‑07) | Agent submits common SQLi and XSS payloads, inspects network responses for reflected payloads, and checks the DOM for script tags or unexpected HTML. | Validates sanitization and CSP and parameterized queries block exploitation. |
| Edge length bounds (EB‑01‑EB‑04) | Agent uses a data‑driven loop to type strings of lengths min‑1, min, max, max+1 and records validation messages or truncation. | Checks minimum, maximum, and over‑limit behavior. |
| Internationalization (EB‑05‑EB‑08) | Agent switches device locale or injects RTL/Unicode strings, emojis, and combining characters, then confirms that UI renders correctly and results are consistent. | Verifies RTL, emojis, normalization, and CJK handling. |
| Numerics & dates (EB‑09‑EB‑12) |