Common Filters And Sorting Bugs and How to Catch Them

Common Filters And Sorting Bugs and How to Catch Them

Common Filters And Sorting Bugs and How to Catch Them

Filters and sorting are everyday features in web and mobile applications, yet they hide a surprising number of defects that slip past scripted tests. Users encounter missing results, duplicated entries, or chaotic ordering, which erodes trust and can lead to abandoned checkouts or faulty analytics. This guide walks through the most frequent filter and sorting bug patterns, shows how they manifest, explains why they arise, and gives concrete steps to reproduce, detect, fix, and prevent them. By the end you will have a ready‑to‑use test matrix, a checklist for manual and automated checks, and insight into how persona‑driven autonomous exploration surfaces issues that traditional automation overlooks.

1. Why Filters and Sorting Are Bug Prone

Filters and sorting sit at the intersection of data access, UI state, and business logic. A change in any layer—backend query, API pagination, client‑side state management, or UI component—can break the contract that the displayed list matches the user’s intent. Common root causes include:

• Implicit assumptions about data types (e.g., treating strings as numbers).
• Off‑by‑one errors in page or offset calculations.
• Inconsistent handling of null, undefined, or NaN values.
• Locale‑specific collation rules that differ between environments.
• Stale caches or memoized results that outlive underlying data updates.
• Missing secondary sort keys that leave order nondeterministic.
• Logical mistakes when combining multiple filter predicates (AND/OR confusion).
• Performance bottlenecks that cause timeouts only under load.
• Accessibility gaps where filter controls are not keyboard‑navigable.
• Race conditions when asynchronous fetches finish out of order.

Each of these can produce a visible symptom to the user while leaving unit tests green because the tests often hit a happy‑path dataset or mock deterministic responses.

2. Bug Pattern 1 – Off‑by‑One Page Index Errors

What it looks like

A user clicks “Next page” on a paginated table and sees the same items as the previous page, or the last page is completely empty. In mobile apps, infinite scroll may stop prematurely or duplicate the final batch.

Why it happens

Backend APIs often accept page (starting at 1) or offset (starting at 0). If the frontend mixes the two conventions, the calculated offset becomes page * pageSize instead of (page‑1) * pageSize. Conversely, when the backend returns totalCount and the frontend computes totalPages = Math.ceil(totalCount / pageSize), an off‑by‑one in the ceiling calculation can hide the final page when totalCount is an exact multiple of pageSize.

How to reproduce

1. Set page size to 10.
2. Insert exactly 30 records into the system.
3. Navigate to page 3 (should show records 21‑30).
4. Observe duplication or missing data.

Detection

• Unit test – mock the API with known totals and assert that the requested offset matches (page‑1)*size.
• Contract test – use tools like Pact to verify the API documentation matches the client’s offset calculation.
• Exploratory test – run a persona that repeatedly clicks next/previous while varying total record counts (e.g., 0, 1, size‑1, size, size+1, 2*size‑1).

Fix

• Centralize pagination logic in a single utility function.
• Add unit tests that cover boundary conditions: empty set, exact multiple, one‑item remainder.
• Log the computed offset and limit on each request for easy debugging.

Prevention checklist

• [] All pagination calls go through a shared helper.
• [] Helper is tested with zero, exact multiple, and remainder cases.
• [] API contract explicitly states whether page is zero‑ or one‑based.
• [] UI shows a disabled “Next” button when offset + size >= totalCount.

3. Bug Pattern 2 – Incorrect Comparator Logic (NaN, null handling)

What it looks like

Sorting a column of numbers places NaN or null values at unpredictable positions—sometimes at the top, sometimes at the bottom, sometimes causing the entire list to appear unsorted.

Why it happens

JavaScript’s Array.prototype.sort converts elements to strings unless a compare function is supplied. A naïve compare like return a - b yields NaN when either operand is NaN or null, and the sort algorithm treats NaN as “greater than” any number, leading to erratic ordering. Similar issues appear in SQL (ORDER BY col) where NULLS FIRST/LAST defaults differ between databases.

How to reproduce

1. Create a list: [5, null, 2, NaN, 9].
2. Apply the buggy comparator.
3. Observe that null and NaN end up interleaved or at the front.

Detection

• Unit test – feed arrays containing null, undefined, NaN, and negative numbers; assert the output matches a known sorted order (e.g., all non‑numeric values at the end).
• Property‑based test – generate random arrays with a known sort key and verify that the comparator is transitive and antisymmetric.
• Automated UI test – sort a column in the UI and verify that the visual order matches the expected order using a screenshot comparison or DOM inspection.

Fix

• Implement a safe comparator:

function safeNumberComparator(a, b) {
  if (a === null && b === null) return 0;
  if (a === null) return 1;   // treat null as larger
  if (b === null) return -1;
  if (Number.isNaN(a) && Number.isNaN(b)) return 0;
  if (Number.isNaN(a)) return 1; // NaN after numbers
  if (Number.isNaN(b)) return -1;
  return a - b;
}

• In SQL, explicitly declare ORDER BY col NULLS LAST (or FIRST) to match UI expectations.
• Add lint rules that flag bare a - b comparators without null/NaN guards.

Prevention checklist

• [] All custom comparators are reviewed for null/NaN handling.
• [] Unit test suite includes edge‑value cases for every sortable field.
• [] Database queries specify null ordering explicitly.
• [] UI tests assert that sorting does not move null/NaN rows unexpectedly.

4. Bug Pattern 3 – Locale‑Specific Sorting Failures

What it looks like

A list of names sorted alphabetically places “Ångström” after “Z” in a Swedish locale but before “A” in an English locale, causing confusion for users who expect a consistent order across language settings.

Why it happens

Developers often rely on the default JavaScript String.prototype.localeCompare without specifying a locale, which defaults to the runtime environment’s locale. In CI containers the locale may be C, while production servers run en_US.UTF-8. The result is different sort orders between test and production.

How to reproduce

1. Set environment locale to C.
2. Sort an array containing ['Ångström', 'Zebra', 'Apple'].
3. Observe order: ['Zebra', 'Apple', 'Ångström'].
4. Switch locale to sv_SE and repeat; order changes.

Detection

• Unit test – call the sorting function with a known set of international characters and assert the output matches the expected order for each supported locale.
• Contract test – verify that the API returns data already sorted in the requested locale, or that the client applies the locale correctly.
• Exploratory test – toggle the app’s language setting and verify that sorting behaves consistently.

Fix

• Always pass an explicit locale to localeCompare:

function sortByName(a, b, locale = 'en') {
  return a.name.localeCompare(b.name, locale, { sensitivity: 'base' });
}

• Store the preferred locale in user settings or accept it as an API query parameter (?sortLocale=sv).
• In backend SQL, use ORDER BY col COLLATE "sv_SE" when the locale is known.

Prevention checklist

• [] No bare localeCompare calls without a locale argument.
• [] Accept locale as a configurable parameter, fallback to a defined default (e.g., en-US).
• [] Unit tests cover each supported locale with a representative character set.
• [] Documentation states which locale governs sorting for each endpoint.

5. Bug Pattern 4 – Stale Cache Leading to Inconsistent Results

What it looks like

After deleting a record, the user still sees it in the filtered list until they manually refresh the page. In some cases, applying a filter after an edit shows outdated counts, causing mistrust in the UI.

Why it happens

Frontend state management libraries (Redux, MobX, React Query) often cache query results to avoid unnecessary network calls. If the cache invalidation logic does not listen to the relevant mutation events (e.g., deleteItem, updateItem), the stale data persists.

How to reproduce

1. Load a list of 20 items, apply a filter that matches 5 of them.
2. Delete one of the matching items via a separate API call.
3. Observe that the filtered list still shows 5 items, and the deleted item is present.
4. Refresh the page; the list now shows 4 items.

Detection

• Integration test – perform a mutation (create/update/delete) and immediately assert that a dependent query reflects the change without a manual refetch.
• Network mock – use tools like MSW to spy on outgoing requests; ensure that after a mutation the client either invalidates the cache or issues a fresh request.
• Exploratory test – run a persona that alternates between editing and filtering rapidly, checking for UI mismatches.

Fix

• In React Query, call queryClient.invalidateQueries({ queryKey: ['items'] }) after a mutation.
• In Redux, attach a reducer that removes the deleted item’s ID from the state slice used by the filter selector.
• Consider using optimistic updates: remove the item from the UI instantly, then revert on error.

Prevention checklist

• [] Every mutation that alters the filtered dataset triggers cache invalidation or state update.
• [] Unit tests for mutation‑query coupling are present.
• [] E2E scenarios verify that filter results stay in sync after CRUD operations.
• [] Logging shows when cache is cleared versus when a fresh fetch occurs.

6. Bug Pattern 5 – Missing Secondary Sort Keys Causing Non‑Deterministic Order

What it looks like

When two records have identical primary sort values (e.g., same timestamp), their order changes each time the list is reloaded, leading to flickering UI and difficulty reproducing bugs.

Why it happens

A sort comparator only considers the primary key. If the comparator returns 0 for equal keys, the underlying sort algorithm may leave the items in whatever order they happened to be in the array, which can vary due to asynchronous data fetching or pagination.

How to reproduce

1. Insert two records with identical createdAt values (e.g., both at 2024-01-01T12:00:00Z).
2. Sort by createdAt descending.
3. Reload the page multiple times; note that the two records swap positions intermittently.

Detection

• Unit test – provide an array with duplicate primary keys and assert that the secondary key (e.g., id) determines final order.
• Snapshot test – render the list, capture the DOM order, reload, and ensure the snapshot does not change.
• Load test – simulate concurrent inserts with the same timestamp and verify order stability.

Fix

• Extend the comparator to break ties with a deterministic secondary field:

function compareItems(a, b) {
  if (a.timestamp !== b.timestamp) return b.timestamp - a.timestamp;
  return a.id.localeCompare(b.id); // assuming string IDs
}

• In SQL, add the secondary column to the ORDER BY clause: ORDER BY timestamp DESC, id ASC.
• Ensure that the secondary key is unique or at least sufficiently varied to guarantee stability.

Prevention checklist

• [] Every sort definition includes a tie‑breaker field.
• [] Unit tests verify stable ordering when primary keys repeat.
• [] API documentation lists the full sort tuple for each endpoint.
• [] UI tests assert that repeated sorts produce identical sequences.

7. Bug Pattern 6 – Filter Combination Logic Errors (AND/OR Misuse)

What it looks like

A user selects “Category = Electronics” AND “Price < $100”. The results show items that are either electronics or cheap, effectively treating the criteria as OR. Conversely, a filter meant to show “Sale OR New Arrival” may incorrectly require both conditions, hiding valid items.

Why it happens

When building query objects or SQL WHERE clauses programmatically, developers sometimes mistakenly use || instead of && (or vice versa) when combining predicate functions, or they mishandle nested condition groups. UI state that stores each filter as an independent boolean can also lead to logic errors when the UI combines them incorrectly.

How to reproduce

1. Create a dataset:

• Item A: Category = Electronics, Price = $120
• Item B: Category = Books, Price = $80
• Item C: Category = Electronics, Price = $90

1. Apply filters: Category = Electronics AND Price < $100.
2. Expected result: only Item C.
3. Observe that Item A and B also appear (OR behavior) or that nothing appears (incorrect AND).

Detection

• Unit test – feed a known dataset and a set of filter toggles; assert the output matches the truth table of the intended logical expression.
• Property‑based test – generate random filter combinations and compare the client‑side filtered list against a reference implementation (e.g., using Array.filter with a pure function).
• Exploratory test – use a persona that rapidly toggles multiple filters and checks for impossible states (e.g., showing items that violate any active filter).

Fix

• Centralize filter evaluation in a pure function:

function passesAllFilters(item, filters) {
  return Object.entries(filters).every(([key, value]) => {
    if (value === null) return true; // no filter
    return item[key] === value;
  });
}

• For OR‑style filters, maintain a separate list and use .some.
• Write UI unit tests that simulate clicking each filter checkbox and verify the resulting list against the pure function.

Prevention checklist

• [] All filter logic is covered by unit tests with truth‑table validation.
• [] Filter state is immutable; derived lists are computed via pure functions.
• [] Code reviews check for accidental ||/&& swaps in complex conditionals.
• [] End‑to‑end scenarios validate that the UI never shows an item that contradicts an active filter.

8. Bug Pattern 7 – Performance‑Related Sorting Timeouts

What it looks like

On a product listing page with 100 000 items, clicking the “Sort by price” button causes the UI to freeze for several seconds, sometimes triggering a “page unresponsive” warning in the browser.

Why it happens

Client‑side sorting of large arrays blocks the main thread. If the dataset is fetched unsorted from the backend and the frontend attempts to sort it in‑place, the operation can exceed the frame budget (≈16 ms). Similar timeouts occur on the server when an unindexed ORDER BY forces a filesort on millions of rows.

How to reproduce

1. Load a list with 150 000 records (e.g., via a seed script).
2. Open the browser’s performance profiler.
3. Trigger the sort action and record the main‑thread blocking time.
4. Observe a blocking period > 200 ms.

Detection

• Benchmark test – use a script that times Array.sort on a cloned array of realistic size; assert the duration stays under a threshold (e.g., 50 ms).
• Lighthouse audit – run the performance audit and ensure “Main‑thread work” does not spike excessively on sort interactions.
• Backend explain – run EXPLAIN on the sorting query to verify index usage; flag any query that shows Using filesort.

Fix

• Push sorting to the backend whenever possible, ensuring the relevant column is indexed.
• If client‑side sorting is unavoidable, use web workers or requestIdleCallback to offload the work.
• Implement virtual scrolling so only the visible slice is sorted; the rest remains untouched.
• Add a loading spinner and disable the sort control while the operation is in progress.

Prevention checklist

• [] Sorting operations exceeding 50 ms on the main thread are flagged in performance tests.
• [] Backend queries that require sorting have appropriate indexes verified via EXPLAIN.
• [] Virtual scrolling or pagination is used for lists > 5 000 rows.
• [] UI shows a non‑blocking indicator during any sort that may take > 200 ms.

9. Bug Pattern 8 – Accessibility‑Related Filter UI Bugs

What it looks like

A screen‑reader user announces that the “Apply Filters” button is unavailable, or keyboard users cannot reach the filter dropdown because it is trapped inside a modal that loses focus.

Why it happens

Developers often build custom filter widgets using