Common File Upload Bugs and How to Catch Them

Common File Upload Bugs and How to Catch Them

Common File Upload Bugs and How to Catch Them

File upload functionality is a common feature in modern applications, yet it remains one of the most error‑prone areas. Attackers and everyday users alike can trigger crashes, data leaks, or denial‑of‑service simply by interacting with the upload widget in unexpected ways. This guide walks through the most frequent bug patterns, explains why they arise, shows what they look like to users, and provides concrete steps to reproduce, detect, fix, and prevent each issue. The material is organized so you can copy the test matrix and checklist into your own test plan and start hunting these defects today.

1. Why File Upload Is a Hotspot for Bugs

Upload endpoints sit at the intersection of several subsystems: HTTP parsing, input validation, file system interaction, downstream processing (image resizing, virus scanning, etc.), and access control. Each layer introduces assumptions that can be violated. When a developer trusts the client‑provided filename or MIME type without verification, the door opens for injection, path traversal, or privilege escalation. When size limits are missing or poorly enforced, a malicious user can exhaust disk space or memory. When temporary files are not cleaned up securely, race conditions or information disclosure appear. Because the flow touches so many moving parts, scripted tests that only hit the “happy path” often miss the edge cases that surface only under stress or with atypical user behavior.

1.1 Complexity of validation

Validation is not a single check; it is a chain of checks that must all pass. A typical validation pipeline might look like:

1. Verify Content‑Type header matches an allowed list.
2. Inspect file extension against a whitelist.
3. Read the first few bytes to confirm magic numbers (file signatures).
4. Enforce size limits before writing to disk.
5. Scan the file for known malware patterns.
6. Store the file using a generated safe name.
7. Set appropriate permissions and log the upload.

If any step is missing, incorrectly ordered, or based on client‑controlled data, an attacker can slip through. For example, relying only on the extension lets a user rename a .exe to .jpg and bypass a whitelist that only checks the suffix.

1.2 Interaction with storage and processing

After validation, the file is often handed off to other services: a thumbnail generator, a document parser, or a search indexer. Those services may have their own expectations about file format, size, or naming conventions. A file that passes upload validation can still cause a downstream crash if the processor assumes, say, that a PDF is not encrypted or that an image width fits in a 16‑bit integer. Moreover, storage systems may impose limits on path length, character set, or number of inodes, which can turn a seemingly innocuous upload into a denial‑of‑service when many large files are uploaded in quick succession.

2. Bug Pattern #1: Missing or Weak File Type Validation

2.1 What happens

The endpoint accepts any file regardless of its actual content, trusting only the client‑supplied MIME type or extension. An attacker uploads a script (e.g., shell.php) disguised as an image. If the application later serves uploaded files statically, the script can be executed, leading to remote code execution (RCE). Even when files are not served directly, some frameworks treat files with certain extensions as executable during deployment (e.g., PHP auto‑prepend).

2.2 How to reproduce

1. Prepare a file with a benign extension but malicious content, such as:

   <?php system($_GET['cmd']); ?>

Save it as evil.jpg.

1. Use a tool like curl to POST the file:

   curl -F "upload=@evil.jpg;type=image/jpeg" https://example.com/upload

1. If the response indicates success and the file is stored, request it directly:

   curl https://example.com/uploads/evil.jpg?cmd=id

A return of the uid output confirms execution.

2.3 Detection (manual & automated)

*Manual*: Maintain a list of dangerous extensions (.php, .jsp, .asp, .sh, .py, .pl) and attempt uploads with those extensions while varying the Content‑Type header. Observe whether the server stores the file unchanged and whether it is later accessible or executable.

*Automated*: In a unit test, mock the validation layer and assert that a request with a disallowed extension returns a 4xx error. In an integration test, use a security scanner (e.g., OWASP ZAP) with an active upload rule that tries to upload a PHP shell and checks for a 200 response containing the shell output.

2.4 Fix and prevention

• Never trust client‑provided MIME type or extension as the sole authority.
• Verify the file’s magic numbers (e.g., JPEG starts with FF D8 FF E0).
• Maintain a strict whitelist of allowed mime types and extensions; reject anything else.
• Store uploaded files with a randomly generated name and serve them only through a controlled handler that sets Content-Disposition: attachment and strips executable headers.
• If the application must display images, re‑encode them (e.g., using ImageMagick) to strip any embedded scripts.

3. Bug Pattern #2: Insufficient Size Limits Leading to DoS

3.1 What happens

When the upload endpoint does not enforce a maximum file size, or when the limit is set unrealistically high, an attacker can upload gigabyte‑sized files. This consumes disk space, exhausts temporary storage, or blocks the upload thread pool, causing legitimate users to experience timeouts or failures. In cloud environments, excessive storage usage can lead to unexpected billing spikes.

3.2 How to reproduce

1. Generate a large file (e.g., 5 GB) using dd:

   dd if=/dev/zero of=huge.bin bs=1M count=5000

1. Attempt to upload it via the web form or API.

1. Monitor server resources (disk, memory, CPU) during the upload. If the upload proceeds without error and the server’s available space drops sharply, the size limit is missing or ineffective.

3.3 Detection (manual & automated)

*Manual*: Try uploading files just above the advertised limit (if any) and files that are multiples of that limit. Look for error responses (413 Payload Too Large) versus success.

*Automated*: In a CI pipeline, include a test that sends a multipart request with a body larger than the configured maxUploadSize (e.g., set in nginx client_max_body_size or application config). Assert that the response status is 413 and that no file is written to disk.

3.4 Fix and prevention

• Enforce a hard size limit at the earliest possible point (web server, load balancer, or application filter).
• Return a clear 413 error with a helpful message when the limit is exceeded.
• Stream the upload to a temporary file while checking the running byte count; abort if the limit is surpassed before completing the read.
• Set up monitoring alerts for rapid increases in upload‑related storage usage.

4. Bug Pattern #3: Path Traversal via Filename

4.1 What happens

If the application uses the user‑supplied filename (or part of it) to construct a storage path without proper sanitization, an attacker can include directory traversal sequences (../) to write or overwrite files outside the intended upload directory. This can lead to remote code execution (by overwriting a server script), denial of service (by corrupting configuration files), or information disclosure (by reading sensitive files).

4.2 How to reproduce

1. Craft a filename like ../../etc/passwd.
2. Upload a benign file with that name.
3. Check whether the file appears in /etc/passwd (or a similarly sensitive location) on the server.

A quick test with curl:

curl -F "upload=@id.txt;filename=../../etc/passwd" https://example.com/upload

Then verify the file’s contents:

curl https://example.com/uploads/../../etc/passwd

If you see the password file, the traversal succeeded.

4.3 Detection (manual & automated)

*Manual*: Use a list of traversal payloads (../, ..\\, ..%2f, %2e%2e%2f) and attempt uploads. After each, run a script that checks common sensitive paths for the uploaded file.

*Automated*: Write a test that mocks the file‑system abstraction and asserts that the resolved path stays within the configured base directory. Many web frameworks provide utilities (e.g., Path.GetFullPath in .NET) that can be used in assertions.

4.4 Fix and prevention

• Never concatenate user input directly into a file system path.
• Extract only the basename of the filename (path.basename in Node, os.path.basename in Python) and discard any parent directory components.
• Generate a unique identifier (UUID or hash) for the stored file and keep the original filename only in metadata, not in the storage path.
• Validate that the final path starts with the intended upload directory using a prefix check (if (!realPath.startsWith(uploadBase)) reject).

5. Bug Pattern #4: Race Conditions in Temporary File Handling

5.1 What happens

Many frameworks write the incoming multipart stream to a temporary file before moving it to its final location. If the temporary file name is predictable (e.g., based on a timestamp or process ID) and the application does not use atomic rename operations, an attacker can guess the temporary name and read or modify the file while it is still being written. This can lead to information leakage (reading a partially uploaded sensitive document) or tampering (injecting malicious code into a file that will later be executed).

5.2 How to reproduce

1. Identify the temporary directory used by the upload handler (often /tmp or a config value).
2. Upload a moderately sized file (a few MB) while repeatedly scanning the temp directory for files matching a pattern (e.g., upload_*).
3. If you can open and read the file before the upload completes, a race condition exists.

A simple script:

#!/bin/bash
while true; do
    ls -t /tmp/upload_* 2>/dev/null | head -n1 | while read f; do
        if [ -f "$f" ]; then
            echo "Found temp file: $f"
            head -c 100 "$f"
            break 2
        fi
    done
    sleep 0.01
done &
curl -F "upload=@large.pdf" https://example.com/upload

5.3 Detection (manual & automated)

*Manual*: Enable verbose logging of the upload process to see when the temporary file is created and moved. Insert a small delay (e.g., using a breakpoint or a sleep in a custom handler) and attempt to read the temp file during that window.

*Automated*: In an integration test, replace the temporary file factory with a mock that returns a known path. Then, from a separate thread, try to open that path before the test signals completion. Assert that any read attempt fails with a permission error or returns empty data.

5.4 Fix and prevention

• Use a cryptographically secure random name for temporary files (e.g., os.tempnam with a random suffix).
• Open the file with O_EXCL | O_CREAT flags to ensure the file does not already exist.
• Perform all writes to the temp file, then rename it atomically to the final destination using rename(2), which is atomic on the same filesystem.
• Set restrictive permissions on the temporary file (e.g., 0600) so only the process that created it can read or write it.

6. Bug Pattern #5: Insecure Direct Object Reference (IDOR) in Uploaded Files

6.1 What happens

After a file is uploaded, the application often returns an identifier or a direct URL that references the stored file (e.g., /uploads/12345.pdf). If access control checks are missing or flawed, a user can guess or enumerate identifiers belonging to other users and download, modify, or delete those files. This is a classic IDOR vulnerability that can lead to privacy breaches, tampering with shared resources, or privilege escalation.

6.2 How to reproduce

1. Upload a file as user A and note the returned ID or URL (e.g., https://example.com/files/9876).
2. Log out and log in as user B (or use an unauthenticated session).
3. Attempt to access the URL from step 1. If the file is served or a modification endpoint accepts the request, an IDOR exists.

6.3 Detection (manual & automated)

*Manual*: Perform a brute‑force or sequential scan of likely identifiers (numeric IDs, UUIDs) while logged in as a low‑privilege user. Record any successful accesses to files that belong to other accounts.

*Automated*: Use an API testing tool (e.g., Postman or REST‑Assured) to iterate over a range of IDs and assert that each request returns 403 or 404 when the requesting user lacks permission. In a unit test, mock the authorization service and verify that the handler calls authorize(user, file) before serving the file.

6.4 Fix and prevention

• Never expose raw internal identifiers in URLs; use indirect references such as signed tokens or encrypted IDs that embed the owner’s ID and an expiration timestamp.
• Always perform an authorization check that confirms the requesting user has permission (read/write/delete) on the specific file object before processing the request.
• Log all access attempts to uploaded files for anomaly detection.
• Consider using a reference monitor or access control list (ACL) service that centralizes permission decisions.

7. Bug Pattern #6: Lack of Virus/Malware Scanning

7.1 What happens

An application that accepts user‑provided files without scanning for known malware can become a distribution vector for trojans, ransomware, or cryptominers. Even if the uploaded file is never executed on the server, it may be served to other users who download and open it, leading to client‑side compromise.

7.2 How to reproduce

1. Obtain a test malware sample (e.g., the EICAR test file, which is harmless but detected by most AV engines).
2. Upload the EICAR file as if it were a legitimate document.
3. Check whether the upload is accepted without any warning or quarantine action.

If the server stores the file and later serves it, the lack of scanning is confirmed.

7.3 Detection (manual & automated)

*Manual*: Maintain a set of known test signatures (EICAR, a few benign virus test files) and attempt uploads. Verify that the response includes a rejection or that the file is moved to a quarantine bucket.

*Automated*: Integrate with an antivirus API (e.g., ClamAV’s clamdscan) in a test harness. After each upload, call the scanner on the stored file and assert that the scan result is clean; if the file is known malicious, assert that the upload was blocked or the file is quarantined.

7.4 Fix and prevention

• Integrate an on‑access or on‑upload virus scanning step using a reputable engine (ClamAV, commercial AV, or cloud‑based scanning services).
• Reject the upload if any threat is detected, returning a clear 400‑level error with a user‑friendly message.
• Optionally, quarantine the file for further analysis instead of immediate deletion, to avoid false positives disrupting legitimate workflows.
• Keep virus definitions up to date via automated updates.

8. Bug Pattern #7: Incorrect Handling of Multipart/Form‑Data Parsing (e.g., boundary issues)

8.1 What happens

HTTP multipart requests rely on a boundary string to separate parts. If the server’s parser incorrectly handles boundary detection—such as failing to respect quoted boundaries, mishandling line endings, or allowing boundary strings to appear inside part data—an attacker can craft a request that causes the parser to misinterpret where a part ends. This can lead to:

• Bypassing size limits (by hiding extra data inside a part that the parser thinks is another part).
• Injecting arbitrary headers or parts that modify server state (HTTP request smuggling‑style).
• Causing the server to crash or enter an infinite loop when trying to locate the next boundary.

8.2 How to reproduce

1. Start with a normal multipart upload request captured via a proxy (e.g., Burp Suite).
2. Modify the boundary value in the Content-Type header to include a newline or a sequence that matches part of the file payload.
3. For example, set boundary to ----WebKitFormBoundaryxyz and embed the same string inside the file contents.
4. Send the request and observe whether the server accepts the upload, returns an error, or hangs.

A concrete curl example:

BOUNDARY="----WebKitFormBoundaryxyz"
PAYLOAD=$(cat <<EOF
--$BOUNDARY
Content-Disposition: form-data; name="upload"; filename="test.txt"
Content-Type: text/plain

hello world
--$BOUNDARY
Content-Disposition: form-data; name="extra"
Content-Type: text/plain

malicious
--$BOUNDARY--
EOF
)
curl -i -H "Content-Type: multipart/form-data; boundary=$BOUNDARY" \
     --data-binary "$PAYLOAD" https://example.com/upload

If the server treats the inner boundary as a real delimiter, it may see two files (test.txt and extra) and potentially store the second part where it shouldn’t.

8.3 Detection (manual & automated)

*Manual*: Use a fuzzing tool like ffuf or a custom script that iterates over malformed boundary values (missing trailing --, extra spaces, embedded CRLF) and monitors for abnormal responses (500, empty body, delayed response).

*Automated*: Write a unit test that feeds the raw byte stream of a multipart request to the parser library directly (bypassing the HTTP layer) and asserts that the number of parsed parts matches the expected count. Include test cases where the boundary appears inside part data and verify that the parser treats it as literal data, not a delimiter.

8.4 Fix and prevention

• Use a battle‑tested multipart parser (e.g., Apache Commons FileUpload, multer for Node.js, django.core.handlers.wsgi for Python) rather than rolling your own.
• If you must implement custom parsing, strictly follow RFC 7578: the boundary must be preceded by -- and followed by either -- (final part) or CRLF. Never treat a boundary that appears inside part data as a delimiter unless it is exactly preceded by -- and followed by CRLF or --.
• Limit the maximum number of parts and the maximum header size to mitigate resource exhaustion.
• Log parsing errors and reject malformed requests with a 400 Bad Request.

9. Bug Pattern #8: Inadequate Access Controls on Uploaded Files (Overwrite, Privilege Escalation)

9.1 What happens

When the upload endpoint allows a user to specify a destination path or filename that maps to an existing file, a malicious user can overwrite critical application files (e.g., configuration, scripts, logs) or other users’ data. This can lead to privilege escalation (by overwriting a sudoers file or a web shell), data loss, or bypassing integrity checks.

9.2 How to reproduce

1. Identify a scenario where the application lets the user set a subfolder or filename (e.g., a profile picture upload that lets the user choose a folder like avatars//).
2. Attempt to upload a file with a path that traverses outside the intended directory or targets a known sensitive file (e.g., ../../../web.config).
3. Verify whether the target file is replaced with the uploaded content.

9.3 Detection (manual & automated)

*Manual*: Maintain a list of sensitive paths relative to the upload root (e.g., ../, ../../etc/passwd, ../app/config.yml). Try each as a filename or subfolder component and check the resulting file system.

*Automated*: In a test environment with a disposable file system, seed a known file (e.g., important.txt) with unique content. Run the upload API with a malicious path and then read important.txt. If the content changed, the test fails.

9.4 Fix and prevention

• Disallow any user‑controlled path components; generate the storage path server‑side (e.g., /uploads//).
• If a user‑chosen filename is required for display purposes, store it only in metadata and never use it to construct the file system location.
• Set the uploaded file’s permissions to the minimum required (e.g., 0644 for read‑only, 0600 for private).
• Use an immutable storage layer (e.g., write‑once object storage) where overwriting an existing key is either impossible or requires explicit versioning.

10. Bug Pattern #9: Metadata Exploitation (EXIF, Embedded Scripts)

10.1 What happens

Files often carry metadata that is ignored during basic validation but later processed by downstream components. For instance, an image’s EXIF block can contain arbitrary strings, and some libraries will render those strings as part of the image or even execute them if they are mistakenly interpreted as code (e.g., PHP’s exif_read_data with certain options). Similarly, PDFs can embed JavaScript that runs when the document is opened in a vulnerable viewer.

If the application extracts and displays metadata without sanitization, an attacker can store XSS payloads in EXIF fields that get rendered in a gallery page, leading to stored cross‑site scripting.

10.2 How to reproduce

1. Take a benign JPEG and inject a script into the EXIF UserComment tag using a tool like exiftool:

   exiftool -UserComment='<script>alert(1)</script>' image.jpg

1. Upload the modified image.
2. View the image page or any endpoint that displays EXIF data. If the script executes, metadata exploitation is present.

10.3 Detection (manual & automated)

*Manual*: Use a metadata injection tool to add known XSS or SSRF payloads to common fields (EXIF, IPTC, XMP, PDF /JavaScript). After upload, inspect the rendered page for unsanitized output.

*Automated*: In an integration test, after upload, call an endpoint that returns the metadata (e.g., /api/image/123/meta). Assert that the returned values are properly escaped or stripped of HTML/script tags. You can also use a DOM‑based XSS scanner (e.g., OWASP ZAP) on the gallery page.

10.4 Fix and prevention

• Strip or sanitize all non‑essential metadata before storing or serving the file. Libraries like mozjpeg or ImageMagick can strip EXIF with -strip.
• If certain metadata must be preserved (e.g., orientation), extract only the needed fields and discard the rest.
• Treat any metadata that will be displayed as untrusted user input: apply output encoding appropriate to the context (HTML, JavaScript, attribute).
• For PDFs, consider using a library that flattens or removes JavaScript (e.g., qpdf --linearize).

11. Bug Pattern #10: Upload Flow Interruptions (Cancel, Network Loss) Leading to Inconsistent State

11.1 What happens

Upload implementations often assume a successful transfer from start to finish. If the user cancels the request, the network drops, or the server times out mid‑stream, the application may leave behind a partially written temporary file, an incomplete database record, or a locked resource. Later cleanup routines might fail to remove the orphaned data, causing disk space leaks or stale references that confuse subsequent operations (e.g., a thumbnail generator trying to process a half‑written file).

11.2 How to reproduce

1. Initiate a large file upload (several MB) using a tool that allows you to abort after a few seconds (e.g., curl with --speed-limit and --speed-time).
2. Monitor the upload directory and any associated database or queue for entries created during the transfer.
3. Cancel the upload and check whether any temporary file or record remains.

Example using curl:

curl -F "upload=@bigfile.zip" --speed-limit 1000 --speed-time 10 https://example.com/upload

If after the command exits you find a file like /tmp/upload_12345.part still present, the cleanup on abort is missing.

11.3 Detection (manual & automated)

*Manual*: Use a network throttling tool (e.g., tc on Linux) to introduce latency or packet loss during an upload. Observe server side logs for error handling and check for leftover artifacts.

*Automated*: In a test harness, simulate a socket closure after a certain number of bytes have been transmitted. Assert that any temporary files created are removed and that no database rows are left in a “pending” state. Many testing frameworks provide hooks to mock the underlying transport and inject errors.

11.4 Fix and prevention

• Write uploads to a temporary file with a unique name and only rename/move it to the final location after the entire stream has been read successfully.
• Use a transaction or a two‑step commit: first record the upload intent in a durable store (e.g., a “upload_jobs” table with status pending), then upon successful completion update the status to completed. On failure or abort, a background job can clean up any resources linked to the pending job.
• Set appropriate timeouts on the server side (read timeout, request timeout) and ensure the timeout handler performs cleanup.
• Provide a client‑side mechanism to resume or cancel cleanly (e.g., HTTP PATCH with range headers) if your use case requires it.

12. How Persona‑Driven Autonomous Exploration Finds These Bugs

Scripted test suites typically follow predetermined paths: they log in, navigate to the upload page, upload a valid file, and assert success. Real users, however, behave in varied ways: some are curious and try odd file names, some are impatient and cancel mid‑upload, some attempt to break the system with large payloads, and others use accessibility tools that interact differently with the UI.

An autonomous QA agent that models multiple user personas can exercise the upload endpoint in ways that static scripts miss. By varying behavior profiles—such as the “curious” persona that tries every possible file extension, the “impatient” persona that repeatedly aborts requests, the “adversarial” persona that crafts malformed multipart bodies, and the “elderly” persona that relies on screen‑reader navigation—the agent explores a far richer state space.

12.1 What SUSA Does

SUSA (SUSATest) is an autonomous testing platform that, given an APK or a web URL, automatically explores the application without pre‑written test cases. It builds a model of the UI and API endpoints, then drives interactions using a library of personas. For file upload, SUSA will:

• Attempt uploads with a matrix of file types, sizes, and names generated on the fly.
• Vary HTTP headers (Content‑Type, Content‑Length, boundary) to trigger parser edge cases.
• Simulate network interruptions and cancellations using its built‑in fault injection.
• Check for crashes, ANRs, dead ends, and accessibility violations after each interaction.
• Record which flows (e.g., login → upload → view) succeed or fail, producing a PASS/FAIL verdict per path.

Because Susa learns from each run, it remembers which inputs caused errors and prioritizes similar variations in subsequent executions, increasing the chance of catching subtle bugs like race conditions or metadata‑based XSS.

12.2 Example Run

Imagine testing a web image gallery. SUSA’s “adversarial” persona might generate the following sequence:

1. Load the gallery page, locate the upload button.
2. Set the Content-Type header to image/jpeg but attach a file whose magic numbers correspond to a ZIP archive.
3. Append a malformed boundary string that includes the sequence -- inside the part data.
4. After the server responds with a 200, immediately issue a second request that aborts the TCP connection after uploading only half the file.

During this run, SUSA could detect:

• A crash in the image processing library when it tries to read the ZIP as a JPEG (Bug #1).
• A parser error that leads to a 500 response (Bug #7).
• A leftover temporary file in /tmp after the aborted upload (Bug #10).

The platform would flag each anomaly, capture logs, and suggest a regression test (e.g., an Appium script for Android or a Playwright script for the web) that reproduces the exact sequence.

12.3 Why Personas Matter

• Curious: tries extensions like .htaccess, .trailer, .tmp to discover weak validation.
• Impatient: repeatedly clicks cancel or navigates away during upload, exposing cleanup gaps.
• Novice: uses the default UI without altering headers, catching issues