search

bolt-lightningTesting Failures and Errors

Testing involves intentionally triggering errors and seeing how the system behaves. But it's not enough to just state "yes, the error pop-up appeared as expected, all is good".

circle-exclamation

Errors aren’t just about whether they occur, but how they are surfaced, how meaningful they are to the user, and what happens next.

hashtag
A simple example

Drawing

Regardless of context, does this error dialog have problems?

  • "The message is vague". This might be intentional. Revealing technical details to non-technical users is likely to be inappropriate, useless, and might even expose sensitive information.

  • "OK, so what do I do now?" - Missing information on how to recover. Do I navigate back? Can I close the window?

  • "What happens when I click OK/Cancel?" - given the message, there is no obvious difference between the two buttons.

  • "Shouldn't OK be on the right?" - bonus question. There is no right or wrong, as it is platform-dependent. On Windows, affirmative actions are on the left; on macOS, Linux, and Android, they are on the right. The web? It depends. See UX StackExchange discussionarrow-up-right.

This example is contrived yet realistic, but it highlights that error events require thought, design, and testing.

hashtag
Structured Categories for Testing Errors & Failures

Follow the CLEAR LOG mnemonic.

It captures user-facing and backend aspects of error testing. Not all points apply to all situations.

hashtag
C – Clarity

  • Is the error message clear and understandable for the intended audience?

    • "Oops, something went wrong." → "Payment failed due to expired card."

  • Is the technical detail level appropriate (end-user vs. admin vs. developer)?

hashtag
L – Localization

  • Is the error message shown in the correct language/locale?

    • Unlike UI text, error messages are often hard-coded into the code by omission

  • Do translations preserve meaning, politeness, and cultural appropriateness?

hashtag
E – Explanation

  • Does the error explain why it happened (without leaking sensitive info)?

  • Does it guide the user on next steps or recovery options?

  • Are error codes meaningful and consistent?

hashtag
A – Actionability

  • What can the user do after seeing the error? Are the options non-ambiguous?

    • “OK/Cancel” → “Try Payment With Another Card / Cancel Booking.”

  • Are recovery options clear (retry, cancel, contact support)?

  • Are destructive actions (like “Delete anyway”) clearly warned?

  • Are users given the "Submit error report" option? If they do, does the submission work?

hashtag
R – Recovery

  • Can the user continue working without losing data?

  • If the app crashes, is there auto-save, rollback, or undo?

  • Does retry work under varying conditions (network restored, permissions fixed)?

hashtag
L – Logging

  • Is the error logged in the backend/system logs?

  • Does the log capture enough diagnostic detail for debugging, or will the developer struggle?

  • Is sensitive information (passwords, PII) excluded from logs?

    • Does the backend log contain a payment gateway error code, but not the full card number?

hashtag
O – On-Time

  • Does the error appear promptly, without long freezes or hangs?

  • Are timeouts handled gracefully?

  • Are cascading failures prevented (e.g., one error doesn’t spawn multiple popups)?

hashtag
G – Gracefulness

  • How gracefully does the system degrade under error conditions?

  • Does the UI remain usable (no frozen screens, unresponsive buttons)?

  • Is error styling consistent (colors, icons, modal placement)?

PreviousDo Regression Testing: RCRCRCNextReporting with Testing Metrics

Last updated