Craft a Useful Bug Report
A problem well stated is a problem half solved.
Writing a bug report seems simple on the surface. In practice, it's not.
Unclear, incomplete, missing context, missing or incorrect steps to reproduce, no useful attachments such as logs or screenshots, no links to previously reported similar issues, weak justification... the list goes on.
"The quality of your reports is probably the single most important factor that determines your credibility as a tester"1 — James Bach, Michael Bolton
This page is meant to help everyone who reports bugs. It shows what a good bug report looks like to beginners and acts as a reminder to current professionals.
🕵️Step 1: Investigate First
Before you write the report, be sure to investigate with RIMGEN.
Also, consider to MIP - Mention In Passing. Talk to the right developer directly without a report and get a simple bug fixed without formal paperwork.
Remember that testing is ultimately about creating a product of the highest quality possible. Unless your workplace has toxic metrics or KPIs, such as evaluating testers based on bug report count, minimizing the friction required to get a bug fixed is desirable.
📄 Step 2: Write a Bug Report
Below is a template of satisfactory completeness. Bug Report format varies by company.
SUT version where the bug happens: for example 1.x or 2.3.y
Description1: What is the problem? What exactly happened?
Oracle: WHY is it a problem? (see section below) e.g. Contradicts requirements, behavior differs from previous version, fails some widely-accepted standard, or other
Exact steps to reproduce (no more, no less), including relevant config/env details If the bug is intermittent (not 100% reproducible) - include detailed information on your attempts to reproduce it, ideally on different machines/environments
If appropriate: 1. Any information derived from RIMGEN 2. "Same behavior happens / doesn't happen in older version Y, therefore..." 3. Logs (relevant extracts, not large dumps). If unsure, include the entire large log, but try to point to relevant lines. 4. Screenshots/recordings 5. Fix suggestion / expected behavior (neutral, non-prescriptive tone). No need to include obvious things such as "should not crash" or "text should not overflow the button border".
1 Description and tone should vary, depending on whether it's a "normal bug" or "improvement suggestion".
Normal bug example: "This doesn't work as per requirements."
Improvement suggestion: "This seems OK, or doesn't contradict the explicit requirements, but it could be better, because... {personal anecdote, logical reasoning why it's confusing, comparison with competition, etc. See Oracles below.}"
🔮 Oracles and HICCUPS
— I closed your bug report as "Not a bug"
— But it is a bug!
— Why do you say so? How do you know?
An oracle is a means by which we recognize a problem. An oracle completes the sentence "It is a bug because..."
An oracle can be strong (does not meet formal requirements) or weak (a personal opinion - "I find it confusing").
Use the HICCUPS mnemonic2 to choose an oracle and use it in your bug report.
H - History
Is the product consistent with its past versions?
"The behavior is different from the previous version, and there was no request to change it."
I - Image
Product is consistent with the image the company wants to project
"We provide security services, yet our website has security holes."
"We are a design studio, yet our website uses Comic Sans."
C - Comparable Products
Unless the product is bleeding-edge and visionary, it should probably be at least somewhat consistent with comparable products
"This is OK, but our competition does it BETTER. Should we be as good as them at least?"
C - Claims
Product is consistent with internal artifacts: requirements, design documents, and other
Product is consistent with public-facing things: website claims, instruction manuals, public announcements
U - Users' Desires/Expectations
Do not ignore feedback from reasonable, relevant end users. Feedback from different users may be contradictory
P - Product
Consistent behavior within the product itself
"On this screen, 'Delete' button is in bright red, but it's grey not on this other screen."
"On this screen, pressing 'Cancel' triggers a confirmation dialog, but not on this other screen."
S - Statutes & Standards
Compliant with laws, regulations, and applicable standards
❌ Common Mistakes in Bug Reports
Poorly worded title
It is the face of the report. It is also difficult to make it concise and informative at the same time. Invest some thought into it.
When you submit 10 reports titled "System crashes", it becomes difficult to browse through bug backlogs without clicking into each one.
Description that is too short
Can't reproduce because not enough information
"Not a bug" because no justification provided
Description that is too long
Not contradictory to the previous point.
Non-essential information is noise to anyone reading the report
Many-in-one
1 bug report describing multiple defects that just might happen on the same screen
Risk: 1 defect gets fixed, the other accidentally ignored.
Submit separate reports for separate problems
Exceptions exist, e.g. "The button font style AND font size are inconsistent with other buttons". Use experience and common sense.
Disrespectful, incoherent, poor grammar
Unprofessional tone
Poorly structured (written rambling)
Written without proper punctuation or with typos
False positive
A bug report that is not a real bug. Incorrect behavior was caused by poor understanding, incorrect environment configuration, etc.
Dangerous! Submit too many false positives and be seen as a time-waster.
Getting the severity wrong
"THIS IS CRITICAL", when in reality, it's a mid-severity problem.
"Not a big deal", leading to the report getting lost in the backlog, but later it turned out to be severe.
Check E of RIMGEN
References
Last updated