Table of Contents
Sanity Testing and Smoke Testing are confusing terms. By the end of this blog post, we will be looking at clearing your doubts. Such as, can your app handle a system testing effort? Or a regression testing effort? Smoke testing vs Sanity testing has been a very interesting topic for testers all over. Is automation testing necessary for a sanity testing effort? Is that one of your Software Testing strategies?
Smoke Testing:
Interviewer: What is Smoke Testing?
Interviewee: Basically, a hardware term. You plugin a mobile charger into the power supply point. Switch the power on. The charger burns, and goes up in smoke. Product is dead.
Image credit: “smoke” by Brett Samuel, CC BY-SA 2.0
Interviewer: So how does it apply to Software
Interviewee: Does your product – a web app/mobile app, launch successfully? Or does it crash as soon as it launches? Also,
Does your mobile device (if you are testing a mobile app) become hotter in a few seconds.
Is there a burning smell emanating from your device/laptop/desktop?
Is there an unusual sound emanating from your device/laptop/desktop?
That would be a major risk.
Sanity Testing:
Interviewer: What is Sanity testing?
Interviewer: or .. should I say, What does a sanity check mean?
Interviewee: Assuming your app launches successfully, what would be the next step? A cursory look at all the major features of the application. Testing them at a shallow level, not going deep. For example logging in to an app with only valid credentials, no invalid data. Once logged in, have a cursory look at the main features of the app.
Interviewer: So, what are the deliverables?
Interviewee: A Sanity test report. The report contains, the details of failed and passed tests. It should also have a statement on whether we should go ahead with a major testing effort. And….. a few more things
Interviewer: Such as?
Interviewee: Subjective feelings also, slow network, User experience etc. something just isn’t right
For a video on the same, please watch it here
Interviewer: Subjective feelings huh? Interesting. Why would you say that?
Interviewer: Restricting yourself to just pass and fail is biased. A case of False dichotomy
Interviewer: So let us suppose your Sanity Test report does have a few failed tests. What do you do?
Interviewee: Raise bug reports. Wait for the next build where the fixes are made. This way a part of the entry criteria can be fulfilled.
Interviewer: What if they say, please ignore failed tests, and continue with other tests which go deeper.
Interviewee: Even if all the tests passed, I would be wary of such a situation. I would also like to see if the below conditions are met:
- All the test data is available for the major testing effort
- Assess the availability of resources, (Network, devices, access) and plan for the risk mitigation.
Interviewer: Hmm.. What could be the reasoning behind this?
Interviewee: Sanity / Smoke testing efforts are basically the first hurdles. Sanity testing, especially, is like a test to pass – no difficult tests. These tests are done to see if the app can handle a major testing effort. Such as, a system testing effort, or a regression testing effort. Any bugs, need to be addressed, else a risk is being transferred to the Software Testing team.
Automation Testing:
Interviewer: Why would you automate your Sanity Testing suite? Has that been one of your Software testing strategies?
Interviewee: Yes, if your team gets several builds in a day. And almost the same tests need to be executed manually. Then, automation of the Sanity tests can add some value. Also, Sanity testing of APIs can also be considered.
Software testing strategies:
Interviewer: Is there any extra dimension that you introduced when you did Sanity Testing
Interviewee: Yes. For some test cases, I introduced a column in the test case document, just for the logs. Please read about it here.
Let us know what you think about the key differences between smoke testing vs sanity testing in the comments below
One Response