I wrote this after I read a discussion on LinkedIn in the “Software Testing & Quality Assurance” group.
A group member asked two months ago the following:
“Is it possible to fully automate all QA and testing?
I believe that, if the company did little and bad, only scripted manual testing or no testing at all, it is possible to fully automate the Testing. There is no general consensus regarding this as there isn’t for nothing on this planet. Also, it might be possible to lead the departmental transformation, if the case is as I mentioned before.
If this is not the situation, than the questions at hand are wrong ones.
I will continue with some textual analysis.
Firstly, “Quality Assurance” is a group of words used stupidly in most cases in the industry. Many times it is used to denote the Testing Team, that do not and should not ensure quality. They should only do their best to Test the product in their given boundaries / context and inform the stakeholders on the found results. Other bad use of the words would be to point to Managers that ensure compliance with the ISO and other standards or to Managers that plan the procedures and workflow. In the analysed text it’s not used properly either. This expression is mostly a marketing tool, that pleasantly tickles the ears of the inexperienced boss. Who wouldn’t want a person that ensures that you have a quality product at a small cost?
Secondly, “all testing” will never be done by no means. It’s similar in magnitude to counting all living creatures on Earth.
More, we work in a context. The mere fact it is possible doesn’t mean we should start doing it. Shooting oneself in the foot might be possible, but might not be the best thing one should do. We should consider also costs, risks and benefits to customer. What if the cost to fully automatically Test a software are 1000 times greater? What if the chances of finding new important bugs fast are 100 times smaller? Would one still do it?
The question formulated by the author it is similar to this post’s title. In some cases both, testing and programming, can be fully automated. My guess is that these situations are not the majority.
In the LinkedIn discussion Google, James Whittaker and the famous “Test is dead” speech are mentioned. See my From developers, with hate post related to them.
Now, I would propose different set of questions and answers that are more relevant in the industry:
1. Should one fire all the Manual Exploratory Testers and replace them with automatic Unit Tests?
R: No. In most cases I know, this would be a stupid thing to do. If the Testers never did a good enough job you should start invest in training, books and conferences. In an extreme case, you should fire them and hire new guys.
2. How can 100% Unit Test code coverage can be obtained?
R: This kind of metrics are stupid, irrelevant, confusing and most of the time used for lying to someone. Be aware of “all” and “100%”, it might point to someone that is inexperienced, can’t think out of the box or trying to lye to you.
3. Is Automatic Testing less expensive than Manual Exploratory Testing?
R: It depends. In some cases YES and in others NO. Automatic Tests do not find new important bugs, they only check that old (good, bad, efficient or not) assumptions are met. In software development there is an important chance that the highest cost might be in the unknown. Are you willing to ignore this probability? Also, Unit Tests are highly inflexible and will need a lot of maintenance.
4. Will A.T. tell you better than a human if the team is building the right product?
R: In most cases A.T. suck big time at Usability Testing. A Unit Test will have nothing to tell you about the beauty of the product and how appealing might it be to the client. At the moment, computers don’t inform us that an application is very hard to learn and use.
So, are we at the moment in time where one could replace human feelings, innovation and intellect with machine?
Can we respond with YES to “Is it possible to fully automate all programming?”?
If not, Manual Exploratory Test is not dead.