AI to drive Product Quality?

Posted: December 1, 2016 in Uncategorized

Let me start with some of the popular transitions around the idea of “Product Quality” that went on to create many jobs at different levels. I have also represented each transition as a level.

The Transition levels are not mutually exclusive. Each Transition level may or may not co-exist during the course of existence of the Universe!


Each of these levels could exist in different ratios at different points of time spanning across the existence Software Engineering (I will leave it to you to guess/predict/compute the time span of Software Engineering).

Testing was carried out alongside Development. Then testing began gaining importance and was beginning to be prioritised in Software Development cycle. Thereafter it was being handed over to a separate group called Quality Assurance to share the load and bring in a user acceptance perspective to Testing. Testing was still mostly manual. Quality Assurance began automating many of the repetitive tasks confined to testing activity and Automation Testing mostly marks the beginning of transition to Quality Engineering in order to address the various inefficiencies impacting the quality of a software product.

Automation was not just confined to testing activity alone, but also automation of other repetitive tasks such as automation of releases with CI/CD systems (evolved into Release Engineering), then automation of infrastructure provisioning and cloud based infrastructure picking up pace (evolved into DevOps) and sometimes Release Engineering and DevOps together being called as Site Reliability Engineering. In general, I think Quality Engineering is a super set of Release Engineering, DevOps and Site Reliability Engineering. There is no denying of fact that all of these roles including Testing did exist (or were carried out by a lone warrior called Developer) even before they were re-christened with specific names.

Let us look at what next after Quality Engineering at level 4.

To do so, I will deconstruct Quality Engineering along the lines of Testing. As you know, Quality Engineering is different levels of automation of different aspects of a Software Development cycle and all subsets mostly aligned with a common goal of ‘developing a piece of software efficiently‘.


The various stages of indicates various degrees of automation of testing process. All of the levels usually have the 3 common feedback parameters namely bugs, false positives and execution times, each having varying degrees of impact on each stage. It is this impact that leads Quality Engineering to transition between different stages and we choose a Stage to solve our problem of testing based on the implementation cost of automation + cost of resolution of feedback.

In the first glance, it might occur that the implementation is costliest at Stage 4 relative to Stage 1, 2 or 3. While I claim, that it is not always true. The cost of addressing the feedback plays a significant role in decision-making.

Consider if there was a Stage 0 with zero automation (meaning manual testing only). It is quite predictable from our experience how Stage 2 is cheaper than Stage 0 and Stage 1 as well. For a fairly complex system, the execution time in Stage 0, is quite large and thus, impacting the cost of addressing feedback (assuming infinite energy on the part of the tester to test the system).

Now, consider a system of greater complexity and also catering to more diverse and unknown/unexplored real-world territories, the size of test data set could grow exponentially and there is a direct impact on redundant testing and execution time. This leads us to pursue other approaches to optimize test data set such as the application of Orthogonal Array Testing(OAT) technique. Furthermore, to improve the testing “effectiveness“, help from the field of Data Science could be taken to more deeply understand the behavior of the system under test.

Now, let us take this a little further, say, I build an automated source of “truth” system and validate the behavior of the application in real world, which could then notify of possible bugs, we could flag them as valid or false positive and enhance (or train) this “truth” system. Did I just say “train”? Well, I leave the rest to your imaginations!

To conclude, testing and system-testability (will post a new write-up on system-testability soon) can also take the role of being a feature of an application/system given the need for the system and availability of resources and we eventually converge back to being Developer. It just seems like an alternative career track/journey to being a Developer driven by the Quality Engineering philosophy. Hence, engineer out a solution for a Quality problem that you are facing like a Developer does!


Drop your thoughts here...

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s