All About the Apps
Showing results for 
Search instead for 
Do you mean 

What types of software testing matter to you?

Michael-Deady ‎10-15-2012 10:00 AM - edited ‎09-18-2015 04:08 PM

One question you should ask yourself is “If I have X number of dollars per year for testing, where should I test and what types of testing will give more bang for my buck?” The second part of that same question is “What is the risk or exposure to my company if I do?”


First of all, this is a subjective and suggestive blog and is open for interpretation and/or discussion. Think of it this way, doesn’t the guy who kicked the anthill just hate the ants, not the actual hill? Over the years I’ve seen companies commit to software testing only to dump everything into functional or performance testing. They neglect other types of testing that could have been more successful in saving money or limiting the exposure of a company to a great deal of risk. Are you now stuck with this scenario? If you could start your testing group over again where would you focus your company’s money and resources?


What do you test for?


We can identify functional defects and performance issues; what and how we test can for the most part be unproductive and inadequate. An example: it’s been proven that most defects can be traced back to the requirements document. Yet, we still only test after the application has been developed. The other part is: how cost effective is it to fix the same defects during later stages in the process instead of in the requirement document—where most defects can be traced back to. If you fix a defect in the design phase, it could save your company 70 percent of the cost of fixing that same defect in the implementation phases of the development process.  We know this, yet we still wait to test at the end of Software Development Lifecycle (SDLC).


A similar argument can be made about application security testing, but 80 percent of requirements don’t even mention word “security” in the design model or documents.  With this in mind, it’s somehow implied that security is the number one priority for most companies. This needs to be re-evaluated.


Who performs your tests?


Another issue that faces this type of testing is the use of unskilled or untrained labor to write and/or execute the tests. This is a poor use of time and for the most part counterproductive.  While the testing environment is a great place to learn a system or an application, you are exposing the company to a learning curve, which at best will generate nothing more than unfavorable results.  The only way to mitigate the introduction of untrained labor is to have very detailed test cases, which usually not the case.


I understand that every case, company and application are very unique – for the rest of this blog I will use general terms that can apply to most situations. They are listed in important in the broadest since of the term “importance”.


  • The first is simply called verification and validation—this should be every tester, analyst and developer’s motto. Verification and validation are independent of each other; however they have been proven to yield high success rates while lowering both cost and risk. There is really not a fancy or automated tool for this type of testing.  There are options that are similar to the build-in workflow of ALM and very simple to implement.
  1. Think of the verification stage of testing as dealing with everything that is static, written and/or documented.  This pertains to the designing, and/or testing of the application. Verification is basically made up of several walkthroughs of the document or artifact with all of the players involved in the SDLC of an application. This should be a very controlled meeting and should generate one log that lists defects, questions, recommendations, omissions and/or approvals.     
  2. Validation is everything dynamic or the byproduct of the verification. During this stage the application is validated to make sure it operates as designed. These types of testing include unit, integration, system, security, etc….

The V&V process has been adapted to most types of methodology including Agile and has a high Return on Investment (ROI)—more than any other type of process mentioned below.


  • Software Security currently doesn’t have high ROI unless something is found; however it does greatly reduce your company’s exposure and risk. During several site visits I found that software security and testing were two different groups and they didn’t even track issues in the same systems. To increase software security ROI, the issues or risk should be tracked in the same fashion that development tracks defects.

In one case, a company had a paid great deal of money for security testing software. The software sent out a monthly report listing of vulnerability of the applications currently in production to each application owner. I asked them how they followed up and resolution process group engagement plan. They told me it was up to each application to address security vulnerability. I suggested that for the next month the security group request a read receipt on every report sent out. When the last of about 300 reports where sent out, they had less than 10 percent return as read and seven came back as incorrect application owner.


 Just a FYI, the theory of “cover your posterior” (CYA) doesn’t work when you’re facing the CIO dealing with a security breach.  I highly recommend that you invest in a suite security tool and place accountability on the testing groups to validate both production and development.

Terrorists are now funding hacker groups to attack and steal information and/or money by finding and exploring security breaches in your I.T. infrastructure.  Protect your software by making sure the security you have in place is actually working and being held accountable.


  • Unit  and Intogreation testing is by far is the best way of finding defects than any other type of testing. However, most development groups don’t follow any type of formal unit and intogreation testing process. Even if they did, 90 percent of the testing is normally done by the same person writing the code.  We have made it so hard to track issues and even harder to enforce even the simplest of formal unit and intogreation process.


Think of it this way, when I start writing code I write about 100 lines of code and find a syntax error or bug for every 4 lines of code. That is just before lunch. It then takes me a day to document the number of bugs found in the code. And that is considering the code took just a couple of hours of work that morning. 


Nothing is more effective than white box testing. That is why I’m one of the biggest supporters of a training testing group on gray box testing and root cause analysis. This allows the tester and developer groups to work in tandem.


The best way to set up successful unit testing is to integrate with your development environment (IDE) with a Requirements and Defect's management system. This will allow developers to log issues along with requirements and areas of risk.


 I can think an example from was when I was developing. I would find an area of what I would call “questionable coding skills” or “mysterious requirements”. I would convey the issues to my management, which ultimately would go ignored until it caused a defect issue in production. By integrating your IDE and defect tracking system you could link problem code or ambiguous requirements directly to defects and requirement system. This could save everyone a great deal of grief.


  • Yes, Functional Testing has a large impact on how an application behaves. It also has a high ROI and reduces exposure, but that all really depends on how you’re dividing up the work loads, tools and type of methodologies. For example: if you’re doing most of the regression testing manually, then your ROI will drop and your risk will increase over time. Devoting too much time to new functionality will also decrease testing group values. Most of this type of testing is redundant to unit and user acceptant testing. If you really want to make functional testing successful I would recommend two types of methodology or process.  I prefer systematic and exploratory testing; they yield the best results when compared to other functional types of testing, either new or regression.
  1. Systematic Testing emphasizes prevention by focusing on the risk area and stresses continuous improvement. While this process offers a high success rate, it is very time consuming and can be very documentation intensive. Systematic testing may not be the most Agile friendly method but it does have a great track record with other iterative types of methodologies.
  2. Exploratory Testing has lower ROI yielded then systematic testing; what you lose in quality you gain in quantity. Risk does increase some with this method. To be successful you need tools like Sprinter for documenting test cases. However, I believe that exploratory can be very successful when in an agile development environment.

If you really want to make Testing Matter it may not be the type of testing but how you communicate and track issue within the SDLC process?  


This blog is a general outline of the four biggest testing processes and/or methods that make your software development lifecycle be more successful. If you would like me to go into more in-depth with a different type of testing, please contribute to the discussion. If you would like me to lengthen the blog to cover more type of testing or rearrange the order please comment.  I would like to discuss with (my peers) the reader and get the records straight on what type of testing really matter.



Other Blogs you may like:





About the Author


Michael Deady is a Pr. Consultant & Solution Architect for Teksystems, center on quality, aimed at client's satisfaction, and long-term success. Perceived by clients, peers, and supervisors as a leader with the proven ability to lead development and quality assurance teams through software-development life cycle phases, to ensure quality of new products. He specializes in software development, testing, and security. He also loves science fiction movies and anything to do with Texas.

27 Feb - 2 March 2017
Barcelona | Fira Gran Via
Mobile World Congress 2017
Hewlett Packard Enterprise at Mobile World Congress 2017, Barcelona | Fira Gran Via Location: Hall 3, Booth 3E11
Read more
Each Month in 2017
Software Expert Days - 2017
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all