Protect Your Assets
Showing results for 
Search instead for 
Do you mean 

WebInspect and Web Application Scanner Comparisons

markpainter ‎08-17-2012 11:22 AM - edited ‎09-09-2015 11:41 AM

IBM has been making some noise about their recent showing in Shay Chen’s web application scanner comparison study. While Shay’s results show a lot of things,  they don’t show that Appscan is a better solution.  Not by a long shot. 

 

For the sake of comparison, here’s how WebInspect ranked in different categories:

 

#1 – WIVET (Web Input Vector Extractor Teaser)  

#1 – Coverage features (tied)

#1 – Input Vectors (tied)

#1 – XSS (tied)

#2 – Audit Features Comparison

#2 – RFI

#4 – SQLi (the difference between 1st and 4th was .74%, and only included detections, not false positives. Otherwise, the results would have changed.)

 

The WIVET category is arguably the most important, and one that WebInspect won. If you can’t find a page, how can you test if for vulnerabilities?

 

What’s not included in Shay’s results are some scoring issues that helped us lose some points. In certain categories Shay used the WebInspect ‘All Checks’ policy to maintain consistency across all his tests. This unfortunately resulted in a number of false positives because certain checks that are included as a fail-safe mechanism do simple pattern matching as opposed to the more intelligent checks used in other WebInspect policies.  In other words, our ‘All Checks’ policy is the kitchen sink approach. We throw everything we can at an application, and some of that stuff isn’t necessarily pretty.  Our default scanning policy is the ‘Standard’ policy specifically for that reason. To his credit, Shay is fair in that he used the same criteria for every scanner. Here was his comment on the matter: 

 

“…the All Checks policy is not tagged as experimental and the consumer does not have any obvious leads that using it might affect the accuracy, and thus, I have no workaround for this issue.”

 

We can concede that point. We’d much rather Shay maintain a level playing field than change anything because we weren’t specific enough in our description.

 

It will be interesting to see the results of Shay’s next set of tests. We are most definitely looking forward to the competition. 

About the Author

markpainter

Events
Each Month in 2016
Online
Software Expert Days - 2016
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
Sep 30
Seattle, WA
OpenStack Days Seattle
OpenStack Days Seattle, September 30, is the largest gathering of OpenStack users and prospective users in the Pacific Northwest region.
Read more
View all