Security VARs See 'Huge Opportunity' To Tout Their Own Testing As Security Vendor Infighting Escalates

With an escalating soap opera brewing as security vendors take shots at each other's technology and trustworthiness, security VARs say they see a "huge opportunity" to emphasize their own testing of the vendors' products over third-party evaluations.

"I don't put very much weight in [third-party testing]," said one security VAR executive, who did not want to be named. "I think there's a lack of credible third-party testing out there." Popular third-party testing providers include AV-TEST, AV Comparatives, SE Labs and NSS Labs.

Instead, the executive, who has tested many current endpoint security products, said his business relies on its own extensive testing, which vets vendors for over six months against 200 different criteria to help customers decide which solutions are the best fit for their business. That presents a "huge opportunity" for his business, he said, to be able to provide customers customized added value based on their specific business needs.

[Related: Cylance, Sophos Trade Heated Words, With A Reseller Partner Caught In The Middle]

"We're an unbiased company. We did our analysis to help us help our customers," the executive said. "It's a huge opportunity."

Matt Johnson, CEO at Baltimore-based Phalanx Secure Solutions, said his company follows similar practices, saying it "doesn't rely on a lot of third-party tests" to choose technology. Johnson said Phalanx CISO Johnny Collins works to vet all the security technologies the company sells – a three- to four-month process that includes a full rundown of the technology and penetration testing. For clients, that's important, he said, because it provides validity behind the technology and establishes the solution provider as a trusted advisor.

"We pitch it as doing our due diligence," Johnson said. "Not only do we look at outside reports, but we also went through and examined the products and put our full weight [behind it for our clients]."

The comments come after three weeks of drama among multiple security vendors. The weeks kicked off with a war of words between Sophos and Cylance, with Sophos slamming the next-generation endpoint security startup in a blog post, saying its technology was inferior to its own in tests. Cylance fired back, slamming the move as "dirty tactics" by Sophos and alleging that the company disabled features that would make the technology work as advertised.

Sophos said in a blog post about the issue that one way Cylance could help settle the score is to engage in third-party industry testing, saying the startup has been "absent from virtually all public, independent third-party tests" but one.

"We believe that customers should be wary of vendors who fail to participate in public tests because there is no way to hold them publicly accountable for their marketing claims," the blog post said.

Sophos wasn't the only endpoint vendor targeting Cylance. In late 2015, Symantec claimed in a blog post that its technology was more effective than that of the startup in its own tests, as well as third-party evaluations.

In a separate set of attacks, Palo Alto Networks went after competitor Fortinet on Twitter for allegedly failing to disclose its configurations for recent NSS Labs tests of its data center intrusion prevention systems.

Why don't you show the world your DC IPS test configuration, ? We at Palo Alto Networks did...

ICYMI

Everyone can get 100% if you (a) configure for that purpose and (b) accept high false-positive rates, . Be transparent

In an email to CRN, Chad Skipper, vice president product testing and certifications at Cylance, argued that third-party testing organizations aren't the best way to rank technologies, as it is "comparing apples to oranges" and can be biased based on methodologies or relationships.

"This represents a serious challenge both for endpoint security vendors and organizations seeking to fairly evaluate various products and approaches. Until independent testing organizations take into account the foundational differences between various endpoint security technologies and update testing methodologies accordingly, their results are going to be unavoidably flawed or incomplete," Skipper said.

Both Cylance and Sophos advocated in respective blog posts for partners and customers to test solutions for themselves.

However, some security vendors expressed frustration at the infighting in the security space. Invincea also wrote its own blog post, urging partners to step away from the "mudslinging" and focus on their own concrete third-party testing, commonly accepted standards. Invincea also explicitly stated what makes its technology different from those of competitors.

"There is a tipping point where marketing can go too far without backing up their claims, causing widespread skepticism across an industry. I think we’ve hit that point in cybersecurity and all of us (not just vendors) will suffer the consequences," the blog post said. "Let’s not single out Cylance and Sophos as the only guilty parties, because everyone in the security industry plays a role in the creation of this dynamic."