Executives at security appliance maker WatchGuard Technologies said the firm is accepting at least some of the results of a battery of extensive tests on its next-generation firewall that ultimately caused the appliance to fall well below its competitors.
NSS Labs tested nine NGFW appliances provided by Check Point, Dell-SonicWall, Fortinet, Juniper Networks, Palo Alto Networks, Sourcefire, Stonesoft and WatchGuard. The results of the study were issued by the firm last week.
In tests conducted on the WatchGuard XTM 2050 appliance, the company was able to detect roughly half of the evasion techniques the testing team pitted against it. The widely known hacker techniques, such as HTML obfuscation, help disguise an attack to avoid being detected by the security appliance and is weighted heavily in the NSS Labs' tests.
"Although WatchGuard demonstrated a good level of exploit protection, it was let down by its poor antievasion capabilities and a suboptimal price-performance ratio," NSS Labs noted in its report.
Frank Artes, director of research at NSS Labs, said the company also had some shortcomings to its system management capabilities. The company's system manager was not as scalable and mature as other vendor products. WatchGuard was the only vendor tested by NSS Labs that was given a "caution" designation.
"It really hurts them when other aspects that they should be doing aren't scoring at 100 percent," Artes said. "We expect WatchGuard, if it puts in the tweaks necessary, between now and next year it could be sitting shoulder to shoulder with Check Point."
WatchGuard also failed to integrate with NSS Labs' Active Directory implementation, making it unable to correctly enforce user-based policies. The appliance also scored lower than its peers in blocking remote attacks against vulnerable applications, blocking about 85 percent of the remote attacks. It had an overall rate of blocking of 91 percent of exploits.
Dave Taylor, vice president of corporate strategy at Seattle-based WatchGuard, said the technology is tuned as a unified threat management appliance and not as a firewall. The firm's appliance was tested by NSS Labs in the firewall category last year and the overall results came out better.
"It absolutely is not an accurate representation of what our customers will find. Our customers that run our product love it, and we are perceived very highly in the market," Taylor said. "We don't optimize for NGFW; we optimize for UTM." Taylor said the results were "frustrating," explaining to CRN that while other vendor NGFW appliances do single-pass testing, WatchGuard uses seven technologies to scan network packets for malware and anomalous activity. The firm's development team continues to work on making the appliance easy to deploy and maintain, and technology partners are constantly boosting their effectiveness, Taylor said. WatchGuard also has failover and system redundancy capabilities, he said.
NEXT: Corrupted Firmware Caused Delay