Report Archive for

Click on a report title to go to the download page for that report.

IBM Corporation Podcast: IBM Security Network Protection XGS 7100 Next-Generation Intrusion Prevention System (IPS)

Document number: 216167
Release Date: 12 Dec 2016

Tolly Group Founder Kevin Tolly hosts a twelve-minute podcast discussing Tolly's IBM Security Network Protection XGS 7100 Next-Generation Intrusion Prevention System (IPS) evaluation published in Tolly report #216106.

The Podcast provides an overview of the XGS 7100 efficacy and performance testing along with a Q&A with IBM's Jordan Carlson, WW Portfolio Marketing Manager, Security Analytics & Network Security, IBM Security.

Download is in mp3 format.
Intrusion Detection/Prevention Systems
IBM Corporation Webinar - IBM XGS 7100 Next-Generation Intrusion Prevention System (IPS) Efficacy and Performance Evaluation

Document number: 216128
Release Date: 04 May 2016

IBM commissioned Tolly to evaluate the effectiveness and performance of its IBM Security Network Protection XGS 7100 appliance. The IBM XGS 7100 is a next-generation intrusion prevention appliance, and the model tested was outfitted with 8x10GbE ports. Efficacy testing encompassed attack detection/ blocking of various threat corpora, both with and without SSL/TLS inbound/ outbound inspection enabled. Performance testing included multi-protocol throughput with SSL/TLS disabled and SSL/TLS inbound enabled, along with HTTP connections per second. The system detected 100% of publicly-disclosed exploits tested along with successfully blocking 100% of the McAfee Evader test suite.

View the on-demand webcast with Kevin Tolly presenting the study results along with Sr. Product Manager, IBM Security, Eric York by clicking on the link below which will take you to the IBM webcast landing page.

IBM Webcast View On-demand Landing Page.

The Tolly report can be downloaded from here.
(Note: There is no download for this webcast item.)
Intrusion Detection/Prevention Systems
IBM Corporation IBM Security Network Protection XGS 7100 Next-Generation Intrusion Prevention System (IPS) Efficacy and Performance Evaluation

Document number: 216106
Release Date: 12 Feb 2016

Network security threats continue to grow not only in number, but also in type and sophistication. Organizations need protection from zero-day attacks, advanced persistent threats, intricate evasion techniques, and more. And while security is a key objective, maintaining network performance and availability remain top priorities as well. Organizations need both proactive protection and performance from their network security solutions.

IBM commissioned Tolly to evaluate the effectiveness and performance of its IBM Security Network Protection XGS 7100 appliance. The IBM XGS 7100 is a next-generation intrusion prevention appliance, and the model tested was outfitted with 8x10GbE ports. Efficacy testing encompassed attack detection/ blocking of various threat corpora, both with and without SSL/TLS inbound/ outbound inspection enabled. Performance testing included multi-protocol throughput with SSL/TLS disabled and SSL/TLS inbound enabled, along with HTTP connections per second. The system detected 100% of publicly-disclosed exploits tested along with successfully blocking 100% of the McAfee Evader test suite.
Intrusion Detection/Prevention Systems
WINS WINS Sniper ONE 40G Intrusion Prevention System (IPS) Efficacy and Performance Evaluation

Document number: 215144
Release Date: 30 Sep 2015

In a large scale network, security solutions are installed to detect and protect the various network incidents. Moreover, enterprise/ISP administrators are using various security products to protect the service/internal resources by the characteristics of their service/internal resources (DNS Server, VoIP connection, Internet, etc). The security solutions should provide the stable performance without interrupting the data flow. SNIPER ONE is a security solution which stably detects and blocks the attacks in large scale network such as Mobile LTE, higher education, financial, service provider, enterprise data center, etc. The user can activate the different security policy by services like IPS, Anti-DDoS, VoIP, DNS, and so on.

WINS commissioned Tolly to evaluate the effectiveness and performance of its Sniper ONE 40G Intrusion Prevention System (IPS). The Sniper ONE is an inline IPS outfitted with four 10GbE ports. Testing encompassed attack detection/blocking in various application environments, packet level and Layer 7 (HTTP) throughput as well as an evaluation of some key features of the IPS. The system detected 98% of attacks and blocked 93% of attacks
Intrusion Detection/Prevention Systems
IBM Corporation Podcast: IBM Security Network Intrusion Prevention System GX7800 Comparative Efficacy and Performance Evaluation

Document number: 213130
Release Date: 07 Jun 2013

Tolly Group Director of Engineering John Tolly hosts a nine-minute podcast discussing Tolly's IBM GX7800 comparative evaluation published in Tolly report 212148.

John is joined by IBM's Senior Operations Manager Clinton McFadden.

Stream from the Tolly Podcast page.

Download is in mp3 format.
Intrusion Detection/Prevention Systems
IBM Corporation IBM Security Network Intrusion Prevention System GX7800 Comparative Efficacy and Performance Evaluation

Document number: 212148
Release Date: 06 Dec 2012

Enterprise-class networks today are facing more advanced threats from a multitude of sources than ever before. Effective threat protection solutions must defend against real-world threats that are evolving quickly, and at the same time deliver high levels of performance and availability. IBM commissioned Tolly to evaluate their protocol-based IBM Security Network Intrusion Prevention System GX7800 and compare its efficacy to that of a Snort-based device, a signature- based platform.

Tolly engineers conducted many different performance tests with the GX7800 and achieved a maximum of 35.7 Gbps throughput under mixed traffic loads. This demonstrates a great tolerance for network surges, growth and capacity over IBM's published performance characteristics. Tolly also evaluated the GX7800’s efficacy and functionality.

Tests showed the GX7800 to be more effective blocking publicly-available exploits than Snort and dramatically more effective when blocking mutated exploits - blocking 100% compared to 52% for Snort.
Intrusion Detection/Prevention Systems
Wedge Networks Wedge Networks BeSecure NDFP-2040 Web Security Appliance: Anti-virus Effectiveness and Performance Comparison Versus Fortinet FortiGate 3600A

Document number: 209141
Release Date: 15 Nov 2009

Wedge Networks commissioned Tolly to benchmark the anti-virus effectiveness and network performance of their BeSecure NDP-2040 Web Security Gateway and compare that with Fortinet's FortiGate 3600A security appliance.

Test were run using both the Extended WildList (August 2009) and the VX Heavens virus collections.
Intrusion Detection/Prevention Systems
Wedge Networks 稳捷网络 (Wedge Networks) BeSecure NDP-2040 WEB 安全设备与 Fortinet FortiGate 3600A

Document number: 209141ZH
Release Date: 14 Nov 2009

如今的 Web 2.0 网络不仅要求具备有效的病毒检测能力,还要求在不同有效载荷大小的情况下都具备稳定的性能。

虽然现在的防火墙和统一威胁管理 (UTM) 解决方案能够提供有效的防火墙能力,但要实现全面的防毒检测和良好的吞吐性能,通常需要使用一种与之互补的解决方案,例如稳捷网络 (Wedge Networks) BeSecure WEB 安全设备。
Intrusion Detection/Prevention Systems
Symantec Corporation Symantec Corporation Symantec Endpoint Protection 11.0 vs. McAfee Total Protection for Endpoint Performance Impact on Microsoft Office Usage

Document number: 208321
Release Date: 12 Sep 2008

Symantec Corporation commissioned The Tolly Group to evaluate the impact of two Enterprise class endpoint security offerings on host client performance: Symantec Endpoint Protection 11.0 compared with McAfee Total Protection for Endpoint. The Tolly Group installed Symantec Endpoint Protection 11.0 which provides anti-virus, anti-spyware and host intrusion prevention functionality in a single agent against the corresponding products in the McAfee Total Protection for Endpoint Bundle.

The Tolly Group benchmarked file “open” and “save/close” times, as well as memory usage on an unprotected Microsoft Windows Vista SP1 system and compared these with execution times on the protected systems.

Tests were conducted in July 2008.

Intrusion Detection/Prevention Systems
Trustwave Trustwave TS-1000 High-Speed Intrusion Prevention Appliance IPS Performance and Security Effectiveness Evaluation

Document number: 208296
Release Date: 25 Jun 2008

Trustwave commissioned The Tolly Group to evaluate its TS- 1000 High-Speed Intrusion Prevention Appliance to determine its performance levels along with its security effectiveness.

The Tolly Group measured the Layer 3 zero-loss (􀀁0.001% acceptable packet loss) throughput and latency of the TS-1000 in firewall only mode, with intrusion prevention active, and while under attack. Engineers also measured the TS-1000’s support for concurrent TCP connections and its connection setup rate. Finally, engineers subjected the TS-1000 to a variety of security attacks and identified the product’s effectiveness at protecting servers.

Tests were conducted in March 2008.

Intrusion Detection/Prevention Systems
Net Optics Net Optics, Inc. Net Optics 10/100/1000 iBypass Switch with Heartbeat Evaluation of Uninterrupted In-line Protection for IPS Appliances

Document number: 208291
Release Date: 20 May 2008

Net Optics, Inc. commissioned The Tolly Group to evaluate its 10/100/ 1000 iBypass Switch with Heartbeat across different network and device disruption scenarios in a simulated network that paired the bypass switch in-line with an IPS device.

The Tolly Group examined the bypass switch’s in-line protection solution using a leading enterprise IPS device. The goal was to introduce real-world failures to validate the capabilities provided through an integrated solution. Engineers measured the fail-over time of the Net Optics bypass switch in various fail-over scenarios and verified its additional network monitoring capabilities.

Tests were conducted in April 2008.

Intrusion Detection/Prevention Systems
Cymtec Cymtec Systems, Inc. Cymtec Sentry™ Propagation Protection Solution Detection Accuracy and Network Performance Evaluation

Document number: 208279
Release Date: 20 May 2008

Cymtec Systems, Inc. commissioned The Tolly Group to evaluate its Cymtec Sentry Propagation Protection Solution.

The Tolly Group examined the network performance and accuracy of the Cymtec Sentry. Tests measured throughput and latency; accuracy of threat detection; fail-over response; traffic shaping; and session limits. The Cymtec Sentry is composed of hardware appliances that are placed in specific network segments where propagation protection is required. All appliances are then managed by the Cymtec Sentry Management Console software. This controls the Cymtec Sentry Appliances installed on the network including configuration, current threats, alerts, and statistical information.

Tests were conducted in April 2008.

Intrusion Detection/Prevention Systems
Reflex Security, Inc. Reflex Interceptor 1000 Performance Evaluation of Network Security Switch under Severe Attack Strain and Fail-Open Scenarios

Document number: 207241
Release Date: 23 Oct 2007

Reflex Security, Inc. commissioned The Tolly Group to measure the performance of the vendor’s Reflex Interceptor 1000, a network security switch with eight GbE ports (four in, four out) that provides 1 Gigabit (Gbps) throughput for medium to large enterprises.

Engineers measured the performance of the Interceptor 1000 across four pairs of GbE interfaces, both with and without exposing the device to a taxing load of security threats. Engineers also measured the number of concurrent TCP connections sustained across the Interceptor 1000 and examined how the unit responds during an invoked power failure. Tests were conducted in August 2007.

Intrusion Detection/Prevention Systems
Colubris Networks White Paper Sponsored by Colubris Networks: Evaluating Wireless IPS Systems

Document number: 207231
Release Date: 23 May 2007

This Tolly Group white paper, commissioned by Colubris Networks, Inc., focuses on the key issues users must consider when evaluating wireless intrusion prevention systems. For the report, The Tolly Group evaluated the Colubris RF Manager, a multi-faceted WIPS designed to protect enterprise network infrastructures from wireless attacks.

The Tolly Group assessed the capability of the Colubris RF Manager to detect and block a range of wireless threats — from dealing with rogue APs, to detection and prevention of access point (AP) MAC address spoofing, to detection and prevention of Denial of Service (DoS) attacks, and several others.

Tolly Group engineers measured the effectiveness of the Colubris RF Manager against two other products: AirMagnet Inc.’s AirMagnet Enterprise and Aruba Networks Aruba Mobility Controller.

Intrusion Detection/Prevention Systems
3Com Corp. TollyEdge White Paper Series: Benchmarking Strategies for Wireless Intrusion Prevention Systems

Document number: 207216
Release Date: 19 May 2007

This comprehensive 32-page TollyEdge: Benchmarking Strategies for Wireless Intrusion Prevention Systems white paper identifies the chief factors with regards to protection, performance and ease-of-use for WIPS offerings that users need to address, and The Tolly Group offers its insights into the most practical way to benchmark these essential criteria.

The report provides unique perspectives from 3Com and AirDefense. The report aims to help readers understand the key issues they must consider, and the key metrics and processes they should employ to effectively benchmark any WIPS products.

Intrusion Detection/Prevention Systems
NetClarity, Inc. NetClarity Auditor Enterprise, Auditor Branch & Protection for Windows Manejo de Vulnerabilidad de Activos y Sistema Preventivo de Intrusos Evaluación en el Manejo de Vulnerabilidad, Características del Network Admission Control (NAC)

Document number: 207183ES
Release Date: 01 Feb 2007

NetClarity delegó a The Tolly Group a desarrollar pruebas en sus productos de seguridad Auditor Enterprise™, Auditor Branch™ y Protection for Windows™ en áreas del manejo de vulnerabilidad, Network Admission Control (NAC) y características de seguridad para terminales.

Las pruebas evaluaron diferentes características e implementacion de los productos al proveer soluciones de seguridad cumpliendo con las regulaciones acorde a la industria. Las pruebas también evaluaron su facilidad de instalación y su continuo manteniemiento en los productos. Las pruebas se realizaron en Enero del 2007.

Intrusion Detection/Prevention Systems
NetClarity, Inc. NetClarity Auditor Enterprise, Auditor Branch & Protection for Windows Vulnerability Management Appliances and Host-based Intrusion Prevention System Evaluation of Vulnerability Management, Network Admission Control Features and Endpoint Security Package

Document number: 207183
Release Date: 01 Feb 2007

NetClarity commissioned The Tolly Group to evaluate its Auditor Enterprise, Auditor Branch appliances and Protection for Windows endpoint security solution in terms of security vulnerability management, Network Admission control (NAC) and endpoint security features.

Tests evaluated the various features of the products and evaluated how they interoperated to provide a comprehensive security solution that also helps ensure and document regulatory compliance. Tests also evaluated the ease of deployment and ongoing maintenance of the products. Tests were conducted in January 2007.

Intrusion Detection/Prevention Systems
The Tolly Group TollyEdge White Paper Series: Benchmarking Strategies for Wireless Intrusion Prevention Systems

Document number: 207117
Release Date: 21 Jan 2007

This comprehensive 32-page TollyEdge: Benchmarking Strategies for Wireless Intrusion Prevention Systems white paper identifies the chief factors with regards to protection, performance and ease-of-use for WIPS offerings that users need to address, and The Tolly Group offers its insights into the most practical way to benchmark these essential criteria.

The report provides unique perspectives from AirDefense and AirTight Networks. The report aims to help readers understand the key issues they must consider, and the key metrics and processes they should employ to effectively benchmark any WIPS products.

Intrusion Detection/Prevention Systems
InfoExpress InfoExpress Dynamic Network Access Control Competitive “Ease-of-Use” Comparison versus Cisco Network Admission Control and Cisco Clean Access

Document number: 207165
Release Date: 19 Jan 2007

InfoExpress commissioned The Tolly Group to evaluate its Dynamic NAC (DNAC) 5 versus Cisco Systemss, Inc.’s Cisco Network Access Control (NAC) 2.0 and Cisco Clean Access (CCA) 4.0.

Tests concentrated on the effort necessary – in terms of number of steps required – to deploy and maintain the NAC solutions under test, and the potential impact of each step on the existing network infrastructure. Tolly Group engineers audited the process of deploying the NAC solution in a representative network, and also documented the effort involved in performing routine maintenance of each NAC solution. Testing was conducted in November 2006.

Intrusion Detection/Prevention Systems
Siemens Enterprise Networks Tolly Group White Paper Series sponsored by Siemens: Evaluating Wireless Intrusion Prevention Systems vs. Cisco and Network Chemistry

Document number: 206156
Release Date: 31 Oct 2006

Siemens commissioned The Tolly Group in September 2006 to conduct a comprehensive hands-on evaluation of Siemens HiPath Wireless Manager HiGuard. This white paper provides a detailed evaluation of the capabilities of the Siemens HiGuard versus rival products from Cisco Systems and Network Chemistry.

Tolly Group engineers examined a variety of capabilities delivered by the Siemens HiGuard, a multi-faceted integrated WIPS designed to protect enterprise network infrastructures from wireless attacks. The Siemens HiGuard detected 100% of security threats launched against it.

This September 2006 white paper provides a comprehensive look at the Siemems HiPath Wireless Manager HiGuard and the necessary tools for corporations to effectively evaluate WIPS.

Intrusion Detection/Prevention Systems
Cetacea Networks Corp. Cetacea Networks OrcaFlow® Embedded Technology for Ethernet Switches, Evaluation of Network-based Anomaly Detection Feature

Document number: 206140
Release Date: 07 Aug 2006

Cetacea Networks Corp. commissioned The Tolly Group to evaluate its OrcaFlow® network-based anomaly detection (NBAD) technology, a novel software approach to monitoring and detecting a wide array of security threats.

Tolly Group engineers conducted a variety of test scenarios involving OrcaFlow. A baseline test measured the ability of the software to handle “normal” network traffic loads without any adverse impact of switch performance and with minimal to no false positives introduced during the monitoring process. A high volume multiport monitoring test examined OrcaFlow’s ability to rapidly detect the traffic anomalies, identify the software’s ability to detect a breadth of threats and do so with minimal false positives.

Additionally, Tolly Group engineers verified OrcaFlow’s compatibility between SNMP MIB II devices and sFlow® capable devices. Finally, engineers measured the processor utilization on Ethernet switches and bandwidth usages by OrcaFlow® TeraSAR™ sensor to determine its impact on host devices.

Intrusion Detection/Prevention Systems
Cetacea Networks Corp. Cetacea Networks OrcaFlow® TeraSAR TS100-P4 Traffic Anomaly Sensor, Evaluation of Network-based Anomaly Detection Device

Document number: 206142
Release Date: 19 Jun 2006

Cetacea Networks Corp. commissioned The Tolly Group to evaluate its OrcaFlow® TeraSAR TS100-P4 network-based anomaly detection (NBAD) platform, a novel software approach to monitoring and detecting a wide array of security threats. The OrcaFlow TeraSAR is available in models which will monitor from 512 to 10,240 switched Ethernet ports.

Tolly Group engineers conducted a variety of test scenarios involving OrcaFlow. A baseline test measured the ability of the software to handle “normal” network traffic loads without any adverse impact of switch performance and with minimal to no false positives introduced during the monitoring process. A high volume multiport monitoring test examined OrcaFlow’s ability to rapidly detect the traffic anomalies, identify the software’s ability to detect a breadth of threats and do so with minimal false positives.

Additionally, Tolly Group engineers verified OrcaFlow’s compatibility between SNMP MIB II devices and sFlow® capable devices. Finally, engineers measured the processor utilization on Ethernet switches and bandwidth usages by OrcaFlow® TeraSAR™ sensor to determine its impact on host devices.

Intrusion Detection/Prevention Systems
IntruGuard Devices, Inc. IntruGuard Devices, Inc. IG2000 Rate-Based Intrusion Prevention System, Layer 2-4 DoS/DDoS Attack Mitigation and Performance Evaluation

Document number: 206129
Release Date: 03 May 2006

IntruGuard Devices, Inc. commissioned The Tolly Group to evaluate the performance of its IG2000 Rate-Based Intrusion Prevention System (RBIPS), a stateful security appliance designed to monitor bidirectional traffic, intercept DoS/DDoS floods and other anomalies such as state-anomalies, header anomalies, network scans, dark-address scans, and port scans, etc.

The Tolly Group validated the performance of the IG2000, as well as the appliance’s effectiveness at detecting and mitigating a variety of high-rate attacks at Layers 2, 3 and 4. Additionally, Tolly Group engineers measured the latency of the appliance and effectiveness under attack.

Intrusion Detection/Prevention Systems
Siemens Enterprise Networks White Paper Sponsored by Siemens: Evaluating Wireless IPS Systems

Document number: 206119
Release Date: 18 Apr 2006

This Tolly Group white paper, commissioned by Siemens, focuses on the key issues users must consider when evaluating wireless intrusion prevention systems. For the report, The Tolly Group evaluated HiPath Wireless Manager Advanced (HWMA), a multi-faceted WIPS designed to protect enterprise network infrastructures from wireless attacks.

The Tolly Group assessed the capability of HWMA to detect and block a range of wireless threats — from dealing with rogue APs, to detection and prevention of access point (AP) MAC address spoofing, to detection and prevention of Denial of Service (DoS) attacks, and several others.

Tolly Group engineers measured the effectiveness of HWMA against two other products: AirMagnet Inc.’s AirMagnet Enterprise and Aruba Networks Aruba Mobility Controller. Tests were conducted at AirTight Networks facilities in Mountain View, CA during December 2005.
Intrusion Detection/Prevention Systems
Wiresoft Net, Inc. Tolly Group White Paper Series: Securing SMB Networks Without Breaking the Bank

Document number: 206113
Release Date: 07 Apr 2006

SMBs need a multipurpose security platform that provides complete security protection. The dilemma that SMBs face today is that the IT market abounds with supplier after supplier that offer point solutions for security.

Wiresoft Net, Inc. commissioned The Tolly Group to evaluate its Wiresoft Sentry Security Suite, a versatile platform that offers a variety of security services including transparent virus scanning, challenge response spam blocking, stateful packet firewalling, VPN services (PPTP and IPSec) and more.

Tolly Group engineers conducted a battery of performance tests on the Sentry Security Suite, such as its effectiveness at blocking spam traffic, the aggregate throughput delivered while operating as a firewall, the aggregate throughput delivered across a VPN connection and the aggregate throughput while scanning Web traffic for viruses. Engineers also evaluated a number of functions, such as set up, hardware reliability and failover protection.

Intrusion Detection/Prevention Systems
Symantec Corporation Symantec Gateway Security Version 3.0, Firewall Performance and Security Capability Benchmark versus Cisco ASA 5520 and Juniper NetScreen-500

Document number: 206108
Release Date: 06 Feb 2006

Symantec Corp. commissioned The Tolly Group to evaluate its Symantec Gateway Security solution, a full-inspection firewall with integrated dynamic routing and VLAN support, intrusion prevention, anti-virus, anti-spam, URL and Dynamic Document Review- based (DDR) content filtering, VPN (IPSec and SSL), and intrusion detection.

Tolly Group engineers evaluated the capability of the Symantec Gateway Security (SGS) Version 3.0 software running on a Symantec Gateway Security 5660 to identify and to block network attacks common to enterprise networks. Engineers also examined the SGS capability to isolate and to block suspicious network and audit events, and as well as block common evasion techniques used to deceive security appliances and affect end users. Additionally, engineers measured the aggregate throughput delivered by the SGS 5660 while configured to scan all traffic for attacks, as well as the connection set-up rate and the maximum number of sustained connections supported. Finally, engineers evaluated the graphical user interface (Security Gateway Management Interface) of the SGS. Tests were conducted during September and November 2005.

Engineers benchmarked the SGS appliance against a Cisco Systems, Inc. Adaptive Security 5520 appliance and a Juniper Networks, Inc. NetScreen-500 integrated firewall/IPSec VPN security appliance.
Intrusion Detection/Prevention Systems
AirTight Networks, Inc. White Paper Sponsored by AirTight Networks: Evaluating Wireless IPS Systems

Document number: 206103
Release Date: 06 Feb 2006

This Tolly Group white paper, commissioned by AirTight Networks, Inc., focuses on the key issues users must consider when evaluating wireless intrusion prevention systems. For the report, The Tolly Group evaluated SpectraGuard Enterprise, a multi-faceted WIPS designed to protect enterprise network infrastructures from wireless attacks.

The Tolly Group assessed the capability of SpectraGuard Enterprise to detect and block a range of wireless threats — from dealing with rogue APs, to detection and prevention of access point (AP) MAC address spoofing, to detection and prevention of Denial of Service (DoS) attacks, and several others.

Tolly Group engineers measured the effectiveness of SpectraGuard Enterprise against two other products: AirMagnet Inc.’s AirMagnet Enterprise and Aruba Networks Aruba Mobility Controller. Tests were conducted at AirTight Networks facilities in Mountain View, CA during December 2005.
Intrusion Detection/Prevention Systems
Reflex Security, Inc. "Talking Outside the Box": Podcast Interview with Reflex Security CTO Hezi Moore and Performance Evaluation of Reflex IPS-100 Appliance

Document number: 206101
Release Date: 31 Jan 2006

This 14-minute podcast focuses on intrusion prevention appliances. The podcast reviews the chief findings from a performance evaluation of Reflex Security Inc.’s IPS-100 intrusion prevention appliance.

In addition, Kevin Tolly, President/CEO/Founder of The Tolly Group interviews Reflex Security CTO Hezi Moore on the issues users face when deploying IPS appliances and achieving optimal performance.

Details of the test can be found in document 205136.

Click below to download the "podcast" MP3 audio file.
Intrusion Detection/Prevention Systems
Reflex Security, Inc. Reflex Security IPS100 Intrusion Prevention Appliance, Performance, Security and Usability Evaluation

Document number: 205136
Release Date: 20 Nov 2005

Reflex Security, Inc. commissioned The Tolly Group to test the Reflex IPS100 network intrusion prevention appliance. The Reflex IPS blocks a comprehensive range of malicious traffic, including HTTP attacks, Denial-of-Service attempts, scans, backdoor exploits, floods, viruses, and worms. The Tolly Group validated the performance of the Reflex IPS, as well as the appliance’s effectiveness at detecting and preventing a variety of attacks. The Tolly Group also evaluated the system’s reliability, reporting and ease of use.

Tolly Group engineers conducted a battery of performance tests, focusing on HTTP throughput across the Reflex IPS appliance under normal conditions, and when subjected to attack traffic generated by Blade Software IDS Informer. They also performed a security test to measure the number of IDS Informer attacks blocked by the Reflex IPS100 while handling HTTP traffic in the background, and tests were also conducted to verify that the Reflex IPS100 appliance could block E-mails infected with worms and virus.

Intrusion Detection/Prevention Systems
Shenick Network Systems Limited White Paper: Test Tool Evolution Keeps Pace with Network Operator Needs

Document number: 205122
Release Date: 19 Aug 2005

Shenick Network Systems commissioned The Tolly Group in July 2005 to examine the company's diversifEye™ integrated network, application and security performance test system. The diversifEye platform offers a high degree of granular control over each application flow and individual network services to measure Quality of Service (QoS) and Quality of Experience (QoE) with a high degree of accuracy.

QoE is a higher-level abstraction of network measurements. QoE equates to the time required to change a channel in an IP TV application, for instance, or for a Web page to download; QoE identifies the performance measurement that is most meaningful to end users.

The objective of the diversifEye testing was to test an access network with triple-play traffic mixed with Distributed Denial of Service (DDoS) traffic which attempts to overwhelm the network with invalid traffic and thus deny service to valid users and P2P that consumes too much service provider bandwidth. Such triple play traffic represents revenue generating "value-added" services like VoIP and video. Tests focused on Layer 2/3 traffic generation/analysis capabilities, as well as the capability of the diversifEye platform to create traffic conditions to benchmark network equipment and services higher up the protocol stack.
Intrusion Detection/Prevention Systems
NETASQ NETASQ F2000 IPS-Firewall Multiservice Security Appliance Performance Evaluation

Document number: 205120
Release Date: 10 Jul 2005

NETASQ commissioned The Tolly Group to evaluate the NETASQ F2000 IPS-Firewall, a purpose-built network security appliance that combines real-time intrusion prevention, firewall service, IPSec virtual private networking (VPN), clientless SSL VPNs, advanced content filtering, anti-spam, anti-virus and other integrated security services.

Tolly Group engineers focused testing on the performance of the NETASQ F2000 using a mostly default configuration, measuring the device’s zero-loss throughput (while IPS services were active), benchmarking latency introduced by the device under varying traffic loads and conditions. (In its default state, the NETASQ F2000 enables protocol analysis and signature and port-scan detection, among other IPS capabilities.) Tests were conducted at The Tolly Group’s Boca Raton, FL. facilities in May 2005.
Intrusion Detection/Prevention Systems
Symantec Corporation Symantec Network Security 7160 Intrusion Prevention Appliance Performance Evaluation

Document number: 205111
Release Date: 23 Jun 2005

Symantec Corp. commissioned The Tolly Group to evaluate its Symantec Network Security 7160, an eight-port Gigabit Ethernet security appliance that offers intrusion protection while delivering throughput in excess of 1 Gbps.

Tolly Group engineers evaluated the capability of the Symantec Network Security 7160 to detect and block network attacks and threats common to enterprise networks. Engineers also examined the Symantec Network Security 7160’s capability to detect and block suspicious network threats and security risks /audit events. Next, engineers tested the ability for the Symantec Network Security 7160 to continue to block threats when common and advanced evasion techniques were used to deceive the security appliance and affect end users. Additionally, engineers measured the aggregate throughput delivered by the Symantec Network Security 7160 while configured to scan all traffic for attacks, as well as the connection set-up rate and the maximum number of sustained connections supported. Finally, engineers evaluated the management capabilities and ease of use for the Symantec Network Security 7160. Tests were conducted in March 2005.

Intrusion Detection/Prevention Systems
Radware Ltd. White paper: Measuring Key Criteria of Intrusion Prevention Systems

Document number: 205114
Release Date: 20 Jun 2005

Radware, Inc. commissioned The Tolly Group to evaluate its DefensePro 3000, an intrusion prevention switch with DoS protection that combines bandwidth management for attack isolation and traffic shaping to offer enterprise and carrier networks protection against a diverse range of network- and application-level attacks.

The aim of the testing was to evaluate the DefensePro 3000 to determine that it delivers the advanced IPS requirements users need to combat today’s sophisticated security threats.

Tolly Group engineers examined the performance of the DefensePro 3000 in various scenarios to understand the maximum throughput offered by the switch while it actively handled various attacks and processed signature loads. Engineers also evaluated the manner in which the DefensePro 3000 was able to detect and block attacks that utilized common evasion techniques and engineers examined a facility that restricts bandwidth to background applications that otherwise could interfere with strategic application traffic. Finally, The Tolly Group examined a capability of the DefensePro 3000 to implement protection options for different segments of supported networks. Tests were conducted in April 2005 at Radware facilities in Israel.

Also see document 205112.
Intrusion Detection/Prevention Systems
Radware Ltd. Radware, Inc. DefensePro 3000 Throughput Benchmark and Attack Mitigation Evaluation

Document number: 205112
Release Date: 31 May 2005

Radware, Inc. commissioned The Tolly Group to evaluate its DefensePro 3000, an intrusion prevention switch with Denial of Service (DoS) protection that combines bandwidth management for attack isolation and traffic shaping to offer enterprise and carrier networks protection against a diverse range of network- and application-level attacks.

Tests show that the DefensePro 3000 is adept at identifying and blocking attacks with zero instances of false positives while simultaneously handling multi-Gigabit traffic loads. From a performance standpoint, the DefensePro 3000 was able to handle 2.5 Gbps of “real-world” throughput while simultaneously handling either a 40,000-packet per second (pps) worm attack, a 200-Mbps SYN Flood attack or a 100-Mbps DoS attack. Tests also show the DefensePro 3000 is capable of protecting Secure Sockets Layer (SSL) data and can isolate attacks to protect mass mailings and control P2P traffic.

Intrusion Detection/Prevention Systems
Internet Security Systems, Inc. (IBM) Internet Security Systems Proventia Intrusion Prevention Appliance G2000 Throughput, Latency and Failover Performance Evaluation

Document number: 205110
Release Date: 12 Apr 2005

Internet Security Systems Inc. commissioned The Tolly Group to evaluate the performance of its Proventia Intrusion Prevention Appliance G2000, a security appliance designed to monitor all inbound traffic, intercept attack traffic and other security threats and block the attacking stream so it does not reach intended targets. The Proventia G2000 is an eight-port device capable of supporting four monitoring segments with two ports dedicated to each segment. The appliance came with two 10/100/1000 Ethernet management ports and has an advertised throughput of 2 Gbps.

Tolly Group engineers measured the zero-loss Layer 2 bidirectional throughput of the Proventia G2000, as well as the latency of the appliance. In addition, engineers measured the TCP performance in terms of the sustained new connection rate and the maximum simultaneous connections supported. Lastly, Tolly Group engineers evaluated the effectiveness of the Proventia G2000 at thwarting certain Denial of Service (DoS) attacks that otherwise could impact network performance adversely. Tests were conducted in March 2005.

Tests show that the Proventia G2000 is able to deliver network performance on par with typical network switching devices, meaning its presence will not degrade network throughput when it is deployed in an enterprise network. Tests show the Proventia G2000 delivers between 2 Gbps and 5 Gbps of Layer 2, bidirectional, zero-loss throughput (depending on frame size), along with low latency. Moreover the appliance processes TCP connections at rates that are necessary to support scalable enterprise applications and also help repel DoS attacks. Finally, tests show the Proventia G2000 repels DoS, Nimda and Blaster attacks on one network segment without compromising the throughput rate of normal application traffic traversing a second network segment. In essence, good traffic on one network segment remains unaffected by attack traffic that is isolated on a second network segment.

Intrusion Detection/Prevention Systems
Check Point Software Technologies Ltd. White Paper: Improving Security ROI via an Integrated Application Security Solution - JAPANESE VERSION

Document number: 205101JP
Release Date: 23 Feb 2005

Check Point Software Technologies, Inc. commissioned The Tolly Group to conduct a series of tests that demonstrate the effectiveness of the company's Application Intelligence within the Check Point VPN-1 NG Series firewall compared to other offerings and how they handle threatening security exploits. Check Point believes its Check Point VPN-1 NG Series firewall is the only perimeter security gateway to provide protection for the entire perimeter environment -- without requiring the purchase and deployment of a second standalone "intrusion protection" device.

Engineers tested the security attributes of Check Point VPN-1 NG firewall against a Cisco PIX 515E and a Juniper Networks NetScreen-204. The Check Point, Cisco and Juniper security solutions went through 17 rigorous tests that exposed them to a variety of common application-level exploits including SSL, SQL and HTTP-based vulnerabilities. Tests demonstrated that while Cisco’s and Juniper’s solutions are response-based, meaning that they rely on pre-defined signatures to defeat attacks, Check Point’s solution is proactive, protecting the network against attacks before they even occur.

Tests show that the Check Point VPN-1 NG Gateway offers greater depth of protection in comparison to Cisco and Juniper products tested, and also provides application-level security for a greater number of protocols including SQL, HTTP, HTTPS, SOCKS, IPSec, BGP, OSPF, and RIP. Moreover, the Check Point gateway offers integrated IPS, firewall and VPN capabilities in a single device, unlike the rival products that steer users to a companion security device. Finally, the Check Point VPN-1 NG Gateway offers a significant total cost-of-ownership advantage.

Please note that this version does not include the appendix. Please see the English version, Document 205101, for that information..
Intrusion Detection/Prevention Systems
Check Point Software Technologies Ltd. White Paper: Improving Security ROI via an Integrated Application Security Solution - GERMAN VERSION

Document number: 205101DE
Release Date: 23 Feb 2005

Check Point Software Technologies, Inc. commissioned The Tolly Group to conduct a series of tests that demonstrate the effectiveness of the company's Application Intelligence within the Check Point VPN-1 NG Series firewall compared to other offerings and how they handle threatening security exploits. Check Point believes its Check Point VPN-1 NG Series firewall is the only perimeter security gateway to provide protection for the entire perimeter environment -- without requiring the purchase and deployment of a second standalone "intrusion protection" device.

Engineers tested the security attributes of Check Point VPN-1 NG firewall against a Cisco PIX 515E and a Juniper Networks NetScreen-204. The Check Point, Cisco and Juniper security solutions went through 17 rigorous tests that exposed them to a variety of common application-level exploits including SSL, SQL and HTTP-based vulnerabilities. Tests demonstrated that while Cisco’s and Juniper’s solutions are response-based, meaning that they rely on pre-defined signatures to defeat attacks, Check Point’s solution is proactive, protecting the network against attacks before they even occur.

Tests show that the Check Point VPN-1 NG Gateway offers greater depth of protection in comparison to Cisco and Juniper products tested, and also provides application-level security for a greater number of protocols including SQL, HTTP, HTTPS, SOCKS, IPSec, BGP, OSPF, and RIP. Moreover, the Check Point gateway offers integrated IPS, firewall and VPN capabilities in a single device, unlike the rival products that steer users to a companion security device. Finally, the Check Point VPN-1 NG Gateway offers a significant total cost-of-ownership advantage.

Please note that this version does not include the appendix. Please see the English version, Document 205101, for that information..
Intrusion Detection/Prevention Systems
Check Point Software Technologies Ltd. White Paper: Improving Security ROI via an Integrated Application Security Solution

Document number: 205101
Release Date: 23 Feb 2005

Check Point Software Technologies, Inc. commissioned The Tolly Group to conduct a series of tests that demonstrate the effectiveness of the company's Application Intelligence within the Check Point VPN-1 NG Series firewall compared to other offerings and how they handle threatening security exploits. Check Point believes its Check Point VPN-1 NG Series firewall is the only perimeter security gateway to provide protection for the entire perimeter environment -- without requiring the purchase and deployment of a second standalone "intrusion protection" device.

Engineers tested the security attributes of Check Point VPN-1 NG firewall against a Cisco PIX 515E and a Juniper Networks NetScreen-204. The Check Point, Cisco and Juniper security solutions went through 17 rigorous tests that exposed them to a variety of common application-level exploits including SSL, SQL and HTTP-based vulnerabilities. Tests demonstrated that while Cisco’s and Juniper’s solutions are response-based, meaning that they rely on pre-defined signatures to defeat attacks, Check Point’s solution is proactive, protecting the network against attacks before they even occur.

Tests show that the Check Point VPN-1 NG Gateway offers greater depth of protection in comparison to Cisco and Juniper products tested, and also provides application-level security for a greater number of protocols including SQL, HTTP, HTTPS, SOCKS, IPSec, BGP, OSPF, and RIP. Moreover, the Check Point gateway offers integrated IPS, firewall and VPN capabilities in a single device, unlike the rival products that steer users to a companion security device. Finally, the Check Point VPN-1 NG Gateway offers a significant total cost-of-ownership advantage.

Please note that this is a large document - close to 2MB. A shorter version of this document is available, without the 30+ page appendix, from the Check Point web site.
Intrusion Detection/Prevention Systems
Top Layer Networks Top Layer Networks Attack Mitigator IPS 5500 IPS Evaluation versus TippingPoint UnityOne-2400

Document number: 204146
Release Date: 02 Dec 2004

Top Layer Networks, Inc. commissioned The Tolly Group to evaluate its Attack Mitigator IPS 5500, an intrusion prevention system designed to stop network-based threats while allowing legitimate transactions to complete.

The Tolly Group evaluated the effectiveness of the Attack Mitigator IPS 5500 at dealing with single-protocol and mixed-protocol Distributed Denial-of-Service (DDoS) SYN flood attacks. Moreover, engineers examined the capability of the Attack Mitigator IPS 5500 to handle real-time identification and blocking of embedded worms when mixed in otherwise normal traffic. Tests focused on the capability to filter such traffic while monitoring what, if any, degradation that screening caused to the IPS’ connection rate.

Engineers conducted these tests on the Attack Mitigator IPS 5500 and compared the results to a TippingPoint Technologies Inc. UnityOne-2400 IPS. In every test case the Top Layer IPS5500 outperformed the UnityOne-2400. Tests show the IPS5500 is up to 82.5% more effective at blocking embedded worm attacks than the UnityOne-2400 during tests with worms infecting from 10% to 50% of traffic. Tests also demonstrate that the IPS5500 completes 100% of HTTP and mixed protocol connections attempted while under SYN flood attack. Tests were conducted in October 2004.
Intrusion Detection/Prevention Systems
TippingPoint Technologies, a 3Com Company Tipping Point Technologies, Inc.UnityOne Intrusion Prevention Appliances Performance Evaluation

Document number: 203101
Release Date: 01 Feb 2003

TippingPoint Technologies, Inc. commissioned The Tolly Group to evaluate its line of UnityOne Intrusion Prevention Appliances, the UnityOne 2400, 1200 and 400. UnityOne is an intrusion prevention system that blocks worms, Trojans, viruses, hybrid attacks, denial of service attacks and other attacks while delivering switch-like performance. Engineers measured network performance (aggregate throughput and latency), the precision of each device’s security filtering and the efficiency of testing for false positives and false negatives. Product class: Intrusion prevention system Intrusion Detection/Prevention Systems