Abstract

WatchGuard Technologies, Inc. commissioned The Tolly Group to evaluate the Firebox® V60, a four-port Fast Ethernet firewall/VPN security appliance in a variety of real-world zero-loss throughput scenarios and to compare its performance against a Cisco PIX 515E two-port appliance and a NetScreen Technologies, Inc. NetScreen-50 four-port appliance.

Summary of the Test Report:

WatchGuard Technologies commissioned The Tolly Group to benchmark the Firebox V60 firewall/VPN appliance against Cisco’s PIX 515E and NetScreen’s NetScreen-50. The evaluation aimed to measure real-world zero-loss throughput across firewall and VPN scenarios, reflecting configurations with large rule sets and multiple active sessions. The Firebox V60 was tested under rigorous conditions, including configurations with 500 firewall rules and up to 8,000 concurrent sessions, simulating the demands of branch office and remote access deployments.

The Firebox V60 consistently outperformed its rivals, delivering up to 21X the firewall throughput of the Cisco PIX 515E and nearly 1.5X the throughput of the NetScreen-50 in tests of 512-byte frames and larger. In VPN throughput tests, the Firebox V60 achieved up to 188% greater performance than NetScreen-50 and 75% more than PIX 515E, particularly excelling in larger frame sizes. Even under taxing rule sets where Cisco’s device failed to pass traffic, the Firebox V60 maintained throughput levels comparable to baseline, “best-case” configurations.

The Firebox V60’s performance demonstrated resilience under complex configurations, handling deep rule sets without degrading throughput. Its ability to maintain high throughput in both firewall and VPN scenarios, combined with advanced features like stateful packet filtering, DoS protection, VLAN tagging, and robust management tools, solidifies its positioning as a versatile and high-performing security appliance for small to medium-sized enterprises needing reliable, scalable protection across their network edge.

Key Performance Metrics – WatchGuard Firebox V60

• Firewall Throughput (Zero-Loss Performance):

  • Up to 21X more throughput than Cisco PIX 515E.

  • Up to 1.5X more throughput than NetScreen-50 in tests with 512-byte frames and larger.

  • Maintained 199 Mbps throughput in real-world configurations with 500 rules and 500 IP sessions.

  • Outperformed rivals across all packet sizes (64, 512, 1,024, 1,518 bytes) in complex rule-set scenarios.

  • Cisco PIX 515E failed to pass traffic in high rule-count scenarios.

• VPN Throughput (Zero-Loss Performance):

  • Achieved up to 188% more throughput than NetScreen-50.

  • Up to 75% more VPN throughput than Cisco PIX 515E for larger packet sizes.

  • Delivered 109.6 Mbps at 1,024-byte frames and 100 Mbps at 1,518-byte frames.

  • Outperformed rivals in IPSec fragmentation scenarios, efficiently handling maximum Ethernet frame sizes.

• Resiliency Under Load:

  • Performance remained consistent even under heavy configurations (500 IP sessions with 500 rules).

  • Minor throughput reductions (<1 Mbps) in high-rule scenarios, compared to significant drops in competitor devices.

  • Demonstrated efficient rule processing, even when each session required unique rule evaluation.