Tolly Report Archive: 50 Most Recent Reports

Click on report title to go to the report abstract page. Report download requires registration and/or purchase.





Ruijie Networks Co., Ltd. Document number: 221149
Reyee Wi-Fi 6 (802.11ax) Home WLAN Access Points Comparative Performance Evaluation vs. NETGEAR and TP-Link
Release date: 18 November 2021

Wi-Fi 6 is the latest generation Wi-Fi technology that brings improvements in individual device and overall system throughput. Built upon more than 15 years experience in the WLAN market, Ruijie Networks Co., Ltd. developed a series of Wi-Fi 6 access point models to cover different application scenarios. Now, Wi-Fi 6 technology is coming to the home market where demand for Wi-Fi connectivity and bandwidth has surged. Today, It is not uncommon for homes to have dozens of devices connected to Wi-Fi at any given time. In addition to computers and smartphones, Smart TVs, thermostats, cameras, doorbells, lighting, sensors and more now have persistent Wi-Fi connections.

Ruijie commissioned Tolly to evaluate the performance of its Reyee (sub-brand) standalone and mesh Wi-Fi 6 home access points against similar models from NETGEAR and TP-Link. Tests included multiple user tests of up to 78 simultaneous users, single-client maximum performance tests, and signal coverage tests. Tests were run both at 2.4GHz and 5GHz frequencies.




SonicWall Document number: 221148
SonicWall NSsp 13700 Next-Generation Firewall (NGFW) Strategic Analysis vs. Fortinet FG-2600F
Release date: 16 November 2021

Remote working and cloud computing have had a major impact on the way that enterprises conduct business. Universally, the shift to distributed work has put a focus on both security and connectivity. Or, in the context of this report, next- generation firewalls and SD-WANs. Given the array of services available, and the cost differential across vendors, it is prudent to analyze offerings before proceeding with an NGFW acquisition. SonicWall has designed its NSsp 13700 to excel in protection of large enterprises while maintaining an industry-leading price- performance ratio.

SonicWall commissioned Tolly to review the published specifications of its NSsp 13700 and compare it to Fortinet’s FG-2600F. Analysis was primarily focused on evaluating key total cost of ownership (TCO) metrics of the comparable solutions.

The Tolly analysis shows that SonicWall has a dramatically lower 3-year TCO for High-Availability (HA) deployments compared to Fortinet while providing comparable or better security feature sets and performance characteristics. The TCO is enhanced by SonicWall’s High Availability licensing policy allowing a single firewall license to be used for both the primary and secondary (HA) appliance.




VMware, Inc. Document number: 221128ZH
VMware SD-WAN ⽤户体验 使⽤双 WAN 链接随时随地⼯作
Release date: 12 November 2021

SD-WAN 的灵活性和易⽤性使其作为“下⼀代”⼴域⽹,在各类企业和机构中异常受 欢迎。SD-WAN 的 overlay ⽹络在保持稳定的吞吐量性能的同时,还能为终端⽤户 的体验带来显著的好处。VMware SD-WAN™ 可以⾃动补救数据包丢失,即使在 不佳的 WAN ⽹络条件下也能为⽤户提供⾼质量的体验。双 WAN 链路的部署提供 更⼤的灵活性、稳健性和更⾼的总吞吐量,进⼀步增强了⽤户体验。

VMware 和 Intel 委托 Tolly 对 VMware SD-WAN 的服务质量和对较差⽹络状况的 补救能⼒进⾏了基准测试,以评估该⽅案在使⽤双 WAN 链路部署时,承载 IP 语⾳ (VoIP)和随时随地⼯作(WFA)应⽤的终端⽤户体验。此外,Tolly 还对 VMware SD-WAN 的数据包吞吐量进⾏了基准测试。

即使在 WAN 链路有 15% 丢包率,150ms 延迟,20ms 抖动的情况下,VMware SD-WAN 在双链路上也提供了良好的语⾳通话质量(MOS 为 4.31)。请分别参⻅ 图 1 和图 2。 VMware SD-WAN 展示了对双 WAN 链路的⾃动优化,⾃动优先选 择低延迟的链路。从微软 OneDrive 下载⼀个 250MB 的视频,即使在⼀个链接故 障后,下载仍在继续。在两条链路都激活的情况下,单个下载会话能够使⽤两条链 路的叠加带宽,以实现更快的下载。此外,两台 VMware SD-WAN Edge 640 设备 间可实现总共 3Gbps 的设备吞吐量。




VMware, Inc. Document number: 221128
User Experience with VMware SD-WAN Work-from-Anywhere with Dual WAN Links
Release date: 12 November 2021

The flexibility and ease-of-use of SD-WAN has made it extraordinarily popular as the “next-gen” WAN for organizations large and small. SD-WAN’s network overlay topology can also provide significant benefits for end-user experience while maintaining solid throughput performance. VMware SD- WAN™ can automatically remediate packet loss and deliver a high-quality experience even under adverse network conditions. Dual WAN link deployments further enhance the user experience by providing even greater flexibility, robustness and higher aggregate throughput.

VMware and Intel commissioned Tolly to benchmark the quality-of-service and remediation capabilities of VMware SD-WAN to improve the end-user experience for voice-over-IP (VoIP) and work-from-anywhere (WFA) applications using a dual WAN link deployment. Additionally, Tolly benchmarked the packet throughput of VMware SD-WAN.

VMware SD-WAN delivered good voice quality (MOS of 4.31) across dual links even with packet loss levels of 15% and latency of 150ms with 20ms jitter. See Figures 1 & 2 respectively. VMware SD-WAN demonstrated automatic optimization for dual WAN links by automatically preferring the low latency link. File download of a 250MB video from Microsoft OneDrive illustrated that download continued after one link was failed. With both links active, a single download session was able to use bandwidth from both links for a faster download. Additionally, VMware SD-WAN Edge 640 delivered 3Gbps of device throughput across a pair of devices.




Apposite Technologies Document number: 221140
Apposite Netropy Traffic Generation L2-L3 & L4-L7 Ease of Use and Features
Release date: 04 November 2021

Benchmarking is essential if companies are going to deliver on service level agreements (SLA). Validating the traffic handling characteristics of networks, applications and security solutions is the only way to deliver services with confidence. Traffic generators have been available for quite some time. In many cases, though, the complexity and expense of traffic generators has kept them out of all but the larger enterprise labs. Apposite seeks to redefine the traffic generator with its Netropy series. Netropy Traffic Generation delivers a wide range of L2-L3 and L4-L7 functions all using a browser-based GUI and at a price point attractive even to smaller companies.

Apposite Technologies commissioned Tolly to evaluate the ease-of-use features of its Netropy Traffic Generation solution, specifically the Netropy TrafficEngine -one of four applications available. Testing was conducted using the Netropy 10G2 platform that provided four ports capable of 10GbE. In addition to confirming wire throughput capabilities, Tolly exercised a range of throughput testing scenarios.

Tolly tests confirmed that the Netropy 10G2 can send and receive zero-loss, wire-speed 10GbE traffic on both sets of ports simultaneously. Tolly engineers were able to configure both lower-level and upper-level protocol tests without any product training using the browser based GUI.




New H3C Technologies Co., Ltd Document number: 221142
H3C SD-WAN Solution H3C AD-WAN (Application-Driven SD-WAN) Functionality Validation
Release date: 02 November 2021

With the development of cloud computing, mobile Internet and other application modes, and the change of traffic pattern, the network needs to actively "adapt" to the demands of application traffic, and change as needed. The H3C Application-Driven SD- WAN (AD-WAN) solution can improve the remote user application experience, as fast and as secure as accessing local applications. It can meet various networking needs such as enterprise self-built and internet service provider-operated SD-WAN, and support all-scenario connectivity between headquarters, branches, data centers, and the cloud, as well as large-scale flexible networking and ISP POP networking.

H3C commissioned Tolly to evaluate the AD-WAN solution. The results show that the H3C AD-WAN solution can not only implement traditional SD-WAN functions, but also expand and extend SD-WAN.




Spirent Communications Document number: 221151
Tolly on Technology - Leveraging Low-Latency 5G and Cloud Services - Conversation with Spirent
Release date: 26 October 2021

Tolly Group Founder, Kevin Tolly hosts a Video Podcast discussing Leveraging Low-Latency 5G and Cloud Services with Spirent Communications.

The Tolly on Technology Video Podcast series is a thought leadership podcast that interview experts on trends and best practices for IT leaders.

Kevin is joined by Spirent's Ken Rosenberg discusses the challenges of benchmarking 5G+Cloud Compute environments. Find out Spirent's perspective on Leveraging Low-Latency 5G and Cloud Services and what unique challenges it offers users.

Tolly on Technology Landing Page.
(Note: There is no download for this Video Podcast item.)




SonicWall Document number: 221139
SonicWall NSsp 13700 Next-Generation Firewall (NGFW) Strategic Analysis vs. Palo Alto Networks PA-5250
Release date: 19 October 2021

Remote working and cloud computing have had a major impact on the way that enterprises conduct business. Universally, the shift to distributed work has put a focus on both security and connectivity. Or, in the context of this report, next- generation firewalls and SD-WANs. Given the array of services available, and the cost differential across vendors, it is prudent to analyze offerings before proceeding with an NGFW acquisition. SonicWall has designed its NSsp 13700 to excel in protection of large enterprises while maintaining an industry-leading price- performance ratio.

SonicWall commissioned Tolly to review the published specifications of its NSsp 13700 and compare it to Palo Alto Networks’ PA-5250. Analysis was mainly focused on evaluating key total cost of ownership (TCO) metrics of the comparable solutions.

The Tolly analysis shows that SonicWall has a dramatically lower 3-year TCO for high-availability (HA) deployments compared to Palo Alto Networks while providing comparable or better security feature sets and performance characteristics. The TCO is enhanced by SonicWall’s High Availability licensing policy allowing a single firewall license to be used for both the primary and secondary (HA) appliance.




Huawei Technologies, Co.Ltd Document number: 221145ZH
华为 CloudEngine S5736-S 系列交换机 性能测试和功能验证
Release date: 04 October 2021

华为 CloudEngine S5736-S 系列交换机包括 CloudEngine S5736-S 系列多速率交换机和 CloudEngine S5736-S 系列全光交换机。

CloudEngine S5736-S 系列全新一代全万兆多速率以太网交换机,提供 24 端口万兆多速率全电款型,上行 4 个 10GE 端口,同时提供一个扩展卡插槽。可广泛应用于大中型企业园区汇聚/接入、小微型数据中心接入等多种应用场景。

CloudEngine S5736-S 系列新一代标准型全光千兆以太网交换机,提供 48 光口款型,上行 4 个 10GE 端口。可广泛应用于城域接入、企业园区接入和汇聚、数据中心接入等多种应用场景。

Tolly 工程师测试了华为 CloudEngine S5736-S 系列交换机的性能并验证了其功能。




Huawei Technologies, Co.Ltd Document number: 221145
Huawei CloudEngine S5736-S Series Switches Performance Evaluation and Feature Validation
Release date: 04 October 2021

Huawei CloudEngine S5736-S series switches include CloudEngine S5736-S Series Multi-GE Switches and CloudEngine S5736-S Series All- Optical Switches.

CloudEngine S5736-S Series Switches standard multi-GE access switches are for the Wi-Fi 6 era, featuring 24 downlink ports, four 10 GE SFP+ uplink ports, and one extended slot. They are designed to work with Wireless Local Area Network (WLAN) Access Points (APs) in campus networks.

CloudEngine S5736-S Series Switches next generation standard all-optical GE switches have 48 downlink optical ports and four 10 GE uplink ports. They can be used as access or aggregation switches on campus networks or as access switches on Metropolitan Area Networks (MANs) or data center networks.

Tolly engineers evaluated Huawei CloudEngine S5736-S series switches’ performance and validated their features.




Huawei Technologies, Co.Ltd Document number: 221144ZH
华为 CloudEngine S5731-S 系列交换机 性能测试和功能验证
Release date: 04 October 2021

华为 CloudEngine S5731-S 系列交换机是标准型千兆交换机,采用华为自研 ARM 架构 CPU 和 Solar 交换芯片,具有全千兆电口接入和固定万兆上行端口。它们适合部署在企业园区接入和汇聚、数据中心接入等多种应用场景。

Tolly 工程师测试了华为 CloudEngine S5731-S 系列交换机的性能并验证了其功能。CloudEngine S5731-S 可以在本地配置,也可运行在云管理模式,通过华为 Agile Controller 进行管理。CloudEngine S5731-S 系列交换机还支持大量的安全功能,并可以和华为 Cybersecurity Intelligent System(CIS)联动实现全网安全协防,进行加密通信分析(ECA)和威胁诱捕。

测试结果适用于所有华为 CloudEngine S5731-S 型号,包括 CloudEngine S5731-S24T4X、CloudEngine S5731-S24P4X、CloudEngine S5731-S48T4X 以及 CloudEngine S5731-S48P4X。




Huawei Technologies, Co.Ltd Document number: 221144
Huawei CloudEngine S5731-S Series Switches Performance Evaluation and Feature Validation
Release date: 04 October 2021

Huawei CloudEngine S5731-S series switches are standard gigabit switches with GbE copper access ports and 10GbE uplink ports. They are ideal to be deployed in the aggregation or access layer of campus networks, and in the access layer of data center networks.

Tolly engineers evaluated Huawei CloudEngine S5731-S series switches’ performance and validated their features. Built upon Huawei’s high performance Versatile Routing Platform (VRP) software, CloudEngine S5731-S series switches can be configured locally or work in cloud- managed mode to be managed by Huawei Agile Controller. CloudEngine S5731-S series switches also support numerous security features and can interoperate with Huawei Cybersecurity Intelligence System (CIS) for Encrypted Communications Analytics (ECA) and network-wide threat deception.




Huawei Technologies, Co.Ltd Document number: 221138Summary
Competitive Test of Huawei Lossless Ethernet Solution for Hyper-converged Data Center Networks Centralized Storage: Huawei Ethernet-based NoF+ vs. Fibre Channel HPC: Huawei Lossless Ethernet vs. InfiniBand
Release date: 25 September 2021

Traditional data centers may be built on multiple fabrics – Ethernet for server interconnection, Fibre channel for storage networks, and InfiniBand for high performance compute clusters.

With new development of Ethernet, such as lossless Ethernet technologies, high performance Ethernet NICs supporting RDMA and MPI, NVMe over RoCE all-flash storage systems, Ethernet-based data center network solutions started to replace Fibre Channel and InfiniBand solutions to provide equivalent or better performance. Furthermore, Ethernet solutions are typically more cost effective. They are supported by more vendors. And 400G Ethernet switches are widely available on market now while current Fibre Channel switches typically only support up to 64G FC ports. For all these reasons, modern data centers are moving from the multi-protocol mode towards converged all-Ethernet.

Based on the considerations above, Huawei launched the hyper-converged data center networks CloudFabric 3.0 solution. It consists of two components: Huawei CloudEngine data center switches, and Huawei iMaster NCE-Fabric, an intelligent network management and control system.

Huawei commissioned Tolly to evaluate Huawei’s lossless Ethernet data center network solution, and compare it with Fibre Channel in storage area network test cases and with InfiniBand in high performance computing scenarios.




Huawei Technologies, Co.Ltd Document number: 221138ZHSummary
华为超融合数据中心网络无损以太网络方案对比测试 集中式存储:华为 NoF+ 方案 vs. FC 方案 vs. H3C 无损以太网方案 HPC:华为无损以太网络方案 vs. InfiniBand 方案
Release date: 25 September 2021

传统数据中心可能建立在多个架构(Fabric)上,包括用于服务器互连的以太网、用于存储网络的光纤通道(Fibre Channel - FC)以及用于高性能计算集群的 InfiniBand(IB)。

随着以太网的新发展,例如无损以太网技术、支持 RDMA 和 MPI 的高性能以太网网卡、以及 NVMe over RoCE 全闪存存储系统等,基于以太网的数据中心网络解决方案开始取代光纤通道和 InfiniBand 解决方案,并提供等效或更好的表现。此外,以太网解决方案通常更具成本效益,并得到了更多供应商的支持。现在市场上已广泛存在 400G 以太网交换机,而当前的光纤通道交换机通常最多只支持 64G FC 端口。 由于这些原因,现代数据中心正在从多协议模式转向融合全以太网。

基于以上考虑,华为技术有限公司推出了超融合数据中心网络 CloudFabric 3.0 解决方案。它由两部分组成:华为 CloudEngine 数据中心交换机和华为智能网络管控系统 iMaster NCE-Fabric。

华为委托 Tolly 对华为基于以太网的数据中心网络解决方案进行测试,并在存储测试用例中与光纤通道解决方案进行性能对比,以及在高性能计算场景中与 InfiniBand 方案进行性能对比。




Huawei Technologies, Co.Ltd Document number: 221138ZH
华为超融合数据中心网络无损以太网络方案对比测试 集中式存储:华为 NoF+ 方案 vs. FC 方案 vs. H3C 无损以太网方案 HPC:华为无损以太网络方案 vs. InfiniBand 方案
Release date: 25 September 2021

传统数据中心可能建立在多个架构(Fabric)上,包括用于服务器互连的以太网、用于存储网络的光纤通道(Fibre Channel - FC)以及用于高性能计算集群的 InfiniBand(IB)。

随着以太网的新发展,例如无损以太网技术、支持 RDMA 和 MPI 的高性能以太网网卡、以及 NVMe over RoCE 全闪存存储系统等,基于以太网的数据中心网络解决方案开始取代光纤通道和 InfiniBand 解决方案,并提供等效或更好的表现。此外,以太网解决方案通常更具成本效益,并得到了更多供应商的支持。现在市场上已广泛存在 400G 以太网交换机,而当前的光纤通道交换机通常最多只支持 64G FC 端口。 由于这些原因,现代数据中心正在从多协议模式转向融合全以太网。

基于以上考虑,华为技术有限公司推出了超融合数据中心网络 CloudFabric 3.0 解决方案。它由两部分组成:华为 CloudEngine 数据中心交换机和华为智能网络管控系统 iMaster NCE-Fabric。

华为委托 Tolly 对华为基于以太网的数据中心网络解决方案进行测试,并在存储测试用例中与光纤通道解决方案进行性能对比,以及在高性能计算场景中与 InfiniBand 方案进行性能对比.




Huawei Technologies, Co.Ltd Document number: 221138
Competitive Test of Huawei Lossless Ethernet Solution for Hyper-converged Data Center Networks Centralized Storage: Huawei Ethernet-based NoF+ vs. Fibre Channel HPC: Huawei Lossless Ethernet vs. InfiniBand
Release date: 25 September 2021

Traditional data centers may be built on multiple fabrics – Ethernet for server interconnection, Fibre channel for storage networks, and InfiniBand for high performance compute clusters.

With new development of Ethernet, such as lossless Ethernet technologies, high performance Ethernet NICs supporting RDMA and MPI, NVMe over RoCE all-flash storage systems, Ethernet-based data center network solutions started to replace Fibre Channel and InfiniBand solutions to provide equivalent or better performance. Furthermore, Ethernet solutions are typically more cost effective. They are supported by more vendors. And 400G Ethernet switches are widely available on market now while current Fibre Channel switches typically only support up to 64G FC ports. For all these reasons, modern data centers are moving from the multi-protocol mode towards converged all-Ethernet.

Based on the considerations above, Huawei launched the hyper-converged data center networks CloudFabric 3.0 solution. It consists of two components: Huawei CloudEngine data center switches, and Huawei iMaster NCE-Fabric, an intelligent network management and control system.

Huawei commissioned Tolly to evaluate Huawei’s lossless Ethernet data center network solution, and compare it with Fibre Channel in storage area network test cases and with InfiniBand in high performance computing scenarios.




Dell Technologies Document number: 221129
Dell EMC CloudIQ for PowerEdge Server Infrastructure Insights
Release date: 22 September 2021

Managing datacenters can be very challenging. The rise of the internet of things (IoT) has driven growth in distributed edge computing and many system admins support systems from remote locations or home on a regular basis. Real intelligence is needed to manage systems effectively.

With these needs in mind, Dell Technologies several years ago introduced Dell EMC CloudIQ, a cloud-native application that combines machine intelligence and human intelligence, provides IT administrators with a single-pane-of-glass view of their storage systems globally. Dell EMC CloudIQ has now been extended to include Dell EMC PowerEdge Server systems.

Dell Technologies commissioned Tolly to validate the “Time to Value” and “Time to Insight” of CloudIQ in conjunction with managing Dell PowerEdge servers. Tolly found a rapid time to value of under 30 minutes and time to insight that is 3X faster for an average Dell EMC customer.




Keysight Technologies Document number: 221146
Tolly on Technology Video Podcast: Benchmarking Cloud Environments - Conversation with Keysight
Release date: 20 September 2021

Tolly Group Founder Kevin Tolly hosts a Video Podcast discussing Benchmarking Cloud Environments with Keysight Technologies.

The Tolly on Technology Video Podcast series is a thought leadership podcast that interview experts on trends and best practices for IT leaders.

Kevin is joined by Keysight Technologies, Amritam Putatunda, Technical Product Manager to discuss the challenges of benchmarking cloud environments. Find out Keysight Technologies perspective on Benchmarking Cloud Environments and what unique challenges it offers users.

Tolly on Technology Landing Page.
(Note: There is no download for this Video Podcast item.)




Citrix Systems, Inc. Document number: 221100KO
Citrix ADC VPX F5 BIG-IP Virtual Edition과 성능 비교
Release date: 08 September 2021

ADC(Application Delivery Controller)는 인터넷의 보이지 않는 일꾼입니다. 둘 이상의 서버로 구성된 모든 백엔드 애플리케이션은 프런트 엔드에 특정 유형의 ADC를 가지고 있어 사용자에게 전달을 최적화합니다. 
Citrix ADC VPX(구명칭 NetScaler)는 지연 시간이 짧은 고성능 트래픽 관리 기능을 제공하여 최종 사용자 환경을 개선하도록 설계되었습니다.

Citrix Systems 는 Tolly 에게 Citrix ADC VPX 가상 어플라이언스의 성능을
벤치마킹하고 F5 BIG-IP VE(Virtual Edition)와 비교하도록 의뢰했습니다.
테스트는 Amazon Web Services(AWS) 환경에서 실행되었으며 다양한 암호화 프로토콜을 사용하는 ADC 및 웹 애플리케이션 방화벽(WAF) 기능에 대한 테스트가 포함되었습니다. 테스트는 응답성, 트랜잭션 처리량 및 궁극적으로 최종 사용자 경험의 지표로 지연 시간을 측정하는 데 중점을 두었습니다. 테스트는 흐름의 최악의 1%의 지연 시간을 측정하는 P99 지연 시간을 측정했습니다.

Citrix ADC VPX 는 모든 테스트 시나리오에서 F5 BIG-IP VE 보다 성능이 
우수했으며, 비슷한 처리량 수준으로 벤치마킹할 때 낮은(더 좋은 성능)) 지연 시간과 낮은 CPU 사용률을 보였습니다. 그림 1 을 참조하십시오.




Huawei Technologies, Co.Ltd Document number: 221133
Huawei AirEngine Series Wi-Fi 6 Access Points Performance Evaluation and Feature Validation
Release date: 02 September 2021

Wi-Fi 6 is the latest Wi-Fi standard and is widely used by new mobile devices. Compared with earlier standards, Wi-Fi 6 access points (APs) provide higher bandwidth, a larger number of concurrent users, and lower latency.

Tolly verified some of the features of Huawei's latest mid-range Wi-Fi 6 indoor APs with smart antennas: AirEngine 6761-21T, AirEngine 5761-21, AirEngine 5761-11, AirEngine 5761-12W, AirEngine 5761-11W, and AirEngine 5761-11WD.

Tolly engineers also tested the performance of Huawei Wi-Fi 6 APs and compared their performance with those of Aruba, Cisco, H3C, and Ruijie Wi-Fi 6 APs in the same environment. Performance test results show that Huawei AirEngine indoor APs provide excellent performance and outperform counterpart products of other vendors in the areas tested.




NVIDIA Corporation Document number: 221125
NVIDIA Spectrum-3 vs Switch Built with Commodity Silicon 100/200/400GbE Performance Evaluation Evaluating Consistency & Predictability
Release date: 26 August 2021

Public/private cloud computing is the modern infrastructure model for delivering business-critical applications. The network fabric is the foundation of the modern data center. The network bandwidth of each tenant needs to be protected so that they are not negatively impacted by a “noisy tenant” sharing the same network infrastructure. Applications are built assuming that the network will deliver consistent and predictable performance with fair sharing of resources. Not all switches meet these basic requirements.

NVIDIA commissioned Tolly to benchmark the performance, fairness, and tenant isolation capabilities of its NVIDIA Spectrum-3 12.8Tbps switch and compared that to a switch built with a 12.8Tbps Commodity Silicon.




Broadcom Inc. Document number: 221134
Lenovo ThinkSystem Emulex LPe36002 Host Bus Adapter 64G Fibre Channel Performance vs. 32G and 16G HBA Modes
Release date: 24 August 2021

Data storage and retrieval is growing at an extraordinary pace. Analysts note that data grew world-wide by 4x in recent years and that database data warehousing will double in the next five years. Business Intelligence (BI) and Artificial Intelligence (AI) can dramatically increase data retrieval rates.

Next-generation Fibre Channel alone and in conjunction with next-generation PCIe 4.0 technology can provide dramatic improvements in application data throughput. Lenovo’s ThinkSystem Emulex Gen 7 LPe36002 64G Fibre Channel (64G) dual-port host bus adapter (HBA) leverages both 64G and PCIe 4.0.

Broadcom commissioned Tolly to benchmark the performance of the new adapter in terms of raw performance and compared to existing 32G Fibre Channel (32G) HBAs. The adapter demonstrated dual-port performance in excess of 11 million IOPS. Running relational database transaction benchmarks, the adapter effectively doubled performance by dramatically cutting run times. Similarly, the adapter leveraged PCIe 4.0 technology increasing application throughput to ~12,000MBPS compared to ~6,600MBPS maximum achieved on PCIe 3.0 servers.




Xena Networks Document number: 221114
Xena Networks Chimera Network Impairment Emulation Feature & Topology Support Evaluation
Release date: 23 August 2021

Unified Communication as a Service, UCaaS for short, has taken the world by storm of late. The Covid crisis necessitated remote working and remote working fueled UCaaS adoption. Assuring the quality of end-user experience (EUE) for VoIP and other UCaaS applications is an essential requirement. Understanding how VoIP and other apps behave under varying network conditions is a prerequisite for providing good EUE. Xena Networks Chimera provides sophisticated network impairment emulation to accomplish that task.

Xena Networks commissioned Tolly to evaluate the network impairment capabilities and topology support of Chimera. Chimera is available as a compact, 2-port standalone unit or as a module for greater port scaling in the Xena ValkyrieBay chassis.

Tolly tests confirmed the availability of a broad range of impairments that include drop, misordering, duplication, policing, shaping and others. Chimera also demonstrated flow corruption capabilities of IPv4 and also of the frame check sequence (FCS). Importantly, Chimera demonstrated transparent, line- rate traffic forwarding at 10, 25, 40, 50 & 100GbE link speeds.




SonicWall Document number: 221130
SonicWall NSsp 15700 Next-Generation Firewall (NGFW) Strategic Analysis vs. Fortinet FortiGate 3600E
Release date: 18 August 2021

The term firewall has been part of the IT lexicon for over two decades. Today’s threats are radically more sophisticated than in the past and today’s next-generation firewalls have came a long way compared to the original invention - and all NGFWs are not the same. It is important to focus on the security features and performance needs that are most important for your environment. SonicWall has designed its NSsp 15700 to excel in protection of enterprises, service providers, government agencies, and MSSPs, while maintaining an industry-leading price-performance ratio.

SonicWall commissioned Tolly to review the published specifications of its NSsp 15700 and compare it to Fortinet’s FortiGate 3600E. Analysis was mainly focused on evaluating key total cost of ownership (TCO) metrics of the comparable solutions.

The Tolly analysis shows that the SonicWall has a dramatically lower 3-year TCO for HA deployments compared to Fortinet while providing comparable or better security feature set and performance characteristics. The TCO is enhanced by SonicWall’s High Availability (HA) licensing policy allowing a single firewall license to be used for both the primary and backup (HA) appliance.




Huawei Technologies, Co.Ltd Document number: 221137ZH
数据中心自动驾驶网络 华为 CloudFabric 测试报告
Release date: 17 August 2021

Tolly 本次就数据中⼼⽹络⾃动驾驶华为 CloudFabric 在四个维度进⾏了评估:

Day0 规划建设:规划建设环节,包括基于业务意图系统⾃动理解,并转化为⽹络需求,⽣成⽅案,同时完成部署验证等物理⽹络就绪的能⼒。

Day1 业务部署:业务部署环节,包括意图理解,事前仿真,配置⾃动化,事后检验等业务部署时⽹络配置效率及准确性能⼒。

Day2 运维监控:运维监控环节,包括意图监控,问题定界定位、⽅案⽣成、评估决策、⽅案实施、业务验证等智能运维能⼒。

DayN 变更优化:变更优化环节,对⽹络变更或优化的意图进⾏⽅案设计、评估决策,⽅案实施和验证等,以提⾼⽹络运⾏效率的能⼒。

本测试报告对数据中⼼⾃动驾驶⽹络华为 CloudFabric 进行测试和分析,并从 Day0 规划建设、Day1 业务部署、Day2 监控排障、DayN 变更优化四个⽅⾯进⾏阐述。




Viavi Solutions, Inc. Document number: 221131
VIAVI Solutions Observer Benefits of Executive Dashboards - Case Study
Release date: 30 July 2021

Ultimately, an organization’s IT infrastructure exists to provide business value through optimal service delivery. Network session data often holds the key to the source of IT service problems. However, massive collections of packets are of little use without sophisticated analytics to make sense of it all. Furthermore, in complex organizations there are always subsets of information and systems that benefit from custom, unified management.

VIAVI Observer provides customizable executive dashboards that provide for purpose-built monitoring and management of essential resources. Built around VIAVI’s End-User Experience (EUE) Score, these dashboards provide single pane of glass visibility for key resources combined with rapid problem isolation capabilities.

VIAVI commissioned Tolly to review the executive dashboard capabilities in a practical manner via a case study approach. The case study represents an actual large hospital environment. Without benefit of a solution like Observer, problem isolation could be very time consuming and require cross-team support. Tolly found that using Observer’s executive dashboard capability and EUE AI-based scoring, that problem isolation could be completed quickly and by a single Observer operator.




Huawei Technologies, Co.Ltd Document number: 221126
Campus Network Endpoint Profiling Huawei iMaster NCE-Campus vs. Cisco ISE
Release date: 21 July 2021

Endpoint profiling is a core capability required by enterprises to build a "zero-trust" endpoint security system responsible for network access control, permission control, and asset management. With the advancement of the Internet of Things (IoT), enterprises' information and communications technology (ICT) applications are becoming IP-based and masses of endpoints are attempting to access enterprise campus networks. These endpoints vary in type, upgrade frequently, and differ hugely across industries. In addition, it is hard to obtain data samples of endpoints in certain industries. All of this makes it difficult to identify and manage endpoints in different industries. Facing this challenge, enterprises of all sizes are finding ways to apply AI technologies on their campus networks to build a powerful endpoint profiling rule engine that can maximize the endpoint profiling scope and precision.

Huawei commissioned Tolly to independently test the capabilities of Huawei iMaster NCE-Campus and Cisco Identity Services Engine (ISE) in profiling multiple types of endpoints in different industries. The test areas include:

1. Endpoint profiling accuracy: Tolly engineers tested how accurately the two systems can identify the types, vendors, and models of various endpoints (such as IP phones, IP cameras, mobile phones, network devices, and office devices).

2. Unknown endpoint profiling and automated expansion of the endpoint profiling fingerprint library: Tolly engineers tested how well the two systems profile unknown endpoints based on their respective fingerprint libraries.

3. Customized endpoint profiling report display: Tolly engineers compared the endpoint profiling reports generated by the two systems to determine which one can collect and display endpoint statistics more comprehensively.




Huawei Technologies, Co.Ltd Document number: 221105ZH
数据中心自动驾驶网络 华为 CloudFabric 与 H3C AD-DC 对比测试报告
Release date: 20 July 2021

Tolly 本次就数据中⼼⽹络⾃动驾驶华为 CloudFabric 与 H3C AD-DC 在四个维度进⾏了评估:

Day0 规划建设:规划建设环节,包括基于业务意图系统⾃动理解,并转化为⽹络需求,⽣成⽅案,同时完成部署验证等物理⽹络就绪的能⼒。

Day1 业务部署:业务部署环节,包括意图理解,事前仿真,配置⾃动化,事后检验等业务部署时⽹络配置效率及准确性能⼒。 Day2 运维监控:运维监控环节,包括意图监控,问题定界定位、⽅案⽣成、评估决策、⽅案实施、业务验证等智能运维能⼒。

DayN 变更优化:变更优化环节,对⽹络变更或优化的意图进⾏⽅案设计、评估决策,⽅案实施和验证等,以提⾼⽹络运⾏效率的能⼒。




Huawei Technologies, Co.Ltd Document number: 221126ZH
园区网络终端识别能力 华为 iMaster NCE-Campus 与 Cisco ISE 对比测试报告
Release date: 24 June 2021

终端识别作为企业建立终端零信任安全体系(终端准入、权限管控、资产管理等)的基础,其识别能力和精度具有至关重要的意义。随着物联网的不断普及与应用,企业内部 ICT 应用呈现出 IP 化趋势,海量终端接入企业园区网络。这些终端具有种类繁多、更新换代快、⾏业差异⼤等特点,且对于敏感⾏业,其终端数据样本也难以获取。这都给终端识别和管理带来了极大的挑战。因此企业园区网络迫切需要引入 AI 技术、构建强大的终端识别规则引擎来满足企业对终端识别广度和精度的要求。

华为技术有限公司委托 Tolly 独立测试评估华为云园区网络 CloudCampus 3.0 解决方案 iMaster NCE-Campus 园区自动驾驶网络管理控制系统(以下简称华为 iMaster NCE-Campus)与 Cisco Identity Services Engine (ISE) 针对不同行业中多类型终端的识别能力。




Opale Systems Document number: 221120
MultiDSLA Voice Quality Test System Voice Quality Testing with Network Impairments
Release date: 08 June 2021

Unified Communication as a Service, UCaaS for short, has taken the world by storm of late. The Covid crisis necessitated remote working and remote working fueled UCaaS adoption. Assuring the quality of end-user experience (EUE) for VoIP and other UCaaS applications is an essential requirement. Understanding how VoIP apps behave under varying network conditions is a prerequisite for providing good EUE. Opale Systems’ MultiDSLA Voice Quality Test System provides voice generation and measurement capabilities.

Opale Systems commissioned Tolly to evaluate the voice quality testing capabilities of its MultiDSLA solution.

Tolly tests confirmed the need for benchmarking UCaaS/VoIP applications showing that different VoIP solutions respond differently to network impairments such as packet loss and deliver different levels of EUE.

Tolly found the Opale Systems MultiDSLA easy to set up and run. The system automatically analyzed voice quality by comparing the original audio with the received audio and produced empirical voice quality scores.

For more information on the Opale Systems MultiDSLA please visit Teraquant's Web site.





Broadcom Inc. Document number: 221123
Emulex® LPe36002 Host Bus Adapter, 64G Fibre Channel HBA Performance vs. 32G
Release date: 11 May 2021

Data storage and retrieval is growing at an extraordinary pace. Analysts note that data grew world-wide by 4x in recent years and that database data warehousing will double in the next five years. Business Intelligence (BI) and Artificial Intelligence (AI) can dramatically increase data retrieval rates.

Next-generation Fibre Channel alone and in conjunction with next-generation PCIe 4.0 technology can provide dramatic improvements in application data throughput. Broadcom’s Emulex Gen 7 LPe36002 64G Fibre Channel (64G) dual- port host bus adapter (HBA) leverages both 64G and PCIe 4.0.

Broadcom commissioned Tolly to benchmark the performance of its new adapter in terms of raw performance and compared to existing 32G Fibre Channel (32G) HBAs. The adapter demonstrated dual-port performance in excess of 11 million IOPS. Running relational database transaction benchmarks, the adapter effectively doubled performance by dramatically cutting run times. Similarly, the adapter leveraged PCIe 4.0 technology increasing application throughput to ~12,000MBPS compared to ~6,600MBPS maximum achieved on PCIe 3.0 servers.




Huawei Technologies, Co.Ltd Document number: 221122ZH
华为 CloudEngine S5735S-L 系列交换机 性能测试和功能验证
Release date: 31 March 2021

华为 CloudEngine S5735S-L 系列交换机是精简型千兆接入交换机,具有 GE 接入端口以及 GE 或 10GE 上行端口。它们适合部署在企业园区接入、千兆到桌面等多种应用场景。

Tolly 工程师测试了华为 CloudEngine S5735S-L 系列交换机的性能并验证了其功能。基于华为的高性能 VRP 软件平台,CloudEngine S5735S-L 系列交换机支持灵活的以太组网,提供多样的安全控制,并且支持基础的 IPv4 和 IPv6 三层路由协议。




Huawei Technologies, Co.Ltd Document number: 221121ZH
华为 CloudEngine S5735-L 系列交换机 性能测试和功能验证
Release date: 31 March 2021

华为 CloudEngine S5735-L 系列交换机是精简型千兆接入交换机,具有 GE 接入端口以及 GE 或 10GE 上行端口。它们适合部署在企业园区接入、千兆到桌面等多种应用场景。

Tolly 工程师测试了华为 CloudEngine S5735-L 系列交换机的性能并验证了其功能。基于华为的高性能 VRP 软件平台,CloudEngine S5735-L 系列交换机支持灵活的以太组网,提供多样的安全控制,并且支持基础的 IPv4 和 IPv6 三层路由协议。




Huawei Technologies, Co.Ltd Document number: 221121
Huawei CloudEngine S5735-L Series Switches Performance Evaluation and Feature Validation
Release date: 31 March 2021

Huawei CloudEngine S5735-L series switches are simplified gigabit switches with GbE access ports and GbE or 10GbE uplink ports. They are ideal to be deployed in the access layer of campus networks.

Tolly engineers evaluated Huawei CloudEngine S5735-L series switches’ performance and validated their features. Built upon Huawei’s high performance Versatile Routing Platform (VRP) software, CloudEngine S5735-L series switches support flexible Ethernet networking, various security features, and basic IPv4 / IPv6 Layer 3 routing protocols.




Keysight Technologies Document number: 221118
Keysight Vision X Network Packet Broker Performance & Features
Release date: 26 March 2021

Network visibility is essential to network security and performance management. In practical terms, visibility relies heavily on a packet broker’s ability to process tapped traffic reliably and intelligently. All tapped traffic from switched port analyzer (SPAN) and test access point (TAP) ports must have duplicate packets eliminated and be delivered without loss to network security tools. Another key to complete visibility is application intelligence. Network packet brokers (NPB) are required to detect, report, filter and present application metadata to security monitoring tools in a reliable and consistent way.

Keysight commissioned Tolly to report on the advanced packet processing capability (deduplication, in particular) and the application intelligence of its Vision X network packet broker.

Throughout the testing described, Keysight Vision X demonstrated high packet processing of 2Tbps system-wide combined with no-drop packet deduplication and rapid, accurate detection of applications.




Dell Technologies Document number: 221119
Infographic: Dell EMC PowerEdge XE7100 Storage Server
Release date: 08 March 2021

Dell Technologies commissioned Tolly to study the specifications and characteristics of its Dell EMC PowerEdge XE7100 and provide analysis and context as to where the device sits in terms of advancing the industry.

View the Infographic on the key takeaways from the Dell EMC PowerEdge XE7100 analysis.

The full Tolly report can be downloaded from here.




SonicWall Document number: 221102
SonicWall NSa 2700 Next-Generation Firewall (NGFW) Strategic Analysis vs. Fortinet FortiGate 100F NGFW
Release date: 04 March 2021

The term firewall has been part of the IT lexicon for over two decades. Today’s threats are radically more sophisticated than in the past and today’s next- generation firewalls have came a long way compared to the original invention - and all NGFWs are not the same. It is important to focus on the security features and performance needs that are most important for your environment. SonicWall has designed its NSa 2700 to excel in protection while maintaining an industry- leading price-performance ratio.

SonicWall commissioned Tolly to review the published specifications of its NSa 2700 and compare it to Fortinet’s FortiGate 100F. Analysis was mainly focused on evaluating key total cost of ownership (TCO) metrics of the comparable solutions.

The Tolly analysis shows that the SonicWall has a dramatically lower 3-year TCO for HA deployments compared to Fortinet while providing comparable or better security feature set and performance characteristics. The TCO is enhanced by SonicWall’s High Availability (HA) licensing policy allowing a single firewall license to be used for both the primary and backup (HA) appliance.




GreatHorn, Inc. Document number: 221107
GreatHorn Cloud Email Security Platform vs. Proofpoint Essentials
Release date: 01 March 2021

Threat actors are always seeking new methods to access and compromise corporate data. Email is a preferred path into organizations for many hackers as virtually every organization depends upon email and, almost universally, email is a communications path between outside and inside. As companies strive to erect barriers to hackers, the hackers strive to find new and often very sophisticated ways to overcome those barriers and access corporate data.

GreatHorn’s Cloud Email Security Platform is a purpose-built security solution focused on meeting the demands of today’s cloud- based email offerings. As organizations have shifted to cloud email platforms (Google Workspace, formerly G-Suite and Microsoft 365), and phishing attacks have evolved to include zero-day attacks and increasingly complex social engineering tactics, a multi-layered approach to more effectively respond to risk across each stage of the entire email lifecycle is required.

GreatHorn commissioned Tolly to evaluate the email security feature set and capabilities of its solution and compare that with capabilities of Proofpoint. Specifically, this comparison was made with Proofpoint Essentials.




Citrix Systems, Inc. Document number: 221117
Webinar: Citrix ADC Performance Comparison
Release date: 25 February 2021

Application Delivery Controllers (ADCs) are the unseen workhorses of the internet. Every back-end application that consists of two or more servers has an ADC of some type on the front end to optimize delivery to the user. Citrix ADC VPX (formerly NetScaler) is designed to enhance end-user experience by providing high performance traffic management coupled with low latency.

Join Jason Poole, Product Marketing at Citrix and Kevin Tolly, Founder of The Tolly Group, for a webinar covering the recent benchmark testing that showed performance advantage of Citrix ADC and what Citrix ADC means for your applications delivery choices.

Citrix Webinar View On-Demand Landing Page.

The Tolly report can be downloaded from here.




Cambium Networks, Ltd. Document number: 221116
Webinar: Cambium Networks Wi-Fi 6 Access Points TCO and Performance Comparison
Release date: 19 February 2021

Wi-Fi 6 brings WLAN performance to a new level with aggregate throughput potential measured in Gigabits per second. The advent of Wi-Fi 6 also brings economics to a new level as higher performance and higher density impact price-performance and total cost of ownership (TCO) calculations.

Join Bruce Miller, VP of Enterprise Marketing at Cambium Networks and Kevin Tolly, Founder of The Tolly Group, for a webinar covering the recent benchmark testing that showed Wi-Fi 6 wireless access points from Cambium Networks outperformed competitors Aruba, Meraki (Cisco) and Ruckus (Commscope) in terms of performance and Total Cost of Ownership (TCO).

Cambium Networks Webinar View On-Demand Landing Page.

The Tolly report can be downloaded from here.




Dell Technologies Document number: 221111
Dell EMC Networking Data Center Switches Z9332F-ON & Z9432F-ON 400GbE Performance Evaluation
Release date: 16 February 2021

Data center architects require high-capacity, high-performance switches at the core of the network to provide the best application performance for users. Dell EMC offers a complete set of data center purpose built Layer 2 and Layer 3 switches that not only provide 400GbE interfaces but are also part of the open networking innovation supporting multiple Network Operating Systems (NOS) options.

Dell EMC commissioned Tolly to benchmark the overall performance, latency and power consumption of two of its data center switches: the Z9332F-ON (32x 400GbE + 2x 10GbE) and the Z9432F-ON (32x 400GbE + 2x 10GbE).

The Dell EMC switches delivered 100% line-rate RFC2544 Layer 2 and Layer 3 throughput with 32x 400GbE ports in full mesh topology at frame sizes of 512- bytes and greater. Tolly engineers also benchmarked Layer 2/Layer 3 latency, power consumption as well as MAC/FIB/ARP/ACL/IGMP/PIM table capacities.




Cambium Networks, Ltd. Document number: 221110
Cambium Networks Wi-Fi 6 Access Points TCO and Performance vs. Aruba, Meraki and Ruckus
Release date: 10 February 2021

Wi-Fi 6 brings WLAN performance to a new level with aggregate throughput potential measured in Gigabits per second. The advent of Wi-Fi 6 also brings economics to a new level as higher performance and higher density impact price-performance and total cost of ownership (TCO) calculations.

Cambium Networks commissioned Tolly to benchmark the performance of its tri-radio XV3-8 (8x8/4x4 MU-MIMO) and XV2-2 (2x2 MU-MIMO) Wi-Fi 6 Access Points (APs) and comparable APs from Aruba, Meraki (Cisco) and Ruckus (CommScope). Performance tests encompassed low-client density and high-client density scenarios with various traffic types. Tolly Engineers also calculated price-performance and 5-year TCO. Cambium acquired Xirrus in 2019 and the current products combine technology from both companies.

The Cambium Networks XV3-8 price-performance was more than twice that of the nearest competitor while offering TCO that is an average of 38% lower than Aruba, Meraki and Ruckus. The XV3-8 averaged 47% higher bidirectional performance (in tri- radio mode) vs. competitive 8x8/4x4 APs and outperformed Meraki and Ruckus in the 100-client high-density test.

The Cambium Networks XV2-2 price-performance was twice that of the nearest competitor while offering TCO 35% lower than Aruba, 49% lower than Meraki and 44% lower than Ruckus. The XV2-2 outperformed the others in bidirectional and downstream throughput and delivered comparable or better performance in the 50-client high- density test.




VMware, Inc. Document number: 221115
Webinar: VMware Work From Home
Release date: 09 February 2021

Tolly benchmarked the quality of service and remediation capabilities of VMware SD-WAN to improve the end user experience for VoIP and work from home (WFH) applications.

Join Shefali Chinni, Senior Product Marketing Manager at VMware and Kevin Tolly, Founder of The Tolly Group, for a webinar covering the quality-of-service and remediation capabilities of VMware SD-WAN to improve the end-user experience for voice- over-IP (VoIP) and work-from-home (WFH) applications.

VMware Webinar View On-demand Landing Page.

The Tolly report can be downloaded from here.




Aukua Systems Inc Document number: 221113
Aukua MGA2510 Ethernet Test Platform Network Impairment Emulation Testing
Release date: 08 February 2021

Unified Communication as a Service, UCaaS for short, has taken the world by storm of late. The Covid crisis necessitated remote working and remote working fueled UCaaS adoption. Assuring the quality of end-user experience (EUE) for VoIP and other UCaaS applications is an essential requirement. Understanding how VoIP apps behave under varying network conditions is a prerequisite for providing good EUE. Aukua System’s MGA2510 Ethernet Test Platform provides a network impairment capability to do just that.

Aukua Systems commissioned Tolly to evaluate the network impairment capabilities and ease-of-use of its MGA2510 Ethernet Test Platform configured as a network impairment generator. The MGA2510 is FPGA-based and can be reconfigured with a few clicks to serve as an inline network capture/analysis device or a network traffic generator.

Tolly tests confirmed the need for benchmarking UCaaS/VoIP applications showing that different VoIP solutions respond differently to network impairments such as packet loss and deliver different levels of EUE. See Figure 1. Tolly found the Aukua MGA2510 simple to set up, configure and run. Tests also showed the ease of which the platform can be reconfigured to serve as a network capture device.




Dell Technologies Document number: 221109
Dell EMC PowerEdge XE7100 Storage Server Scalability, Efficiency & Flexibility Analysis
Release date: 04 February 2021

Nothing seems to grow faster than the need for storage. Few in the world of IT would argue otherwise. With IoT, image, and video objects feeding “big data” analytics systems, that growth curve can be steep as new systems are placed online. Dell Technologies has engineered its latest storage server with a focus on high-density, high-capacity storage customers.

Dell Technologies commissioned Tolly to study the specifications and characteristics of its Dell EMC PowerEdge XE7100 and provide analysis and context as to where the device sits in terms of advancing the industry.

Tolly found that the Dell EMC PowerEdge XE7100 Storage Server provides significant scalability and efficiency benefits both compared to prior generation systems and compared to current generation systems of a leading competitor. Significantly, the unit provides module server sled options that allow for high-performance optimization for specific application focus areas such storage/archival, intelligent video analysis (IVA), and media streaming.




Spirion, LLC. Document number: 221104
Spirion Data Discovery Accuracy Evaluation
Release date: 02 February 2021

Knowing the location of personal and sensitive personal data is essential for organizations of all sizes. You cannot secure what you cannot locate. The elements and composition of personal can vary dramatically across applications. One size certainly does not fit all. Business, government, legal, and medical environments have different and overlapping definitions of what is personal data. Thus, it is critically important to be able to customize any data discovery solution. Spirion provides for easy and flexible tuning to enable accurate definition of personal data.

Spirion commissioned Tolly to evaluate its data discovery and protection solution. The evaluation included a proof-of-concept (PoC) accuracy demonstration, installation, setup and filtering configuration.

Testing illustrated that Spirion could deliver 98.5% accuracy in a 200 document corpus with filtering configuration and tuning accomplished in under four hours.




Viavi Solutions, Inc. Document number: 221103DE
VIAVI Solutions Observer Scoring und Analyse des Endnutzererlebnisses
Release date: 27 January 2021

Letzten Endes soll die IT-Infrastruktur eines Unternehmens durch eine optimale Bereitstellung von Diensten einen Geschäftswert erwirtschaften. Häufig sind es die Daten einer Netzwerksitzung, die wichtige Informationen zur Ursache von Störungen an den IT-Diensten beinhalten. Wenig hilfreich ist allerdings die massive Aufzeichnung von Paketen ohne eine anschließende systematische Analyse. Die von VIAVI ermöglichte Bewertung (Scoring) des Endnutzererlebnisses (End-User Experience, EUE) schöpft das Leistungspotenzial der Netzwerkdaten in vollem Umfang aus. Im Rahmen der Verarbeitung dieser Daten auf Grundlage des maschinellen Lernens (ML) werden auf jede Netzwerkkonversation 30 gängige sowie von VIAVI entwickelte kritische Leistungsindikatoren (KPI) angewendet.

VIAVI hat Tolly beauftragt, seine EUE-Scoring-Lösung Observer zu testen und deren Leistungsmerkmale und Funktionen zu dokumentieren. Tolly hat die Lösung untersucht und die wichtigsten Eigenschaften bestätigt. Dabei wurde geprüft, wie die aufgezeichneten Daten genutzt werden können, um Leistungsstörungen umgehend zu erkennen und einzugrenzen. Über die intuitive Benutzeroberfläche kann eine marginale oder kritische (schlechte) Leistung schnell erkannt werden. Anschließend wurde eine detailliertere Analyse mit automatisch generierten Karten zur Darstellung (Mapping) von Abhängigkeiten zwischen den Anwendungen durchgeführt und die Störung genau auf eine spezifische Datenbank-Komponente des Servers eingegrenzt. Abbildung 1 zeigt ein Beispiel einer Abhängigkeitskarte.




Viavi Solutions, Inc. Document number: 221103
VIAVI Solutions Observer End-User Experience Score & Analysis
Release date: 27 January 2021

Ultimately, an organization’s IT infrastructure exists to provide business value through optimal service delivery. Network session data often holds the key to the source of IT service problems. However, massive collections of packets are of little use without sophisticated analytics to make sense of it all. VIAVI End-User Experience (EUE) Score analysis harnesses the power of your network data. Machine learning-based processing then applies 30 well-known and VIAVI developed KPIs for every network conversation on your network.

VIAVI commissioned Tolly to review its Observer End-User Experience Score solution and document key features and functionality. Tolly explored the solution and confirmed key capabilities. Tolly reviewed how actual data could be used to recognize and isolate a performance problem quickly. Using the intuitive interface, marginal or critical (poor) performance could be spotted quickly. Then, drill-down navigation and auto-generated dependency maps were used to isolate the problem down to a specific server’s database component.




SlashNext, Inc. Document number: 221108
SlashNext Phishing Protection Accuracy vs. Leading Email Security Providers
Release date: 22 January 2021

Phishing attacks are not new but the sophistication of the attacks and, thus, their effectiveness is always growing. Instead of mass attacks many bad actors use targeted, “spear phishing” attacks. SlashNext has developed next-gen AI phishing threat intelligence and protection to help combat this omnipresent threat.

SlashNext commissioned Tolly to benchmark the accuracy of its phishing protection solution and compare that with several leading vendors of email security solutions. Tests were conducted with Microsoft Defender (Safe Links) and Proofpoint Essentials Business (URL Defense). An informal test was also run against the advanced threat protection offering, including their URL rewriting feature, from another market leading provider (Vendor X). The test corpus consisted of zero-hour phishing URLs submitted to the PhishTank community site as well as those discovered by SlashNext’s Threat Labs.

SlashNext delivered significantly greater accuracy on the zero-hour PhishTank URLs than the other solutions. Significantly, the other solutions could only identify 21% to 49% of the recent and zero-hour phishing URLs selected by Tolly from the SlashNext AI Phishing Detection Database.




Cambium Networks, Ltd. Document number: 221106
Cambium Networks Wi-Fi 6 Access Points TCO and Performance vs. Aruba and Meraki
Release date: 15 January 2021

Wi-Fi 6 brings WLAN performance to a new level with aggregate throughput potential measured in Gigabits per second. The advent of Wi-Fi 6 also brings economics to a new level as higher performance and higher density impact price-performance and total cost of ownership (TCO) calculations.

Cambium Networks commissioned Tolly to benchmark the performance of its tri-radio XV3-8 (8x8/4x4 MIMO) and XV2-2 (2x2 MIMO) Wi-Fi 6 Access Points (APs) and comparable APs from Aruba and Meraki (Cisco). Performance tests encompassed low- client density and high-client density scenarios with various traffic types. Tolly Engineers also calculated price-performance and 5-year TCO. Cambium acquired Xirrus Wi-Fi Networks in 2019 and the Wi-Fi 6 products tested are based on converged technology from both companies.

The Cambium Networks XV3-8 price-performance was more than twice that of the nearest competitor while offering TCO that is 40% lower than Aruba and Meraki. The XV3-8 averaged 52% higher bidirectional performance (in tri-radio mode) vs. competitive 8x8 APs and outperformed Meraki in the 100-client high-density test.

The Cambium Networks XV2-2 TCO was 48% lower than Meraki and 35% lower than Aruba. The XV2-2 outperformed the others in bidirectional and downstream throughput and delivered comparable or better performance in the 50-client high-density test.