Report Archive for

Click on a report title to go to the download page for that report.

EdgeWave, Inc. EdgeWave ePrism Email Security 2800 Anti-spam Effectiveness and Feature Comparison Versus Solutions from Barracuda, Cisco and Google (2009 Test)

Document number: 212119
Release Date: 31 Jul 2012

In testing conducted in 2009, EdgeWave’s ePrism Email Security 2800 (2800) achieved a higher percentage of spam detection in this February 2009 test than Barracuda and Google offerings, and delivered performance on par with the Cisco IronPort C150, at a fraction of the cost per user mailbox.

The ePrism 2800 also resulted in up to 90% fewer false positives than Barracuda and Google offerings. Finally, it offers a hybrid solution combining appliance technology with hosted services for backup.
Anti-Spam
Red Condor, Inc. Red Condor Message Assurance Gateway 2700: Anti-spam Effectiveness and Feature Comparison versus Google Message Security (Postini)

Document number: 209108
Release Date: 30 May 2009

Red Condor commissioned Tolly to evaluate the anti-spam effectiveness of its Message Assurance Gateway 2700 against Google Message Security (Postini).

Testing focused on the anti-spam effectiveness in terms of spam detection percentage, spam error percentage and false positive rate of the solutions under test during a calendar week of testing per solution, while deployed in the live corporate network of The Tolly Group.

The testing was conducted in accordance with Tolly Common Test Plan #1058, Anti-spam Gateway v1.0.

Click below to download the free test report.
Anti-Spam
Red Condor, Inc. Red Condor Message Assurance Gateway 2700: Anti-spam Effectiveness and Feature Comparison versus Barracuda Spam Firewall 300

Document number: 209109
Release Date: 16 May 2009

Red Condor commissioned Tolly to evaluate the anti-spam effectiveness of its Message Assurance Gateway 2700 against Barracuda Networks’ Spam Firewall 300.

Testing focused on the anti-spam effectiveness in terms of spam detection percentage, spam error percentage and false positive rate of the solutions under test during a calendar week of testing per solution, while deployed in the live corporate network of The Tolly Group.

The testing was conducted in accordance with Tolly Common Test Plan #1058, Anti-spam Gateway v1.0.

Click below to download the free test report.
Anti-Spam
Red Condor, Inc. Red Condor Message Assurance Gateway 2700: Anti-spam Effectiveness and Feature Comparison versus Solutions From Barracuda, Cisco and Google

Document number: 209107
Release Date: 22 Apr 2009

Red Condor commissioned Tolly to evaluate the anti-spam effectiveness of its Message Assurance Gateway 2700 against competing solutions from Cisco Systems’ IronPort C150 Email Security Appliance, Barracuda Networks’ Spam Firewall 300 and Google’s Message Security powered by Postini.

Testing focused on the anti-spam effectiveness in terms of spam detection percentage, spam error percentage and false positive rate of the solutions under test during a calendar week of testing per solution, while deployed in the live corporate network of The Tolly Group.

The testing was conducted in accordance with Tolly Common Test Plan #1058, Anti-spam Gateway v1.0.

Click below to download the free report.
Anti-Spam
Abaca Abaca Technology Corp. Email Protection Gateway™ EPG 1000 with ReceiverNet™ Premium Service Anti-Spam Effectiveness Evaluation

Document number: 208274
Release Date: 02 Sep 2008

Abaca Technology Corporation commissioned The Tolly Group to evaluate the effectiveness of the Abaca Email Protection Gateway™ EPG 1000 with ReceiverNet™ Premium Service E-mail security solution at blocking spam, virus and phishing attacks.

Tolly Group engineers conducted a live evaluation of Abaca’s Email Protection Gateway appliance and ReceiverNet Service with realistic traffic patterns and sufficient duration to emulate real-world performance. Tolly Group engineers measured the solution’s detection rate (the number of false positives and false negatives, as well as the false positive and false negative error rates) to report the effectiveness and accuracy of the Abaca E-mail security solution.

Tests were conducted during June 2008.

Anti-Spam
Fortinet, Inc. FortiMail-100™ Anti-spam Effectiveness and Feature Comparison vs. Barracuda Networks Spam Firewall 200

Document number: 207259
Release Date: 14 Jan 2008

Fortinet, Inc. commissioned The Tolly Group to measure the effectiveness of the company’s FortiMail-100™ multi-layered E-mail security appliance at blocking spam and virus messages.

Tolly Group engineers tested the spam detection effectiveness of the FortiMail-100 against a Barracuda Networks Spam Firewall 200. In accordance with The Tolly Group’s Fair Testing Charter, Barracuda was invited to review the test methodology, offer suggestions for its product and comment on its results.

Engineers measured the percentage of spam blocked, the number of “false positives,” “false negatives” and virus messages detected per product. Engineers also validated a number of E-mail security features and deployment flexibility. Tests were conducted in October and November 2007.

Anti-Spam
Fortinet, Inc. FortiMail-4000A™ Anti-spam Effectiveness and Feature Comparison vs. IronPort Systems C350 E-mail Security Appliance

Document number: 207258
Release Date: 11 Dec 2007

Fortinet, Inc. commissioned TheTolly Group to measure the effectiveness of the company’s FortiMail-4000A™ multi-layered E-mail security appliance at blocking spam and virus messages.

Tolly Group engineers tested the performance of the FortiMail-4000A against an IronPort Systems C350 E-mail Security Appliance. In accordance with The Tolly Group’s Fair Testing Charter, IronPort was invited to review the test methodology, offer suggestions for optimal configuration of its product and comment on its results.

Engineers measured the percentage of spam blocked, the number of “false positives,” “false negatives” and virus messages detected per product. Engineers also validated a number of E-mail security features and deployment flexibility. Tests were conducted in November 2007.

Anti-Spam
Cloudmark, Inc. "Talking Outside the Box:" Podcast Interview with Cloudmark's Jamie De Guerre and Overview of Tolly Up-to-spec Testing

Document number: 207205
Release Date: 12 Feb 2007

This 20-minute podcast focuses on the effectiveness of anti-spam solutions for both Service Provider and Enterprise markets. Tolly Group President/CEO Kevin Tolly provides an overview of a recent Tolly Group Up-to-spec test of Cloudmark’s Authority for Service Providers.

In addition, Charlie Bruno, Executive Editor of The Tolly Group, discusses the ramifications of the test with Cloudmark's Jamie De Guerre. Details of the test can be found in Tolly Group document 206147.

Anti-Spam
Cloudmark, Inc. Cloudmark Authority™ for Service Providers Symantec Brightmail Anti-Spam Version 6.0とのスパム検出率の比較テスト- version in Japanese

Document number: 206147JA
Release Date: 28 Nov 2006

Cloudmark社は、Cloudmark Authority for Service Providersの、スパムブロックおよび誤検出防止における精度を測定するよう The Tolly Groupに委託しました。

Tolly Groupのエンジニアは、Cloudmark Authorityの性能をSymantec Brightmail Anti-Spam Version 6.0 と比較してテストしました。エンジニアは、製品のスパムに対する有効性を相対的に計測するため、テスト機器によってブロックされたスパムの比率、誤検出の比率を測定しました。テストは2006年の7月から8月に行われました。

Anti-Spam
Cloudmark, Inc. Cloudmark Authority for Service Providers Anti-Spam Effectiveness versus Symantec Brightmail Anti-Spam Version 6.0

Document number: 206147
Release Date: 28 Nov 2006

Cloudmark, Inc. commissioned The Tolly Group to measure the effectiveness of the Cloudmark Authority for Service Providers at blocking spam and guarding against false positives.

Tolly Group engineers tested the performance of Cloudmark Authority against the Symantec Brightmail Anti-Spam Version 6.0. Engineers measured the percentage of spam blocked by the tested devices, and the percentage of false positives to determine the relative effectiveness of the products at fighting spam. Testing was conducted in July and August of 2006.
Anti-Spam
Mirapoint, Inc. Tolly Group White Paper Series: Exploring Exchange-Compatible Appliance-Based Messaging

Document number: 206131
Release Date: 11 May 2006

Mirapoint, Inc. commissioned The Tolly Group to build a microcosm of an enterprise E-mail environment and validate specific capabilities of the vendor’s Message Server appliance related to its ability to provide a “drop in” replacement to Microsoft’s Exchange E-mail application.

This hands-on evaluation proved conclusively that Mirapoint’s Message Server appliance can provide a viable alternative to Microsoft’s Windows-based Exchange Server mail transport application providing full functionalitiy to Outlook clients via Mirapoint’s Outlook SynQ technology while offering platform and security advantages over Microsoft Exchange.

This 23-page white paper comes complete with comparison charts that assess the compatibility of Mirapoint’s management, maintenance, integration and E-mail/calendaring features/functions with Microsoft Outlook. Plus accompanying screen shots provide a window into the Mirapoint Message Server’s “look and feel.”

Anti-Spam
Sana Security, Inc. Sana Security Primary Response SafeConnect, Malware Detection and Removal/CPU and Memory Utilization

Document number: 206125
Release Date: 25 Apr 2006

Sana Security, Inc. commissioned The Tolly Group to evaluate its Primary Response SafeConnect, a software program designed for Windows-based operating systems (2000, XP) to detect and remove malicious programs.

Primary Response SafeConnect detected and removed 100% — all 183 — of the malware programs that invaded the host system from visiting 4,280 suspicious Web sites. The test results conclusively demonstrate that the program successfully accomplished its main objective — identifying and removing programs showing signs of suspicious behavior, such as trojans, rootkits, Spyware, adware and more. Engineers also measured the CPU utilization and memory utilization of Primary Response SafeConnect during peak program usage.
Anti-Spam
Wiresoft Net, Inc. Tolly Group White Paper Series: Securing SMB Networks Without Breaking the Bank

Document number: 206113
Release Date: 07 Apr 2006

SMBs need a multipurpose security platform that provides complete security protection. The dilemma that SMBs face today is that the IT market abounds with supplier after supplier that offer point solutions for security.

Wiresoft Net, Inc. commissioned The Tolly Group to evaluate its Wiresoft Sentry Security Suite, a versatile platform that offers a variety of security services including transparent virus scanning, challenge response spam blocking, stateful packet firewalling, VPN services (PPTP and IPSec) and more.

Tolly Group engineers conducted a battery of performance tests on the Sentry Security Suite, such as its effectiveness at blocking spam traffic, the aggregate throughput delivered while operating as a firewall, the aggregate throughput delivered across a VPN connection and the aggregate throughput while scanning Web traffic for viruses. Engineers also evaluated a number of functions, such as set up, hardware reliability and failover protection.

Anti-Spam
VirtualConnect Technologies, Inc. "Road Test" White Paper – Mailprotector: Intelligent Spam and Virus Filtering Service, Leveraging Third-party Filtering Services to Improve Security and Reduce Overhead

Document number: 205104
Release Date: 21 Apr 2005

VirtualConnect Technologies commissioned The Tolly Group, Inc. to conduct a “road test” of its Mailprotector E-mail spam, virus and attack filtering hosted solution. The test took place during January 2005 and included an on-site audit of the hosting infrastructure located in Atlanta, GA. The Tolly Group also deployed and used the VirtualConnect Mailprotector since October 2004.

Tolly Group observations show that Mailprotector usage results in productivity boosts for employees who no longer must grapple with spam, data network managers can expect a reduction in WAN link usage since far less spam should be congesting the pipes, and companies also demonstrate that they are protecting employees from offensive spam or unsolicited porn content, that left unaddressed could expose the organization to potential lawsuits.

Further, the intangible costs for demoing Mailprotector versus other types of inline services (such as a Salesforce.com) are far less, since there is virtually no investment of time or resources to conduct a tight integration with data stores. All that is required to get going is for mail administrators to redirect incoming messages to the VirtualConnect Mailprotector servers.
Anti-Spam
NetIQ Corp. NetIQ MailMarshal Ver.5.5 SMTP Competitive Anti-Spam Filtering Effectiveness versus SurfControl E-mail Filter Ver.4.7

Document number: 204121
Release Date: 12 Jul 2004

NetIQ Corp. commissioned The Tolly Group to examine the effectiveness of its MailMarshal SMTP Ver 5.5, an anti-spam E-mail filtering tool. NetIQ asked The Tolly Group to compare the filtering performance of MailMarshal against SurfControl, another SMTP-based anti-spam filtering tool from the company of the same name.

Tolly Group engineers measured the products’ spam detection rate and gathered evidence of false positives. Tests were conducted in February/March 2004 at The Tolly Group’s Boca Raton, Fl. facilities.

Anti-Spam
Mailblocks, Inc (AOL) Mailblocks Inc. Competitive Evaluation of Anti-Spam Solutions: Challenge Response vs. Filtering

Document number: 203121
Release Date: 16 Oct 2003

Mailblocks commissioned The Tolly Group to evaluate the effectiveness of its Mailblocks Challenge/Response 2.0 anti-spam service against five anti-spam solutions that utilize advanced filtering techniques: Brightmail, Inc.’s Brightmail, Microsoft Corp.’s Hotmail, Network Associates, Inc.’s McAfee SpamKiller, Postini, Inc.’s Postini and Symantec Corp.’s SpamAlert. Unlike the filtering-based anti-spam offerings, Mailblocks Challenge/Response 2.0 utilizes a Challenge/Response architecture.

The Tolly Group set out to measure the effectiveness of the various anti-spam solutions to block a variety of spam types from overrunning user inboxes. Moreover, tests sought to identify the effectiveness of the various solutions tested at allowing legitimate mail messages to be delivered to recipients without any false positives that would block the delivery of legitimate mail.
Anti-Spam
Mailblocks, Inc (AOL) Competitive Evaluation of Spam-Blocking Solutions: The C/R Capabilities of DigiPortal Software’s ChoiceMail, EarthLink spamBlocker, Mailblocks Inc.’s Mailblocks Challenge Response 2.0, Qurb, Inc.’s Qurb and SpamArrest’s SpamArrest

Document number: 203507
Release Date: 20 Sep 2003

This Tolly Group white paper provides a hands-on independent evaluation of the effectiveness of several anti-spam solutions based on Challenge/ Response technology.

The report, “Competitive Evaluation of Spam-blocking E-mail Solutions,” which was commissioned by Mailblocks, Inc., examined five popular offerings that are currently available to consumers: DigiPortal Software’s ChoiceMail; Internet service provider Earthlink’s spamBlocker; Mailblocks’ Challenge/Response 2.0; Qurb, Inc.’s Qurb and SpamArrest from the company of the same name.

Although similarities exist between the various offerings tested, The Tolly Group found that buyers have a range of features and functionality from which to choose when selecting a Challenge/Response product/service that will effectively protect the integrity of their incoming E-mail.

Key features identified in the white paper include: disposable aliases, safe sender status, consolidated E-mail, whitelists and blocklists and customized challenge messages. To learn more about the reports findings, download this free white paper.
Anti-Spam