Untangle

1
Antivirus Fight Club!
August 8th _at_ LinuxWorld
2
Before we begin
Please submit any viruses for the test at
http//virus.untangle.com/
3
Background - who we are
Untangle provides an open source network gateway
platform.
We are not an antivirus companyWe are not a
testing company
4
Background - why we are doing this

• 2005 Untangle researches antivirus to add to
  the network gateway platform
• after testing we choose clam (open source) and
  one other vendor
• 2006 Untangle seeks Testing Labs for
  certification (stickers!)
• 2006 Testing Lab refused to test AV product,
  because use of open source
• wont tell us why
• wont provide test results
• wont provide test set

Something fishy is going on here
5
What is the AV FightClub?
A simple test of real-world anti-virus detection
by different AV engines

• What AV FightClub is not
• Zero-day test
• Functionality comparison
• Not coverage testing

Two important things!

• Open - for samples participation discussion
• Transparent - simple, verify run at home

6
The Test
Each vendor is subjected to

• Small Set of test viruses (eicar)
• Set of in-the-wild viruses
• Set of user-submitted viruses (minus
  non-viruses, not in-the-wild viruses, and
  phish)

Scored by of viruses identified and performance
if applicable
All vendors should catch all these viruses
7
The Vendors
Vendors
Engines with linux support (clam, kasperskey,
fprot, sophos, globalhauri)Gateway Appliances
(sonicwall, fortinet, watchguard)Windows
solutions (norton/symance, mcafee)
8
Questions?
predictions?
9
Lets get started

• zip up the test set for windows tests
• deposit on web server for gateway appliance tests

10
F-Prot
Vendor
Version 4.6.8
Updated 2007-08-08
Method Linux Client
11
Sophos
Vendor
Version 4.20.0
Updated 2007-08-08
Method Linux Client
12
GlobalHauri
Vendor
Version SDK 4.0. engine 2007-08-07
Updated 2007-08-08
Method Linux Client
13
Kasperksy
Vendor
Version Kav4fs 5.5.27
Updated 2007-08-08
Method Linux Client
14
Norton/Symantec
Vendor
Version Norton Antivirus 2007
Updated 2007-08-08
Method Windows Client
15
McAfee
Vendor
Version 7.2.147
Updated 2007-08-08
Method Windows Client
16
Sonicwall
Vendor
Version Sonicwall 1260 (3.2.0.5-54e)
Updated 2007-08-08
Method Gateway Appliance
17
Fortinet
Vendor
Version Fortinet 50A (2.8.0-520)
Updated 2007-08-08
Method Gateway Appliance
18
Watchguard
Vendor
Version Watchguard x20e (8.5.1-8138)
Updated 2007-08-08
Method Gateway Appliance
19
Clam
Vendor
Version 0.91-1-1ubuntu3
Updated 2007-08-08
Method Linux Client
20
Results
21
Results 2
22
Conclusions
conclusions

• Open Source solution (Clam) doesnt suck . In
  fact, its excellent!
• Many vendors are poor. Some are selling dead
  donkeys!

outstanding questions

• Why hasnt this been pointed out?
• Is there something wrong with the way we test
  antivirus today?

23
Thanks for coming!
Contact
Dirk Morris dmorris_at_untangle.com
Remember
Dont believe me? Try this at home. The test set
will be available on http//virus.untangle.com (pa
ssword on zip file is a)