Checkvir Realtime Anti-Malware Testing and Certification

1
Checkvir Realtime Anti-Malware Testing and
Certification

• Dr. Ferenc Leitold, Veszprog Ltd.
• fleitold_at_veszprog.hu
• www.checkvir.com

2
Contents

• Purpose of Checkvir testing
• Testing methodology
• Technical background
• Testing procedures
• Current state
• Difficulties
• Questions

3
Purpose of Checkvir testing

• Problems
• Big number of updates
• Cloud technology
• Solutions are continually changing
• Testing all versions are impossible

Number of updates / day
AVG 1,7
ESET 2,6
F-PROT 1,2
F-Secure 5
Kaspersky 23,2
McAfee 35,4
Panda 44,7
Sophos 5,4
Sunbelt 0,6
Symantec 233,4
VirusBuster 1
source AV-Test.org
4
Purpose of Checkvir testing

• Testing all versions are impossible
• Executes tests as frequently as possible
• Automatic methods have to be developed
• Big number of computers have to be used

5
Purpose of Checkvir testing

• The main purposes
• Provide reliable, correct and exact
  information mainly about
• effectiveness
• performance
• in a balanced way
• (AMTSOs principle)
• Provide naming cross-reference information

performance
effectiveness
6
Testing methodology
update
test
Unpack previous image
Unpack last image
Initialize testing
AV update
Execute test(s)
no
New version?
Save results and reports
yes
Pack and save the new image
Analyze results
Publish results
7
Testing methodologyTechnical background
firewall
malware proxy server
webserver
controller
firewall router
archiver
clients
8
Testing methodologyTesting procedures

• Malware knowledge (detection, disinfection)
• against known, unknown malware and clean files
• on-demand, on-access and proactive executions
• Container checking capabilities
• archives, email clients data files,
• Speed
• on-demand, on-access
• boot time
• Functionality
• Stability

speed
knowledge
9
Testing methodologyTesting procedures
Why the speed is so important?
10
Testing methodologyTesting procedures
11
Testing methodologyTesting procedures

• Testing bootup time
• What is more important?
• BOOTUP TIME or SECURE BOOTING
• DEMO

12
Testing methodologyTesting procedures
13
Testing methodologyTesting procedures
Bootup protection test
Avast AVG Avira Bitdefender Eset
e-Trust F-Prot F-Secure
Fortinet Ikarus Kaspersky Microsoft Rising Sophos
Symantec Trend Micro VirusBuster
14
Testing methodologyTesting procedures
Bootup protection test
Avast AVG Avira Bitdefender Eset
e-Trust F-Prot F-Secure
Fortinet Ikarus Kaspersky Microsoft Rising Sophos
Symantec Trend Micro VirusBuster
15
Testing methodologyTesting procedures
Bootup protection test
Avast AVG Avira Bitdefender Eset
e-Trust F-Prot F-Secure
Fortinet Ikarus Kaspersky Microsoft Rising Sophos
Symantec Trend Micro VirusBuster
16
Testing methodologyTesting procedures
Bootup protection test
Avast AVG Avira Bitdefender Eset
e-Trust F-Prot F-Secure
Fortinet Ikarus Kaspersky Microsoft Rising Sophos
Symantec Trend Micro VirusBuster
17
Testing methodologyProactive tests vs. AM cloud
technology

• Problems
• AM products use cloud technology
• gt traffic should be allowed
• Malware use cloud technology
• gt traffic should be allowed
• gt How can we protect the world?
• gt How can we provide exactly the same environment
  for solutions?

18
Testing methodologyProactive tests vs. AM cloud
technology
firewall
malware proxy server
webserver
controller
firewall router
archiver
clients
19
Testing methodologySettings

• By default, DEFAULT settings are used
• Minimal functionality is required
• Execute tests without user interaction
• Automatically clean the infected file
• (if not possible -gt delete)
• Report file generation

20
Current state

• What is working now?
• The frame system
• The website
• Automatic procedures of some products
• Preliminary selection and validation of the
  samples

21
Current state
22
Current state
23
Current state
24
Difficulties

• Viewpoint of the average user
  Automatic methods
• Testing environment
• Funcionality problems
• Truncate report file
• Stability problems

25
Questions