Honeycomb Automated IDS Signature Generation using Honeypots

1
Honeycomb Automated IDS SignatureGeneration
using Honeypots

• Christian Kreibich
• Jon Crowcroft

2
Motivation

• Wed like to characterize suspicious traffic
• IDS signatures are a way to do this
• How to focus on relevant traffic? (Evil Bit ?)
• Honeypots have no production value
• Their traffic is suspicious by definition
• Thus look for patterns in honeypot traffic

3
Honeycomb

• Name? Nice double meaning ...

4
Honeycomb

• Name? Nice double meaning ...
• Combing for patterns in honeypot traffic

5
Honeycombs Architecture
6
Honeycombs Algorithm
7
Pattern Detection (I)

• Stream reassembly

8
Pattern Detection (II)

• Longest-common-substring (LCS) on pairs of
  messages
• m1 fetaramasalatapatata
• m2 insalataramoussaka
• Can be done in O(m1 m2) using suffix trees
• Implemented libstree, generic suffix tree library
• No hardcoded protocol-specific knowledge

9
Pattern Detection (II)

• Longest-common-substring (LCS) on pairs of
  messages
• m1 fetaramasalatapatata
• m2 insalataramoussaka
• Can be done in O(m1 m2) using suffix trees
• Implemented libstree, generic suffix tree library
• No hardcoded protocol-specific knowledge

10
Pattern Detection (III)

• Horizontal detection
• LCS on pairs of messages
• each message independent
• e.g. (persistent) HTTP

11
Pattern Detection (IV)

• Vertical detection
• concatenates incoming messages
• LCS on pairs of strings
• for interactive flows and to mask TCP dynamics
• e.g. FTP, Telnet, ...

12
Signature Pool

• Limited-size queue of current signatures
• Relational operators on signatures
• sig1 sig2 all elements equal
• sig1 ? sig2 elements differ
• sig1 ? sig2 sig1 contains subset of sig2s
  facts
• signew sigpool signew ignored
• signew ? sigpool signew added
• signew ? sigpool signew added
• sigpool ? signew signew augments sigpool
• Aggregation on destination ports

13
Results

• We ran Honeycomb on an unfiltered cable modem
  connection for three days
• Honeyd setup
• fake FTP, Telnet, SMTP, HTTP services, all
  Perl/Shell scripts.
• Other ports traffic sinks
• Some statistics
• 649 TCP connections, 123 UDP connections
• Full traffic volume 1MB
• approx. 30 signatures created
• No wide-range portscanning

14
TCP Connections
HTTP
Kuang2 Virus/Trojan
SMB
NetBIOS
Microsoft SQL Server
15
UDP Connections
NetBIOS
Messenger Service
Slammer
16
Signatures created Slammer

• Honeyd log
• 2003-05-08-022643.0385 udp(17) S 81.89.64.111
  2943 192.168.169.2 14342003-05-08-022743.0404
  udp(17) E 81.89.64.111 2943 192.168.169.2 1434
  376 02003-05-08-095838.0807 udp(17) S
  216.164.19.162 1639 192.168.169.2
  14342003-05-08-095938.0813 udp(17) E
  216.164.19.162 1639 192.168.169.2 1434 376
  02003-05-08-171524.0072 udp(17) S
  66.28.200.226 6745 192.168.169.2
  14342003-05-08-171624.0083 udp(17) E
  66.28.200.226 6745 192.168.169.2 1434 376 0
• Signature
• alert udp any any -gt 192.168.169.2/32 1434 (msg
  "Honeycomb Thu May 8 09h58m38 2003 " content
  "04 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
  01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
  01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
  01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
  01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
  01 01 01 01 01 01 01 01 01 01 01 01 01 01 01 01
  01 DC C9 B0BEB 0E 01 01 01 01 01 01
  01pAEB01pAEB90 90 90 90 90 90 90 90hDC
  C9 B0BB8 01 01 01 011C9 B1 18PE2 FD501 01
  01 05P89E5Qh.dllhel32hkernQhounthickChGetTfB9
  llQh32.dhws2 fB9etQhsockfB9toQhsendBE 18 10
  AEB8DED4PFF 16P8DEE0P8DEF0PFF
  16PBE 10 10 AEB8B 1E 8B 03U8B ECQt05 BE
  1C 10 AEBFF 16 FF D01C9QQP81 F1 03 01 04 9B
  81 F1 01 01 01 01Q8DECCP8BEC0PFF
  16j11j02j02 FF D0P8DEC4P8BEC0PFF
  16 89 C6 09 DB 81 F3ltaD9 FF 8BEB4 8D 0C_at_8D
  14 88 C1 E2 04 01 C2 C1 E2 08)C2 8D 04 90 01 D8
  89EB4j10 8DEB0P1C9Qf81
  F1x01Q8DE03P8BEACPFF D6 EB" )
• Full worm detected

17
Signatures created CodeRedII

• Hit more than a dozen times
• alert tcp 80.0.0.0/8 any -gt 192.168.169.2/32 80
  (msg "Honeycomb Tue May 6 11h55m20 2003 "
  flags A flow established content "GET
  /default.ida?XXXXXXXXXXXXXXXXXXXXX
• XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
  XXXu9090u6858ucbd3u7801u9090u6858ucbd3u780
  1u9090u6858ucbd3u7801u9090u9090u8190u00c3
  u0003u8b00u531bu53ffu0078u0000u00a
  HTTP/1.00D 0AContent-type text/xml0AContent-l
  ength 3379 0D 0A 0D 0A C8 C8 01 00E8 03 00
  00 00 CC EB FEdgFF600 00dg8900 00 E8 DF
  02 00 00h04 01 00 00 8D 85\FE FF FFPFFU9C
  8D 85\FE FF FFPFFU98 8B_at_10 8B 08 89
  8DXFE FF FF FFUE404 04 00 00 0F 94 C104
  08 00 00 0F 94 C5 0A CD 0F B6 C9 89 8DTFE FF FF
  8Bu08 8109A 02 00 00 0F 84 C4 00 00 00
  C7F09A 02 00 00 E8 0A 00 00 00CodeRedII00 8B
  1CFFUD8f0B C0 0F 95 858FE FF FF C7
  85PFE FF FF 01 00 00 00j00 8D 85PFE FF
  FFP8D 858FE FF FFP8BE08 FFp08 FF 90 84
  00 00 00 80 BD8FE FF FF 01thSFFUD4 FFUEC
  01E84iBDTFE FF FF,01 00 00 81 C7,01 00
  00 E8 D2 04 00 00 F7 D0 0F AF C7
  89F48DE88Pj00 FFu08 E8 05 00 00 00 E9 01
  FF FF FFj00j00 FFUF0PFFUD0OuD2
  E805 00 00iBDTFE FF FF 00\05 81 C7
  00\05WFFUE8j00j16 FFU8CjFF FFUE8
  EB F9 8BF4)E84jdFFUE8 8D 85ltFE FF
  FFPFFUC0 0F B7 85ltFE FF FF88 88 00
  00sCF 0F B7 85gtFE FF FF 83 F8 0AsC3fC7
  85pFF FF FF 02 00fC7 85rFF FF
• Full worm due to vertical detection server
  replies before all signature-relevant packets seen

18
Signatures detected others

• alert tcp 64.201.104.2/32 any -gt 192.168.169.2/32
  1080,3128,4588,6588,8080 (msg "Honeycomb Mon May
  5 19h04m12 2003 " flags S flow stateless )

19
Signatures detected others

• alert tcp 64.201.104.2/32 any -gt 192.168.169.2/32
  1080,3128,4588,6588,8080 (msg "Honeycomb Mon May
  5 19h04m12 2003 " flags S flow stateless )
• alert udp 81.152.239.141/32 any -gt
  192.168.169.2/32 135 (msg "Honeycomb Thu May 8
  12h57m51 2003 " content "15 00 00 00 00 00 00
  00 15 00 00 00YOUR EXTRA PAYCHEQUE00 E1 04x0C
  00 00 00 00 00 00 00 0C 00 00 0080.4.124.4100
  01 00 00 00 00 00 0001 00 00 Amazing
  Internet Product Sells Itself!0D 0AResellers
  Wanted!
  GO TO.....
  
  
  www.Now4U2.co.uk" )

20
Signatures detected others

• alert tcp 64.201.104.2/32 any -gt 192.168.169.2/32
  1080,3128,4588,6588,8080 (msg "Honeycomb Mon May
  5 19h04m12 2003 " flags S flow stateless )
• alert udp 81.152.239.141/32 any -gt
  192.168.169.2/32 135 (msg "Honeycomb Thu May 8
  12h57m51 2003 " content "15 00 00 00 00 00 00
  00 15 00 00 00YOUR EXTRA PAYCHEQUE00 E1 04x0C
  00 00 00 00 00 00 00 0C 00 00 0080.4.124.4100
  01 00 00 00 00 00 0001 00 00 Amazing
  Internet Product Sells Itself!0D 0AResellers
  Wanted!
  GO TO.....
  
  
  www.Now4U2.co.uk" )
• alert tcp 80.4.218.53/32 any -gt 192.168.169.2/32
  80 (msg "Honeycomb Thu May 8 07h27m33 2003 "
  flags PA flow established content "GET
  /scripts/root.exe?/cdir HTTP/1.00D 0AHost
  www0D 0AConnnection close0D 0A 0D" )

21
Signature Usability

• LCS blindly calculates longest substring
• alert tcp any any -gt 81.100.86.44/32 445 (msg
  "Honeycomb Fri Jul 18 02h40m22 2003 " flags PA
  flow established content "00 00 00 85
  FFSMBr00 00 00 00 18SC8 00 00 00 00 00 00 00
  00 00 00 00 00 00 00 FF FE 00 00 00 00 00b00
  02PC NETWORK PROGRAM 1.000 02LANMAN1.000
  02Windows for Workgroups 3.1a00 02LM1.2X00200
  02LANMAN2.100 02NT LM 0.12" )
• Generated signatures not necessary useful for
  everyday use

22
Signature Usability (II)

• But this distraction can be interesting
• alert tcp 62.219.50.70/32 any -gt
  147.237.72.91/32 80 (msg "Honeycomb Sun Nov 9
  19h03m09 2003 " flags A flow established
  content "F45dYN1pL3zRApBOj2WCKnO2hiH9UgFzTlLwkFg0
  OPehaFKCk1gYadTVTcrsHbcz5Gd4qg.94xMs7cRE0ivx8.GVNN
  3YK1yCn8AU8WnuJtrcsEyTtwrH2ivX.w5UvBFGTN8y56ISLjiD
  eCBxjQVfdZGRllRB9jOG5m70m9keYyNsW2g51WiGzsOY2MCkaw
  AoxAMFsh3rwRLVBtqGLGiXsm9SIrsEF23jQ6nbJM3knX6AbQqf
  qMBEMxApEgnWqK4xq0ZmmRaWj84uNmyTD3ZBg1KUkXUaAlBEnt
  zhFJIhpWfDaWefyBBf4WsBFzfCO.YFBHIzam2N9GrJhwSHc7vo
  wkdGXXWuvdpqHJowhbLG6KvHZVjoFkUXqwOaTTK22z0osT9cAR
  .mRBXmrtCwe5wViX9EWaGHgocWqviXkBbvYZuns5IrXQv28kBD
  m4oMoWl7JLvzZ-Wd-18qj.jztV mDPNc0FHsv2N4U4qczZzBss
  fp6S.8W0Azj9R1wLkjpP Xjr9r8ZOmE7Jyq1-MET-2gW9ETIe
  tlqd39CjftUnszxCDDAZnsXZeuT1C3xDwefCHI344MF
  K45Fi4GrZRKHJWUkJkKW622tnCAqR3zRF.MxBrkNcfeVcDkv2f
  OE0PF8AUCfiewxcA4x1mu3niSnlx1T-hRcb0l1Q983X8ANPFI8
  H4vM-TQ vhMkHsvN0nxsUrh9xBm.YZL6Nc300YNle4DGK
  FNz.8HIQ9ID8mRIGJSGzcPHaq7EXAo67nnkHWw58d4udtwsbrr
  7NN48v5zjKtBlpklHTTcqjYsKsVWDhqEzDqFMrplBvgHfnjtKU
  IsBQsLIKgEAu9vXH5tWu3ef4nPT.7Tz9i8pb3DyZBMyqAf6TkY
  G5z.UUeZP5BrTTc2XFOY1xfRieOzb.5qgE1GyXMojMNWZqTuZK
  MWVzW8ZMNXx3ARaxpNCD-LB8oWxCtruMqb-mOuxR2NkMfZMFnL
  sIouUzQtGZ8RsY2NJEz." )

23
Summary

• System detects patterns in network traffic
• Using honeypot traffic, the system creates useful
  signatures
• Good at worm detection
• Todo list
• Ability to control LCS algorithm (whitelisting?)
• Tests with higher traffic volume
• Experiment with approximate matching
• Better signature reporting scheme

24
Thanks!

• Shoutouts a13x hØ
• No machines were harmed or compromised in the
  making of this presentation.
• www.cl.cam.ac.uk/cpk25/honeycomb/