A Management Support Tool Using Haskell and LDAP

1
A Management Support Tool UsingHaskell and LDAP

• Chris Ryder
• Simon Thompson
• Dominic Steinitz

2
What am I going to talk about?

• Project undertaken in summer 2000 with BA
• Uses Haskell to connect to an LDAP directory
• In use within BAs Information Security dept.
• Update on whats happened since

3
Whats LDAP ?

• Lightweight Directory Access Protocol
• A directory is a lightweight database optimised
  for reading
• BA has an LDAP directory containing entries for
  every employee
• System Authentication
• On-Line telephone directories
• Pay-roll
• etc

4
Whats LDAP ?

• Entries have attributes with values
• Entries must have a unique distinguished name
  (DN)
• uid1234,oupeople,obaplc.com
• Entries in BAs directory also have
• common name (CN)
• user ID (UID)
• employee number (employeeNumber)
• Entries may also have a manager attribute which
  contains the DN of the persons manager

5
The Project

• Chameleon gives each manager a score
• BA wanted a tool to calculate aggregate scores
  for managers based on their subordinates scores
• Wanted to be able to drill-down through the
  results
• Wanted to experiment with Haskell for this task

6
The Project

• The program has to
• Build a model of the management hierarchy
• Insert scores information from Chameleon
• Calculate aggregate scores
• Display the results
• The management structure is contained within the
  LDAP directory as back-pointers
• Chameleon outputs a text file of UID / score pairs

7
Haskell LDAP

• Two options to connect Haskell to LDAP
• Talk LDAP via sockets (complicated)
• Use a library (simple)
• LDAP libraries available for C, Java, Perl
• but not Haskell
• Tried Java library using Lambada
• Settled on C library using HaskellDirect

8
Haskell LDAP

• This is what we ended up with
• Unfortunately its slow lots of copying of
  strings

9
Haskell LDAP
query
attributes we want back
ldapSearch uid cn, dn, uid
Haskell Stub
HDirect Glue Code
String gt char String gt char
C stub
OpenLDAP C Library
10
Haskell LDAP

• Retrieving results is even more work
• do
• err lt- ldapPrimGetNextResult
• dn lt- ldapPrimGetDN
• (err,name,value) lt- ldapPrimGetFirstAttr
• (err,name,value) lt- ldapPrimGetNextAttr
• -- Repeat last line until an error occurs
• Constrained to C API
• Very inefficient to do lots of small queries

11
Building the management tree

• Each entry in the LDAP directory has a manager
  attribute
• Pointer to the entries manager (using DN)
• May be empty
• May point to a non-existent manager
• Need to build the tree in a bottom-up manner
• Need to be able to find a person from their DN

12
Output of Chameleon

• Chameleon outputs scores as comma separated
  fields
• Outputs User ID / score pairs (plus a few bits of
  misc. info that we ignore)
• May be more than one score per person
• Easy to parse scores, but need to work out which
  person they relate to from their User ID

13
How it was implemented

• Iguana was written using a layered structure

14
How it was implemented

• People are represented using this data structure
• data Person
• Person -- Subs Scores Agg Score
• LDAPEntry Int Float (Maybe Float)
• data LDAPEntry
• LDAPEntry
• String -- DN
• String -- manager DN (or "")
• String -- CN
• String -- UID

15
How it was implemented

• The program
• Reads in every entry from the LDAP directory into
  an IOArray (mutable array)
• Builds a hash table mapping DNs to indices
• Creates the tree by adding subordinates indices
  into their managers data structure
• Re-builds the hash table mapping UIDs to indices
• Read the scores file into the array
• Walk over the tree calculating the aggregate
  scores
• Output the results

16
So far so good Whats the problem?

• Memory usage !
• What uses the memory ?
• Strings Haskell strings seem quite inefficient
• The hash tables
• Haskell lists
• Laziness
• Why are these a problem ?
• approx. 100,000 entries in the LDAP directory

17
Strings

• Haskell strings take significantly more space
  than C strings (obvious)
• The program needs to store DN, CN, UID and
  Manager attributes
• CN and Manager are typically 60-70 characters
• Used MD5 checksums of CN and Manager
• 128 Bits (represented as Hex string)
• Small chance of two DNs hashing to same value

18
Hash tables

• The hash tables are needed because
• LDAP lookups are slow (using our LDAP binding)
• Linear searches through the array are slow
• Balanced trees (handwritten and FiniteMap) used
  large amounts of memory
• Use an IOArray to build a simple hash table using
  chaining to handle collisions
• relatively quick lookups with reduced memory
  usage

19
Lists

• Lists appear to have large memory overhead
• Lists are quite inefficient as the get larger
• Found we needed to use mutable arrays for the
  large lists we needed
• Makes the program look quite imperative

20
Laziness

• Laziness proved to be more of a hindrance than
  help
• Few places where it more than doubled memory
  usage (sometimes much more)
• Most other places it didnt have much effect
• Difficult to determine whether laziness was
  useful or not without trial-and-error

21
Conclusions

• HDirect / FFI is useful for boundaries of
  Haskell
• Can be forced into an imperative style
• Can be slow
• Understanding memory usage is not clear
• Profilers can help
• No documentation on interpreting results
• Large data sets may force an imperative approach

22
Update

• Dan Russell, Chris Reade, Phil Molyneux, David
  Martland and Barry Avery at Kingston Business
  School and Dominic Steinitz at BA are working on
  a pure Haskell LDAP library.
• Tested a partially complete version with Iguana
• Missing a few features Wildcard () searches
• Memory usage is higher (approx 50 higher)