Massive Data Analysis Lab (MassDAL)

1
Massive Data Analysis Lab (MassDAL)
S. Muthukrishnan CS Dept
2
MassDAL

• Agenda Gather, manage and process massive data
  logs----Web, IP/wireless traffic data, location
  trajectories of objects, sensor readings of
  physical world.
• Key Challenges
• Scale Beyond the traditional human scale. Eg.,
  IP data at a single router interface for an hour
  exceeds total yearly worldwide credit card
  transactions!
• Data Collection probes/sensors with associated
  data quality and communication problems.
• Need breakthroughs in Mathematics, Algorithms,
  Systems and Engineering, to meet these
  challenges.
• Potential Major impact in Homeland Security,
  Telecom, Transportation and Society-at-large.

3
State of MassDAL

• Mathematics and Computer Science.
• Algorithmic tools for embedding vectors, strings,
  trees and other objects for compact
  representation.
• Algorithmic tools for analyzing data summaries
  for heavy hitters, deviants, clustering, decision
  trees, etc.
• Invited talks at ACM, SIAM, European conferences
  in Algorithms, Databases, Statistics, and Data
  Mining on novel models and algorithms.
• Over dozen research papers in last 2 years on
  experience with massive data analysis.
• Supported by NSF grants. Partner MIT, DIMACS.

4
State of MassDAL

• Science
• Developing wearable sensors for tracking location
  of objects as well as interactions between
  objects. Measuring behavioral data.
• Current partner Telcordia. Their initial
  investment 300k/3 months (est). Potential
  parter in works Los Alamos National Lab.
• Potential Analysis of social networks for
  Epidemiology and Homeland Security, and health
  industry.

5
State of MassDAL

• Engineering.
• Consulting in analysis of wireless network logs.
  ATT Wireless, 3rd largest in US, 20 Million
  customers. Terabytes/month. Fully operational,
  telco-grade!
• Incorporated novel algorithms in operational IP
  network data analysis tools. Partner Gigascope.
• Developed principled approach to data cleaning
  and data quality monitoring for operational IP
  network. Partner PACMAN.
• Developed new burst-detection algorithms for text
  streams. Partner DIMACS, Monitoring message
  streams.

6
Future

• See http//cs.rutgers.edu/muthu/massdal.html

7
Future of MassDAL

• Research Need breakthrough research in
  mathematics, systems, databases, algorithms,
  sensor networking.
• Expand data domains.
• Potential partners Google, NJ auto insurance
  fraud data, USPTO patent data, AWS location
  trajectories, etc.
• Build state-of-art facility at Rutgers.
• Secure, 24X7, data hosting and analysis
  infrastructure capable of gathering and
  processing petabytes of data/month across
  domains, data sources, etc. Unique in the world!
• Potential.
• Every wireless, telecom, internet service
  provider is looking to farm out this crucial
  piece of their operations. Estimated market for
  these services 100s of millions in US per
  year. Crucial for NJ State. Interest from
  multiple VCs now.