The Big Data Combat

1
The Big Data Combat

• SPEC INDIA

2
What is it all about?

• The size of the data generated in the world
  explodes
• Data is being constantly gathered by various
  sources
• The Data keeps increasing many folds every day
• Technology gears up to combat BIG DATA

3
Big Data

• A large unstructured big volume data set
• Too complex to be handled by commonly used
  database management systems
• RDBMS
• DBMS
• Big data uses statistical inference to determine
  parameters from a large volume of data
• Regressions
• Nonlinear Relationships
• Data Dependencies

4
Sources of Data Today

• The Internet
• Mobile Devices
• Remote Sensing
• Software Logs
• Cameras
• Microphones
• Radio Frequency Identification (RFID)
• Wireless Sensor Networks

5
The Challenges

• In the Growth Digitization of This Global
  Information Storage

6
Volume

• BIG Volumes
• The unceasing increase in the amount of data
• Created everyday
• Overwhelming in size

7
Velocity

• Velocity _at_ The Speed Of Light
• Speed of Data in and out
• Transactions
• Business Analysis

8
Variety

• Variety Spices up Big Data too
• Data Types
• Data Sources
• Challenges in
• Capture
• Curate
• Store
• Interpretation
• Meaningful Analys
• Search
• Data Visualization

9
Big Data Rollout

• Steps for a mature and meaningful data set
• Data Profiling
• Data Cleansing
• Data Integration of structured and unstructured
  data
• Data Merging
• Data Migration
• Data Replication
• ETL / ELT / ETLT Design and Development
• Interfacing legacy systems with the modern
  approach

10
Big Data Tools

• Hadoop, a distributed file system
• MapReduce, a framework for data abstractions
• Hive for data summarization and adhoc queries
• Pig for parallel processing
• HBase, a structured storage for large tables
• Sqoop for data integration of Hadoop with RDBMS
• Flume for data transfers of log data to
  centralized data repositories

11
It is Big is getting bigger too!

•  
• Visit
• http//www.spec-india.com/services/bi-bigdata-data
  base-services.html
• to request a FREE POC to Test Drive our services