Apache Ignite

1
What Is Apache Ignite ?

• An in memory distributed database
• A cluster based processing platform
• Open source / Apache 2.0 license
• A key / value store
• Written in Java, C, C, SQL
• Multiple API's available
• Multiple deployment options
• Developed by GridGain Systems

2
What Is Apache Ignite ?

• Persistence is turned off
• A distributed in-memory database
• In memory data grid
• Use SQL or key-value APIs
• Persistence is turned on
• A distributed, horizontally scalable database
• Guarantees full data consistency
• Is resilient to full cluster failures

3
How Does Ignite Work ?

• Persistence can be toggled
• Small data sets can be stored in memory
• Large datasets can use disk
• Memory can be used as a caching mechanism
• RDBMS or NoSQL integration
• Partitioning or replication of data across
  cluster
• Horizontally scaleable

4
How Does Ignite Work ?

• Use Ignite as a distributed cache
• Implements JCache specification (JSR 107)
• Supports ACID transactions
• Supports SQL ( with limitations )
• Supports Java, C, and .NET
• Has a feature rich key-value API
• Offers two user interface ( UI ) options
• Supports collocated processing

5
How Does Ignite Work ?

• Ignite has client and server nodes in it's
  cluster
• Server nodes are for storage and computation
• Client nodes are connection points to the
  database
• They may be embedded in client applications
• Application code written in Java, C or C
• Ignite supports ODBC, JDBC and REST

6
Ignite DataGrid
7
Ignite DataGrid

• An in-memory distributed key-value store
• Horizontal scaleable
• Add nodes on demand
• Scale to hundreds of nodes
• View as a distributed partitioned hash map
• Every cluster node owns a portion of the data
• As cluster (server) nodes are added more data is
  cached

8
Ignite Console
9
GridGain Console
10
Ignite Machine Learning
11
Ignite Machine Learning

• Multiple development API's
• A set of simple / efficient ML functions
• Massively scaleable
• Co-location of data and processing
• Minimised the need for costly data transfers
• Minimised ETL as ML functions act on distributed
  data
• Algorithms support
• Classification, Regression, Clustering
• Recommendation, Preprocessing

12
Ignite Deployment Options

• Multiple deployment options
• Docker
• Amazon AWS
• Google Cloud
• Mesos
• YARN
• VMWare
• Microsoft Azure
• Kubernetes

13
Available Books

• See Big Data Made Easy
• Apress Jan 2015
• See Mastering Apache Spark
• Packt Oct 2015
• See Complete Guide to Open Source Big Data
  Stack
• Apress Jan 2018
• Find the author on Amazon
• www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
• Connect on LinkedIn
• www.linkedin.com/in/mike-frampton-38563020

14
Connect

• Feel free to connect on LinkedIn
• www.linkedin.com/in/mike-frampton-38563020
• See my open source blog at
• open-source-systems.blogspot.com/
• I am always interested in
• New technology
• Opportunities
• Technology based issues
• Big data integration