An introduction to Apache S4

1
Apache S4

• What is it ?
• Building Blocks
• Characteristics
• Architecture
• Why was it needed ?

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
2
Apache S4 What is it ?

• S4 SSSS
• Simple Scalable Streaming System
• For data processing
• Aims to hide processing complexity
• A platform for data processing development
• Distributed
• Scalable
• Fault Tolerant
• High performance

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
3
Apache S4 What is it ?

• It is modular
• Written in Java
• Currently in the Apache incubator
• Offers plug-ins
• For complex data stream processing
• Production deployment at Yahoo

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
4
Apache S4 Building Blocks

• What are the S4 building blocks ?
• Message queues
• Processors
• Serializer
• Checkpointing backend
• Plus
• A simple API

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
5
Apache S4 Characteristics

• Cluster management uses Zoo Keeper
• Symmetric nodes
• No centralization
• No single point of failure
• No limit on nodes
• Throughput increases linearly
• Fault tolerant failover to standby server

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
6
Apache S4 Architecture
www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
7
Apache S4 Why was in needed ?
www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
8
Contact Us

• Feel free to contact us at
• www.semtech-solutions.co.nz
• info_at_semtech-solutions.co.nz
• We offer IT project consultancy
• We are happy to hear about your problems
• You can just pay for those hours that you need
• To solve your problems