Real-time Analytics on a High Performance Database Platform
3 May 2014 Apigee, Bangalore
Vast amounts of data are being processed within a few milliseconds in every organization and sector. Here lies the importance of architecture and technology choices to ensure that systems perform fast and as quickly as possible.
In this run-up event, we present two different perspectives: one, of choosing available technology to enable real-time analytics. Second, of architecting and building a system for the organization to ensure that data is processed real time and can present information for making decisions and sales.
Talks and Workshops
Real-time Analytics on a High Performance Database Platform [ talk ]
Dr. Srini speaks on functionality and efficiency, in the context of Aerospike's multiple deployments at 1M+ TPS.
The techniques described include how to:
- Use an in-memory NoSQL database for low latency access
- Exploit parallelism using a combination of DRAM and flash
- Use flexible schema and dynamic definition of access paths for efficient query retrieval
- Increase number of records that can be accessed in real-time using efficient secondary indexes
- Size deployments precisely for predictable price/performance
- Configure the system to ensure 100% uptime
- Define operator friendly procedures for launching a system at Internet scale
Dr. Srini V. Srinivasan, Aerospike founder and vice president of engineering & operations brings 20-plus years of experience in designing, developing and operating Web-scale infrastructures, and he holds over a dozen patents in database, Internet, mobile, and distributed system technologies. Previously, Vice President of Engineering, Yahoo! and Chief Architect of IBM’s DB2 internet products.
Hyperion: The Flipkart way [ talk ]
How do you know what's going on in the apps and on your services? The key lies in being able to get insights into what's happening on your app and in your production systems quickly and without failure.
In this talk we discuss Hyperion: a fault tolerant, real time, distributed event tracking and analysis system. The system allows clients to push, store and analyze event data to gain long term and short term insight in a collection of related systems. It is designed to be scalable, flexible and reliable, and provides real-time histograms, trends and query features. Hyperion is built as a platform for event storage, query and processing. It provides capabilities to the clients to build very powerful apps on top of it. We'll go through the architecture, performance and learnings from the system.
Santanu Sinha is an architect working on real-time analytics and next generation systems at flipkart.com. Prior work include developing the distributed content ingestion framework that powers meta and content ingestion for all books and ebooks on flipkart.com.
Introduction To Real-Time Analytics On Aerospike [ workshop ]
The workshop is divided into two phases.
Phase-1: Installation and Setup
- Installing Aerospike
- Forming Cluster
- Putting load on Aerospike
- Take down nodes and bring them up
- Monitoring using AMC
- Dynamically changing some config parameters
Phase-2: Running an analytics use case.
- Load dataset
- Create secondary index
- Create a Streaming Aggregation UDF and registering it
- Triggering the Aggregation LUA UDF
- Check the results
- Play with the Aggregation logic
Laptop running Centos/Ubuntu/Debian Linux (could be VM).
10:30 AM - 10:45 AM
Registrations & checkin
10:45 AM - 11:00 AM
11:00 AM - 11:45 AM
talk: Real-time Analytics on a High Performance Database Platform
Interested in attending? Please login. You can use your existing Twitter or Google account, and if you have previously voted on a session proposal or attended a hacknight, you already have a HasGeek account.