Data is the core of all things at Uber and every aspect of the Uber experience is powered by data - this ranges from systems that connect the rider to a driver, predict the correct fare for your ride, calculate in-app ETAs, optimal routing for Uber pool, restaurant recommendations in Uber eats, to online fraud detection. The Data org at Uber is a central team that builds the foundational services and platforms that power Uber’s analytics, machine learning, AI, & business intelligence applications. This talk will focus on the infrastructure and platforms that Uber have internally built that power data ingestion, storage, streaming/batch analytics, and machine learning for thousands of operators, data scientists, and engineers at Uber. In this talk, speaker will also cover some unique challenges relating to managing their petabyte scale data lake across a hybrid on-prem & cloud based infrastructure. Speaker will do a deep dive into Uber’s batch analytics infrastructure comprising of Hadoop/HDFS, Hive, Spark, Presto, and YARN/Mesos/Peloton and how these systems help them build the foundation required for the next 5 years of growth.

Speaker Bio:

Hari Subramanian leads several efforts within Uber data infrastructure including building a highly scalable cross-datacenter data processing layer using Apache Spark and SQL analytics-as-a-Service using Apache Hive. He also works in the intersection of infrastructure, machine learning, and data science and builds platforms that help Uber’s data scientists take machine learning models seamlessly from conception to production. Prior to Uber, Hari led software engineering teams at Amazon and built and operated cloud compute infrastructure for AWS EC2. Apart from his management experience, Hari has also tinkered with hypervisors, kernels, and storage systems as a systems software developer at VMware and played the role of a product manager on various occasions including at his own startup. Hari is also an inventor with over 10 granted software patents to his name and a speaker at local meetups and conferences.