This event has ended. Visit the official site or create your own event on Sched.
  • Registration is open every day from 7:30 a.m. until the close of the day’s sessions. 
  • 30-minute refreshment breaks are at 10:00 a.m. and 2:30 p.m. daily (Wednesday afternoon break is at 2:00 p.m.). 
  • Lunch is 12:00-1:00 p.m. daily in the Catalina Ballroom (Monday lunch is in the Shutters Hospitality Suite)
Thursday, May 5 • 11:30am - 12:00pm
Evolution of a Data-Streaming Solution

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

In this talk, we will describe a business need for data collection and streaming that led to implementation of a data-streaming solution for delivering messages from many clients to various end points including Kafka, HDFS, Mongo DB, and Splunk. The talk will walk attendees through the evolution of the solution from local use of Active MQ and relational databases to a real-time data-streaming solution: Pivotal’s Spring Extreme Data (XD).

We will cover scalability considerations, observed performance, and security considerations. We will describe why the tool was selected, the various solution patterns implemented and discarded, and the final solution topology. We will also discuss deployment considerations for the public cloud, including the secure transport of data between data centers.

Get the slides.

Watch the video.

avatar for Joseph Paulchell

Joseph Paulchell

Director, Software Engineer, Capital One
I am currently working on data streaming solutions to enable our Big Data strategy, primarily using Spring Extreme Data to send data to Kafka, Hadoop, Mongo, and ELK stacks. I am investing heavily in Amazon Web Services and working toward certification as an architect to better understand... Read More →

Thursday May 5, 2016 11:30am - 12:00pm PDT
Fairbanks A