Introduction to Spring Cloud Dataflow (I)

Introduction

This is the first of a series of posts on how to develop data-driven micro-services with Spring Cloud Dataflow (SCDF from now on).  For now, we will see what is the approach proposed by this framework and how to build locally the basic components: Source, Sink and Processor.

Also, if you are familiar with Spring, we will take a look to the already-made components that are available for you to use so you don´t have to reinvent the wheel.

Contents

• What is Spring Cloud DataFlow?
• Introduction to the API: A simple Producer (Source), Consumer (sink) setup.
• Prerequisites: Kafka and Zookeeper.
• Coding a simple Source: HTTP Poller that retrieves live stock prices.
• Coding a simple Sink that store the results in a file.
• Writing an additional Batch Source.
• Summary & Resources.

Video: Lambda architecture with Spring XD and Spark

Summary

Today I do not have any piece of code worth showing you. Instead, I would like to quickly comment on this YouTube video, which I have seen recently, that is coming from one the SpringOne2GX 2015 event (and later released to the public).

Note: The fact the these talks are published more and more often (like what happens with the DockerCon talks) kinda puts me off from paying the high premium involved in physically assisting to these events...