O'Reilly logo

Streaming Systems by Tyler Akidau, Reuven Lax, Slava Chernyak

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 8. Streaming SQL

Let’s talk SQL. In this chapter, we’re going to start somewhere in the middle with the punchline, jump back in time a bit to establish additional context, and finally jump back to the future to wrap everything up with a nice bow. Imagine Quentin Tarantino held a degree in computer science and was super pumped to tell the world about the finer points of streaming SQL, so offered to ghostwrite this chapter with me; it’s sorta like that. Minus the violence. And the vintage soundtrack. And Uma Thurman.

What is streaming SQL?

I would argue that the answer to this question has eluded our industry for decades. In all fairness, the database community has understood ~99% of the answer for quite a while now. But I have yet to see a truly cogent and comprehensive definition of streaming SQL that encompasses the full breadth of robust streaming semantics. That’s what we’ll try to come up with here, although it would hubris to assume we’re 100% of the way there now. Maybe 99.1%? Baby steps.

Before we dive in, it’s also worth highlighting that the vision for streaming SQL presented in this chapter is the result of a collaborative discussion between the Apache Calcite, Apache Flink, and Apache Beam communities. Julian Hyde, the lead developer on Calcite, has long pitched his vision for what streaming SQL might look like. In 2016, members of the Flink community integrated Calcite SQL support into Flink itself, and began adding streaming-specific features such as windowing ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required