O'Reilly logo

Streaming Systems by Reuven Lax, Slava Chernyak, Tyler Akidau

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 8. Streaming SQL

Let’s talk SQL. In this chapter, we’re going to start somewhere in the middle with the punchline, jump back in time a bit to establish additional context, and finally jump back to the future to wrap everything up with a nice bow. Imagine Quentin Tarantino held a degree in computer science and was super pumped to tell the world about the finer points of streaming SQL, so offered to ghostwrite this chapter with me; it’s sorta like that. Minus the violence.

What is streaming SQL?

I would argue that the answer to this question has eluded our industry for decades. In all fairness, the database community has understood ~99% of the answer for quite a while now. But I have yet to see a truly cogent and comprehensive definition of streaming SQL that encompasses the full breadth of robust streaming semantics. That’s what we’ll try to come up with here, although it would be hubris to assume we’re 100% of the way there now. Maybe 99.1%? Baby steps.

Regardless, I want to point out up front that most of what we’ll discuss in this chapter is still purely hypothetical as of the time of writing. This chapter and the one that follows (covering streaming joins) both describe an idealistic vision for what streaming SQL could be. Some pieces are already implemented in systems like Apache Calcite, Apache Flink, and Apache Beam. Many others aren’t implemented anywhere. Along the way, I’ll try to call out a few of the things that do exist in concrete form, but given what a moving ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required