O'Reilly logo

Streaming Systems by Reuven Lax, Slava Chernyak, Tyler Akidau

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 6. Streams & tables

You have reached the part of the book where we talk about streams and tables. If you recall, back in Chapter 1, we briefly discussed two important, but orthogonal dimensions of data: cardinality and constitution. Up until now, we’ve focused strictly on the cardinality aspects (bounded vs unbounded), and otherwise ignored the constitution aspects (stream vs table). This has allowed us to learn about the challenges brought to the table by the introduction of unbounded datasets, without worrying too much about the lower-level details that really drive the way things work. We’re now going to expand our horizons, and see what the added dimension of constitution brings to the mix.

Though it’s a bit of a stretch, one way to think about this shift in approach is to compare the relationship of classical mechanics to quantum mechanics. You know how in physics class they teach you a bunch of classical mechanics stuff like Newtonian theory etc, and then once you think you’ve more or less mastered that, they come along and tell you it was all bunk, and classical physics only tells you part of the picture, and there’s actually this other thing called quantum mechanics that really explains how things work at a lower level, but it didn’t make sense to complicate matters up front by trying to teach you both at once, and oh wait we also haven’t fully reconciled everything between the two yet so just squint at it and trust us that it all makes sense somehow? Well this ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required