O'Reilly logo

Mastering Scala Machine Learning by Alex Kozlov

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Basic components of a data-driven system

In short, a data-driven architecture contains the following components—at least all the systems I've seen have them—or can be reduced to these components:

  • Data ingest: We need to collect the data from systems and devices. Most of the systems have logs, or at least an option to write files into a local filesystem. Some can have capabilities to report information to network-based interfaces such as syslog, but the absence of persistence layer usually means potential data loss, if not absence of audit information.
  • Data transformation layer: It was also historically called extract, transform, and load (ETL). Today the data transformation layer can also be used to have real-time processing, where the aggregates ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required