To get started, we will compare two topics:
- Stream ingestion: This is a process where you introduce a service/API for receiving messages from your producers. Such a service is designed to ingest data only—it does nothing more (such as transforming or analyzing). To perform any kind of analysis of ingested data, you have to introduce your own processors.
- Stream analysis: This is a process where you actually analyze the data. You search for anomalies, duplicates, or malformed data, process it, and push it further to other services for storing, presenting, and triggering other actions.
To make things even clearer, we can take a look at the following diagram:
It shows the four steps of data processing: ...
