Appendix. Capturing data on the web

As you’ve learned in this book, intelligent applications are those that can change their behavior based on information. It follows, then, that we must have a mechanism for the capture and access of data. Because we’re talking about web-scale processing, it stands to reason that we may need a system designed with the following in mind:

  • Volume— Our system should be capable of dealing with web-scale data.
  • Scalability— Our system should be configurable with changing load.
  • Durability— Outages or network blips shouldn’t affect the eventual consistent state of data.
  • Latency— We shouldn’t expect to wait long periods of time between data being generated and data being processed.
  • Flexibility— Access to the data ...

Get Algorithms of the Intelligent Web, Second Edition now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.