Cover by Brian Carper, Christophe Grand, Chas Emerick

Safari, the world’s most comprehensive technology and business learning platform.

Find the exact information you need to solve a problem on the fly, or go deeper to master the technologies and skills you need to succeed

Start Free Trial

No credit card required

O'Reilly logo

À la Carte Message Queues

Let’s combine all of what we’ve learned about how to effectively use CouchDB from Clojure to implement an asynchronous work queue based on the events being emitted into the logging database we considered earlier. First, a refresher on what that data looks like:

Example 15-4. Sample event data

{:evt-type "auth/new-user" :username "Chas"}
{:evt-type "auth/new-user" :username "Dave"}
{:evt-type "sales/purchase" :username "Chas" :products ["widget1"]}
{:evt-type "sales/purchase" :username "Robin" :products ["widget14"]}
{:evt-type "sales/RFQ" :username "Robin" :budget 20000}

Now, you could add a single watch for changes in the logging database, and instead of echoing those events to *out*, do something useful with them; that’s a perfectly valid thing to do, especially if your requirements are relatively simple. However, most real-world systems need some additional levers to pull to control event processing loads and more ably support modular design requirements.

We can assume that our typical website or application will be generating all sorts of events; some will only need to be retained for some period of time, some will need to go into a long-term archive, and others will need to be acted upon as soon as possible. Earlier, we defined a hierarchy over the concrete event types in our sample dataset. Let’s now build a parallel hierarchy that defines how different types of events should be consumed:

Example 15-5. Defining a partial event processing hierarchy

(ns eventing.processing) ...

Find the exact information you need to solve a problem on the fly, or go deeper to master the technologies and skills you need to succeed

Start Free Trial

No credit card required