Beautiful Data

Conclusion: Mediation via Gnip

Amidst raging tides bringing more heterogeneous APIs ashore every day, some consistency is warranted. As a message-oriented middleware service, Gnip promises to "deliver the Web's data" using a middleman data brokerage approach. Representing the data publisher's desire to share via an array of inbound transport protocols, Gnip shuttles normalized data from those publishers to Gnip consumers in real time (sub-60-second latency from message receipt to rebroadcast). Regardless of inbound data format inconsistencies, Gnip "cleans" and normalizes the data into a canonical format as a service to the Gnip consumer. The result is a single integration with Gnip's API, and consistently formatted data for the consuming application.

You can access many social data APIs through Gnip's single interface, all in "real time." Gnip's framework is biased toward the event-driven model described earlier, in order to promote a more efficient data flow across the network and between applications at large. However, polling is supported as well. Requiring polling-based applications to switch to an event-based model in order to leverage the normalizing benefits Gnip provides would be too heavy handed. Without simultaneous, wholesale adoption of yet-to-be-completed standards around Publish/Subscribe frameworks and format consistencies, a broker in the middle is necessary.

The current infrastructure has done a great job carrying things thus far, and online commerce has boomed. ...