SPARQL + pubsubhubbub = sparqlPuSH

There have been lots of discussion recently regarding dynamics and notification in the Semantic Web realm, including various vocabularies for describing changes and approaches for notifying them – as Leigh recently blogged about it.
Last month, while visiting Kno.e.sis, Pablo an I worked on an approach using pubsubhubbub for RDF changes notification, that I’m happy to announce today.

The result is sparqlPuSH, an interface that can be plugged on any SPARQL endpoint and that broadcast notifications to clients interested in what’s happening in the store using the pubsubhubbub protocol. At a glance, anyone can register a particular query to the RDF store (e.g. list all microblog posts, or list any changes made by X, using the Changesets vocabulary) and results are provided in an RSS / Atom feed that is then sync-ed using pubsubhubbub: each time new data corresponding the register query is added into the store, the store itself notifies the interested parties of such updates.
Practically, this means that you can be notified in real-time of any change happening in a SPARQL endpoint.

The following video describes how the approach works as well as shows a related use-case and you can download its source at http://code.google.com/p/sparqlpush/.
It can be used as an interface on the top of any SPARQL endpoint and also comes with an ARC2 interface (if you’re using a different endpoint, the interactions happen via HTTP and use requires that your endpoint provides JSON SPARQL query results).

We believe that a push system like this for RDF notification can change lots of things regarding RDF data management and how to make sense of it, in real-time. In addition, we hope that such approach could be generalised not only to SPARQL endpoints, but to resource themselves, so that one resource can ping a pubsubhubbub hub when it changes, the notifications being then broadcasted to interested parties.

Leave a Reply

Your email address will not be published. Required fields are marked *