Developers

The Refinder API

Refinder is a Web service that allows users to link arbitrary items that they need to do their daily work. This includes concrete digital objects like files, emails, and Web pages, but also real-world entities like persons and organizations, and abstract entities like projects, to-do-items, or topics. .com manages these items, allows to relate them to each other, and provides automatically computed recommendations (lists of things that may be relevant to another thing).

Developers, Get Your API Consumer Key

To develop an application that talks to the getRefinder.com web service, you need to obtain a consumer key (also called Client Identifier in RFC 5849) and consumer secret (also called Client Shared-Secret in RFC 5849). At the moment, this can be done only by sending an email to dev@gnowsis.com. Please specify in this email:

  • who you are,
  • what you want to develop, and
  • which programming language(s) and platform(s) you target.

Your data will be verified, and when we think it's OK we will send you a consumer key and consumer secret. Please permit up to three working days for this procedure. Note that as long as Refinder is in alpha or beta mode we reserve the right to revoke authorization for consumers at any time without giving any reason.

Authentication

Authentication is a precondition to access services. Refinder deals with private data, so every request must be properly authenticated. For this purpose, Refinder uses OAuth 1.0 (specified by RFC 5849).

In this document, we do not describe in detail how OAuth works, because this would be a duplication of efforts. Therefore, we refer to the OAuth specification and to the available guides to OAuth, for example this one.

Once you have received your consumer key and secret you can use the protocol specified by OAuth to get a request token, verifying it, and to get an access token (more about this workflow). The Refinder API defines the following URLs for this:

The Refinder API uses HMAC-SHA1 as signature method. User verification can be done by providing a callback URL which will be called by the Refinder Web service. When no callback URL is specified, the Refinder Web Application displays a verification code to the user which they can pass over to the requesting client via an out-of-band method like copy&paste (more details).

Every request to the Refinder API must be correctly authenticated and signed using OAuth. In the following, whenever we talk about issuing HTTP requests, we mean to issue a signed request, although we do not explicitly describe the procedure required for this every time.

Modelling Things

Things have the following characteristics:

  • Each thing is identified by a globally unique, stable URI. This URI is constructed by concatenating the Refinder service URL, a "things" type identifier, a UUID, and a fragment "#t". Thus, a thing URI may look like this: http://www.getrefinder.com/things/103c9c90-b590-11df-8d81-0800200c9a66/#t
    Note that the "#t" fragment identifier is required to distinguish the thing (which may be a person, an organization, or a project) from its digital representation (which may be an HTML or RDF document about the thing). For more information about this, read Cool URIs for the Semantic Web.
    In a local context (e.g., within an application) a thing may also be identified only by its UUID, since this is assumed to be globally unique even without the service URL. However, when data about things is exchanged between applications, the full URI as described before must always be used.
  • Each thing has one type, which (similar to object-oriented languages) defines the thing's characteristics, like the attributes that can be applied to it. The thing type is also identified by a unique URI. Refinder allows any URI to be used as type; however certain types are handled by the system in a certain manner (e.g., choosing icons). These types are described in Thing Types and Attributes.
  • Things can be described by a set of attributes. Each attribute is identified by a URI and a data type (which is in turn identified by a URI).
  • Each thing has one owner, which is the user that has created the thing. The owner has the right to view, modify, and delete a thing.
  • Refinder automatically stores the date and time of the creation, as well as the date and time of the last update for each thing.

Thing Types

As described before, each thing has one type which is identified by an URI. The thing type determines which attributes can be stored together with the thing. Any URI can be used as thing type; for the following types, Refinder assigns certain characteristics (e.g., display of icons): (URIs are abbreviated with namespaces for better readability):

Type URI
Note pimo:Note
Bookmark nfo:Bookmark
Person pimo:Person
Organization pimo:Organization
Location pimo:Location
Topic pimo:Topic
Event pimo:Event
Project pimo:Project
Task pimo:Task
File nfo:FileDataObject
Email nmo:Email
Question pimo:Question
RSS Item rss:item
Tweet twitter:status




Serialization

The Refinder API uses RDF to represent things. RDF is an abstract, graph-based data model. Basically it can be used to make statements about resources, whereas each resource is identified by a URI. Statements have the form "subject"-"predicate"-"object", where subject is the resource that the statement is about, predicate identifies a characteristic of the resource that is described in this statement, and object is the value of the characteristic. An example statement could be:

<http://www.getrefinder.com/things/5229f226-14d7-428c-aebe-b1a258a82a26/#t> <http://www.w3.org/2000/01/rdf-schema#label> "Dirk Hageman"

A set of an arbitrary number of such statements is called a graph. RDF itself is an abstract model; there exist many ways to serialize graphs. The Refinder platform supports the formats RDF/XML, Turtle/N3, and N-Triples. The description of a thing, consisting of a couple of statements, looks as follows in the Turtle/N3 format:

@prefix pimo: <http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#> .
@prefix nao: <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

 <http://www.getrefinder.com/things/4815f7ae-776a-4c59-b2e2-9cc09b9212b0/#t> a pimo:Thing , pimo:Event ;
 rdfs:label "CID Project Meeting Belfast June 2010" ;
 pimo:dtStart "2010-06-05T09:00:00"^^xsd:dateTime ;
 pimo:dtEnd "2010-06-07T18:00:00"^^xsd:dateTime ;
 pimo:creator <http://www.getrefinder.com/users/claudia> ;
 nao:created "2010-02-10T13:12:00"^^xsd:dateTime ;
 nao:lastModified "2010-03-23T18:11:00"^^xsd:dateTime .

The same graph, serialized using the RDF/XML format, looks like this:

<?xml version="1.0"?>
<rdf:RDF xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:pimo="http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#" xmlns:xsd="http://www.w3.org/2001/XMLSchema#" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:nao="http://www.semanticdesktop.org/ontologies/2007/08/15/nao#">
 <pimo:Thing rdf:about="http://www.getrefinder.com/things/4815f7ae-776a-4c59-b2e2-9cc09b9212b0/#t">
 <rdf:type rdf:resource="http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#Event" />
 <rdfs:label>CID Project Meeting Belfast June 2010</rdfs:label>
 <pimo:dtStart rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2010-06-05T09:00:00</pimo:dtStart>
 <pimo:dtEnd rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2010-06-07T18:00:00</pimo:dtEnd>
  <pimo:creator rdf:resource="http://www.getrefinder.com/users/claudia" />
 <nao:created rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2010-02-10T13:12:00</nao:created>
 <nao:lastModified rdf:datatype="http://www.w3.org/2001/XMLSchema#dateTime">2010-03-23T18:11:00</nao:lastModified>
 </pimo:Thing>
</rdf:RDF>

You can use services like http://any23.org to convert RDF from one serialization formats to another. It is strongly recommended, however, that you do not serialize or deserialize RDF data "by hand" in your application, but leave this task to existing parsers, which are well-tested and efficient. Recommended RDF libraries include Jena (for Java), RDFlib (for Python), RAP (for PHP) and SemWeb (for C#).

When you are working with the Refinder API, things (and all other objects) need to be sent via HTTP. HTTP provides the option to switch different serialization formats for RDF data, which can be chosen by using content negotiation. Using the HTTP headers "Accept" (for requesting data from the server) or "Content-Type" (for sending data to the server) one defines the format that the payload of the request (or the expected answer from the server) is serialized. The W3C maintains a list of registered MIME types for different RDF formats. As default and fallback (if no format can be negotiated), the Refinder API uses RDF/XML serialization.
 

Creating, Reading, Updating, and Deleting Things

The Refinder API follows the REST principles, which say that all operations are based on states of resources. Refinder treats all things as individual resources, and upon creation they are automatically assigned a globally valid, stable URI.

To create a thing, the thing's representation must be sent via a HTTP POST request to the URL http://www.getrefinder.com/things/. The representation is an RDF graph as described in the previous section, whereas instead of the thing URI a blank node must be specified, since the URI of the thing will be defined by the Refinder Web Service. Further, provenance attributes of the thing (like its owner, its creation date, and the date of last modification) cannot be passed to the server since these are automatically defined. Therefore, a typical representation of a thing that is sent to the Refinder API looks as follows:

@prefix pimo: <http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#> .
@prefix nao: <http://www.semanticdesktop.org/ontologies/2007/08/15/nao#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

 [] a pimo:Event ;
 rdfs:label "CID Project Meeting Belfast June 2010" ;
 pimo:dtStart "2010-06-05T09:00:00"^^xsd:dateTime ;
 pimo:dtEnd "2010-06-07T18:00:00"^^xsd:dateTime .

When the creation of the thing was successful, the Refinder Web API returns with a HTTP 201 (Created) status code. Further, the response contains a Location header that contains the URI of the newly created thing (for instance, http://www.getrefinder.com/things/caafd800-bcd2-11df-851a-0800200c9a66/#t).

This URI can be directly used to retrieve the thing's representation, via a HTTP GET request. When the thing is found, HTTP 200 (OK) is returned, whereas the body of the request contains the RDF representation of the thing. When a thing does not exist under the specified URI, a HTTP 404 (Not Found) status is returned.

A thing can be updated (modified) by sending an updated representation of its attributes to its URI via HTTP PUT. Note that to update a single field of the thing (e.g., its label), the entire representation must be sent. It is not sufficient to send only a partial representation, since that would cause the other, unspecified attributes to be removed from the Refinder data base.

Things can be deleted using the HTTP DELETE method to the thing URI.

Search

Refinder provides access to its search feature via the API: just issue a request to http://www.getrefinder.com/things/search/?q=querystring and a list of things will be returned that match the specified search criteria. This is the same operation as entering querystring into the "search" text field of the Refinder Web Application. Note that any special characters in "querystring" must be URL-encoded with UTF-8 encoding.

Alternatively, you can locate things via their data objects or data sources.

Data Sources and Data Objects

The concepts of data sources and data objects are used to connect the "logical" things within Refinder (persons, events, tasks, projects, etc.) to the concrete, "physical" data items that are floating around on computers. For this purpose, each thing in Refinder may be associated to an arbitrary number of data objects, each of which corresponds to a physical unit of data. Data objects, in turn, can be grouped into data sources, which form logically closed connections of data objects.
For instance, a data source might be an email account on an IMAP server. Each email within this account might be a data object.
Data objects can be used to check whether for a given physical unit of data a corresponding thing in Refinder already exists. Even more importantly, they are used by the Refinder Web Application and the Refinder Sidebar to allow the user to directly open the data unit, regardless of where the thing appears. Thus, it is possible to directly open a file from a Refinder sidebar that is embedded in Outlook, and it is possible to open an appointment in Outlook from a Refinder sidebar that is embedded in Firefox. This works in conjunction with the Refinder URL Handler.
The Refinder API allows you to register and retrieve data sources, to associate things with data objects, and to retrieve things based on their associated data sources and data objects.

Registering Data Sources

To register a data source (for instance, an account on a Web platform) you issue a POST request to http://www.getrefinder.com/datasources/. The request body must contain an RDF description about the data source, which follows the following template:

@prefix datasource: <http://www.getrefinder.com/ns/2010/06/datasource#> .
@prefix pimo: <http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .

[] a datasource:DataSource;
 datasource:type <http://www.getrefinder.com/2010/06/ds#IMAP>;
 rdfs:label "Bernhard's IMAP account at GreatMail";
 datasource:physical "imap://bernhard@imap.greatmail.com".

The values for datasource:type, rdfs:label, and datasource:physical can be defined freely by the client. The type of the data source must be a URI; it is used on the one hand by the Refinder Web Application to define the visual appearance of the data source, and on the other hand by the Refinder URL Handler to determine the correct action to be taken when the user wants to open a data object that is part of this data source. The rdfs:label property is a human-readable description of the data source, and the datasource:physical property is used to specify the physical properties of the data source; in our example, it describes the login name and the IMAP server which are represented by this data source. The physical property can be used to search whether a data source for a given physical system already exists.

If the creation of the datasource was successful, the server replies with HTTP 201, the Location header contains a URI of the form http://www.getrefinder.com/datasources/<uuid>/#ds, whereas the UUID uniquely identifies the datasource. This UUID should be remembered because it is required to associate data objects to the data source.

Associating Data Objects to Things

After a thing was created (see Things) one may associate data objects to the things. This happens by issueing a PUT (!) request to the URL http://www.getrefinder.com/things/<thing-uuid>/dataobjects/. The content of the PUT request should contain association statements between the thing and one or more data objects, as follows:

@prefix pimo: <http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#> .

<http://www.getrefinder.com/things/{thing-uuid}/#t> pimo:groundingOccurrence <refinder:{datasource-uuid}:{dataobject-id}> .

As one can see, data objects use a special kind of URI, called Refinder Data Object URI. A Refinder data object URI consists of three parts: the URL schema identifier ("refinder:"), the UUID of the datasource this object is associated with, and the local ID of the dataobject, relative to the data source.

An example of a concrete data object URL is refinder:be67b770-bccd-11df-851a-0800200c9a66:%2fUsers%2fbernhard%2ftest.txt (note that the local-id part of the URL is URL-encoded, this must always be done!). Assuming that the referenced data source is a file system, this URL refers to the data object within this data source. The interpretation of the local ID part of the Refinder data object URL depends on the type of the data source: objects within a file system are referenced differently than objects on a Web application, or objects within an email account. It obeys the Refinder URL handler to correctly interpret the local part w.r.t. the type of the data source.

Additionally, a client may associate a clickable URL with the data object, which is used in the Refinder User Interface to render a different link than the Refinder URL. This is in particular useful for all data objects that can be reached with a standard URL and therefore require no special URL handler. A clickable URL can be associated to the data object by adding a second statement to the data object RDF description:
 

@prefix pimo: <http://www.semanticdesktop.org/ontologies/2007/11/01/pimo#> .
@prefix dataobject: <http://www.getrefinder.com/ns/2010/06/dataobject#> .

<http://www.getrefinder.com/things/{thing-uuid}/#t> pimo:groundingOccurrence <refinder:{datasource-uuid}:{dataobject-id}> .
<refinder:{datasource-uuid}:{dataobject-id}> dataobject:clickable-url <http://some-url.com/clickme> .
 

Note that when a thing is associated with one or more data objects, the thing can no longer be modified through the Refinder Web Application, it can only be modified through the Refinder API.

Retrieving Things via Data Sources or Data Objects

When data sources and/or data objects have been associated to things, the things can be retrieved via two API calls:

GET http://www.getrefinder.com/things/search/?dataobject=<dataobject-uri> returns all things that are associated to the data object. For the data object, the full Refinder Data Object URI must be specified (it must be URL-encoded). Instead of a data object URI, a clickable URL may also be passed to as parameter.

GET http://www.getrefinder.com/things/search/?datasource=<datasource-uuid> returns all things that are associated to data objects within the specified data source. The UUID of the data source must be provided as parameter.