Checkpoint

Centralized identity store and authentication broker for web applications. It supports a number of authentication mechanisms and is provided via a http-interface.

Top contributors

Grove

Grove is a structured document storage module for storing things like comments, blog posts, events etc. and organize these documents for easy retrieval later.

Top contributors

Kudu

Ranks content in collections.

Top contributors

Tiramisu

Manages image, audio, video, and document uploads and processing.

Top contributors

Snitch

A service to track reports of objectionable content and the corresponding moderator decisions.

Top contributors

Sherlock

Sherlock munges content of various pebbles and passes it to Elasticsearch for indexing. Sherlock also offers an endpoint for querying this content.

Top contributors

Origami

Top contributors

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

GET /api/checkpoint/v1/access_groups

List all groups for the current realm.

Response Description
200 [JSON]

POST /api/checkpoint/v1/access_groups/:label

Create a new group.

Only for gods of the realm

Parameter Type Description
label String

A unique (within the realm) identifier for this group.
Response Description
201 [JSON]

PUT /api/checkpoint/v1/access_groups/:label

Create or update new group.

Only for gods of the realm

Parameter Type Description
label String

A unique (within the realm) identifier for this group.
Response Description
200 [JSON]
201 [JSON]

GET /api/checkpoint/v1/access_groups/:identifier

Retrieve metadata for a group.

Parameter Type Description
identifier String

The id or label of the group.
Response Description
200 [JSON]

GET /api/checkpoint/v1/identities/:id/memberships

Get all group memberships for an identity.

Parameter Type Description
id String

The id of the identity ('me' for current identity).
Response Description
200 [JSON]
404 No such identity in this realm.

GET /api/checkpoint/v1/identities/:id/access_to/:path

Ask if an identity has (restricted) access to a given path.

Parameter Type Description
id Required String

The id of the identity ('me' for current identity).
path Required String

The path for which access status is being checked
Response Description
200 [JSON]

GET /api/checkpoint/v1/access_groups/:identifier/memberships

Get all memberships for a group.

Parameter Type Description
identifier String

The id or label of the group.
Response Description
200 [JSON]

DELETE /api/checkpoint/v1/api/checkpoint/v1/access_groups/:label

Delete a group.

Only for gods of the realm.

Parameter Type Description
identifier String

The id or label of the group.
Response Description
200 [JSON]

PUT /api/checkpoint/v1/access_groups/:identifier/subtrees/:location

Add a path to the group.

The members of the group will be able to read all restricted content within the paths added to the group. I.e. if the path 'acme.secrets' is added to the group 'secret_cabal', its members will be able to read the secrets that are posted with the restricted flag within this path. This would include 'post.secret_dossier:acme.secrets.top_secret.sinister_plans$3241'.

Only for gods of the realm.

Parameter Type Description
identifier String

The id or label of the group.
location String

The path to add to the group (i.e. acme.secrets).
Response Description
200 [JSON]

DELETE /api/checkpoint/v1/access_groups/:access_group_identifier/subtrees/:location

Remove a path from a group.

The path must be specified exactly as it has been added to the group. No magic will remove other granted locations that may fall within the subtree of the specified location.

Only for gods of the realm

Parameter Type Description
group_identifier String

The id or label of the group.
location String

The path to remove from the group (i.e. acme.secrets).
Response Description
200 [JSON]

PUT /api/checkpoint/v1/access_groups/:access_group_identifier/memberships/:identity_id

Add a member to the group.

Only for gods of the realm.

Parameter Type Description
group_identifier String

The id or label of the group.
identity_id String

The id of the identity to add to the group.
Response Description
204 [JSON]
409 The identity is from a different realm than the group.

DELETE /api/checkpoint/v1/access_groups/:access_group_identifier/memberships/:identity_id

Delete a group membership.

Only for gods of the realm.

Parameter Type Description
group_identifier String

The id or label of the group.
identity_id String

The id of the identity to add to the group.
Response Description
200 [JSON]

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

GET /api/checkpoint/v1/accounts/:provider/:uid

Get an account by provider and uid. Requires god permissions or that the given account is associated with the current session identity.

Only provided to god-identities.

Parameter Type Description
provider Required String

The provider of the account. E.g. github, twitter, facebook.
uid Required String

The id of the account according to the provider.
Response Description
200 [JSON]
404 No such account.

GET /api/checkpoint/v1/identities/:id/accounts

Get all accounts for an identity. Requires god permissions or that the identity is the same as the current session identity.

Only provided to the identity in question or god.

Parameter Type Description
id Required String

The id of the identity. ('me' for current identity).
Response Description
403 This is not you or you are not god!
404 No such identity.

GET /api/checkpoint/v1/identities/:id/accounts/:provider

Get an account for an identity. Requires god permissions or that the identity is the current session identity.

Only provided to the identity in question or god.

Parameter Type Description
id Required String

The identity id. Can be a numeric id or the string 'me'.
provider Required String

The provider of the account, e.g. 'github', 'twitter', 'facebook'.
Response Description
200 [JSON]
404 No such identity.

POST /api/checkpoint/v1/identities/:id/accounts/:provider/:uid

Create or updates an account and associate with an identity. If there is no current identity, then a new identity is created, unless `identity_id` is provided. Requires god permissions.

Thomax sez: Bug which I don't want to fix: Take care not to wrap account in {account: {uid: 123, etc: ...}}, just put the account object: {uid: 123 etc: ...} on the request body.

Parameter Type Description
provider Required String

The provider of the account. E.g. github, twitter, facebook.
uid Required String

The id of the account according to the provider.
account Required JSON

The account record (see readme).
Response Description
201 [JSON]
404 No such identity.

DELETE /api/checkpoint/v1/identities/:id/accounts/:provider/:uid

Deletes an account. Requires god permissions.

Only for gods and self.

Parameter Type Description
provider Required String

The provider of the account. E.g. github, twitter, facebook.
uid Required String

The id of the account according to the provider.
Response Description
204 Gone.
404 No such identity.

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

POST /api/checkpoint/v1/logout

Log out from current session.

Link to this endpoint to provide a way for the user to log out.

Parameter Type Description
redirect_to Required String

Where to redirect the user after login.
Response Description
301 Redirect to target address.

GET /api/checkpoint/v1/login/:provider

Log in with a given provider.

When a user wants to log in to your application, direct her to this endpoint. Checkpoint will take care of the authentication process and redirect the user to the target address with a valid session.

Parameter Type Description
provider Required String

The provider to log in via. E.g. twitter, facebook, google.
redirect_to Required String

Where to redirect the user after login.
force_dialog Boolean

Force login dialog with the provider (not supported by all providers).
Response Description
301 Redirect to target address.

GET /api/checkpoint/v1/login/anonymous

Log in anonymously.

Some applications require the user to perform actions that need an identity, but still do not require any authenitcation. Redirect the users browser here to get an anonymous user session. The identity will be a valid checkpoint identity with no accounts or profile.

Parameter Type Description
redirect_to String

Where to redirect the user after login (NOTE: required if non-xhr request).
Response Description
200 (if xhr request) Logged in
301 Redirect to target address.
403 (if xhr request) IP-address is hot. Please redirect user to 'login/anonymous'
409 (if xhr request) Logged in already

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

GET /api/checkpoint/v1/bannings/:path

Get all effective bans for a given path

GET /api/checkpoint/v1/bannings/mine/:path

Check if current session is banned on a given path Returns a json object with the boolean property `banned`, and if this is true, the `path` property containst the path the user is banned from

PUT /api/checkpoint/v1/bannings/:path/identities/:identity

Make it so the given identity is banned from the provided path. Will replace any more specific bans shadowed by this ban. Will not do anything if a more general, equivalent ban is allready in place. Returns a list of the bans that are actually in effect to comply with this directive.

DELETE /api/checkpoint/v1/bannings/:path/identities/:identity

Delete any effective bans that preclude the given identity from accessing the given path. May delete a more general ban if required. Returns all bans that were removed.

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

GET /api/checkpoint/v1/callbacks

Get all callbacks for the current realm. Requires god permissions.

Response Description
200 [Collection of callbacks]
403 You are not god

POST /api/checkpoint/v1/callbacks

Create a callback. Requires god permissions.

This method does nothing if a callback allready exists for the provided params.

Parameter Type Description
callback.path Required String

The path the callback shall protect.
callback.url Required String

The url which will allow/disallow an action.
Response Description
201 [Attributes of callback]
400 You forgot to namespace your record - or, there were no attributes
403 You are not god
404 No such realm

GET /api/checkpoint/v1/callbacks/:id

Get a specific callback. Requires god permissions.

Response Description
200 [Attributes of callback]
403 You are not god

DELETE /api/checkpoint/v1/callbacks/:id

Delete a specific callback. Requires god permissions.

Response Description
200 [Attributes of former callback]
403 You are not god

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

GET /api/checkpoint/v1/domains/:name

Get metadata for a domain.

Parameter Type Description
name Required String

The domain name.
Response Description
200 [JSON]
404 Not found.

POST /api/checkpoint/v1/realms/:label/domains

Add a domain to a realm.

To use checkpoint on a given domain it must be added to a realm first.

Only gods of the realm may do this.

Parameter Type Description
label Required String

The realm.
name Required String

The domain name.
Response Description
201 [JSON]
403 The domain is connected to a different realm.
409 You are not a god in this realm.

DELETE /api/checkpoint/v1/realms/:label/domains/:name

Delete a domain from a realm.

Only gods of the realm may do this.

Parameter Type Description
label Required String

The realm.
name Required String

The domain name.
Response Description
204 Ok.
403 The domain is connected to a different realm.
409 You are not a god in this realm.

GET /api/checkpoint/v1/domains/:name/allows/:origin

Test if a domain associated with Checkpoint trusts an abritary domain

Parameter Type Description
name Required String

The domain name associated with Checkpoint.
origin Required String

The abritary domain name to test against.
Response Description
200 [JSON] allowed: true/false
404 No associated domain name.

POST /api/checkpoint/v1/realms/:label/domains/:name/origins

Add an origin host to a domain.

Add a host to the domain's origins

Only gods of the realm may do this.

Parameter Type Description
label Required String

The realm.
name Required String

The domain name.
origin Required String

The origin domain name.
Response Description
201 OK
403 The domain is connected to a different realm.
409 You are not a god in this realm.

DELETE /api/checkpoint/v1/realms/:label/domains/:name/origins/:origin

Delete an origin host from a domain.

Only gods of the realm may do this.

Parameter Type Description
label Required String

The realm.
name Required String

The domain name.
origin Required String

The origin domain name.
Response Description
204 Ok.
403 The domain is connected to a different realm.
409 You are not a god in this realm.

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

POST /api/checkpoint/v1/identities/:identity_id/fingerprints

Post a new fingerprint to an identity

Parameter Type Description
fingerprints Required Array

An array of strings to add as fingerprints
Response Description
200 [JSON] The identity object containing the complete list of fingerprints, including the newly added ones

GET /api/checkpoint/v1/fingerprints/:fingerprints/identities

Get identities by fingerprints

Parameter Type Description
fingerprints Required String

A comma-separated list of fingerprints
Response Description
200 [JSON] A collection of identities

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

POST /api/checkpoint/v1/identities

Create a new identity.

Typically a new identity is created implicitly by logging in for the first time in a new realm. This endpoint is used for importing accounts from legacy systems.

Only for gods. Check readme for details on the parameters.

Parameter Type Description
identity Required Hash

The attributes of the new identity.
account Required Hash

The attributes of the default account.
Response Description
201 [JSON]

POST /api/checkpoint/v1/identities/:id

Update attributes for an identity.

Only for gods. Check readme for details on the parameters.

Parameter Type Description
id Required Integer

The id of the identity to update.
identity Required Hash

The updated attributes.
Response Description
200 [JSON]

GET /api/checkpoint/v1/identities/:id

Retrieve one or more identities including profiles.

Parameter Type Description
id Required Integer

The identity id or a comma-separated list of ids, 'me' for current user.
Response Description
200 [JSON]

GET /api/checkpoint/v1/identities/find

Find identities, match against 'name', 'nickname', 'email' attributes on the identities' accounts.

Only for gods.

Parameter Type Description
q Required String

limit Integer

The maximum amount of posts to return.
offset Integer

The index of the first result to return (for pagination).
Response Description
200 [JSON]

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

POST /api/checkpoint/v1/realms

Create a realm.

Parameter Type Description
realm Required Hash

The attributes of the realm.
domain Required Hash

The attributes of the primary domain.
Response Description
201 [JSON] The realm, along with a (god) identity and a session key.

GET /api/checkpoint/v1/realms

List all realms.

Response Description
200 [JSON] the realms

POST /api/checkpoint/v1/sessions

Create a session.

Only gods may create sessions for other identities.

Parameter Type Description
identity_id Required Integer

The id of the identity to create a session for.
Response Description
200 [JSON] Session metadata.

GET /api/checkpoint/v1/realms/:label

Get metadata for a realm.

Response Description
200 [JSON] The realm. Includes god sessions for the realm if current id is root.

GET /api/checkpoint/v1/sessions/:key

Get metadata for a session.

Only gods may inspect the sessions of other identities.

Parameter Type Description
key Required String

The key of the session.
Response Description
200 [JSON] Session metadata.

DELETE /api/checkpoint/v1/sessions/:key

Delete a session key.

Deleting a session key effectively logs out the user with that key.

Only gods may delete sessions of other identities.

Parameter Type Description
key Required Integer

The hash of the session.
Response Description
200 [JSON] Session metadata.

GET /api/checkpoint/v1/domains/:name/realm

Get a realm by its domain name. (Oh, inverted world!)

Response Description
200 [JSON] The realm.

Checkpoint

Checkpoint is a centralized authentication broker for web applications that supports a number of authentication mechanisms via an http interface. Checkpoint can take care of logging your users into your application and keep track of session and access privileges across services.

Build Status

Concepts

Basic config

To initiate authentication, you first need to have a realm with a domain set up for your application:

$ bundle exec ./bin/checkpoint realm create example -t "Example Security Realm" -d example.org

Checkpoint is provided as an http service and needs to be mapped into the url-space of your application using some proxy mechanism. The standard root url for checkpoint is:

/api/checkpoint/v1/

In production this mapping is done with ha-proxy. In development a rack proxy will be provided.

Typical usage

Given that your basic config is set up, your user can log in by being sent to the appropriate login action. A "Log in with Twitter" link should direct the browser to the following url:

/api/checkpoint/v1/login/twitter

An authentication process will commence possibly taking your user via twitter to confirm their identity. If login is successful your user is returned to your application at:

/login/succeeded

The session key for the logged in user is now stored in the cookie named 'checkpoint.session'. This is a 512 bit hash that can be used with all Pebble-compliant web-services to identify your current user and her credentials. (Unsuccessful logins are returned to: /login/failed)

Currently Checkpoint supports the following authentication mechanisms:

Sessions

The basic purpose of Checkpoint is providing and managing sessions for your users. A session in Checkpoint is represented by a 512 bit string of random garbage, the 'session string'. This string can be passed around to all pebbles compliant web services as proof of identity.

To check the identity for a specific session, this call to checkpoint could be used:

/api/checkpoint/v1/identity/me?session=10e9pde6ww4kr5nv7y9k54kei1dj1lfe9s [...]

Pebbles expect to find the session string in one of two places. First it looks for a url-parameter named 'session', if it is not found there it will attempt to retrieve it from a cookie named 'checkpoint.session'. If neither of these are present the request will be processed without authentication.

Fingerprinting

For each account registered with an identity, one or more fingerprints are recorded for posterity based on the account information. The fingerprint is an SHA-256 hash computed from the immutable parts of the account information, such as one's Twitter UID, mobile number or similar.

The fingerprint obscures the original details but still permits the application to determine if a future credential has been fingerprinted, thus making it possible to ban Twitter users, mobile numbers, etc. without having the original information at hand.

Checkpoint exposes an API that can be used to check for banned fingerprints. This API uses the Pebblebed path system, so that the banning system can be used not only for blocking logins, but for fine-grained access control such as blocking users from posting in a forum.

Callbacks

Pebbles have default rules guarding who gets to do what to a given object. Callbacks allow implementation of custom policies for creating, updating and deleting objects.

A callback is registered with a path and an url. If a callback is registered with the path 'a.b.c' it will be consulted for any action within that path, i.e. it will be called when someone wants to update post:a.b.c.d.e.f.g$11 or post:a.b.c$1, but not post:a$1or post:z.y.x$666.

When a pebble consults checkpoint to determine if an action should be allowed, the callback will post to the url. The post will recieve the parameters method, uid and identity. method will be either 'create', 'update' or 'delete', uid is the uid of the object the action will be taken on, identity is the id of the identity attempting to perform the action. The callback should return a json-hash with the key allowed. If the action should be allowed, allowed should be true, if it should be denied: false. If the callback does not have a specific opinion on the matter it should respond with an empty hash. If the callback chooses to deny the action, it should provide a reason in the response hash. This is a human readable string that may be presented to a user.

At the time of writing, only Grove actually consults checkpoint callbacks. In the future all pebbles must support this. To support callbacks, a pebble must GET /callbacks/allowed/:method/:uid before performing the action. The allowed key in the response will be either true, false or default. true means 'categorically allowed, override default behavior', false means 'categorically' denied, while default means 'apply internal rules, the callbacks had nothing to say about this'. Typically this means no callback was defined for the path in question. If the action has been denied, the response will also include the url of the denying callback along with a textual reason for the denial.

Tagging

Identities support arbitrary sets of tags. (A tag can be any string, including spaces.) The main use case is to allow apps to tag the origin of identities when there may be multiple applications sharing the same realm.

For example, if you develop apps A and B, and these apps tag their identities accordingly, then it's trivial to determine which identities came from which app. If, say, app B is discontinued or must be moved to a separate realm, it's easy to purge/move the identities if they have been tagged correctly.

Tagging is entirely-freeform and can serve any purpose, however.

Custom strategy and provider

A customized installation of Checkpoint might need to implement its own provider and strategy for authentication against an existing user/password store. Refer to config/strategies-example.rb and the checkpoint-strategy gem to set up a Checkpoint compliant provider and strategy.

Known weaknesses

Installation

Get the code

$ git clone git://github.com/bengler/checkpoint.git

Configure ActiveRecord

$ cp config/database-example.yml config/database.yml
$ $EDITOR config/database.yml

Bootstrap

$ ./bin/bootstrap

The bootstrap script will check for dependencies and set up a development environment if PostgreSQL and Memcached are installed.

Testing

$ bundle exec rake db:test:prepare
$ bundle exec rspec spec

A note on configuring the Evernote strategy for sandbox use

When developing evernote-apps you will typically start with a sandbox account. To make the auth-mechanism use your sandbox-account, add a site-key to the evernote service-keys, like this:

evernote:
  consumer_key: <key>
  consumer_secret: <secret>
  site: 'https://sandbox.evernote.com'

GET /api/checkpoint/v1/transfer

Redirect safely to any domain within the realm while preserving the session.

Requires that both the origin domain and target domain is attached to the same realm. This method falls back to a common redirect if the origin and target domain is the same. To transfer session from 'acme.no' to an url on 'acme-other-place.org' you would direct the user to an url like this: 'http://acme.no/api/v1/checkpoint/transfer?target_url=http%3A%2F%2Facme-other-place.org'

This is an endpoint for browsers.

Parameter Type Description
target_url Required Integer

The full url (including host name) to redirect the user.
Response Description
301 Redirect.

Grove

Build Status

Grove is a generic document store layered on top of an actual database such as PostgreSQL. It can store and index structured documents like comments, blog posts, events etc. and organize these documents for easy retrieval later.

Data model

In Grove, a document is any dictionary-type object that can be represented with a JSON hash, including nested hashes. Grove organizes these documents inside posts; a post represents the "posting" of a document. All Grove APIs deal with posts, not documents directly.

A post collects the following information:

Document class

The document class is an application-specific period-delimited string of identifiers. This can be used to filter queries, and can be used to distinguish documents of different types from each other.

Examples of typical class names:

Note that the first identifier must always be post. This signals to other applications that this specific object is handled by Grove.

Paths

Grove's document database can be viewed as a hierarchy of folders. Every document must be associated with at least one path. Paths with wildcards are typically used to query Grove for content.

A path is a period-delimited list of folder names. The first name must be the realm of the document (see Checkpoint for more on realms). The second name is by convention an application identifier, while the rest of the path is application-specific.

Examples of paths:

A document has one canonical path, which is where the "original" document is stored. If you need the document to appear in multiple places in the folder hierarchy you may post it to multiple paths, which will act like "symlinks" to the document, and enable the document to appear in query results as if it were stored in all the provided paths; in reality, the original document is always returned. If the underlying document is updated, it will be updated for all paths.

Folders are created automatically whenever a document is posted; you don't have to manually create them. Any path you postulate is acceptable as long as it is within the realm of your application.

Child objects

It is a convention in Pebble applications to put children of an object in a "subfolder" of its canonical path. A subpath is (conventionally) generated by appending the numeric ID of the parent object to the path and storing the children there.

For example, an article in the football blog:

post.article:acmecorp.blogs.football$323

The numeric ID here is 323. Comments on that article would be inside the path:

post.comment:acmecorp.blogs.football.323

A comment posted to that path would get an UID such as this:

post.comment:acmecorp.blogs.football.323$534

Tags

A set og tags may be applied to any document and subsequently be used to constrain results in queries. A tag is an opaque identifier that may contain letters, digits and underscores (but not spaces).

Occurrences

A document may also be organized on a timeline. A document may have any number of timestamps (occurrences) attached to it. Each timestamp is labeled. This can be used to model start-/end-times for events, or due dates for tasks.

When querying Grove, the result set can be constrained to documents with a specific labeled occurrence and optionally only documents with such an occurrence within a specified time window. This would typically be used to retrieve events that occur on a specific date, or tasks that are overdue.

Synchronization from external sources

When synchronizing data from external sources, you should give the document an external ID (external_id). The external ID may be any string, it may e.g. be the URL or database ID of the source object. The important thing is that it is invariant for the given source object, and that it is unique within the realm of your application. This ensures that updates written by multiple concurrent workers never results in duplicates.

Additionally, Grove supports external documents. If the content of the source document is synchronized to Grove as an external_document (not document) and local edits are written to the document field, Grove ensures that consecutive synchronization operations will not overwrite local edits, while fields that do not have local edits will still be updated from source. An example:

UIDs

Across all pebbles Grove documents are identified by their UIDs. The UID of a Grove document always has base class post. UIDs have the format:

<klass>:<canonical path>$<id>

Typical UIDs will look like this:

Custom permission policies

Grove supports Checkpoint callbacks. You may override Grove's internal rules about who has permissions to create, update and delete what by implementing callbacks.

See the Checkpoint documentation for details on how to do this.

Sensitive and protected data

Every post can have data set in the sensitive field. This data is only readable (and writeable) by an identity which has edit access to the post.

Similarly, the protected column is only readable (and writerable) by an identity which has the god flag set.

Tags

Every post can have a list of tags. A tag is a basic keyword containing only non-space characters. Tags can be used to query documents.

Published status

A post is either visible or invisible. When published is true, it is always visible; when false, it is only visible if a query provides the parameter unpublished=include or unpublished=only.

Deleted

A post can be marked as deleted. When deleted is true, it is invisible to queries unless the query provides the parameter deleted=include.

API

Querying

To query for one or more documents, one performs as GET to:

/api/grove/v1/posts/<UID>?...

For example:

/api/grove/v1/posts/post:acmecorp.*?tags=unpaid

The UID part is the class, path and ID to search, and forms the basic query. All parts of this UID can contain wildcards, eg. *:acmecorp.invoices.*.

Additional parameters:

POST /api/grove/v1/posts/:uid

Create or update a post.

When creating new posts specify uid without the oid part, (e.g. 'post.event:acme.calendar'), if you specify the full uid with oid (e.g. 'post.event:acme.calendar$3242') this is considered an update of the specified document.

If you specify only some of the post attributes they will be replaced without touching the other attributes. E.g. specify only 'document' to update attributes without touching occurrences, tags or paths.

Parameter Type Description
uid Required String

The uid of the post (omitting oid).
post Required JSON

The post to create (see readme for details).
post[document] JSON

The attributes of the post.
post[tags] JSON

Array of tags for the post.
post[external_id] JSON

The external_id of the document.
post[paths] JSON

Array of synonymous paths for the post.
post[occurrences] JSON

Hash of arrays of timestamps for this post. E.g. {"start_time" => ['2012-11-14T10:54:22+01:00']}
merge Boolean

Attributes will be merged on top of post. This allows partial updates. Conflicts will be retried.
Response Description
204 Success.
403 Forbidden. This is not your post, and you are not god.
404 No such post.
409 The external_id is in use on a post with a different path.

PUT /api/grove/v1/posts/:uid

Update a post.

If you specify only some of the post attributes they will be replaced without touching the other attributes. E.g. specify only 'document' to update attributes without touching occurrences, tags or paths.

Parameter Type Description
post Required JSON

The post to create (see readme for details).
uid String

The uid of the post (omitting oid).
external_id String

The external_id of the post to update.
post[document] JSON

The attributes of the post.
post[external_id] JSON

The external_id of the document.
post[tags] JSON

Array of tags for the post.
post[paths] JSON

Array of synonymous paths for the post.
post[occurrences] JSON

Hash of arrays of timestamps for this post.
merge Boolean

Attributes will be merged on top of post. This allows partial updates. Conflicts will be retried.
Response Description
204 Success.
403 Forbidden. This is not your post, and you are not god.
404 No such post.
409 The external_id is in use on a post with a different path.

DELETE /api/grove/v1/posts/:uid

Delete a post.

By default only the original creator or a god user may delete posts. To override this behavior you must implement a PSM callback. An uid or an external_id must be specified.

Parameter Type Description
uid String

The uid of the post.
external_id String

The external_id of the post.
delete_multiple_posts Boolean

Set to true of you're God and want to delete multiple posts. Enables filtered_by which gives you some freedom to tweak the scope.
Response Description
204 Success.
403 This is not your post and you are not god!
404 No such post.

GET /api/grove/v1/posts/:uid

Query posts retrieving either a specific post or a collection of posts according to your criteria.

Due to optimizations, only very basic visibility processing is properly supported when using comma-separated uids in the query. Generally this should only be used for published, not-deleted posts with unrestricted visibility.

Parameter Type Description
uid String

The uid of a specific post, a comma separated list of uids or a wildcard. uid query (e.g. "*:acme.invoices.*")
external_id Integer

The external_id of the post you want.
tags String

Constrain query by tags. Either a comma separated list of required tags or a boolean expression like 'paris & !texas' or 'closed & (failed | pending)'.
created_by Integer

Only documents created by this checkpoint identity will be returned.
created_after String

Only documents created after this date (yyyy.mm.dd) will be returned.
created_before String

Only documents created before this date (yyyy.mm.dd) will be returned.
since String

Only documents created or updated after this timestamp (ISO 8601) will be returned.
unpublished String

If set to 'include', accessible unpublished posts will be included with the result. If set to 'only', only accessible unpublished posts will be included with the result.
deleted String

If set to 'include', accessible deleted posts will be included with the result.
occurrence[label] String

Require that the post have an occurrence with this label.
occurrence[from] String

The occurrences must be later than this time. Time stamp (ISO 8601).
occurrence[to] String

The occurrences must be earlier than this time. Time stamp (ISO 8601).
occurrence[order] String

'asc' or 'desc'. The posts will be ordered by their occurrences in the specified order.
limit Integer

The maximum amount of posts to return.
offset Integer

The index of the first result to return (for pagination).
sort_by String

Name of field to sort by. Defaults to 'created_at'.
direction String

Direction of sort. Defaults to 'desc'.
raw Boolean

If `true`, does not return the merged document, but instead provides the raw `document`, `external_document` and `occurrences` separately.
editable string

If `only`, return only posts writable for current_identity. Defaults to `include`.
Response Description
200 JSON.
403 Forbidden (the post is restricted, and you are not invited!)
404 No such post.

GET /api/grove/v1/posts/:uid/tags

Find all tags of documents that match the given UID. Returns a hash of tags and occurrence counts. The resulting tag list will be a superset of all tags of the matched documents.

Parameter Type Description
uid String

A wildcard uid query (e.g. "*:acme.invoices.*").
tags String

Constrain query by tags. Either a comma separated list of required tags or a boolean expression like 'paris & !texas' or 'closed & (failed | pending)'.
unpublished String

If set to 'include', accessible unpublished posts will be counted too.
deleted String

If set to 'include', accessible deleted posts will be counted too.
created_by Integer

Only documents created by this checkpoint identity will be counted.
created_after String

Only documents created after this date (yyyy.mm.dd) will be counted.
created_before String

Only documents created before this date (yyyy.mm.dd) will be returned.
limit Integer

The maximum amount of posts to return.
offset Integer

The index of the first result to return (for pagination).
Response Description
200 JSON.
403 Forbidden (the post is restricted, and you are not invited!)
404 No such post.

GET /api/grove/v1/posts/:uid/count

Counts the maximum posts that would be returned by a call to GET /api/grove/v1/posts/:uid

Parameter Type Description
uid String

A wildcard uid query (e.g. "*:acme.invoices.*").
tags String

Constrain query by tags. Either a comma separated list of required tags or a boolean expression like 'paris & !texas' or 'closed & (failed | pending)'.
unpublished String

If set to 'include', accessible unpublished posts will be counted too.
deleted String

If set to 'include', accessible deleted posts will be counted too.
created_by Integer

Only documents created by this checkpoint identity will be counted.
created_after String

Only documents created after this date (yyyy.mm.dd) will be counted.
created_before String

Only documents created before this date (yyyy.mm.dd) will be returned.
limit Integer

The maximum amount of posts to return.
offset Integer

The index of the first result to return (for pagination).
Response Description
200 JSON.
403 Forbidden (the post is restricted, and you are not invited!)
404 No such post.

PUT /api/grove/v1/posts/:uid/touch

Touch a post (updating the updated_at field)

Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

POST /api/grove/v1/posts/:uid/undelete

Undelete a post.

Only gods or members of an accessgroup may undelete posts. Posts lose their external_id when deleted. Undeleted posts will have their old external_ids stashed in the document under the key `external_id`.

Parameter Type Description
uid Required String

The uid of the post.
Response Description
200 Ok.
403 You don't have permission to undelete this post.

POST /api/grove/v1/posts/:uid/tags/:tags

Add tags to a post

Parameter Type Description
uid Required String

The uid of the post.
tags Required String

A comma separated list of tags to add.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

PUT /api/grove/v1/posts/:uid/tags/:tags

Replace tags for a post.

Parameter Type Description
uid Required String

The uid of the post.
tags Required String

A comma separated list of tags to set.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

DELETE /api/grove/v1/posts/:uid/tags/:tags

Remove tags for a post.

Parameter Type Description
uid Required String

The uid of the post.
tags Required String

A comma separated list of tags to remove.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

POST /api/grove/v1/posts/:uid/paths/:path

Add a synonymous path to the document

Parameter Type Description
uid Required String

The uid of the post.
path Required String

The path to add.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

DELETE /api/grove/v1/posts/:uid/paths/:path

Remove a synonymous path to the document

Parameter Type Description
uid Required String

The uid of the post.
path Required String

The path to remove.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

GET /api/grove/v1/posts/:uid/since/:cursor

Returns posts in ID order, with limited filtering support. Always sorts by ID order. Response will contain next cursor.

Parameter Type Description
uid Required String

The uid of a specific post, a comma separated list of uids or a wildcard. uid query (e.g. "*:acme.invoices.*")
cursor Required String

Previous cursor. Pass 'start' to begin from beginning.
external_id Integer

The external_id of the post you want.
tags String

Constrain query by tags. Either a comma separated list of required tags or a boolean expression like 'paris & !texas' or 'closed & (failed | pending)'.
created_by Integer

Only documents created by this checkpoint identity will be returned.
created_after String

Only documents created after this date (yyyy.mm.dd) will be returned.
created_before String

Only documents created before this date (yyyy.mm.dd) will be returned.
since String

Only documents created or updated after this timestamp (ISO 8601) will be returned.
unpublished String

If set to 'include', accessible unpublished posts will be included with the result. If set to 'only', only accessible unpublished posts will be included with the result.
deleted String

If set to 'include', accessible deleted posts will be included with the result.
limit Integer

The maximum amount of posts to return.
exclude_attributes String

Comma-separated list of top-level attributes that can be excluded from the results to optimize performance. Defaults to none.
raw Boolean

If `true`, does not return the merged document, but instead provides the raw `document`, `external_document` and `occurrences` separately.
editable string

If `only`, return only posts writable for current_identity. Defaults to `include`.
Response Description
200 JSON.
403 Forbidden (the post is restricted, and you are not invited!)
404 No such post.

POST /api/grove/v1/posts/:uid/occurrences/:event

Add an occurrence to the post

Parameter Type Description
uid Required String

The uid of the post.
event Required String

The kind of occurrence to add (e.g. 'start_time').
at Required String

Time stamp (ISO 8601) to add.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

DELETE /api/grove/v1/posts/:uid/occurrences/:event

Delete one group of occurrences.

Will delete every occurrence of the specified kind.

Parameter Type Description
uid Required String

The uid of the post.
event Required String

The kind of occurrences to delete (e.g. 'start_time').
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

PUT /api/grove/v1/posts/:uid/occurrences/:event

Replace occurrences in one group with a single occurrence.

Parameter Type Description
uid Required String

The uid of the post.
event Required String

The kind of occurrence to replace (e.g. 'start_time').
at Required String

Time stamp (ISO 8601) to replace any existing occurrences with.
Response Description
200 JSON.
403 Forbidden (This is not your post, and you are not god!)
404 No such post.

Kudu

Build Status

Feedback service.

Ack

A single piece of feedback is known as an ack.

Kudu supports flexible feedback strategies (kind), e.g. * classic kudos ("+1") * votes: upvote, downvote, and neutral ("+1", "-1", and "0"), * ratings: e.g. 1 through 5 stars ("+1", "+2", "+3", "+4", "+5") * arbitrary scores, e.g. -17, +32, +100

An Ack is provided by an identity for some object, as identified by a uid. It also must specify a kind, which is essentially a feedback strategy defined in the application, and a value.

For example:

Ack.new(:identity => 7, :external_uid => 'post:a.b.c$34', :kind => 'rating', :value => 4)

Endpoints

Get acks for the current identity. :uids can be a comma delimited list of uids, or a single uid.

GET /acks/:uids

Post feedback for an item:

POST /acks/:uid

:kind is required, and must be a valid label. :value is required and must be an integer. :identity is required and is retrieved from checkpoint using the session key. :external_uid is also required.

Scores

The aggregate scores for each :uid are available through the /scores endpoints.

Endpoints

All scores for an object, grouped by :kind

GET /scores/:uid

All scores of a given kind:

GET /scores/:uid/:kind

Fetch ranked lists. These need to be ranked by an attribute on score. uid includes a wildcard path.

GET /scores/:uid/:kind/rank/:by

Fetch mixes of ranked, randomized scores, segmented by different groups. Go look at the code. Seriously.

GET /scores/:uid/sample

Stats

TODO: implement endpoints at /stats/:path/:more_stuff.

These aggregate data about :scores and :kind (can't aggregate scores of different kinds. Apples and Oranges).

This will deliver stats for paths (i.e. an app, or a region). Currently this is sort of implemented with some hacks, for dittforslag (how many contributors, top contributors, ranked lists of various things -- most controversial, most popular, etc.

This requires support for wildcard paths, which is on the block for the next few days.

GET /acks/:uid

Get a single ack

Response Description
200 JSON

DELETE /acks/:uid

Delete a single ack

Response Description
200 JSON

GET /acks/:uid/:kind

Get a single ack for a uid, kind and current identity.

Will fail without an identity (can be passed in as parameter, or using current session). Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource.
kind Required String

Kind.
identity String

Checkpoint identity id.
Response Description
200 JSON

POST /acks/:uid/:kind

Create a single ack for current identity and the specified kind. Updates the ack if it exists.

Will fail without a current identity. Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource.
kind Required String

Kind.
ack[value] Required Integer

The value of the ack.
Response Description
200 JSON if updated
201 JSON if created

PUT /acks/:uid/:kind

Update a single ack for current identity and the specified kind.

Will fail without a current identity. Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource.
kind Required String

Kind.
Response Description
200 JSON if updated
404 JSON if ack not found

DELETE /acks/:uid/:kind

Delete a single ack for current identity Delete a single ack for current identity and the specified kind.

Will fail without a current identity. Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource.
kind Required String

Kind.
Response Description
200 JSON if deleted
404 JSON if ack not found

Kudu

Build Status

Feedback service.

Ack

A single piece of feedback is known as an ack.

Kudu supports flexible feedback strategies (kind), e.g. * classic kudos ("+1") * votes: upvote, downvote, and neutral ("+1", "-1", and "0"), * ratings: e.g. 1 through 5 stars ("+1", "+2", "+3", "+4", "+5") * arbitrary scores, e.g. -17, +32, +100

An Ack is provided by an identity for some object, as identified by a uid. It also must specify a kind, which is essentially a feedback strategy defined in the application, and a value.

For example:

Ack.new(:identity => 7, :external_uid => 'post:a.b.c$34', :kind => 'rating', :value => 4)

Endpoints

Get acks for the current identity. :uids can be a comma delimited list of uids, or a single uid.

GET /acks/:uids

Post feedback for an item:

POST /acks/:uid

:kind is required, and must be a valid label. :value is required and must be an integer. :identity is required and is retrieved from checkpoint using the session key. :external_uid is also required.

Scores

The aggregate scores for each :uid are available through the /scores endpoints.

Endpoints

All scores for an object, grouped by :kind

GET /scores/:uid

All scores of a given kind:

GET /scores/:uid/:kind

Fetch ranked lists. These need to be ranked by an attribute on score. uid includes a wildcard path.

GET /scores/:uid/:kind/rank/:by

Fetch mixes of ranked, randomized scores, segmented by different groups. Go look at the code. Seriously.

GET /scores/:uid/sample

Stats

TODO: implement endpoints at /stats/:path/:more_stuff.

These aggregate data about :scores and :kind (can't aggregate scores of different kinds. Apples and Oranges).

This will deliver stats for paths (i.e. an app, or a region). Currently this is sort of implemented with some hacks, for dittforslag (how many contributors, top contributors, ranked lists of various things -- most controversial, most popular, etc.

This requires support for wildcard paths, which is on the block for the next few days.

POST /scores/:uid/:kind/touch

Create a score.

Create a score, i.e. in order to preserve creation date. This is idempotent. Subsequent posts to the same URL will not have any effect.

Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource, or a wildcard UID indicating a collection of resources.
kind Required String

Kind to count.
Response Description
201 JSON

GET /api/kudu/v1/scores/:uid/:kind

Get detailed scores for resources with a specific kind.

Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource, or a wildcard UID indicating a collection of resources.
kind Required String

Kind to count.
limit Integer

Maximum number of results. Defaults to 20.
offset Integer

Index of the first results. Defaults to 0.
rank String

Field to sort by. Available fields are "total_count", "positive_count", "negative_count", "neutral_count", "positive", "negative", "average", "controversiality", "histogram","positive_score", "negative_score", "average_score".
updated_after String

Date or time for last time the score updated. E.g. 2014-12-24.
direction String

Sort order. Defaults to "desc".
Response Description
200 JSON

GET /api/kudu/v1/scores/:path/:kind/sample

Get a mix of scores for resources on a path.

Mix is a sample of score fields, e.g. "positive", "controversial" etc

Kind isn't taken into consideration. Kind would typically be "votes", "downloads", "likes" etc.

Parameter Type Description
path Required String

Path to resources.
kind Required String

Kind to count.
segments Required String

A list of fields to fetch scores from. See example.
limit Required Integer

Maximum number of results.
shuffle Boolean

Set to true in order to randomize samples.
Response Description
200 JSON

Kudu

Build Status

Feedback service.

Ack

A single piece of feedback is known as an ack.

Kudu supports flexible feedback strategies (kind), e.g. * classic kudos ("+1") * votes: upvote, downvote, and neutral ("+1", "-1", and "0"), * ratings: e.g. 1 through 5 stars ("+1", "+2", "+3", "+4", "+5") * arbitrary scores, e.g. -17, +32, +100

An Ack is provided by an identity for some object, as identified by a uid. It also must specify a kind, which is essentially a feedback strategy defined in the application, and a value.

For example:

Ack.new(:identity => 7, :external_uid => 'post:a.b.c$34', :kind => 'rating', :value => 4)

Endpoints

Get acks for the current identity. :uids can be a comma delimited list of uids, or a single uid.

GET /acks/:uids

Post feedback for an item:

POST /acks/:uid

:kind is required, and must be a valid label. :value is required and must be an integer. :identity is required and is retrieved from checkpoint using the session key. :external_uid is also required.

Scores

The aggregate scores for each :uid are available through the /scores endpoints.

Endpoints

All scores for an object, grouped by :kind

GET /scores/:uid

All scores of a given kind:

GET /scores/:uid/:kind

Fetch ranked lists. These need to be ranked by an attribute on score. uid includes a wildcard path.

GET /scores/:uid/:kind/rank/:by

Fetch mixes of ranked, randomized scores, segmented by different groups. Go look at the code. Seriously.

GET /scores/:uid/sample

Stats

TODO: implement endpoints at /stats/:path/:more_stuff.

These aggregate data about :scores and :kind (can't aggregate scores of different kinds. Apples and Oranges).

This will deliver stats for paths (i.e. an app, or a region). Currently this is sort of implemented with some hacks, for dittforslag (how many contributors, top contributors, ranked lists of various things -- most controversial, most popular, etc.

This requires support for wildcard paths, which is on the block for the next few days.

GET /api/kudu/v1/acks/:uid/:kind/count

Get the score count for resources with a specific kind.

The count is the sum of total_count all scores for a give UID.

Kind is typically "votes", "downloads", "likes" etc.

Parameter Type Description
uid Required String

UID denoting a resource, or a wildcard UID indicating a collection of resources.
kind Required String

Kind to count.
Response Description
200 JSON

Tiramisu

Build Status

A service for managing user-originated uploads and asset-processing.

Uses S3 for storage, Tootsie for transcoding and ImageMagick for file identification.

Getting Started

git clone git@github.com:pebblestack/tiramisu.git tiramisu

Make sure you have ImageMagick installed via your favorite package manager. Confirm that you are able to run identify on the command line.

Config

Your configuration goes in config/services.yml.

cp config/services-example.yml config/services.yml

You have one configuration for each environment.

Tiramisu uses the environment variable RACK_ENV to determine current environment.

---
development:
  S3:
    access_key_id: <aws access key>
    secret_access_key: <aws secret>
    bucket: <your bucket name>

production:
  ...

Api

Upload an image by posting multipart form-data with the asset in the field "file". E.g

curl -F "file=@mypicture.jpg" "http://tiramisu.dev/api/tiramisu/v1/images/:uid"

The :uid is in the form of a partial pebbles-uid without the object id. In other words, something like this:

`image:myclient.myapp.kittens`

myclient.myapp.kittens is used to scope the image by client, service, user or other criteria. It is a '.'-separated string of labels.

If the operation succeeds a complete uid is returned with a generated object id, like this:

`image:myclient.myapp.kittens$20120105134610-1498-bp5o`

If the image format is not supported, the action fails with 400 and the content {"error":"format-not-supported"}, if processing times out it fails with the content {"error":"timeout"}.

This action will write progress updates to a streamed HTTP response while submitting to tootsie/uploading image to S3. Clients may listen for changes to the response in order to provide progress feedback to the user.

If you require processing notifications from tootsie, specify the query-parameter notification_url. The format of these messages is documented in Tootsie's README

curl -F "file=@mypicture.jpg" "http://tiramisu.dev/api/tiramisu/v1/images/:uid?notification_url=..."

Typical response:

{"percent":0,"status":"received"}
{"percent":20,"status":"transferring"}
// ...
{"percent":75,"status":"transferring"}
{"percent":90,"status":"transferring"}
{"percent":100,"status":"completed","image": { // NOTE: formatted on mulitple lines for readability. In reality, this is a one-liner
    "id":"image:myclient.myapp.something$20120105134610-1498-bp5o",
    "baseurl":"http://yourbucketname.s3.amazonaws.com/myclient/myapp/something/20120105134610-1498-bp5o",
    "sizes":[
        {
            "width":100,
            "square":true,
            "url":"http://yourbucketname.s3.amazonaws.com/myclient/myapp/something/20120105134610-1498-bp5o/100.jpg"
        },
        // ... other sizes
        {
            "width":5000,
            "square":false,
            "url":"http://yourbucketname.s3.amazonaws.com/myclient/myapp/something/20120105134610-1498-bp5o/5000.jpg"
        }
    ],
    "original":"http://yourbucketname.s3.amazonaws.com/myclient/myapp/something/20120105134610-1498-bp5o/original.jpeg",
    "aspect":1.4988290398126465
    }
}

If the transaction fails, progress reports {"progress": 100, "status":"failed", "message": "<error message>"} and closes the connection.

NOTE: the sizes hash for an image after a successfull post request does not indicate whether the image is ready to be displayed or not (that is, processed by tootsie and available at the S3 URL).

The client can verify that the image is present by trying to fetch the url, and checking if that request fails or not.

This is best implemented in javascript using an image object of which we listen for load or error events. Example implementation with jQuery:

var poll = function (url) {
  var retries = 0;
  var loader = new Image();
  var attempt = function() {
    jQuery(loader).one('load', function () {
      // image is ready
    });
    jQuery(loader).one('error', function () {
      // image is not ready continue polling
      setTimeout(attempt, 1000);
    });
    loader.src = url + "?retry=" + retries++; // Need a different url every time, cause Opera will cache it even if it fails
  };
  attempt();
};

Known issues/TODO

Deployment

When configuring this service for production mind the following points:

POST /api/tiramisu/v1/files/:uid

Post a job to store a document.

Parameter Type Description
uid Required String

The partial Pebbles uid (species:path, without oid).
file Required File

Multipart form field containing the file to upload.
Response Description
200 A stream of JSON objects that describe the status of the transfer. When status is 'completed' an additional key, 'metadata' will be present containing full uid, and path to the file. On error, the response will be JSON containing the error message. The status will always be 200.

POST /api/tiramisu/v1/audio_files

Post a job to transcode and store an audio file.

Parameter Type Description
uid Required String

The partial Pebbles Uid (species:path, without oid).
Response Description
200 A stream of JSON objects that describe the status of the transfer. When status is 'completed', an additional key, 'metadata' will be present containing data about the transcoded formats urls of transcoded files.

POST /api/tiramisu/v1/images/:uid

Post a job to scale and store an image.

Parameter Type Description
uid Required String

The partial Pebbles uid (species:path, without oid).
file Required File

Multipart form field containing the image to upload.
notification_url String

The endpoint where you wish to receive notification when the transfer and scaling job has been completed.
force_jpeg boolean

Will convert all sizes/versions to jpeg. Defaults to true. Set this to false in order to support gif and png transcoding
force_orientation string

Will force set an orientation on image
Response Description
200 A stream of JSON objects that describe the status of the transfer. When status is 'completed' an additional key, 'metadata' will be present containing the full uid as well as information about sizes, aspect ratio, and the paths to the stored images. On error, the response will be JSON containing the error message. The status will always be 200.

DELETE /api/tiramisu/v1/images/:path

Delete a single image from S3.

Parameter Type Description
path Required String

The whole image URL minus hostname, e.g. apdm/oa/kittens/20160208121232-809-a5x0/original.jpg.
Response Description
200 [JSON] {deleted: true} if the delete operation succeeded, else an error message.

GET /api/tiramisu/v1/audio_files/:uid/status

Get transcoding status of uploaded audio file.

Parameter Type Description
uid Required String

The complete Pebbles uid (including oid).
Response Description
200 A JSON structure that is exactly like the metadata key returned from the upload endpoint, except with an appended 'ready' key which is true if the transcoding is completed or false if it is in progress.

Snitch

Build Status

A service to track reports of objectionable content and the corresponding moderator decisions

Getting Started

git clone git@github.com:bengler/snitch.git
cd snitch

rake db:bootstrap

GET /api/snitch/v1/items/:uid

Return a paginated list of unprocessed reported resources (i.e. for a moderator to review).

You may also input a list of full UIDs, e.g GET /items/a.b.c$1,a.b.c$2,a.b.c$3. For this type of query pagination will not be possible as it always returns the exact items and in the exact order as the input uid list. If an item is not found, it will output a null item in the same position as the input.

Parameter Type Description
scope String

The scope of the reports to fetch. Must be any of "pending" (any reported items that have no registered decision), "processed" (Items that have been decided upon), "reported" (all reported items, including items that have recieved a decision), "fresh" (any fresh content that has not been marked as seen by a moderator). If not specified. Default scope is "pending".
sort_by String

Any of "created_at", "updated_at" or "action_at" Defaults to "created_at".
order String

Either "asc" or "desc". Defaults to "desc".
offset Integer

Index of the first returned hit. Defaults to 0.
limit Number

Maximum number of returned hits. Defaults to 10.

POST /api/snitch/v1/items/:uid

Notify snitch of the existence of a resource

This is used to notify snitch of the existence of a new resource (i.e. for moderating resources as it is submitted) To let snitch know a moderator has seen an item, post an action of the kind "seen" (any other action will also mark the item as seen).

:uid is eventually stored as :external_uid on the Item

POST /api/snitch/v1/reports/:uid

Report a resource as objectionable.

Uses checkpoint to attach a user to the report, but accepts anonymous reports.

Parameter Type Description
uid Required String

Pebbles uid denoting a resource
kind String

Application-specific tag to denote kind of objection
comment String

A comment from the reporting user
Response Description
200 OK

GET /api/snitch/v1/items/:uid/count

Return count on how many items there arefor a wildcard uid

Parameter Type Description
scope String

The scope of the reports to fetch. Must be any of "pending" (any reported items that have no registered decision), "processed" (Items that have been decided upon), "reported" (all reported items, including items that have recieved a decision), "fresh" (any fresh content that has not been marked as seen by a moderator). If not specified. Default scope is "pending".

POST /api/snitch/v1/items/:uid/reset

Resets an item's seen status

This is useful for items that have been seen, then edited later on, and should be reviewed by a moderator again.

Only the seen status is reset. Decisions, actions and reports are kept for the item.

POST /api/snitch/v1/items/:uid/actions

Register a moderator decision

Actual removal of content is not performed by snitch, but this will remove the item from the default list returned by GET /items. Currently the user reporting a decision must be god of the given realm, but this should be considered a temporary solution until a proper concept of "moderators" is in place. Both the decision and the decider is registered with the item in question.

Requires moderator access

Parameter Type Description
uid Required String

A full or a wildcard uid for what to register desicion on.
action Required JSON

action[kind] Required String

One of "kept", "removed", "seen", "edited", "recommended" or "recommendation_revoked".
action[rationale] String

Optionally you can provide a "rationale" label to explain the reason for the action. Example rationales could be "irrelevant", "adhominem", "hatespeech", "legal", etc.
action[message] String

An additional human readable explanation of the action. This should be directed at the offender as this message in the future may be provided to the original poster.

GET /api/snitch/v1/items/:uid/actions

Get lists of moderator decisions this will only go as far back as 30 days for performance reasons. By passing a date to the parameter :since you may page even further back.

Returns a paginated list of recent actions on items matching the wildcard uid sorted by date. By default

GET /api/snitch/v1/items/:uid/reports

Get a list of submitted reports for the given item

Returns a paginated list of submitted reports for the given item

Parameter Type Description
uid Required String

Pebbles uid denoting a resource

Sherlock

  ____  _               _            _    
 / ___|| |__   ___ _ __| | ___   ___| | __
 \___ \| '_ \ / _ \ '__| |/ _ \ / __| |/ /
  ___) | | | |  __/ |  | | (_) | (__|   <
 |____/|_| |_|\___|_|  |_|\___/ \___|_|\_\

Build Status

Sherlock is a no-hassle service which handles all the indexing and query needs of a content-brokering Pebble. For example, consider using Sherlock if you need to full-text query Grove content.

Warning: Experimental software

Sherlock is currently used by several pebbles in production, but the API is still under continous development so expect both behavior and interfaces to change.

Sherlock has three interfaces

  1. AMQP. Sherlock subscribes to a message queue. Incoming messages are evaluated, normalized and passed via…
  2. ...HTTP requests to Elasticsearch for indexing.
  3. Accept client HTTP search queries, translate the query, pass it to Elasticsearch and route the search result back to the client as an HTTP response.

Installation

Sherlock needs two external services to function:

  1. RabbitMQ
  2. Elasticsearch

RabbitMQ

brew install rabbitmq

Elasticsearch

brew install elasticsearch

Stop elasticsearch

launchctl unload -wF ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch.plist

Add these lines to your elasticsearch.yml ('brew info elasticsearch' to locate the file):

# Turn off automatic index creation
action.auto_create_index: false

Start elasticsearch

launchctl load -wF ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch.plist

Sherlock

git clone git@github.com:bengler/sherlock.git
cd sherlock
bundle install

Run the integration tests to see if Sherlock is playing well with RabbitMQ and Elasticsearch

rspec spec/integration/

Start the update listener

./bin/update_listener start --daemon

Usage

Index something:

curl -XPUT 'http://localhost:9200/an_index_of_books/book/1' -d '{"title":"Island","author":"Aldous Huxley", "year_of_publication":"1962"}'

Also, anything added to grove, or updated in grove, will be put on the River, which Sherlock then picks up and sends to Elasticsearch for indexing.

You can now do queries such as:

http://sherlock.dev/api/sherlock/v1/search/my_index/*:*/?q=foo+bar
http://sherlock.dev/api/sherlock/v1/search/my_index/*:*
http://sherlock.dev/api/sherlock/v1/search/development/post.greeting:*?limit=100

Predefined mappings

Some use-cases require predefined mappings. E.g. we don't want a UID to be string-tokenized because this will cause dash to look like a whitespace and thus not return hits on a query.

If you need to predefine how ES analyzes a particular part of you data, do so by appending your stuff to config/predefined_es_mappings.json.

Other handy stuff

Drop all indexes

./bin/sherlock drop_all_indices

Empty all queues of messages

./bin/sherlock empty_all_queues

...or you can also scratch the queue itself with

rabbitmqadmin purge queue=response.report.river

Test how text is analyzed

curl -XGET 'localhost:9200/development_dna/_analyze' -d 'as sly as a fox'

Type mapping of fields in an index

curl -XGET 'http://localhost:9200/sherlock_development_apdm/_mapping'

GET /api/sherlock/v1/mapping/:realm/?:klass?

Get Elasticsearch mappings.

Take a look at how Elasticsearch has chosen to index data

Parameter Type Description
realm Required String

Name of realm containing the searchable data.
klass String

Name of a specific indexed datatype.
Response Description
200 JSON

Sherlock

  ____  _               _            _    
 / ___|| |__   ___ _ __| | ___   ___| | __
 \___ \| '_ \ / _ \ '__| |/ _ \ / __| |/ /
  ___) | | | |  __/ |  | | (_) | (__|   <
 |____/|_| |_|\___|_|  |_|\___/ \___|_|\_\

Build Status

Sherlock is a no-hassle service which handles all the indexing and query needs of a content-brokering Pebble. For example, consider using Sherlock if you need to full-text query Grove content.

Warning: Experimental software

Sherlock is currently used by several pebbles in production, but the API is still under continous development so expect both behavior and interfaces to change.

Sherlock has three interfaces

  1. AMQP. Sherlock subscribes to a message queue. Incoming messages are evaluated, normalized and passed via…
  2. ...HTTP requests to Elasticsearch for indexing.
  3. Accept client HTTP search queries, translate the query, pass it to Elasticsearch and route the search result back to the client as an HTTP response.

Installation

Sherlock needs two external services to function:

  1. RabbitMQ
  2. Elasticsearch

RabbitMQ

brew install rabbitmq

Elasticsearch

brew install elasticsearch

Stop elasticsearch

launchctl unload -wF ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch.plist

Add these lines to your elasticsearch.yml ('brew info elasticsearch' to locate the file):

# Turn off automatic index creation
action.auto_create_index: false

Start elasticsearch

launchctl load -wF ~/Library/LaunchAgents/homebrew.mxcl.elasticsearch.plist

Sherlock

git clone git@github.com:bengler/sherlock.git
cd sherlock
bundle install

Run the integration tests to see if Sherlock is playing well with RabbitMQ and Elasticsearch

rspec spec/integration/

Start the update listener

./bin/update_listener start --daemon

Usage

Index something:

curl -XPUT 'http://localhost:9200/an_index_of_books/book/1' -d '{"title":"Island","author":"Aldous Huxley", "year_of_publication":"1962"}'

Also, anything added to grove, or updated in grove, will be put on the River, which Sherlock then picks up and sends to Elasticsearch for indexing.

You can now do queries such as:

http://sherlock.dev/api/sherlock/v1/search/my_index/*:*/?q=foo+bar
http://sherlock.dev/api/sherlock/v1/search/my_index/*:*
http://sherlock.dev/api/sherlock/v1/search/development/post.greeting:*?limit=100

Predefined mappings

Some use-cases require predefined mappings. E.g. we don't want a UID to be string-tokenized because this will cause dash to look like a whitespace and thus not return hits on a query.

If you need to predefine how ES analyzes a particular part of you data, do so by appending your stuff to config/predefined_es_mappings.json.

Other handy stuff

Drop all indexes

./bin/sherlock drop_all_indices

Empty all queues of messages

./bin/sherlock empty_all_queues

...or you can also scratch the queue itself with

rabbitmqadmin purge queue=response.report.river

Test how text is analyzed

curl -XGET 'localhost:9200/development_dna/_analyze' -d 'as sly as a fox'

Type mapping of fields in an index

curl -XGET 'http://localhost:9200/sherlock_development_apdm/_mapping'

GET /api/sherlock/v1/search/:realm/:uid

Search indexed data.

Search indexed data using various parameters. As in all pebbles data is scoped by realm.

Documents with restricted=true is only accessible by god sessions or users with checkpoint-specified access to that path.

Parameter Type Description
realm Required String

Name of realm containing the searchable data.
uid String

uid denoting a resource, or a wildcard uid indicating a collection of resources.
q String

Query string.
limit Integer

Maximum number of returned hits. Defaults to 10.
offset Integer

Index of the first returned hit. Defaults to 0.
sort_by String

Document attribute to sort the result set by. Defaults to an internally calculated relevancy score. Seperate multiple fields with comma.
order String

Order in which to sort the returned hits. Defaults to DESC. Supports multiple fields sorting with individual order by seperating by comma in the same order as sort_by.
range[attribute] String

Attribute to perform a ranged query by. DEPRECATED, use min/max.
range[from] String

Minimum accepted value for a ranged query. DEPRECATED, use min/max.
range[to] String

Maximum accepted value for a ranged query. DEPRECATED, use min/max.
min[name_of_attribute] String

Minimum accepted value for a ranged query.
max[name_of_attribute] String

Maximum accepted value for a ranged query.
fields[name_of_attribute] String

Require a named attribute to have a specific value. Use "null" to indicate a missing value. Use 'value1|value2' to indicate 'or'.
deleted String

How to treat the deleted attribute. Accepts 'include' or 'only'. Default is to not include these records. Getting a deleted record requires access to its path.
unpublished String

How to treat unpublished records in the result. Accepts 'include' or 'only'. Default is to not include these records. Getting an unpublished record requires access to its path.
nocache Boolean

Bypass cache for guest users. Default is false.
return_fields String

Return only these named fields, separated by comma. E.g. "realm,document.author,updated_at"
Response Description
200 JSON

POST /api/sherlock/v1/search/:realm/:uid

Search indexed data. Same params as the GET endpoint

Response Description
200 JSON

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/:uid

Find one or more affiliation.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

POST /api/origami/v1/:realm/affiliations

Create a affiliation.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

PUT /api/origami/v1/:realm/affiliations/:identifier

Update a affiliation.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label (alphanumeric) that uniquely identifies this affiliation.
Response Description
201 Successfully created.
403 Unauthorized.

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/:uid

Find one or more associate.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

POST /api/origami/v1/:realm/associates

Create an associate.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

PUT /api/origami/v1/:realm/associates/:identifier

Update an associate.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label (alphanumeric) that uniquely identifies this associate.
Response Description
201 Successfully created.
403 Unauthorized.

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

POST /:realm/associates/:id_or_label/tags/:tags

Add tags for an associate.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label of associate
tags Required String

A comma separated list of tags to set.
Response Description
200 JSON.
403 Unauthorized.
404 Associate not found in realm

PUT /:realm/associates/:id_or_label/tags/:tags

Replace tags for an associate.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label of associate
tags Required String

A comma separated list of tags to set.
Response Description
200 JSON.
403 Unauthorized.
404 Associate not found in realm

DELETE /:realm/associates/:id_or_label/tags/:tags

Remove tags for a post.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label of associate
tags Required String

A comma separated list of tags to remove.
Response Description
200 JSON.
403 Unauthorized.
404 Associate not found in realm

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/:uid

Find one or more capacity.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

POST /api/origami/v1/:realm/capacities

Create a capacity.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

DELETE /api/origami/v1/:realm/capacities/:id

Delete a capacity.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id Required Integer

The numeric id of the capacity.
Response Description
204 Successfully deleted.
403 Unauthorized.
404 No such capacity.

PUT /api/origami/v1/:realm/capacities/:identifier

Update a capacity.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label (alphanumeric) that uniquely identifies this capacity.
Response Description
201 Successfully created.
403 Unauthorized.

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/:uid

Find one or more group.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

POST /api/origami/v1/:realm/groups

Create a group.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

GET /api/origami/v1/:realm/:uid/capacities

Find all capacities in a group

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid of the group
limit Integer

Max results to return
offset Integer

The index of the first result to return
Response Description
200 JSON

PUT /api/origami/v1/:realm/groups/:identifier

Update a group.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label (alphanumeric) that uniquely identifies this group.
Response Description
201 Successfully created.
403 Unauthorized.

GET /api/origami/v1/:realm/:unit_uid/capacities

Find all capacities in a unit

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid of the unit
limit Integer

Max results to return
offset Integer

The index of the first result to return
Response Description
200 JSON

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

POST /api/origami/v1/:realm/organizations

Create an organization.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/resources/:uid

A type-neutral query. Provide any valid uid-query for any kind of Origami resource. E.g. list a number of objects of varying types and have them all returned together here.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid, a comma separated list of uids or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

DELETE /api/origami/v1/:realm/resources/:uid

A type-neutral delete. Provide any valid uid-query for any kind of Origami resource. E.g. list a number of objects of varying types and have them all deleted together here.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid, a comma separated list of uids. Wildcard uids are rejected for safety.
Response Description
200 JSON
403 Unauthorized.

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/:uid

Find one or more role.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

POST /api/origami/v1/:realm/roles

Create a role.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

PUT /api/origami/v1/:realm/roles/:identifier

Update a role.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label (alphanumeric) that uniquely identifies this role.
Response Description
201 Successfully created.
403 Unauthorized.

Origami

Build Status

A service to handle data on organizations and the people associated with them.

Dependencies

Basic Object Model

Organization

Unit

A business unit or organizational unit. These are organized in a hierarchy (mechanism TBD).

Group

A group within a Unit. Typically a comittee or workgroup.

Associate

A person who is associated with a realm. They could be members or volunteers or employees or students or whatnot. The type of associate is qualified through affiliations, groups, and capacities.

Affiliation

Describes the relationship between an Associate and an Organization. Typically used to model membership or employee status. A given Associate can have any number of Affiliations with an Organization, e.g. both member and employee.

Capacity

A person is associated with a specific unit in the organization via a role. This role is a simple label. Roles do not need to have counterparts in the Role table.

Role

A Role is a fleshed out description of a role in the realm.

Origami API

organization

units

associates

There is currently no checking for uniqueness across the realm. I.e. you can have as many Anne Olsen as you like.

capacities

There is no PUT for capacities. You would have to identify it by :id, because associates may have multiple capacities (both different roles in the same group, and same role in different groups). If you want to update one, then delete the old one and make a new one.

Role

Maintenance

Imports

When you do an import, a batch gets created, and each entry in the import is a batch item.

To protect against multiple simultaneous imports for a source and realm, we use a batch proxy of which there can only be one.

When the import is completed, the batch proxy still exists, so no further imports can be performed.

When the batch is flushed (moved from batch items into the actual organization hierarchy), then the batch proxy is deleted, and another import can be performed.

(sorry about the braindump format of this import note).

Getting started, development wise

createuser origami
createdb -O origami origami_development
bundle install
rake db:migrate
rake db:test:prepare
rspec spec

GET /api/origami/v1/:realm/:uid

Find one or more unit.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid or a wildcard uid search
Response Description
200 JSON
403 Unauthorized.

POST /api/origami/v1/:realm/units

Create a unit.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
Response Description
201 Successfully created.
403 Unauthorized.

GET /api/origami/v1/:realm/:uid/hierarchy

Get a subtree starting at a given unit.

Restricted to God users. Also: Extremely slow. Should be re-implemented to use paths.

Parameter Type Description
realm Required String

The Checkpoint security realm.
uid Required String

The pebbles uid for the unit.
Response Description
200 JSON
403 Unauthorized.

PUT /api/origami/v1/:realm/units/:identifier

Update a unit.

Restricted to God users.

Parameter Type Description
realm Required String

The Checkpoint security realm.
id_or_label Required String

The id (numeric) or label (alphanumeric) that uniquely identifies this unit.
Response Description
201 Successfully created.
403 Unauthorized.