granary package

Reference documentation.

as2

Convert between ActivityStreams 1 and 2.

AS2: http://www.w3.org/TR/activitystreams-core/

AS1: http://activitystrea.ms/specs/json/1.0/
http://activitystrea.ms/specs/json/schema/activity-schema.html
granary.as2.from_as1(obj, type=None, context=u'https://www.w3.org/ns/activitystreams')[source]

Converts an ActivityStreams 1 activity or object to ActivityStreams 2.

Parameters:
  • obj – dict, AS1 activity or object/
  • type – string, default type if type inference can’t determine a type.
  • context – string, included as @context

Returns: dict, AS2 activity or object

granary.as2.to_as1(obj, use_type=True)[source]

Converts an ActivityStreams 2 activity or object to ActivityStreams 1.

Parameters:
  • obj – dict, AS2 activity or object
  • use_type – boolean, whether to include objectType and verb

Returns: dict, AS1 activity or object

atom

Convert between ActivityStreams 1 and Atom.

Atom spec: https://tools.ietf.org/html/rfc4287 (RIP atomenabled.org)

class granary.atom.Defaulter(init={})[source]

Bases: collections.defaultdict

__init__(init={})[source]

x.__init__(…) initializes x; see help(type(x)) for signature

__hash__() <==> hash(x)[source]
__weakref__

list of weak references to the object (if defined)

granary.atom.activities_to_atom(activities, actor, title=None, request_url=None, host_url=None, xml_base=None, rels=None, reader=True)[source]

Converts ActivityStreams 1 activities to an Atom feed.

Parameters:
  • activities – list of ActivityStreams activity dicts
  • actor – ActivityStreams actor dict, the author of the feed
  • title – string, the feed <title> element. Defaults to ‘User feed for [NAME]’
  • request_url – the URL of this Atom feed, if any. Used in a link rel=”self”.
  • host_url – the home URL for this Atom feed, if any. Used in the top-level feed <id> element.
  • xml_base – the base URL, if any. Used in the top-level xml:base attribute.
  • rels – rel links to include. dict mapping string rel value to string URL.
  • reader – boolean, whether the output will be rendered in a feed reader. Currently just includes location if True, not otherwise.
Returns:

unicode string with Atom XML

granary.atom.activity_to_atom(activity, xml_base=None, reader=True)[source]

Converts a single ActivityStreams 1 activity to an Atom entry.

Kwargs are passed through to activities_to_atom().

Parameters:
  • xml_base – the base URL, if any. Used in the top-level xml:base attribute.
  • reader – boolean, whether the output will be rendered in a feed reader. Currently just includes location if True, not otherwise.
Returns:

unicode string with Atom XML

granary.atom.atom_to_activities(atom)[source]

Converts an Atom feed to ActivityStreams 1 activities.

Parameters:atom – unicode string, Atom document with top-level <feed> element
Returns:list of ActivityStreams activity dicts
granary.atom.atom_to_activity(atom)[source]

Converts an Atom entry to an ActivityStreams 1 activity.

Parameters:atom – unicode string, Atom document with top-level <entry> element
Returns:dict, ActivityStreams activity
granary.atom.html_to_atom(html, url=None, fetch_author=False, reader=True)[source]

Converts microformats2 HTML to an Atom feed.

Parameters:
  • html – string
  • url – string URL html came from, optional
  • fetch_author – boolean, whether to make HTTP request to fetch rel-author link
  • reader – boolean, whether the output will be rendered in a feed reader. Currently just includes location if True, not otherwise.
Returns:

unicode string with Atom XML

facebook

Facebook source class. Uses the Graph API.

https://developers.facebook.com/docs/graph-api/using-graph-api/

The Audience Targeting ‘to’ field is set to @public or @private based on whether the Facebook object’s ‘privacy’ field is ‘EVERYONE’ or anything else. https://developers.facebook.com/docs/reference/api/privacy-parameter/

Retrieving @all activities from get_activities() (the default) currently returns an incomplete set of activities, ie NOT exactly the same set as your Facebook News Feed: https://www.facebook.com/help/327131014036297/

This is complicated, and I still don’t fully understand how or why they differ, but based on lots of experimenting and searching, it sounds like the current state is that you just can’t reproduce the News Feed via Graph API’s /me/home, FQL’s stream table, or any other Facebook API, full stop. :(

Random details:

See the fql_stream_to_post() method below for code I used to experiment with the FQL stream table.

class granary.facebook.FacebookId(user, post, comment)

Bases: tuple

comment

Alias for field number 2

post

Alias for field number 1

user

Alias for field number 0

class granary.facebook.Facebook(access_token=None, user_id=None)[source]

Bases: granary.source.Source

Facebook source class. See file docstring and Source class for details.

access_token

string, optional, OAuth access token

user_id

string, optional, current user’s id (either global or app-scoped)

__init__(access_token=None, user_id=None)[source]

Constructor.

If an OAuth access token is provided, it will be passed on to Facebook. This will be necessary for some people and contact details, based on their privacy settings.

Parameters:
  • access_token – string, optional OAuth access token
  • user_id – string, optional, current user’s id (either global or app-scoped)
get_actor(user_id=None)[source]

Returns a user as a JSON ActivityStreams actor dict.

Parameters:user_id – string id or username. Defaults to ‘me’, ie the current user.
get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, fetch_news=False, event_owner_id=None, **kwargs)[source]

Fetches posts and converts them to ActivityStreams activities.

See method docstring in source.py for details.

Likes, top-level replies (ie comments), and reactions are always included. They come from the ‘comments’, ‘likes’, and ‘reactions’ fields in the Graph API’s Post object: https://developers.facebook.com/docs/reference/api/post/

Threaded comments, ie comments in reply to other top-level comments, require an additional API call, so they’re only included if fetch_replies is True.

Mentions are never fetched or included because the API doesn’t support searching for them. https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875

Additional args:
fetch_news: boolean, whether to also fetch and include Open Graph news
stories (/USER/news.publishes). Requires the user_actions.news permission. Background in https://github.com/snarfed/bridgy/issues/479
event_owner_id: string. if provided, only events owned by this user id
will be returned. avoids (but doesn’t entirely prevent) processing big non-indieweb events with tons of attendees that put us over app engine’s instance memory limit. https://github.com/snarfed/bridgy/issues/77
get_event(event_id, owner_id=None)[source]

Returns a Facebook event post.

Parameters:
  • id – string, site-specific event id
  • owner_id – string
Returns:

dict, decoded ActivityStreams activity, or None if the event is not found or is owned by a different user than owner_id (if provided)

get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]

Returns an ActivityStreams comment object.

Parameters:
  • comment_id – string comment id
  • activity_id – string activity id, optional
  • activity_author_id – string activity author id, optional
  • activity – activity object (optional)
get_share(activity_user_id, activity_id, share_id, activity=None)[source]

Returns an ActivityStreams share activity object.

Parameters:
  • activity_user_id – string id of the user who posted the original activity
  • activity_id – string activity id
  • share_id – string id of the share object
  • activity – activity object (optional)
get_albums(user_id=None)[source]

Fetches and returns a user’s photo albums.

Parameters:user_id – string id or username. Defaults to ‘me’, ie the current user.
Returns:sequence of ActivityStream album object dicts
get_reaction(activity_user_id, activity_id, reaction_user_id, reaction_id, activity=None)[source]

Fetches and returns a reaction.

Parameters:
  • activity_user_id – string id of the user who posted the original activity
  • activity_id – string activity id
  • reaction_user_id – string id of the user who reacted
  • reaction_id – string id of the reaction. one of: ‘love’, ‘wow’, ‘haha’, ‘sad’, ‘angry’
  • activity – activity object (optional)
create(obj, include_link=u'omit', ignore_formatting=False)[source]

Creates a new post, comment, like, or RSVP.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose contents will be a dict with ‘id’ and ‘url’ keys for the newly created Facebook object (or None)

preview_create(obj, include_link=u'omit', ignore_formatting=False)[source]

Previews creating a new post, comment, like, or RSVP.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose contents will be a unicode string HTML snippet or None

create_notification(user_id, text, link)[source]

Sends the authenticated user a notification.

Uses the Notifications API (beta): https://developers.facebook.com/docs/games/notifications/#impl

Parameters:
  • user_id – string, username or user ID
  • text – string, shown to the user in the notification
  • link – relative string URL, the user is redirected here when they click on the notification. Note that only the path and query parameters are used! they’re combined with the domain in your Facebook app’s Game App URL: https://developers.facebook.com/docs/games/services/appnotifications#parameters

Raises: urllib2.HTPPError

post_url(post)[source]

Returns a short Facebook URL for a post.

Parameters:post – Facebook JSON post
comment_url(post_id, comment_id, post_author_id=None)[source]

Returns a short Facebook URL for a comment.

Parameters:
  • post_id – Facebook post id
  • comment_id – Facebook comment id
classmethod base_id(url)[source]

Guesses the id of the object in the given URL.

Returns:string, or None
base_object(obj, verb=None, resolve_numeric_id=False)[source]

Returns the ‘base’ silo object that an object operates on.

This is mostly a big bag of heuristics for reverse engineering and parsing Facebook URLs. Whee.

Parameters:
  • obj – ActivityStreams object
  • verb – string, optional
  • resolve_numeric_id – if True, tries harder to populate the numeric_id field by making an additional API call to look up the object if necessary.
Returns:

dict, minimal ActivityStreams object. Usually has at least id, numeric_id, and url fields; may also have author.

post_to_activity(post)[source]

Converts a post to an activity.

Parameters:post – dict, a decoded JSON post
Returns:an ActivityStreams activity dict, ready to be JSON-encoded
post_to_object(post, type=None)[source]

Converts a post to an object.

TODO: handle the sharedposts field

Parameters:
  • post – dict, a decoded JSON post
  • type – string object type: None, ‘post’, or ‘comment’
Returns:

an ActivityStreams object dict, ready to be JSON-encoded

comment_to_object(comment, post_id=None, post_author_id=None)[source]

Converts a comment to an object.

Parameters:
  • comment – dict, a decoded JSON comment
  • post_id – optional string Facebook post id. Only used if the comment id doesn’t have an embedded post id.
  • post_author_id – optional string Facebook post author id. Only used if the comment id doesn’t have an embedded post author id.
Returns:

an ActivityStreams object dict, ready to be JSON-encoded

share_to_object(share)[source]

Converts a share (from /OBJECT/sharedposts) to an object.

Parameters:share – dict, a decoded JSON share
Returns:an ActivityStreams object dict, ready to be JSON-encoded
user_to_actor(user)[source]

Converts a user or page to an actor.

Parameters:user – dict, a decoded JSON Facebook user or page
Returns:an ActivityStreams actor dict, ready to be JSON-encoded
event_to_object(event, rsvps=None)[source]

Converts an event to an object.

Parameters:
  • event – dict, a decoded JSON Facebook event
  • rsvps – sequence, optional Facebook RSVPs
Returns:

an ActivityStreams object dict

event_to_activity(event, rsvps=None)[source]

Converts a event to an activity.

Parameters:
  • event – dict, a decoded JSON Facebook event
  • rsvps – list of JSON Facebook RSVPs
Returns:

an ActivityStreams activity dict

rsvp_to_object(rsvp, type=None, event=None)[source]

Converts an RSVP to an object.

The ‘id’ field will ony be filled in if event[‘id’] is provided.

Parameters:
  • rsvp – dict, a decoded JSON Facebook RSVP
  • type – optional Facebook RSVP type, one of RSVP_FIELDS
  • event – Facebook event object. May contain only a single ‘id’ element.
Returns:

an ActivityStreams object dict

album_to_object(album)[source]

Converts a photo album to an object.

Parameters:album – dict, a decoded JSON Facebook album
Returns:an ActivityStreams object dict
privacy_to_to(obj, type=None)[source]

Converts a Facebook privacy field to an ActivityStreams to field.

privacy is sometimes an object: https://developers.facebook.com/docs/graph-api/reference/post#fields

…and other times a string: https://developers.facebook.com/docs/graph-api/reference/album/#readfields

Parameters:obj – dict, Facebook object (post, album, comment, etc)
Returns:ActivityStreams to object, or None if unknown type: string object type: None, ‘post’, or ‘comment’
Return type:dict
fql_stream_to_post(stream, actor=None)[source]

Converts an FQL stream row to a Graph API post.

Currently unused and untested! Use at your own risk.

https://developers.facebook.com/docs/technical-guides/fql/ https://developers.facebook.com/docs/reference/fql/stream/

TODO: place, to, with_tags, message_tags, likes, comments, etc., most require extra queries to inflate.

Parameters:
  • stream – dict, a row from the FQL stream table
  • actor – dict, a row from the FQL profile table
Returns:

dict, Graph API post

Here’s example code to query FQL and pass the results to this method:

resp = self.urlopen('https://graph.facebook.com/v2.0/fql?' + urllib.urlencode(
    {'q': json.dumps({
      'stream': '''\
        SELECT actor_id, post_id, created_time, updated_time,
          attachment, privacy, message, description
        FROM stream
        WHERE filter_key IN (
          SELECT filter_key FROM stream_filter WHERE uid = me())
        ORDER BY created_time DESC
        LIMIT 50
        ''',
      'actors': '''\
        SELECT id, name, username, url, pic FROM profile WHERE id IN
          (SELECT actor_id FROM #stream)
        '''})}))

results = {q['name']: q['fql_result_set'] for q in resp['data']}
actors = {a['id']: a for a in results['actors']}
posts = [self.fql_stream_to_post(row, actor=actors[row['actor_id']])
         for row in results['stream']]
static parse_id(id, is_comment=False)[source]

Parses a Facebook post or comment id.

Facebook ids come in different formats:

  • Simple number, usually a user or post: 12
  • Two numbers with underscore, usually POST_COMMENT or USER_POST: 12_34
  • Three numbers with underscores, USER_POST_COMMENT: 12_34_56
  • Three numbers with colons, USER:POST:SHARD: 12:34:63 (We’re guessing that the third part is a shard in some FB internal system. In our experience so far, it’s always either 63 or the app-scoped user id for 63.)
  • Two numbers with colon, POST:SHARD: 12:34 (We’ve seen 0 as shard in this format.)
  • Four numbers with colons/underscore, USER:POST:SHARD_COMMENT: 12:34:63_56
  • Five numbers with colons/underscore, USER:EVENT:UNKNOWN:UNKNOWN_UNKNOWN Not currently supported! Examples: 111599105530674:998145346924699:10102446236688861:10207188792305341_998153510257216 111599105530674:195181727490727:10102446236688861:10205257726909910_195198790822354

Background:

Parameters:
  • id – string or integer
  • is_comment – boolean
Returns:

Some or all fields may be None.

Return type:

FacebookId

resolve_object_id(user_id, post_id, activity=None)[source]

Resolve a post id to its Facebook object id, if any.

Used for photo posts, since Facebook has (at least) two different objects (and ids) for them, one for the post and one for each photo.

This is the same logic that we do for canonicalizing photo objects in get_activities() above.

If activity is not provided, fetches the post from Facebook.

Parameters:
  • user_id – string Facebook user id who posted the post
  • post_id – string Facebook post id
  • activity – optional AS activity representation of Facebook post
Returns:

Facebook object id or None

Return type:

string

urlopen(url, _as=<type 'dict'>, **kwargs)[source]

Wraps urllib2.urlopen() and passes through the access token.

Parameters:_as – if not None, parses the response as JSON and passes it through _as() with this type. if None, returns the response object.
Returns:decoded JSON object or urlopen response object
urlopen_batch(urls)[source]

Sends a batch of multiple API calls using Facebook’s batch API.

Raises the appropriate urllib2.HTTPError if any individual call returns HTTP status code 4xx or 5xx.

https://developers.facebook.com/docs/graph-api/making-multiple-requests

Parameters:urls – sequence of string relative API URLs, e.g. (‘me’, ‘me/accounts’)
Returns:sequence of responses, either decoded JSON objects (when possible) or raw string bodies
urlopen_batch_full(requests)[source]

Sends a batch of multiple API calls using Facebook’s batch API.

Similar to urlopen_batch(), but the requests arg and return value are dicts with headers, HTTP status code, etc. Only raises urllib2.HTTPError if the outer batch request itself returns an HTTP error.

https://developers.facebook.com/docs/graph-api/making-multiple-requests

Parameters:requests

sequence of dict requests in Facebook’s batch format, except that headers is a single dict, not a list of dicts, e.g.:

[{'relative_url': 'me/feed',
  'headers': {'ETag': 'xyz', ...},
 },
 ...
]
Returns:sequence of dict responses in Facebook’s batch format, except that body is JSON-decoded if possible, and headers is a single dict, not a list of dicts, e.g.:
[{'code': 200,
  'headers': {'ETag': 'xyz', ...},
  'body': {...},
 },
 ...
]

flickr

Flickr source class.

Uses Flickr’s REST API https://www.flickr.com/services/api/

TODO: Fetching feeds with comments and/or favorites is very request intensive right now. It would be ideal to find a way to batch requests, make requests asynchronously, or make better calls to the API itself. Maybe use flickr.activity.userPhotos (https://www.flickr.com/services/api/flickr.activity.userPhotos.html) when group_id=SELF.

class granary.flickr.Flickr(access_token_key, access_token_secret, user_id=None, path_alias=None)[source]

Bases: granary.source.Source

Flickr source class. See file docstring and Source class for details.

__init__(access_token_key, access_token_secret, user_id=None, path_alias=None)[source]

Constructor.

If they are not provided, user_id and path_alias will be looked up via the API on first use.

Parameters:
  • access_token_key – string, OAuth access token key
  • access_token_secret – string, OAuth access token secret
  • user_id – string, the logged in user’s Flickr nsid. (optional)
  • path_alias – string, the logged in user’s path_alias, replaces user_id in canonical profile and photo urls (optional)
call_api_method(method, params=None)[source]

Call a Flickr API method.

upload(params, file)[source]

Upload a photo or video via the Flickr API.

create(obj, include_link=u'omit', ignore_formatting=False)[source]

Creates a photo, comment, or favorite.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose content will be a dict with ‘id’, ‘url’, and ‘type’ keys (all optional) for the newly created Flickr object (or None)

preview_create(obj, include_link=u'omit', ignore_formatting=False)[source]

Preview creation of a photo, comment, or favorite.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose description will be an HTML summary of what publishing will do, and whose content will be an HTML preview of the result (or None)

get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, **kwargs)[source]

Fetches Flickr photos and converts them to ActivityStreams activities.

See method docstring in source.py for details.

Mentions are not fetched or included because they don’t exist in Flickr. https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875

get_actor(user_id=None)[source]

Get an ActivityStreams object of type ‘person’ given a Flickr user’s nsid. If no user_id is provided, this method will make another API request to find out the currently logged in user’s id.

Parameters:user_id – string, optional
Returns:dict, an ActivityStreams object
user_to_actor(resp)[source]

Convert a Flickr user dict into an ActivityStreams actor.

get_comment(comment_id, activity_id, activity_author_id=None, activity=None)[source]

Returns an ActivityStreams comment object.

Parameters:
  • comment_id – string comment id
  • activity_id – string activity id, required
  • activity_author_id – string activity author id, ignored
  • activity – activity object (optional)
photo_to_activity(photo)[source]

Convert a Flickr photo to an ActivityStreams object. Takes either data in the expanded form returned by flickr.photos.getInfo or the abbreviated form returned by flickr.people.getPhotos.

Parameters:photo – dict response from Flickr
Returns:dict, an ActivityStreams object
like_to_object(person, photo_activity)[source]

Convert a Flickr favorite into an ActivityStreams like tag.

Parameters:
  • person – dict, the person object from Flickr
  • photo_activity – dict, the ActivityStreams object representing the photo this like belongs to
Returns:

dict, an ActivityStreams object

comment_to_object(comment, photo_id)[source]

Convert a Flickr comment json object to an ActivityStreams comment.

Parameters:
  • comment – dict, the comment object from Flickr
  • photo_id – string, the Flickr ID of the photo that this comment belongs to
Returns:

dict, an ActivityStreams object

get_user_image(farm, server, author)[source]

Convert fields from a typical Flickr response into the buddy icon URL.

ref: https://www.flickr.com/services/api/misc.buddyicons.html

user_id()[source]

Get the nsid of the currently authorized user. The first time this is called, it will invoke the flickr.people.getLimits api method.

https://www.flickr.com/services/api/flickr.people.getLimits.html

Returns:a string
path_alias()[source]

Get the path_alias of the currently authorized user. The first time this is called, it will invoke the flickr.people.getInfo api method.

https://www.flickr.com/services/api/flickr.people.getInfo.html

Returns:a string
user_url(user_id)[source]

Convert a user’s path_alias to their Flickr profile page URL.

Parameters:user_id (string) – user’s alphanumeric nsid or path alias
Returns:string, a profile URL
photo_url(user_id, photo_id)[source]

Construct a url for a photo given user id and the photo id :param user_id: alphanumeric user ID or path alias :type user_id: string :param photo_id: numeric photo ID :type photo_id: string

Returns:string, the photo URL
classmethod base_id(url)[source]

Used when publishing comments or favorites. Flickr photo ID is the 3rd path component rather than the first.

github

GitHub source class. Uses the v4 GraphQL API and the v3 REST API.

API docs: https://developer.github.com/v4/ https://developer.github.com/v3/ https://developer.github.com/apps/building-oauth-apps/authorization-options-for-oauth-apps/#web-application-flow

class granary.github.GitHub(access_token=None)[source]

Bases: granary.source.Source

GitHub source class. See file docstring and Source class for details.

access_token

string, optional, OAuth access token

__init__(access_token=None)[source]

Constructor.

Parameters:access_token – string, optional OAuth access token
classmethod base_id(url)[source]

Extracts and returns a USERNAME:REPO:ID id for an issue or PR.

Parameters:url – string
Returns:string, or None
graphql(graphql, kwargs)[source]

Makes a v4 GraphQL API call.

Parameters:graphql – string GraphQL operation

Returns: dict, parsed JSON response

rest(url, data=None, **kwargs)[source]

Makes a v3 REST API call.

Uses HTTP POST if data is provided, otherwise GET.

Parameters:data – dict, JSON payload for POST requests

Returns: requests.Response

get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, public_only=True, **kwargs)[source]

Fetches issues and comments and converts them to ActivityStreams activities.

See Source.get_activities_response() for details.

Not comprehensive! Uses the notifications API (v3 REST).

Also note that start_index and count are not currently supported.

https://developer.github.com/v3/activity/notifications/ https://developer.github.com/v3/issues/ https://developer.github.com/v3/issues/comments/

fetch_likes determines whether emoji reactions are fetched: https://help.github.com/articles/about-conversations-on-github#reacting-to-ideas-in-comments

The notifications API call supports Last-Modified/If-Modified-Since headers and 304 Not Changed responses. If provided, etag should be an RFC2822 timestamp, usually the exact value returned in a Last-Modified header. It will also be passed to the comments API endpoint as the since= value (converted to ISO 8601).

get_actor(user_id=None)[source]

Fetches nd returns a user.

Args: user_id: string, defaults to current user

Returns: dict, ActivityStreams actor object

get_comment(comment_id, **kwargs)[source]

Fetches and returns a comment.

Parameters:comment_id – string comment id (either REST or GraphQL), of the form REPO-OWNER_REPO-NAME_ID, e.g. snarfed:bridgy:456789

Returns: dict, an ActivityStreams comment object

render_markdown(markdown, owner, repo)[source]

Uses the GitHub API to render GitHub-flavored Markdown to HTML.

https://developer.github.com/v3/markdown/ https://github.github.com/gfm/

Parameters:
  • markdown – unicode string, input
  • and repo (owner) – strings, the repo to render in, for context. affects git hashes #XXX for issues/PRs.

Returns: unicode string, rendered HTML

create(obj, include_link=u'omit', ignore_formatting=False)[source]

Creates a new issue or comment.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose contents will be a dict with ‘id’ and ‘url’ keys for the newly created GitHub object (or None)

preview_create(obj, include_link=u'omit', ignore_formatting=False)[source]

Previews creating an issue or comment.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose contents will be a unicode string HTML snippet or None

existing_labels(owner, repo)[source]

Fetches and returns a repo’s labels.

Parameters:
  • owner – string, GitHub username or org that owns the repo
  • repo – string

Returns: set of strings

classmethod issue_to_object(issue)[source]

Converts a GitHub issue or pull request to ActivityStreams.

Handles both v4 GraphQL and v3 REST API issue and PR objects.

https://developer.github.com/v4/object/issue/ https://developer.github.com/v4/object/pullrequest/ https://developer.github.com/v3/issues/ https://developer.github.com/v3/pulls/

Parameters:issue – dict, decoded JSON GitHub issue or PR
Returns:an ActivityStreams object dict, ready to be JSON-encoded
classmethod pr_to_object(issue)

Converts a GitHub issue or pull request to ActivityStreams.

Handles both v4 GraphQL and v3 REST API issue and PR objects.

https://developer.github.com/v4/object/issue/ https://developer.github.com/v4/object/pullrequest/ https://developer.github.com/v3/issues/ https://developer.github.com/v3/pulls/

Parameters:issue – dict, decoded JSON GitHub issue or PR
Returns:an ActivityStreams object dict, ready to be JSON-encoded
classmethod comment_to_object(comment)[source]

Converts a GitHub comment to ActivityStreams.

Handles both v4 GraphQL and v3 REST API issue objects.

https://developer.github.com/v4/object/issue/ https://developer.github.com/v3/issues/

Parameters:comment – dict, decoded JSON GitHub issue
Returns:an ActivityStreams comment dict, ready to be JSON-encoded
classmethod reaction_to_object(reaction, target)[source]

Converts a GitHub emoji reaction to ActivityStreams.

Handles v3 REST API reaction objects.

https://developer.github.com/v3/reactions/

Parameters:
  • reaction – dict, decoded v3 JSON GitHub reaction
  • target – dict, ActivityStreams object of reaction
Returns:

an ActivityStreams reaction dict, ready to be JSON-encoded

classmethod user_to_actor(user)[source]

Converts a GitHub user to an ActivityStreams actor.

Handles both v4 GraphQL and v3 REST API user objects.

https://developer.github.com/v4/object/user/ https://developer.github.com/v3/users/

Parameters:user – dict, decoded JSON GitHub user
Returns:an ActivityStreams actor dict, ready to be JSON-encoded

googleplus

Google+ source class.

The Google+ API currently only returns public activities and comments, so the Audience Targeting ‘to’ field is always set to @public. https://developers.google.com/+/api/latest/activities/list#collection

class granary.googleplus.GooglePlus(auth_entity=None, access_token=None)[source]

Bases: granary.source.Source

Google+ source class. See file docstring and Source class for details.

The Google+ API already exposes data in ActivityStreams format, so this is mostly just a pass through.

__init__(auth_entity=None, access_token=None)[source]

Constructor.

Currently, only auth_entity is supported. TODO: implement access_token.

Parameters:
  • access_token – string OAuth access token
  • auth_entity – oauth-dropins.googleplus.GooglePlusAuth
get_actor(user_id=None)[source]

Returns a user as a JSON ActivityStreams actor dict.

Parameters:user_id – string id or username. Defaults to ‘me’, ie the current user.

Raises: GooglePlusAPIError

get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, **kwargs)[source]

Fetches posts and converts them to ActivityStreams activities.

See method docstring in source.py for details. app_id is ignored.

Replies (comments), likes (+1s), and shares (reshares) each need an extra API call per activity. The activity has total counts for them, though, so we only make those calls when we know there’s something to fetch. https://developers.google.com/+/api/latest/comments/list https://developers.google.com/+/api/latest/people/listByActivity

We also batch those calls into a single HTTP request, so there are at most two HTTP requests total, one to get activities and optionally one to get new responses. https://developers.google.com/api-client-library/python/guide/batch

Mentions are not currently fetched or included because the API doesn’t explicitly support searching for them. It could be approximated, though: https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875

get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]

Returns an ActivityStreams comment object.

Parameters:
  • comment_id – string comment id
  • activity_id – string activity id, optional
  • activity_author_id – string activity author id. Ignored.
  • activity – activity object (optional)
postprocess_activity(activity)[source]

Massage G+’s ActivityStreams dialect into our dialect, in place.

Parameters:activity – ActivityStreams activity dict.
postprocess_comment(comment)[source]

Hack to pretend comment activities are comment objects.

G+ puts almost everything in the comment activity, not the object inside the activity. So, copy over the content and use the activity itself.

postprocess_actor(actor)[source]

Massage G+’s ActivityStreams dialect into our dialect, in place.

Parameters:actor – ActivityStreams actor dict.
user_to_actor(actor)

Massage G+’s ActivityStreams dialect into our dialect, in place.

Parameters:actor – ActivityStreams actor dict.
maybe_add_tags(batch, activity, cached, cache_updates, collection, verb)[source]

Fetches and adds ‘like’ or ‘share’ tags to an activity.

Just adds a request and callback to the batch. Does not execute the batch.

Converts +1s to like and reshares to share activity objects, and stores them in place in the ‘tags’ field of the activity’s object. Details: https://developers.google.com/+/api/latest/people/listByActivity

Parameters:
  • batch – BatchHttpRequest
  • activity – dict, G+ activity that was +1ed or reshared
  • cached – dict of cache values. (not cache object above.)
  • cache_updates – dict, values to write back to cache
  • collection – string, ‘plusoners’ or ‘resharers’
  • verb – string, ActivityStreams verb to populate the tags with
html_to_activities(html)[source]

Converts HTML from https://plus.google.com/ to ActivityStreams activities.

Parameters:html – unicode string
Returns:list of ActivityStreams activity dicts

instagram

Instagram source class.

Instagram’s API doesn’t tell you if a user has marked their account private or not, so the Audience Targeting ‘to’ field is currently always set to @public. http://help.instagram.com/448523408565555 https://groups.google.com/forum/m/#!topic/instagram-api-developers/DAO7OriVFsw https://groups.google.com/forum/#!searchin/instagram-api-developers/private

class granary.instagram.Instagram(access_token=None, allow_comment_creation=False, scrape=False)[source]

Bases: granary.source.Source

Instagram source class. See file docstring and Source class for details.

__init__(access_token=None, allow_comment_creation=False, scrape=False)[source]

Constructor.

If an OAuth access token is provided, it will be passed on to Instagram. This will be necessary for some people and contact details, based on their privacy settings.

Parameters:
  • access_token – string, optional OAuth access token
  • allow_comment_creation – boolean, optionally disable comment creation, useful if the app is not approved to create comments.
  • scrape – boolean, whether to scrape instagram.com’s HTML (True) or use
  • API (the) –
urlopen(url, **kwargs)[source]

Wraps urllib2.urlopen() and passes through the access token.

get_actor(user_id=None, **kwargs)[source]

Returns a user as a JSON ActivityStreams actor dict.

Parameters:
  • user_id – string id or username. Defaults to ‘self’, ie the current user.
  • kwargs – if scraping, passed through to get_activities_response(). Raises AssertionError if provided and not scraping.

Raises: InstagramAPIError

get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, scrape=False, cookie=None, ignore_rate_limit=False, **kwargs)[source]

Fetches posts and converts them to ActivityStreams activities.

See method docstring in source.py for details. app_id is ignored. Supports min_id, but not ETag, since Instagram doesn’t support it.

http://instagram.com/developer/endpoints/users/#get_users_feed http://instagram.com/developer/endpoints/users/#get_users_media_recent

Likes are always included, regardless of the fetch_likes kwarg. They come bundled in the ‘likes’ field of the API Media object: http://instagram.com/developer/endpoints/media/#

Mentions are never fetched or included because the API doesn’t support searching for them. https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875

Shares are never fetched included since there is no share feature.

Instagram only supports search over hashtags, so if search_query is set, it must begin with #.

May populate a custom ‘ig_like_count’ property in media objects. (Currently only when scraping.)

Parameters:
  • scrape – if True, scrapes HTML from instagram.com instead of using the API. Populates the user’s actor object in the ‘actor’ response field. Useful for apps that haven’t yet been approved in the new permissions approval process. Currently only supports group_id=SELF. Also supports passing a shortcode as activity_id as well as the internal API id. http://developers.instagram.com/post/133424514006/instagram-platform-update
  • cookie – string, only used if scrape=True
  • ignore_rate_limit – boolean, for scraping, always make an HTTP request, even if we’ve been rate limited recently
  • ** – see Source.get_activities_response()
Raises:

InstagramAPIError

get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]

Returns an ActivityStreams comment object.

Parameters:
  • comment_id – string comment id
  • activity_id – string activity id, required
  • activity_author_id – string activity author id. Ignored.
  • activity – activity object, optional. Avoids fetching the activity.
get_share(activity_user_id, activity_id, share_id, activity=None)[source]

Not implemented. Returns None. Resharing isn’t a feature of Instagram.

create(obj, include_link=u'omit', ignore_formatting=False)[source]

Creates a new comment or like.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult. if successful, content will have and ‘id’ and

’url’ keys for the newly created Instagram object

preview_create(obj, include_link=u'omit', ignore_formatting=False)[source]

Preview a new comment or like.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult. if successful, content and description

will describe the new instagram object.

media_to_activity(media)[source]

Converts a media to an activity.

http://instagram.com/developer/endpoints/media/#get_media

Parameters:media – JSON object retrieved from the Instagram API
Returns:an ActivityStreams activity dict, ready to be JSON-encoded
media_to_object(media)[source]

Converts a media to an object.

Parameters:media – JSON object retrieved from the Instagram API
Returns:an ActivityStreams object dict, ready to be JSON-encoded
comment_to_object(comment, media_id, media_url)[source]

Converts a comment to an object.

Parameters:
  • comment – JSON object retrieved from the Instagram API
  • media_id – string
  • media_url – string
Returns:

an ActivityStreams object dict, ready to be JSON-encoded

like_to_object(liker, media_id, media_url)[source]

Converts a like to an object.

Parameters:
  • liker – JSON object from the Instagram API, the user who does the liking
  • media_id – string
  • media_url – string
Returns:

an ActivityStreams object dict, ready to be JSON-encoded

user_to_actor(user)[source]

Converts a user to an actor.

Parameters:user – JSON object from the Instagram API
Returns:an ActivityStreams actor dict, ready to be JSON-encoded
base_object(obj)[source]

Extends the default base_object() to avoid using shortcodes as object ids.

static id_to_shortcode(id)[source]

Converts a media id to the shortcode used in its instagram.com URL.

Based on http://carrot.is/coding/instagram-ids , which determined that shortcodes are just URL-safe base64 encoded ids.

html_to_activities(html, cookie=None, count=None)[source]

Converts Instagram HTML to ActivityStreams activities.

The input HTML may be from:

Parameters:
  • html – unicode string
  • cookie – string, optional sessionid cookie to be used for subsequent HTTP fetches, if necessary.
  • count – integer, number of activities to return, None for all
Returns:

tuple, ([ActivityStreams activities], ActivityStreams viewer actor)

jsonfeed

Convert between ActivityStreams and JSON Feed.

JSON Feed spec: https://jsonfeed.org/version/1

granary.jsonfeed.activities_to_jsonfeed(activities, actor=None, title=None, feed_url=None, home_page_url=None)[source]

Converts ActivityStreams activities to a JSON feed.

Parameters:
  • activities – sequence of ActivityStreams activity dicts
  • actor – ActivityStreams actor dict, the author of the feed
  • title – string, the feed title
  • home_page_url – string, the home page URL
  • feed_url – the URL of the JSON Feed, if any. Included in the feed_url field.
Returns:

dict, JSON Feed data, ready to be JSON-encoded

granary.jsonfeed.jsonfeed_to_activities(jsonfeed)[source]

Converts a JSON feed to ActivityStreams activities and actor.

Parameters:jsonfeed – dict, JSON Feed data
Returns:(activities, actor) tuple, where activities and actor are both ActivityStreams object dicts
Raises:ValueError, if jsonfeed isn’t a valid JSON Feed dict

microformats2

Convert ActivityStreams to microformats2 HTML and JSON.

Microformats2 specs: http://microformats.org/wiki/microformats2

granary.microformats2.get_string_urls(objs)[source]

Extracts string URLs from a list of either string URLs or mf2 dicts.

Many mf2 properties can contain either string URLs or full mf2 objects, e.g. h-cites. in-reply-to is the most commonly used example: http://indiewebcamp.com/in-reply-to#How_to_consume_in-reply-to

Parameters:objs – sequence of either string URLs or embedded mf2 objects
Returns:list of string URLs
granary.microformats2.get_html(val)[source]

Returns a string value that may have HTML markup.

Parameters:value – mf2 property value, either string or {‘html’: ‘<p>str</p>’, ‘value’: ‘str’} dict
Returns:string or None
granary.microformats2.get_text(val)[source]

Returns a plain text string value. See get_html.

granary.microformats2.activity_to_json(activity, **kwargs)[source]

Converts an ActivityStreams activity to microformats2 JSON.

Parameters:
  • activity – dict, a decoded JSON ActivityStreams activity
  • kwargs – passed to object_to_json
Returns:

dict, decoded microformats2 JSON

granary.microformats2.object_to_json(obj, trim_nulls=True, entry_class=u'h-entry', default_object_type=None, synthesize_content=True)[source]

Converts an ActivityStreams object to microformats2 JSON.

Parameters:
  • obj – dict, a decoded JSON ActivityStreams object
  • trim_nulls – boolean, whether to remove elements with null or empty values
  • entry_class – string or sequence, the mf2 class(es) that entries should be given (e.g. ‘h-cite’ when parsing a reference to a foreign entry). defaults to ‘h-entry’
  • default_object_type – string, the ActivityStreams objectType to use if one is not present. defaults to None
  • synthesize_content – whether to generate synthetic content if the object doesn’t have its own, e.g. ‘likes this.’ or ‘shared this.’
Returns:

dict, decoded microformats2 JSON

granary.microformats2.json_to_object(mf2, actor=None, fetch_mf2=False)[source]

Converts a single microformats2 JSON item to an ActivityStreams object.

Supports h-entry, h-event, h-card, and other single item times. Does not yet support h-feed.

Parameters:
  • mf2 – dict, decoded JSON microformats2 object
  • actor – optional author AS actor object. usually comes from a rel=”author” link. if mf2 has its own author, that will override this.
  • fetch_mf2 – boolean, whether to fetch additional pages via HTTP if necessary, e.g. to determine authorship: https://indieweb.org/authorship
Returns:

dict, ActivityStreams object

granary.microformats2.html_to_activities(html, url=None, actor=None)[source]

Converts a microformats2 HTML h-feed to ActivityStreams activities.

Parameters:
  • html – string HTML
  • url – optional string URL that HTML came from
  • actor – optional author AS actor object for all activities. usually comes from a rel=”author” link.
Returns:

list of ActivityStreams activity dicts

granary.microformats2.activities_to_html(activities)[source]

Converts ActivityStreams activities to a microformats2 HTML h-feed.

Parameters:obj – dict, a decoded JSON ActivityStreams object
Returns:string, the content field in obj with the tags in the tags field converted to links if they have startIndex and length, otherwise added to the end.
granary.microformats2.object_to_html(obj, parent_props=None, synthesize_content=True)[source]

Converts an ActivityStreams object to microformats2 HTML.

Features:

  • linkifies embedded tags and adds links for other tags
  • linkifies embedded URLs
  • adds links, summaries, and thumbnails for attachments and checkins
  • adds a “via SOURCE” postscript
Parameters:
  • obj – dict, a decoded JSON ActivityStreams object
  • parent_props – list of strings, the properties of the parent object where this object is embedded, e.g. [‘u-repost-of’]
  • synthesize_content – whether to generate synthetic content if the object doesn’t have its own, e.g. ‘likes this.’ or ‘shared this.’
Returns:

string, the content field in obj with the tags in the tags field converted to links if they have startIndex and length, otherwise added to the end.

granary.microformats2.json_to_html(obj, parent_props=None)[source]

Converts a microformats2 JSON object to microformats2 HTML.

See object_to_html for details.

Parameters:
  • obj – dict, a decoded microformats2 JSON object
  • parent_props – list of strings, the properties of the parent object where this object is embedded, e.g. ‘u-repost-of’
Returns:

string HTML

granary.microformats2.hcard_to_html(hcard, parent_props=None)[source]

Renders an h-card as HTML.

Parameters:
  • hcard – dict, decoded JSON h-card
  • parent_props – list of strings, the properties of the parent object where this object is embedded, e.g. [‘p-author’]
Returns:

string, rendered HTML

granary.microformats2.render_content(obj, include_location=True, synthesize_content=True, render_attachments=False)[source]

Renders the content of an ActivityStreams object as HTML.

Includes tags, mentions, and non-note/article attachments. (Note/article attachments are converted to mf2 children in object_to_json and then rendered in json_to_html.)

Note that the returned HTML is included in Atom as well as HTML documents, so it must be HTML4 / XHTML, not HTML5! All tags must be closed, etc.

Parameters:
  • obj – decoded JSON ActivityStreams object
  • include_location – whether to render location, if provided
  • synthesize_content – whether to generate synthetic content if the object doesn’t have its own, e.g. ‘likes this.’ or ‘shared this.’
Returns:

string, rendered HTML

granary.microformats2.find_author(parsed, **kwargs)[source]

Returns the author of a page as a ActivityStreams actor dict.

Parameters:
  • parsed – return value from mf2py.parse()
  • kwargs – passed through to mf2util.find_author()
granary.microformats2.get_title(mf2)[source]

Returns an mf2 object’s title, ie its name.

Parameters:mf2 – dict, parsed mf2 object (ie return value from mf2py.parse())

Returns: string title, possibly ellipsized

granary.microformats2.first_props(props)[source]

Converts a multiply-valued dict to singly valued.

Parameters:props – dict of properties, where each value is a sequence
Returns:corresponding dict with just the first value of each sequence, or ‘’ if the sequence is empty
granary.microformats2.tags_to_html(tags, classname)[source]

Returns an HTML string with links to the given tag objects.

Parameters:
  • tags – decoded JSON ActivityStreams objects.
  • classname – class for span to enclose tags in
granary.microformats2.object_urls(obj)[source]

Returns an object’s unique URLs, preserving order.

granary.microformats2.author_display_name(hcard)[source]

Returns a human-readable string display name for an h-card object.

granary.microformats2.maybe_linked_name(props)[source]

Returns the HTML for a p-name with an optional u-url inside.

Parameters:propsmultiply-valued properties dict
Returns:string HTML
granary.microformats2.img(src, alt=u'')[source]

Returns an <img> string with the given src, class, and alt.

Parameters:
  • src – string url or dict with value and (optionally) alt
  • alt – string, alt attribute value, or None
Returns:

string

granary.microformats2.vid(src, poster=u'')[source]

Returns an <video> string with the given src and class

Parameters:
  • src – string, url of the video
  • poster – sring, optional. url of the poster or preview image
Returns:

string

granary.microformats2.aud(src)[source]

Returns an <audio> string with the given src and class

Parameters:src – string, url of the audio
Returns:string
granary.microformats2.maybe_linked(text, url, linked_classname=None, unlinked_classname=None)[source]

Wraps text in an <a href=…> iff a non-empty url is provided.

Parameters:
  • text – string
  • url – string or None
  • linked_classname – string, optional class attribute to use if url
  • unlinked_classname – string, optional class attribute to use if not url
Returns:

string

granary.microformats2.maybe_datetime(str, classname)[source]

Returns a <time datetime=…> elem if str is non-empty.

Parameters:
  • str – string RFC339 datetime or None
  • classname – string class name
Returns:

string

source

Source base class.

Based on the OpenSocial ActivityStreams REST API: http://opensocial-resources.googlecode.com/svn/spec/2.0.1/Social-API-Server.xml#ActivityStreams-Service

Uses the ‘to’ field of the Audience Targeting extension to indicate an activity’s privacy settings. It’s set to a group with alias @public or @private, or unset if unknown. http://activitystrea.ms/specs/json/targeting/1.0/#anchor3

class granary.source.CreationResult(content, description, abort, error_plain, error_html)

Bases: tuple

abort

Alias for field number 2

content

Alias for field number 0

description

Alias for field number 1

error_html

Alias for field number 4

error_plain

Alias for field number 3

exception granary.source.RateLimited(*args, **kwargs)[source]

Bases: exceptions.BaseException

Raised when an API rate limits us, and we may have a partial result.

partial

the partial result, if any. Usually a list.

__init__(*args, **kwargs)[source]

x.__init__(…) initializes x; see help(type(x)) for signature

__weakref__

list of weak references to the object (if defined)

granary.source.html_to_text(html, **kwargs)[source]

Converts string html to string text with html2text.

Parameters:**kwargs – html2text options https://github.com/Alir3z4/html2text/blob/master/docs/usage.md#available-options
granary.source.load_json(body, url)[source]

Utility method to parse a JSON string. Raises HTTPError 502 on failure.

granary.source.creation_result(content=None, description=None, abort=False, error_plain=None, error_html=None)[source]

Create a new CreationResult.

create() and preview_create() use this to provide a detailed description of publishing failures. If abort is False, we should continue looking for an entry to publish; if True, we should immediately inform the user. error_plain text is sent in response to failed publish webmentions; error_html will be displayed to the user when publishing interactively.

Parameters:
  • content – a string HTML snippet for preview_create() or a dict for create()
  • description – string HTML snippet describing the publish action, e.g. @-reply’ or ‘RSVP yes to this event’. The verb itself is surrounded by a <span class=”verb”> to allow styling. May also include <a> link(s) and embedded silo post(s).
  • abort – a boolean
  • error_plain – a string
  • error_html – a string
Returns:

a CreationResult

granary.source.object_type(obj)[source]

Returns the object type, or the verb if it’s an activity object.

Details: http://activitystrea.ms/specs/json/1.0/#activity-object

Parameters:obj – decoded JSON ActivityStreams object
Returns:ActivityStreams object type
Return type:string
class granary.source.SourceMeta[source]

Bases: type

Source metaclass. Registers all source classes in the sources global.

class granary.source.Source[source]

Bases: future.types.newobject.newobject

Abstract base class for a source (e.g. Facebook, Twitter).

Concrete subclasses must override the class constants below and implement get_activities().

Class constants:

  • DOMAIN: string, the source’s domain
  • BASE_URL: optional, the source’s base url
  • NAME: string, the source’s human-readable name
  • FRONT_PAGE_TEMPLATE: string, the front page child template filename
  • AUTH_URL: string, the url for the “Authenticate” front page link
  • EMBED_POST: string, the HTML for embedding a post. Should have a %(url)s placeholder for the post URL and (optionally) a %(content)s placeholder for the post content.
  • POST_ID_RE: regexp, optional, matches valid post ids. Used in post_id().
  • HTML2TEXT_OPTIONS: dict mapping string html2text option names to values https://github.com/Alir3z4/html2text/blob/master/docs/usage.md#available-options
user_url(user_id)[source]

Returns the URL for a user’s profile.

get_actor(user_id=None)[source]

Fetches and returns a user.

Args: user_id: string, defaults to current user

Returns: dict, ActivityStreams actor object

get_activities(*args, **kwargs)[source]

Fetches and returns a list of activities.

See get_activities_response() for args and kwargs.

Returns:list, ActivityStreams activity dicts
get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, **kwargs)[source]

Fetches and returns ActivityStreams activities and response details.

Subclasses should override this. See get_activities() for an alternative that just returns the list of activities.

If user_id is provided, only that user’s activity(s) are included. start_index and count determine paging, as described in the spec: http://activitystrea.ms/draft-spec.html#anchor14

app id is just object id: http://opensocial-resources.googlecode.com/svn/spec/2.0/Social-Data.xml#appId

group id is string id of group or @self, @friends, @all, @search: http://opensocial-resources.googlecode.com/svn/spec/2.0/Social-Data.xml#Group-ID

The fetch_* kwargs all default to False because they often require extra API round trips. Some sources return replies, likes, and shares in the same initial call, so they may be included even if you don’t set their kwarg to True.

Parameters:
  • user_id – string, defaults to the currently authenticated user
  • group_id – string, one of @self’, @all’, @friends’, @search’. defaults to @friends
  • app_id – string
  • activity_id – string
  • start_index – int >= 0
  • count – int >= 0
  • etag – string, optional ETag to send with the API request. Results will only be returned if the ETag has changed. Should include enclosing double quotes, e.g. ‘“ABC123”’
  • min_id – only return activities with ids greater than this
  • cache – object with get(key), set_multi(dict), and delete_multi(list) methods. In practice, this is App Engine’s memcache interface: https://developers.google.com/appengine/docs/python/memcache/functions Used to cache data that’s expensive to regenerate, e.g. API calls.
  • fetch_replies – boolean, whether to fetch each activity’s replies also
  • fetch_likes – boolean, whether to fetch each activity’s likes also
  • fetch_shares – boolean, whether to fetch each activity’s shares also
  • fetch_events – boolean, whether to fetch the user’s events also
  • fetch_mentions – boolean, whether to fetch posts that mention the user
  • search_query – string, an optional search query, only for use with @search group_id
  • kwargs – some sources accept extra kwargs. See their docs for details.
Returns:

response values based on OpenSocial ActivityStreams REST API.

http://opensocial-resources.googlecode.com/svn/spec/2.0.1/Social-API-Server.xml#ActivityStreams-Service http://opensocial-resources.googlecode.com/svn/spec/2.0.1/Core-Data.xml

It has these keys: * items: list of activity dicts * startIndex: int or None * itemsPerPage: int * totalResults: int or None (e.g. if it can ‘t be calculated efficiently) * filtered: False * sorted: False * updatedSince: False * etag: string etag returned by the API’s initial call to get activities

Return type:

dict

Raises:
  • ValueError – if any argument is invalid for this source
  • NotImplementedError – if the source doesn’t support the requested operation, e.g. Facebook doesn’t support search.
classmethod make_activities_base_response(activities, *args, **kwargs)[source]

Generates a base response dict for get_activities_response().

See get_activities() for args and kwargs.

create(obj, include_link=u'omit', ignore_formatting=False)[source]

Creates a new object: a post, comment, like, share, or RSVP.

Subclasses should override this. Different sites will support different functionality; check each subclass for details. The actor will usually be the authenticated user.

Parameters:
  • obj – ActivityStreams object. At minimum, must have the content field. objectType is strongly recommended.
  • include_link – string. ‘include’, ‘omit’, or ‘if truncated’; whether to include a link to the object (if it has one) in the content.
  • ignore_formatting – whether to use content text as is, instead of converting its HTML to plain text styling (newlines, etc.)
Returns:

contents will be a dict. The dict may be None or empty. If

the newly created object has an id or permalink, they’ll be provided in the values for ‘id’ and ‘url’.

Return type:

CreationResult

preview_create(obj, include_link=u'omit', ignore_formatting=False)[source]

Previews creating a new object: a post, comment, like, share, or RSVP.

Returns HTML that previews what create() with the same object will do.

Subclasses should override this. Different sites will support different functionality; check each subclass for details. The actor will usually be the authenticated user.

Parameters:
  • obj – ActivityStreams object. At minimum, must have the content field. objectType is strongly recommended.
  • include_link – string. Whether to include a link to the object (if it has one) in the content.
  • ignore_formatting – whether to use content text as is, instead of converting its HTML to plain text styling (newlines, etc.)
Returns:

contents will be a unicode string HTML snippet (or None)

Return type:

CreationResult

get_event(event_id)[source]

Fetches and returns an event.

Parameters:id – string, site-specific event id
Returns:decoded ActivityStreams activity, or None
Return type:dict
get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]

Fetches and returns a comment.

Subclasses should override this.

Parameters:
  • comment_id – string comment id
  • activity_id – string activity id, optional
  • activity_author_id – string activity author id, optional. Needed for some sources (e.g. Facebook) to construct the comment permalink.
  • activity – activity object, optional. May avoid an API call if provided.

Returns: dict, ActivityStreams comment object

Raises:ValueError – if any argument is invalid for this source
get_like(activity_user_id, activity_id, like_user_id, activity=None)[source]

Fetches and returns a ‘like’.

Default implementation that fetches the activity and its likes, then searches for a like by the given user. Subclasses should override this if they can optimize the process.

Parameters:
  • activity_user_id – string id of the user who posted the original activity
  • activity_id – string activity id
  • like_user_id – string id of the user who liked the activity
  • activity – activity object, optional. May avoid an API call if provided.

Returns: dict, ActivityStreams ‘like’ activity object

get_reaction(activity_user_id, activity_id, reaction_user_id, reaction_id, activity=None)[source]

Fetches and returns a ‘reaction’.

Default implementation that fetches the activity and its reactions, then searches for this specific reaction. Subclasses should override this if they can optimize the process.

Parameters:
  • activity_user_id – string id of the user who posted the original activity
  • activity_id – string activity id
  • reaction_user_id – string id of the user who reacted
  • reaction_id – string id of the reaction
  • activity – activity object, optional. May avoid an API call if provided.

Returns: dict, ActivityStreams ‘reaction’ activity object

get_share(activity_user_id, activity_id, share_id, activity=None)[source]

Fetches and returns a ‘share’.

Parameters:
  • activity_user_id – string id of the user who posted the original activity
  • activity_id – string activity id
  • share_id – string id of the share object or the user who shared it
  • activity – activity object, optional. May avoid an API call if provided.

Returns: dict, an ActivityStreams ‘share’ activity object

get_rsvp(activity_user_id, event_id, user_id, event=None)[source]

Fetches and returns an RSVP.

Parameters:
  • activity_user_id – string id of the user who posted the event. unused.
  • event_id – string event id
  • user_id – string user id
  • event – AS event activity (optional)

Returns: dict, an ActivityStreams RSVP activity object

get_blocklist()[source]

Fetches and returns the current user’s block list.

…ie the users that the current user is blocking. The exact semantics of “blocking” vary from silo to silo.

Returns:sequence of actor objects
user_to_actor(user)[source]

Converts a user to an actor.

The returned object will have at least a ‘url’ field. If the user has multiple URLs, there will also be a ‘urls’ list field whose elements are dicts with ‘value’: URL.

Parameters:user – dict, a decoded JSON silo user object
Returns:ActivityStreams actor, ready to be JSON-encoded
Return type:dict
static postprocess_activity(activity)[source]

Does source-independent post-processing of an activity, in place.

Right now just populates the title field.

Parameters:activity – activity dict
static postprocess_object(obj)[source]

Does source-independent post-processing of an object, in place.

Populates location.position based on latitude and longitude.

Parameters:object – object dict
static original_post_discovery(activity, domains=None, cache=None, include_redirect_sources=True, **kwargs)[source]

Discovers original post links.

This is a variation on http://indiewebcamp.com/original-post-discovery . It differs in that it finds multiple candidate links instead of one, and it doesn’t bother looking for MF2 (etc) markup because the silos don’t let you input it. More background: https://github.com/snarfed/bridgy/issues/51#issuecomment-136018857

Original post candidates come from the upstreamDuplicates, attachments, and tags fields, as well as links and permashortlinks/permashortcitations in the text content.

Parameters:
  • activity – activity dict
  • domains – optional sequence of domains. If provided, only links to these domains will be considered original and stored in upstreamDuplicates. (Permashortcitations are exempt.)
  • cache – optional, a cache object for storing resolved URL redirects. Passed to follow_redirects().
  • include_redirect_sources – boolean, whether to include URLs that redirect as well as their final destination URLs
  • kwargs – passed to requests.head() when following redirects
Returns:

([string original post URLs], [string mention URLs]) tuple

static actor_name(actor)[source]

Returns the given actor’s name if available, otherwise Unknown.

static is_public(obj)[source]

Returns True if the object is public, False if private, None if unknown.

…according to the Audience Targeting extension https://developers.google.com/+/api/latest/activities/list#collection

Expects values generated by this library: objectType group, alias @public or @private.

Also, important point: this defaults to true, ie public. Bridgy depends on that and prunes the to field from stored activities in Response objects (in bridgy/util.prune_activity()). If the default here ever changes, be sure to update Bridgy’s code.

static add_rsvps_to_event(event, rsvps)[source]

Adds RSVP objects to an event’s attending fields, in place.

Parameters:
  • event – ActivityStreams event object
  • rsvps – sequence of ActivityStreams RSVP activity objects
static get_rsvps_from_event(event)[source]

Returns RSVP objects for an event’s attending fields.

Parameters:event – ActivityStreams event object
Returns:sequence of ActivityStreams RSVP activity objects
static activity_changed(before, after, log=False)[source]

Returns whether two activities or objects differ meaningfully.

Only compares a few fields: object type, verb, content, location, and image. Notably does not compare author and published/updated timestamps.

This has been tested on Facebook posts, comments, and event RSVPs (only content and rsvp_status change) and Google+ posts and comments (content, updated, and etag change). Twitter tweets and Instagram photo captions and comments can’t be edited.

Parameters:after (before,) – dicts, ActivityStreams activities or objects
Returns:boolean
classmethod embed_post(obj)[source]

Returns the HTML string for embedding a post object.

classmethod embed_actor(actor)[source]

Returns the HTML string for embedding an actor object.

classmethod tag_uri(name)[source]

Returns a tag URI string for this source and the given string name.

base_object(obj)[source]

Returns the ‘base’ silo object that an object operates on.

For example, if the object is a comment, this returns the post that it’s a comment on. If it’s an RSVP, this returns the event. The id in the returned object is silo-specific, ie not a tag URI.

Subclasses may override this.

Parameters:obj – ActivityStreams object
Returns:dict, minimal ActivityStreams object. Usually has at least id; may also have url, author, etc.
classmethod base_id(url)[source]

Guesses the id of the object in the given URL.

Returns:string, or None
classmethod post_id(url)[source]

Guesses the post id of the given URL.

Returns:string, or None

twitter

Twitter source class.

Uses the v1.1 REST API: https://dev.twitter.com/docs/api

TODO: collections for twitter accounts; use as activity target?

The Audience Targeting ‘to’ field is set to @public or @private based on whether the tweet author’s ‘protected’ field is true or false. https://dev.twitter.com/docs/platform-objects/users

class granary.twitter.OffsetTzinfo(utc_offset=0)[source]

Bases: datetime.tzinfo

A simple, DST-unaware tzinfo from given utc offset in seconds.

__init__(utc_offset=0)[source]

Constructor.

Parameters:utc_offset – Offset of time zone from UTC in seconds
utcoffset(dt)[source]

datetime -> minutes east of UTC (negative for west of UTC).

dst(dt)[source]

datetime -> DST offset in minutes east of UTC.

__weakref__

list of weak references to the object (if defined)

class granary.twitter.Twitter(access_token_key, access_token_secret, username=None)[source]

Bases: granary.source.Source

Twitter source class. See file docstring and Source class for details.

__init__(access_token_key, access_token_secret, username=None)[source]

Constructor.

Twitter now requires authentication in v1.1 of their API. You can get an OAuth access token by creating an app here: https://dev.twitter.com/apps/new

Parameters:
  • access_token_key – string, OAuth access token key
  • access_token_secret – string, OAuth access token secret
  • username – string, optional, the current user. Used in e.g. preview/create.
get_actor(screen_name=None)[source]

Returns a user as a JSON ActivityStreams actor dict.

Parameters:screen_name – string username. Defaults to the current user.
get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, **kwargs)[source]

Fetches posts and converts them to ActivityStreams activities.

XXX HACK: this is currently hacked for bridgy to NOT pass min_id to the request for fetching activity tweets themselves, but to pass it to all of the requests for filling in replies, retweets, etc. That’s because we want to find new replies and retweets of older initial tweets. TODO: find a better way.

See source.Source.get_activities_response() for details. app_id is ignored. min_id is translated to Twitter’s since_id.

The code for handling ETags (and 304 Not Changed responses and setting If-None-Match) is here, but unused right now since Twitter evidently doesn’t support ETags. From https://dev.twitter.com/discussions/5800 : “I’ve confirmed with our team that we’re not explicitly supporting this family of features.”

Likes (ie favorites) are scraped from twitter.com HTML, since Twitter’s REST API doesn’t offer a way to fetch them. You can also get them from the Streaming API, though, and convert them with streaming_event_to_object(). https://dev.twitter.com/docs/streaming-apis/messages#Events_event

Shares (ie retweets) are fetched with a separate API call per tweet: https://dev.twitter.com/docs/api/1.1/get/statuses/retweets/%3Aid

However, retweets are only fetched for the first 15 tweets that have them, since that’s Twitter’s rate limit per 15 minute window. :( https://dev.twitter.com/docs/rate-limiting/1.1/limits

Quote tweets are fetched by searching for the possibly quoted tweet’s ID, using the OR operator to search up to 5 IDs at a time, and then checking the quoted_status_id_str field https://dev.twitter.com/overview/api/tweets#quoted_status_id_str

Use the group_id @self to retrieve a user_id’s timeline. If user_id is None or @me, it will return tweets for the current API user.

group_id can be used to specify the slug of a list for which to return tweets. By default the current API user’s lists will be used, but lists owned by other users can be fetched by explicitly passing a username to user_id, e.g. to fetch tweets from the list @exampleuser/example-list you would call get_activities(user_id=’exampleuser’, group_id=’example-list’).

Twitter replies default to including a mention of the user they’re replying to, which overloads mentions a bit. When fetch_mentions is True, we determine that a tweet mentions the current user if it @-mentions their username and:

  • it’s not a reply, OR
  • it’s a reply, but not to the current user, AND * the tweet it’s replying to doesn’t @-mention the current user
fetch_replies(activities, min_id=None)[source]

Fetches and injects Twitter replies into a list of activities, in place.

Includes indirect replies ie reply chains, not just direct replies. Searches for @-mentions, matches them to the original tweets with in_reply_to_status_id_str, and recurses until it’s walked the entire tree.

Parameters:activities – list of activity dicts
Returns:same activities list
fetch_mentions(username, tweets, min_id=None)[source]

Fetches a user’s @-mentions and returns them as ActivityStreams.

Tries to only include explicit mentions, not mentions automatically created by @-replying. See the get_activities() docstring for details.

Parameters:
  • username – string
  • tweets – list of Twitter API objects. used to find quote tweets quoting them.
  • min_id – only return activities with ids greater than this
Returns:

list of activity dicts

get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]

Returns an ActivityStreams comment object.

Parameters:
  • comment_id – string comment id
  • activity_id – string activity id, optional
  • activity_author_id – string activity author id. Ignored.
  • activity – activity object, optional
get_share(activity_user_id, activity_id, share_id, activity=None)[source]

Returns an ActivityStreams ‘share’ activity object.

Parameters:
  • activity_user_id – string id of the user who posted the original activity
  • activity_id – string activity id
  • share_id – string id of the share object
  • activity – activity object, optional
get_blocklist()[source]

Returns the current user’s block list.

May make multiple API calls, using cursors, to fully fetch large blocklists. https://dev.twitter.com/overview/api/cursoring

Block lists may have up to 10k users, but each API call only returns 100 at most, and the API endpoint is rate limited to 15 calls per user per 15m. So if a user has >1500 users on their block list, we can’t get the whole thing at once. :(

Returns:

sequence of actor objects

Raises:
  • source.RateLimited if we hit the rate limit. The partial
  • attribute will have the list of user ids we fetched before hitting the
  • limit.
get_blocklist_ids()[source]

Returns the current user’s block list as a list of Twitter user ids.

May make multiple API calls, using cursors, to fully fetch large blocklists. https://dev.twitter.com/overview/api/cursoring

Subject to the same rate limiting as get_blocklist(), but each API call returns ~4k ids, so realistically this can actually fetch blocklists of up to 75k users at once. Beware though, many Twitter users have even more!

Returns:

sequence of string Twitter user ids

Raises:
  • source.RateLimited if we hit the rate limit. The partial
  • attribute will have the list of user ids we fetched before hitting the
  • limit.
create(obj, include_link=u'omit', ignore_formatting=False)[source]

Creates a tweet, reply tweet, retweet, or favorite.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose content will be a dict with ‘id’, ‘url’, and ‘type’ keys (all optional) for the newly created Twitter object (or None)

preview_create(obj, include_link=u'omit', ignore_formatting=False)[source]

Previews creating a tweet, reply tweet, retweet, or favorite.

Parameters:
  • obj – ActivityStreams object
  • include_link – string
  • ignore_formatting – boolean
Returns:

a CreationResult whose content will be a unicode string HTML snippet (or None)

upload_images(images)[source]

Uploads one or more images from web URLs.

https://dev.twitter.com/rest/reference/post/media/upload

Note that files and JSON bodies in media POST API requests are not included in OAuth signatures. https://developer.twitter.com/en/docs/media/upload-media/uploading-media/media-best-practices

Parameters:images – sequence of AS image objects, eg: [{‘url’: ‘http://picture’, ‘displayName’: ‘a thing’}, …]
Returns:list of string media ids or CreationResult on error
upload_video(url)[source]

Uploads a video from web URLs using the chunked upload process.

Chunked upload consists of multiple API calls:

  • command=INIT, which allocates the media id
  • command=APPEND for each 5MB block, up to 15MB total
  • command=FINALIZE

https://dev.twitter.com/rest/reference/post/media/upload-chunked https://dev.twitter.com/rest/public/uploading-media#chunkedupload

Parameters:url – string URL of images
Returns:string media id or CreationResult on error
urlopen(url, parse_response=True, **kwargs)[source]

Wraps urllib2.urlopen() and adds an OAuth signature.

base_object(obj)[source]

Returns the ‘base’ silo object that an object operates on.

Includes special handling for Twitter photo and video URLs, e.g. https://twitter.com/nelson/status/447465082327298048/photo/1 https://twitter.com/nelson/status/447465082327298048/video/1

Parameters:obj – ActivityStreams object
Returns:dict, minimal ActivityStreams object. Usually has at least id and url fields; may also have author.
tweet_to_activity(tweet)[source]

Converts a tweet to an activity.

Parameters:tweet – dict, a decoded JSON tweet
Returns:an ActivityStreams activity dict, ready to be JSON-encoded
tweet_to_object(tweet)[source]

Converts a tweet to an object.

Parameters:tweet – dict, a decoded JSON tweet
Returns:an ActivityStreams object dict, ready to be JSON-encoded
user_to_actor(user)[source]

Converts a tweet to an activity.

Parameters:user – dict, a decoded JSON Twitter user
Returns:an ActivityStreams actor dict, ready to be JSON-encoded
retweet_to_object(retweet)[source]

Converts a retweet to a share activity object.

Parameters:retweet – dict, a decoded JSON tweet
Returns:an ActivityStreams object dict
streaming_event_to_object(event)[source]

Converts a Streaming API event to an object.

https://dev.twitter.com/docs/streaming-apis/messages#Events_event

Right now, only converts favorite events to like objects.

Parameters:event – dict, a decoded JSON Streaming API event
Returns:an ActivityStreams object dict
favorites_html_to_likes(tweet, html)[source]

Converts the HTML from a favorited_popup request to like objects.

e.g. https://twitter.com/i/activity/favorited_popup?id=434753879708672001

Parameters:html – string
Returns:list of ActivityStreams like object dicts
static rfc2822_to_iso8601(time_str)[source]

Converts a timestamp string from RFC 2822 format to ISO 8601.

Example RFC 2822 timestamp string generated by Twitter:
‘Wed May 23 06:01:13 +0000 2007’
Resulting ISO 8610 timestamp string:
‘2007-05-23T06:01:13’
user_url(username)[source]

Returns the Twitter URL for a given user.

status_url(username, id)[source]

Returns the Twitter URL for a tweet from a given user with a given id.

tweet_url(tweet)[source]

Returns the Twitter URL for a tweet given a tweet object.