granary package
Reference documentation.
as2
Convert between ActivityStreams 1 and 2.
AS2: http://www.w3.org/TR/activitystreams-core/
- AS1: http://activitystrea.ms/specs/json/1.0/
http://activitystrea.ms/specs/json/schema/activity-schema.html
- granary.as2.from_as1(obj, type=None, context='https://www.w3.org/ns/activitystreams')[source]
Converts an ActivityStreams 1 activity or object to ActivityStreams 2.
- Parameters
obj – dict, AS1 activity or object
type – string, default type if type inference can’t determine a type.
context – string, included as @context
Returns: dict, AS2 activity or object
atom
Convert between ActivityStreams 1 and Atom.
Atom spec: https://tools.ietf.org/html/rfc4287 (RIP atomenabled.org)
- class granary.atom.Defaulter(init={})[source]
Bases:
collections.defaultdict
Emulates Django template behavior that returns a special default value that can continue to be referenced when an attribute or item lookup fails. Helps avoid conditionals in the template itself. https://docs.djangoproject.com/en/1.8/ref/templates/language/#variables
- __str__()
Return str(self).
- __eq__(value, /)
Return self==value.
- __weakref__
list of weak references to the object (if defined)
- granary.atom.activities_to_atom(activities, actor, title=None, request_url=None, host_url=None, xml_base=None, rels=None, reader=True)[source]
Converts ActivityStreams 1 activities to an Atom feed.
- Parameters
activities – list of ActivityStreams activity dicts
actor – ActivityStreams actor dict, the author of the feed
title – string, the feed <title> element. Defaults to ‘User feed for [NAME]’
request_url – the URL of this Atom feed, if any. Used in a link rel=”self”.
host_url – the home URL for this Atom feed, if any. Used in the top-level feed <id> element.
xml_base – the base URL, if any. Used in the top-level xml:base attribute.
rels – rel links to include. dict mapping string rel value to string URL.
reader – boolean, whether the output will be rendered in a feed reader. Currently just includes location if True, not otherwise.
- Returns
unicode string with Atom XML
- granary.atom.activity_to_atom(activity, xml_base=None, reader=True)[source]
Converts a single ActivityStreams 1 activity to an Atom entry.
Kwargs are passed through to
activities_to_atom()
.- Parameters
xml_base – the base URL, if any. Used in the top-level xml:base attribute.
reader – boolean, whether the output will be rendered in a feed reader. Currently just includes location if True, not otherwise.
- Returns
unicode string with Atom XML
- granary.atom.atom_to_activities(atom)[source]
Converts an Atom feed to ActivityStreams 1 activities.
- Parameters
atom – unicode string, Atom document with top-level <feed> element
- Returns
list of ActivityStreams activity dicts
- granary.atom.atom_to_activity(atom)[source]
Converts an Atom entry to an ActivityStreams 1 activity.
- Parameters
atom – unicode string, Atom document with top-level <entry> element
- Returns
dict, ActivityStreams activity
- granary.atom.html_to_atom(html, url=None, fetch_author=False, reader=True)[source]
Converts microformats2 HTML to an Atom feed.
- Parameters
html – unicode string
url – string URL html came from, optional
fetch_author – boolean, whether to make HTTP request to fetch rel-author link
reader – boolean, whether the output will be rendered in a feed reader. Currently just includes location if True, not otherwise.
- Returns
unicode string with Atom XML
facebook
Facebook source class. Supports both Graph API and scraping HTML.
https://developers.facebook.com/docs/graph-api/using-graph-api/
- class granary.facebook.FacebookId(user, post, comment)
Bases:
tuple
- property comment
Alias for field number 2
- property post
Alias for field number 1
- property user
Alias for field number 0
- granary.facebook.now_fn(tz=None)
Returns new datetime object representing current time local to tz.
- tz
Timezone object.
If no tz is specified, uses local timezone.
- class granary.facebook.Facebook(access_token=None, user_id=None, scrape=False, cookie_c_user=None, cookie_xs=None)[source]
Bases:
granary.source.Source
Facebook source class. See file docstring and Source class for details.
- access_token
string, optional, OAuth access token
- user_id
string, optional, current user’s id (either global or app-scoped)
- scrape
boolean, whether to scrape mbasic.facebook.com’s HTML (True) or use the API (False)
- cookie_c_user
string, optional c_user cookie to use when scraping
- cookie_xs
string, optional xs cookie to use when scraping
- __init__(access_token=None, user_id=None, scrape=False, cookie_c_user=None, cookie_xs=None)[source]
Constructor.
If an OAuth access token is provided, it will be passed on to Facebook. This will be necessary for some people and contact details, based on their privacy settings.
If scrape is True, cookie_c_user and cookie_xs must be provided.
- Parameters
access_token – string, optional OAuth access token
user_id – string, optional, current user’s id (either global or app-scoped)
scrape – boolean, whether to scrape mbasic.facebook.com’s HTML (True) or use the API (False)
cookie_c_user – string, optional c_user cookie to use when scraping
cookie_xs – string, optional xs cookie to use when scraping
- user_url(id)
Returns the URL for a user’s profile.
- get_actor(user_id=None)[source]
Returns a user as a JSON ActivityStreams actor dict.
- Parameters
user_id – string id or username. Defaults to ‘me’, ie the current user.
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, fetch_news=False, event_owner_id=None, **kwargs)[source]
Fetches posts and converts them to ActivityStreams activities.
See method docstring in source.py for details.
Likes, top-level replies (ie comments), and reactions are always included. They come from the ‘comments’, ‘likes’, and ‘reactions’ fields in the Graph API’s Post object: https://developers.facebook.com/docs/reference/api/post/
Threaded comments, ie comments in reply to other top-level comments, require an additional API call, so they’re only included if fetch_replies is True.
Mentions are never fetched or included because the API doesn’t support searching for them. https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875
- Additional args:
- fetch_news: boolean, whether to also fetch and include Open Graph news
stories (/USER/news.publishes). Requires the user_actions.news permission. Background in https://github.com/snarfed/bridgy/issues/479
- event_owner_id: string. if provided, only events owned by this user id
will be returned. avoids (but doesn’t entirely prevent) processing big non-indieweb events with tons of attendees that put us over app engine’s instance memory limit. https://github.com/snarfed/bridgy/issues/77
- get_event(event_id, owner_id=None)[source]
Returns a Facebook event post.
- Parameters
id – string, site-specific event id
owner_id – string
- Returns
dict, decoded ActivityStreams activity, or None if the event is not found or is owned by a different user than owner_id (if provided)
- get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]
Returns an ActivityStreams comment object.
- Parameters
comment_id – string comment id
activity_id – string activity id, optional
activity_author_id – string activity author id, optional
activity – activity object (optional)
Returns an ActivityStreams share activity object.
- Parameters
activity_user_id – string id of the user who posted the original activity
activity_id – string activity id
share_id – string id of the share object
activity – activity object (optional)
- get_albums(user_id=None)[source]
Fetches and returns a user’s photo albums.
- Parameters
user_id – string id or username. Defaults to ‘me’, ie the current user.
- Returns
sequence of ActivityStream album object dicts
- get_reaction(activity_user_id, activity_id, reaction_user_id, reaction_id, activity=None)[source]
Fetches and returns a reaction.
- Parameters
activity_user_id – string id of the user who posted the original activity
activity_id – string activity id
reaction_user_id – string id of the user who reacted
reaction_id – string id of the reaction. one of: ‘love’, ‘wow’, ‘haha’, ‘sad’, ‘angry’, ‘thankful’, ‘pride’, ‘care
activity – activity object (optional)
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a new post, comment, like, or RSVP.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose contents will be a dict with ‘id’ and ‘url’ keys for the newly created Facebook object (or None)
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Previews creating a new post, comment, like, or RSVP.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose contents will be a unicode string HTML snippet or None
- create_notification(user_id, text, link)[source]
Sends the authenticated user a notification.
Uses the Notifications API (beta): https://developers.facebook.com/docs/games/notifications/#impl
- Parameters
user_id – string, username or user ID
text – string, shown to the user in the notification
link – relative string URL, the user is redirected here when they click on the notification. Note that only the path and query parameters are used! they’re combined with the domain in your Facebook app’s Game App URL: https://developers.facebook.com/docs/games/services/appnotifications#parameters
Raises: urllib2.HTPPError
- post_url(post)[source]
Returns a short Facebook URL for a post.
- Parameters
post – Facebook JSON post
- comment_url(post_id, comment_id, post_author_id=None)[source]
Returns a short Facebook URL for a comment.
- Parameters
post_id – Facebook post id
comment_id – Facebook comment id
- classmethod base_id(url)[source]
Guesses the id of the object in the given URL.
- Returns
string, or None
- base_object(obj, verb=None, resolve_numeric_id=False)[source]
Returns the ‘base’ silo object that an object operates on.
This is mostly a big bag of heuristics for reverse engineering and parsing Facebook URLs. Whee.
- Parameters
obj – ActivityStreams object
verb – string, optional
resolve_numeric_id – if True, tries harder to populate the numeric_id field by making an additional API call to look up the object if necessary.
- Returns
dict, minimal ActivityStreams object. Usually has at least id, numeric_id, and url fields; may also have author.
- post_to_activity(post)[source]
Converts a post to an activity.
- Parameters
post – dict, a decoded JSON post
- Returns
an ActivityStreams activity dict, ready to be JSON-encoded
- post_to_object(post, type=None)[source]
Converts a post to an object.
TODO: handle the sharedposts field
- Parameters
post – dict, a decoded JSON post
type – string object type: None, ‘post’, or ‘comment’
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- comment_to_object(comment, post_id=None, post_author_id=None)[source]
Converts a comment to an object.
- Parameters
comment – dict, a decoded JSON comment
post_id – optional string Facebook post id. Only used if the comment id doesn’t have an embedded post id.
post_author_id – optional string Facebook post author id. Only used if the comment id doesn’t have an embedded post author id.
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
Converts a share (from /OBJECT/sharedposts) to an object.
- Parameters
share – dict, a decoded JSON share
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- user_to_actor(user)[source]
Converts a user or page to an actor.
- Parameters
user – dict, a decoded JSON Facebook user or page
- Returns
an ActivityStreams actor dict, ready to be JSON-encoded
- event_to_object(event, rsvps=None)[source]
Converts an event to an object.
- Parameters
event – dict, a decoded JSON Facebook event
rsvps – sequence, optional Facebook RSVPs
- Returns
an ActivityStreams object dict
- event_to_activity(event, rsvps=None)[source]
Converts a event to an activity.
- Parameters
event – dict, a decoded JSON Facebook event
rsvps – list of JSON Facebook RSVPs
- Returns
an ActivityStreams activity dict
- rsvp_to_object(rsvp, type=None, event=None)[source]
Converts an RSVP to an object.
The ‘id’ field will ony be filled in if event[‘id’] is provided.
- Parameters
rsvp – dict, a decoded JSON Facebook RSVP
type – optional Facebook RSVP type, one of RSVP_FIELDS
event – Facebook event object. May contain only a single ‘id’ element.
- Returns
an ActivityStreams object dict
- album_to_object(album)[source]
Converts a photo album to an object.
- Parameters
album – dict, a decoded JSON Facebook album
- Returns
an ActivityStreams object dict
- privacy_to_to(obj, type=None)[source]
Converts a Facebook privacy field to an ActivityStreams to field.
privacy is sometimes an object: https://developers.facebook.com/docs/graph-api/reference/post#fields
…and other times a string: https://developers.facebook.com/docs/graph-api/reference/album/#readfields
- Parameters
obj – dict, Facebook object (post, album, comment, etc)
- Returns
ActivityStreams to object, or None if unknown type: string object type: None, ‘post’, or ‘comment’
- Return type
- fql_stream_to_post(stream, actor=None)[source]
Converts an FQL stream row to a Graph API post.
Currently unused and untested! Use at your own risk.
https://developers.facebook.com/docs/technical-guides/fql/ https://developers.facebook.com/docs/reference/fql/stream/
TODO: place, to, with_tags, message_tags, likes, comments, etc., most require extra queries to inflate.
- Parameters
stream – dict, a row from the FQL stream table
actor – dict, a row from the FQL profile table
- Returns
dict, Graph API post
Here’s example code to query FQL and pass the results to this method:
resp = self.urlopen('https://graph.facebook.com/v2.0/fql?' + urllib.urlencode( {'q': json_dumps({ 'stream': '''\ SELECT actor_id, post_id, created_time, updated_time, attachment, privacy, message, description FROM stream WHERE filter_key IN ( SELECT filter_key FROM stream_filter WHERE uid = me()) ORDER BY created_time DESC LIMIT 50 ''', 'actors': '''\ SELECT id, name, username, url, pic FROM profile WHERE id IN (SELECT actor_id FROM #stream) '''})})) results = {q['name']: q['fql_result_set'] for q in resp['data']} actors = {a['id']: a for a in results['actors']} posts = [self.fql_stream_to_post(row, actor=actors[row['actor_id']]) for row in results['stream']]
- email_to_object(html)[source]
Converts a Facebook HTML notification email to an AS1 object.
Returns: dict, AS1 object, or None if email html couldn’t be parsed
- Parameters
html – string
- scraped_to_activities(scraped, log_html=False, **kwargs)[source]
Converts HTML from an mbasic.facebook.com timeline to AS1 activities.
- Parameters
scraped – str, HTML
log_html – boolean
kwargs – unused
Returns: tuple: ([AS activities], AS logged in actor (ie viewer))
- scraped_to_activity(scraped, log_html=False, **kwargs)[source]
Converts HTML from an mbasic.facebook.com post page to an AS1 activity.
- Parameters
scraped – str, HTML from an mbasic.facebook.com post permalink
log_html – boolean
kwargs – unused
Returns: tuple: (dict AS activity or None, AS logged in actor (ie viewer))
- merge_scraped_reactions(scraped, activity)[source]
Converts and merges scraped likes and reactions into an activity.
New likes and emoji reactions are added to the activity in ‘tags’. Existing likes and emoji reactions in ‘tags’ are ignored.
- Parameters
scraped – str, HTML from an mbasic.facebook.com/ufi/reaction/profile/browser/ page
activity – dict, AS activity to merge these reactions into
- Returns
list of dict AS like/react tag objects converted from scraped
- scraped_to_actor(scraped)[source]
Converts HTML from an mbasic.facebook.com profile about page to an AS1 actor.
- Parameters
scraped – str, HTML from an mbasic.facebook.com post permalink
Returns: dict, AS1 actor
- static parse_id(id, is_comment=False)[source]
Parses a Facebook post or comment id.
Facebook ids come in different formats:
Simple number, usually a user or post: 12
Two numbers with underscore, usually POST_COMMENT or USER_POST: 12_34
Three numbers with underscores, USER_POST_COMMENT: 12_34_56
Three numbers with colons, USER:POST:SHARD: 12:34:63 (We’re guessing that the third part is a shard in some FB internal system. In our experience so far, it’s always either 63 or the app-scoped user id for 63.)
Two numbers with colon, POST:SHARD: 12:34 (We’ve seen 0 as shard in this format.)
Four numbers with colons/underscore, USER:POST:SHARD_COMMENT: 12:34:63_56
Five numbers with colons/underscore, USER:EVENT:UNKNOWN:UNKNOWN_UNKNOWN Not currently supported! Examples: 111599105530674:998145346924699:10102446236688861:10207188792305341_998153510257216 111599105530674:195181727490727:10102446236688861:10205257726909910_195198790822354
Background:
- Parameters
id – string or integer
is_comment – boolean
- Returns
Some or all fields may be None.
- Return type
- resolve_object_id(user_id, post_id, activity=None)[source]
Resolve a post id to its Facebook object id, if any.
Used for photo posts, since Facebook has (at least) two different objects (and ids) for them, one for the post and one for each photo.
This is the same logic that we do for canonicalizing photo objects in get_activities() above.
If activity is not provided, fetches the post from Facebook.
- Parameters
user_id – string Facebook user id who posted the post
post_id – string Facebook post id
activity – optional AS activity representation of Facebook post
- Returns
Facebook object id or None
- Return type
string
- urlopen(url, _as=<class 'dict'>, **kwargs)[source]
Wraps
urllib2.urlopen()
and passes through the access token.- Parameters
_as – if not None, parses the response as JSON and passes it through _as() with this type. if None, returns the response object.
- Returns
decoded JSON object or urlopen response object
- urlopen_batch(urls)[source]
Sends a batch of multiple API calls using Facebook’s batch API.
Raises the appropriate
urllib2.HTTPError
if any individual call returns HTTP status code 4xx or 5xx.https://developers.facebook.com/docs/graph-api/making-multiple-requests
- Parameters
urls – sequence of string relative API URLs, e.g. (‘me’, ‘me/accounts’)
- Returns
sequence of responses, either decoded JSON objects (when possible) or raw string bodies
- urlopen_batch_full(requests)[source]
Sends a batch of multiple API calls using Facebook’s batch API.
Similar to urlopen_batch(), but the requests arg and return value are dicts with headers, HTTP status code, etc. Only raises
urllib2.HTTPError
if the outer batch request itself returns an HTTP error.https://developers.facebook.com/docs/graph-api/making-multiple-requests
- Parameters
requests –
sequence of dict requests in Facebook’s batch format, except that headers is a single dict, not a list of dicts, e.g.:
[{'relative_url': 'me/feed', 'headers': {'ETag': 'xyz', ...}, }, ... ]
- Returns
sequence of dict responses in Facebook’s batch format, except that body is JSON-decoded if possible, and headers is a single dict, not a list of dicts, e.g.:
[{'code': 200, 'headers': {'ETag': 'xyz', ...}, 'body': {...}, }, ... ]
flickr
Flickr source class.
Uses Flickr’s REST API https://www.flickr.com/services/api/
TODO: Fetching feeds with comments and/or favorites is very request intensive right now. It would be ideal to find a way to batch requests, make requests asynchronously, or make better calls to the API itself. Maybe use flickr.activity.userPhotos (https://www.flickr.com/services/api/flickr.activity.userPhotos.html) when group_id=SELF.
- class granary.flickr.Flickr(access_token_key, access_token_secret, user_id=None, path_alias=None)[source]
Bases:
granary.source.Source
Flickr source class. See file docstring and Source class for details.
- __init__(access_token_key, access_token_secret, user_id=None, path_alias=None)[source]
Constructor.
If they are not provided, user_id and path_alias will be looked up via the API on first use.
- Parameters
access_token_key – string, OAuth access token key
access_token_secret – string, OAuth access token secret
user_id – string, the logged in user’s Flickr nsid. (optional)
path_alias – string, the logged in user’s path_alias, replaces user_id in canonical profile and photo urls (optional)
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a photo, comment, or favorite.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose content will be a dict with ‘id’, ‘url’, and ‘type’ keys (all optional) for the newly created Flickr object (or None)
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Preview creation of a photo, comment, or favorite.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose description will be an HTML summary of what publishing will do, and whose content will be an HTML preview of the result (or None)
- delete(id)[source]
Deletes a photo. The authenticated user must have created it.
- Parameters
id – int or string, photo id to delete
Returns: CreationResult, content is Flickr API response dict
- preview_delete(id)[source]
Previews deleting a photo.
- Parameters
id – int or string, photo id to delete
Returns: CreationResult
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, **kwargs)[source]
Fetches Flickr photos and converts them to ActivityStreams activities.
See method docstring in source.py for details.
Mentions are not fetched or included because they don’t exist in Flickr. https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875
- get_actor(user_id=None)[source]
Get an ActivityStreams object of type ‘person’ given a Flickr user’s nsid. If no user_id is provided, this method will make another API request to find out the currently logged in user’s id.
- Parameters
user_id – string, optional
- Returns
dict, an ActivityStreams object
- get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]
Returns an ActivityStreams comment object.
- Parameters
comment_id – string comment id
activity_id – string activity id, optional
activity_author_id – string activity author id, ignored
activity – activity object, optional. Avoids fetching the activity.
- photo_to_activity(photo)[source]
Convert a Flickr photo to an ActivityStreams object. Takes either data in the expanded form returned by flickr.photos.getInfo or the abbreviated form returned by flickr.people.getPhotos.
- Parameters
photo – dict response from Flickr
- Returns
dict, an ActivityStreams object
- like_to_object(person, photo_activity)[source]
Convert a Flickr favorite into an ActivityStreams like tag.
- Parameters
person – dict, the person object from Flickr
photo_activity – dict, the ActivityStreams object representing the photo this like belongs to
- Returns
dict, an ActivityStreams object
- comment_to_object(comment, photo_id)[source]
Convert a Flickr comment json object to an ActivityStreams comment.
- Parameters
comment – dict, the comment object from Flickr
photo_id – string, the Flickr ID of the photo that this comment belongs to
- Returns
dict, an ActivityStreams object
- get_user_image(farm, server, author)[source]
Convert fields from a typical Flickr response into the buddy icon URL.
ref: https://www.flickr.com/services/api/misc.buddyicons.html
- user_id()[source]
Get the nsid of the currently authorized user. The first time this is called, it will invoke the flickr.people.getLimits api method.
https://www.flickr.com/services/api/flickr.people.getLimits.html
- Returns
a string
- path_alias()[source]
Get the path_alias of the currently authorized user. The first time this is called, it will invoke the flickr.people.getInfo api method.
https://www.flickr.com/services/api/flickr.people.getInfo.html
- Returns
a string
- user_url(user_id)[source]
Convert a user’s path_alias to their Flickr profile page URL.
- Parameters
user_id (string) – user’s alphanumeric nsid or path alias
- Returns
string, a profile URL
github
GitHub source class. Uses the v4 GraphQL API and the v3 REST API.
API docs: https://developer.github.com/v4/ https://developer.github.com/v3/ https://developer.github.com/apps/building-oauth-apps/authorization-options-for-oauth-apps/#web-application-flow
- class granary.github.GitHub(access_token=None)[source]
Bases:
granary.source.Source
GitHub source class. See file docstring and Source class for details.
- access_token
string, optional, OAuth access token
- __init__(access_token=None)[source]
Constructor.
- Parameters
access_token – string, optional OAuth access token
- classmethod base_id(url)[source]
Extracts and returns a USERNAME:REPO:ID id for an issue or PR.
- Parameters
url – string
- Returns
string, or None
- graphql(graphql, kwargs)[source]
Makes a v4 GraphQL API call.
- Parameters
graphql – string GraphQL operation
Returns: dict, parsed JSON response
- rest(url, data=None, parse_json=True, **kwargs)[source]
Makes a v3 REST API call.
Uses HTTP POST if data is provided, otherwise GET.
- Parameters
data – dict, JSON payload for POST requests
json – boolean, whether to parse the response body as JSON and return it as a dict. If False, returns a
requests.Response
instead.
Returns: dict decoded from JSON response if json=True, otherwise
requests.Response
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, public_only=True, **kwargs)[source]
Fetches issues and comments and converts them to ActivityStreams activities.
See
Source.get_activities_response()
for details.Not comprehensive! Uses the notifications API (v3 REST).
Also note that start_index and count are not currently supported.
https://developer.github.com/v3/activity/notifications/ https://developer.github.com/v3/issues/ https://developer.github.com/v3/issues/comments/
fetch_likes determines whether emoji reactions are fetched: https://help.github.com/articles/about-conversations-on-github#reacting-to-ideas-in-comments
The notifications API call supports Last-Modified/If-Modified-Since headers and 304 Not Changed responses. If provided, etag should be an RFC2822 timestamp, usually the exact value returned in a Last-Modified header. It will also be passed to the comments API endpoint as the since= value (converted to ISO 8601).
- get_actor(user_id=None)[source]
Fetches nd returns a user.
Args: user_id: string, defaults to current user
Returns: dict, ActivityStreams actor object
- get_comment(comment_id, **kwargs)[source]
Fetches and returns a comment.
- Parameters
comment_id – string comment id (either REST or GraphQL), of the form REPO-OWNER_REPO-NAME_ID, e.g. snarfed:bridgy:456789
Returns: dict, an ActivityStreams comment object
- render_markdown(markdown, owner, repo)[source]
Uses the GitHub API to render GitHub-flavored Markdown to HTML.
https://developer.github.com/v3/markdown/ https://github.github.com/gfm/
- Parameters
markdown – unicode string, input
repo (owner and) – strings, the repo to render in, for context. affects git hashes #XXX for issues/PRs.
Returns: unicode string, rendered HTML
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a new issue or comment.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose contents will be a dict with ‘id’ and ‘url’ keys for the newly created GitHub object (or None)
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Previews creating an issue or comment.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose contents will be a unicode string HTML snippet or None
- existing_labels(owner, repo)[source]
Fetches and returns a repo’s labels.
- Parameters
owner – string, GitHub username or org that owns the repo
repo – string
Returns: set of strings
- issue_to_object(issue)[source]
Converts a GitHub issue or pull request to ActivityStreams.
Handles both v4 GraphQL and v3 REST API issue and PR objects.
https://developer.github.com/v4/object/issue/ https://developer.github.com/v4/object/pullrequest/ https://developer.github.com/v3/issues/ https://developer.github.com/v3/pulls/
- Parameters
issue – dict, decoded JSON GitHub issue or PR
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- pr_to_object(issue)
Converts a GitHub issue or pull request to ActivityStreams.
Handles both v4 GraphQL and v3 REST API issue and PR objects.
https://developer.github.com/v4/object/issue/ https://developer.github.com/v4/object/pullrequest/ https://developer.github.com/v3/issues/ https://developer.github.com/v3/pulls/
- Parameters
issue – dict, decoded JSON GitHub issue or PR
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- comment_to_object(comment)[source]
Converts a GitHub comment to ActivityStreams.
Handles both v4 GraphQL and v3 REST API issue objects.
https://developer.github.com/v4/object/issue/ https://developer.github.com/v3/issues/
- Parameters
comment – dict, decoded JSON GitHub issue
- Returns
an ActivityStreams comment dict, ready to be JSON-encoded
- reaction_to_object(reaction, target)[source]
Converts a GitHub emoji reaction to ActivityStreams.
Handles v3 REST API reaction objects.
https://developer.github.com/v3/reactions/
- Parameters
reaction – dict, decoded v3 JSON GitHub reaction
target – dict, ActivityStreams object of reaction
- Returns
an ActivityStreams reaction dict, ready to be JSON-encoded
- user_to_actor(user)[source]
Converts a GitHub user to an ActivityStreams actor.
Handles both v4 GraphQL and v3 REST API user objects.
https://developer.github.com/v4/object/user/ https://developer.github.com/v3/users/
- Parameters
user – dict, decoded JSON GitHub user
- Returns
an ActivityStreams actor dict, ready to be JSON-encoded
instagram
Instagram source class.
Instagram’s API doesn’t tell you if a user has marked their account private or not, so the Audience Targeting ‘to’ field is currently always set to @public. http://help.instagram.com/448523408565555 https://groups.google.com/forum/m/#!topic/instagram-api-developers/DAO7OriVFsw https://groups.google.com/forum/#!searchin/instagram-api-developers/private
- class granary.instagram.Instagram(access_token=None, allow_comment_creation=False, scrape=False, cookie=None)[source]
Bases:
granary.source.Source
Instagram source class. See file docstring and Source class for details.
- __init__(access_token=None, allow_comment_creation=False, scrape=False, cookie=None)[source]
Constructor.
If an OAuth access token is provided, it will be passed on to Instagram. This will be necessary for some people and contact details, based on their privacy settings.
- Parameters
access_token – string, optional OAuth access token
allow_comment_creation – boolean, optionally disable comment creation, useful if the app is not approved to create comments.
scrape – boolean, whether to scrape instagram.com’s HTML (True) or use the API (False)
cookie – string, optional sessionid cookie to use when scraping.
- get_actor(user_id=None, **kwargs)[source]
Returns a user as a JSON ActivityStreams actor dict.
- Parameters
user_id – string id or username. Defaults to ‘self’, ie the current user.
kwargs – if scraping, passed through to get_activities_response(). Raises AssertionError if provided and not scraping.
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, scrape=False, cookie=None, ignore_rate_limit=False, **kwargs)[source]
Fetches posts and converts them to ActivityStreams activities.
See method docstring in source.py for details. app_id is ignored. Supports min_id, but not ETag, since Instagram doesn’t support it.
http://instagram.com/developer/endpoints/users/#get_users_feed http://instagram.com/developer/endpoints/users/#get_users_media_recent
Likes are always included, regardless of the fetch_likes kwarg. They come bundled in the ‘likes’ field of the API Media object: http://instagram.com/developer/endpoints/media/#
Mentions are never fetched or included because the API doesn’t support searching for them. https://github.com/snarfed/bridgy/issues/523#issuecomment-155523875
Shares are never fetched or included since there is no share feature.
Instagram only supports search over hashtags, so if search_query is set, it must begin with #.
May populate a custom ‘ig_like_count’ property in media objects. (Currently only when scraping.)
- Parameters
scrape – if True, scrapes HTML from instagram.com instead of using the API. Populates the user’s actor object in the ‘actor’ response field. Useful for apps that haven’t yet been approved in the new permissions approval process. Currently only supports group_id=SELF. Also supports passing a shortcode as activity_id as well as the internal API id. http://developers.instagram.com/post/133424514006/instagram-platform-update
cookie – string, only used if scrape=True
ignore_rate_limit – boolean, for scraping, always make an HTTP request, even if we’ve been rate limited recently
** – see
Source.get_activities_response()
- get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]
Returns an ActivityStreams comment object.
- Parameters
comment_id – string comment id
activity_id – string activity id, optional
activity_author_id – string activity author id. Ignored.
activity – activity object, optional. Avoids fetching the activity.
Not implemented. Returns None. Resharing isn’t a feature of Instagram.
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a new comment or like.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
- a CreationResult. if successful, content will have and ‘id’ and
’url’ keys for the newly created Instagram object
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Preview a new comment or like.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
- a CreationResult. if successful, content and description
will describe the new instagram object.
- media_to_activity(media)[source]
Converts a media to an activity.
http://instagram.com/developer/endpoints/media/#get_media
- Parameters
media – JSON object retrieved from the Instagram API
- Returns
an ActivityStreams activity dict, ready to be JSON-encoded
- media_to_object(media)[source]
Converts a media to an object.
- Parameters
media – JSON object retrieved from the Instagram API
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- comment_to_object(comment, media_id, media_url)[source]
Converts a comment to an object.
- Parameters
comment – JSON object retrieved from the Instagram API
media_id – string
media_url – string
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- like_to_object(liker, media_id, media_url)[source]
Converts a like to an object.
- Parameters
liker – JSON object from the Instagram API, the user who does the liking
media_id – string
media_url – string
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- user_to_actor(user)[source]
Converts a user to an actor.
- Parameters
user – JSON object from the Instagram API
- Returns
an ActivityStreams actor dict, ready to be JSON-encoded
- base_object(obj)[source]
Extends the default base_object() to avoid using shortcodes as object ids.
- static id_to_shortcode(id)[source]
Converts a media id to the shortcode used in its instagram.com URL.
Based on http://carrot.is/coding/instagram-ids , which determined that shortcodes are just URL-safe base64 encoded ids.
- scraped_to_activities(html, cookie=None, count=None, fetch_extras=False)[source]
Converts scraped Instagram HTML to ActivityStreams activities.
The input HTML may be from:
a user’s feed, eg https://www.instagram.com/ while logged in
a user’s profile, eg https://www.instagram.com/snarfed/
a photo or video, eg https://www.instagram.com/p/BBWCSrfFZAk/
- Parameters
html – unicode string
cookie – string, optional sessionid cookie to be used for subsequent HTTP fetches, if necessary.
count – integer, number of activities to return, None for all
fetch_extras – whether to make extra HTTP fetches to get likes, etc.
- Returns
tuple, ([ActivityStreams activities], ActivityStreams viewer actor)
- html_to_activities(html, cookie=None, count=None, fetch_extras=False)
Converts scraped Instagram HTML to ActivityStreams activities.
The input HTML may be from:
a user’s feed, eg https://www.instagram.com/ while logged in
a user’s profile, eg https://www.instagram.com/snarfed/
a photo or video, eg https://www.instagram.com/p/BBWCSrfFZAk/
- Parameters
html – unicode string
cookie – string, optional sessionid cookie to be used for subsequent HTTP fetches, if necessary.
count – integer, number of activities to return, None for all
fetch_extras – whether to make extra HTTP fetches to get likes, etc.
- Returns
tuple, ([ActivityStreams activities], ActivityStreams viewer actor)
- scraped_to_activity(html, **kwargs)[source]
Converts HTML from photo/video permalink page to an AS1 activity.
- Parameters
html – string, HTML from a photo/video page on instagram.com
kwargs – passed through to scraped_to_activities
- Returns
(AS activity or None, AS logged in actor (ie viewer))
- Return type
- scraped_to_actor(html, **kwargs)[source]
Extracts and returns the logged in actor from any Instagram HTML.
- Parameters
html – unicode string
Returns: dict, AS1 actor
- merge_scraped_comments(scraped, activity)[source]
Converts and merges scraped comments (replies) into an activity.
- Parameters
scraped – str or sequence, scraped JSON comments
activity – dict, AS activity to merge these comments into
- Returns
list of dict AS comment objects converted from scraped
- Raises
ValueError – if scraped is not valid JSON
- merge_scraped_reactions(scraped, activity)[source]
Converts and merges scraped likes and reactions into an activity.
New likes and emoji reactions are added to the activity in ‘tags’. Existing likes and emoji reactions in ‘tags’ are ignored.
Supports both legacy and v2 Instagram JSON.
- Parameters
scraped – str or dict, scraped JSON likes
activity – dict, AS activity to merge these reactions into
- Returns
list of dict AS like tag objects converted from scraped
- Raises
ValueError – if scraped is not valid JSON
jsonfeed
Convert between ActivityStreams and JSON Feed.
JSON Feed spec: https://jsonfeed.org/version/1
- granary.jsonfeed.activities_to_jsonfeed(activities, actor=None, title=None, feed_url=None, home_page_url=None)[source]
Converts ActivityStreams activities to a JSON feed.
- Parameters
activities – sequence of ActivityStreams activity dicts
actor – ActivityStreams actor dict, the author of the feed
title – string, the feed title
home_page_url – string, the home page URL
feed_url – the URL of the JSON Feed, if any. Included in the feed_url field.
- Returns
dict, JSON Feed data, ready to be JSON-encoded
- granary.jsonfeed.jsonfeed_to_activities(jsonfeed)[source]
Converts a JSON feed to ActivityStreams activities and actor.
- Parameters
jsonfeed – dict, JSON Feed data
- Returns
(activities, actor) tuple, where activities and actor are both ActivityStreams object dicts
- Raises
ValueError, if jsonfeed isn't a valid JSON Feed dict –
mastodon
Mastodon source class.
Mastodon is an ActivityPub implementation, but it also has a REST + OAuth 2 API independent of AP. API docs: https://docs.joinmastodon.org/api/
May also be used for services with Mastodon-compatible APIs, eg Pleroma: https://docs-develop.pleroma.social/backend/API/differences_in_mastoapi_responses/
- class granary.mastodon.Mastodon(instance, access_token, user_id=None, truncate_text_length=None)[source]
Bases:
granary.source.Source
Mastodon source class. See file docstring and Source class for details.
- instance
string, base URL of Mastodon instance, eg https://mastodon.social/
- user_id
integer, optional, current user’s id (not username!) on this instance
- access_token
string, optional, OAuth access token
- __init__(instance, access_token, user_id=None, truncate_text_length=None)[source]
Constructor.
If user_id is not provided, it will be fetched via the API.
- Parameters
instance – string, base URL of Mastodon instance, eg https://mastodon.social/
user_id – string or integer, optional, current user’s id (not username!) on this instance
access_token – string, optional OAuth access token
truncate_text_length – int, optional character limit for toots, overrides the default of 500
- classmethod embed_post(obj)[source]
Returns the HTML string for embedding a toot.
https://dev.twitter.com/docs/embedded-tweets
- Parameters
obj – AS1 dict with at least url, and optionally also content.
Returns: string, HTML
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, include_shares=True, fetch_events=False, fetch_mentions=False, search_query=None, start_index=0, count=0, cache=None, **kwargs)[source]
Fetches toots and converts them to ActivityStreams activities.
See
Source.get_activities_response()
for details.
- get_actor(user_id=None)[source]
Fetches and returns an account.
Args: user_id: string, defaults to current account
Returns: dict, ActivityStreams actor object
- get_comment(comment_id, **kwargs)[source]
Fetches and returns a comment.
- Parameters
comment_id – string status id
**kwargs – unused
Returns: dict, ActivityStreams object
- Raises
ValueError – if comment_id is invalid
- status_to_activity(status)[source]
Converts a status to an activity.
- Parameters
status – dict, a decoded JSON status
- Returns
an ActivityStreams activity dict, ready to be JSON-encoded
- status_to_object(status)[source]
Converts a status to an object.
- Parameters
status – dict, a decoded JSON status
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- user_to_actor(account)[source]
Converts a Mastodon account to an AS1 actor.
- Parameters
account – dict, Mastodon account
Returns: dict, AS1 actor
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a status (aka toot), reply, boost (aka reblog), or favorite.
https://docs.joinmastodon.org/methods/statuses/
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns: CreationResult whose content will be a dict with ‘id’, ‘url’, and
‘type’ keys (all optional) for the newly created object (or None)
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Preview creating a status (aka toot), reply, boost (aka reblog), or favorite.
https://docs.joinmastodon.org/methods/statuses/
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns: CreationResult whose content will be a unicode string HTML
snippet (or None)
- base_object(obj)[source]
Returns the ‘base’ Mastodon object that an object operates on.
If the object is a reply, boost, or favorite of a Mastodon post - on any instance - this returns that post object. The id in the returned object is the id of that remote post on the local instance. (As a Mastodon style id, ie an integer in a string, not a tag URI.)
Uses Mastodon’s search API on the local instance to determine whether a URL is a Mastodon post, and if it is, to find or generate an id for it on the local instance.
Discovered via https://mastodon.social/@jkreeftmeijer/101245063526942536
- Parameters
obj – ActivityStreams object
- Returns
dict, minimal ActivityStreams object. Usually has at least id; may also have url, author, etc.
- upload_media(media)[source]
Uploads one or more images or videos from web URLs.
https://docs.joinmastodon.org/methods/statuses/media/ https://docs.joinmastodon.org/user/posting/#attachments
- Parameters
media – sequence of AS image or stream objects, eg: [{‘url’: ‘http://picture’, ‘displayName’: ‘a thing’}, …]
Returns: list of string media ids for uploaded files
- delete(id)[source]
Deletes a toot. The authenticated user must have authored it.
- Parameters
id – int or string, toot id (on local instance) to delete
Returns: CreationResult, content is dict with url and id fields
- preview_delete(id)[source]
Previews deleting a toot.
- Parameters
id – int or string, toot id (on local instance) to delete
Returns: CreationResult
- get_blocklist_ids()[source]
Returns the current user’s block list as a list of integer account ids.
May make multiple API calls to fully fetch large block lists. https://docs.joinmastodon.org/methods/accounts/blocks/
- Returns
sequence of integer Mastodon account ids on the current instance
meetup
Meetup.com source class.
- class granary.meetup.Meetup(access_token)[source]
Bases:
granary.source.Source
- classmethod embed_post(obj)[source]
Returns the HTML string for embedding an RSVP from Meetup.com.
- Parameters
obj –
obj – AS1 dict with at least url, and optionally also content.
Returns: string, HTML
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a new object: a post, comment, like, share, or RSVP.
Subclasses should override this. Different sites will support different functionality; check each subclass for details. The actor will usually be the authenticated user.
- Parameters
obj – ActivityStreams object. At minimum, must have the content field. objectType is strongly recommended.
include_link – string. ‘include’, ‘omit’, or ‘if truncated’; whether to include a link to the object (if it has one) in the content.
ignore_formatting – whether to use content text as is, instead of converting its HTML to plain text styling (newlines, etc.)
- Returns
- contents will be a dict. The dict may be None or empty. If
the newly created object has an id or permalink, they’ll be provided in the values for ‘id’ and ‘url’.
- Return type
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Previews creating a new object: a post, comment, like, share, or RSVP.
Returns HTML that previews what
create()
with the same object will do.Subclasses should override this. Different sites will support different functionality; check each subclass for details. The actor will usually be the authenticated user.
- Parameters
obj – ActivityStreams object. At minimum, must have the content field. objectType is strongly recommended.
include_link – string. Whether to include a link to the object (if it has one) in the content.
ignore_formatting – whether to use content text as is, instead of converting its HTML to plain text styling (newlines, etc.)
- Returns
contents will be a unicode string HTML snippet (or None)
- Return type
microformats2
Convert ActivityStreams to microformats2 HTML and JSON.
Microformats2 specs: http://microformats.org/wiki/microformats2 ActivityStreams 1 specs: http://activitystrea.ms/specs/
- granary.microformats2.get_string_urls(objs)[source]
Extracts string URLs from a list of either string URLs or mf2 dicts.
Many mf2 properties can contain either string URLs or full mf2 objects, e.g. h-cites. in-reply-to is the most commonly used example: http://indiewebcamp.com/in-reply-to#How_to_consume_in-reply-to
- Parameters
objs – sequence of either string URLs or embedded mf2 objects
- Returns
list of string URLs
- granary.microformats2.get_html(val)[source]
Returns a string value that may have HTML markup.
- Parameters
value – mf2 property value, either string or {‘html’: ‘<p>str</p>’, ‘value’: ‘str’} dict
- Returns
string or None
- granary.microformats2.activity_to_json(activity, **kwargs)[source]
Converts an ActivityStreams activity to microformats2 JSON.
- Parameters
activity – dict, a decoded JSON ActivityStreams activity
kwargs – passed to object_to_json
- Returns
dict, decoded microformats2 JSON
- granary.microformats2.object_to_json(obj, trim_nulls=True, entry_class='h-entry', default_object_type=None, synthesize_content=True)[source]
Converts an ActivityStreams object to microformats2 JSON.
- Parameters
obj – dict, a decoded JSON ActivityStreams object
trim_nulls – boolean, whether to remove elements with null or empty values
entry_class – string or sequence, the mf2 class(es) that entries should be given (e.g. ‘h-cite’ when parsing a reference to a foreign entry). defaults to ‘h-entry’
default_object_type – string, the ActivityStreams objectType to use if one is not present. defaults to None
synthesize_content – whether to generate synthetic content if the object doesn’t have its own, e.g. ‘likes this.’ or ‘shared this.’
- Returns
dict, decoded microformats2 JSON
- granary.microformats2.json_to_object(mf2, actor=None, fetch_mf2=False)[source]
Converts a single microformats2 JSON item to an ActivityStreams object.
Supports h-entry, h-event, h-card, and other single item times. Does not yet support h-feed.
- Parameters
mf2 – dict, decoded JSON microformats2 object
actor – optional author AS actor object. usually comes from a rel=”author” link. if mf2 has its own author, that will override this.
fetch_mf2 – boolean, whether to fetch additional pages via HTTP if necessary, e.g. to determine authorship: https://indieweb.org/authorship
- Returns
dict, ActivityStreams object
- granary.microformats2.html_to_activities(html, url=None, actor=None, id=None)[source]
Converts a microformats2 HTML h-feed to ActivityStreams activities.
- Parameters
html – unicode string HTML or
requests.Response
url – optional string URL that HTML came from
actor – optional author AS actor object for all activities. usually comes from a rel=”author” link.
id – string, optional id of specific element to extract and parse. defaults to the whole page.
- Returns
list of ActivityStreams activity dicts
- granary.microformats2.activities_to_html(activities)[source]
Converts ActivityStreams activities to a microformats2 HTML h-feed.
- Parameters
obj – dict, a decoded JSON ActivityStreams object
- Returns
string, the content field in obj with the tags in the tags field converted to links if they have startIndex and length, otherwise added to the end.
- granary.microformats2.object_to_html(obj, parent_props=None, synthesize_content=True)[source]
Converts an ActivityStreams object to microformats2 HTML.
Features:
linkifies embedded tags and adds links for other tags
linkifies embedded URLs
adds links, summaries, and thumbnails for attachments and checkins
adds a “via SOURCE” postscript
- Parameters
obj – dict, a decoded JSON ActivityStreams object
parent_props – list of strings, the properties of the parent object where this object is embedded, e.g. [‘u-repost-of’]
synthesize_content – whether to generate synthetic content if the object doesn’t have its own, e.g. ‘likes this.’ or ‘shared this.’
- Returns
string, the content field in obj with the tags in the tags field converted to links if they have startIndex and length, otherwise added to the end.
- granary.microformats2.json_to_html(obj, parent_props=None)[source]
Converts a microformats2 JSON object to microformats2 HTML.
See object_to_html for details.
- Parameters
obj – dict, a decoded microformats2 JSON object
parent_props – list of strings, the properties of the parent object where this object is embedded, e.g. ‘u-repost-of’
- Returns
string HTML
- granary.microformats2.hcard_to_html(hcard, parent_props=None)[source]
Renders an h-card as HTML.
- Parameters
hcard – dict, decoded JSON h-card
parent_props – list of strings, the properties of the parent object where this object is embedded, e.g. [‘p-author’]
- Returns
string, rendered HTML
- granary.microformats2.render_content(obj, include_location=True, synthesize_content=True, render_attachments=False, render_image=False, white_space_pre=True)[source]
Renders the content of an ActivityStreams object as HTML.
Includes tags, mentions, and non-note/article attachments. (Note/article attachments are converted to mf2 children in object_to_json and then rendered in json_to_html.)
Note that the returned HTML is included in Atom as well as HTML documents, so it must be HTML4 / XHTML, not HTML5! All tags must be closed, etc.
- Parameters
obj – decoded JSON ActivityStreams object
include_location – boolean, whether to render location, if provided
synthesize_content – boolean, whether to generate synthetic content if the object doesn’t have its own, e.g. ‘likes this.’ or ‘shared this.’
render_attachments – boolean, whether to render attachments, eg links, images, audio, and video
render_image – boolean, whether to render the object’s image(s)
white_space_pre – boolean, whether to wrap in CSS white-space: pre. If False, newlines will be converted to <br> tags instead. Background: https://indiewebcamp.com/note#Indieweb_whitespace_thinking
- Returns
string, rendered HTML
- granary.microformats2.find_author(parsed, **kwargs)[source]
Returns the author of a page as a ActivityStreams actor dict.
- Parameters
parsed – dict, parsed mf2 object (ie return value from mf2py.parse())
kwargs – passed through to mf2util.find_author()
- granary.microformats2.get_title(mf2)[source]
Returns an mf2 object’s title, ie its name.
- Parameters
mf2 – dict, parsed mf2 object (ie return value from mf2py.parse())
Returns: string title, possibly ellipsized
- granary.microformats2.first_props(props)[source]
Converts a multiply-valued dict to singly valued.
- Parameters
props – dict of properties, where each value is a sequence
- Returns
corresponding dict with just the first value of each sequence, or ‘’ if the sequence is empty
- granary.microformats2.tags_to_html(tags, classname, visible=True)[source]
Returns an HTML string with links to the given tag objects.
- Parameters
tags – decoded JSON ActivityStreams objects.
classname – class for span to enclose tags in
visible – boolean, whether to visibly include displayName
- granary.microformats2.author_display_name(hcard)[source]
Returns a human-readable string display name for an h-card object.
- granary.microformats2.maybe_linked_name(props)[source]
Returns the HTML for a p-name with an optional u-url inside.
- Parameters
props – multiply-valued properties dict
- Returns
string HTML
- granary.microformats2.img(src, alt='')[source]
Returns an <img> string with the given src, class, and alt.
- Parameters
src – string url or dict with value and (optionally) alt
alt – string, alt attribute value, or None
- Returns
string
- granary.microformats2.vid(src, poster='')[source]
Returns an <video> string with the given src and class
- Parameters
src – string, url of the video
poster – sring, optional. url of the poster or preview image
- Returns
string
- granary.microformats2.aud(src)[source]
Returns an <audio> string with the given src and class
- Parameters
src – string, url of the audio
- Returns
string
- granary.microformats2.maybe_linked(text, url, linked_classname=None, unlinked_classname=None)[source]
Wraps text in an <a href=…> iff a non-empty url is provided.
- Parameters
text – string
url – string or None
linked_classname – string, optional class attribute to use if url
unlinked_classname – string, optional class attribute to use if not url
- Returns
string
- granary.microformats2.maybe_datetime(str, classname)[source]
Returns a <time datetime=…> elem if str is non-empty.
- Parameters
str – string RFC339 datetime or None
classname – string class name
- Returns
string
- granary.microformats2.size_to_bytes(size)[source]
Converts a string file size to an integer number of bytes.
- Parameters
size – string, may be either integer bytes or human-readable approximation, eg 7MB or 1.23 kb
Returns: integer, bytes, or None if size can’t be parsed
- granary.microformats2.prefix_image_urls(activity, prefix)[source]
Adds a prefix to all image url fields, eg to inject a caching proxy.
Skips any image url fields that already start with the prefix. URLs are not URL-encode before adding the prefix. (This is currently used with https://cloudimage.io/ , which doesn’t URL-decode.)
- Parameters
activity – dict, AS1 activity. Modified in place.
prefix – string
pixelfed
Pixelfed source class, heavily based on Mastodon.
Pixelfed’s API is a clone of Mastodon’s. https://docs.pixelfed.org/technical-documentation/api-v1.html
- class granary.pixelfed.Pixelfed(instance, access_token, user_id=None, truncate_text_length=None)[source]
Bases:
granary.mastodon.Mastodon
Pixelfed source class.
reddit
Reddit source class.
Reddit API docs: https://github.com/reddit-archive/reddit/wiki/API https://www.reddit.com/dev/api https://www.reddit.com/prefs/apps
PRAW API docs: https://praw.readthedocs.io/
- class granary.reddit.Reddit(refresh_token)[source]
Bases:
granary.source.Source
Reddit source class. See file docstring and Source class for details.
- classmethod post_id(url)[source]
Guesses the post id of the given URL.
- Parameters
url – string
- Returns
string, or None
- praw_to_actor(praw_user)[source]
Converts a PRAW Redditor to an actor.
Makes external calls to fetch data from the Reddit API.
https://praw.readthedocs.io/en/latest/code_overview/models/redditor.html
Caches fetched user data for 5m to avoid repeating user profile API requests when fetching multiple comments or posts from the same author. Background: https://github.com/snarfed/bridgy/issues/1021
Ideally this would be part of PRAW, but they seem uninterested: https://github.com/praw-dev/praw/issues/131 https://github.com/praw-dev/praw/issues/1140
- Parameters
user – PRAW Redditor object
- Returns
an ActivityStreams actor dict, ready to be JSON-encoded
- user_to_actor(user)[source]
Converts a dict user to an actor.
- Parameters
user – JSON user
- Returns
an ActivityStreams actor dict, ready to be JSON-encoded
- praw_to_object(thing, type)[source]
Converts a PRAW object to an AS1 object.
Currently only returns public content.
Note that this will make external API calls to lazily load some attributes.
- Parameters
thing – a PRAW object, Submission or Comment
type – string to denote whether to get submission or comment content
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- praw_to_activity(thing, type)[source]
Converts a PRAW submission or comment to an activity.
Note that this will make external API calls to lazily load some attributes.
https://praw.readthedocs.io/en/latest/code_overview/models/submission.html https://praw.readthedocs.io/en/latest/code_overview/models/comment.html
- Parameters
thing – a PRAW object, Submission or Comment
type – string to denote whether to get submission or comment content
- Returns
an ActivityStreams activity dict, ready to be JSON-encoded
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=None, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, **kwargs)[source]
Fetches submissions and ActivityStreams activities.
Currently only implements activity_id, search_query and fetch_replies.
- get_actor(user_id=None)[source]
PLACEHOLDER. Returns an empty dict.
Only here because the granary.io API needs this to emit Atom data.
TODO: implement.
- get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]
Returns an ActivityStreams comment object.
- Parameters
comment_id – string comment id
activity_id – string activity id, Ignored
activity_author_id – string activity author id. Ignored.
activity – activity object, Ignored
rss
Convert between ActivityStreams and RSS 2.0.
RSS 2.0 spec: http://www.rssboard.org/rss-specification
Apple iTunes Podcasts feed requirements: https://help.apple.com/itc/podcasts_connect/#/itc1723472cb
Notably: * Valid RSS 2.0. * Each podcast item requires <guid>. * Images should be JPEG or PNG, 1400x1400 to 3000x3000. * HTTP server that hosts assets and files should support range requests.
- granary.rss.from_activities(activities, actor=None, title=None, feed_url=None, home_page_url=None, hfeed=None)[source]
Converts ActivityStreams activities to an RSS 2.0 feed.
- Parameters
activities – sequence of ActivityStreams activity dicts
actor – ActivityStreams actor dict, the author of the feed
title – string, the feed title
feed_url – string, the URL for this RSS feed
home_page_url – string, the home page URL
hfeed – dict, parsed mf2 h-feed, if available
- Returns
unicode string with RSS 2.0 XML
source
Source base class.
Based on the OpenSocial ActivityStreams REST API: http://opensocial-resources.googlecode.com/svn/spec/2.0.1/Social-API-Server.xml#ActivityStreams-Service
Uses the ‘to’ field of the Audience Targeting extension to indicate an activity’s privacy settings. It’s set to a group with alias @public or @private, or unset if unknown. http://activitystrea.ms/specs/json/targeting/1.0/#anchor3
- class granary.source.CreationResult(content, description, abort, error_plain, error_html)
Bases:
tuple
- property abort
Alias for field number 2
- property content
Alias for field number 0
- property description
Alias for field number 1
- property error_html
Alias for field number 4
- property error_plain
Alias for field number 3
- exception granary.source.RateLimited(*args, **kwargs)[source]
Bases:
BaseException
Raised when an API rate limits us, and we may have a partial result.
- partial
the partial result, if any. Usually a list.
- __weakref__
list of weak references to the object (if defined)
- granary.source.html_to_text(html, baseurl='', **kwargs)[source]
Converts string html to string text with html2text.
- Parameters
baseurl – str, base URL to use when resolving relative URLs. Passed through to HTML2Text().
**kwargs – html2text options https://github.com/Alir3z4/html2text/blob/master/docs/usage.md#available-options
- granary.source.load_json(body, url)[source]
Utility method to parse a JSON string. Raises HTTPError 502 on failure.
- granary.source.creation_result(content=None, description=None, abort=False, error_plain=None, error_html=None)[source]
Create a new
CreationResult
.create()
andpreview_create()
use this to provide a detailed description of publishing failures. If abort is False, we should continue looking for an entry to publish; if True, we should immediately inform the user. error_plain text is sent in response to failed publish webmentions; error_html will be displayed to the user when publishing interactively.- Parameters
content – a string HTML snippet for
preview_create()
or a dict forcreate()
description – string HTML snippet describing the publish action, e.g. ‘@-reply’ or ‘RSVP yes to this event’. The verb itself is surrounded by a <span class=”verb”> to allow styling. May also include <a> link(s) and embedded silo post(s).
abort – a boolean
error_plain – a string
error_html – a string
- Returns
- granary.source.object_type(obj)[source]
Returns the object type, or the verb if it’s an activity object.
Details: http://activitystrea.ms/specs/json/1.0/#activity-object
- Parameters
obj – decoded JSON ActivityStreams object
- Returns
ActivityStreams object type
- Return type
string
- granary.source.merge_by_id(obj, field, new)[source]
Merges new items by id into a field in an existing AS1 object.
Merges new items by id into the given field. If it exists, it must be a list. Requires all existing and new items in the field to have ids.
- Parameters
obj – dict, AS1 object
field – str, name of field to merge new items into
new – sequence of AS1 dicts
- class granary.source.SourceMeta(name, bases, class_dict)[source]
Bases:
type
Source metaclass. Registers all source classes in the sources global.
- class granary.source.Source[source]
Bases:
object
Abstract base class for a source (e.g. Facebook, Twitter).
Concrete subclasses must override the class constants below and implement
get_activities()
.Class constants:
DOMAIN: string, the source’s domain
BASE_URL: optional, the source’s base url
NAME: string, the source’s human-readable name
FRONT_PAGE_TEMPLATE: string, the front page child template filename
AUTH_URL: string, the url for the “Authenticate” front page link
EMBED_POST: string, the HTML for embedding a post. Should have a %(url)s placeholder for the post URL and (optionally) a %(content)s placeholder for the post content.
POST_ID_RE: regexp, optional, matches valid post ids. Used in post_id().
HTML2TEXT_OPTIONS: dict mapping string html2text option names to values https://github.com/Alir3z4/html2text/blob/master/docs/usage.md#available-options
- TRUNCATE_TEXT_LENGTH: integer, optional. Character limit to truncate to.
Defaults to Twitter’s limit, 280 characters as of 2019-10-12.
- TRUNCATE_URL_LENGTH: integer, optional. Number of characters that URLs count
for. Defaults to Twitter’s, 23 as of 2019-10-12.
OPTIMIZED_COMMENTS: boolean, whether
get_comment()
is optimized and only fetches the requested comment. If False,get_comment()
fetches many or all of the post’s comments to find the requested one.
- get_actor(user_id=None)[source]
Fetches and returns a user.
Args: user_id: string, defaults to current user
Returns: dict, ActivityStreams actor object
- get_activities(*args, **kwargs)[source]
Fetches and returns a list of activities.
See get_activities_response() for args and kwargs.
- Returns
list, ActivityStreams activity dicts
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, include_shares=True, fetch_events=False, fetch_mentions=False, search_query=None, scrape=False, **kwargs)[source]
Fetches and returns ActivityStreams activities and response details.
Subclasses should override this. See
get_activities()
for an alternative that just returns the list of activities.If user_id is provided, only that user’s activity(s) are included. start_index and count determine paging, as described in the spec: http://activitystrea.ms/draft-spec.html#anchor14
app id is just object id: http://opensocial-resources.googlecode.com/svn/spec/2.0/Social-Data.xml#appId
group id is string id of group or @self, @friends, @all, @search: http://opensocial-resources.googlecode.com/svn/spec/2.0/Social-Data.xml#Group-ID
The fetch_* kwargs all default to False because they often require extra API round trips. Some sources return replies, likes, and shares in the same initial call, so they may be included even if you don’t set their kwarg to True.
- Parameters
user_id – string, defaults to the currently authenticated user
group_id – string, one of ‘@self’, ‘@all’, ‘@friends’, ‘@search’. defaults to ‘@friends’
app_id – string
activity_id – string
start_index – int >= 0
count – int >= 0
etag – string, optional ETag to send with the API request. Results will only be returned if the ETag has changed. Should include enclosing double quotes, e.g. ‘“ABC123”’
min_id – only return activities with ids greater than this
cache – object with get(key), set_multi(dict), and delete_multi(list) methods. In practice, this is App Engine’s memcache interface: https://developers.google.com/appengine/docs/python/memcache/functions Used to cache data that’s expensive to regenerate, e.g. API calls.
fetch_replies – boolean, whether to fetch each activity’s replies also
fetch_likes – boolean, whether to fetch each activity’s likes also
include_shares – boolean, whether to include share activities
fetch_shares – boolean, whether to fetch each activity’s shares also
fetch_events – boolean, whether to fetch the user’s events also
fetch_mentions – boolean, whether to fetch posts that mention the user
search_query – string, an optional search query, only for use with @search group_id
scrape – boolean, whether to scrape activities from HTML (etc) instead of using an API. Not supported by all sources.
kwargs – some sources accept extra kwargs. See their docs for details.
- Returns
- response values based on OpenSocial ActivityStreams REST API.
http://opensocial-resources.googlecode.com/svn/spec/2.0.1/Social-API-Server.xml#ActivityStreams-Service http://opensocial-resources.googlecode.com/svn/spec/2.0.1/Core-Data.xml
It has these keys: * items: list of activity dicts * startIndex: int or None * itemsPerPage: int * totalResults: int or None (e.g. if it can ‘t be calculated efficiently) * filtered: False * sorted: False * updatedSince: False * etag: string etag returned by the API’s initial call to get activities
- Return type
- Raises
ValueError – if any argument is invalid for this source
NotImplementedError – if the source doesn’t support the requested operation, e.g. Facebook doesn’t support search.
- classmethod make_activities_base_response(activities, *args, **kwargs)[source]
Generates a base response dict for
get_activities_response()
.See
get_activities()
for args and kwargs.
- scraped_to_activities(scraped, count=None, fetch_extras=False, cookie=None)[source]
Converts scraped HTML (or JSON, etc) to AS activities.
Used for scraping data from the web instead of using an API. Useful for sources with APIs that are restricted or have difficult approval processes.
- Parameters
scraped – str, scraped data from a feed of posts
count – integer, number of activities to return, None for all
fetch_extras – whether to make extra HTTP fetches to get likes, etc.
cookie – string, optional cookie to be used for subsequent HTTP fetches, if necessary.
- Returns
([AS activities], AS logged in actor (ie viewer))
- Return type
- scraped_to_activity(scraped)[source]
Converts scraped HTML (or JSON, etc) to a single AS activity.
Used for scraping data from the web instead of using an API. Useful for sources with APIs that are restricted or have difficult approval processes.
- Parameters
scraped – str, scraped data from a single post permalink
- Returns
(AS activity or None, AS logged in actor (ie viewer))
- Return type
- scraped_to_actor(scraped)[source]
Converts HTML from a profile page to an AS1 actor.
- Parameters
html – string, HTML from a profile page
Returns: dict, AS1 actor
- merge_scraped_reactions(scraped, activity)[source]
Converts and merges scraped likes and reactions into an activity.
New likes and emoji reactions are added to the activity in ‘tags’. Existing likes and emoji reactions in ‘tags’ are ignored.
- Parameters
scraped – str, HTML or JSON with likes and/or emoji reactions
activity – dict, AS activity to merge these reactions into
- Returns
list of dict AS like/react tag objects converted from scraped
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a new object: a post, comment, like, share, or RSVP.
Subclasses should override this. Different sites will support different functionality; check each subclass for details. The actor will usually be the authenticated user.
- Parameters
obj – ActivityStreams object. At minimum, must have the content field. objectType is strongly recommended.
include_link – string. ‘include’, ‘omit’, or ‘if truncated’; whether to include a link to the object (if it has one) in the content.
ignore_formatting – whether to use content text as is, instead of converting its HTML to plain text styling (newlines, etc.)
- Returns
- contents will be a dict. The dict may be None or empty. If
the newly created object has an id or permalink, they’ll be provided in the values for ‘id’ and ‘url’.
- Return type
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Previews creating a new object: a post, comment, like, share, or RSVP.
Returns HTML that previews what
create()
with the same object will do.Subclasses should override this. Different sites will support different functionality; check each subclass for details. The actor will usually be the authenticated user.
- Parameters
obj – ActivityStreams object. At minimum, must have the content field. objectType is strongly recommended.
include_link – string. Whether to include a link to the object (if it has one) in the content.
ignore_formatting – whether to use content text as is, instead of converting its HTML to plain text styling (newlines, etc.)
- Returns
contents will be a unicode string HTML snippet (or None)
- Return type
- delete(id)[source]
Deletes a post.
Generally only supports posts that were authored by the authenticating user.
- Parameters
id – silo object id
- Returns
CreationResult
- preview_delete(id)[source]
Previews deleting a post.
- Parameters
id – silo object id
- Returns
CreationResult
- get_event(event_id)[source]
Fetches and returns an event.
- Parameters
id – string, site-specific event id
- Returns
decoded ActivityStreams activity, or None
- Return type
- get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]
Fetches and returns a comment.
Subclasses should override this.
- Parameters
comment_id – string comment id
activity_id – string activity id, optional
activity_author_id – string activity author id, optional. Needed for some sources (e.g. Facebook) to construct the comment permalink.
activity – activity object, optional. May avoid an API call if provided.
Returns: dict, ActivityStreams comment object
- Raises
ValueError – if any argument is invalid for this source
- get_like(activity_user_id, activity_id, like_user_id, activity=None)[source]
Fetches and returns a ‘like’.
Default implementation that fetches the activity and its likes, then searches for a like by the given user. Subclasses should override this if they can optimize the process.
- Parameters
activity_user_id – string id of the user who posted the original activity
activity_id – string activity id
like_user_id – string id of the user who liked the activity
activity – activity object, optional. May avoid an API call if provided.
Returns: dict, ActivityStreams ‘like’ activity object
- get_reaction(activity_user_id, activity_id, reaction_user_id, reaction_id, activity=None)[source]
Fetches and returns a ‘reaction’.
Default implementation that fetches the activity and its reactions, then searches for this specific reaction. Subclasses should override this if they can optimize the process.
- Parameters
activity_user_id – string id of the user who posted the original activity
activity_id – string activity id
reaction_user_id – string id of the user who reacted
reaction_id – string id of the reaction
activity – activity object, optional. May avoid an API call if provided.
Returns: dict, ActivityStreams ‘reaction’ activity object
Fetches and returns a ‘share’.
- Parameters
activity_user_id – string id of the user who posted the original activity
activity_id – string activity id
share_id – string id of the share object or the user who shared it
activity – activity object, optional. May avoid an API call if provided.
Returns: dict, an ActivityStreams ‘share’ activity object
- get_rsvp(activity_user_id, event_id, user_id, event=None)[source]
Fetches and returns an RSVP.
- Parameters
activity_user_id – string id of the user who posted the event. unused.
event_id – string event id
user_id – string user id
event – AS event activity (optional)
Returns: dict, an ActivityStreams RSVP activity object
- get_blocklist()[source]
Fetches and returns the current user’s block list.
…ie the users that the current user is blocking. The exact semantics of “blocking” vary from silo to silo.
- Returns
sequence of actor objects
- get_blocklist_ids()[source]
Returns the current user’s block list as a list of silo-specific user ids.
- Returns
- sequence of user ids, not globally unique across other sources. May be
integers or strings.
- user_to_actor(user)[source]
Converts a user to an actor.
The returned object will have at least a ‘url’ field. If the user has multiple URLs, there will also be a ‘urls’ list field whose elements are dicts with ‘value’: URL.
- Parameters
user – dict, a decoded JSON silo user object
- Returns
ActivityStreams actor, ready to be JSON-encoded
- Return type
- static postprocess_activity(activity)[source]
Does source-independent post-processing of an activity, in place.
Right now just populates the title field.
- Parameters
activity – activity dict
- static postprocess_object(obj)[source]
Does source-independent post-processing of an object, in place.
Populates location.position based on latitude and longitude.
- Parameters
object – object dict
- static original_post_discovery(activity, domains=None, cache=None, include_redirect_sources=True, include_reserved_hosts=True, **kwargs)[source]
Discovers original post links.
This is a variation on http://indiewebcamp.com/original-post-discovery . It differs in that it finds multiple candidate links instead of one, and it doesn’t bother looking for MF2 (etc) markup because the silos don’t let you input it. More background: https://github.com/snarfed/bridgy/issues/51#issuecomment-136018857
Original post candidates come from the upstreamDuplicates, attachments, and tags fields, as well as links and permashortlinks/permashortcitations in the text content.
- Parameters
activity – activity dict
domains – optional sequence of domains. If provided, only links to these domains will be considered original and stored in upstreamDuplicates. (Permashortcitations are exempt.)
include_redirect_sources – boolean, whether to include URLs that redirect as well as their final destination URLs
include_reserved_hosts – boolean, whether to include domains on reserved TLDs (eg foo.example) and local hosts (eg http://foo.local/, http://my-server/)
cache – deprecated, unused
kwargs – passed to requests.head() when following redirects
- Returns
([string original post URLs], [string mention URLs]) tuple
- static is_public(obj)[source]
Returns True if the object is public, False if private, None if unknown.
…according to the Audience Targeting extension: http://activitystrea.ms/specs/json/targeting/1.0/
Expects values generated by this library: objectType group, alias @public, @unlisted, or @private.
Also, important point: this defaults to true, ie public. Bridgy depends on that and prunes the to field from stored activities in Response objects (in bridgy/util.prune_activity()). If the default here ever changes, be sure to update Bridgy’s code.
- static add_rsvps_to_event(event, rsvps)[source]
Adds RSVP objects to an event’s attending fields, in place.
- Parameters
event – ActivityStreams event object
rsvps – sequence of ActivityStreams RSVP activity objects
- static get_rsvps_from_event(event)[source]
Returns RSVP objects for an event’s attending fields.
- Parameters
event – ActivityStreams event object
- Returns
sequence of ActivityStreams RSVP activity objects
- static activity_changed(before, after, log=False)[source]
Returns whether two activities or objects differ meaningfully.
Only compares a few fields: object type, verb, content, location, and image. Notably does not compare author and published/updated timestamps.
This has been tested on Facebook posts, comments, and event RSVPs (only content and rsvp_status change) and Google+ posts and comments (content, updated, and etag change). Twitter tweets and Instagram photo captions and comments can’t be edited.
- Parameters
before – dicts, ActivityStreams activities or objects
after – dicts, ActivityStreams activities or objects
- Returns
boolean
- static append_in_reply_to(before, after)[source]
appends the inReplyTos from the before object to the after object, in place
- Parameters
before – dicts, ActivityStreams activities or objects
after – dicts, ActivityStreams activities or objects
- classmethod embed_post(obj)[source]
Returns the HTML string for embedding a post object.
- Parameters
obj – AS1 dict with at least url, and optionally also content.
Returns: string, HTML
- base_object(obj)[source]
Returns the ‘base’ silo object that an object operates on.
For example, if the object is a comment, this returns the post that it’s a comment on. If it’s an RSVP, this returns the event. The id in the returned object is silo-specific, ie not a tag URI.
Subclasses may override this.
- Parameters
obj – ActivityStreams object
- Returns
dict, minimal ActivityStreams object. Usually has at least id; may also have url, author, etc.
- classmethod base_id(url)[source]
Guesses the id of the object in the given URL.
- Returns
string, or None
- truncate(content, url, include_link, type=None, quote_url=None)[source]
Shorten text content to fit within a character limit.
Character limit and URL character length are taken from the TRUNCATE_TEXT_LENGTH and TRUNCATE_URL_LENGTH class constants
- Parameters
content – string
url – string
include_link – string
type – string, optional: ‘article’, ‘note’, etc.
quote_url – string URL, optional. If provided, it will be appended to the content, after truncating.
Return: string, the possibly shortened and ellipsized text
- __weakref__
list of weak references to the object (if defined)
twitter
Twitter source class.
Uses the v1.1 REST API: https://developer.twitter.com/en/docs/api-reference-index
The Audience Targeting ‘to’ field is set to @public or @private based on whether the tweet author’s ‘protected’ field is true or false. https://dev.twitter.com/docs/platform-objects/users
- class granary.twitter.OffsetTzinfo(utc_offset=0)[source]
Bases:
datetime.tzinfo
A simple, DST-unaware tzinfo from given utc offset in seconds.
- __init__(utc_offset=0)[source]
Constructor.
- Parameters
utc_offset – Offset of time zone from UTC in seconds
- utcoffset(dt)[source]
datetime -> timedelta showing offset from UTC, negative values indicating West of UTC
- __weakref__
list of weak references to the object (if defined)
- class granary.twitter.Twitter(access_token_key, access_token_secret, username=None, scrape_headers=None)[source]
Bases:
granary.source.Source
Twitter source class. See file docstring and Source class for details.
- __init__(access_token_key, access_token_secret, username=None, scrape_headers=None)[source]
Constructor.
Twitter now requires authentication in v1.1 of their API. You can get an OAuth access token by creating an app here: https://dev.twitter.com/apps/new
- Parameters
access_token_key – string, OAuth access token key
access_token_secret – string, OAuth access token secret
username – string, optional, the current user. Used in e.g. preview/create.
scrape_headers – dict, optional, with string HTTP header keys and values to use when scraping likes
- get_actor(screen_name=None)[source]
Returns a user as a JSON ActivityStreams actor dict.
- Parameters
screen_name – string username. Defaults to the current user.
- get_activities_response(user_id=None, group_id=None, app_id=None, activity_id=None, start_index=0, count=0, etag=None, min_id=None, cache=None, fetch_replies=False, fetch_likes=False, fetch_shares=False, include_shares=False, fetch_events=False, fetch_mentions=False, search_query=None, scrape=False, **kwargs)[source]
Fetches posts and converts them to ActivityStreams activities.
XXX HACK: this is currently hacked for bridgy to NOT pass min_id to the request for fetching activity tweets themselves, but to pass it to all of the requests for filling in replies, retweets, etc. That’s because we want to find new replies and retweets of older initial tweets. TODO: find a better way.
See
source.Source.get_activities_response()
for details. app_id is ignored. min_id is translated to Twitter’s since_id.The code for handling ETags (and 304 Not Changed responses and setting If-None-Match) is here, but unused right now since Twitter evidently doesn’t support ETags. From https://dev.twitter.com/discussions/5800 : “I’ve confirmed with our team that we’re not explicitly supporting this family of features.”
Likes (nee favorites) are scraped from twitter.com, since Twitter’s REST API doesn’t offer a way to fetch them. You can also get them from the Streaming API, though, and convert them with streaming_event_to_object(). https://dev.twitter.com/docs/streaming-apis/messages#Events_event
Shares (ie retweets) are fetched with a separate API call per tweet: https://dev.twitter.com/docs/api/1.1/get/statuses/retweets/%3Aid
However, retweets are only fetched for the first 15 tweets that have them, since that’s Twitter’s rate limit per 15 minute window. :( https://dev.twitter.com/docs/rate-limiting/1.1/limits
Quote tweets are fetched by searching for the possibly quoted tweet’s ID, using the OR operator to search up to 5 IDs at a time, and then checking the quoted_status_id_str field https://dev.twitter.com/overview/api/tweets#quoted_status_id_str
Use the group_id @self to retrieve a user_id’s timeline. If user_id is None or @me, it will return tweets for the current API user.
group_id can be used to specify the slug of a list for which to return tweets. By default the current API user’s lists will be used, but lists owned by other users can be fetched by explicitly passing a username to user_id, e.g. to fetch tweets from the list @exampleuser/example-list you would call get_activities(user_id=’exampleuser’, group_id=’example-list’).
Twitter replies default to including a mention of the user they’re replying to, which overloads mentions a bit. When fetch_mentions is True, we determine that a tweet mentions the current user if it @-mentions their username and:
it’s not a reply, OR
it’s a reply, but not to the current user, AND * the tweet it’s replying to doesn’t @-mention the current user
- Raises
NotImplementedError – if fetch_likes is True but scrape_headers was not provided to the constructor.
- fetch_replies(activities, min_id=None)[source]
Fetches and injects Twitter replies into a list of activities, in place.
Includes indirect replies ie reply chains, not just direct replies. Searches for @-mentions, matches them to the original tweets with in_reply_to_status_id_str, and recurses until it’s walked the entire tree.
- Parameters
activities – list of activity dicts
- Returns
same activities list
- fetch_mentions(username, tweets, min_id=None)[source]
Fetches a user’s @-mentions and returns them as ActivityStreams.
Tries to only include explicit mentions, not mentions automatically created by @-replying. See the
get_activities()
docstring for details.- Parameters
username – string
tweets – list of Twitter API objects. used to find quote tweets quoting them.
min_id – only return activities with ids greater than this
- Returns
list of activity dicts
- get_comment(comment_id, activity_id=None, activity_author_id=None, activity=None)[source]
Returns an ActivityStreams comment object.
- Parameters
comment_id – string comment id
activity_id – string activity id, optional
activity_author_id – string activity author id. Ignored.
activity – activity object, optional
Returns an ActivityStreams ‘share’ activity object.
- Parameters
activity_user_id – string id of the user who posted the original activity
activity_id – string activity id
share_id – string id of the share object
activity – activity object, optional
- get_blocklist()[source]
Returns the current user’s block list.
May make multiple API calls, using cursors, to fully fetch large blocklists. https://dev.twitter.com/overview/api/cursoring
Block lists may have up to 10k users, but each API call only returns 100 at most, and the API endpoint is rate limited to 15 calls per user per 15m. So if a user has >1500 users on their block list, we can’t get the whole thing at once. :(
- Returns
sequence of actor objects
- Raises
attribute will have the list of user ids we fetched before hitting the –
limit. –
- get_blocklist_ids()[source]
Returns the current user’s block list as a list of Twitter user ids.
May make multiple API calls, using cursors, to fully fetch large blocklists. https://dev.twitter.com/overview/api/cursoring
Subject to the same rate limiting as get_blocklist(), but each API call returns ~4k ids, so realistically this can actually fetch blocklists of up to 75k users at once. Beware though, many Twitter users have even more!
- Returns
sequence of string Twitter user ids
- Raises
attribute will have the list of user ids we fetched before hitting the –
limit. –
- create(obj, include_link='omit', ignore_formatting=False)[source]
Creates a tweet, reply tweet, retweet, or favorite.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose content will be a dict with ‘id’, ‘url’, and ‘type’ keys (all optional) for the newly created Twitter object (or None)
- preview_create(obj, include_link='omit', ignore_formatting=False)[source]
Previews creating a tweet, reply tweet, retweet, or favorite.
- Parameters
obj – ActivityStreams object
include_link – string
ignore_formatting – boolean
- Returns
a CreationResult whose content will be a unicode string HTML snippet (or None)
- upload_images(images)[source]
Uploads one or more images from web URLs.
https://dev.twitter.com/rest/reference/post/media/upload
Note that files and JSON bodies in media POST API requests are not included in OAuth signatures. https://developer.twitter.com/en/docs/media/upload-media/uploading-media/media-best-practices
- Parameters
images – sequence of AS image objects, eg: [{‘url’: ‘http://picture’, ‘displayName’: ‘a thing’}, …]
- Returns
list of string media ids or
CreationResult
on error
- upload_video(url)[source]
Uploads a video from web URLs using the chunked upload process.
Chunked upload consists of multiple API calls:
command=INIT, which allocates the media id
command=APPEND for each 5MB block, up to 15MB total
command=FINALIZE
https://developer.twitter.com/en/docs/media/upload-media/uploading-media/chunked-media-upload
- Parameters
url – string URL of images
- Returns
string media id or
CreationResult
on error
- delete(id)[source]
Deletes a tweet. The authenticated user must have authored it.
- Parameters
id – int or string, tweet id to delete
Returns: CreationResult, content is Twitter API response dict
- preview_delete(id)[source]
Previews deleting a tweet.
- Parameters
id – int or string, tweet id to delete
Returns: CreationResult
- urlopen(url, parse_response=True, **kwargs)[source]
Wraps
urllib2.urlopen()
and adds an OAuth signature.
- base_object(obj)[source]
Returns the ‘base’ silo object that an object operates on.
Includes special handling for Twitter photo and video URLs, e.g. https://twitter.com/nelson/status/447465082327298048/photo/1 https://twitter.com/nelson/status/447465082327298048/video/1
- Parameters
obj – ActivityStreams object
- Returns
dict, minimal ActivityStreams object. Usually has at least id and url fields; may also have author.
- tweet_to_activity(tweet)[source]
Converts a tweet to an activity.
- Parameters
tweet – dict, a decoded JSON tweet
- Returns
an ActivityStreams activity dict, ready to be JSON-encoded
- tweet_to_object(tweet)[source]
Converts a tweet to an object.
- Parameters
tweet – dict, a decoded JSON tweet
- Returns
an ActivityStreams object dict, ready to be JSON-encoded
- user_to_actor(user)[source]
Converts a user to an actor.
- Parameters
user – dict, a decoded JSON Twitter user
- Returns
an ActivityStreams actor dict, ready to be JSON-encoded
- retweet_to_object(retweet)[source]
Converts a retweet to a share activity object.
- Parameters
retweet – dict, a decoded JSON tweet
- Returns
an ActivityStreams object dict
- streaming_event_to_object(event)[source]
Converts a Streaming API event to an object.
https://dev.twitter.com/docs/streaming-apis/messages#Events_event
Right now, only converts favorite events to like objects.
- Parameters
event – dict, a decoded JSON Streaming API event
- Returns
an ActivityStreams object dict
- static rfc2822_to_iso8601(time_str)[source]
Converts a timestamp string from RFC 2822 format to ISO 8601.
- Example RFC 2822 timestamp string generated by Twitter:
‘Wed May 23 06:01:13 +0000 2007’
- Resulting ISO 8610 timestamp string:
‘2007-05-23T06:01:13’