`` tags instead of converting them to
\`s so that GitHub renders HTML entities like ``>`` inside them
instead of leaving them escaped.
`Background. `__
- JSON Feed:
- Handle malformed attachments better.
- microformats2:
- Don’t crash on string ``context`` fields.
- ``html_to_activities()``: limit to ``h-entry``, ``h-event``, and
``h-cite`` items
(`#192 `__).
- The ``cache`` kwarg to ``Source.original_post_discovery()`` now has
no effect. ``webutil.util.follow_redirects()`` has its own built in
caching now.
- Added Meetup.com support for publishing RSVPs.
.. _section-8:
2.2 - 2019-11-02
~~~~~~~~~~~~~~~~
- Add Mastodon support!
- Add Python 3.7 support, and improve overall Python 3 compatibility.
- Update a number of dependencies.
- Switch from Python’s built in ``json`` module to
`ujson `__ to speed up JSON
parsing and encoding.
- Add ``duration`` and ``size`` support to ActivityStreams 1 and 2,
RSS, and microformats2 HTML and JSON. `microformats2 support is still
emerging for both `__.
Both integer seconds and `ISO 8601 string
durations `__ are
supported for ``duration``. Integer bytes is used for ``size``
everywhere. microformats2 HTML also includes human-readable strings,
eg ``5.1 MB``.
(`#169 `__)
- Twitter:
- ``[preview]_create()``: detect attempts to upload `images over
5MB `__
and return an error.
- Facebook:
- Add ``get_activities(scrape=True)`` for scraping HTML from
`m.facebook.com `__. Requires ``c_user``
and ``xs`` cookies from a logged in session
(`snarfed/bridgy#886 `__).
- `Upgrade Graph API version from 2.10 to
4.0. `__
- Atom:
- Bug fix for de-duping images in attachments.
- RSS:
- Wrap all ```` element contents in ``CDATA`` sections.
- Render images in ```` with HTML ```` tags
(`#175 `__).
- ``from_activities()`` bug fix: don’t crash when converting
multiple attachments to enclosures in a single item. (RSS only
supports one enclosure per item, so we now only include the first,
and log a warning if the activity has more.)
.. _section-9:
2.1 - 2019-09-04
~~~~~~~~~~~~~~~~
- Convert AS2 ``Mention`` tags to AS1 ``objectType`` ``mention``
(non-standard) and vice versa
(`snarfed/bridgy-fed#46 `__).
- Twitter:
- Bug fix for large block list fetches that get rate limited after a
few successful requests.
- Handle HTTP 403 + error code 200 when fetching retweets for a
protected or otherwise unavailable tweet
(`bridgy#688 `__).
- Demote @-mentions from
`person-tags `__ to
`mentions `__. Specifically, this
means they’ll no longer get rendered with ``u-category`` mf2.
- Instagram:
- Disabled in the REST API entirely due to Instagram’s aggressive
rate limiting and blocking
(`bridgy#655 `__).
- Update scraping to handle replies in new
``edge_media_to_parent_comment`` field
(`#164 `__).
- Use cookie for all scraping HTTP requests, not just for likes.
- microformats2:
- Revise whitespace handling; use ``white-space: pre`` CSS in HTML
output.
- Facebook:
- Bug fix: don’t interpret ``photo.php`` as username in post URLs.
- Atom:
- Switch from ``white-space: pre`` CSS back to converting newlines
to ``
``\ s because some feed readers (`eg
NewsBlur `__)
follow it too strictly and don’t even line wrap.
- RSS:
- Default title to ellipsized content.
.. _section-10:
2.0 - 2019-03-01
~~~~~~~~~~~~~~~~
*Breaking change*: drop Google+ since `it shuts down in
March `__. Notably, this
removes the ``googleplus`` module.
.. _section-11:
1.15 - 2019-02-28
~~~~~~~~~~~~~~~~~
- Add RSS 2.0 output!
(`#124 `__)
- All silos:
- Switch users’ primary URLs from web site to silo profile
(`#158 `__).
- GitHub:
- Don’t enclose bare URLs in ``<``/``>``
(`snarfed/bridgy#850 `__).
- Atom:
- Bug fix for actors and attachments with multiple image URLs.
- Bug fix for attachment author objects with no properties.
- Google+:
- Drop from web UI and REST API since `consumer Google+ is shutting
down
entirely `__
(`more `__).
- Switch from deprecated global API endpoint to G+ endpoint.
Background in
`snarfed/bridgy#846 `__,
`Google blog
post `__
`and
docs `__.
- Instagram:
- Fix individual photo/video link urls for multi-photo/video posts.
- Handle `user-provided alt
text `__
(`#159 `__).
- Twitter:
- Update max video upload size from 5MB to 512MB
(`#162 `__).
- ``/url``: Return HTTP 400 when fetching the user’s URL results in an
infinite redirect.
.. _section-12:
1.14 - 2018-11-12
~~~~~~~~~~~~~~~~~
Add ``delete()``. Currently includes Twitter and Flickr support. \*
Instagram: \* Make extra HTTP fetch (with cookie) to get individual
likes
(`snarfed/bridgy#840 `__).
\* Update scraping logic to handle feed HTML changes. \* Link @-mentions
in comments as well as photo/video captions. \* GitHub: \*
``create``/``preview_create`` bug fixes for issues and comments on
private repos. \* Handle HTTP 410 Gone responses from REST API, eg when
a repo has been deleted or issues for the repo disabled. \* Twitter: \*
Add ``delete()`` and ``preview_delete()`` for deleting tweets. \*
Flickr: \* Add ``delete()`` and ``preview_delete()`` for deleting
photos. \* microformats2: \* Add
`follow-of `__ support. \* Only use
quotation-of property for quote tweets, not URLs.
(`#155 `__) \* If a tag
has startIndex/length, it gets linkified in the content, so don’t also
emit an mf2 child or HTML h-cite for it.
(`#155 `__ \* Atom: \*
Encode ``&``\ s in author URL and email address too. (Thanks
`sebsued `__!) \* AS2: \* Add ``Follow``
support.
.. _section-13:
1.13 - 2018-08-08
~~~~~~~~~~~~~~~~~
- Twitter:
- Support ISO 8601 formatted created_at timestamps, which the
`archive download
uses `__,
as well as RFC 2822 from the API.
- ``create()`` and ``preview_create()``: support RSVPs. Tweet them
as normal tweets with the RSVP content.
(`snarfed/bridgy#818 `__)
- ``create()`` and ``preview_create()``: support alt text for
images, via AS1 ``displayName``.
(`snarfed/bridgy#756 `__).
- Instagram:
- Add global rate limiting lock for scraping. If a scraping HTTP
request gets a 429 or 503 response, we refuse to make more
requests for 5m, and instead short circuit and return the same
error. This can be overridden with a new ``ignore_rate_limit``
kwarg to ``get_activities()``.
- GitHub:
- Add ``tag`` support to ``create``/``preview_create`` to add
label(s) to existing issues
(`snarfed/bridgy#811 `__).
- Escape HTML characters (``<``, ``>``, and ``&``) in content in
``create()`` and ``preview_create()``
(`snarfed/bridgy#810 `__).
- ``get_activities()`` and ``get_comment()`` now return
``ValueError`` instead of ``AssertionError`` on malformed
``activity_id`` and ``comment_id`` args, respectively.
- ``get_activities()`` bug fix for issues/PRs with no body text.
- Switch from GraphQL to REST API for creating comments and
reactions, since GraphQL hits authorization errors on many org
repos.
(`snarfed/bridgy#824 `__)
- Improve GraphQL support for comments and users.
- Atom:
- Shorten and ellipsize feed title when necessary
(`#144 `__).
- microformats2:
- Upgrade mf2py to improve a few things like `implied p-name
detection `__
and whitespace handling
(`#142 `__, fixes
`#145 `__,
`snarfed/bridgy#756 `__,
`snarfed/bridgy#828 `__).
- Support ``alt`` attribute in ```` tags
(`snarfed/bridgy#756 `__).
.. _section-14:
1.12 - 2018-03-24
~~~~~~~~~~~~~~~~~
- Add Python 3 support! Granary now requires either Python 2.7+ or
Python 3.3+.
- Instagram:
- Fix scraping profile pages.
- Twitter:
- Update character counting to handle Twitter change that now
auto-links *all* ccTLDs.
`Background. `__
- GitHub:
- Bug fix for ``get_activities()`` with deleted issues and repos.
- microformats2:
- ``object_to_json()``: convert tags to simple strings in the
``category`` property, not full nested objects like ``h-card``\ s
(`#141 `__).
- Special case GitHub issues that are in-reply-to a repo or its
``/issues`` URL to be objectType ``issue``.
- Render simple string categories in HTML output.
This release is intentionally small and limited in scope to contain any
impact of the Python 3 migration. It *should* be a noop for existing
Python 2 users, and we’ve tested thoroughly, but I’m sure there are
still bugs. Please file issues if you notice anything broken!
.. _section-15:
1.11 - 2018-03-09
~~~~~~~~~~~~~~~~~
- Add GitHub!
- ``get_activities()`` supports issues and pull requests, including
comments and reactions. It’s currently based on notifications, so
it’s best effort, not comprehensive, and only includes recently
active issues/PRs.
- ``create()`` and ``preview_create()`` support issues, comments,
`stars `__, and
`reactions `__.
- Twitter:
- Prefer MP4 and other video/… content types to HLS (.m3u8) etc.
`Background. `__
- Prefer HTTPS URLs for media images.
- ``get_activities()``: Support @-prefixed usernames in ``user_id``.
- Facebook:
- Support new `recurring aka multi-instance
events `__.
``create()`` and ``preview_create()`` now only support RSVPs to
individual instances of multi-instance events, to match the
Facebook API itself.
- Try harder to find original (full) sized photo URLs, specifically
``_o.jpg`` files instead of ``_s.jpg``.
- ``create()`` bug fix for photo and image URLs with unicode
characters.
- Fixed bug where ``get_activities(user_id=...)`` included the
authenticated user’s own recent photos, albums, and news
publishes.
- Instagram:
- Extract more user (``author``) data from scraped profile pages.
- Fix home page feed scraping.
- microformats2, Atom:
- Add enclosures for image attachments.
- Bug fixes for rendering image, video, and audio attachments inside
shares and attachments. De-dupe images.
- microformats2:
- Handle simple string-only author properties.
- Add ``fetch_mf2`` kwarg to ``json_to_object()`` for fetching
additional pages over HTTP to determine authorship.
- Generate explicit blank ``p-name`` in HTML to prevent old flawed
`implied p-name
handling `__
(`#131 `__).
- Fix ``share`` verb handling in ``activity_to_json()`` and
``activities_to_html()``
(`#134 `__).
- Remember which content contains HTML, preserve newlines in it, and
don’t translate those newlines to ``
``\ s
(`#130 `__).
- Atom:
- Fix timezone bugs in ``updated`` and ``published``.
- JSON Feed:
- Omit title from items if it’s the same as the content. (Often
caused by microformats2’s implied ``p-name`` logic.)
.. _section-16:
1.10 - 2017-12-10
~~~~~~~~~~~~~~~~~
- Moved web site and REST API to `granary.io `__!
`granary-demo.appspot.com `__ now
301 redirects.
- Twitter:
- Update the publish character limit to 280.
`Background. `__
- Fix a `bug in preview_create that auto-linked @-mentions inside
URLs `__,
e.g. Medium posts.
- Support videos and animated GIFs in ``get_activities()`` etc.
- Instagram:
- Add cookie query param to REST API to allow scraping that logged
in user’s feed.
- HTML (including Atom content):
- Render image, video, and audio attachments more often and
consistently.
- Include microformats2 ``u-photo``, ``u-video``, and ``u-audio``
classes more often and consistently.
- Atom:
- Add ``atom_to_activities()`` for converting full feed documents.
- Add to REST API and web UI.
- Include source URL in ``rel=alternate`` link as well as
actor/author URL
(`#151 `__).
- JSON Feed:
- Fix bug that omitted title in some cases
(`#122 `__).
.. _section-17:
1.9 - 2017-10-24
~~~~~~~~~~~~~~~~
- Add `ActivityStreams
2.0 `__! New ``as2``
module includes ``to_as1()`` and ``from_as1()`` functions. Currently
supported: articles, notes, replies, likes, reposts, events, RSVPs,
tags, attachments.
- Atom:
- Add new ``atom_to_activity()`` function for converting Atom to
AS1.
- Add email field to author, if provided.
- JSON Feed:
- Raise ValueError on bad (non-dict) input.
- REST API:
- Add ``as2`` value for ``format`` and ``input``. Revise existing
ActivityStreams and microformats2 value names to ``as1``,
``as1-xml``, and ``mf2-json``. Old values ``activitystreams``,
``json``, ``json-mf2``, and ``xml`` are still accepted, but
deprecated.
.. _section-18:
1.8 - 2017-08-29
~~~~~~~~~~~~~~~~
- Add `JSON Feed `__ support to both library and
REST API.
- Twitter:
- Add ``get_blocklist()``.
- Bug fix for creating replies, favorites, or retweets of video
URLs, e.g. https://twitter.com/name/status/123/video/1 .
- Bug fix for parsing favorites HTML to handle a small change on
Twitter’s side.
- ``post_id()`` now validates ids more strictly before returning
them.
- Facebook:
- Improve heuristic for determining privacy of wall posts from other
users.
- Support GIFs in comments (attachment types
``animated_image_autoplay`` and ``animated_image_share``).
- Upgrade Graph API from
`v2.6 `__
to
`v2.10 `__.
- Instagram:
- Update scraping to handle new home page (ie news feed) JSON
schema, which changed sometime around 2017-02-27. (Profile pages
and individual photo/video permalinks still haven’t changed yet.)
- microformats2:
- Add `u-featured `__ to
ActivityStreams ``image``.
- Improve ``h-event`` support.
- Minor whitespace change (added
.. raw:: html
) when rendering locations as HTML.
- ``post_id()`` now validates ids more strictly before returning
them.
- Fix bugs in converting latitude and longitude between
ActivityStreams and mf2.
- Google+:
- Update HTML scraping to handle changed serialized JSON data
format.
- Atom:
- Add new ``activity_to_atom()`` function that renders a single
top-level ```` instead of ````.
- Add new ``reader`` query param for toggling rendering decisions
that are specific to feed readers. Right now, just affects
location: it’s rendered in the content when ``reader=true`` (the
default), omitted when ``reader=false``.
- Include author name when rendering attached articles and notes
(e.g. quote tweets).
- Only include AS ``activity:object-type`` and ``activity:verb``
elements when they have values.
- Render AS image and mf2 u-photo if they’re not already in content.
- Render ``thr:in-reply-to`` from ``object.inReplyTo`` as well as
``activity.context.inReplyTo``.
- REST API:
- Fix bugs in html => json-mf2 and html => html conversions.
- Upgrade brevity to 0.2.14 for a couple
`bug `__
`fixes `__.
.. _section-19:
1.7 - 2017-02-27
~~~~~~~~~~~~~~~~
- microformats2:
- Interpret h-cite and
`u-quotation-of `__
(experimental) as attachments, e.g. for quote tweets.
- Convert `audio `__ and
`video `__ properties to AS
attachments.
- Twitter:
- Linkify @-mentions and hashtags in ``preview_create()``.
- Support creating quote tweets from attachments with Twitter URLs.
- When converting quote tweets to AS, strip quoted tweet URL from
end of text.
- Raise ValueError when ``get_activities()`` is passed
``group_id='@search'`` but not ``search_query``.
- Instagram:
- Improve HTML scraping error handling.
- Support `multi-photo/video
posts `__.
- Facebook:
- Disable creating “interested” RSVPs, since Facebook’s API doesn’t
allow it.
- Atom:
- Support `media
enclosures `__
for audio and video attachments.
- Source.get_activities(): start raising ValueError on bad argument
values, notably invalid Facebook and Twitter ids and Instagram search
queries.
- Fix rendering and linkifying content with Unicode high code points
(ie above the 16-bit Basic Multilingual Plane), including some emoji,
on “narrow” builds of Python 2 with ``--enable-unicode=ucs2``, which
is the default on Mac OS X, Windows, and older \*nix.
.. _section-20:
1.6 - 2016-11-26
~~~~~~~~~~~~~~~~
- Twitter:
- Handle new “extended” tweets with hidden reply-to @-mentions and
trailing URLs for media, quote tweets, etc. Background:
https://dev.twitter.com/overview/api/upcoming-changes-to-tweets
- Bug fix: ensure like.author.displayName is a plain unicode string
so that it can be pickled normally, e.g. by App Engine’s memcache.
- Bug fix: handle names with emoji correctly in
favorites_html_to_likes().
- Bug fix: handle search queries with unicode characters.
- Atom:
- Render full original quoted tweet in retweets of quote tweets.
- microformats2 HTML:
- Optionally follow and fetch rel=“author” links.
- Improve mapping between microformats2 and ActivityStreams ‘photo’
types. (mf2 ‘photo’ type is a note or article *with* a photo, but
AS ‘photo’ type *is* a photo. So, map mf2 photos to underlying
type without photo.)
- Support location properties beyond h-card, e.g. h-adr, h-geo,
u-geo, and even when properties like latitude and longitude appear
at the top level.
- Error handling: return HTTP 502 for non-JSON API responses, 504 for
connection failures.
.. _section-21:
1.5 - 2016-08-25
~~~~~~~~~~~~~~~~
- REST API:
- Support tag URI for user id, app id, and activity id.
- Twitter:
- Better error message when uploading a photo with an unsupported
type.
- Only include original quote tweets, not retweets of them.
- Skip fetching retweets for protected accounts since the API call
always 403s.
- Flickr:
- Better username detection. Flickr’s API is very inconsistent about
username vs real name vs path alias. This specifically detects
when a user name is probably actually a real name because it has a
space.
- Uploading: detect and handle App Engine’s 10MB HTTP request limit.
- Bug fix in create: handle unicode characters in photo/video
description, hashtags, and comment text.
- Atom:
- Bug fix: escape &s in attachments’ text (e.g. quote tweets).
- Bug fix: handle multiply valued ‘object’ fields in ActivityStreams
1 activities.
- GitHub:
- Switch creating comments and reactions from GraphQL to REST API
(`bridgy#824 `__.
.. _section-22:
1.4.1 - 2016-06-27
~~~~~~~~~~~~~~~~~~
- Bump oauth-dropins requirement to 1.4.
.. _section-23:
1.4.0 - 2016-06-27
~~~~~~~~~~~~~~~~~~
- REST API:
- Cache silo requests for 5m by default, 60m for Instagram because
they aggressively blocking scraping. You can skip the cache with
the new cache=false query param.
- Facebook:
- Upgrade from API v2.2 to v2.6.
https://developers.facebook.com/docs/apps/changelog
- Add reaction support.
- De-dupe event RSVPs by user.
- Twitter:
- Switch create() to use brevity for counting characters.
https://github.com/kylewm/brevity
- Fix bug in create() that occasionally incorrectly escaped ., +,
and - characters.
- Fix text rendering bug when there are multipl photos/videos.
- When replying to yourself, don’t add a self @-mention.
- Instagram:
- Fix bugs in scraping.
- Upgrade to requests 2.10.0 and requests-toolbelt 0.60, which support
App Engine.
.. _section-24:
1.3.1 - 2016-04-07
~~~~~~~~~~~~~~~~~~
- Update `oauth-dropins `__
dependency to >=1.3.
.. _section-25:
1.3.0 - 2016-04-06
~~~~~~~~~~~~~~~~~~
- Support posting videos! Currently in Facebook, Flickr, and Twitter.
- Instagram:
- Add support for scraping, since they’re `locking down their API
and requiring manual
approval `__.
- Linkify @-mentions in photo captions.
- Facebook:
- Fetch `Open Graph
stories `__
aka ``news.publish`` actions.
- Many bug fixes for photo posts: better privacy detection, fix bug
that attached comments to wrong posts.
- Twitter:
- Handle all photos/videos attached to a tweet, not just the first.
- Stop fetching replies to @-mentions.
- Atom:
- Render attachments.
- Add ``xml:base``.
- microformats2:
- Load and convert h-card.
- Implement full post type discovery algorithm, using mf2util.
https://indiewebcamp.com/post-type-discovery
- Drop support for h-as-\* classes, both incoming and outgoing.
They’re deprecated in favor of post type discovery.
- Drop old deprecated ``u-like`` and ``u-repost`` properties.
- Misc bug fixes.
- Set up Coveralls.
.. _section-26:
1.2.0 - 2016-01-11
~~~~~~~~~~~~~~~~~~
- Improve original post discovery algorithm. (`bridgy
#51 `__)
- Flickr tweaks. (`bridgy
#466 `__)
- Add mf2, activitystreams, atom, and search to interactive UI.
(`#31 `__,
`#29 `__)
- Improved post type discovery (using mf2util).
- Extract user web site links from all fields in profile
(e.g. description/bio).
- Add fabricated fragments to comment/like permalinks
(e.g. #liked-by-user123) so that object urls are always unique
(multiple silos).
- Improve formatting/whitespace support in create/preview (multiple
silos).
- Google+:
- Add search.
- Facebook:
- Fetch more things in get_activities: photos, events, RSVPs.
- Support person tags in create/preview.
- Prevent facebook from automatically consolidating photo posts by
uploading photos to “Timeline Photos” album.
- Include title in create/preview.
- Improve object id parsing/resolving.
- Improve tag handling.
- Bug fix for fetching nested comments.
- Misc improvements, API error/flakiness handling.
- Flickr:
- Create/preview support for photos, comments, favorites, tags,
person tags, location.
- Twitter:
- Create/preview support for location, multiple photos.
- Fetch quote tweets.
- Fetching user mentions improvements, bug fixes.
- Fix embeds.
- Misc AS conversion improvements.
- microformats2:
- Improve like and repost rendering.
- Misc bug fixes.
- Set up CircleCI.
.. _section-27:
1.1.0 - 2015-09-06
~~~~~~~~~~~~~~~~~~
- Add Flickr.
- Facebook:
- Fetch multiple id formats, e.g. with and without USERID\_ prefix.
- Support threaded comments.
- Switch from /posts API endpoint to /feed.
- Google+:
- Support converting plus.google.com HTML to ActivityStreams.
- Instagram:
- Support location.
- Improve original post discovery algorithm.
- New logo.
.. _section-28:
1.0.1 - 2015-07-11
~~~~~~~~~~~~~~~~~~
- Bug fix for atom template rendering.
- Facebook, Instagram: support access_token parameter.
.. _section-29:
1.0 - 2015-07-10
~~~~~~~~~~~~~~~~
- Initial PyPi release.