Library Of Congress Abandons Twitter Archive

The Library of Congress has abandoned a project to archive every post made on Twitter. It blames the sheer number of "tweets" now being made, along with an increasing reliance on images and videos.

The library is the world's largest and collects all manner of documents from newspapers to government forms. As a general rule it, doesn't aim to collect all possible data from a source but made an exception for Twitter in 2010.

At that point it reached an agreement with Twitter to get a copy of its complete archive of posts, then dating back to 2006. The idea was to "document the emergence of online social media for future generations." As well as maintaining a complete collection by continuing to add new posts to the archive, the library also aimed to find a better way to organize the archive, particularly to make it easier to see all the content from a specific day.

A paper from the library explaining the change of heart says it continuously reviews the documents and data it archives. It says its decided to stop archiving the data at the end of the year, giving it a complete record of the first 12 years of the site.

Text-Only Posts No Longer Enough

According to the library, this decision comes for three reasons.

The first is that the number of posts made each day has increased dramatically. (Source: npr.org) Secondly, the agreement with Twitter only covered the text of posts. The library says this is making the archive less relevant as so much of the context of Twitter now comes from images, videos and links to external websites.

Finally, it cited the recent increase of the maximum number of characters in a single post from 140 to 280. The library didn't say if this was simply because it made an appropriate end point to the project, or if it had caused technical problems with its archiving set-up.

Deleted Tweets Could Re-Emerge

The library has not yet made the archive of posts publicly available. It says it needs to figure out the best way to do this in a cost-efficient manner which makes it useful to searchers - plus, it must comply with the original Twitter agreement. In particular, it needs to decide if and how it should include tweets that the user later deleted.

Some future tweets will be archived but only on a "very selective basis." These will be "thematic and event-based, including events such as elections, or themes of ongoing national interest, e.g. public policy." (Source: loc.gov)

What's Your Opinion?

Is the Library of Congress right to abandon the project now? Do partial or complete archives of Twitter provide a genuinely useful social history of the US? Should the archive include deleted tweets when it goes public?