Necessary Utility for Successful Crowd Sourcing

Overview

Crowd sourcing has been used in digital humanities in a variety of different ways – from collecting history through initiatives like the Veterans History Project to work like the transcription and correction of text sets when computational analysis is actually less functional than the work of the human brain.  Examples of this type of work include projects like Transcribe Bentham and Papers of the War Department or NYPL’s Building Inspector and What’s on the Menu?

While crowd sourcing can be an effective tool for digital historians, it is important to think carefully about how a crowd sourcing project should be designed in order to maximize its utility.  Factors to consider include the relative complexity of the task, the relative ease of use of the interface for contributors as well as ways to attract and maintain connections with contributors.

The Tasks at Hand

In designing a project, digital historians must consider the complexity of the task.   Activities such as correcting  texts or maps that have been digitized using  OCR or georectification technologies fall on the more accessible tasks on the crowd sourcing spectrum because they generally give the contributors a basis from which to work.  Projects such as Building Inspector and Trove fall into this category.

However, scanned images of texts can vary in difficulty.  Both What’s on the Menu? and Transcribe Bentham rely on images of scanned texts; however, the skill needed for transcription varies greatly.  The NYPL Menu project requires contributors to transcribe menus that appear in print form while the Bentham project utilizes handwritten manuscripts from the 18th and 19th centuries.  The complexity of the subject matter and archaic spellings in the Bentham project also contribute to the level of difficulty.  This explains a great deal about the relative completion rates for each project.   The Menu project achieved its goal of having 9,000 menus transcribed in the first three months of its 2011 launch and at the moment the site is out of menus for transcription.  The Bentham project began a year earlier and has yet to complete half of the project’s scope.  While the Bentham project is clearly more complex in scope and substance, the disparity between the two projects speaks to the difficulty in transcribing handwriting over typescript.

However, crowd sourcing can be applied for more than just processing information but gathering it as well.  Projects like StoryCorps and the Veterans History Project  require a high level of engagement and preparation on the part of the interviewer to be successful.  The Veteran’s History Project recommends that contributors be in at least tenth grade, though teachers have been successful in scaffolding the experience for younger students.

The Interface

Digital humanities project designers must be attentive to the the ways in which contributors interact with the digital material as well.  While Transcribe Bentham is a complex project requiring a good bit of background knowledge both about the texts themselves and the etiquette of transcription, the website containing the instructions isn’t visually appealing and overwhelms the first time contributor with information.

Transcribe Bentham editing window

The platform for transcription is also a bit unwieldy as zooming in and out of the text or across it seemed more difficult than it needed to be.  The side-by-side comparison was also less than ideal in navigating the document concurrently with transcription.  Utilizing an over-under format adopted in the second version of the Papers of the War Department (PWD) interface would improve things.

Two of the easiest interfaces are those designed by the NYPL Labs – Building Inspector and What’s on the Menu?  While both are simpler tasks than the transcription required in Bentham and PWD, the layout for each is straightforward and uncluttered.  In fact the Building Inspector interface is playful and almost game-like, encouraging contributors to “Kill Time. Make History.”  In keeping with the idea of killing time, I tried the site on my iPhone, and it worked just fine, creating a potential competitor to other waiting games I am tempted to play.

Attracting and Maintaining the Contributor Community

Certainly creating a workable user interface is a large part of maintaining a contributing community, and the ability to gather feedback from users is one of the many reasons its a good idea to register users.  Early on project designers discovered that most of the work on the Bentham Project was being done by a handful of contributors.

Leaderboard in Transcribe Bentham community

However, in this case project designers realized that having a fewer number of devoted contributors may be more advantageous than monitoring hundreds or even thousands of novice or sporadic “helpers.”

Contributor page in Transcribe Bentham community

 

They capitalized on that knowledge by creating a Leaderboard and Facebook style pages for contributors, giving them status for their contributions.  This encourages a sense of community within the project, capitalizing on the fact that it takes a pretty special person to devote time to transcribing Bentham’s works.

Building Inspector keeps track of the user’s “score” during each session and has a link to broadcast those contributions via the user’s personal Twitter account, helping to reward contributors for their work.

However, the most important aspect of crowd sourced projects involve actually attracting individuals to a project in the first place.  Building Inspector again capitalized on its playful vibe by coordinating its launch via Twitter with an article in Wired highlighting a “new game” instead of digital history project.  Social media outlets such as Facebook and Instagram are other ways to attract people to the work of crowd sourced digital history.

In other cases the ways in which the projects themselves open up new sources of scholarship can attract more contributors as well.  Smart crowd source project designers can think about ways engaging with content first hand might connect to projects in schools through educational outreach.  Transcribe Bentham Inside and Outside the Classroom seeks to do this by placing Bentham into the curricular framework and encouraging class visits. And, the ability for students to contribute to the Veteran’s History Project helps teacher Jamie Sawatzky to develop a growing number of young historians through Rocky Run Middle School’s iWitness to History Day each year.

Maximizing Utility

Successful crowd sourcing in digital public history projects can be done but not without regard to structures inside and strategies outside the scope of the project that will attract and maintain users interest in the project.  Making sure that the task is appropriate for the target audience and the project’s goals are also key factors in ensuring that the project doesn’t become lost in the crowd.

Leave a Reply

Your email address will not be published. Required fields are marked *