Badges: Revisiting the Peer-Review Process

On 17 April 2012, The Faculty Advisory Council at Harvard University issued a memorandum expressing concern over the rising costs of library subscriptions to scholarly journals, and strongly encouraging faculty and graduate students to submit their work to open access journals as a way of transferring prestige away from print and other for profit journals with high subscription costs. Indeed, concern over rising subscription costs, public access to information, and often-lengthy turn-around times for submitted materials have led to the emergence of numerous open access journals in recent years, with numbers increasing almost exponentially.

As of 2013-02-05 [15:55:30], The Directory of Open Access Journals (DOAJ) lists 8621 journals, 104 of which have been added since 1 January 2013. In order to be included in the directory, a research journal must be free and open to the public without delay, and must “exercise quality control on submitted papers through an editor, editorial board and/or a peer-review system.”[1] Since open access journals are also online journals (online content delivery significantly decreases the costs associated with traditional print publication), many journals, like Southern Spaces and Vectors, are also very interested in delivering scholarly content through a variety of alternative media. Because they are not tied to the economically motivated decision-making processes governing major print publishers, open access journals arguably have more freedom to publish and encourage alternative modes of scholarship, and, perhaps, to provide a legitimate space for otherwise marginal voices. Crucial to the legitimacy of open access journals, however, is a commitment to traditional processes of establishing authority, a ‘blind’ process whereby expert opinion is validated by expert opinion, anonymous though it may be.

So here’s the beginning of a half-baked (and somewhat utopian) idea…

[Image Creative Commons licensed / Flickr user fczuardi]

Recently, the idea of ‘badges’ has entered into discussions about assessment. Badges are a new approach to credentialing, a way of demonstrating competency, perhaps in the absence of formal training, through recognition by institutions, on the one hand, or peers, on the other. Most basically, having demonstrated some skill mastery or other characteristic (like ‘helpfulness,’ for example) an individual receives a badge (some kind of graphic) that they can display on their website and/or social network profiles that confers legitimacy to claims about skill-sets and aptitudes, by linking each back to the badge issuer. Although still in its infancy, the idea of using badges as a universal peer assessment framework has been most fully developed by the Mozilla Open Badges Project through the creation of the open source Open Badges Infrastructure. The strongest proponent of the use of badges in schools as a legitimate alternative to grades, is Cathy N. Davidson, co-founder of HASTAC (Humanities, Arts, Science, and Technology Advanced Collaboratory), an international organization dedicated to rethinking the future of learning for the information age.[2]

My half-baked idea goes something like this: Why not eliminate journals entirely? Why not revisit every part of the traditional approach to academic publishing, including the peer-review process? In place of academic journals, why not encourage authors to host content themselves and self-publish their work in a way that decentralizes knowledge production?

According to the Budapest Open Access Initiative, “There are many degrees and kinds of wider and easier access to this literature. By ‘open access’ to this literature, we mean its free availability on the public internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. The only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited.” A decentralized approach to academic publication that encourages scholars to self-publish on personal websites fulfills every condition of Open Access. The greatest drawback to this kind of approach (and a longstanding reason for distrusting self-publication in general), is that it bypasses peer-review, an 18th century system of legitimizing knowledge production. In other words, open access to scholarship doesn’t really mean much if there is no way to judge the quality and credibility of the work. This is where badges come in.

I produce a piece of scholarship and put it online. If it is found to be of merit by others in my community, or by others who discover my work organically, then they can give me a badge, a seal of approval that vouches for the quality of my work. The meaning of a badge, then, becomes a transparent reflection of the quality of the work by way of the credibility of the issuer. On the one hand, the ‘value’ of the badge is determined by the amount of recognized expertise the issuer has to evaluate the work. The fact that a badge is not anonymous means that different badges carry different weights, and in a way that is often obscured by the peer-review process (not all peer reviewers have the same level of expertise, which is why there is almost no article that is so bad that one can’t find a journal to publish it). On the other hand, the issuer of a badge will be forever tied to the life of the work, and in such a way as to put their credibility on the line alongside the original author. Consequently, issuers are discouraged from ‘badging’ willy nilly, as if they were ‘liking’ posts on Facebook. According to the scheme, assessments of the merit of a work would involve a consideration of the author and the work itself, but also the expertise of badge issuers. This additional information would also be very helpful hermeneutically, as a way of locating a work with respect to the communities it touches and the range of meanings it might have.

Okay, so maybe this idea isn’t even really half-baked, but it is an idea. Realistically, even if this or something like it were good in theory, in practice replacing the old peer-review model would require that nearly everyone jumped on board at once, lest the badging mavericks find themselves cut off and set adrift from the larger, legitimate, and legitimizing community of scholars. Implementing this kind of framework would most certainly be fraught with logistical and technical challenges as well. The point of this thought experiment, however, is not so much about proposing an alternative reality, but rather bringing to light the fact that, in challenging the hegemonic authority of major publishers as the only credible mechanism for delivering legitimate scholarship, open access approaches to publication do not at the same time challenge the notion of centralized authority per se. The world wide web offers us more than just an opportunity to share knowledge openly, but also the opportunity to challenge more basic assumptions about how knowledge is (and should be) produced. Upon re-evaluation, It may in fact be that journals continue to be the best way of delivering knowledge and ratifying expert opinion. The opportunity for decentralization afforded by the internet, however, allows us to call our standard ways of doing things into question, and in so doing, to transform passive assumptions about the way things are into active decisions about the ways things should be.