scholarly communication
Korey Jackson
More than gatekeeping
Close-up on open access evaluation in the Humanities
had the privilege to join the SPARC-ACRL when OA proponents themselves talk about
I Forum at the 2014 ALA Annual Confer- a separate genre of scholarly content, there s
ence in Las Vegas, (viva!). This year s forum a possibility that we do that content a dis-
theme was  Evaluating the Quality of Open service, especially given OA s already hard
Access Content, and I was tasked with fought (and ongoing) battle with valuation.
exploring new modes of evaluating humani- OA scholarship has long faced a  brand
ties scholarship in a talk titled  Close-up on challenge, in part because it disrupts the
Open Access Evaluation in the Humanities. traditional (largely journal-driven) scholarly
What follows is a slightly expanded version publishing marketplace. It s become such a
of that talk. well-rehearsed part of the OA drama that
A quick rumination (and perhaps it hardly needs rehashing. But here s the
provocation) to get us started: Since I m thumbnail version: ideally, OA scholarship
a humanist, I want to take the humanist s is revolutionary because its business model
license and do a little close reading of this rejects the notion of  business as short-
year s theme title. Really, I just want to hand for crass, consumer-side profiteering.
think critically about the last phrase in the In doing so, however, it also rejects some
title: Open Access (OA) Content. It seems of the market risks that bestow the kind
pretty natural on the surface to talk about of prestige that can only be bestowed by
content distributed openly in this way. The a capitalist value system. Rejecting values
trouble is that when we frame openly dis- that lead to overpriced products is a good
tributed content as something in need of thing (especially if you re a consumer of
specialized evaluation we end up talking those products). But there s a baby in that
about something other than a distribution bathwater: when things cost money, we
or business model. We end up talking about tend to ascribe more than monetary value to
a wholly separate scholarly genre. In effect, those things because capital is cultural, too.
we affirm the peculiarity of OA content by So, OA scholarship has had it pretty tough,
insisting on referring to that content in terms wanting to be valued in the same way its
of its distribution scheme. Researchers and
other consumers of academic content don t
Korey Jackson is Gray Family Chair for Innovative
usually speak of  paywall or  toll access
Library Services at Oregon State University
content; we don t talk about books, manu- Libraries, email: [e-mail]
scripts, or journals as  paid for. They re
Contact series editors Zach Coble, digital scholarship
simply books, manuscripts, and journals.
specialist at New York University, and Adrian Ho, director
And, yes, maybe it would do us good to
of digital scholarship at the University of Kentucky
Libraries, at [e-mail] with article ideas
think more overtly about the costs of schol-
arship to individuals and to institutions. But � 2014 Korey Jackson
C&RL News November 2014 542
paywall counterparts are, but needing at the
same time to disavow paywall value metrics. But there s a baby in that bathwater:
And that challenge is evident in the when things cost money, we tend to
amount of time we ve spent advocating ascribe more than monetary value to
for OA s parity with paywall content. Over those things because capital is cul-
ten years ago, Peter Suber faced down the tural, too. So, OA scholarship has had it
issue of OA journal quality.1  The rigor of pretty tough, wanting to be valued in
peer review, he writes,  is independent of the same way its paywall counterparts
the price, medium, and funding model of are, but needing at the same time to
a journal. OA may threaten the profits and disavow paywall value metrics.
market position of some publishers, but it
does not threaten the quality of published
science. While he s speaking to the smaller of Impactstory)2 refer to any kind of mea-
circle of STEM publishers, his remarks apply surement of impact outside of traditional
beyond any strict disciplinary boundaries, metric types like citation count, h-index,
and even beyond the specific context of ar- etc. These measurements can take shape
ticle processing charges (APCs) addressed in as blog post mentions, Twitter citations,
Suber s post. The practice that contributes to use of an article within citation managers
a journal s selectiveness and prestige peer like Mendeley and Zotero, user downloads
review need not change simply because a on data-sharing platforms like figshare for
journal chooses to flip the script and freely the most part, any online arena that doesn t
distribute already-paid-for content. have representation within traditional metric
Of course saying as much doesn t make rubrics can potentially be tracked through
a thing true. Not only that, but for content various altmetric engines (provided that
that exists outside of the framework of the these arenas offer the right kinds of APIs to
journal article, peer review is simply not the allow aggregation of this data).
central (or even a very important) gatekeep- Greg Tananbaum offers a concise ac-
ing function. All of which leads me to think count of altmetrics and the more specific
that it might just be time for OA to ditch the subcategory of  article-level metrics in his
banner of paywall parity altogether and opt  Article-Level Metrics: A SPARC Primer, 3
for a new standard, one that says something where he writes that these alternative schol-
along the lines of:  content is content re- arly barometers  open the door to measures
gardless of how it gets into our hands; peer of both the immediacy and the socialization
review is one way to vet content, but peer of an article.
review is largely a game of branding& not Some examples of altmetric applications
quite as gimmicky as, say,  As Seen on TV, and services include Priem s own Impact-
but with similar fluctuations in reliability story, the aptly named Altmetric,4 and Plum
and scope. Analytics.5 Each of these offers a slight
It may be the only choice for vetting variation on the theme, but all are focused
when content remains closed behind pay- on Tananbaum s idea of the  socialization
walls. But it s not the only choice when of scholarship, tracking timely usage, men-
content can be freely disseminated. Okay, tions, and other signals of general circula-
that particular slogan might have a hard time tion among an engaged, online audience.
fitting on a banner, but you get the idea. It is true, however, that the humanities
This freeing up of choice for how we have been slow to adopt altmetrics at the
want to vet content is where altmetrics come level of even the individual scholar, let alone
in. Altmetrics (a term coined by Jason Priem, as a larger component of, say, promotion
a PhD candidate at University of North Car- and tenure review. But this sluggishness has
olina-Chapel Hill s iSchool and cofounder less to do with any particular resistance to
November 2014 543 C&RL News
the  alt in altmetrics and more to do with but it became a platform allowing authors
a generalized suspicion of counting. to comment on each other s work prior to
As Jason Baird Jackson remarks,  In publication a strikingly novel practice in
many humanities fields, those scholars the often underappreciated world of edited
have intuitions and beliefs about the most collections.
important journals, but specific measure- " Dougherty has recently completed a
ments of impact are simply not the coin similar project, Web Writing: Why and How
of the humanities realm. More succinctly: for Liberal Arts Teaching and Learning.9
 They don t know which to be more ner- Much like Writing History, Web Writing has
vous about, altmetrics or all metrics.  Any been released as an OA, openly reviewable
kind of metric entails the risk of promoting manuscript. With authors and community
short-sightedness, Jackson says.  I think the readers all working collaboratively on re-
humanists are particularly sensitive to this. 6 view, the book is again a standout example
This kind of skepticism has kept many of how edited collections can benefit from
from delving into the stats-laden world embracing openness not only as a business
of metrics, but it hasn t kept pioneering model, but as an essential component of
scholars from exploring and refining the the craft of knowledge creation. Dougherty
landscape of new-model review and evalu- offers an insightful behind-the-scenes for
ation. In fact, there are quite a few exciting those curious about the book s evolution in
developments in OA humanities evalua- a final section: Editorial Process and Intel-
tion developments that mirror those in lectual Property Policy.10
the STEM fields, but also help to point out " DHCommons journal is an Alliance of
what some of this evaluation is also about: Digital Humanities Organizations project de-
finding new readers and creating deeper signed to provide ongoing peer review ser-
(read: not superficial or crass) markets. vices to digital humanities scholarship. The
Examples include: journal s ambitions, according to its editorial
" Kathleen Fitzpatrick s Planned Ob- statement, are  to bridge the  evaluation
solescence, a seminal work on the history, gap between the Digital Humanities and
present, and possible future of scholarly more traditional disciplinary scholarship.
communication in higher education. Before As the editors explain,  Digital projects often
publication by NYU Press, the book was continue for many years as a continuum of
available as a CommentPress manuscript,7 work. Rather than building to a single pub-
meaning that it could be commented on by lication moment as monographs do, digital
anyone in the community who was willing projects often mark progress through a series
to toss in a hat and provide commentary. of significant milestones. DHCommons will
The book received a great deal of atten- provide a concrete way to certify the value
tion when it was released, in no small part of long-standing, influential, but unfinished
because of its status as an artifact of com- projects to colleagues unfamiliar with the
munity editing. contours of digital scholarship. 11
" Jack Dougherty and Kirsten Naw- " Open Library of Humanities12 very de-
rotzki s Writing History in the Digital liberately borrows from the Public Library of
Age,8 which used the same CommentPress Science (PLOS) and their flagship publication
platform and invited commentary from a PLOS ONE, seeking to introduce this model
hybrid of community readers and select of editorial-gatekeeping-plus-community-
editors from the University of Michigan review to humanities scholarship. Rather
Press, where the book was under contract. than charging authors APC, the Open Library
In this case, the edited collection provided of Humanities is looking to sustain nonprofit
contributing authors with feedback about business operations through what it calls
their specific submission. Not only this,  Library Partnership Subsidies. 13 In essence,
C&RL News November 2014 544
it asks the library community to support con-
tent creation at the beginning of its lifecycle, . . . any online production, especially
rather than at the end, and to pay quite a bit those that engage OA distribution
less to do so. It s a noble and provocative models, counts; and that information
model that, if successful, has the potential delivery and access are decidedly
to forge stronger ties between libraries, where libraries have the most impor-
librarians, and the researchers who depend tant role to play. . .
on library resources to produce scholarship.
All of these venues are concerned with
providing effective evaluation for content,
but they re equally about community build- /sparc-article-level-metrics-primer, accessed
ing and content amplification. And they September 28, 2014.
signal a possible opportunity for libraries to 4. Altmetric, http://www.altmetric.com/,
begin encouraging and helping to develop accessed September 28, 2014.
better outlets for humanistic OA publishing 5. Plum Analytics, www.plumanalytics.
and review. There s been a lot of discussion com/, accessed September 28, 2014. See
lately about the library s role in the growing also www.plumanalytics.com/metrics.html
field of digital humanities, and about what for a comprehensive overview of different
constitutes the  digital in digital humani- metric types.
ties. My answer is broad: that any online 6. J. Howard,  Rise of  Altmetrics Revives
production, especially those that engage OA Questions About How to Measure Impact of
distribution models, counts; and that infor- Research, Chronicle of Higher Education,
mation delivery and access are decidedly June 3, 2013, http://chronicle.com/article
where libraries have the most important role /Rise-of-Altmetrics-Revives/139557/ ac-
to play, whether that role is about educating cessed September 28, 2014.
scholars about options for publication, about 7. K. Fitzpatrick,  Planned Obsolescence:
things scholars will want and need to know Publishing, Technology, and the Future of
before embarking on such publication, or the Academy, http://mcpress.media-com-
about the many styles of metrics and altmet- mons.org/plannedobsolescence/, accessed
rics that can be marshaled to help showcase September 28, 2014.
the quality and impact of their endeavors. 8. J. Dougherty and K. Nawrotzki, eds.,
In the end, better and more varied evalu- Writing History in the Digital Age, open ac-
ation is not merely a function of gatekeep- cess version: http://writinghistory.trincoll.
ing, but a step toward freeing content from edu/, accessed September 28, 2014.
profiteerism and allowing it to freely enter 9. J. Dougherty, ed., Web Writing: Why
the terrain of real knowledge sharing. and How for Liberal Arts Teaching and
Learning, http://webwriting.trincoll.edu/,
Notes accessed September 28, 2014.
1. P. Suber,  Objection-reply: Whether 10. Ibid, http://webwriting.trincoll.edu
the upfront payment model corrupts peer /how-this-book-evolved/process/.
review at open-access journals, SPARC 11. DHCommons, http://dhcommons.org
Open Access Newsletter, Issue #71, http:// /journal, accessed September 28, 2014.
legacy.earlham.edu/~peters/fos/newslet- 12. Open Library of Humanities, https://
ter/03-02-04.htm#objreply, accessed Sep- www.openlibhums.org/, accessed Septem-
tember 28, 2014. ber 28, 2014.
2. Impactstory: about, https://impactsto- 13. Library Partnership Subsidies, https://
ry.org/about, accessed September 28, 2014. www.openlibhums.org/2014/04/07/library-
3. G. Tananbaum,  Article-Level Metrics: A partnership-subsidies-lps/, accessed Sep-
SPARC Primer, www.sparc.arl.org/resource tember 28, 2014.
November 2014 545 C&RL News