Peer Review Working Group Recap

Peer review is an integral component of traditional academic publishing, but in the world of open textbooks, there is currently no common standard for reviewing in open textbooks. To address this need, Rebus Community has formed a working group to explore peer review for open textbooks. The group held its first meeting May 3 to discuss how the community of open textbook practitioners could build a coherent strategy and best practices around peer review for open textbooks.

Below is a recap of the meeting. You can read next steps and how to get involved here.

Deb Quentel from CALI started the session by describing CALI’s editorial review process for the lessons and textbooks they produce. Deb said CALI has an editorial review board consisting of potential reviewers in specific subject areas that she reaches out to first when looking for reviewers. If no one has the relevant expertise, she asks board members to leverage their networks to find appropriate reviewers. If that fails, the author may suggest potential peers as reviewers. Finally, Deb scouts reviewers online from relevant websites and publications.

In CALI’s peer review process, reviewers are asked to look at subject matter accuracy and whether the content covers the terrain to be expected. Reviewers also evaluate the sophistication of the end-of-chapter questions that are included for students. Reviewers provide a no-more-than-two-page memo of feedback. At CALI, peer reviewers are anonymous and usually review one chapter of a text. About 75% of the chapters for any given book get reviewed. Deb outlines her process in further detail here.

Next, Hugh McGuire of Rebus Community outlined various types of pre- and post-publication review that he hypothesized may need to be done to an open textbook:

• Subject matter expert review (a formal, pre-publication review, similar to traditional peer review)
• Open review, in which anyone can make comments (less formal)
• Student and faculty feedback on a published, beta version of the textbook
• Ongoing feedback (how does feedback get back to the authors following publication?)
• Revision and updates (especially for subjects that change frequently)

Karen Lauritsen from the Open Textbook Network gave an overview of OTN’s review process. All new OTN member organizations are able to send faculty to workshops as part of their professional development in open textbooks. In return, they are required to provide a light review for one book in the Open Textbook Library and are given a rubric from which to do so. Karen said that roughly 40% of these reviewers end up adopting a textbook from the library for classroom use.

The discussion then moved on to standardizing types of peer review processes for open textbooks, perhaps by using markers or badges to transparently delineate which processes were employed on a given text. Billy Meinke from the University of Hawaii shared the Mozilla Science Contributorship Badges and noted that ideally peer review badges would exist both for content and for contributors. This would serve the dual purpose of giving recognition to reviewers along with credit they can use in their tenure and promotion process.

Billy said that Mozilla Science Badges were linked to ORCID ids. Hugh said Rebus will be looking for ways to do the same. Karen added that the markers ought to be non-hierarchical, but still provide adequate recognition.

Nicholas Persa from UW-Madison acknowledged the need for one portal of access that enables and encourages community review – a model that Rebus is hoping to build.

Participants agreed on many topics, such as the need to incentivize reviewers either with payment or recognition, the need for post-publication review in classrooms and continual updating of open textbooks, and badges or markers to validate both content and reviewers. However, a number of questions remained unanswered: what specific value can we offer reviewers, particularly when monetary payment is not an option? Should pre-publication review be done before copyediting? How do we begin making these markers and defining levels of completion for peer review?

Universities such as The University of British Columbia seem to be making strides in recognizing the importance of OERs on tenure committees, and as this spreads to other institutions, there will be a growing need to develop best practices and standards for processes like reviewing.

The Rebus Working Group will continue to work on these and other issues pertaining to peer review. Next steps will be posted on the forum. Join the conversation on the Rebus Community forum thread as we try to figure it all out!

You can watch the video recap here.

Additional Resources:

• Peter Potter shared AAUP’s best practices for peer review.
• Billy Meinke noted an organization focused on giving credit to peer reviewers.
• Peter Potter shared an article on the possible future of peer review.
• Rebus’ Agenda and Notes from the WG meeting