For Open Textbooks to be widely adopted in academia, issues pertaining to peer review, curation, and testing must be addressed. The curation and testing of Open Textbooks is essential to Rebus’ goal of producing scalable models for OER publishing. Rebus has partnered with the Open Textbook Network to discuss pertinent problems, especially those around content management, in terms of peer review and accessibility. While Rebus’ work with the Forum is concentrated in one space and the organization has engaged deeply with it thus far, the work on Peer Review, Curation, and Testing is currently in its nascent stages. As Rebus is still in the early stages of discovering the nuances of these processes, the following section provides an outline of Rebus’ current workflow and the direction it is taking, rather than a systematized critique of its limitations and the directions for improvement.

3.1 Peer Review Working Group: Revising the Review Process

A key challenge in the adoption of Open Textbooks in universities is that faculty are often skeptical of the “quality, comprehensiveness, clarity and currency of open textbooks” as these are perceived to be low-cost, low-quality alternatives to high-cost, high-quality printed textbooks (CARL 2014). The absence of appropriate marketing of the peer review process in the production of Open Textbooks legitimizes these attitudes.

Peer review is valuable for indicating the quality of educational content, and is integral to the production of textbooks, scholarly monographs, and journals. Its presence signals to the prospective user of a text that it has passed through rigorous quality control, and that its content is suitable for discussion and use in the classroom. In the age of content abundance, when the barriers to content production are minimal, peer review performs an important role in signaling the trustworthiness of a particular book. Indeed, many instructors reject Open Textbooks “because they question the quality of freely available resources, especially when such sources aren’t connected to a reputable publishing house, institution, or author” (Pitt 2015). While traditional publishing houses are assumed to have vetted the content and quality of textbooks through open or closed peer review, the same does not hold true for Open Textbook creators. Thus, marking the scholarly legitimacy of Open Textbooks is, for Rebus, an essential step in the process of distributing such texts.

Rebus is trying to develop a set of standardized approaches for peer reviewing Open Textbooks, and is using various models on existing projects. Peer review has taken place on four texts thus far: Financial Strategy for Public Managers, Literature Review for Education and Nursing Students, The Science of Human Nutrition, and Media Innovation and Entrepreneurship. With Financial Strategy, both chapter and full-text reviews were conducted. Authors could recommend potential reviewers, but remained unaware of the specific chapters each individual reviewed (similar to double non-blind review^[1]). Reviewers for Literature Review were solicited on the forum and by cold-calling subject librarians. The author was unaware of reviewers’ identities but the reviewers knew the author (single-blind review^[2]). Each chapter was reviewed by a single subject-matter expert, with one exception that was reviewed by two people. With both Human Nutrition and Media Innovation, project leads contacted potential reviewers through their own networks, and filtered this feedback before passing it along to authors.

As is evident from the above description, the process of reviewing Open Textbooks is varied and inconsistent. This inconsistency is not specific to Rebus or Open Textbooks, but typifies peer review in the textbook industry, and in academia more broadly (McLaughlin 2016; Tennant et. al 2017). Organizations conduct peer review in several different ways, and no standard exists in terms of the academic rank and number of reviewers involved, or with regard to the level of blindness^[3]. These range from “commission[ing] reviews from a mixture of lecturers and student reviewers” to “expansive peer review” practices with a dozen or more reviewers, to post-publication “peer validation” by tenured or tenure-track faculty (Watson 2016, Fenton 2016, Talbert 2012). Rebus is discovering that the process varies according to book in question: it is currently documenting notable kinds of peer review to inform collaborators, Open Textbook creators, and the general public of the relevant best practices.

Rebus has set up a peer review working group, comprised of five sub-groups, each dealing with particular aspects of the peer review process. The first sub-group is the Open Textbook Review Board, which helps find reviewers for Open Textbooks. The second sub-group works to define the purpose and structure of the different kinds of review that could be valuable to an Open Textbook. They also aim to guide the development of dedicated tools and resources for each kind of review. Sub-group three means to conduct an environmental scan of peer review tools, processes, and to inform the development of new review processes. In relation to this, the fourth sub-group discusses the creation of standard markers for Open Textbook review to clearly identify the kinds of review any given textbook has undergone. Finally, the fifth sub-group examines ideas for recognizing and rewarding reviewer contributions.

The Peer Review Working Group is currently developing these processes for Open Textbooks. In documenting and educating the public about several key forms of peer review, Rebus is developing new practices and norms of communication. These new norms attempt to convey the quality and legitimacy of Rebus’ Open Textbooks. For example, the review markers produced will, ideally, be placed on book covers. Their presence will guarantee the quality of the peer-reviewed Open Textbook to end-users, making it more marketable to students and faculty at universities. These efforts are necessary for making Open Textbooks, both Rebus’ and otherwise, viable as educational resources.

3.2 Accessibility Working Group: Promoting Access

For Rebus, accessibility entails “making it easy for people with disabilities (say people with visual impairments, people with learning disabilities, among others) to access content” (Mays 2017). In the publishing industry, accessibility often becomes an afterthought, incorporated into books only after publication. This approach limits a book’s reach, and many individuals cannot use it on initial release. Instead, institutions must remediate^[4] these books to make them accessible to all students. This process is labour intensive, requiring significant time and resources. At the University of Washington, a staff member revealed that student or temporary workers spent 1800 hours remediating textbooks this year, totaling an expenditure of almost $27,000 (Rebus and OTN 2017). They found that non-STEM books were more quickly remediated, at a rate of 100 pages an hour, while STEM books took much longer, at a rate of 10 pages an hour (Rebus and OTN 2017). This difference is likely due to the presence of tables, images, charts, and non-plain text content in these books. Most troubling, however, is that once the book is made accessible at an institution, copyright and funding restrictions can prevent it from being shared with other institutions. So, each institution must spend time, labour, and capital to duplicate remediation for its students, even if the work has already been done elsewhere. Such practices waste resources—a problem easily avoided through openly available texts that incorporate accessibility into their production.

Given its commitment to producing books that all students may use right from publication, Rebus has created a Working Group to incorporate accessibility upstream in the authoring process, rather than downstream after the book has been produced. This would enable textbooks to be read by students with impairments or disabilities without any wait time. This process would also minimize or eliminate the cost of institutions remediating these textbooks. Rebus intends to make sure that authors making Open Textbooks through the organization are aware of good accessibility practices from the start of this process, and wishes to help them easily implement these practices. For instance, author guides will include a section on accessibility and will encourage authors to provide alternative text for images, charts, and tables. Rebus also intends to provide a standard accessibility audit process for after an Open Textbook is created and to create a mechanism to “fix” accessibility problems found in this process. Lastly, Rebus thinks it is useful to have an “accessibility stamp of approval” for projects that have successfully passed the accessibility audit, to indicate the book’s accessibility to potential adopters.

As Rebus incorporates accessibility fully into its production processes, particular issues need to be resolved. Primarily, Rebus needs to account for the fact that, as with most scholarly publishing, all of its authors and contributors are volunteers, and only some have grant funding from other institutions. In order to make its production processes truly open, Rebus needs to consider ways of making collaboration viable for individuals not already employed by research institutions, and who do not have a reasonable basic salary to supplement their volunteer work. Moreover, there is also an issue regarding the use of images and other non-textual elements in these books. Whenever non-textual elements are added they need to be remediated at later stages in the project. These remediations are often carried out by non-subject experts at individual universities, who take longer to complete the process and cannot match experts in the suitability and specificity of the information they provide—a fact Rebus has learned through its work on the Media Innovation and Entrepreneurship textbook. Resolving this issue requires improved coordination between authors and Rebus with regard to how accessibility is to be incorporated into the text.

Spreadsheets and tabular information present a similar issue. While working on Financial Strategy for Public Managers, Rebus received images of spreadsheets from authors. According to the Accessibility Working Group, however, all tabular information must be presented using tabular markup (in HTML) and displaying such information as an image constitutes bad practice. Moreover, when a table is presented as an image, it needs to be accompanied with alternative text that describes the image in detail. Given that Rebus did not make such requirements clear to authors initially, this presented substantial challenges during the later parts of the production cycle. Standardization and proper communication of accessibility requirements should thus be an important goal for Rebus’ Accessibility Working Group.

Moving forward, Rebus should make the importance of accessibility clear to authors by incorporating best practices into author guides, thus resolving issues of inaccessibility early on. These best practices would ask authors to use images only when needed, to provide alternative text for images, and to avoid displaying information through colour alone. By holding authors responsible for these core aspects of accessibility, Rebus can increase awareness of the challenges of remediating books for all users and also reduce the amount of remediation required. Rebus should also prepare guides for editors, who can suggest changes with accessibility in mind. Lastly, Rebus’ Open Textbook project leads should be aware of the time needed to make books accessible, and should budget this into the overall project timeline accordingly. Collectively, these approaches will lead to a much more integrated approach to accessibility in Open Textbooks, saving Rebus both time and money.