As you move through the Digital Readiness Levels, take time to review and reflect on your work on a regular basis. Evaluation is both outward-looking — how is our work making a difference for our users and our community? and inward-looking — what have we learned? How can we improve or evolve our work?

As you move towards a new level in any of the focus areas, consider the following questions:

Is it documented?

• Are new directions and decisions represented in existing policies, such as a collection development policy or a rights and reproductions policy? Do policies need to be revised or do new policies need to be created?
• Are new procedures or processes written down, such as steps for using a scanner or standards for how to describe content? Is there enough information provided that someone new to the organization would be able to carry out these processes on their own?
• Are partnerships or arrangements with other parties, such as a digitization vendor or a content contributor, documented in some way, like a contract, Memorandum of Understanding, or letter of commitment?

Is it sustainable?

• Is there a plan for how this work will continue to be supported by staff and/or volunteers from year to year?
• What work needs to be done to maintain relationships with partners or collaborators?
• Is funding committed for any recurring costs, such as a cloud storage service?
• Are best practices and widely-adopted standards being used, so that digital work is “future proof”–that is, it’s compatible with commonly-used platforms and systems?
• Is there an exit strategy for any tools or partnerships? For instance, can content be removed from a platform if necessary?

Is it appropriate, relevant and/or accessible?

• Are new digital initiatives in line with the organization’s mission and values? Do digital initiatives reflect community needs and respect community priorities?
• If information is available publicly, is it violating any privacy, copyright or ethical considerations?
• Can all potential users access the content? What can be done to improve accessibility for people with disabilities?

Is it working?

• Are we doing what we set out to do? What were our goals and have we achieved them, partially or fully?
• What have we learned and what will we do differently next time?
• What lessons learned would be helpful to other organizations?
• Have we communicated our project decisions and progress to stakeholders and partners, including any funders?

EVALUATE: BRONZE LEVEL

Identifying Audiences

Who are the people your digitization projects will reach? “Everyone!” is a good umbrella goal, but taking steps to define specific user groups and their needs will help you connect with them more effectively. Start by brainstorming a list of key audiences, or undertake a community mapping exercise to identify potential user groups^[1]. Think beyond your go-to groups (genealogists, teachers) to other community members you might not already reach (new arrivals, young families).

For each user group, consider:

• Where are these users? Do they gather regularly, virtually or in person? Where can you find them?
• What is this group’s current relationship with your organization or project?
• How does your work benefit this group? Why should they care?
• What do you want community members to do with your organization or project? What is the “call to action”?
• What challenges or barriers to engagement, such as language or Internet access, do they face and how will you address those?
• What else do you need to learn about this group to facilitate new, more or better engagement?
• Consider that your role in the community may evolve over time. What is your plan for reassessing community needs periodically?

User Personas

To gain an even deeper understanding of your audiences and the best ways to connect with them, you might consider outlining some user personas. User personas are fictional but realistic representations of core audience segments. They’re a good way to keep community members’ perspectives and needs at the heart of your project development and outreach efforts.

Building Community Connections

Developing and maintaining relationships with community members gives you insight into how your organization’s digital work can best support and serve your community.

Remember the following “rules of engagement” when interacting with community members^[2]:

• Introduce yourself authentically (not just your title, organization, or degree/credentials)
• Always ask permission to engage with an individual or community group
• Reassure folks that you will protect and honor their legacy
• Share copies of any project outcomes and project updates with community members
• Follow up continually – these are never “one and done” interactions

Some community-centered questions to ask as you plan digital projects:

• How do you measure the needs of your community?
• How will digital work serve the needs of your community and organization?
• Does your community have access to computers or other means to use and experience digital content?
• What resources exist that can be shared? (Digitization equipment, knowledge and expertise, workspace, digital or physical storage space)

EVALUATE: SILVER LEVEL

Understanding Use

Do you know how users are searching for, finding, and using your digital content? Gathering data and stories about collection use can help you make the case for allocating funding to digital projects and can inform your decisions about what to digitize next. A solid plan for documenting and evaluating collection use will also strengthen your grant proposals and fundraising efforts.

One way to approach evaluation is simply by paying attention to what your users say and do. What are they searching for in your digital collections? Are they finding what they were looking for? What discoveries have they made? With these kinds of interactions, you might want to follow up to ask for more information, either informally or using some kind of survey tool.

Methods for gathering information about use:

• Evaluation surveys
• Log of patron contacts (for example, keeping track of the number of research inquiries or reproduction requests you receive)
• Attendance numbers at project launch events or other related programming
• Focus groups or individual interviews
• Social listening (for example, observing the comments on photos posted to Facebook or Instagram for recurring themes)
• Web analytics

Web Analytics

Google and other search engines, and Facebook and other social media platforms, all track lots and lots of information referred to as web analytics. Relying only on a single metric, like pageviews, won’t give you a full picture of use and engagement, but considering web analytics in combination with user feedback, research requests, and other data can help you understand and demonstrate the impact of your digital work.

Web analytics might help you answer questions such as:

• What do people search for that brings them to your digital collection? (keywords)
• Where were they before they came to your website? (referrals)
• How long do they stay on your website? (bounce rate)
• Which items in your collection get the most attention? (pageviews)

EVALUATE: GOLD LEVEL

Wrapping Up a Digital Project

As you approach the end of a project, think ahead to how you will wrap it up. Tying up loose ends, gathering documentation, and taking time to reflect will help make your next project even more successful!

Steps in wrapping up a digital project:

• Connect with partners. Make sure you’re on the same page regarding any ongoing commitments. Consider establishing an MoU or other written agreement if you don’t already have one in place.
• Help transition your team members. If you hired short-term staff, offer to help them with references or a resume and cover letter review, if appropriate.
• Wrap up the financial and administrative end. Close out any contracts, confirm all invoices are paid, submit required grant reports, etc.
• Preserve project documentation. Assemble any workflows and training materials, meeting minutes, reports, partnership agreements, donor agreements, and logins and passwords for hosting platforms, cloud storage, software tools, or shared workspaces such as Dropbox or Google Suite

• Evaluate and reflect. Arrange a project “debrief” with your team shortly after your project ends. Invite project staff, volunteers, leaders, or other key stakeholders to join you. Discuss the successes, challenges, and lessons learned, and invite honest feedback about their experiences.

Share What You Learned

You don’t have to be a large, well-funded organization to share your project findings and outcomes with others in your field. Small and rural libraries, archives, historical societies, and museums are always interested in hearing “how we did it” stories from organizations similar to their own.

Opportunities for sharing what you’ve learned with other practitioners:

• Present a session or a poster at a regional or state-level conference
• Write a blog post or newsletter article for your state library association, regional archives forum, or other similar organization
• Post any workflows you created to the Library Workflow Exchange
• Host a virtual or in-person brown-bag lunch
• Share on email lists or online communities for local historians, genealogists, or archivists