As evaluators, we have a variety of data collection approaches we can use that allow us to understand our partners’ work. As part of one ongoing program evaluation, we’re using case studies to dive deep into the work being done by grantee cohorts in several states. One element of our case study methodology is to conduct site visits with project teams across the country, and we recently spent two days in Cleveland, Ohio, getting to know our grantee partners better and learning about their organization and the work they’ve been doing. Virtual engagement through platforms like Zoom have allowed us to learn from and build relationships with partners who live across the country. However, in-person site visits can contribute to an evaluation in a number of ways, particularly if you can spend a longer period at your research site. We wanted to share our reflections about using site visits as part of the evaluation process, and some of the benefits we saw from this experience.

Being in Cleveland for several days provided several opportunities for different kinds of data collection. We’d arranged interviews with our key contacts at the grantee organization, as well as with several other staff and partners they are working with. We watched our partners at work in their planning meetings and a team dinner. We also had the chance to see the organization’s offices, and go on a tour of the city and the neighborhood where they are doing community organizing.

Throughout these formal data collection sessions, being face-to-face with our research partners let us observe and learn more about the greater context of our partners’ work and experiences, and to hear about their priorities. In addition to having our questions answered during interviews, we got to chat before and after about other things interviewees wanted to share with us. When we conducted interviews in our partner organization’s offices, they could point to brainstorming documents that they still had hanging on the walls, and show us T-shirts they had produced as part of one of their campaigns. For another set of interviews, we met project partners at a local community space and they shared stories about the development of that neighborhood as we got coffee and sat down to begin the interview. During the project meetings that we sat in on, we were able to observe how people moved in a shared space together, deferring to each other, making room for certain people to lead discussion, all of which offered insight into partner dynamics that would have been difficult to understand through a computer screen. And during our tour of the neighborhood, our grantee partner told us stories about the history of Cleveland, the neighborhood we were in, and what kind of health challenges exist, while pointing out local churches involved in organizing, the limited number of after school programs for kids, and noting how housing quality had been impacted by changing investment in the area over time. We were able to learn a significant amount from what people shared with us, and the increased understanding of the context by being there with people was an important contribution to our learnings.

In addition to significant data collection, we also enjoyed forming a stronger relationship with our project partners. With virtual engagement, relationship building starts when the call begins and ends when the call ends, supplemented by the occasional email back and forth. However, while visiting in person, we talked about our lives while waiting in line for our scones and coffee before one interview started, and chatted about the NBA finals while loading up our plates at the post-planning-meeting team dinner. We also got to know people we otherwise wouldn’t have had the chance to meet. During our office visit, we heard from grantee organization staff not as involved in the project we were evaluating, who shared their own perspectives on the organization and its goals for improving health care for their community. These relationships not only enrich the evaluation, but are rewarding in and of themselves.

Another unexpected benefit of conducting this multi-day site visit was the gift of dedicating our time entirely to one evaluation. At ICH, each of us are on multiple projects at once, working with a variety of partners and balancing different responsibilities and timelines. This is part of what makes our job fun! However, spending two days with a colleague, working together on a single project, allowed us to fully immerse ourselves in the work. We had time to debrief together about everything we had learned at the end of each day and identify topics we wanted to learn more about the next day. We also had plenty of time just to brainstorm about how we could take what we learned and apply it to the broader evaluation project. It was great to slow down and dig in.

Although site visits can strengthen evaluation work, they are not always the right fit for every project. They place a burden on the partners who host us, and longer visits are a significant time investment. However, in the right circumstances, a site visit can be a great strategy to probe a partner’s work and see the impact firsthand. We came out of this trip with A LOT of data. We’re excited to use the analysis that emerged from spending time with each other and immersing ourselves in days of brainstorming, focused work, and relationship building to develop impactful reporting to share with all our partners.