by Virginia A. Koch and Eric M. Chrisman
Assessment data can be full of power as well as pitfalls. Data-driven reports can enhance leaders’ ability to understand mega-trends, digest rapidly changing information, develop actionable insights, and meaningfully engage stakeholders. Assessment efforts can elevate stakeholder concerns, bolster calls to action, and fuel paradigm shifts when backed by transformational leaders. Conversely, assessment reports can also collect dust on a shelf when inadequate political will, perceived irrelevance, or other priorities deter action. So, while most campus housing departments these days are swimming in statistics, the most successful ones know that all the data in the world don’t do any good unless they are put into action.
Many campus housing leaders rely on assessment findings to guide their decision-making, advocate for stakeholder concerns, reduce mistakes based on faulty assumptions, and enhance performance by building on success and incorporating student and staff feedback. During uncertain times, as epitomized over the last 20 months, having data to turn to becomes even more valuable. As April Konvalinka, executive director of housing and residence life at the University of Central Florida, explains, “Like our housing colleagues worldwide, the pandemic launched our department and institution into crisis mode, and we found ourselves pivoting each time we received new information. Our assessment findings and occupancy data became critical navigation tools, assuring us when we were headed in the right direction and challenging us to think differently when the results differed from our expectations. Our team used our assessment data to inform decisions on reopening our housing program, addressing concerns for student and staff well-being, and evaluating the effectiveness of our modified programs and services.”
Data-guided decision-making, though, has value even during normal times. In an Occasional Paper of the National Institute for Learning Outcomes Assessment, Karen Singer-Freeman and Christine Robinson identified what they called the “grand challenges in assessment.” A grand challenge is “a problem requiring collective cooperation within a community of scholars to successfully resolve.” For them, the challenges include the need for assessment to “produce visible and actionable assessment findings that drive innovation,” inform budgetary decisions, result in immediate improvements, increase equity, measure strategic progress over time, and effectively communicate “relevant, timely, and contextualized information about the full range of experiences to stakeholders.”
How can housing professionals collect, analyze, and share data to move the needle on the grand challenges in assessment? Examples from three United States campuses recognized by Skyfactor Benchworks in their 2021 Assessment & Impact Awards highlight how data-driven insights can be used to improve the residential experience and engage a variety of stakeholders. As one example, Richard DeShields, dean of student engagement and auxiliary services at Miles Community College, notes that campus leaders were “intentional to administer assessment annually because data gets old fast as students cycle through our campus every couple of years.” Miles’ housing staff discussed the assessment findings with students in terms of sustainability, recycling, and food equity. “We followed up with student focus groups to seek input into food preferences. Our efforts resulted in more vegan and locally sourced options and increased student satisfaction,” says DeShields. “The data helped us know where we were going, and involving students in making sense of the data was invaluable.”
Another example was at Oxford College of Emory University, where housing staff have been using their Skyfactor Resident Assessment data to track their efforts to cultivate a diverse and inclusive campus environment. As Michele Hempfling, now director of community engaged learning and internships and past director of residential education and services for 11 years, explains, “We used the annual Resident Assessment to benchmark against six peer institutions and more than 250 participating campuses and showcase to stakeholders the exceptional work of our staff and campus partners. Our longitudinal data provides perspective on our rise to become a top performer on Skyfactor’s ‘Diverse Interactions’ factor for the last seven years and motivates us to strive for excellence in our diversity, equity, and inclusion efforts.”
Many campus housing leaders rely on assessment findings to guide their decision-making, advocate for stakeholder concerns, reduce mistakes based on faulty assumptions, and enhance performance by building on success and incorporating student and staff feedback.
A third example came from Scott M. Helfrich, director of university housing and conference services at Millersville University. His staff have been expanding their living-learning community program over the past three years with the goal of having one in five students living on campus involved in an LLC by 2025. “Our regular assessment enables us to share the results and student feedback with key stakeholders including current and potential LLC faculty partners, resident assistants, and influential university leaders. We leverage our results to advocate for resources such as new partnerships and funding for LLC scholarships and are able to demonstrate how we contribute to the university’s strategic plan.”
In other cases, data can be used to move the needle on issues and concerns for an entire profession. In partnership with higher education associations and public health entities, the Association of College & University Housing Officers-International leaders and staff collected, analyzed, and disseminated data as housing professionals scrambled to address a multitude of concerns throughout the COVID-19 pandemic. Thousands of housing professionals contributed as survey and straw poll participants, virtual roundtable panelists, and content experts in online community discussion boards and a variety of ACUHO-I publications. From March to August 2020, an international group of housing professionals compiled just-in-time data regarding housing operational responses and packaged the information into online reports for quick distribution. “With the rapidly evolving nature of the pandemic, providing timely and accurate data to our members was the top priority for the Association,” says Mary DeNiro, ACUHO-I CEO and executive director. This work continued into 2021 when ACUHO-I’s Re-Entry to Fall 2021 Work Group mined data to provide recommendations and resources for campus leaders to consider as their housing staff tackled new challenges in restoring standard operations in a COVID-influenced environment.
How can housing professionals avoid common assessment pitfalls including misunderstood or mistrusted data? Presenters of a 2021 ACUHO-I educational session titled “From Dread to Success: How to Handle Unexpected Assessment Results” addressed this issue head on. The presenters, Amanda Knerr and Ardell Sanders from the office of residential life at Indiana State University and Sherry Woosley from Skyfactor Benchworks, provided six concrete strategies for making sense of unexpected or unwanted results, both good and bad, and sharing those results with stakeholders. Sanders cautions that housing professionals must avoid viewing data in a bubble by creating a core team to review and chew on assessment results. As Sanders explains, “It’s important to assemble a trusted and diverse team with different dispositions, experience levels, content and contextual knowledge, and perspectives. Your core team needs to be willing to challenge the assessment results with the right questions.”
As assessment grows in utility, reach, and purpose, higher education leaders are wise to consider the political capital they will need to muster to put data to use.
Woosley suggests that, when data is confusing, “You can ask: how did we measure the item? What did we actually ask? Are we measuring what we think we are measuring?” In addition, Knerr emphasizes that “the core team can consider the data collection methods: When was the data collected? What else was going on at the time that could affect the results?” They wrapped up the presentation with three additional strategies for making sense of data. They recommend considering the context (how did COVID mitigation strategies impact student engagement?), entertaining alternative explanations (what other conditions could explain the results?), and, finally, digging deeper into the results (can the data be disaggregated?).
The challenge of sharing assessment findings is complicated by the growing propensity to mistrust data or question scientific research. For instance, throughout the pandemic housing professionals witnessed firsthand how beliefs about the coronavirus, vaccines, and mitigation strategies affected campus occupancy as well as staff concerns about in-person work in the residence halls. As the pandemic drags on, the emotions associated with these beliefs are powerful, compelling, and intensifying. How can this emotional cycle be explained? In a 2019 Hidden Brain podcast, host Shankar Vedantam interviewed University College London researcher Tali Sharot, who explained a phenomenon called confirmation bias, which she describes as “our tendency to take in any kind of data that confirms our prior convictions and to disregard data that does not conform to what we already believe.” She further explains that “even if the data is based on very good science, it has to be communicated in a way that would really tap into people's needs, their desires. If people are afraid, we should address that.” While other factors may also contribute to this mistrust, the eroding of trust in data and research is troubling, and its effects may be felt for decades to come.
As assessment grows in utility, reach, and purpose, higher education leaders are wise to consider the political capital they will need to muster to put data to use. During an August 4, 2021 Inside Higher Ed webinar on assessment culture, editor and panelist Scott Jaschik remarked that “any good assessment tool is going to uncover problems” and challenged “do you have the time and resources to address the problems?” A leading voice in higher education issues, Jaschik notes that “in our polarized political climate, leaders at American public institutions face tough decisions when assessment data exposes findings that may be unpopular with elected officials or their appointees. It is important to anticipate the costs of engaging in the political sphere. No one wants to get blindsided.” For example, should public institution administrators survey campus stakeholders about mandatory COVID-19 testing if state officials are against testing? “Before launching an assessment, leaders need to ask strategic questions that will help them focus on who to assess and why,” suggests Jaschik.
While assessment is often associated with concrete issues such as occupancy, it also can be optimized in areas such as increasing equity. Anne Lundquist, assistant vice president of campus strategy at Anthology, a higher education data and strategy company, and co-editor of the upcoming book Reframing Assessment to Center Equity, notes that “it is possible to develop tools that lift all students’ lived experiences, cultures, and identities. Rather than using a deficit model for and about non-white students, where the use of assessment results centers around mitigating perceived student weaknesses, equity-centered assessment scrutinizes institutional practices, policies, programs and culture for systemic barriers to student success.” Lundquist encourages housing professionals to “engage multiple stakeholders throughout the entire assessment process including collaborative evaluation design, data analysis, and storytelling.” She also challenges individuals to “interrupt their habitual way of doing things. Ask important questions such as ‘Who is at the table? Who is missing?’”
Campus housing professionals must manage a complex network of people, budgets, facilities, policies, and practices. Fortunately, a growing incorporation of assessment resources can be utilized to better understand these many factors, and these findings can be used to make improvements. Prudent leaders can link their assessments to divisional and institutional strategic plans. Acting wisely, housing professionals can avoid assessment pitfalls and leverage strategic and transparent assessments to drive innovation, promote equity, and make a difference in the lives of students and staff living on their campuses.
Virginia A. Koch, Ph.D., is an assessment and research specialist, and Eric M. Chrisman is an administrative coordinator for assignments. Both work in the Department of Housing and Residence Life at the University of Central Florida in Orlando.
