by Matt Venaas and Sherry Woosley
Whether it be survey data or occupancy rates, every longitudinal data set collected over enough years has that point where the number jumped or dropped dramatically. It is that single point – the outlier – on the line chart that sticks out like a soaring mountain or a plunging crater and, as a result, gets labeled with an asterisk. These outliers are the result of many things. Perhaps a housing department made a major shift in staffing or underwent a reorganization. Maybe there was a drastic change in the department’s budget. Or, perchance, there was an unexpected issue or series of critical events.
Welcome to 2020.
It would be an understatement to say that the COVID-19 pandemic has forced unexpected and unprecedented changes on colleges and universities around the world. Budgets, facilities, staffing, services, and more were affected dramatically. In the wake of campus closures, many campuses or departments chose not to try collecting key outcome metrics. The need to focus on health and safety issues rightfully topped all others.
But the unexpected did not stop with the closing of campuses. Housing and residence life professionals were forced to confront other profound issues related to both higher education and society. The killing of George Floyd triggered widespread protests and increased scrutiny, bringing renewed frustration and focus on policing, systemic racism, and social justice issues. Add in staggering unemployment, economic uncertainty, and a host of unanswered questions around the pandemic and it becomes glaringly apparent that, for the foreseeable future, higher education will be anything but normal. That asterisk everyone hoped would only be attached to 2019-20 is likely to stick around for multiple years.
So what does this mean for housing departments and assessment efforts? What data is needed? What will the data mean during this unusual time? Should assessment efforts even continue?
In short, absolutely. In fact, they have to.
In the face of so many challenges, campuses could maybe be forgiven for simply trying to survive the year. However, for many campus housing professionals, simply surviving isn’t enough. Rather, a commitment to caring is an underlying foundation of their work. To care this year means collecting data to better understand the unique experiences of students and staff. Students need a safe, productive, and welcoming environment. Staff must have the space, resources, and confidence to excel as professionals. And everyone wants to do good work. In general, assessment can be the tool to understand not only what steps were taken in the face of these unique situations, but also what was and was not successful so that practices can be improved moving forward.
Assessment will also inform work in the short term. In his most recent book, The Infinite Game, author and motivational speaker Simon Sinek wrote about how humans repeatedly practice to have the skills needed to direct behaviors. But, as he discussed with retired Navy Seal commander Rich Diviney in a recent YouTube video, that practice/skills approach only works in a known situation. In times of uncertainty, one has to lean on different attributes, including situational awareness and adaptability. Right now, ambiguity is constant, and the pace of change is dizzying. Collecting data in the moment helps housing professionals to better understand and adapt to the situation when one can’t rely on previous knowledge and experiences. Many departments and staff are trying and implementing new practices, from enhanced cleaning in facilities to virtual engagement with residents. The only way to know the extent to which these new practices are working is to collect data about them.
Finally, housing and residence life will need data in the long term. Without data, any explanation can be suggested and argued, which means that discussions are more often won through power, charisma, and strong voices. Data can change the dynamic, allowing good information to successfully challenge or confirm deeply held opinions that may or may not be true or accurate. In the long term, data can inform planning, justify budgets and staffing, and show how – especially in difficult times – housing and residence life departments make a lasting positive difference in the lives of college students. It may be difficult to recognize now, but campus housing professionals will need more than anecdotes to show the impact of current efforts.
Guiding Principles
While there’s no doubt that assessment needs to continue, this does not mean that campus housing professionals should simply carry on with a plan similar to that of the previous years. An overhaul or adjustments may be necessary. If new steps are taken, however, it’s important that these new approaches stay student-centered, inspired, and focused on the bigger picture.
More than ever, the focus and attention of housing and residence life professionals should be on students. Economic impacts, sustained isolation, and global discord have played out every day since the onset of COVID-19. A Skyfactor survey of more than 13,000 students conducted earlier this year, showed that 55% of students were concerned about income, 31% about job security, and 13% about food security. In addition, protests against systemic racism and the government response to the protests have raised conversations about social justice and how these issues could exacerbate trauma for students. Assessment efforts must be mindful of this fundamental, widespread, and sustained impact. Empathy and care must play a key role in assessment development.
In the midst of uncertainty and even trauma, it is dangerous to approach assessment as something a department should do simply out of a sense of obligation. Rather, it should tap into individual and institutional curiosity to inspire more value. Recent research has highlighted the advantages of leveraging curiosity in both personal and professional endeavors. In a 2018 article on curiosity in the Harvard Business Review, behavioral scientist Francesca Gino wrote, “When our curiosity is triggered, we think more deeply and rationally about decisions and come up with more creative solutions,” and “When we are curious, we view tough situations more creatively.”
What does curiosity look like in the context of housing assessment? Ask questions. What is different this year? What impact would that have on the student experience? In what ways are students struggling? What unexpected good things are happening in new environments? How do students see this program, activity, or service? All of these are typical questions for assessment, but curiosity encourages one to think more deeply than just how to measure something. It’s about wanting to better understand the what and why of the work.
Finally, assessment efforts during an asterisk year should still reflect the fundamentals of good practice by focusing on the big picture. What are the program goals? How do those fit into the goals of the department and institution? Are there bigger frameworks or theories to consider? For instance, while the process may look different this year, departments will still go through program review. Institutions will still go through accreditation processes or at least continue to collect data for future accreditations. What data will be needed, even during a pandemic, to show progress and outcomes? In the midst of day-to-day demands, campus housing professionals need to remain focused on the big picture. Many tough conversations are already happening, especially around budgets, staffing, and resources. More will happen in the future. A big picture focus will ensure that they are ready for those conversations.
Campus housing professionals will need data to tell the story of their work, how they matter, how they make a difference, and even why they are worth it. They will need to discuss facilities, staffing and roles, programs and processes, outcomes, and budgets. Interestingly, one can even use poor results to illustrate what happens to outcomes. For instance, a sense of belonging and community will decrease when students are isolated at home, and the lack of welcome week activities will create transition issues for some students. Thinking ahead to the stories that staff and departments will need to tell about their work can help highlight the big picture and provide information about what data should be collected and shared.
Assessment should continue, but in the time of COVID-19 it’s going to have to look different as campuses must adjust, experiment, and prioritize. These changes will range from the micro to the macro, but all are important. For example, many of a campus’s existing surveys will need to be modified. If all residents are living in single bedrooms, questions about roommates should be removed. If shared spaces like lounges are off-limits to ensure social distancing, then questions about lounges should be removed. Similarly, questions may need to be added. In cases where activities are virtual instead of in-person, questions could be added related to satisfaction with the technology used. Small edits may be enough to ensure that the data are relevant to the current campus context.
Without data, any explanation can be suggested and argued, which means that discussions are more often won through power, charisma, and strong voices. Data can change the dynamic, allowing good information to successfully challenge or confirm deeply held opinions that may or may not be true or accurate.
Housing programs may need to consider different timing for gathering data. Waiting until the end of the year to assess quality-of-life may be too late, so such surveys should perhaps be moved to mid-year so they can identify and address issues before the year is over. If this data is to be collected via focus groups, those most likely will need to be conducted virtually. Just as activities were adapted to virtual environments, focus group facilitators can adapt their activities, incorporate virtual polls, and use chat features to prompt and collect discussion. Regardless of whether it’s a survey or a focus group, housing professionals should rethink or adjust strategies related to invitations, timelines, and incentives. Invitations should include language reflecting the new challenges that students currently face. Typical incentives such as bookstore and coffee shop gift cards may need to be adjusted to better reflect virtual or social distancing options.
This unpredictable academic year can be an opportunity to break from current practices and try new approaches. Virtual environments create different kinds of data. New programs mean opportunities to rethink outcomes and how they are measured. Plus, while the campus is focused on high-stakes issues like health and safety, new assessment strategies or experimental activities will receive less attention and thus be less risky to try.
One strategy is to add formative assessment activities to training exercises. Many departments have been conducting staff training virtually. Asynchronous activities often include self-paced learning. Worksheets and reflections can become assessment documents. Virtual meetings can incorporate engagement tools that collect formative information. The University of South Carolina in Columbia, for example, added iClicker to their staff training sessions to engage staff in activities. Questions asked during a session also gauged knowledge and influenced pacing. After sessions, built-in exit polls were used to determine which concepts were still unclear.
It is important to recognize that the new conditions may make it more difficult to gather information. Understanding the student experience is tougher because informal interactions with students are limited, masks hide non-verbal cues, and virtual sessions make reading the environment or mood problematic. Spotting students or staff who are struggling can be more challenging in social-distanced environments. One strategy to bypass this would be to use short or pulse surveys. For instance, the University of Tennessee at Martin used a short survey from Skyfactor to do a systematic check-in with all students. Learning from that approach, Skyfactor is experimenting with short retention surveys that can be deployed quickly and used more frequently this fall.
In addition, campuses more than ever will need to use existing data sources and triangulate the information. In previous years, housing and residence life professionals were able to make decisions relying heavily on experience and knowledge related to the calendar and existing practices. Many data sources can provide ways to answer those questions and test gut reactions in the current context. By examining the questions students ask in emails or reviewing data collected in staff forms, employing the curiosity perspective highlighted by Francesca Gino can provide new insights. Consistent information in multiple places provides credibility to the insights.
Whether it is the economy, sports records, or student satisfaction surveys, 2020 will forever carry with it an asterisk. In that light, it is important to recognize that campus housing professionals are not going to be able to do everything they are accustomed to doing, especially as the current situation continues to evolve. Departments may be forced to delay, skip, or cancel some assessment efforts. Housing and residence life professionals will have to focus on the key activities and prioritize. When reviewing projects and tasks, it will be useful to look for projects that can be postponed or dropped and to do so within the context of higher level goals and strategy. Does the assessment strategy collect key data points needed for program review? Are data points tied to particular outcomes? Does the project provide information for taking quick or necessary action? If not, perhaps it can wait. Erin Bentrim, divisional director of student affairs research and assessment at the University of North Carolina Charlotte, used the term strategic abandonment to describe how departments may have to proceed.
Despite the inevitable asterisk, though, assessment efforts should not stop. They may need to be modified or scaled back, and programs are going to have to prioritize what data to collect and how. By keeping in mind the big picture, the purpose of the work, and the students and staff affected, assessment efforts can continue. In fact, those efforts need to continue now more than ever.
Matt Venaas is a research manager at Macmillan Learning and a previous recipient of the Robert P. Cook Talking Stick Article of the Year Award. Sherry Woosley is the senior director of analytics and research at Macmillan Learning.
