by Erick Montenegro and Gavin Henning
Horace Mann, the influential American educator and architect of the country’s public education system, once said that education – beyond all other devices – is a great equalizer. This was true in 1848 when he uttered these words, and it remains true today. Now, and perhaps more than ever, the importance of an education – specifically of holding quality higher education credentials – can determine entire families’ futures and future generations.
But what happens when the promise of education remains unmet for some, generation after generation? When learning outcomes attainment varies by student demographics? When the return on investment in a college education is disproportionately less for some student populations than it is for others? The answer is that inequities grow. Society continues to further divide and stratify instead of equalize.
Within this conversation around equity – specifically the ways it manifests in higher education through graduation rates, campus climate, norms, assumptions, and the uneven return on educational investment – there also is one regarding the purpose of assessment. If assessment practitioners can approach their work through an equity lens, it will allow them to serve as a vehicle for just educational experiences and equality through equitable assessment praxis.
At its core, equity means being mindful of fairness and giving everyone their fair share in specific situations. It is also the process that provides us with the underlying principles to reach equality; equity is equality turned into action (Smith & Gorard, 2006). Equity means acting to remedy injustices. It involves interrogating, reflecting, and improving practice, policy, and perspectives. It necessitates intentionality and commitment. Through being ever mindful of equity, assessment practitioners can make inequities visible and demonstrate where and when injustice and differential experiences create and sustain inequities within institutions (hooks, 1998; Pasque et al., 2012). But to even do equity work, we need to identify where those inequities take place, how they manifest, and for whom. We have to go beyond the conversational layer on issues of access, retention, and graduation amongst different student groups toward dismantling the environments and customs that create these metrics. Equity requires using this knowledge and data to change the status quo.
Assessment can be a valuable tool in either addressing and resolving inequities on campus or perpetuating them. Assessment – the systematic “gathering and use of evidence of student learning in decision making and in strengthening institutional performance and public accountability” (Kuh et al., 2015, p. 2) – can help identify areas where student populations are experiencing inequitable outcomes attainment, inform decision making to address identified inequities, and explore how changes have impacted students. But conducting assessment to uncover and remedy inequities requires intentionality and conviction at every step of the assessment process, even in the purpose of undertaking assessment in the first place. An equity lens can give assessment practitioners cause to reflect on the assessment process as a whole to ensure that we do the following:
1. check biases and ask reflective questions throughout the assessment process to address assumptions and positions of privilege;
2. use multiple sources of evidence appropriate for the students being assessed and assessment effort;
3. include student perspectives and take action based on perspectives;
4. increase transparency in assessment results and actions;
5. ensure collected data can be meaningfully disaggregated and interrogated; and
6. make evidence-based changes that address issues of equity that are context-specific. (Montenegro & Jankowski, 2020, p. 13)
Assessment in higher education has been used primarily for two purposes: accountability and continuous improvement, which can also be seen as summative and formative types of assessment, respectively (Ewell, 2009). The goal of accountability is to ensure that colleges and universities are achieving their espoused goals and student learning outcomes. In other words, are colleges and universities doing what they say they are doing? The goal of continuous improvement is to find opportunities to improve courses, programs, and services to advance learning. The focus for improvement can be on an individual course, an academic program, a cocurricular program or service, or the institution. Ideally, regardless of level of focus, assessment processes are embedded in everyday practice, providing critical feedback about what is working well and what needs to be improved.
Ewell (2009) argued that while accountability and improvement are two main purposes of assessment, they can be in tension with one another. Resulting from state and federal performance mandates, accountability requires an organization to use evidence to demonstrate achievement of expected standards or outcomes “in a posture of institutional compliance, or at least appearance of it” (Ewell, 2009, p. 9). Improvement, on the other hand, centers on making changes to processes to increase student learning, development, and success (Ewell, 2009). Not only are these purposes incentivized differently, but each uses data differently. Thus, while both accountability and improvement seem to be worthy goals of assessment, they may conflict with each other. Despite this encumbrance, both roles are critical in the effective functioning of colleges and universities and both can further equity and inclusion.
Approaches to assessment are undergirded by systems of assumptions, beliefs, and theories regarding the construction of reality and knowledge, which in turn affect how assessment is implemented. These systems or paradigms, often unconscious, are philosophies or worldviews for making sense of the world (Patton, 2014) or a set of beliefs that guides action (Lincoln et al., 2011). These paradigms influence the extent to which equity can be integrated into assessment. Paradigms have their foundation in research, but those research paradigms are relevant to assessment work. Kivunja and Kuyini (2017) argued that there are four research paradigms that can be applied to educational contexts: positivist, interpretivist/constructivist, pragmatic, and critical/transformativist.
The positivist paradigm includes a number of assumptions, including the tenet that knowledge is objective, measurable, and generalizable (Egbert & Sandeen, 2014). Another belief is that there are universal, unchanging facts, and to comprehend these facts researchers must be external to that which is being researched (Taylor & Medina, 2013). The main theme of positivism is objectivity: the belief that knowledge can only be generated by unbiased observation of the phenomenon from a distance. The underlying assumptions of positivism and the procedures built upon it have implications for equity in that data gathered through quantitative methods are aggregations of individual responses with a goal of discovering what is common amongst participants. In an attempt to be objective, the results of a positivist approach are general, not specific; lifeless, not robust; blunt. Quantitative assessment results do not take into account the subjectivity of students’ experiences and thus only provides limited understanding of a phenomenon. Assessment built upon a positivist paradigm is often inequitable because individuality and the context for data is not part of the assessment process.
The goal of the interpretivist/constructivist paradigm is to understand the subjective element of the human experience (Guba & Lincoln, 1989), providing the context that a positivist approach cannot. Emphasis is placed on the individual and their interpretation of their experience and the world around them (Kivunja & Kuyini, 2017), and an important belief is that reality is socially constructed (Bogdan & Biklen, 1998). There is an assumption that the researcher (or assessor) is connected to and engaged with the phenomenon rather than objectively distanced from it as in the positivist paradigm. Assessment emanating from an interpretivist/constructivist paradigm utilizes qualitative data collection and analysis methods. As such, there is a value placed on the individual interpretation of reality, and culture and context are paramount to understanding. This focus on individuality and context makes assessment based on this paradigm more equitable than one based on a positivist paradigm.
At its core, equity means being mindful of fairness and giving everyone their fair share in specific situations. It is also the process that provides us with the underlying principles to reach equality; equity is equality turned into action.
A pragmatic paradigm bridges positivist and interpretivist/constructivist paradigms in an effort to implement methods that are most appropriate, or practical, for the phenomenon being studied (Kivunja & Kuyini, 2017), whether they be quantitative or qualitative. This paradigm further supports the integration of equity and assessment as it is not beholden to any one set of assumptions regarding reality and knowledge, and multiple methodologies can be employed, providing a more complete picture of complex phenomena from various vantage points. Pragmatism breaks free from a traditional objective or subjective stance to broaden the set of assumptions underlying assessment practice.
The main foci of a critical paradigm are the roles of power and oppression in understanding reality. There is recognition of the impact of power and oppression on individual interpretation and social construction of a phenomenon (Bronner, 2011). Key assumptions within this paradigm include concern with power relationships and social structures that perpetuate them, the acknowledgment that some versions of reality are privileged over others, and an emphasis on the construction of reality rather than the discovery of reality (Kivunja & Kuyini, 2017). This approach supports the use of both qualitative and quantitative methods of data collection and analyses to expose the social structures that perpetuate oppression and to identify strategies to dismantle these structures. Assessment based on this paradigm has the most promise for integrating equity and assessment. It builds on interpretivist/constructivist assessment to take into account subjectivity but recognizes that power affects reality and thus equity. It affords the use of multiple methods for a more complete understanding of a phenomenon. Equity-centered assessment cannot be undertaken without acknowledgment of the role that power and oppression play in student learning, development, and success and recognition that assessment can be a vehicle to address power and oppression when it is used to identify solutions.
An emerging research paradigm being applied to assessment is an Indigenous paradigm, which is rooted in four axial assumptions: responsibility, respect, reciprocity, and rights and regulations (Snow et al., 2016). Assumptions of this paradigm include the belief that people are connected to each other, the earth, and the cosmos; that knowledge is relational, constructed, shared, and cannot be discovered or owned by any one person; and that equity, equality, and inclusion are key values (Wilson, 2008). Assessment rooted in this paradigm emphasizes the role of participants as equitable partners with assessors in the assessment process, as co-creators of knowledge as part of an inclusive, connected, collaborative process.
Each paradigm has a set of beliefs, a perspective, a lens that is applied to the assessment questions and unconsciously affects methods, data collection systems, policies, and the institutional structures put in place to implement and support assessment processes and practices. Thus, recognizing and scrutinizing the paradigms that undergird assessment are needed if we are to intentionally integrate equity into assessment practice.
Shifting assessment from accountability and improvement to inquiry provides another way to answer key questions in higher education. Inquiry seeks to answer “why” questions rather than “what” questions. “Why” questions attempt to understand the reasons for something happening rather than simply describing a phenomenon. Inquiry takes into account the relational nature of reality, acknowledging that student experience and learning does not occur in a vacuum, but takes place in institutions of higher education that are steeped in centuries-old systems of power and oppression.
An example could be the graduation rates across student identity groups. An assessment approach that tries to answer a “what” question would identify that there are differences in graduation rates for white students compared to those for Black/African American, Latinx, Asian/Asian American, and Native students.
Until underlying paradigms are understood, assessment methods rooted in these beliefs cannot address inequity in higher education. Moving beyond a focus on accountability and improvement, implementing assessment as inquiry with equity in mind can be a powerful tool for change.
While it can be easy to think that assessment is free from the concerns of environmental microaggressions or won’t feed into the effects of a negative campus climate, the reality is that practices such as not disaggregating data, not involving the perspective of students in assessment, using language in outcomes and assignments that is not easily understood by students, and lacking transparency in grading are examples of the same negative ones that contribute to inequities.
Assessment is not apolitical. Assessment is not fully objective. Assessment happens within a campus context that is situated within a larger society, and assessment is conducted by human beings with specific preferences, perspectives, experiences, and dispositions. Those conducting assessment need to be well versed on the issues of equity that impact students in their classroom, in their program, and in their institution.
Equity means acting to remedy injustices. It involves interrogating, reflecting, and improving practice, policy, and perspectives. It necessitates intentionality and commitment.
In assessment, environmental microaggressions take the form of using aggregate data, eliminating small sample sizes, and not including students’ voices. These are systematic issues and systems of power and subjugation that turn assessment into a tool for the perpetuation of inequities. Using aggregate data, doing surface-level data disaggregation, or simply identifying gaps in data analysis is not in itself equitable assessment. Equitable assessment requires a deeper dive into the data, disaggregated by different student populations, its sub-populations (e.g., further analyzing Latinx populations by comparing the outcomes of Mexican Americans and Cuban Americans), and the intersection between different student characteristics (e.g., first-generation Mexican American students who also identify as female and are commuter students; Montenegro & Jankowski, 2020). Failing to take this deeper dive into the data can lead to false understanding and missing potential inequities, not to mention the issues that arise when populations are continuously disregarded due to small sample sizes (Montenegro & Jankowski, 2020). If assessment practitioners continuously do assessment in ways that are not mindful of inequities, then we are doing our students a disservice.
There also cannot be equitable assessment without purposefully including the student voice in the assessment process (Montenegro & Jankowski, 2017, 2020). Nothing sends a clearer message to students of not being valued than to be excluded from processes that ultimately have a direct impact on their own success. Meaningful student involvement in assessment can help mitigate issues created when assumptions are made about students without evidence, can help make the assessment process more relevant for different student populations, and can inform changes that minimize negative unintended consequences while strengthening their intended impacts (Montenegro & Jankowski, 2020).
Assessment can help institutions meet their goals, set benchmarks, and make evidence-based changes to improve. It is a process that helps institutions be at their best and be accountable, but institutions cannot do this when there are inequities happening on campus. Equity ensures that students of color are in the best position to succeed, attain the learning outcomes set forth by their institutions, and ultimately graduate. Assessment and equity work together to more effectively help institutions and students achieve their goals.
Conducting more equitable assessment is not a radically different way of doing assessment. In fact, it relies on a sound assessment process already being in place and adding an intentional focus on equity throughout. Equitable assessment involves more instances of reflection to interrogate the assessment process in place, to consider how the assessor’s identity impacts the assessment, to identify which perspectives are missing from the assessment process, and to act accordingly to address any and all issues.
It should go without saying, but equitable assessment does not mean we use assessment to only identify and close equity gaps (Montenegro & Jankowski, 2020). However, seeking to identify and fix equity gaps should be part of the purpose of assessment. Without conducting assessment from an equity perspective, there is a risk of further contributing to the marginalization of students of color and thus failing on the promise of education for an ever-growing number of college students. It is a moral and practical imperative to conduct equitable assessment, and it begins by interrogating our own practice. We need to look inward and ask what role higher education assessment processes have in either perpetuating or remedying the inequities affecting students.
Erick Montenegro, Ph.D., is a Fellow with the National Institute for Learning Outcomes Assessment focused on equity-minded assessment practice. He currently is the director of communications for Credential Engine. Gavin Henning, Ph.D., is professor of higher education and director of the Master of Higher Education Administration and Doctorate of Education Programs at New England College in Henniker, New Hampshire. Over his more than 20 years in higher education he has been a professor, assessment practitioner, and student affairs administrator.