Date of Award
2026
Document Type
Open Access Dissertation
Degree Name
Doctor of Philosophy in Computational Science and Engineering (PhD)
Administrative Home Department
Department of Social Sciences
Advisor 1
Mark Rouleau
Committee Member 1
Michelle Jarvie-Eggart
Committee Member 2
Adrienne Minerick
Committee Member 3
Laura Brown
Abstract
Higher education institutions routinely evaluate equity initiatives using measures of participation, engagement, and satisfaction. While these approaches provide useful information about program delivery, they offer limited insight into whether interventions change how participants understand institutional inequities. As a result, important dimensions of organizational learning and awareness often remain difficult to detect, particularly within STEM higher education environments where evaluation systems frequently emphasize measurable outputs over deeper indicators of change.
This dissertation examines how awareness-related outcomes can be identified, measured, and communicated within equity intervention contexts. Using a three-study design, the research first presents a systematic review of equity intervention evaluation practices in STEM higher education, revealing a persistent reliance on satisfaction-based assessment approaches. It then develops and validates a perception-shift evaluation protocol using data from 248 participants across six iterations of Michigan Technological University's Advocates & Allies program. Findings demonstrate that participants consistently reported less favorable, yet more informed, perceptions of campus equity following workshop participation, suggesting that increased awareness may serve as an observable precursor to institutional change. Finally, the dissertation introduces the Data Storytelling Evaluation Protocol (DSEP), a computational framework designed to translate complex evaluation findings into formats accessible to institutional stakeholders.
Collectively, the findings challenge the assumption that participant satisfaction is an adequate indicator of intervention effectiveness and demonstrate the value of measuring awareness as an evaluative outcome. The dissertation contributes conceptual, methodological, and computational approaches that support more meaningful evaluation of equity initiatives and provide institutions with new ways to recognize, interpret, and respond to evidence of change.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Recommended Citation
Colbert, Karen, "Beyond Satisfaction: Developing a Data Storytelling Evaluation Protocol For STEM Equity Program Assessment", Open Access Dissertation, Michigan Technological University, 2026.
https://digitalcommons.mtu.edu/etdr/2186