DIMENSIONS OF EQUITY IN RPPS – A FRAMEWORK TO GUIDE PARTNERSHIP DISCUSSIONS

Stacey Sexton | SageFox Consulting Group

Jean Ryoo | Center X – Computer Science Equity Project, University of California, Los Angeles

Lisa Garbrecht | STEM Evaluation Services, The University of Texas at Austin

Renee Fall | National Center for Computer Science Education, College of St. Scholastica

Volume 2 Issue 3 (2020), pp. 12-18

INTRODUCTION

Research-practice partnerships (RPPs) have been growing in number across the nation as various fields–from medicine to education–realize their potential to address persistent gaps between research, practice, and policy. More specifically, RPPs are meant to better align research questions, methods, and intervention designs with the persistent problems that practitioners identify in their work, supporting the discovery of new knowledge that is more relevant to practitioners’ needs and interests. At the same time, RPPs can help ensure the implementation of evidence-based practices as educational projects become better informed by such relevant research. This is because RPPs bring together researchers and practitioners to jointly negotiate research questions, data collection, analysis, and findings dissemination through long-term, mutually beneficial collaborations (Coburn, Penuel, & Geil, 2013; Tseng, 2012). 

RPPs are also at the forefront of current Computer Science for All (CSforAll) efforts seeking to broaden participation in computing for the many young women, low-income communities of color, rural communities, English Learners, and students with disabilities who have been denied equal access to quality education experiences in the field (see, for example, Margolis et al., 2008/2017). Funders such as the National Science Foundation have been encouraging projects that leverage RPPs to develop a stronger knowledge base as well as to implement projects with broader positive impacts regarding students’ learning experiences in computing (see for example NSF 17-525, 18-537). To support shared learning across these projects, CSforALL and SageFox Consulting Group have partnered as RPPforCS to facilitate convenings and community-building among all RPPs funded through the NSF program (Sexton, Zarch, & DeLyser, 2020). The mission of RPPforCS is two-fold: (i) to organize the community of funded projects to maximize cross-project learning and knowledge sharing; and (ii) to conduct research with this community to understand how RPPs develop and change over time in the context of growing a new discipline within K-12 education. As of May 2020, RPPforCS has hosted 25 webinars, 2.5 in person gatherings, and have produced several Theme Studies and Research Practice Briefs with the goal of promoting cross-project learning and collaboration.

At one such convening that took place before the 2020 Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT) Conference, workshops were held to address various issues of interest to CSforAll RPPs. One workshop–entitled “Problems of Practice: Keeping the Focus on Equity in your RPP” led by Florence Sullivan, Jean Ryoo, Jill Denner, and Sneha Veeragoudar–was organized to support RPPs in learning from one another through identifying and discussing multi-level problems of practice concerning equity in RPPs. Over the course of the first hour, the assembled group worked to converge on a deeper understanding of the equity challenges faced by RPPs in order to shape a broader discussion encompassing the complexity of equity work from a number of perspectives. In the second hour, the discussion moved toward the articulation of strategies to continually center equity in RPP projects through a visualization activity. Within the context of the workshop, this meant either, 1) ensuring that projects/interventions consider issues of equity while resulting in equitable outcomes for students and educators, or 2) considering how to maintain respectful relationships within RPPs that equally honor the work and roles of both practitioners and researchers. Workshop attendees formed groups to create visualizations as tools to further their discussions.

In one such small group, four researchers (the co-authors of this piece) came together to reflect on ways to keep equity at the center of measurement and research / evaluation efforts within an RPP. The authors used the visualization activity to develop a framework that RPP teams can use to structure conversations around equity within their work. This framework includes questions that can support RPP discussions in every phase of a collaboration, from the early design process to the dissemination of findings. The co-authors are involved in RPPs focused on broadening participation in computing specifically, and that is reflected in our language throughout this piece. We believe this framework applies to any RPP examining issues of equity, however, and is not limited to those focusing exclusively on CS education. In what follows, we share details regarding the framework, literature informing this tool, and its potential applications in RPPs.

A FRAMEWORK FOR FACILITATING RPP EQUITY CONVERSATIONS

When designing this framework, we began our work with a focus on how to measure equity in educational outcomes. We  quickly realized, however, that a more holistic approach was needed to understand equity that moves beyond a focus on outcomes data while accounting for systems and structures of an RPP and within education. We reflected on the challenges of discussing issues of equity within various aspects of RPPs and how difficult it can be to find a starting place for open conversations across partners. As such, we decided that a useful tool would be a framework that outlines discussion questions to surface equity concerns within three domains: Equity in the Partnership, Equity in the Research, and Equity in the Practice / Implementation. Equity in the Partnership refers to equity within the actual RPP itself in regards to relationships and interactions across practitioners and researchers; Equity in the Research refers to all matters related to formation of research questions, data collection, data analysis, and engagement of the public with findings; Equity in the Implementation refers to all teaching/learning practices associated with the program or intervention under study. These three domains intersect to form what is the center circle in our diagram (see Figure 1 below). This overlap of the three domains at the center represents the ultimate aim of recognizing the utility of an RPP in deepening the understanding of equity in a context and developing a more relevant strategy for achieving equitable outcomes. 

We recognize that even engaging with this framework requires a willingness from both researchers and practitioners to ask tough questions, expose uncomfortable truths about existing structures, practices, and policies, and to (potentially) commit to addressing these. This framework does not address tensions that may arise as the partnership navigates questions such as, Whose job is it to identify structures / systems / policies that are counter-productive, or may have unintended consequences? Whose job is it to change them? What happens if the practice side partners refuse or simply can’t, given the highly political nature of these decisions? What role is the RPP expected to play then? As these questions emerge in the context of these discussions, we encourage teams to work through them as they are able. More generally, we hope that this framework can provide a structure for partnerships looking to delve deeply into equity work, and we presume that partnerships may discover boundaries (actions or ideas that are off-limits), barriers, and opportunities for advancing equity as a result of engaging with the framework.

Figure 1: Framework for Facilitating RPP Equity Conversations

A FRAMEWORK FOR FACILITATING RPP EQUITY CONVERSATIONS

>> Equity in the Research Practice PARTNERSHIP

Whether in the early stages of relationship-building or later phases of disseminating collaborative lessons learned, it is valuable to regularly reflect on the health of an RPP and the nature of power dynamics across partners. It is challenging for RPPs to enact equitable education interventions and research if relationships and interactions within the partnership itself are inequitable. Furthermore, embracing the reality that humans are imperfect beings and that we all have room to grow and be better at challenging oppressive thoughts, actions, and institutional structures, continuous reflection and dialogue within RPPs about whether or not the collaboration is equitably engaging and respecting all partners is important. 

In this reflection on centering equity in conversations about RPP relationships and work, we leaned on the questions and ideas raised by Henrick, McGee, and Penuel (2019) as well as Ryoo, Choi, and McLeod (2015). We developed a list of guiding questions for consideration within RPPs themselves as they engage with the implementation of educational interventions as well as collecting / analyzing data. More specifically, the following questions can support conversations about whether or not RPP interactions and efforts are effectively guided by a shared equity lens or not:

  • How has the RPP defined “equity” in regards to the shared work, mission/vision of the RPP, and partner relationships?
  • Has the RPP undergone a process to really understand their communities, unique sociocultural and historical contexts, and local needs?
  • Has equity and representation (roles, skill, identity, etc.) been considered in the design of the RPP membership?
  • Has the RPP built trust and cultivated healthy relationships among its members?
  • Has the RPP gained a shared understanding and appreciation of the diversity of skills and knowledge across the partnership that can inform the shared work?
  • How is the RPP attending to both rigorous research goals and practice partner organization goals?
  • Has the RPP developed a shared vocabulary and understanding of the topic of inquiry?
  • How are differential workloads, responsibilities, and roles being considered across the partnership?
  • Are practitioners represented in the RPP and given adequate space to share their perspectives?

These questions serve as a starting point for considerations of equity within the RPP regarding relations, expectation, effort, and representation across practice and research.

>> Equity in the RESEARCH

A key activity for many RPPs is conducting and using rigorous research to inform action (Henrick et al, 2017). Tseng, Fleischman and Quintero (2018) additionally speak to the pitfalls associated with traditional education research practices and propose RPPs as a potential model for democratizing evidence in education. 

While Tseng, Fleischman, and Quintero (2018) were speaking specifically about bringing researchers and practitioners together to define research agendas and make meaning of evidence produced through research, we believe this charge can be expanded to include democratized access to the entirety of the research process itself. Leaning on this concept of democratization, we drafted several guiding questions for partnerships to consider as they engage in the various stages of their research work.

Research Design

  • Are research and evaluation measures in line with the equity-related practice and implementation goals set by the project?
  • Have you considered how data categories have been constructed, and in what ways such categorical construction may mislabel, dehumanize, or misguide our understandings of minoritized populations?
  • How are research methods attending to the diverse understandings of culture and knowledge in learning spaces and/or partnerships? 
  • How are research methods attending to the power hierarchies of partnerships and educational contexts?

Data Collection/Analysis

  • How are data collection processes attending to participants’/partners’ needs, interests, and comfort levels?
  • How are data collection processes moving beyond just preventing harm to actually benefiting those involved?
  • Are teachers, school, and/or district personnel involved in analyzing and sense-making of the data collected?

Implementation

  • Were practitioners who wished it provided opportunities to understand the research processes such that they could meaningfully contribute to research-related activities of the RPP?
  • Have students been given the opportunity to understand the research being done, their role, and be involved as more than a research subject?

 Public Engagement

  • Has attention been given to how the broader community is engaging with results of the RPP specifically to impact practitioners’ work (i.e., more than a peer-reviewed article, or report to school/district leaders)?

These guiding questions build off of the reflective work of considering equity as presented in the previous section and also inform considerations of equity as they are built into the practice and implementation efforts of a partnership.

>> Equity in CS Education PRACTICE/IMPLEMENTATION

To achieve the ultimate goal of inclusive CS education for all students, RPPs must take into account the complexity of equity in the educational systems in which they are situated. Understanding and measuring equity in an RPP’s implementation goes beyond the diversity of students participating in CS education, as a demographic indicator alone may not reveal intersectional disparities and/or the systemic structures that contribute to pervasive inequities. Fletcher and Warner (2020) provide a framework to assess equity at four levels of the education system which are progressive and build upon each other, including Capacity for, Access to, Participation in, and Experience of CS education (CAPE). Looking at equity within these four levels can provide a more holistic understanding of disparities in computing education (Warner, Fletcher, & Garbrecht, 2019) and can help identify systems, policies, and/or practices that RPPs can address to achieve equitable student outcomes. 

Along with utilizing the CAPE Framework discussed above, we leaned on Santo et al.’s (2019) research on school districts’ conceptualization of equity in RPP implementation and Goode’s (2019) presentation of Microsoft’s Guide to Inclusive Computer Science Education to develop the following questions that can guide RPPs in their assessment of issues of equity and in their RPP’s design and practice at multiple levels of CS education.

Capacity 

  • How are the RPP’s strategies and implementation informed by and responsive to school and teacher capacity (e.g., resources, supports, skills) to support high quality, equitable CS education? 
  • What structures, systems and/or policies are in place that impact issues of equity in educational outcomes?
  • What role does the RPP have in addressing capacity issues in the educational setting? Do different members have different roles?

Access 

  • Who has access to CS education in the schools? What CS pathways are available and for whom?
  • How is the RPP addressing structural barriers and beliefs systems that may inhibit access? Are the changes being undertaken by the RPP having an impact on the structural barriers and beliefs systems that may inhibit access?

Participation

  • Who is participating in CS? Does CS participation reflect school-level demographics?
  • What processes and/or strategies is the RPP supporting to address gaps in student participation (e.g, teacher professional development, gatekeeper engagement, recruitment, policies)? Is there evidence that RPP-driven processes and strategies are having an impact on the gaps or barriers they were designed to address?  

Experience

  • Are the curriculum, pedagogy, and learning space culturally responsive? Does this impact learning?
  • How is the RPP supporting teachers in providing culturally responsive lessons,  practices, and experiences customized for their students and context?
  • How are student engagement, agency, and identity supported and assessed?

>> Equity at the NEXUS: Understanding Equity in CS Education via an RPP

Equity can be infused into each of the three dimensions of RPPs focused on broadening participation in computing education: the partnership, the research, and the practice or implementation of educational innovations or interventions. The framework thus provides questions to assist partners in deeper discussion of how equity is expressed in these three areas. At the intersection of these dimensions is the nexus of this work, which reflects the value of RPPs as a vehicle for understanding and achieving equity in CS educational contexts. Answers to questions such as these (below) following discussions of equity in each dimension may support RPPs in better understanding or defining equity goals and directions at any point in their work.

  • How is “equity” defined in educational practice, enacted in your partnership’s behavior, and studied through research in your RPP?
  • What are the barriers to achieving your equity goals and from where do they arise (implementation, research, or  partnership)?
  • How can relationships, roles, and responsibilities across the partnership be leveraged to potentially address the barriers to equity that you have identified?
  • How is your implementation of equity-focused practice informing research?
  • How is research on equity informing practice?
  • How is your partnership enhancing or inhibiting equity-focused practice? 
  • How is your partnership informing research?

CONCLUSION

This framework was designed as a resource to support RPPs in structuring conversations on equity throughout all phases of their work. The framework can be used in a retreat or workshop setting with RPP teams examining questions from each dimension to facilitate discussions on how they are conceptualizing, assessing, and addressing equity in their partnership, research, and practice. Questions in this framework can serve as a starting point, prompting RPPs to identify and discuss other questions that may be more relevant to equity in their RPP. Through these discussions, teams can gain a deeper understanding of equity in computer science education within the context of their RPPs, illuminate areas of success or concern, and identify ways to strengthen their strategies for achieving equitable outcomes. We encourage RPPs to try out this framework with their teams and to explore formats and identify questions that are most pertinent to their projects. We intend to build on this framework in an iterative process and welcome your questions, feedback, and suggestions about utilizing the framework to support equity in RPPs. 

The co-authors would like to acknowledge the NSF CSforAll funding that supports the RPPs in which we conduct research and evaluation as these experiences have greatly informed the development of this framework. We would also like to thank CSforALL, SageFox, RPPforCS, and facilitators for convening the 2020 RESPECT pre-conference workshop that sparked the development of this framework, especially the inspiring workshop facilitation by Florence Sullivan, Jean Ryoo, Jill Denner, and Sneha Veeragoudar.  

 

Stacey Sexton is an Evaluator with SageFox Consulting Group [ssexton@sagefoxgroup.com]; Jean Ryoo is the Center X – Director of Research for the Computer Science Equity Project at University of California, Los Angeles [jeanryoo@ucla.edu]; Lisa Garbrecht is the Expanding Pathways in Computing (EPIC) Director of STEM Evaluation Services at The University of Texas at Austin Texas Advanced Computing Center, [lgarbrecht@tacc.utexas.edu]; and Renee Fall is a Senior Research Scholar for the National Center for Computer Science Education at College of St. Scholastica [rfall@css.edu].

 

Suggested citation: Sexton, S., Ryoo, J., Garbrecht, L., & Fall, R. (2020). Dimensions of Equity in RPPs – A Framework to Guide Partnership Discussions. NNERPP Extra, 2(3), 12-18.

WORKS CITED

Coburn, C.E., Penuel, W.R., & Geil, K.E. (2013). Research-Practice Partnerships: A strategy for leveraging research for educational improvement in school districts. New York, NY: William T. Grant Foundation.

Fletcher, C.L. & Warner, J. R., (2020). Summary of the CAPE Framework for Assessing Equity in Computer Science Education. Retrieved from https://www.tacc.utexas.edu/epic/research 

Goode, J. (2019). Microsoft Guide to Inclusive Computer Science Education. Presented at the National Girls Collaborative Project National Webinar. April 23, 2019. https://ngcproject.org/microsoft-philanthropies-and-inclusive-computer-science-education

Henrick, E.C., Cobb, P., Penuel, W.R., Jackson, K., & Clark, T. (2017). Assessing Research-Practice Partnerships: Five Dimensions of Effectiveness. New York, NY: William T. Grant Foundation.

Henrick, E., McGee, S., & Penuel, W. (2019). Attending to issues of equity in evaluating research-practice partnership outcomes. NNERPP Extra, 1(3), 8-13. Retrieved from http://nnerppextra.rice.edu/attending-to-issues-of-equity-in-evaluating-rpps/

Margolis, J., Estrella, R., Goode, J., Jellison-Holme, J., & Nao, K. (2008/2017). Stuck in the shallow end: Education, race, and Computing. Cambridge, MA: MIT Press.

Ryoo, J.J., Choi, M. & McLeod, E. (2015). Building equity in research-practice partnerships. Research+Practice Collaboratory. Retrieved from https://www.exploratorium.edu/sites/default/files/pdfs/BuildingEquity_Oct2015.pdf

Santo, R., DeLyser, L.A., Ahn, J., Pellicone, A., Aguiar, J. & Wortel-London, S. (2019). Equity in the Who, How and What of Computer Science Education: K12 School District Conceptualizations of Equity in ‘CS for All’ Initiatives. Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT), Minneapolis, MN, USA, 2019, pp. 1-8. Retrieved from https://ieeexplore.ieee.org/document/8985901 

Sexton, S., Zarch, R., & DeLyser, L. (2020). Spotlight: An RPP approach to computer science work. NNERPP Extra, 2(1): 18-20.

Tseng, V. (2012). Partnerships: Shifting the dynamics between research and practice. New York, NY: William T. Grant Foundation.

Tseng, V., Fleischman, S., & Quintero, E. (2018). Democratizing Evidence in Education. In Connecting Research and Practice for Educational Improvement: Ethical and Equitable Approaches, Bronwyn Bevan and William Penuel (Eds.). New York, NY: Routledge.

Warner, J. R., Fletcher, C. L., & Garbrecht, L. S. (2019l). Better data, better progress: Methods for measuring inequities in computer science education. American Educational Research Association Annual Meeting, Toronto, Canada. Retrieved from https://utexas.app.box.com/s/llovod6afe7ucuwzlrvf1bgbfdnayfgb

NNERPP | EXTRA is a quarterly magazine produced by the National Network of Education Research-Practice Partnerships  |  nnerpp.rice.edu