OVERVIEW

The Research Artifact

OpenSciEd is a field-level initiative in science education to develop freely available, Next Generation Science Standards (NGSS)-aligned, K-12 science curricula and professional learning. The inquiryHub research-practice partnership is a lead organization in a consortium developing and testing high school courses in biology, chemistry, and physics that address all of the NGSS, including Earth and space science standards. The units we are developing follow a unique “storyline approach” (Reiser et al., 2021) designed to be relevant to students and coherent from their point of view, where units are designed to anticipate and guide students to answering their own questions about a given phenomenon or design problem. The inquiryHub partnership’s work is supported with funding from the Carnegie Corporation of New York, the Walton Family Foundation, the Bill and Melinda Gates Foundation, and the William and Flora Hewlett Foundation. 

This research “artifact” is an example of an artifact where ideas from research are embedded in tools designed to directly impact teaching and learning outcomes (see Ikemoto & Honig, 2018). In particular, OpenSciEd uses a “phenomenon-based” approach to problem-based learning in science, for which there is a strong evidence base for promoting deeper learning. Recent experimental studies have found this approach to be effective in promoting the kinds of three-dimensional learning outcomes of the NGSS that require students to apply core ideas, practices, and crosscutting concepts to explaining phenomena and solving problems (Krajcik et al, 2022; Schneider et al., 2022).  

In addition, the curriculum materials have been informed by broad testing in a large sample of classrooms across the country, using methods adapted from improvement science (see Edelson et al., 2021 for a description of the approach used in supporting the development of middle school materials). As such, the materials reflect the experiences of students, the insights and feedback of teachers, and guidance provided by state level partners along the way, as elaborated below.

iHub: Our Mission

The inquiryHub research-practice partnership was founded in 2007 to support the ongoing work of curriculum and instructional leaders in Denver Public Schools (Colorado) in mathematics and science education. The mission of the RPP is to ensure that all students, but particularly those from systemically marginalized groups and communities, experience STEM learning that is coherent from the student point of view, personally relevant, and connected to important matters of concern to their communities. In addition, we aim to support students in becoming active contributors to collaborative knowledge-building efforts that give them a feeling for what disciplinary practice could be.

The partnership works toward this mission by engaging in efforts to design and test new instructional materials (Penuel, A-R. Allen, et al., 2022), approaches to professional learning (Penuel, C. Allen, et al., 2022), and assessments (Penuel et al., 2019). We work to build more coherent and equitable instructional guidance infrastructures (Hopkins et al., 2013) at the district level. We call this work infrastructuring work (Penuel 2019a, 2019b), inspired by interdisciplinary scholars in the information sciences (Star & Ruhleder, 1996) and computing (Pipek & Wulf, 2009).

A Design-Based Implementation Research (DBIR) Approach

The inquiryHub is an example of a design-based research-practice partnership, in which co-design and testing of innovations is a leading activity. The RPP has been an incubator for the approach to research called Design-Based Implementation Research (DBIR; Fishman et al., 2013). DBIR adheres to four principles that iHub projects follow: 

1. Teams form around shared goals that address persistent problems of educational inequity identified through negotiation among multiple stakeholders’ perspectives and values;
2. To improve practice, teams commit to iterative, collaborative design;
3. To promote quality in the research and development process, teams develop theory, knowledge, and practical tools related to both learning and implementation through systematic inquiry; and
4. To promote lasting improvements, DBIR is concerned with developing capacity for sustaining change in systems.

DBIR is an approach that requires long-term research-practice partnerships. The work of infrastructuring, for example, requires partners with strong bonds of relational and politicized trust (see Vakil et al., 2016), because the work of producing more equitable systems frequently requires risk taking and taking difficult stands. Also, changing infrastructure requires persistent and ongoing efforts to align elements of infrastructure with one another and to a vision for equitable teaching and learning (Penuel, 2019b).

WHY THIS WORK

When OpenSciEd issued a request for proposals in 2020 for a high school developers consortium to develop materials for high school, inquiryHub was well positioned to bid on the request, because the RPP already had co-designed biology and chemistry courses aligned to Colorado Academic Standards in science, which are based on A Framework for K-12 Science Education (NASEM, 2012). We began developing these initial materials together ahead of standards adoption, beginning in summer 2014. Our intent was never to develop a full curriculum or to make the materials widely available beyond Denver. However, because Denver Public Schools leaders insisted on a full course in biology and teachers demanded one in chemistry, the team built both. Soon after, educators and leaders in other states found the iHub materials. The state of Louisiana’s initiative to promote high quality instructional materials adoption was particularly influential in helping the inquiryHub partnership “up its game” to be able to provide professional learning opportunities for teachers at the scale of a state. Today, inquiryHub materials are in use in more than 45 states. We’re in the process of now updating the materials, unit-by-unit, replacing them with new OpenSciEd units over the next year until three free courses in biology, chemistry, and physics become widely available and adoptable by districts by July 2024. A goal is to receive a “green” rating (the highest) for quality from EdReports, an independent review organization for instructional materials, to ensure broad adoption.

The OpenSciEd development project has been an extension of our work in Denver and builds on our approach to partnering strategically with other organizations to build the kinds of teams we need for specific lines of work. The core iHub partners who have been involved as part of OpenSciEd have been Denver Public Schools and the University of Colorado Boulder, Northwestern University (Evanston, IL), BSCS Science Learning (Colorado Springs, CO), and the Charles A. Dana Center at the University of Texas. The new materials in development support iHub’s core mission of promoting equitable and meaningful opportunities for science learning to students. In this project, iHub and its partners are working under contract for a nonprofit organization, OpenSciEd, which will be responsible for the continued curation and widespread distribution of materials. In addition, there is another, wider partnership of state education agencies that make up a key set of stakeholders in the effort.

Denver Public Schools plays a special role as a “pilot district” within the project. As a pilot district, we have continued our commitment to engaging Denver teachers as co-designers, as leaders of professional learning, and as people who test out the materials in their classrooms. One of us (Douglas), has been a lead writer for biology units. 

The need for the OpenSciEd project derives from the fact that even 10 years after the NGSS were developed, there remain precious few instructional materials aligned to the standards, particularly for high school. And while iHub provided aligned materials in biology and chemistry, our materials did not integrate the Earth and space science standards, and we had no course in physics. OpenSciEd provided us with an opportunity as a partnership to continue progressing toward our aim of designing materials for the full complement of the NGSS by revising and extending our materials, to make them better and more comprehensive. In addition, the specifications provided to us by OpenSciEd would demand that we “up our game” with respect to supporting students’ sensemaking and promoting equity in ways we were eager to pursue.

Unlike our typical process for designing projects, OpenSciEd’s request for proposals specified a clear set of development tasks; at the same time, we were given some latitude about what kinds of questions we wanted to answer from the field test. Here, we decided together to build on instruments and approaches that we had been testing and refining for years in iHub. A major focus of our efforts over the years has been to gather data from students on their perceptions of their science learning, with an eye toward promoting epistemic justice, that is, ensuring that students from groups and communities owed an education debt by our society have opportunities to express their thinking and have their ideas respected and taken up in knowledge-building activities. Some of the approaches to data collection and instruments were also adapted from the OpenSciEd middle school field test, and iHub researchers had been involved in that effort, as well.

WHAT THE WORK EXAMINES

Here, we focus on our RPP’s work of field testing the materials we developed. The basic research questions we are asking in our field tests are: What are students’ experiences of the curriculum, and how do they vary by race/ethnicity and by gender? How do teachers judge and experience the materials? What do they learn from professional development? Can students learn from the materials?

In the field test, the team developed a nimble approach grounded in the principles of improvement science (Bryk et al., 2015; Edelson et al., 2021). A challenge we had to address was the need to gather data and produce reports every three months for each unit development and testing cycle that could inform key stakeholders, including the writers of the units, which included university-based and district-based educators. Each cycle involved testing a single unit in biology, chemistry, and physics, along with associated professional learning materials for each. That required us to streamline data collection to focus on teacher surveys and interviews after professional learning workshops and after teaching units, student exit tickets, and a student survey. We added a pre-post data collection to study student learning for two of the units in each course, but this data collection was distinct and required us to spend more than a year co-designing the assessments with writers. We also received feedback from an external review process led by WestEd using a rubric to evaluate alignment of units to the NGSS.

In this project, Denver educators and leaders were part of the writing and testing process, but there were other practitioners involved in all aspects of the work. A consortium of state science leaders from ten states were partners to the larger initiative, and they provided input on the materials themselves as well as the field testing research. With respect to the research, they had opportunities to make sense of data from each cycle of the field test, pose questions of researchers, and ask researchers to pursue additional questions to address puzzles in the data. Those opportunities for sensemaking took place at monthly Zoom meetings where Developers Consortium members were present, as were state leaders and staff from OpenSciEd. Also, interested educators who were part of the field test from those 10 states participated in co-revision workshops for the units that they field tested. In co-revision meetings with writers, educators engaged with feedback from other teachers and reflected on their own experiences to make improvements to the units.

FINDINGS

Over 300 high school teachers (reaching over 15,000 students) field tested biology, chemistry, or physics units designed to  support all students in meeting the vision for science learning described in A Framework for K-12 Science Education (NRC, 2012) and the Next Generation Science Standards (NGSS; NGSS Lead States, 2013). Each teacher received up to six different intensive professional learning workshops to prepare them to use the materials to support equitable learning in science.

There are many dimensions to the findings, but here we focus on those connected to teacher professional learning. It is important to note that those involved in writing materials were not responsible for collecting data related to teachers’ experiences; instead, a different partner in the Consortium, the Charles A. Dana Center at the University of Texas Austin, has led the data collection and analysis for the field test. 

From the standpoint of teachers, the professional learning (PL) prepared them well for several aspects of teaching OpenSciEd units. From self-report surveys, 81 percent of teachers said the PL had prepared them to support students in using science and engineering practices to figure out pieces of disciplinary core ideas. This is an important finding, since a major shift in the standards calls for students to develop proficiency in practices in ways that are integrated with understanding of disciplinary core ideas and crosscutting concepts. Seventy-nine percent said the PL prepared them to have students pose questions to drive the learning of the unit, while 77 percent said it prepared them to push students deeper to revise explanations of phenomena. Further, 74 percent said the PL prepared them to motivate the next step in investigating the anchoring phenomenon in the unit. Last, 72 percent said it prepared them to help students put pieces together of knowledge learned across the unit. These three sets of numbers tell us that teachers became more comfortable with key elements of the storyline instructional model, a finding that resonates with our findings from middle school (see Penuel et al., 2023).

Somewhat more challenging was convincing teachers that the units met their standards better than other materials did. Just 64 percent said this was true, in their estimation. State leaders helped us make sense of these findings, based on their own experience in working with teachers in their states. They said that teachers were likely reporting this to be true in part because teachers’ understanding of the standards themselves remains limited. The research team members also suspect many hold a view of the standards as primarily focused on disciplinary core ideas, based on their comments and responses to follow up interview questions. When state leaders developed these conjectures, members of the field test research team followed up to ask more about how materials fell short, and the data they gathered confirmed some of our own conjectures. 

A second challenge and surprise that remained was helping teachers to know how to link phenomena being studied to students’ own interests and experiences. Seventy-two percent of teachers said the PL helped them to do that. Many remained concerned about interest dropping off toward the end of units, something they call “phenomenon fatigue.” As we revised materials for both units and PL, we have been highlighting strategies for helping build relevance in materials for students.

Findings are still being developed for the overall project, and we do not yet know what effects the materials have had on student learning.

WHAT THE PARTNERSHIP LEARNED

This is the first project of our partnership that is intentionally both inward and outward facing. That is, it is intended to continue a line of capacity-building and improvement work that directly supports Denver Public Schools, while also making materials that can be used everywhere. DPS teachers have grown and stretched their skills as writers and leaders of professional learning in ways that would not have been possible without this opportunity. And the instructional materials we have developed have greatly improved, in terms of their quality. We have learned that we can do this kind of work together, but it does require us to take extra care to make sure that DPS voice is a prominent one in a wider effort that has many different stakeholders. DPS district leaders have learned how valuable the co-design process is in scaling reform ideas (curriculum) in a large district. The co-designing teachers serve as highly respected voices, through PL facilitation, aiding in successfully growing the number of piloting teachers participating in the project each year. Initially there were 6 teachers involved in the co-design/piloting process and now there are 25 teachers piloting.

IMPACT AND USE OF THE WORK

This is a project where research findings are less likely to be taken up than are the instructional materials themselves. Of course, these materials are grounded in research on how people learn and reflect a principled approach to curriculum design grounded in what we know about how best to promote meaningful engagement in science and engineering practices to explain phenomena and solve problems.

So far, two units from each course have been publicly released and are available for use by any educator, free of cost. In addition, materials for professional development are freely available at the OpenSciEd web site. All of the units undergo external review by the NextGen Science group at WestEd. All six of the released units have earned high ratings for quality by this group.

It is too soon to know how findings from the field test have impacted policy and practice more broadly, but the field test data, input from state leaders, and teacher feedback as part of co-revision meetings have all been used extensively to support improvements to the units themselves.

OPEN QUESTIONS AND NEXT STEPS 

We have many open questions, particularly related to professional learning and student learning. The OpenSciEd materials demand major shifts to most teachers’ instruction. To support these shifts, OpenSciEd provides extensive professional development – a multi-day “launch” followed by two-day workshops for every unit in every course. The conditions under which districts can adopt this approach are not yet known. We also need to design a study to test the impacts of the materials on student learning. Our assessments are a first step toward doing so, but we still need to analyze additional validity information about their instructional sensitivity.

As materials are released, we will be continuing to support DPS teachers in implementing them. In addition, we will be developing joint research projects to address questions of mutual interest regarding their use.

This article was co-written by members of the inquiryHub (iHub) research-practice partnership: Douglas Watkins is Manager of K-12 Science Curriculum & Instruction at Denver Public Schools and Bill Penuel is Director of iHub University of Colorado Boulder, Professor of Educational Psychology and Learning Sciences in the School of Education at the University of Colorado, Boulder, PI at the National Center for Research in Policy & Practice (NCRPP), Co-PI of the Research+Practice Collaboratory, and a contributing author to the LearnDBIR website.

REFERENCES

Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America’s schools can get better at getting better. Harvard University Press. 

Edelson, D. C., Reiser, B. J., McNeill, K. L., Mohan, A., Novak, M., Mohan, L., Affolter, R., McGill, T. A. W., Bracey, Z. B., Noll, J. D., Kowalski, S. M., Novak, M., Lo, A. S., Landel, C., Krumm, A., Penuel, W. R., Van Horne, K., González-Howard, M., & Suárez, E. (2021). Developing research-based instructional materials to support large-scale transformation of science teaching and learning: The approach of the OpenSciEd middle school program. Journal of Science Teacher Education, 32(7), 780-804. https://doi.org/10.1080/1046560X.2021.1877457   

Hopkins, M., Spillane, J. P., Jakopovic, P., & Heaton, R. M. (2013). Infrastructure redesign and instructional reform in mathematics. Elementary School Journal, 114(2), 200-224. https://doi.org/https://doi.org/10.1086/671935  

Ikemoto, G. S., & Honig, M. I. (2010). Tools to deepen practitioners’ engagement with research: The case of the Institute for Learning. In C. E. Coburn & M. I. Honig (Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 93-108). Rowman & Littlefield. 

Krajcik, J., Schneider, B., Miller, E. A., Chen, I.-C., Bradford, L., Baker, Q., Bartz, K., Miller, C., Li, T., Codere, S., & Peek-Brown, D. (2022). Assessing the effect of project-based learning on science learning in elementary schools. American Educational Research Journal, 60(1), 70-102. https://doi.org/10.3102/00028312221129247    

National Research Council. (2012). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. National Research Council. https://doi.org/10.17226/13165  

Penuel, W. R. (2019a). Co-Design as infrastructuring with attention to power: Building collective capacity for equitable teaching and learning through Design-Based Implementation Research. In J. M. Pieters, J. M. Voogt, & N. N. P. Roblin (Eds.), Collaborative curriculum design for sustainable innovation and teacher learning (pp. 387-401). Springer. https://doi.org/10.1007/978-3-030-20062-6_21  

Penuel, W. R. (2019b). Infrastructuring as a practice of design-based research for supporting and studying equitable implementation and sustainability of innovations. Journal of the Learning Sciences, 28(4-5), 659-677. https://doi.org/10508406.2018.1552151  

Penuel, W. R., Turner, M. L., Jacobs, J. K., Van Horne, K., & Sumner, T. (2019). Developing tasks to assess phenomenon-based science learning: Challenges and lessons learned from building proximal transfer tasks. Science Education, 103(6), 1367-1395. https://doi.org/10.1002/sce.21544  

Penuel, W. R., Allen, A.-R., Henson, K., Campanella, M., Patton, R., Rademaker, K., Reed, W., Watkins, D. A., Wingert, K., Reiser, B. J., & Zivic, A. (2022). Learning practical design knowledge through co-designing storyline science curriculum units. Cognition and Instruction, 40(1), 148-170. https://doi.org/10.1080/07370008.2021.2010207  

Penuel, W. R., Allen, C. D., Manz, E., & Heredia, S. C. (2022). Design-Based Implementation Research as an approach to studying teacher learning in research-practice partnerships focused on equity. In A. C. Superfine, S. Goldman, & M.-L. M. Ko (Eds.), Teacher learning in changing contexts: Perspectives from the learning sciences (pp. 217-237). Routledge. https://doi.org/10.4324/9781003097112-15  

Penuel, W. R., Allen, A.-R., Deverel-Rico, C., Singleton, C., & Pazera, C. (2023). How teachers’ knowledge of curriculum supports partnering with students in their science learning. Journal of Science Teacher Education. https://doi.org/10.1080/1046560X.2023.2167508  

Pipek, V., & Wulf, V. (2009). Infrastructuring: Toward an integrated perspective on the design and use of information technology. Journal of the Association for Information Systems, 10(5), 447-473. https://doi.org/10.17705/1jais.00195  

Reiser, B. J., Novak, M., McGill, T. A. W., & Penuel, W. R. (2021). Storyline units: An instructional model to support coherence from the students’ perspective. Journal of Science Teacher Education, 32(7), 805-829. https://doi.org/10.1080/1046560X.2021.1884784

Schneider, B., Krajcik, J., Lavonen, J., Salmela-Aro, K., Klager, C., Bradford, L., Chen, I.-C., Baker, Q., Touitou, I., Peek-Brown, D., Dezendorf, R. M., Maestrales, S., & Bartz, K. (2022). Improving science achievement—Is it possible? Evaluating the efficacy of a high school chemistry and physics project-based learning intervention. Educational Researcher, 51(2), 109-121. https://doi.org/10.3102/0013189X211067742  

Star, S. L., & Ruhleder, K. (1996). Steps toward an ecology of infrastructure: Design and access for large information spaces. Information Systems Research, 7(1), 111-134. https://doi.org/10.1287/isre.7.1.111  

Vakil, S., de Royston, M. M., Nasir, N. i. S., & Kirshner, B. (2016). Rethinking race and power in design-based research: Reflections from the field. Cognition and Instruction, 34(3), 194-209. https://doi.org/10.1080/07370008.2016.1169817

Suggested citation: Watkins, D., & Penuel, B. Learning to Co-Design Instructional Materials for Scale and Spread. NNERPP Extra, 5(3), 2-10.