University of Southern California Dissertations and Theses

Teacher curriculum supplementation as phenomenon and process

(USC Thesis Other)

Teacher curriculum supplementation as phenomenon and process

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content i

TEACHER CURRICULUM SUPPLEMENTATION AS PHENOMENON AND PROCESS

by

DANIEL MARK SILVER

A Dissertation Presented to the
FACULTY OF THE USC GRADUATE SCHOOL
UNIVERSITY OF SOUTHERN CALIFORNIA
In Partial Fulfillment of the
Requirements for the Degree
DOCTOR OF PHILOSOPHY
(URBAN EDUCATION POLICY)

December 2022

Copyright 2022 Daniel Mark Silver

ii

Acknowledgements
Every well-worn expression about taking villages and getting by with help from
friends applies here. If I were to individually thank every person whose kindness, patience,
thoughtfulness, feedback, friendship, mentorship, and love shaped this project in some
way, I would be writing for another week or two and you would have to flip past a much-
longer-than-anticipated acknowledgements section on your way to the dissertation. Still, I
want to express gratitude to a few people who were especially important to this project
and to my development into the person who put it together.
Nikki Fiske was my first teacher to insist that I was a writer, and (despite my third-
grade protests) pushed me onto the path where I would develop my written voice. Laura
Marion was my first teacher who refused to accept adequate-but-lazy work from me, and
(despite my tenth-grade protests) cemented my identity as a clear and purposeful thinker
and writer. My high school and college coaches Rick Caragher and Mitch Baker taught me to
pare a task down to its bones, find what matters, and relentlessly focus on that, discomfort
or no, until the task is done. Susan Engel saw me as a researcher before I did, and I will be
forever grateful to her for plucking me out of a 200-level college Psychology of Education
course, offering to advise me on a senior thesis, and being my first model of what calm,
unwavering, trusting academic mentorship looks like.
If those teachers prepared me to complete a dissertation, these were my reason to.
Dana Ingall, Nola Kosowsky, and Kat Needham were daily examples of the relentless,
selfless, and often thankless work in schools striving to truly serve students. You are
incredible educators and mentors, and any successes I had as a teacher I owe to you. I keep
old class pictures of my third graders on my desk, and when work starts to feel like a grind,
iii

I take thirty seconds to remember something brilliant they said or did, and suddenly any
grind is completely worth it. I’m proud and grateful to have been a teacher to each of them,
and I hope my work one day contributes to a stronger education system for them.
My close friends shaped this project more than they can know. I appreciate Jeffrey
Hemphill for the free-wheeling education discussion over our trial of miles, our miles of
trials, that helped me think about close to everything more systematically than before.
Chris Welch and Katerina Belkin taught me how to think compassionately and act with
others in mind, and I strive to live up to their examples daily. I always leave a conversation
with Imogen Herrick thinking a little more deeply and a little more empathetically about a
tricky research or life problem.
I would not be the researcher I am without my colleagues Amie Rapaport, Marshall
Garland, Clare Baek, Michael Feinberg, and Isabel Clay. I have benefitted immensely from
your time, expertise, humor, and friendship.
I want to acknowledge especially the mentorship of Shira Haderlein, Anna Saavedra,
and Morgan Polikoff. Shira, you were my Ph.D. big sis, and without your generosity with
your time and expertise, I would not have gotten far. (Realistically, I’d still be figuring out
Stata.) Anna, your warm, precise mentorship and your genuine care is one of the main
reasons I continued in the Ph.D. program when it didn’t feel worth continuing a few years
ago. And Morgan, you are a consummate leader-by-example, and you will probably be the
model of professional mentorship, clear and respectful communication, timeliness,
purposeful writing, generosity, drive for useful solutions, and more that I strive to emulate
for the rest of my career. I feel so lucky to know each of you.
iv

It is important to mention those without whom this dissertation truly could not
have happened—the researchers who shared unpublished work to include in the review in
Paper 1, the elementary teachers who took the survey in Paper 2 and especially those who
participated in cognitive interviews and in the follow-up observation and interview in
Paper 3, some even during their summer vacation! Many mentioned a desire to inform
relevant research as practitioners, and I hope this dissertation honors that desire. I also
gratefully acknowledge a Rossier Internal Research Grant as well as years of rolled-over
Rossier Research and Travel Grants for making this project possible.
Thank you as well to the other members of my dissertation committee, Drs. Julie
Marsh and Stephen Aguilar. You are model scholars, and your teaching, mentorship, and
feedback on dissertation proposals improved the project and made me a better researcher.
Finally, thank you to my family. My grandparents Tana and Allen Warren were
pioneers in the supplemental teaching materials industry, and growing up in a family with
ties to teacher curriculum supplementation undoubtedly shaped my research interests. My
grandparents Irene and Jerry Silver are shining examples of how to stay grounded and
prioritize loved ones while remaining inquisitive and engaged in the world. My mom
Tamara taught me that you can make progress on any problem by asking the right
questions, and my dad Michael gave me a model for pursuing answers to those questions
with humility and hard work. My brother Jeffrey is a true friend who never fails to make me
smile. And last but not least, Kieres, thank you for putting up with late nights, lots of
keyboard tapping, and pointed questions and discussion over the course of this project.
Thank you even more for being you—steady, funny, selfless, driven, kind, and the best
partner a person could ask for. I love and appreciate you all.
v

Table of Contents

Acknowledgements ......................................................................................................................... ii
Table of Contents ............................................................................................................................ v
List of Tables ................................................................................................................................. vi
List of Figures ............................................................................................................................... vii
Abstract ........................................................................................................................................ viii
Introduction ..................................................................................................................................... 1
Paper 1: A Theoretical Framework for Studying Teachers’ Curriculum Supplementation ........... 5
Method ........................................................................................................................................ 7
Systematic Review .................................................................................................................... 10
The Teacher Curriculum Supplementation Framework ........................................................... 33
Conclusion ................................................................................................................................ 45
Paper 2: A Survey Tool for Measuring Teachers’ Curriculum Supplementation ........................ 47
Literature Review...................................................................................................................... 49
Sample....................................................................................................................................... 58
Method ...................................................................................................................................... 61
Results ....................................................................................................................................... 66
Limitations and Future Directions ............................................................................................ 95
Discussion and Conclusion ....................................................................................................... 97
Paper 3: Observing Teachers’ Curriculum Supplementation ....................................................... 99
Literature Review.................................................................................................................... 101
Sample..................................................................................................................................... 108
Method .................................................................................................................................... 112
Results ..................................................................................................................................... 121
Limitations and Future Directions .......................................................................................... 144
Discussion and Conclusion ..................................................................................................... 146
References ................................................................................................................................... 149
Appendix A. Survey Item-to-Construct Crosswalk .................................................................... 173
Appendix B. Cognitive Interview Protocol ................................................................................ 179
Appendix C. Full Survey Text with Changes Noted .................................................................. 180
Appendix D. Exploratory Factor Analysis Results ..................................................................... 191
Appendix E. Recruitment Email and Observation/Interview Protocol ....................................... 194
Appendix F. Coding Frames ....................................................................................................... 199

vi

List of Tables

Table 1. Findings from the Supplementation-relevant Literature ................................................. 11
Table 2. Comparing Analytic Sample to U.S. Public School Elementary Teacher Population ... 60
Table 3. Common Characteristics of the TCSF’s Four Main Dimensions of Supplementation ... 71
Table 4. Interpretation of Factors as Subscales ............................................................................ 72
Table 5. Relationships between Survey Subscales and Theoretically Distinct Constructs .......... 78
Table 6. Relationships Among Survey Subscales......................................................................... 84
Table 7. Participant Characteristics ............................................................................................ 110
Table 8. Skills in Supplementation Simulation Prompts, by Grade Level ................................. 114
Table 9. Selected Statements about the Credibility and Authenticity of the Supplementation
Simulation ................................................................................................................................... 122
Table 10. High-Level Topic Coverage during Observations and Interviews ............................. 123
Table 11. Supplement Feature Topic Coverage .......................................................................... 124
Table 12. Illustrative Statements about Supplement Features .................................................... 125
Table 13. Reasons for Supplementation Topic Coverage ........................................................... 127
Table 14. Supplement Source Topic Coverage ........................................................................... 129
Table 15. Supplement Use Pattern Topic Coverage ................................................................... 131
Table 16. Participant 9’s Most Common Codes ......................................................................... 134
Table 17. Approaches to Self-Created Materials, by Gender ..................................................... 141
Table 18. Selected Discussion of the Connection between Teaching Experience and Curriculum
Supplementation ......................................................................................................................... 142

Table A1. Survey Items, Target Constructs, and Subscales ....................................................... 173

Table D1. Supplement Source Factor Loadings ......................................................................... 191
Table D2. Supplement Source Rotated Factors .......................................................................... 191
Table D3. Supplement Reason Factor Loadings ......................................................................... 192
Table D4. Supplement Reason Rotated Factors ......................................................................... 192
Table D5. Supplement Feature Rotated Factors ......................................................................... 193
Table D6. Supplement Feature Factor Loadings ........................................................................ 193

vii

List of Figures

Figure 1. The Teacher Curriculum Supplementation Framework ................................................ 35
Figure 2. Supplementation Subscales ........................................................................................... 70
Figure 3. Relationships between Supplement Subscales and Commitment to Teaching Scale .... 78
Figure 4. Relationships between Supplement Subscales and Commitment to Mathematics
Education Reform Scale ............................................................................................................... 79
Figure 5. Relationships between Supplement Subscales and Orientation toward Official Material
Scale .............................................................................................................................................. 81
Figure 6. Relationships between Source Subscales and Reason, Use Pattern, and Feature
Subscales ....................................................................................................................................... 85
Figure 7. Relationships between Reason Subscales and Use Pattern and Feature Subscales ....... 85
Figure 8. Relationships between Use Pattern Subscales and Feature Subscales .......................... 86
Figure 9. How Often Do You Use Each of the Following Sources to Find Supplemental
Materials? ...................................................................................................................................... 88
Figure 10. To What Extent Are Each of the Following Reasons a Factor When You Decide to
Use a Supplemental Curriculum Material? ................................................................................... 89
Figure 11. To What Extent Are Each of the Following Features an Important Consideration
When You Look for or Create a Supplemental Material? ............................................................ 90
Figure 12. Use Patterns ................................................................................................................. 91
Figure 13. Intersecting Sets of Supplementation Dimensions and Their Characteristics ............. 93

viii

Abstract

It is reasonable to imagine that teachers have supplemented their officially-adopted
textbooks with supplemental materials for as long as schools and districts have provided
materials to their teachers. National surveys have shown teacher curriculum
supplementation to be widely-practiced across U.S. classrooms, but despite a robust
research literature on official textbooks, teachers’ supplementation practices have only
become a focus of empirical work in the last decade or less. Perhaps because it is a
relatively new area of research, teacher curriculum supplementation is often conceived in
disparate ways that do not always translate well across research studies, and there are still
many open questions about the process of teacher curriculum supplementation, in practice.
This dissertation contains three papers designed to deepen our understanding of teacher
curriculum supplementation. In the first paper, I systematically review existing
supplementation-relevant literature, highlight seven areas of consensus, and offer the
Teacher Curriculum Supplementation Framework (TCSF) as a theoretical framework for
studying this important practice. In the second paper, I design a survey measure of
supplementation grounded in the TCSF, then collect and analyze evidence about the tool’s
ability to provide valid inferences about the practice. In the third, my coauthor and I
conduct observations of in-service teachers during a simulated supplementation activity,
then interview them about their practice, yielding insights about teachers’ in-the-moment
tendencies and considerations while they supplement their official curriculum materials. I
believe teacher curriculum supplementation is a lever that can be—and often already is—
harnessed for instructional improvement, and this dissertation provides a conceptual
grounding for future study of the practice alongside new evidence about its nature.
1

Introduction
Textbooks have been fixtures of U.S. classrooms for nearly two centuries, in wide use
even before the proliferation of compulsory schooling laws in many states (Cuban, 1984; Tyack,
1976). Teachers in these earliest days often assigned textbook passages for students to memorize,
recite, and write about, “designed to compel students to assimilate knowledge and practice skills
in a particular fashion. It was a fashion dictated by the textbooks usually—and often with
dogmatic determination” (Finkelstein, 1970, p. 23). This heavy reliance on the textbook is
understandable—teachers were commonly expected to teach all subjects, often at multiple grade
levels and with little more than a middle school education themselves (Foster, 1999).
But as early as the turn of the twentieth century, we also have evidence of teachers
choosing to supplement their official textbooks with outside resources: in teaching U.S. history
to his sixth grade students in 1900, William Chatfield wrote that in addition to assigning “lessons
from the text,” “Maps and pictures are freely used to illustrate the work, the former being drawn
upon the blackboard and copied by the pupil…The pictures are gathered from many sources and
are distributed to the class.” (Cuban, 1984, p. 30).
It is reasonable to imagine that teachers have used supplemental materials alongside their
textbooks for as long as textbooks have existed, although teachers may have received mixed
messages as to the value of supplementation, at least over recent decades. For example,
university-based teacher training programs in the 1980’s tended to look down on adherence to
textbook lessons as uncreative (Ball & Feiman-Nemser, 1988), but federal and state education
policy in the 2000’s privileged fidelity to a high-quality textbook as a core tenet of effective
teaching (Superfine et al., 2015). Today, websites dedicated to supplemental curriculum
materials are decried by some as “the fast food of education” (Bennet, 2019) and heralded by
2

others as a “smash success” (Jones, 2018). Regardless of messaging, K-12 instruction in the U.S.
has commonly included teacher supplementation of officially-adopted curriculum materials.
Despite a robust research literature on instruction and curriculum dating back well over
fifty years, this literature only rarely engaged with teacher curriculum supplementation prior to
the last decade or so. When it did, supplementation was often treated as something teachers
surely did, but not something worthy of rigorous empirical study. For example, in Larry Cuban’s
seminal 1984 book How Teachers Taught: Constancy and Change in American Classrooms
1890-1980, supplemental curriculum materials are mentioned only once in over 300 pages, in
recounting that despite their aspirations of hyper-individualized instruction, the “primary tools of
instruction” in many 1960’s open classrooms were simply “the textbook followed by workbooks
and supplementary readings” (Cuban, 1984, p.209). The relative absence of supplemental
materials from the empirical literature on curriculum and instruction is surprising, given
consistent findings that curriculum materials can influence how teachers teach (Davis & Krajcik,
2005; Remillard, 2005) and what students learn (Agodini et al., 2010; Bhatt & Koedel, 2012;
Jackson & Makarin, 2018; Koedel et al., 2017).
Two events in the last fifteen years have raised the profile of teacher curriculum
supplementation enough that it has become not only a popularly-discussed topic in the media
(e.g., Marcin, 2020; Shelton et al., 2020) but also has started appearing more regularly in
empirical study of teaching and curriculum. The first event was the adoption of the Common
Core State Standards by 51 U.S. states and territories in 2010-2013, aligning assessments and
standards of grade-level mastery for a majority of students for the first time in U.S. history
(Greene, 2016; Webster & Thatcher, 2014).
3

The second was the proliferation of online virtual resource pools in the early- and mid-
2010’s (Cummings, 2015). Virtual resource pools are websites that host large numbers of
supplemental instructional materials, downloadable for free or for a small fee. These materials
are typically created and downloaded for use by practicing teachers, as opposed to being created
by professional curriculum writers and adopted by school or district administrators (see Hu &
Torphy, 2020, for a larger discussion of the term “virtual resource pool” and a more formal
definition). Although virtual resource pools had existed for years before widespread adoption of
the Common Core Standards (the largest, Teachers Pay Teachers, was founded in 2006), it is
likely that the adoption of the Common Core Standards broadened the market for any given
supplemental material many times over, since savvy supplement creators could market their
“Common Core-aligned” materials to teachers in any of 40-plus states who were desperate for
guidance around enacting the new set of standards.
In the years following these events, empirical work began to address questions about the
nature and quality of supplemental curriculum materials (e.g., Polikoff & Dean, 2019), the
supplemental curriculum market (e.g., Shelton et al., 2021), teachers’ reasons for engaging in
these markets (e.g., Shapiro et al., 2019), and the identities and motivations of supplement
creators (e.g., Shelton & Archambault, 2019). This literature tends to address questions about
who supplements, why they supplement, and what supplements are like, but there is little
common conceptualization of supplementation as a phenomenon across studies. In addition,
relatively little work has focused on how teacher curriculum supplementation occurs as a
process, including questions about how exactly and to what extent teachers supplement and what
factors teachers consider as they supplement. Answers to questions like these might provide
guidance for policies and professional learning aimed at helping teachers supplement more
4

effectively or at to helping district and state officials select official curriculum materials more in
line with teachers’ expressed needs.
This dissertation consists of three papers aimed at deepening our understanding of teacher
curriculum supplementation as both a phenomenon and a process. In the first paper, I
systematically review existing supplementation-relevant literature, highlight key areas of
consensus, and propose a novel framework for researchers to use as a conceptual starting point in
future studies of teacher curriculum supplementation. In the second, I use that framework to
develop and provide validity evidence for a survey tool designed to measure various dimensions
of teacher curriculum supplementation. In the third, my coauthor and I observe the
supplementation process, carrying out a qualitative content analysis of simulated lesson planning
sessions during which teachers engage in supplementation. Taken together, I hope these papers
contribute theory-enriching and policy-relevant insights into the important and understudied
process of teacher curriculum supplementation.

5

Paper 1: A Theoretical Framework for Studying Teachers’ Curriculum Supplementation
The relationship between teacher and curriculum over the last decades has been anything
but consistent. Before the ubiquity of the internet, nearly all schools issued their teachers
textbooks, but many traditional, university-based teacher training programs taught that great
teachers create their own lessons without using the textbook (Ball & Cohen, 1996; Ball &
Feiman-Nemser, 1988). The standards movement, prominent in federal education policy since at
least 2001’s No Child Left Behind Act, centered fidelity to a high-quality textbook as the mark
of an effective teacher (Superfine, Marshall, & Kelso, 2015). But the explosion of the internet
over the last two decades and the standardization brought on by school-level implementation of
the Common Core Standards in the mid-2010’s has allowed for the rise of “virtual resource
pools,” which have again transformed how teachers use curriculum. Virtual resource pools are
websites that host curriculum resources for download by teachers to use either alongside or
instead of their school-adopted textbooks, often for a small fee and often uploaded by practicing
educators rather than traditional curriculum developers (Hu & Torphy, 2020). While already
popular, moves to remote schooling in light of the COVID-19 pandemic led to another period of
wild growth for such websites, with some reporting revenue increases of up to 20% during this
period (Marcin, 2020).
The largest virtual resource pool, Teachers Pay Teachers (TPT), has been praised for
“allowing dedicated teachers to earn additional money for their hard work” (Boitnott, 2015),
with the platform’s top 14 sellers having earned over $1 million each as of 2015. But the site’s
open-upload policy has led many to view it as “the fast food of education,” rife with attractive
but low-quality resources (Bennett, 2019; Chapman, Wright, & Pascoe, 2019), and some have
pointed out the prevalence of harmful materials that incorporate racist, sexist, or xenophobic
6

tropes (PBS News Hour, 2018; Shelton, Archambault, & Harris, 2020). Despite these problems,
TPT claims that more than two-thirds of U.S. teachers have used its materials, a statement
supported by independent, nationally-representative teacher surveys (e.g., Opfer, Kaufman, &
Thompson, 2017).
Although supplementation is not a new phenomenon, the rise of virtual resource pools
with millions of inexpensive, ready-made, always-available materials ranging in scope from
single activities to full lesson and unit plans has contributed to sweeping “changes to the
organizational ecology of curriculum materials” (Hodge, Salloum, & Benko, 2019, p. 426).
These changes gave rise to large increases in teachers’ use of virtual resource pools, ushering in
an era of easy access to supplemental materials that persists to this day (Kaufman, Opfer,
Bongard, & Pane, 2018). Commensurate with the rise of virtual resource pools, there has been a
flurry of supplementation-relevant empirical research over the last half-decade. However, this
work tends to center phenomena closely related to supplementation (e.g., virtual resource use,
teacher social media use) rather than supplementation itself, and thus conceives of teachers’
curriculum supplementation in ways that do not always translate well across studies. The result is
a somewhat fragmented literature of teacher curriculum supplementation. This paper is an
attempt to give researchers who study teacher curriculum supplementation a common language
by highlighting key areas of consensus in the body of recent supplementation-relevant literature
and by providing a framework following from these findings for use in future empirical work.
Specifically, I address the following questions:
1. What does the body of recent literature relevant to teacher curriculum
supplementation suggest about the prevalence and quality of materials from virtual
resource pools and the characteristics of their authors?
7

2. What does this literature suggest about teachers’ decision-making around whether to
supplement, where to seek out supplemental curriculum materials, and how to enact
those resources?
3. How might researchers conceive of teacher curriculum supplementation in future
empirical studies, given these findings from the literature?
Teacher curriculum supplementation is best understood as: any premeditated, additive
change that a teacher makes to their official curriculum materials. Because elements of this
definition follow from themes that emerge in the literature, I discuss this definition in more detail
after the systematic literature review. The rest of the paper is as follows. I begin by relating the
methods I used to conduct the systematic review, then address the first question by summarizing
what the current literature base tells us about supplemental materials themselves (findings 1, 2,
and 7 in the systematic review). I then address the second question by summarizing what the
literature tells us about teachers’ decision making related to supplementation (findings 3, 4, 5,
and 6 in the systematic review). To address the third question, I then present the Teacher
Curriculum Supplementation Framework, a tool to help researchers conceptualize and study
teacher curriculum supplementation. I aim for the review to be a useful reference and for the
framework to be a powerful tool for future researchers of curriculum supplementation looking to
build a more coherent knowledge base around this important phenomenon.
Method
The ubiquity of virtual resource pools can be linked to Common Core-driven
standardization and the school-level rollout of Common Core assessments, which created a
single set of standards in common across 42 states and created a new market for “Common Core-
aligned” supplemental materials, especially in the window before schools were able to purchase
8

and adopt fully-aligned textbooks (Greene, 2016). This school-level rollout began in most states
in approximately 2015, and analyses at the time found many supplemental curriculum providers
among the top overall providers of Common Core-aligned materials (Hodge, Salloum, & Benko,
2016; Smarter Balanced, 2020). For the purposes of the first two questions, then, I systematically
reviewed literature from this most recent era of curriculum supplementation, from 2015 to
December of 2020, when I ran my final systematic literature search. I began with searches
1
of
the reference databases Education Resources Information Center (ERIC) and PsycInfo, restricted
to 2015 and later. (In a subsequent “snowballing” search, described below, I relaxed this timing
restriction.) My search parameters were intentionally broad because supplementation is not yet a
well-bounded area of inquiry, but often appears in studies of curriculum enactment, curriculum
efficacy, standards implementation, and related areas. I designed my search to minimize the
likelihood that it would fail to find a supplementation-relevant article from the Common Core
era, but the wide net I cast ensured that my search would yield many irrelevant articles.
As of December 1, 2020, the search returned 646 hits in ERIC and 240 in PsycInfo (see
footnote 1 for search string). I then performed a relevance screening of the article abstracts
(Valentine, 2019) and retained those that were: (1) written in English, (2) carried out in a U.S.,
K-12 context, if empirical, (3) teacher-focused, and (4) relevant to curriculum use. The first
criterion is a necessity because English is the only language in which I am fluent. The second
criterion bounds inquiry to U.S., K-12 contexts because expectations around curriculum and
standardization may be quite different in other countries or for older or younger students. The
third criterion bounds this study to teacher curriculum supplementation as opposed to, for

1
The search string I used was: ab(teacher*) AND ab("k-12" OR "k12" OR "elementary" OR "middle" OR "high"
OR "primary" OR "secondary") AND (ab(supplement*) OR ab(textbook*) OR ab((instructional OR teaching OR
curriculum OR curricular) n1 material*)) OR pinterest OR "teachers pay teachers" OR (exten* n1 curricul*) OR
(supplement* n1 curricul*)
9

example, changes a school administrator makes to a district-adopted textbook. The fourth
criterion is necessary because of the broadness of my search term and mainly filters out studies
that are off-topic. (For example, my search would return a study of behavior management that
happened use the phrase “extended curriculum” anywhere in its text, which is not relevant to this
review.) At this early relevance screening stage, I erred on the side of retaining studies: any time
I was not sure whether a study met a particular criterion, I treated it as though it had. After
screening and unduplicating results returned in both the ERIC and PsycInfo searches, there were
64 articles retained in my collection of Common Core era, supplementation-relevant work.
Again, the fact that a high proportion of studies were screened out at this stage is due to the
broadness of my search string.
After this initial screening, I performed a “snowballing” search of the reference sections
of articles retained after screening (Greenhalgh & Peacock, 2005), this time relaxing the
restriction that articles be published in 2015 or later. This step reduced the likelihood that I
would omit an article missed in my initial search while ensuring that I would include in my
review the most-relevant earlier works that the body of Common Core era supplementation-
relevant literature draws on. After screening the abstracts of the snowballed articles for the same
four criteria I applied in the initial screening process, I was left with a collection of 124 articles.
To access key unpublished work, I contacted scholars whose research groups had published three
or more articles since 2015 with teacher curriculum supplementation as a substantive focus to
ask for any ongoing or unpublished projects to include in the review (Rothstein & Hopewell,
2019). This search yielded another fourteen articles (138 total), all of which were relevant to the
review. During my initial reads of the articles, I ended up excluding another 16 from the review
for failing to meet various screening criteria: 10 for not being in a U.S., K-12 context (most of
10

these were carried out in the UK), three for not being teacher-focused (one focused on student-
created materials, another was a general critique of the What Works Clearinghouse, and a third
was a set of guidelines for researchers), and three for not being relevant to curriculum use (one
was about teacher leadership and two were lab studies of lesson planning, divorced from
curriculum). This left a collection of 122 articles to review that either center supplementation in
the Common Core era, are relevant to supplementation in the Common Core era, or substantively
inform pieces in either former category.
Systematic Review
The goal of this review is to present a structured dive into the current state of
supplementation-relevant research, as a reference for researchers interested in this important
topic. To help this review stand on its own and build an understanding of the context within
which supplementation occurs, I begin with a brief, non-exhaustive summary of key studies of
official curriculum materials, nearly always textbooks. To address this paper’s first two
questions, I then structure the systematic review around seven key findings from the
supplementation-relevant literature, listed in Table 1. In discussing the literature that builds to
these seven findings, I strive to include and comment on all supplementation-relevant work from
2015 through the end of 2020. I also include studies from before this era where they provide
important findings or context. Finally, because supplementation-relevant work utilizes a wide
range of methodologies and is often exploratory in nature, I will make no attempt to
quantitatively aggregate study results.

11

Table 1. Findings from the Supplementation-relevant Literature
1 Teacher curriculum supplementation is massive in scope
2 Supplement creators are a relatively homogenous group
3 Teachers supplement to fill needs perceived in their contexts, their students, or themselves
4 The sources teachers use to access supplements may relate to their supplementation goals
5 Teachers have stable “default” supplement use patterns, shaped by personal characteristics
and context
6 Supplements’ specific features relate to whether and how teachers enact them
7 Supplements from virtual resource pools are generally low quality and sometimes pose
potential harm to users

Context: Textbook Adoption, Use, and Effectiveness
Textbook adoption is a process generally localized at the state or district level and has
been portrayed historically as inefficient, ineffective, and sometimes-corrupt (English, 1980; Farr
& Tulley, 1985; Follett, 1985; Fordham Institute, 2004). Newer studies suggest that the process
has become more teacher- and student-centered in recent years. For example, in interviewing a
stratified random sample of California district leaders, Polikoff and colleagues identified distinct
“winnowing” and “evaluating” phases in the adoption process for math materials (Polikoff, et al.,
2020). The California decision makers cited access to technology, perceived teacher buy-in,
student demographics, and textbook appearance as reasons to adopt a textbook, although
alignment to Common Core standards was rarely mentioned as a key consideration.
Although all researchers understand that teachers do not simply parrot a textbook to their
students verbatim, Ball and Cohen (1996) were some of the first to concretize a distinction
between the “intended” curriculum written in textbooks and the “enacted” curriculum actually
taught. Much empirical evidence lends credence to this distinction (e.g., Gay, Barry, Rothrock, &
Pelkey, 2020; Grossman & Thompson, 2008; Rowan, Camburn, & Correnti, 2004; Tarr, Chavez,
Reys, & Reys, 2006), and it is present in nearly all subsequent textbook-centric work. Later
research grounded in mathematics and science education refined these ideas by emphasizing that
the relationship between teacher and textbook is bidirectional, with curriculum materials
12

themselves often informing instruction in ways just as powerful as a teacher’s own contexts and
areas of expertise (e.g., Davis & Krajcik, 2005; Remillard, 2005). Under the conception that
curriculum materials often serve “educative” roles for teachers as well as students, the
importance of studying these materials, whether officially-adopted or supplemental, is clear.
Others have conceived of teachers’ curriculum enactment as occurring in discrete “attending,”
“interpreting,” and “responding” phases (Dietiker, Males, Amador, & Earnest, 2018).
While many researchers worked to address the question, “How do teachers use
textbooks?” another group asked directly, “How good are the textbooks teachers use?”
Generally, studies that compared textbooks to external measures of quality like learning
standards found the textbooks lacking (Polikoff, 2015). Studies that compared textbooks to one
another tended to find that they meaningfully varied in their quality as measured by student
outcomes, but often disagreed as to which textbooks were best. A randomized control trial found
that the most effective of four popular elementary math programs boosted student performance
on a standardized math test by roughly the amount that we might expect from a seven-student
reduction in class size (Agodini & Harris, 2010), although later quasi-experimental analyses that
included some of the same curricula often found that different textbooks were most effective
(Bhatt & Koedel, 2012; Bhatt, Koedel, & Lehmann, 2013; Koedel, Li, Polikoff, Hardaway, &
Wrabel, 2017). The studies claimed to have isolated causal textbook effects, although they
emphasized that they examined average effects (not accounting for professional development,
fidelity of implementation, etc.), which may not hold much practical use. Local averages may be
more applicable to a curriculum adopters’ decisions because adoption decisions are made for a
specific context.
13

A more recent comparative textbook study using a state-representative sample of schools
from each of six states found no student achievement differences by math textbook, in contrast to
earlier work (Blazar, et al., 2019). That research team reconciled their null finding with earlier
literature by pointing to a marked decrease in the centrality of textbook content over the last
decade. They noted that just 7% of teachers in their sample used their textbook exclusively while
26% of teachers used curriculum materials found online in more than half their lessons. Polikoff,
Campbell, and Korn (2018) also noted the diminishing role of the textbook in a recent review of
curriculum content and effects, where their first recommendation for future inquiry is an
increased focus on non-textbook curriculum materials. (Assertions about increasing or
diminishing importance of officially-adopted textbooks to classroom instruction are complicated,
however, by a relative dearth of literature centering the relationship between intended and
enacted curriculum prior to the last two decades or so.)
Empirical evidence suggests that the textbook’s importance to teachers’ enacted
curriculum may be diminishing, giving way in many cases to teacher-adopted supplemental
curriculum materials. The remainder of this review presents seven key findings from recent
supplementation-relevant literature.
Finding 1: Teacher Curriculum Supplementation is Massive in Scope
Prevalence Among Teachers
The internet’s affordances for curriculum materials have been long noted, with scholars
extolling the educational advantages of “resource-based learning environments” and open
education resources as alternatives to traditional textbooks that lend themselves naturally to
individual-level scaffolding (Hill & Hannafin, 2001; Hylen, 2015; Torphy, et al., 2018). Hodge,
Salloum, and Benko (2019) catalogued many of the transformations to the curriculum market
14

over the first decades of the twenty-first century. Traditionally, textbook publishers were
curriculum suppliers and schools and districts were curriculum consumers. The rise of
curriculum-rating websites like EdReports.org introduced non-supplier, non-consumer “demand
influencers” to the market, and the standardization brought on by Common Core’s adoption
encouraged some governments to sponsor curriculum writing directly, encroaching on for-profit
publishers’ traditional roles. Perhaps the greatest disruption to the market, though, came from
virtual resource pools, which allowed teachers to sell materials they had created directly to other
teachers. This blurred the line between curriculum consumer and curriculum producer, allowing
many teachers to occupy both roles. In light of these changes to the curriculum market,
researchers have noted that officially adopted math, science, and English Language Arts
curricula now have only a “very loose relationship with what U.S. teachers actually use in their
classrooms” (Kaufman, et al., 2020).
Teachers have increasingly become curriculum “curators” who must select high-quality
resources from the vast, unregulated ocean of available options (Hu & Torphy, 2020).
Researchers in the Common Core era have documented the pervasiveness of teachers’ curation
of supplemental resources, typically as part of larger curriculum-use studies. Immediately
following Common Core’s adoption, a survey of 403 middle school math teachers found that
over 60% reported regularly supplementing their instruction with materials downloaded from the
internet, in many cases because their school had not yet adapted Common Core-aligned
textbooks (Davis, Choppin, Roth McDuffie, & Drake, 2013). Just a few years later, a survey of
2267 mostly-elementary teachers and administrators found that 79% reported using either
supplements downloaded from the internet or self-created supplements in “some” of “most” of
their lessons each week (Perry, Marple, & Reade, 2017). Another survey of upper elementary
15

teachers found that 23% supplemented their math instruction with non-digital teacher-created
materials and 65% supplemented with digital materials (Polly, 2017). Although these studies
drew from non-representative samples, they showed how widespread supplementation was in
some teacher communities and generally aligned with studies using representative data.
Drawing on data from a nationally-representative teacher panel, Tosh, Doan, Woo, and
Henry (2020) found that over 85% of teachers used digital materials, just over half of which
were supplements, neither required nor recommended by the teacher’s school. This proportion
was higher for teachers of low-income students. Another nationally-representative study found
that the proportion of math teachers who regularly use the virtual resource pool Teachers Pay
Teachers (TPT) increased from 41% in 2015 to 60% in 2017. English teachers displayed a
similar increase, from 42% to 55% (Kaufman, et al., 2018). The proportion of regular Pinterest
users remained stable and Google use declined over the same period, perhaps indicating that
teachers are becoming more specialized in their searches for supplements. A number of other
studies with non-representative teacher samples have also found that between 70% and 90% of
teachers are regular users of virtual resource pools like TPT and Pinterest (Caniglia & Meadows,
2018; Carpenter, Morrison, Craft, & Lee, 2020; Hu, Torphy, Evert, & Lane, 2020; Hunter &
Hall, 2018; Marple, et al., 2017; Shapiro, Sawyer, Dick, & Wismer, 2019).
Taken together, these findings suggest that over two-thirds of U.S. K-12 teachers
currently supplement their curriculum materials using online sources, and that the proportion is
increasing. Although supplementation frequently occurs in non-virtual spaces, I know of no
recent work that focuses explicitly on teachers’ self-created or colleague-created curriculum
materials. Survey data indicates that teachers regularly supplement non-virtually (e.g., Polly,
2017; Opfer, et al., 2017), so this is an area ripe for future research.
16

Scope of Virtual Resource Pools
One study documented the scope of teacher curriculum supplementation differently,
using web-scraping methods on publicly-archived pages to study the structure of the TPT
marketplace (Koehler, Shelton, Carpenter, & Greenhalgh, 2020). Focusing on sales data, the
researchers found that 92% of TPT transactions were for $5 or less and 69% were downloads of
free resources. Despite this, since TPT began in 2006, it has hosted 1.5 billion transactions
totaling $3.9 billion. They reported a strikingly concentrated market, with the top 1% of stores by
sales dollars accounting for 81% of transaction dollars, $3.2 billion. Koehler and colleagues
observed that, at a glance, these super-seller TPT stores are overwhelmingly owned by “White,
attractive, female” teachers, so they wondered whether the full breadth of teaching experiences
and student needs are represented on TPT. A second study found that the 500 most popular U.S.
History resources had similar characteristics to TPT resources as a whole: the resources had an
average price of about $5, and the market was quite top-heavy, with just 13 sellers responsible
for half of the top 500 resources (Harris, Shelton, & Archambault, 2020).
Finding 2: Supplement Creators are a Relatively Homogenous Group
A growing body of work centers the supplement creators, the “teacherpreneurs” (Buckley
& Nzembayie, 2016). In line with the findings from Koehler and colleagues, a survey of about
400 top TPT sellers indicated that teacherpreneurs were mostly White (89%) female (92%)
teachers (72%) with ten or more years of teaching experience (63%) who had completed college
coursework in education (95%; Shelton & Archambault, 2019a). Although only 54% of surveyed
teacherpreneurs reported regularly testing their materials in their own classrooms, 85% reported
that their businesses enhanced their own teaching practices.
17

Surveys of the most successful TPT teacherpreneurs indicated that they treated their TPT
stores more like full businesses than hobbies and attributed their popularity equally to offering
quality products and to frequent self-promotion on social media (Sawyer, Dick, & Sutherland,
2020; Shelton & Archambault, 2020). These teacherpreneurs believed their customers were
looking for products that were easy to use (54% reported this) and standards-aligned (41%), and
which filled a specific curricular need (35%). Just 12% mentioned materials’ rigor or quality as
something their customers looked for in a material. In separate studies, teacherpreneurs reported
deriving satisfaction from positive customer feedback, friendly business competition with fellow
teacherpreneurs, and the additional income (Ingram, 2019; Shelton & Archambault, 2018).
It is important to note that this relatively homogenous picture of teacherpreneurs (white,
female, etc.) is simply what emerges from the existing empirical literature in this review. There
are undoubtedly teacherpreneurs from marginalized backgrounds who are finding success on
platforms like TPT, Instagram, or Twitter, but their experiences have not yet been centered in
empirical study of teacherpreneurs. This is a gap in the literature that should be addressed to
enrich our understanding of the producers of supplemental curriculum materials.
Finding 3: Teachers Supplement to Fill Needs Perceived in Their Contexts, Their Students,
or Themselves
Context-based Motivations for Supplementation
Nearly all supplementation-relevant research assumes that teachers supplement for
specific reasons, but a subset of this research enumerates those reasons in detail. Survey findings
have suggested that any new technology’s use in schools should be understood ecologically,
accounting for teachers’ subject and grade taught and school’s social norms as well as teachers’
personal ideas and beliefs (Zhao & Frank, 2003). Later focus group and survey studies applied
18

this idea to supplementation, finding that teachers often seek out unofficial curriculum materials
because their schools have not provided them all materials to enact the officially-adopted
curriculum or because their school culture is particularly friendly to supplementation (Marple, et
al., 2017; Sawyer & Myers, 2018). A social network analysis of 100 teachers’ Pinterest accounts
found that both individual-level and institution-level factors influenced the sorts of resources that
teachers pinned (Torphy, Liu, Hu, & Chen, 2020). Institution-level factors (e.g., whether two
teachers worked at the same school) tended to predict whether two Pinterest accounts had similar
pinning patterns at all, whereas individual-level factors (e.g., whether two teachers had
approximately the same amount of teaching experience) tended to predict the degree of similarity
in pinning patterns between accounts.
Student-based Motivations for Supplementation
Teachers have commonly cited reasons for engaging in curriculum supplementation like
engaging their students or aligning materials to specific learning needs and styles (Emo, 2015;
Irvine, 2015; Marple, et al., 2017; Polikoff & Dean, 2019; Recker, Dorward, & Nelson, 2004;
Sawyer, et al., 2020; Shapiro, et al., 2019; Webel, Krupa, & McManus, 2015). They have also
reported supplementing to better align their officially-adopted textbooks’ content to learning
standards, to adjust their textbooks’ difficulty, or to correct perceived shortcomings in their
textbooks (Emo, 2015; Hu, Torphy, Chen, & Eagan, 2018; Irvine, 2015; Marple, et al., 2017;
Shapiro, et al., 2019; Webel, Krupa, & McManus, 2015). Others supplement to better assess
student knowledge or to provide students additional choices in their instruction (Emo, 2015;
Recker, Dorward, & Nelson, 2004; Yeigh, Villagomez, Lenski, McElhone, & Larson, 2017).
One recent study took this work a step further and investigated teachers’ motivations for making
changes to their online supplements rather than their official curriculum materials. Teachers in
19

that study reported adapting supplemental materials for many of the same reasons that teachers
tend to adapt official materials, listed above (Sawyer, Dick, Shapiro, & Wilson, 2020).
Increasing student engagement is by far the most commonly-cited student-centered reason for
supplementation in the literature, indicating that teachers who supplement may perceive their
officially-adopted textbooks as dull or otherwise less-than-ideal for their students.
Teacher-based Motivations for Supplementation
Teachers have also cited less student-centric reasons for supplementation, like saving
time, using materials that are easy to understand and implement, or even incorporating “cute”
materials (Irvine, 2015; Marple, et al., 2017; Polikoff & Dean, 2019; Schroeder, Curcio, &
Lundgren, 2019; Webel, Krupa, & McManus, 2015). Teachers also report supplementing to feel
a sense of professional autonomy or to collaborate with other educators (Carpenter, et al., 2020;
Schroeder, Curcio, & Lundgren, 2019; Yeigh, et al., 2017). Some even seek out supplemental
materials to keep themselves from becoming bored by the same curriculum year after year (Emo,
2015; Marple, et al., 2017). Evidence suggests that these motivations are not evenly distributed
across teachers. For example, less-experienced teachers may be more motivated by “cute and
fun” supplements than more-experienced teachers (Schroeder, et al., 2019).
In light of the wide array of contextual, student-based, and teacher-based reasons teachers
may have for supplementation, Lane, Boggs, Chen, and Torphy (2019) presented a framework
for teachers’ use of virtual resources, which may often be, but are not necessarily, supplements.
They proposed conceiving of virtual resource enactment as a product of the resource’s purpose
as a main or supplemental material, the school’s social organization, and the curriculum
enactment process itself. These three factors interact with a variety of characteristics (e.g., local
context, teacher participation in a learning community, etc.) to determine how the virtual
20

resource is enacted. For example, teachers in a school with a relatively autonomous social
organization are likely to enact virtual resources well-tailored to their specific context. Hu,
Torphy, Evert, and Lane (2020) built on these ideas, conceiving of teachers’ virtual resource
curation as an attempt to solve some problem and identifying three distinct modes of virtual
resource curation: self-directed, incidental, and socialized. Although these papers did not present
a theory of supplementation per se, there is a large overlap between supplements and virtual
resources, so many of these ideas are likely relevant to supplementation, at least in the virtual
space.
Finding 4: The Sources Teachers Use to Access Supplements May Relate to Their
Supplementation Goals
Teachers might turn to any of a wide array of sources in seeking supplemental curriculum
materials: virtual resource pools, individual teacher blogs, a professional organization’s website,
their Twitter network, a colleague, a Google search, and more. Teachers may even decide to
forgo all of these sources and create a supplement themselves. There is evidence in the literature
that teachers make this choice purposefully, sourcing their supplements in accordance with their
goals for supplementation. For example, Frank and Torphy (2019) found that teachers do much
of their supplement acquisition via TPT and Pinterest, and tend to use Twitter to keep informed
about important issues in education. Hu & Torphy (2020) contrasted the affordances of social
media (e.g., Twitter, Facebook) with virtual resource pools (e.g., TPT, district repositories),
hypothesizing that these differences may dictate that teachers use these sources in different ways.
Although there is little multi-platform empirical work to directly test this theory, studies of
individual platforms do suggest that teachers’ supplementation goals vary by platform.
21

Twitter
The social networking site Twitter is attractive to teachers for a number of reasons.
Surveys of teachers on social networking sites like Twitter have shown that many use the
platform primarily to find and share resources, collaborate with like-minded teachers, harness a
large and always-responsive sounding board for ideas, or even monetize and promote their own
products and ideas (Carpenter, 2016; Carpenter & Krutka, 2014; Hunter & Hall, 2018; Staudt
Willet, 2019; Trust, Krutka, & Carpenter, 2016). Some teachers report using Twitter to
personalize their own professional development or receive informal mentoring (Carpenter, 2016;
Carpenter & Morrison, 2018; Staudt Willet, 2019; Torphy & Drake, 2019; Trust, Krutka, &
Carpenter, 2016), and some use the platform to find emotional support (Carpenter & Krutka,
2014; Hur & Brush, 2009; Trust, Krutka, & Carpenter, 2016).Teachers have also reported
frustrations associated with engaging in social networks like Twitter such as frequent self-
promoting posts from “edu-influencers” or a general lack of meaningful discourse (Carpenter &
Harvey, 2019). In terms of supplementation, then, Twitter may be teachers’ best option when
they want to generate many ideas or gather many opinions quickly.
Pinterest
As the primary marketing tool for TPT stores and one of just a few sites that incorporates
aspects of both social networks and virtual resource pools, it is little surprise that nearly half of
U.S. teachers use Pinterest weekly or more (Hu & Torphy, 2020; Kaufman, et al., 2018; Pittard,
2017). In fact, “education” is the second most popular category on all of Pinterest, after “home
decor” but before “entertainment” and “food and drink” (Herrington, 2021). Studies suggest that
teachers use Pinterest mainly for gathering and organizing ideas over time by pinning them to
thematic “boards” (Schroeder, et al., 2019). The content pinned to these boards overwhelmingly
22

comes from other teachers, not from expert organizations (Torphy, Hu, Liu, & Chen, 2020). Liu,
Torphy, Hu, Tang, and Chen (2020) quantified the ways teachers’ support diffused across social
networks by building linear growth models of the Pinterest accounts of both early-career and
veteran teachers. They found that early-career teachers were about 67% more likely to pin a
curriculum resource for each veteran colleague who had pinned it, and that veteran colleagues
with whom the teacher did not frequently discuss curriculum were more influential on pinning
behavior than those with whom the teacher did. This finding indicates that early career teachers’
Pinterest use is closely intertwined with their curriculum use. In contrast with Twitter, which
may be best used for a quick supplementation idea, Pinterest may be particularly useful to
teachers who want to curate and maintain a set of supplemental materials for use in their
classrooms.
Instagram
As a newer technology than Twitter or Pinterest, there is less empirical work focused on
teachers’ use of Instagram. The work that does exist indicates that teachers use Instagram largely
to see examples of instructional techniques or supplemental resources in action, which may not
be surprising given Instagram’s heavy visual emphasis (Carpenter, Morrison, Craft, & Lee,
2020). As with Pinterest, studies have shown that teacher-Instagram is highly commercialized,
with widespread marketing of supplemental materials and frequent links to TPT stores (Shelton,
Schroeder, & Curcio, 2020).
Teachers Pay Teachers
As probably the most popular source of supplemental materials, many of teachers’
reasons for visiting TPT have already been outlined in discussion of their motivations for
supplementation, above. However, one supplementation goal frequently appears in studies of
23

TPT, but rarely in studies of other supplementation sources: a desire to save time (Emo, 2015;
Irvine, 2015; Pittard, 2017; Polikoff & Dean, 2019). This indicates that TPT may be especially
convenient relative to other sources of supplements, so teachers may seek supplements from TPT
especially when rushed or exhausted.
Krutka and colleagues (2019) provided a set of guidelines to help educators safely
navigate the profit-driven social media landscape with their students, emphasizing activities to
help students spot misinformation and combat harassment and cyber-bullying. Hashim and
Carpenter (2019) provided a conceptual framework for teachers’ own social media use,
characterizing teachers as utility maximizers who weigh individual preferences and social desires
in deciding whether and how to participate in social media. Teachers’ social media use is an
intriguing and rapidly expanding field in its own right, but for the purposes of this review, social
media is important because it is a context in which curriculum supplementation commonly
occurs. Under Hashim’s and Carpenter’s framework, we might imagine supplementation to serve
either an individual preference (e.g., to deliver a more-effective lesson) or a social desire (e.g., to
become known as a teacher with impeccable materials).
Real-life Social Networks
Schools have long encouraged their teachers to develop in-person networks with their
coworkers to improve their practice. One ethnographic case study found that the networks where
teachers grow most and are most engaged tend to “thrive on conflict and dissent,” allowing
members to have and to voice conflicting ideologies (Achinstein, 2002). A later case study
focused on higher-level contexts, finding that school and district policies powerfully shaped the
structure of teachers’ social networks, the amount of expertise those networks contained, and the
depth of a typical interaction within the networks (Coburn & Russell, 2008). The literature
24

indicates that teachers may not have a particularly specialized view of their real-life networks
with regards to supplementation: they use these networks to seek expertise, reflect on practice,
discuss official and supplemental curriculum, and more (Coburn & Russell, 2008).
Self-created Supplements
Although supplementation often occurs in a social setting, teachers may simply create
supplemental materials themselves. I am not aware of recent empirical work that focuses on
teachers’ self-created supplementation beyond simply documenting that it occurs. The literature
offers little as to what the specific affordances of self-created supplementation are. Speculatively,
one benefit might be full control over the structure and content of the supplemental material,
although this is an area in need of future research.
Finding 5: Teachers Have Stable “Default” Supplement Use Patterns, Shaped by Personal
Characteristics and Context
Even before online supplementation became ubiquitous, some researchers conceived of
teachers’ real-life and online curriculum use not as enactment of a textbook, but as a dynamic
design process, where teachers adapted and modified their official curricula with new materials
in hopes of saving time or engaging their students. They theorized that specific teacher
characteristics like years of experience influence the types of materials they tend to use and how
they use them, implying that different teachers might adapt their curriculum materials quite
differently, but that there should be relatively stable within-teacher patterns of curriculum
adaptation and modification (Brown, 2002; Recker, et al., 2004).
There is empirical support for these ideas. Qualitative case studies have found that
teachers have consistent, “go-to” approaches around when and how they adapt their official
curricula (Drake & Sherin, 2006). A later survey study of teachers’ Pinterest use noted that all
25

participants had a consistent, articulable process for determining whether potential supplements
were of high quality, although that process looked quite different across teachers (Schroeder, et
al., 2019). Studies focused specifically on pre-service teachers’ patterns of online
supplementation found that they are generally uncritical about the quality of downloaded
materials, put a greater-than-ideal emphasis on cute materials, and that the modifications they
make to official curriculum tend not to be particularly educationally meaningful (Forbes &
Davis, 2010; Grote-Garcia & Vasinda, 2014; Rodriguez, Brown, & Vickery, 2020).
Noting teachers’ relatively predictable approaches to curriculum supplementation, some
researchers have attempted to categorize this behavior. Nicol and Crespo (2006) asserted that
teachers may take an “adhering,” “elaborating,” or “creating” approach to their officially-
adopted curriculum, and that teacher factors (e.g., subject matter expertise, personal beliefs, etc.)
and contextual factors (e.g., access to resources, school culture, etc.) determine which approach
teachers take. Sherin and Drake (2009) identified similar factors as contributing to teachers’
curriculum use and supplementation patterns, but expanded the conception to include a timing
component. For example, more-experienced teachers may anticipate student needs and do most
of their supplementation before a lesson, while novice teachers may supplement after the fact, in
response to misconceptions encountered during the lesson. Kaufman and colleagues (2020)
updated these ideas by explicitly accounting for virtual supplements, categorizing teachers as
“by-the-book,” “DIY teachers,” “modifiers,” or “cobblers” with respect to their use of real-life
and virtual curriculum materials. Although their profiles did not distinguish between school-
adopted and teacher-adopted materials (so did not explicitly denote supplementation), the
researchers found that only 26% of teachers in their 12 state-representative sample enacted their
curriculum “by-the-book,” providing at least suggestive evidence that some of what they were
26

measuring may have been supplementation. Each of these frameworks are relevant to
supplementation, but fall short of capturing the phenomenon in a comprehensive way. For
example, none account for a supplemental material’s source, which emerges in the literature as
an important dimension of supplemental materials, related to teachers’ goals for enacting the
material.
Teachers’ approaches to supplementation tend to be stable, but they are not immutable.
Dietiker and Riling (2018) found that professional guidance can alter how teachers enact official
curriculum, even if the guidance is simply from the official teacher guide. Understanding this,
Sawyer, Dick, Shapiro, and Wismer (2019) called for developing teachers’ “critical curation”
abilities, and Hu and Torphy (2020) delineated between teachers’ “thick” (purposeful and
human-led) and “thin” (inscrutable and algorithm-led) supplement curation, arguing that both
teachers’ and search algorithms’ curation must be improved to realize the full potential of virtual
resource pools. Many researchers have taken up this call and developed checklists, question sets,
and guidelines for use by teachers and school leaders to help them supplement their official
curriculum materials more effectively (Gallagher, Swalwell, & Bellows, 2019; Harris, Shelton,
& Archambault, 2020; Hu & Torphy, 2018; Hu, Torphy, & Opperman, 2019; Riling & Dietiker,
2018; Rodriguez, Brown, & Vickery, 2020; Shelton & Archambault, 2019b).
Finding 6: Supplements’ Specific Features Relate to Whether and How Teachers Enact
Them
The broad curriculum literature is clear that specific features of curriculum materials
matter for their effectiveness, so ought to affect how teachers use them. There is no reason to
believe that the features of supplemental materials function any differently than the same
features of traditional materials in terms of educational value. For example, images are generally
27

accepted to aid children’s reading comprehension (Takacs & Bus, 2018), so presence of this
feature should reasonably affect which materials a teacher selects and how the teacher uses them.
Although most features of supplements work analogously to features of official materials
(i.e., a well-chosen picture aids comprehension whether it is part of a supplemental or official
material), certain features are much more likely to show up in supplemental materials. Metadata,
such as the user ratings and comments left for online products, is one such feature. Abramovich
and colleagues studied resources’ metadata on two popular virtual resource pools, TPT and the
exclusive-to-Teach for America TFANet (Abramovich & Schunn, 2012; Abramovich, Schunn,
& Correnti, 2013). They found that the number of ratings or comments a resource had was far
more predictive of the number of downloads than the rating itself (i.e., how many stars) or the
content of the comments (positive or negative). Because search algorithms commonly list high-
traffic materials first without as much emphasis on that traffic’s content (SEMrush, 2019), it is
reasonable to assume that supplements with the most ratings, but not necessarily the highest
ratings, appeared toward the top of these virtual resource pools’ search results. In that case,
Abramovich and colleagues’ findings indicate that the virtual resource pool users in their studies
may not have been particularly discerning when seeking supplemental materials. They may have
satisficed, downloading the first item in the search results that cleared a minimum quality
threshold. Regardless, these findings indicate that teachers are sensitive to supplements’ specific
features, whether their number of ratings and comments or their position in a set of search
results.
Another feature that may be especially prevalent among supplements, especially those
from little-curated virtual resource pools, is “cuteness.” Studies have shown that even low-
quality materials tend to incorporate cute elements (Harris, Shelton, & Archambault, 2020) and
28

that visual appeal is a major consideration of supplement creators (Sawyer, Dick, & Sutherland,
2020). Pre-service teachers especially have been found to attend to cuteness as a feature of
supplemental materials even above educational quality in their downloading decisions
(Gallagher, et al., 2019; Grote-Garcia & Vasinda, 2014).
One supplement feature notably missing from the existing literature is school subject. A
number of studies focus on supplementation within a content area, but rarely compare teachers’
supplementation behaviors across subjects. Research designed to shed light on the content areas
in which teachers are likeliest to use supplemental materials could be particularly valuable.
Finding 7: Supplements from Virtual Resource Pools Are Generally Low Quality and
Sometimes Pose Potential Harm to Users
This section details a number of studies that investigate the effects of open curricula,
virtual resource pools, and other online materials on teachers and students. Missing from this
section are studies that investigate the effects and effectiveness of supplements not available via
the internet, created by the supplementing teacher themselves or by a colleague in the building.
As noted earlier, this is a potentially rich avenue of future inquiry if we are to understand the
phenomenon of teacher curriculum supplementation fully.
Low Academic Quality
Generally, researchers and teams of experts who comment on the quality and
effectiveness of online supplemental materials find them lacking. Researchers have warned that
resources downloaded from the internet are no substitute for good pedagogy, especially for
students with special needs (Beahm, Cook, & Cook, 2019; Miller, 2015). A content analysis of
Pinterest posts related to operations with negative integers found that 31% of pins contained
conceptual errors and just 15% of pins provided any real-world context for the math exercises
29

(Hertel & Wessman-Enzinger, 2017). A later analysis of the 500 most-popular mathematics-
related Pinterest resources found them to be low-rigor, with 53% of resources requiring only
memorization, another 45% requiring basic procedural math knowledge, and just 2% requiring,
in the words of the researchers, “doing math” (Sawyer, et al., 2019). In a follow-up study, the
researchers found that free mathematics resources tended to have lower cognitive demand and
employed more “cute,” non-functional images compared to paid resources (Sawyer, MacNeille,
& Dick, 2020). Hu, Torphy, Opperman, Jansen, and Lo (2018) found that early-career teachers
tended to pin resources that required only memorization or procedural knowledge far more often
than resources that required higher-order thinking, and that they often pinned these sorts of
resources together to the same board.
A study where content experts compared the top high school English Language Arts
resources from multiple virtual resource pools (TPT, Read Write Think, and Share My Lesson)
found that all resources were academically weak, with the majority of main activities requiring
depth of knowledge of just levels 1 or 2 (Polikoff & Dean, 2019). Similarly, in a study of TPT’s
top-rated U.S. History resources, expert raters deemed 70% of the resources “poor” or
“moderate” overall, with 65% containing no clear learning objective, although they noted that
85% of top resources were well-designed and visually appealing (Harris, Shelton, &
Archambault, 2020).
In one of the only studies to connect teacher curriculum supplementation to student
outcomes, Torphy, Chen, Yang, and Chhikara (2019) linked teachers’ Pinterest accounts to
student-level achievement data to model the relationship between supplementation via Pinterest
and students’ math test scores using hierarchical linear models. They found that the overall
quality of a teacher’s pinned resources (as measured by depth of knowledge indicators) predicted
30

test scores in the expected direction—teachers who pinned more rigorous materials had students
who scored higher. Although it is unclear whether the pinning itself is causally linked to the
students’ test scores or simply a by-product of some common cause (e.g., teaching ability), these
findings suggest that, even when considering student-level outcomes, teachers’ supplementation
habits are worthy of investigation.
Potential Harm to Teachers and Teaching
Some researchers have warned that the rise of teacherpreneurs and edu-influencers on
Pinterest, TPT, and various social media platforms has led to a neoliberal “marketization” of
teaching that may disillusion teachers or cause them to leave the profession entirely (Attick,
2017; Bartell, et al., 2019). Pittard (2017) argued that this rise has normalized purchases and
products as a path to being “good enough” as a teacher. This injection of market ideals into
teaching may marginalize consumers (who aren’t “good enough” on their own, so must make up
for it with purchases) and teachers who lack the means to participate in the marketplace, who
cannot even buy their way to “good enough” status.
Empirical evidence suggests that capitalistic ideals are in fact prominent in these spaces.
For example, qualitative studies of Pinterest (Shelton, Schroeder, & Curcio, 2020) and Instagram
(Engman, Lemanski, & Shepard-Corey, 2019) posts by top teacherpreneurs revealed that the
most common type of post was product promotion, often with a link to a TPT store, painting
consumption as one way to achieve teaching success. Another study used data mining techniques
to analyze sources of nearly 200 teachers’ pinned items on Pinterest and found that 82% of
pinned content gave an option to buy something, either via a teacher’s personal blog or via TPT
(Torphy, Hu, Liu, & Chen, 2020). Just 7% of pinned content linked to “expert” education
organizations like the National Council of Teachers of Mathematics or for-profit learning
31

companies like IXL. These findings indicate that a strong consumptive sensibility undergirds
many of teachers’ online interactions, although whether this represents a problem (Pittard, 2017)
or progress (Boitnott, 2015) is up for debate.
Lack of Cultural Relevance and Potential Harm to Students
Aside from being of poor quality for general education purposes, research suggests that
online supplements tend not to be culturally relevant, and that some may even be harmful for
certain students. For example, Polikoff & Dean (2019) found that only 4% of supplemental
resources in their sample of 328 top-rated ELA materials contained support for teaching students
with disabilities or other specific learning needs, with 10% containing limited support and the
remaining 86% containing none. Materials also rarely included texts written by nonwhite authors
or contained “culturally diverse topics.” The study’s expert raters rated just 2% of TPT materials
to have “strong inclusion of diverse perspectives,” with 79% containing none at all.
One study of top-rated U.S. History resources on TPT found that nearly a third make
light of painful parts of history or invite students to embody racist, sexist, or xenophobic roles as
part of role plays (Harris, Shelton, & Archambault, 2020), and another found that resources
related to Black history frequently use cartoonish clip art of Black leaders and invite students to
color their skin, trivializing this history (Rodriguez, Brown, & Vickery, 2020).
Hu, Torphy, and Opperman (2019) introduced a framework for assessing the cultural
relevance of materials found online, and they found a sample of 29 TPT materials tagged by the
authors as culturally relevant to be similarly lacking. Just 10% of the materials would have given
a hypothetical African American student the opportunity to reflect on their own culture, 59%
would have given them the opportunity to reflect on another culture, and only 17% would help
the student to develop a critical consciousness (e.g., Ladson-Billings, 1995).
32

Exceptions to the Rule
The bulk of supplementation-relevant literature indicates that the supplements under
study are generally of poor quality, but there are some bright spots. For example, open curricula
tend to be preferred to traditional textbooks by teachers (Kimmons, 2015; Mason & Kimmons,
2018) and students (Morales & Baker, 2018), and use of open curricula may encourage student
engagement with school (Kim, Lee, Leite, & Huggins-Manley, 2020). It is important to keep in
mind, though, that these studies do not denote the level at which the open curricula are adopted.
Open textbooks may be supplements selected by teachers, but are also commonly adopted at the
school or district level, so may in fact be officially-adopted curriculum. In a similar vein, Smith,
Westberg, and Hejny (2017) found that teachers have generally positive views of the Accelerated
Reader program, but for about half the teachers in their sample, the program was adopted at the
school or district level, so this finding is not purely about a supplemental curriculum material.
In the only published randomized control trial using supplemental materials, Jackson and
Makarin (2018) asked whether a supplemental material can improve student learning under ideal
conditions. They selected high-quality materials and provided teachers training on how to
integrate them smoothly into their official curriculum, eliminating the sizable stumbling blocks
of low supplement quality and instructional incoherence between supplemental and official
curriculum. Under these conditions, students in classes whose teachers received supplements and
training scored 0.09 standard deviations higher than teachers who were not given access to
supplemental curriculum, an effect in line with what one might expect by reducing class size by
15% (Jackson & Makarin, 2018). Another experimental condition where teachers received
access to the supplements but no training suggests that most of this effect was due to the high-
quality supplemental material itself, not the training received.
33

While this experiment showed that supplemental curriculum can boost learning outcomes
relative to officially-adopted materials under ideal circumstances, studies of supplemental
materials typically used in practice find most to be of poor quality. Therefore, it is essential that
we understand teachers’ supplementation decisions in order to develop teachers who supplement
skillfully and responsibly. In the next section, I outline the Teacher Curriculum Supplementation
Framework (TCSF) as a way for researchers to understand the antecedents to and effects of
teachers’ supplementation.
The Teacher Curriculum Supplementation Framework
The wide array of reasons teachers report for supplementation, the dizzying number of
supplement sources and types, and the variety of ways a teacher might enact supplemental
curriculum (see findings 3-6, above) all point to teacher curriculum supplementation being a
complex phenomenon. The ubiquity of supplemental materials in U.S. classrooms (see finding 1,
above) points to teacher curriculum supplementation being an important phenomenon. This
complexity and importance indicate that teacher curriculum supplementation is a worthwhile
avenue of scholarly inquiry, yet many supplementation-relevant studies focus on other, related
constructs like general curriculum use, teachers’ online behavior, or structures of virtual resource
pools, relegating supplementation itself to a supporting role.
Perhaps one reason teachers’ curriculum supplementation rarely stars in scholarly inquiry
is that it is commonly viewed in a piecemeal way, with materials teachers create themselves after
school or borrow from down the hall seen as fundamentally different from materials they
download from a colleague’s blog or a virtual resource pool. Despite important differences
between these practices, I argue that they are all instances of teacher curriculum supplementation
and can thus be understood using similar means. I propose the Teacher Curriculum
34

Supplementation Framework (TCSF) as a useful way for researchers to conceive of teacher
curriculum supplementation, addressing the second question. The TCSF is a flexible yet testable
conceptual tool for systematizing scholarly inquiry around teachers’ curriculum supplementation
and its effects on teachers and students. In the paragraphs that follow, I formally define teacher
curriculum supplementation under the TCSF, then argue that empirical study of teacher
curriculum supplementation ought to consider four dimensions that feature prominently in the
supplementation-relevant literature: the teacher’s reasons for supplementation, the supplement’s
source, the teacher’s supplement use pattern, and features of the supplement itself. The
framework is represented visually in Figure 1. Note that I do not argue that these are the only
four possible dimensions of teacher curriculum supplementation, just that these four emerge as
essential in the current research base. As the body of supplementation-relevant literature grows,
it is likely that additional dimensions also emerge as essential to consider in empirical work. (For
example, if remote schooling brought on by the COVID-19 pandemic remains a widely-available
choice for students and families in the medium term, it will likely be important to consider
teaching modality in empirical supplementation work.)
35

Figure 1. The Teacher Curriculum Supplementation Framework

What is Teacher Curriculum Supplementation?
The TCSF defines supplementation as: a premeditated, additive change that a teacher
makes to their official curriculum materials.
In specifying “premeditated,” I exclude curriculum decisions that occur during
instruction. Thus, supplementation is distinct from constructs like teachers’ “curriculum
strategies,” which may play out in response to “classroom events that transpire” (Sherin &
Drake, 2009). Supplementation under the TCSF carries with it a degree of intent and purpose
that in-the-moment decisions may not have.
In specifying “additive,” I bound this conception of supplementation to materials or
activities that a teacher has added to the official curriculum. This is in contrast with “curricular
adaptations,” which may include additions or subtractions of activities (Riling & Dietiker, 2018).
Teacher Curriculum Supplementation
Reason
• Engage students

• Decrease lesson
planning time

• Enact lessons
not in official
curriculum

• Etc.
Use Pattern
• Minutes per day

• Proportion of all
materials

• Frequency

• Etc.
Supplement Features
• Images

• Computer vs
paper

• Rating

• Etc.
Source
• Virtual resource
pool

• Self-created

• Teacher blog

• Etc.
Effects

• On teachers

• On students
Effectiveness
• According to
agreed-upon
criteria
(academic
growth, academic
mastery, critical
awareness, etc.)
produce
determine
Contextual Factors
Contextual Factors
36

In specifying that supplementation denotes a “change,” I emphasize that it applies only to
some instances of curriculum enactment, defined as “the chain of events whereby a set of
curriculum materials leads to specific instances of classroom instruction” (Sherin & Drake,
2009). A teacher may enact curriculum with perfect fidelity to their officially-adopted textbook,
which is not supplementation.
In specifying that a “teacher” carries out supplementation, the TCSF means that
supplementation is done by the person delivering instruction directly to students. A principal or
dean might make changes to district-adopted curriculum materials, but those changes are likely
to be perceived as “official” by teachers, and the TCSF is interested in understanding and
measuring teachers’ unofficial changes to curriculum. However, the TCSF does not restrict
supplementation to any particular type of teacher: general education teachers can supplement,
but so can special education teachers, literacy specialists, art teachers, and so on. However, some
teachers (e.g., special education teachers) may be expected to alter the school’s curriculum
materials as part of their official duties, which is not considered supplementation under the
TCSF.
Finally, the TCSF defines supplementation as changes to “official curriculum materials,”
which I define as having been adopted via some predetermined process by a higher-level entity
than the teacher, typically the school or district. These materials need not be traditionally
published textbooks; a school that expects its teachers to use the EngageNY open curriculum, for
example, is considered to have an officially adopted curriculum. (In this case, any premeditated,
additive change a teacher makes to the EngageNY lessons is considered supplementation.)
Defining supplementation specifically in relation to official curriculum materials paints it as
distinct from a construct like “virtual instructional resource enactment,” where teachers “locate
37

and download instructional materials to help them with their teaching” (Lane, Boggs, Chen, &
Torphy, 2019). Teachers may enact virtual resources from their official curriculum’s website, for
example, which is not an instance of curriculum supplementation. As a final point of
clarification, a few schools have no curricular expectations for their teachers. In such cases,
teachers’ week-to-week lesson planning is itself a form of curriculum supplementation, although
it may be distinct from typical supplementation in some ways because such teachers may be
more incentivized to supplement than those with ready access to official curriculum materials.
This could be a fruitful area for future research.
Resource curation is a construct currently receiving attention in the literature that is
related to but distinct from teacher curriculum supplementation. There are a few different
definitions of curation, each of which differs from supplementation in some important way. For
example, Torphy and colleagues (2020) define curation as “the act of synthesizing others’ ideas
into a collection” which provides “resources preserved for [the teacher] and others.” This
definition does not account for teachers’ self-made supplements, which studies indicate are quite
prevalent (Polly, 2017). It also emphasizes preservation of the resources, whereas
supplementation is more concerned with a teacher’s decision to use a resource and less with how
they preserve the resource after using it. Gallagher and colleagues (2019) define curation as a
teacher’s reflection “on all of the potential drawbacks and dangers of viral curriculum,” and
Sawyer and colleagues (2020) define it as teachers “pull[ing] and adapt[ing] from what is
available to best meet the needs of their students.” These conceptions of curation do not account
for use of self-created or colleague-created materials (especially the first, which only accounts
for “viral” materials), and they take a more normative stance than curriculum supplementation,
38

defining curation as a process intended to improve students’ experiences. Teachers may
supplement for many reasons, only some of which are likely to improve student experiences.
It is important to note that the somewhat narrow bounding of supplementation under the
TCSF is meant to allow for discrimination among closely-related constructs, not to make value
judgements. Teachers’ textbook adaptation (e.g., Riling & Dietiker, 2018), virtual instructional
resource enactment (e.g., Lane, et al., 2019), and other related phenomena are each essential
topics of study in their own rights. By defining “teacher curriculum supplementation” like it
does, the TCSF emphasizes that, like these other constructs, teacher curriculum supplementation
is a worthy focus of study.
Dimensions of Teacher Curriculum Supplementation
There are four dimensions of supplementation that consistently emerge in recent
supplementation-relevant literature: reason, source, use pattern, and supplement features. These
correspond to findings 3, 4, 5, and 6, respectively, from this paper’s systematic review. In
principle, nearly any combination of reason/source/use pattern/supplement features is possible,
but in practice, the TCSF maintains that there are a smaller number of typical combinations.
Empirical work centering one or a subset of dimensions might be illuminating in itself, but only
by considering the four dimensions together and in context can we build a full understanding of
how teachers supplement and the effects of supplementation on teachers and students. Although
this implies that the TCSF rejects the notion that supplementation as a whole can ever be
meaningfully characterized as “good for students” or “bad for students,” it is perfectly within the
scope of the framework to characterize certain reasons, sources, use patterns, features, or their
combinations as generally good or generally bad for students. For example, the literature
suggests that sourcing resources from virtual resource pools may be generally bad for students
39

(e.g., Harris, Shelton, & Archambault, 2020; Hu, et al., 2018; Polikoff & Dean, 2019). Below, I
describe each dimension, then discuss how these dimensions might interact with one another.
Reasons for Supplementation
Central to the TCSF is the idea that supplementation costs the teacher something.
Concretely, a teacher spends their time and sometimes money to add an activity or lesson to their
curriculum when there is no institutional expectation that they do so. In some cases, there may
even be an explicit expectation that they not do so. Because supplementation carries with it this
cost, the TCSF maintains that teachers do not supplement on a whim; there are always reasons
for any act of supplementation. The TCSF is flexible as to exactly what those reasons might be,
but they are always related to something in teachers’ context (e.g., missing official curriculum
materials; Marple, et al., 2017), the teachers’ students (e.g., to provide more engaging
instruction; Shapiro, et al., 2019), or the teachers themselves (e.g., to save time; Irvine, 2015).
To date, studies that measure teachers’ reasons for supplementation often do so via
surveys (e.g., Davis, et al., 2013; Kaufman, et al., 2018). However, closed-ended survey
questions are restricted to the set of reasons the survey writer can imagine (Fowler, 2008), which
may fall short of the full set of reasons teachers have for supplementation. For example, some
survey-based studies may conceive of supplementation as purely student-motivated, so would
not allow for teachers to cite teacher-based reasons for supplementation. Because of this,
researchers should consider measuring teachers’ reasons for supplementation using interview
methods or open-ended survey responses.
Sources of Supplementation
Just like the TCSF is flexible as to the reasons teachers supplement, it is also flexible
about the source of supplementation. Supplemental materials may come from virtual resource
40

pools like TPT, from a colleague’s recommendation, from a cursory Google search, or any
number of other places. They may even be, and often are, self-created by the teacher. The
literature suggests that teachers source their supplements purposefully (Frank & Torphy, 2019;
Hu & Torphy, 2020), so a supplement’s source may be an important consideration in predicting
or determining its effects. Some research has already begun to examine supplements’ sources as
a dimension potentially related to academic quality. For instance, Polikoff and Dean (2019) find
that the overall quality of materials from TPT is somewhat lower than from Read Write Think or
Share My Lesson. It is sometimes useful to consider only a subset of potential sources (e.g.,
Lane, et al. (2020) provide a useful conceptual framework for understanding teachers’ virtual
resource enactment, separate from their offline supplementation), but the TCSF maintains that
understanding the full spectrum of teachers’ supplementation decision making and the effects of
supplemental materials requires considering a full range of sources.
Typically, researchers collect survey data on where teachers source their supplements
(e.g., Opfer, et al., 2017; Hu, et al., 2020). This is a reasonable route if researchers are only
interested in relatively coarse source measurement (e.g., self-created versus virtual resource pool
versus teacher blog, etc.), but if researchers hope to measure supplements’ sources in finer detail
(e.g., TPT versus Read Write Think versus Share My Lesson, etc.), survey methods may not be
ideal. Surveys are only valid data collection tools insofar as respondents can actually answer the
survey questions (Tourangeau, Rips, & Rasinski, 2000), and it is plausible that a teacher might
not remember each of their supplements’ sources while completing a survey. For this reason,
researchers should consider measuring supplement sources using teacher logs (see Kurz, Elliot,
Kettler, & Yel, 2014, for evidence of the validity of teacher logs as a data collection tool). There
41

is a tradeoff here, with logs likely to supply more precise data but surveys likely to supply the
data more quickly and cheaply.
Supplement Use Patterns
The TCSF also maintains that use patterns, the ways a teacher actually enacts (or plans to
enact) supplements in the classroom, can inform which supplemental materials a teacher selects
and the effects of that teacher’s supplementation. Although use patterns have not received as
much recent empirical attention as reasons for supplementation, they have been measured in
terms of when (Drake & Sherin, 2006), how (Nicol & Crespo, 2006), and to what extent
(Kaufman, et al., 2020) teachers modify official curriculum materials. In addition to these,
teachers’ use patterns may vary in many other ways: the proportion of instructional minutes a
particular supplement occupies, the proportion of all instructional materials supplements
comprise, the time of day the supplement is typically used, and so on. A supplemental material
used every day for five minutes, for example, is likely to have different effects on student
learning than the same materials used every day for thirty minutes or on a single day for five
hours. The TCSF does not prescribe which use patterns are likely to produce which effects, but
maintains that a teacher’s use pattern is important to consider when determining how a
supplement’s effects are felt.
Contemporary studies that incorporate supplement use patterns measure them via survey
responses (Kaufman, et al., 2020). However, as with supplement sources, information about use
patterns may not be particularly salient to teacher respondents, so such survey data may be
subject to large measurement errors as teachers strive to give their best estimates of, for example,
how many times per month they used a particular supplement in the last academic year.
Gathering use pattern data via teacher logs would reduce this error by reducing the length of time
42

teachers must recall in answering questions, and direct classroom observation (in-person or by
recorded video) could eliminate this error, at the cost of the researcher’s time. Surveys are a low-
cost, potentially high-error way to measure teachers’ supplement use patterns, while direct
observations are a high-cost, reduced-error way to measure the same. Teacher logs occupy a
space between the two, and ought to be considered as a data collection tool for future work that
measures teachers’ supplement use patterns.
Supplements’ Specific Features
Specific supplement features are the final dimension to consider when studying teacher
curriculum supplementation. Such features might include images (e.g., Grote-Garcia & Vasinda,
2014; Rodriguez, et al., 2020), ratings and other metadata (e.g., Abramovich & Schunn, 2012),
general visual appeal (e.g., Harris, Shelton, & Archambault, 2020), and more. Considered in
isolation, features of supplements should behave exactly the same way as such features of non-
supplemental materials. For example, a material that includes well-designed visual aids is
friendlier to English Learners than the same material without such aids whether it is
supplemental or officially-adopted. Although their effects may not be unique to supplements, it
is essential to consider these features when studying the effects of supplemental curriculum
materials.
Most studies that measure and discuss specific supplement features rely on the
researchers’ expertise to measure such features. As many of these studies note in their limitations
sections, even researchers who strive to be consistent in, say, determining which resources have
strong “visual design” may have some biases in their measurement (e.g., Harris, Shelton, &
Archambault, 2020). Moreover, such measurements are generally restricted to the supplements
teachers actually use, so do not take into account features of materials that the teacher did not
43

select. To measure this, researchers might observe a teacher’s supplementation process from start
to finish, noting features of the materials they consider and features of those they do not. More
powerfully, an experimental design could measure the effects of some specific supplement
feature on whether a supplement is downloaded or selected for use. Machine learning methods
could even generate measures of the sorts of visual or text features that tend to correlate with
supplement downloads across a virtual resource pool (Hastie, Tibshirani, & Friedman, 2009).
Even this set of methods is non-exhaustive; future research should consider these methods and
others for measuring supplements’ specific features and their effects.
Interactions between the Dimensions of Teacher Curriculum Supplementation
As noted above, the TCSF gives no a priori reason why any specific reason/source/use
pattern/supplement feature combination cannot exist, but acknowledges that some combinations
of characteristics along two or more dimensions show up more than others in practice. We might
think of these combinations as being particularly synergistic. For example, it is theoretically
possible that a teacher hoping to improve their students’ understanding of how math is applied to
real-world problems (reason) sit down and create (source) a black-and-white worksheet
(supplement feature) of fifty multiplication and division problems for their students to complete
in the first 5 minutes of class each day (use pattern). However, this supplement is not something
a researcher should ever expect to observe because (we hope) very few teachers would create a
supplement with “repetitive worksheet” features if their reason for supplementation is to build
real-world math understanding.
Certain combinations of characteristics across two or more dimensions are likely quite
common, though. Researchers might expect “time saving” and “downloaded from TPT” to be a
commonly co-occurring reason/source pair (e.g., Polikoff & Dean, 2019). Similarly, “low
44

cognitive demand worksheet” and “used for ten minutes each morning as students enter the
classroom” might be a commonly co-occurring supplement feature/use pattern pair, and “finds
the textbook unclear,” “self-created,” and “two-thirds of all materials are supplements” might be
a commonly co-occurring reason/source/use pattern triad. Research that highlights these
commonly co-occurring characteristics could powerfully characterize teacher curriculum
supplementation in practice.
As a theoretical paper aimed primarily at a research audience, my focus has largely been
on what the literature suggests about the phenomenon that is teacher curriculum
supplementation, distilled in the large, leftmost box of Figure 1. Ultimately, however, the reason
to study teacher curriculum supplementation must be to improve educational experiences by
determining which types of teacher curriculum supplementation are most effective in which
contexts. Such understandings would allow for school-, district-, and state-level policymakers to
craft policy that empowers teachers to improve their students’ experiences in school via
curriculum supplementation. My goal in embedding effects and effectiveness of supplementation
(the middle and right boxes of Figure 1) directly into the TCSF is to underscore that empirical
work on supplementation should have policy application where possible, and should ultimately
serve students and teachers.
By breaking supplementation down into four distinct dimensions, the TCSF provides a
framework for understanding teachers’ supplementation decisions and the effects of those
supplemental materials on teachers and/or students. Although determining a supplement’s effects
and effectiveness requires consideration of all four dimensions and key aspects of the teacher’s
context, there may be some characteristics along some dimensions that tend to lead to effective
supplements and others that lead to ineffective ones. Research has already found, for example,
45

that high school ELA resources from virtual resource pools tend to be only loosely aligned to the
Common Core Standards and that math resources linked on Pinterest tend not to be very
cognitively demanding (Polikoff & Dean, 2019; Sawyer, et al., 2019). Assuming that an
“effective” supplement is one that contributes to student learning, we might conclude that
resources from sources like TPT and Pinterest tend to be ineffective. On the other hand, we
might imagine that supplements created by a veteran teacher who wants to incorporate activities
more rigorous than those offered in their official curriculum would tend to be quite effective.
Conclusion
Teacher curriculum supplementation is growing, both as part of U.S. teachers’ practice
and as a field of research. Despite this, the body of supplementation-relevant literature tends not
to focus on supplementation specifically, so lacks cohesion and sometimes fails to capture all
aspects the phenomenon. I reviewed recent literature relevant to teacher curriculum
supplementation to identify key consensus findings related to supplemental curriculum materials
(question 1) and teachers’ decision making around supplementation (question 2), then presented
a framework to help researchers conceive of teacher curriculum supplementation in more
targeted ways (question 3). The existing literature reaches a consensus on seven findings about
who is involved in supplementation (see Systematic Review, findings 1 and 2), the key
dimensions of supplementation (Systematic Review, findings 3 through 6), and
supplementation’s educational value (Systematic Review, finding 7). To focus future work, I
propose the Teacher Curriculum Supplementation Framework, a tool which encourages
researchers to attend to each dimension of a given instance of supplementation in order to
determine its effects and effectiveness.
46

Moving forward, our understanding of teacher curriculum supplementation would be
enhanced by studies of: (1) the scope of commonly occurring characteristics within each single
dimension of supplementation, (2) which characteristics commonly co-occur among two and
three dimensions, (3) what the most commonly-used reason/source/use pattern/supplement
feature combinations are, (4) how characteristics within one, two, or three, or all four dimensions
relate to whether and how teachers supplement, and (5) how these characteristics relate to
various student and teacher outcomes. Live observation of teachers’ supplementation processes,
retrospective teacher interviews about these processes, and deeper analyses of supplement
features themselves (using either qualitative comparative or perhaps unsupervised machine
learning methods) seem especially promising for shedding light on some of these remaining
questions. A deeper body of empirical research on teacher curriculum supplementation might
allow school leaders to provide teachers targeted feedback on supplementation or allow
policymakers to incentivize the most effective types of supplementation and discourage less-
effective types. Even traditional textbook publishers could learn what needs teachers express via
their supplementation habits and create materials that anticipate those needs. These advances
might help U.S. teachers supplement more confidently and effectively, or in some cases obviate
the need for supplementation entirely. As a research community, we must develop our
understanding of teacher curriculum supplementation if we are to explore it as a potentially-
powerful lever for improving instruction. Hopefully the Teacher Curriculum Supplementation
Framework is a useful tool in this development and exploration.

47

Paper 2: A Survey Tool for Measuring Teachers’ Curriculum Supplementation
While there has been an explosion in supplementation-relevant empirical work in the last
decade, and particularly since 2017 or so, this body of literature tends to be fractured in terms of
how it conceives of supplementation and thus in how (and whether) it measures supplementation.
For example, Sawyer, Dredger, and colleagues (2020) conceive of supplementation as primarily
a curation activity teachers engage in online in order to improve student learning, measuring the
practice by surveying teachers about their motivations for engaging in it. On the other hand,
Abramovich and Schunn (2012) conceive of supplementation as a digital process in which
teachers attend to metadata to make decisions about which resources to use, deemphasizing the
comparison and retention of preferred materials over time implied by curation but emphasizing
the role of technology in much curriculum supplementation that happens in the virtual sphere.
Neither conception is wrong: teachers often do curate a set of preferred supplemental curriculum
materials over time (Torphy, Liu et al., 2020) and their decisions often are influenced by both
human-driven and technology-driven factors (Hu & Torphy, 2020). The other side of this coin,
though, is that neither conception fully captures the many aspects of the phenomenon of teacher
curriculum supplementation. (These are just two of many conceptions of teacher curriculum
supplementation present in recent supplementation-relevant literature. I outline these in this
study’s literature review and review them in more detail in Paper 1.)
In light of the plethora of useful conceptions of supplementation, but dearth of
comprehensive ones, I reviewed the supplementation-relevant literature from the current, virtual
resource pool-driven era of teacher curriculum supplementation (January 2015 through
December 2020) and attempted to abstract from that body of research a practical but
comprehensive definition and conception of teacher curriculum supplementation, which I call the
48

Teacher Curriculum Supplementation Framework (TCSF; Paper 1). While I believe using the
TCSF is a useful way to conceive of supplementation as a phenomenon, if we are to understand
more about the nuts-and-bolts process of supplementation, we must be able to measure it in a
consistent and comprehensive way. In this study, I create, pilot, and provide early validity
evidence for a survey tool to measure teacher curriculum supplementation. Specifically, I address
the following questions:
1. To what extent does evidence about the survey tool’s content-related validity,
discriminant validity, and internal reliability indicate that it can be used to produce
valid inferences about teacher curriculum supplementation and its dimensions, as
outlined in the TCSF?
2. If evidence suggests that the survey can produce valid inferences about teacher
curriculum supplementation, what do survey responses indicate about the commonly
co-occurring reason/source/use pattern/design feature combinations among
supplementing teachers in this sample?
I begin by reviewing relevant literature, with three aims. First, I will summarize existing
pre-TCSF conceptions of teacher curriculum supplementation to justify the TCSF as a reasonable
theoretical framework to adopt for this study. Second, I will summarize how previous work has
measured supplementation to argue that a more unified, theoretically-grounded measure is
needed. Third, I will briefly summarize the TCSF (introduced in more detail in Paper 1) as a
useful framework around which to construct this measure. After establishing the need for a
measurement tool and justifying grounding this tool in the TCSF, I will outline proposed
methods for its creation.
49

Literature Review
Pre-TCSF Conceptions of Curriculum Supplementation
To justify the use of the TCSF as a conceptual underpinning for the survey tool I develop
in this study, I review conceptions of teacher curriculum supplementation and related phenomena
that exist (whether explicitly stated or implied) in recent supplementation-relevant literature.
While existing conceptions of curriculum use contain useful ideas for thinking about
supplementation, they each fail to capture some important aspect of teacher curriculum
supplementation.
Theories of Curriculum Use
Researchers have written about instruction and curriculum use for decades, and
comprehensively summarizing all such frameworks is beyond the scope of this paper. Instead, I
will summarize and comment on those that most clearly inform supplementation-relevant
empirical work. The first such framework is Pedagogical Content Knowledge (PCK; Cochran et
al., 1991), which sets teachers’ general pedagogical knowledge (which may be shared by any
manager: how to implement group incentives, for example) and general content knowledge
(which may be shared by any subject expert: how to integrate by parts, for example) apart from
pedagogical content knowledge (which is specific to teachers: what sorts of exercises best
preempt common misconceptions about integration by parts, for example). While the PCK
framework did not specifically account for curriculum as initially introduced by Cochran and
colleagues (1991), the paper that most closely informs their framework actually highlights three
types of knowledge essential to consider when conceiving of instruction: “(a) subject matter
content knowledge, (b) pedagogical content knowledge, and (c) curricular knowledge,” the latter
of which refers to a teacher’s ability to adeptly draw on the “full range of programs designed for
50

the teaching of particular subjects and topics at a given level” (Shulman, 1986, p. 9-10). Despite
not including curricular knowledge as initially conceived, the PCK framework (especially
through its successor, TPACK, described in the next section) has informed a number of
supplementation-relevant studies. However, the PCK framework conceptualizes teacher
knowledge as a whole, so is too high-level to be ideal for conceiving of teacher curriculum
supplementation specifically.
In 1996, Ball and Cohen were some of the first to draw a distinction between school
systems’ “intended curriculum,” informed by learning standards and officially-adopted
textbooks, and teachers’ “enacted curriculum,” informed by teachers’ in-the-moment
pedagogical decision-making. While this initial conception did not account for teachers using
supplemental curriculum materials alongside or instead of officially-adopted materials, it was
one of the first times that the research literature explicitly acknowledged that what gets taught in
classrooms may differ from what appears in the textbook. Later research by Ball’s former
doctoral student Remillard (2005; also see Drake & Remillard, 2019) and by Davis and Krajcik
(2005) further clarified the relationship between teacher and textbook to be a “partnership,”
where teachers use their expertise and judgment to make decisions about enacting curriculum
materials, but specific design features of the materials themselves can also be “educative” for
teachers, with aspects of their design influencing how teachers enact them. The Curricular
Noticing framework from Dietiker and colleagues (2018) conceptualizes this interaction between
teacher and textbook more specifically as occurring in three distinct phases: teachers first
“attend” to the features of their textbooks, they then “interpret” these features through the lens of
their own training and professional experience, and finally they “respond” to these
interpretations with tangible actions in the classroom.
51

More recent work on curriculum use tends to account for supplementation, at least to an
extent. A group of researchers at RAND used nationally-representative survey data to conceive
of teachers as “by-the-book,” “DIY,” “modifiers,” or “cobblers” with respect to use of their
official curriculum materials (Kaufman et al., 2020), then included questions about curriculum
supplementation in follow-up interview protocols (Wang et al., 2021). This trend is consistent
with a call from leading curriculum scholars to shift the ways we study and conceive of teachers’
curriculum use to better account for the prominent role that supplementation plays in many
teachers’ practices (Drake, 2021; Polikoff et al., 2018). This shift is much-needed, but these
supplementation-adjacent conceptions of teacher curriculum use are still theoretically
underdeveloped. To do justice to supplementation’s importance in contemporary teaching, we
ought to develop theories that account for supplementation as a construct distinct from general
curriculum use.
Theories of Virtual Curriculum Use
Another strand of research has focused on teachers’ curriculum use specifically in the
virtual space. Since much supplementation, and nearly all supplementation present in the current
research literature, happens online, we might imagine these conceptions of virtual curriculum use
to be especially relevant to curriculum supplementation. But like the conceptions of general
curriculum use outlined above, theories of virtual curriculum use each have useful features, but
tend to fall short of capturing teacher curriculum supplementation in some way.
Iterating on the PCK framework, Mishra and Koehler (2006) introduced the
Technological Pedagogical Content Knowledge framework (TPACK), which emphasizes
effective integration of technology into instruction as its own form of specialized knowledge,
distinct from general pedagogical knowledge or specific pedagogical content knowledge. The
52

framework has been applied in studies of how preservice (Caniglia & Meadows, 2018) and in-
service (Robin, 2009) teachers use the internet, with findings indicating that some of this use
includes seeking out or creating supplemental curriculum materials. TPACK provides useful
ways to conceive of teachers’ interactions with technology, but supplementation is often carried
out in the analog space (Opfer et al., 2017; Polly, 2017), so this is not an ideal theory for
understanding the phenomenon of teacher curriculum supplementation.
Others (e.g., Hertel & Wessman-Enzinger, 2017) have understood virtual curriculum use
through the lens of “connectivism,” which emphasizes the often-social aspects of
supplementation via Pinterest or Twitter, for example, but does not fully account for teachers
who supplement by creating materials themselves or carrying out a quick Google search.
A research group at Michigan State University forwarded a theory of “virtual
instructional resource enactment” which maintains that curriculum use in virtual spaces can be
understood by considering three dimensions: the resource’s status as an official or supplemental
material, the teacher’s own process for enacting curriculum, and the social organization of the
teacher’s school (Lane et al., 2019). This theory notably considers a number of distinct
dimensions of virtual curriculum use, but is not specific to supplementation, so does not include
certain dimensions that may be especially important for supplemental materials like the
supplement’s source, for example.
Theories of Curriculum Curation
A few researchers have studied the ways that teachers search for and maintain a set of
virtual resources for use, a process called “curriculum curation.” This process is closely tied to
supplementation, but the two phenomena are not synonymous: teachers may curate a collection
of supplemental materials for use in their classrooms, but there is no reason to believe that all
53

supplementation includes a curation step. For example, teachers may download and use the first
result in a Google search without reviewing the resource first, or they might create a
supplemental resource themselves. Therefore, understanding how researchers have conceived of
curriculum curation may provide insight into teacher curriculum supplementation but, like the
conceptions outlined above, is not a sufficient lens through which to view the phenomenon.
Researchers who have studied teachers’ navigation of virtual resource pools like
TeachersPayTeachers.com (Abramovich & Schunn, 2012), TFANet (Abramovich et al., 2013),
and YouTube (Fyfield et al., 2020) do not explicitly name their theoretical perspective, but
foreground teachers’ use of heuristics and sensitivity to metadata (e.g., user ratings) and
information dictated by the search algorithm (e.g., the order in which results are displayed) in
seeking and selecting supplemental resources. Unlike TPACK and connectivism, this conception
may be a bit too specific to capture supplementation as a whole; when teachers self-create a
supplemental material, for example, there is no reason to think they would interact with
algorithms or metadata.
Three other groups have developed more explicit conceptions of teacher curriculum
curation. One unnamed conception emphasizes teachers’ formation of “shareable collections of
resources preserved for themselves and others” over time (Torphy, Liu et al., 2020, p. 15; also
see Hu & Torphy, 2020). Another, the Pinning with Pause framework, conceives of curriculum
curation as the act of navigating the “potential drawbacks and dangers of viral curriculum” to
enact high-quality resources (Gallagher et al., 2019, p. 222; also see Rodriguez et al., 2020). A
third, Critical Curation Theory, conceives of curriculum curation as the act of “pull[ing] and
adapt[ing] from what is available to best meet the needs of students” (Sawyer et al., 2019, p. 1).
Supplementation sometimes fits into one or more of these conceptions, but it need not always:
54

supplementing teachers may not look to preserve resources across time, and they may use
supplemental resources created by a colleague (not “viral resources”) or by themselves (not
drawing from “what is available”).
Pre-TCSF Measurement of Supplementation
Existing supplementation-relevant work often measures some aspect of supplementation,
but does not account for all dimensions from the TCSF, summarized in the next section.
Some studies have surveyed or interviewed teachers about their reasons for
supplementation, finding evidence that teachers do not just supplement for reasons related to
their students’ learning, but also for reasons related to themselves or to their school contexts
(Marple et al., 2017; Schroeder et al., 2019; Webel et al., 2015). However, these studies tend not
to account for supplement sources, and I am aware of none that account for anticipated or actual
supplement use patterns or the supplements’ specific design features. Other studies have focused
on what sources teachers use for their supplemental materials, typically using survey methods
(Kaufman et al., 2018; Marple et al., 2017; Opfer et al., 2017). These studies find that teachers
supplement from a wide variety of offline and online sources, including colleagues, virtual
resource pools, individual teacher blogs, and more. However, these studies tend to only
document one dimension of supplementation; they generally do not focus on supplement use,
reasons for supplementation, or supplement features. Comparatively few studies have focused on
supplement use patterns: Kaufman and colleagues (2020) classified teachers’ overall patterns of
curriculum use into categories that account for supplementation, including “modifier” and
“cobbler,” but the study did not focus on the supplementation itself. Polikoff and Silver (2021)
applied the TCSF to study supplement use patterns, but the study was also limited in its ability to
consider other dimensions of supplementation due to a relatively limited set of survey items
55

available in the data. A few studies have also focused on supplements’ specific design features,
finding that supplementing teachers attend to online metadata (Abramovich & Schunn, 2012;
Abramovich et al., 2013) and visual aspects of the materials (Irvine, 2015; Sawyer, Dick,
Shapiro et al., 2020; Schroeder et al., 2019) in their decisions to download and use supplements.
The supplementation-relevant literature has tended to measure supplementation by
focusing on only a single dimension at a time, often grounded in theory that does not fully
capture all aspects of the phenomenon. A valid, reliable, theoretically-grounded measure that
accounts for all dimensions of supplementation could contribute to a more coherent literature
base around teacher curriculum supplementation.
The Teacher Curriculum Supplementation Framework
Existing conceptions of curriculum use and existing ways supplementation has been
measured capture aspects of teacher curriculum supplementation but do not sufficiently account
for all aspects of the phenomenon. Because of this, I proposed the TCSF, which highlights key
dimensions of teacher curriculum supplementation from the body of supplementation-relevant
literature as a way for researchers to conceive of supplementation in empirical work. Here, I will
outline the aspects of the framework most relevant to this study. See Silver (2022) for a full
introduction to the framework.
The TCSF defines supplementation as any premeditated, additive change a teacher makes
to their officially-adopted curriculum materials. This definition distinguishes teacher curriculum
supplementation from related but distinct constructs such as general curriculum enactment
strategies, curriculum adaptation, virtual resource enactment, and curriculum curation.
Additionally, many related constructs assume that teachers alter their officially-adopted
curriculum materials to better serve their students, while the literature that informs the TCSF
56

shows that teachers often supplement for reasons unconnected to their students (e.g., to save
time, because their school is missing key resources to fully implement the official curriculum,
etc.). Therefore, the TCSF maintains that curriculum supplementation may be either helpful or
harmful to student learning, and that one must consider a number of distinct dimensions of
supplementation to understand the likely effects of any individual instance of supplementation on
students and teachers.
The TCSF in its current form identifies four main dimensions of supplementation that
emerge in the literature from 2015 onward, the current, virtual resource pool-influenced “era” of
supplementation. They are: reason for supplementation, supplement source, supplement use
pattern (actual or anticipated), and specific supplement design features. Note that these are
probably not the only important dimensions of supplementation; they are just the ones that
emerge as prominent in the existing body of empirical work. Under the TCSF, considering all
dimensions is essential for understanding the effects of a given instance of supplementation, and
there are certain reasons, sources, use patterns, and design features that may appear together
more commonly than others. For example, the literature (e.g., Polikoff & Dean, 2019) indicates
that teachers who source their supplemental materials from the virtual resource pool Teachers
Pay Teachers (source) often do so to save time (reason). While supplementation as a whole is not
helpful or harmful to students under the TCSF, specific reasons/source/use pattern/feature
combinations may tend either direction. Available evidence suggests that the reason/source pair
given above – teachers who supplement from Teachers Pay Teachers to save time – tends to lead
to supplementation of low educational value (Harris et al., 2020; Polikoff & Dean, 2019; Sawyer,
MacNeille et al., 2020).
57

I created the TCSF to fill a theoretical gap in the supplementation-relevant literature,
since much existing empirical work relies on conceptions of supplementation (or related
phenomena) that do not capture teacher curriculum supplementation in its full complexity. The
framework is designed to highlight the key dimensions of supplementation and hypothesize how
they might interact, based on findings from the supplementation-relevant empirical work
published since 2015 in a U.S., K-12 context (see Silver, 2022, for more detail on which studies
informed this framework). The framework can provide a common language for understanding
teacher curriculum supplementation as a phenomenon, and comes closer than those currently in
use to capturing its full complexity. Therefore, for this study, I conceive of supplementation as
featured in the TCSF.
As a new framework, though, little empirical work employs the TCSF (Polikoff & Silver,
2021, is an exception to this) and no TCSF-aligned work attempts to develop a single survey
measure that reliably produces valid inferences about the dimensions of supplementation.
Developing such a measure is important for similar reasons that the TCSF itself is: the
supplementation-relevant literature tends to measure supplementation in ways that do not
translate well across studies, so a common measure grounded in the TCSF could contribute to a
more coherent understanding of supplementation as a process. Similar efforts to produce
common measures from conceptual frameworks grounded in existing literature have proven
valuable in other areas, such as the study of teachers’ professional development (Desimone,
2009). I hope that this tool helps further systematize empirical study of teacher curriculum
supplementation and provides one way for future research in this area to cleanly build on what
has come before.
58

Sample
Although teachers across all subjects and grade levels supplement, it is reasonable to
believe that they do so in different ways. No literature explicitly investigates differences in
supplementation strategies by grade or subject, but available evidence suggests that their
strategies may differ (e.g., Polikoff & Silver, 2021). Because a large proportion of the
supplementation-relevant literature already focuses in whole or in part on elementary (K-5) math
teachers (e.g., Pittard, 2017; Rodriguez et al., 2020; Sawyer, Macneille et al., 2020; Torphy, Hu
et al., 2020; Torphy, Liu et al., 2020), I recruited a sample of in-service elementary teachers of
math to pilot and provide validity evidence for the supplementation survey measure.
I recruited 280 in-service U.S. elementary math teachers using the online participant
recruitment service CloudResearch, 22 of whom were part of an initial piloting and cognitive
interview process and the remaining 258 of whom were part of the sample that took the finalized
version of the survey. This sample size powered the study to detect true relationships among
items even when they were relatively small in magnitude, with Pearson’s r as low as 0.17 with
conventional alpha and beta of 0.05 and 0.2, respectively. This sample size is also in line with
recommendations from MacCallum and colleagues (1999) for recovering population factors from
a sample given the number and types of items I planned to administer.
CloudResearch is able to quickly recruit members of specific groups (i.e., K-12 teachers)
to participate in academic research and, from prior firsthand experience, CloudResearch works to
provide high data quality relative to less-boutique options like Amazon’s MTurk. I paid
CloudResearch $15 per participant recruited, which included an approximately $11 participant
incentive, processed through CloudResearch. This incentive was substantially higher than
“typical” per-worker payments of $.02 to $2 (CloudResearch.com, 2021) and MTurk average
59

payments of $2 per hour (Hara et al., 2018), which encouraged attentive survey responses. (Zero
of the 280 in-service teachers failed my attention check, for example.) I also provided an
additional $10 as compensation for participation in 20-minute cognitive interviews and $20 for
participation in 40-minute follow-up simulation activities (discussed in detail in Paper 3), all
processed as Amazon gift cards through CloudResearch. These payments were all financed
through Rossier grants: one $1500 internal research grant and multiple years of rolled-over
$1200 research and travel awards.
It is important to note that the 258 teachers who took the finalized survey were not
randomly selected, so I was unable to generalize from their responses to the larger population of
U.S. public elementary school teachers of math. This was slightly less of a concern than it would
typically be, since this study’s main goal was to establish the extent to which the survey tool
could yield valid inferences about curriculum supplementation, with substantive discussion of
that supplementation as only a secondary goal. Still, I will briefly describe differences between
this analytic sample and the overall U.S. public elementary school teacher population along a
few key variables (National Center for Education Statistics, 2019, Table 209.22) so readers can
keep these differences in mind as they read.
In general, studies of participants recruited online from the general population show that
they may be slightly younger and whiter than the overall U.S. population (Redmiles et al., 2019).
However, Table 2 indicates that the analytic sample for this study (i.e., the 258 teachers who
participated in the finalized survey) was quite similar to the U.S. population of elementary
teachers in public schools by race and ethnicity, certification type, and school type. This study’s
analytic sample was slightly older that the overall elementary teaching population, with a greater
concentration of teachers with 3-9 years of experience compared to more novice and more
60

veteran teachers. The analytic sample was distributed relatively evenly across elementary grades.
Although my inability to randomly sample participants for this study meant that I could not rule
out the possibility that the study’s sample differs from the U.S. public elementary school teacher
population in unobserved ways, the sample looked fairly similar to the population along key
observable characteristics.
Table 2. Comparing Analytic Sample to U.S. Public School Elementary Teacher Population
Characteristic Analytic Sample
(%; N=258)
Elementary Teacher Population
Estimate* (%)
Race/Ethnicity
White only 78.3 78.8
Black only 5.8 6.6
Hispanic/Latino only 8.5 10.2
Asian only 2.7 2.1
Other 0.8 0.8
More than one race or
ethnicity
3.5 1.5
Prefer not to say 0.4 .
Age
20-29** 18.4 15.7
30-39 28.5 28.2
40-49 19.1 29.5
50-59 18.0 20.0
60 and over 16.0 6.6
Teaching experience
Less than 3 5.0 9.4
3 to 9 39.53 29.2
10 or more 55.4 61.4
Certification type
Full/Regular 92.3 91.3
Other*** 7.7 8.7
School type
Traditional public 94.2 94.3
Charter 5.8 5.7
Grade taught
Kindergarten 16.3 .
1
st
grade 17.4 .
2
nd
grade 15.9 .
3
rd
grade 18.2 .
4
th
grade 17.4 .
5
th
grade 14.7 .
*Estimates are taken from the 2017-2018 Elementary column of the National Center for Education Statistics Digest of
Education Statistics Table 209.22. https://nces.ed.gov/programs/digest/d19/tables/dt19_209.22.asp
**This category listed as “Under 30” in NCES table.
***Category includes provisional, probationary, emergency, and no certification

61

Method
Finalizing and Administering the Survey Instrument
I drafted an initial version of the survey in consultation with a team of current and former
teachers who reviewed the items for adherence to TCSF constructs. I constructed the survey tool
to gather information across the four main dimensions of supplementation outlined in the TCSF:
teachers’ reasons for using supplemental materials, their sources for those materials, their
anticipated use patterns for the materials, and the features of supplemental materials that are
most important in teachers’ decisions to use them. (See Appendix A for a crosswalk showing the
constructs I designed each survey item to map onto, including those that changed from the initial
to final version.) Within each dimension’s set of questions, I included items specific to trends
that appeared in the supplementation-relevant literature. For example, when asking about
teachers’ reasons for supplementation, I included items related to teachers’ students, to their
school contexts, and to themselves, because those three classes of reasons are most prominent in
the literature. I also included a short set of questions about teachers’ school context and their
own demographic and teaching background to better contextualize the results.
To draw valid inferences from a survey tool, though, the tool must be clear and
intelligible to those taking it. To help ensure this was the case, I then administered the initial
version of the survey to a small number of in-service public elementary school teachers (N=22)
and recruited a subset of this group (N=4) for short cognitive interviews. (Cognitive interview
protocol available in Appendix B.) There were two main goals for this initial round of piloting:
(1) to assess the basic functioning of the survey items (e.g., Do participants understand what they
are being asked? Do the ranges for response options seem generally appropriate?) and (2) to
gather evidence about how respondents engage with each item via think-alouds.
62

I made minor changes to the survey instrument as a result of this initial piloting and the
four cognitive interviews I conducted, outlined in Appendix C. Cognitive interviewees’ think
alouds also served as evidence of the survey instrument’s content-related validity (e.g.,
Desimone & LeFloch, 2004), outlined in the Results section. After the first two cognitive
interviews, I found that interviewees tended to point out similar issues, so I stopped after the
fourth and updated items in accordance with interview findings.
At this stage, I also added a 20-item instrument for measuring “elementary teachers’
commitment to mathematics education reform” (Ross et al., 2003) and a four-item instrument for
measuring teachers’ overall “commitment to teaching” (Ware & Kitsantas, 2007) to the survey in
order to generate measures of constructs similar to but theoretically distinct from teacher
curriculum supplementation for each participant. These measures allowed me to comment on
evidence about the survey instrument’s discriminant validity. I motivate my decision to employ
these particular scales in the next section, then discuss findings related to discriminant validity in
the Results section.
The finalized survey, administered to 258 in-service public elementary school teachers,
ended with an invitation to share an email address to volunteer for an approximately 40-minute
follow-up study. Since this solicitation for email addresses was how I recruited participants for
the study in Paper 3, I discuss this in greater detail there. I used evidence collected from the
small initial round of piloting and the larger round with the finalized survey instrument to
construct arguments for the survey instrument’s ability to yield valid inferences about teacher
curriculum supplementation.
63

Generating Evidence for the Survey Instrument’s Ability to Produce Valid Inferences
To argue that the survey instrument can be used to produce valid inferences about teacher
curriculum supplementation (Research Question 1), I gathered evidence related to four essential
subquestions, each of which builds on the last:
1) Is teacher curriculum supplementation a phenomenon that we can and should measure?
2) Does the survey instrument measure something?
3) Does the survey instrument measure what I claim it does?
4) Does the survey instrument measure what I claim it does in an efficient and parsimonious
manner?
To address subquestion 1, I briefly summarized key ideas from the philosophy of
measurement literature about what makes something measurable, then used evidence from
existing curriculum use literature and from interview data to argue that teacher curriculum use
and supplementation have properties that make them measurable. I also used evidence about the
prevalence of teacher curriculum supplementation to argue that the practice is worth measuring.
To address subquestion 2, I used a factor analysis alongside descriptive statistics and
measures of internal scale reliability to argue that data generated by the survey instrument does
seem to measure something in consistent ways.
Addressing subquestion 3 was more involved. I developed my discussion of the factor
analysis from subquestion 2 to show that the data generated by the survey instrument were
generally (although not entirely) consistent with the predictions of the TCSF. I also presented
interview evidence to show that participants did “grapple with the intended dimensions” of
supplementation as they completed the survey instrument (Desimone & LeFloch, 2004). I
present these two strands of data as evidence of the survey instrument’s content-related validity,
64

since they illustrate the extent to which the instrument accurately captures the teacher curriculum
supplementation construct (Dupuis, 2018).
I also used correlation tables and scatterplots to describe the relationships between survey
subscales and respondents’ commitment to teaching, commitment to mathematics education
reform, and orientation toward their official curriculum material. This discussion built evidence
of the survey instrument’s discriminant validity, showing that it did not measure constructs from
which it was theoretically distinct (Ghaderi, 2018). I chose to relate the survey instrument’s key
subscales to teachers’ commitment to teaching because a skeptical reader might reasonably
believe that curriculum supplementation is simply an artefact of such commitment; the act of
seeking out resources beyond the official textbook might be considered extra work, only
attempted by the most committed teachers. I chose to relate the survey’s subscales to a measure
of commitment to math education reform because a skeptic might reasonably feel that even if
supplementation is not simply an indication of overall commitment to teaching, it might be
indication of commitment to a particularly hands-on, conceptual teaching style not always
prominent in elementary textbooks. I hypothesized that the relationship between various aspects
of teachers’ curriculum supplementation and those commitment scales will be zero or close to it,
indicating that supplementation is not a practice that simply follows from certain orientations
toward teaching.
The TCSF also maintains that teachers do not supplement in a vacuum. They often seek
additional resources based on some perceived shortcoming in their officially-adopted textbook.
Therefore, I also related the supplementation survey’s subscales to a measure of teacher’s regard
for their official material. I hypothesized that this relationship would be negative but small in
magnitude since many teachers may regard the official curriculum highly even as they
65

acknowledge that it may not fully serve every student. (This may not be true for a small minority
of teachers who do not have an officially-provided curriculum. Five of the 258 respondents in
this sample (1.9%) did not have an official curriculum.)
To address the subquestion 4, I assessed the extent to which the survey instrument’s
various subscales related to one another. While I did not hypothesize null relationships among
the subscales, I assessed the extent to which the subscales measured distinct aspects of
supplementation using correlations and visual inspection of scatterplots. In cases where two
subscales were so highly related that they might plausibly measure the same thing, I determined
whether to combine the subscales based on whether there was a strong theoretical distinction
between them or not. This process helped ensure that any reporting of survey results would be as
parsimonious as possible, with all subscales contributing information that was not redundant
with any other.
Analyzing Substantive Findings from the Survey
Since the analyses described above indicated that the survey tool was adequately reliable
and could produce valid inferences about teacher curriculum supplementation, I then briefly and
descriptively analyzed variation in the data at the item level, then investigated the
reason/source/use pattern/design feature combinations that appeared most commonly in the data.
Establishing evidence about which of these combinations appeared most commonly in real-world
supplementation could allow future work to focus on the most common forms of
supplementation and could eventually inform practice and policy by indicating to creators of
professional learning and curriculum which types of supplementation teachers are most likely to
need support with. I used UpSet plots (Lex et al., 2014) to visualize the ways teachers’ responses
across dimensions of supplementation varied and intersected. Of course, this analysis was based
66

on a nonrandom sample and was purely exploratory and descriptive; it could inform future
studies but should not be used to inform practice or policy on its own.
Results
Can and Should Teacher Curriculum Supplementation Be Measured?
For a measurement to be meaningful, it must provide information about the phenomenon
being measured that can be interpreted in the same way by different users of that information at
different times, according to well-defined rules (Finkelstein, 1975; Mari, 2013; Mari et al.,
2017). In short, the thing being measured must be stable enough that changes in measurements
can be interpreted as meaningful, not entirely idiosyncratic. Existing literature exploring the
ways teachers approach curriculum found that their approaches were relatively stable across time
and were informed by their backgrounds and contexts (Drake & Sherin, 2006). Teachers’
tendencies toward their school-provided curriculum sometimes changed as a result of targeted
intervention or new policies (Dietiker & Riling, 2018), but they tended to remain stable unless
changed. These approaches tend to be so stable that policy-adjacent work has adopted a strategy
of classifying teachers into curriculum use “profiles” as a way of better understanding their
practice (Kaufman et al., 2020). Certainly, then, these profiles are the result of “well-defined
rules” which yield information interpretable across users and time.
Little work has explored teachers’ approaches to supplemental curriculum materials in
particular, but interview data from this study indicated that teachers also supplement according to
stable patterns. This was true for supplement sources (e.g., “my go to definitely is slides because
it’s easier to whip something up super quick,” Observation and Interview Participant 3; “I always
go to Common Core Sheets first…and then K-5 Learning is almost always my second choice,
and then Teachers Pay Teachers and then YouTube,” Observation and Interview Participant 15),
67

for supplement use patterns (e.g., “every day, there’s an interactive video, there’s a solve and
share problem, there’s student workbook pages…and it probably is only one extra [activity, to
supplement],” Observation and Interview Participant 7; “when I do my small-group lesson, I
have to use the My Math lesson that they give me… I have other kids in the classroom that are
working on centers. And during that time, they are either on a chromebook doing a supplemental
material such as Splashlearn or there’s other games,” Observation and Interview Participant 12),
and more. Like general curriculum use, teachers’ supplemental curriculum use was not
idiosyncratic—it was stable enough over time that measuring it was a possibility.
Although measureable in principle, if teacher curriculum supplementation were an
unimportant phenomenon, measuring it would amount to little more than an arcane academic
exercise. However, recent national survey evidence has indicated that over 70% of teachers
regularly use online supplemental materials, with some estimates approaching 90% (Kaufman et
al., 2018; Tosh et al., 2020). While not administered to a representative sample, the survey
instrument under study asked about both online and offline supplementation, which others have
not. Under these conditions, 257 of 258 teachers in this study’s sample (99.6%) reported
supplementing their official curriculum at least some of the time. Teacher curriculum
supplementation is widespread enough that developing a more comprehensive understanding of
the process could yield insights relevant to the learning of virtually all U.S. students. Teacher
curriculum supplementation can and ought to be measured.
Does the Survey Instrument Measure Something?
When I ask whether the survey instrument measures “something,” I ask whether a
teacher’s responses to some of the items have any predictive value for their responses to other
items. A lack of such predictive capability would essentially mean that the survey data were
68

randomly distributed: I could not claim to have gained any particular insight into a teacher’s
supplementation practices based on their survey responses. On the other hand, if (for example)
teachers who most frequently Google for supplemental materials also most frequently visit
Teachers Pay Teachers for supplemental materials, then I could claim something about their
supplementation practice. I might claim that such teachers tend to find their supplemental
materials from online sources more often than other teachers.
To explore the extent to which survey responses “hang together” in ways that might
provide information about teacher curriculum supplementation, I performed an exploratory
factor analysis on the question set for each dimension of supplementation except for use pattern.
(Use pattern did not have its own question set, but rather shared items with the source
dimension.) Specifically, I carried out a principal components analysis with varimax rotation to
improve factor interpretability, retaining factors before the “elbow” in a plot of explained
variance on number of factors and retaining items with “moderately high” or higher factor
loadings, greater than 0.4 (Cattell, 1966; Watkins, 2018). At this stage, I was concerned more
with whether or not a factor structure emerged from the survey data and less with interpreting
that factor structure. (I interpret the factors in the context of the TCSF in the next section.) The
results of this factor analysis are presented in Appendix D.
The nine items related to supplement source broke down into two main factors, with three
items loading onto the first and two onto the second. Four of the nine items were unused. While
having 44% of the initial item pool go unused due to low factor loadings may seem high, it was
in line with other work developing scales from teacher surveys (Ware & Kitsantas, 2007). The
two factors explained 67% of variance in responses to supplement source items. (Again, I
interpret factors and comment on which items do not appear in any factors in the next section.)
69

The 17 items related to reasons for supplementation broke down into four factors, each
composed of three to four items, with five items going unused. Two items loaded onto multiple
factors, but those factors were substantively quite similar, so I combined them for analysis. (I
discuss this decision further in the next section.) The four factors explained 80% of variance in
responses to items related to reasons for supplementation.
The 17 items related to supplement features broke down into three factors, each
composed of between two and seven items, with five items going unused. The three factors
explained 72% of the variance in responses to items related to supplement features.
After performing the exploratory factor analysis, I plotted a histogram for each factor,
shown in Figure 2. (The factors are slightly modified from those that appear in Appendix D
based on substantive considerations described in the next section.) These histograms made clear
that there were respondents who fell along nearly the full scale for each retained subscale,
providing further evidence that the survey instrument was capturing information about
something.
70

Figure 2. Supplementation Subscales

Before even beginning to interpret factors in the context of the TCSF, an initial factor
analysis indicated that teachers’ responses were not random. Rather, the emergence of clear
factors indicated that teachers’ responses to certain items held predictive value for their
responses to other items. Additionally, there was considerable variation in respondents’ “scores”
on each factor. Therefore, we can be confident that the survey instrument measured something.
A skeptic might cede this point, but argue that the instrument does not measure teachers’
curriculum supplementation as I claim it does.
Does the Survey Instrument Measure Teacher Curriculum Supplementation?
Evidence from the Exploratory Factor Analysis
In reviewing existing supplementation-relevant literature to inform construction of the
TCSF, four dimensions stood out as essential to consider when looking to understand teachers’
decision making around their supplemental curriculum use. They were: teachers’ reasons for
supplementing, their supplement sources, their anticipated use patterns, and the supplements’
71

design features. Within each of these main dimensions, there were certain characteristics that
appeared especially prevalently. For example, when surveyed, teachers’ reasons for
supplementation tended to relate either to their students, their school contexts, or themselves. A
fuller set of these characteristics is presented in Table 3.
Table 3. Common Characteristics of the TCSF’s Four Main Dimensions of Supplementation
Dimension Characteristics Example Survey Item
Source “Unofficial” online Materials I got from another teachers’
personal blog or website
“Unofficial” offline Materials I created myself
“Official” Materials I checked out from a library

Reason Student-based I want a material that better engages my
students
Teacher-based I want a material that is easier for me to
use
School context-based I have to use a supplemental material
because my school does not provide all the
materials I need to teach the official
curriculum

Use Pattern Prevalence Think about all the materials you use to
teach math across the year. About what
proportion of those materials are
supplemental curriculum materials?
Frequency How often (times per year) do you use
each of the following sources to find
supplemental material?

Design Features Related to teacher usability The material is easily modifiable (e.g.,
word document vs. PDF)
Related to material’s quality The material comes highly recommended
“Fluffy” features The material is visually appealing overall

The contents of Table 3 provided much of my theoretical grounding as I constructed the
survey instrument: I included a battery of items for each of the TCSF’s four main dimensions
(although use pattern and source shared many items), and within each battery, I constructed the
specific items to touch on each dimension’s main characteristics. If these were meaningful
characteristics (i.e., if a teacher’s response on one item related to student-based reasons for
supplementation has predictive power for their responses to other items related to the same), the
72

exploratory factor analysis of the survey data would reveal a factor structure similar to Table 3.
Table 4 presents substantive interpretations of each retained factor from the factor analysis based
on the items retained in each.
Table 4. Interpretation of Factors as Subscales
Dimension Factor Interpretation Notes
Source 1 Unofficial online
sources
Subscales map approximately onto the
TCSF’s predictions, except that official
sources (items 8-9) did not load onto any
factor.
2 Unofficial offline
sources

Reason 1 Teacher-related
reasons
Factors 2, 3, and 4 had items in common, so
I combined them into a single student-related
reasons subscale. Context-related reasons
(items 16-17) did not load onto any factor.
2 Testing
3 Engagement
4 Life beyond school

Use Pattern (n/a) I interpreted responses to “what proportion
of all math materials are supplements?” as a
measure of “prevalence.” I computed
“frequency” from reported supplementation
frequencies across all items related to
supplement source.

Design Features 1 Usability for teacher In a slight departure from the TCSF’s
predictions, data suggest that “usability” may
in fact comprise two factors: material
usability in a vacuum and ease with which
the supplement can be integrated with
existing materials.
2 Integratability into
lessons
3 Quality/suitability
for students

In existing literature, teachers tend to source their supplements from unofficial online
sources, unofficial offline sources, and more-official sources. The two retained factors for items
related to source mapped closely onto two of those source categories: unofficial online sources
and unofficial offline sources, which I retained as subscales for use during substantive analysis.
The unused items were some of teachers’ least-frequently used sources, with between 10 and 25
uses per year, on average, compared to over 50 uses per year for the most-frequently used
sources. Each unused item loaded onto one of the factors that was not retained (items 3 and 4
onto factor 3 and items 8 and 9 onto factor 4), suggesting that these items may be theoretically
73

important despite being relatively uncommon. For this reason, I recommend keeping all nine
items related to source in the survey despite some not loading onto retained factors.
Supplementation-relevant literature suggests that teachers supplement for reasons related
to their students, their school contexts, or themselves. The first retained factor from items related
to reasons for supplementation was interpretable as teacher-related reasons, and while the factor
analysis indicated that student-related reasons loaded onto three factors, all items that loaded
onto these factors were student-related reasons and there were some items that loaded onto
multiple of these factors even after rotation (e.g., items 1 and 5), so it made sense to combine
these factors into a single subscale, interpretable as student-related reasons for supplementation.
Of the unused items, items 6 and 12 loaded onto rotated factor 5 and items 16 and 17 loaded onto
rotated factor 6. Neither factor was retained, but both could be interpreted as context-related
reasons for supplementation, so I recommend keeping those four items in future iterations of the
survey. This fact also suggests that the TCSF’s conception of teachers’ reasons for
supplementation as student-related, teacher-related, or context-related may be a useful one. Items
7 and 14 (lowering difficulty and price) did not load onto any rotated factor with loading greater
than 0.4, but both were frequently-mentioned reasons for supplementation in follow-up
interviews. For theoretical reasons, then, I recommend retaining all items related to reasons for
supplementation even if they do not all appear in the subscales I used during substantive analysis
of these results.
Use pattern is a particularly tricky dimension of supplementation since it can be
conceived in such a wide variety of ways. I took measures of two use patterns that earlier work
has shown to be meaningful: prevalence and frequency (Polikoff & Silver, 2021). Since that
earlier study explored and established definitions for these use patterns, I simply measured them
74

as they had been defined there: prevalence as the proportion of all math materials that are
supplements (measured with a single item that asked exactly that) and frequency as the number
of times per year a teacher supplements (measured by summing teachers’ responses to the
“supplement source” items, since the response scale for those items was in times per year). These
use patterns are distinct ways to think about supplement usage, so it does not make sense to
combine them. Therefore, I did not include use pattern in any factor analysis.
Existing literature suggests that teachers may particularly attend to supplement features
that speak to a material’s usability (for the teacher), its quality (for the student), or more “fluffy”
features (such as aesthetic appeal; Sawyer, Dick, & Sutherland, 2020). The factor analysis
indicated that this conception may need to be modified somewhat. The third retained factor was
interpretable as features related to quality of the material for students (e.g., cultural relevance,
recommendation of colleagues) but the first and second seemed to capture different classes of
features related to the usability of supplements. The first seemed to be usability in general (e.g.,
material accuracy, alignment to teacher style) and the second more specifically spoke to features
that would allow for easy integration into existing curriculum (e.g., ability to implement online,
similarity to official curriculum). It is possible that the sudden move to remote schooling and
need to adapt all materials for online instruction during the COVID-19 pandemic raised the
salience of inegratability in teachers’ practices, which could explain why it does not appear in
prior work but appears as its own factor here. Items related to “fluffy” features hung together
with general usability, but interview data shed potential light on this. A few teachers discussed
the importance of their materials’ visual appeal, but did so in the context of classroom
management: they felt that visually appealing materials would lead their students to misbehave
less since they would be more interested in the materials (e.g., Observation and Interview
75

Participant 7). I retained the factors related to usability, integratability, and material quality as
subscales for substantive analysis. Items 1 through 4 were unused but loaded onto rotated factors
4 and 5, the first two that I did not retain, so I recommend keeping those in the survey instrument
for now. Item 17 did not load onto any factor and was not a particularly salient feature for
teachers, but interview data suggested that teachers commonly thought about student grouping as
a use pattern for their supplemental materials (see Paper 3), so I recommend removing that item
the supplement features section of future survey iterations but adding topically similar items in a
separate section, to measure teachers’ anticipating student grouping patterns.
Following an exploratory factor analysis, the factor structure of the survey data (Table 4)
was quite similar, though not identical, to what would be predicted by the TCSF (Table 3). This
similarity served as evidence consistent with my claim that the survey instrument does measure
teacher curriculum supplementation, as outlined in the TCSF. Based on this analysis, I retained
two to three subscales for each dimension of supplementation to further explore in substantive
analysis of results. The subscales were: unofficial internet sourcing, unofficial offline sourcing,
student-related reasons, teacher-related reasons, supplementation prevalence, supplementation
frequency, features related to usability, features related to integratability, and features related to
supplement quality.
Evidence from Teacher Interviews
A more-straightforward way of building evidence for the survey instrument’s content-
related validity, of confirming that the survey instrument measures teachers’ curriculum
supplementation practices, was to simply ask the teachers completing it. When asked about how
well the survey gathered data about their supplementation practices, cognitive interviewees
reported that it was “great” (Cognitive Interview Participant 2) at gathering information about
76

their supplementation and that “it hit most of the points…that I use” (Cognitive Interview
Participant 3), although one interviewee wished the survey tool more clearly delineated between
curriculum supplementation and curriculum modification (Cognitive Interview Participant 4).
One remarked that the survey items “were appropriate and that perhaps I wouldn’t have even
considered [them]” (Cognitive Interview Participant 1), suggesting that the survey achieved full
coverage of the concept of teacher curriculum supplementation, perhaps even more deeply than
some teachers typically conceive of the practice.
All four cognitive interviewees reported that the survey neither felt repetitive nor felt like
it was missing important aspects of their supplementation practice. Although not a focus of the
follow-up interviews analyzed in Paper 3, these sentiments were echoed by those teachers. This
interview data should be taken as further evidence of the survey’s content-related validity, since
in-service elementary teachers felt that it fully captured their curriculum supplementation
practices.
Evidence from Theoretically Distinct Measures
The TCSF makes no a priori claims about how the dimensions of supplementation relate
to one another, but it does maintain that teacher curriculum supplementation is its own
phenomenon. I chose to measure respondents’ commitment to teaching, commitment to
mathematics education reform practices, and orientation toward their official curriculum
materials and relate them to the supplementation subscales for reasons outlined in the methods
section, above. I hypothesized that the first two would have no relationship with curriculum
supplementation and that the third might have a weakly negative relationship.
Commitment to Teaching Scale. The Commitment to Teaching scale (Ware &
Kitsantas, 2007; full text available in Appendix C) included four items about teachers’ overall
77

satisfaction with their job and their plans to remain in teaching. For this survey sample, the scale
had acceptable internal consistency (Cronbach’s alpha = 0.74) and an exploratory factor analysis
further suggested its unidimensionality. I analyzed the relationships between each of the nine
supplementation subscales from the survey instrument and the Commitment to Teaching scale
using simple correlations (Table 5, column 2). Only one of nine correlations reached significance
at the .05 level, none of the Pearson’s r values exceeded .20 in magnitude, and four of nine
relationships were negatively-signed, all of which make the case that there is no relationship
between teachers’ curriculum supplementation as measured by the survey instrument and their
overall commitment to teaching. This null relationship is visualized in Figure 3, which shows the
scatterplot corresponding to each correlation in the in the second column of Table 5. Visual
inspection of the point clouds does not suggest any particular relationship, and the red best-fit
lines are quite horizontal. Taken together, this suggests that curriculum supplementation is not
simply something that the most (or least) committed teachers do. It is a construct distinct from
commitment to teaching.
78

Figure 3. Relationships between Supplement Subscales and Commitment to Teaching Scale

Table 5. Relationships between Survey Subscales and Theoretically Distinct Constructs
Subscale Commitment
to Teaching
Commitment to
Mathematics Education
Reform Practices
Orientation toward
Official Curriculum
Materials
Source: unofficial online -0.056 0.132 -0.082
Source: unofficial offline -0.012 0.159* -0.091
Reason: student-related 0.137 0.192* 0.088
Reason: teacher-related 0.064 -0.012 -0.021
Use pattern: frequency -0.019 0.131 -0.023
Use pattern: prevalence -0.01 0.032 -0.178*
Feature: usability 0.097 0.299* 0.202*
Feature: integratability 0.153* 0.157* 0.355*
Feature: quality 0.14 0.181* 0.222*
All values are Pearson’s r; *p<.05

Commitment to Mathematics Education Reform Scale. The Commitment to
Mathematics Education Reform scale (Ross et al., 2003; full text available in Appendix C)
included 20 items related to teachers’ perceived importance of conceptual understanding, peer
collaboration, real-world problems, and other hallmarks of strong math pedagogy. In this survey
sample, the scale had acceptable internal consistency (Cronbach’s alpha = 0.78), and while an
exploratory factor analysis suggested multiple factors within the scale, this result appeared to be
79

due to reverse-coded items rather to multidimensionality. Table 5’s third column summarizes the
relationships between this scale and each subscale from the curriculum supplementation survey
instrument. These relationships are visualized in Figure 4. Five of the nine relationships reached
significance at the .05 level, but only one had Pearson’s r with magnitude greater than 0.2. No
obvious relationship between commitment to mathematics education reform practices and the
teacher curriculum subscales emerges from visual inspection of the scatterplots in Figure 4, with
the exception of the relationship with feature: usability subscale, which seems to have a clear
positive relationship. Based on this analysis, teachers who consider features related to a
supplement’s usability especially highly may be especially committed to mathematics education
reform practices, but the relationship between this commitment and the other subscales is either
small in magnitude or null. This suggests that teacher curriculum supplementation as measured
by this survey instrument does not simply follow from a commitment to best teaching practices.
It is its own phenomenon.
Figure 4. Relationships between Supplement Subscales and Commitment to Mathematics
Education Reform Scale

80

Orientation toward Official Material Scale. I created a short, 6-item scale to measure
teachers’ orientation toward their officially-adopted curriculum material, which asked about their
perceptions of the official material’s academic quality, ease of use, and their overall satisfaction
with it (full text available in Appendix C). The scale had high internal reliability (Cronbach’s
alpha = 0.90), and an exploratory factor analysis further confirmed its unidimensionality. Five of
the nine relationships between this scale and the subscales related to curriculum supplementation
(Table 5, column 4) were negative and most were small in magnitude (Pearson’s r < 0.2). Figure
5, which visualizes these relationships, confirms this. However, all three subscales measuring the
extent to which teachers weigh various design features of supplements in their practice related
positively and significantly to teachers’ orientation toward their official material, such that
teachers who regarded their official material more highly also considered features related to
supplement usability, integratability, and quality for students to a greater extent in their decision
making. These positive relationships (especially those for usability and integratability) are also
clearly visible in Figure 5.
81

Figure 5. Relationships between Supplement Subscales and Orientation Toward Official Material
Scale

The fact that any relationships were positive runs counter to what I hypothesized about
supplementation resulting from some shortcoming in the official material. Speculatively, perhaps
this greater attention to supplement features arose precisely because of a high regard for the
official textbook: maybe teachers who believed their official textbook was poor believed that
most any supplemental material would be an improvement, so did not attend as much to the
specific features of supplemental materials. On the other hand, perhaps these materials had a
higher bar to clear for teachers who already believed their official materials were fairly good, so
those teachers may have been more discerning in their decision-making. I did not investigate this
directly in this study, but it could be a useful question for future work to investigate.
Despite a few positive relationships of moderate magnitude (Pearson’s r between 0.2 and
0.35), the relationships between orientation toward the official material and each
supplementation subscale are not consistent enough in direction or magnitude to imply that
82

supplementation is purely a function of a teachers’ orientation toward their officially-adopted
curriculum material.
Evidence from a variety of sources (the subscales themselves, teacher participants,
relationships between the subscales and theoretically distinct constructs) and gathered via a
variety of methods (factor analysis, interview, bivariate correlation) suggested that this survey
instrument could produce valid inferences about teachers’ curriculum supplementation practices.
In particular, it gathered information related to teachers’ supplement sources, reasons for
supplementation, supplement use patterns, and supplement features, as outlined in the TCSF.
Does the Survey Instrument Measure Curriculum Supplementation Parsimoniously?
A last step in arguing that the survey is well-suited for measuring teachers’ curriculum
supplementation is to argue that it did so usefully, without redundancy. Every subscale should
add information about teachers’ curriculum supplementation that the others do not already
provide. I analyzed the bivariate relationships between each survey subscale using methods
similar to those for analyzing the relationships between subscales and theoretically-distinct
constructs, above. These relationships are summarized in Table 6 and visualized in Figures 6-8.
Because these subscales measure different dimensions (or different characteristics within the
same dimension) of the teacher curriculum supplementation construct, I expected that many
relationships would be significant and moderate or large in magnitude. The purpose of this
analysis was to flag any relationships among subscales that were so large in magnitude that one
might argue the two subscales provided redundant information.
At first glance, the relationships between both source subscales and the use pattern:
frequency subscale stand out as particularly large in magnitude, but these relationships are
mechanical—I computed respondents’ frequency of supplementation using their responses to the
83

source items. After those, the only two relationships with Pearson’s r greater than 0.5 were
between theoretically similar constructs (features related to usability and integratability, features
related to quality for students and student-related reasons for supplementing). The majority
(61%) of relationships had Pearson’s r < 0.3, suggesting that although many of these subscales
are related (they do purport to measure different aspects of the same construct, after all), they did
not provide redundant information.

84

Table 6. Relationships Among Survey Subscales

Source:
unofficial online
Source: unofficial
offline
Reason:
student-related
Reason:
teacher-related
Use pattern:
frequency
Use pattern:
prevalence
Feature:
usability
Feature:
integratability
Source: unofficial offline 0.294*
Reason: student-related 0.217* 0.191
Reason: teacher-related 0.158 0.093 0.474*
Use pattern: frequency 0.776* 0.688* 0.273* 0.231*
Use pattern: prevalence 0.333* 0.362* 0.268* 0.257* 0.423*
Feature: usability 0.165 0.15 0.406* 0.195 0.185 -0.036
Feature: integratability 0.159 0.082 0.395* 0.322* 0.205* 0.075 0.511*
Feature: quality 0.146 0.064 0.503* 0.312* 0.195 0.13 0.415* 0.4*
All values are Pearson’s r; *p<.05

85

Figure 6. Relationships between Source Subscales and Reason, Use Pattern, and Feature
Subscales

Figure 7. Relationships between Reason Subscales and Use Pattern and Feature Subscales

86

Figure 8. Relationships between Use Pattern Subscales and Feature Subscales

Substantive Findings from the Survey Instrument
Satisfied that the survey instrument supported valid inferences about teachers’ curriculum
supplementation and did so in ways that were not redundant (Research Question 1), I
investigated a few substantive findings from the survey data (Research Question 2). As
previously noted, these findings should be treated with caution since this study’s sample was a
nonrandom one, albeit one that was observably similar to the population of U.S. elementary
school teachers.
To advance our understanding of teachers’ curriculum supplementation under the TCSF,
we must develop (1) better understanding of which characteristics are most common within each
dimension of supplementation and (2) how those characteristics tend to co-occur across
dimensions (Paper 1).
I investigated which characteristics commonly occur within dimensions in the course of
providing validity evidence, finding that unofficial online and unofficial offline sources were
87

common supplement sources, that teacher- and student-related reasons were common reasons for
supplementation, and that features related to usability, integratability, and quality were types of
features that teachers commonly attended to when supplementing (Table 4). I retained each of
these as a subscale, used to analyze co-occurrence among these characteristics. In some cases, I
recommended retaining items in the survey instrument that did not align with any of these
characteristics because they aligned with theoretically important characteristics (Table 3) or
because they loaded onto factors with eigenvalues large enough that they might have plausibly
been retained in a different (larger or more representative) sample.
Item-level Findings
Although I investigated co-occurrence of characteristics at the subscale level, I also
computed descriptive statistics for each item in the survey, with means displayed in Figures 9-11
(source, reason, feature) and distributions in Figure 12 (use pattern). The data in Figure 9
indicated wide variation in teachers’ sourcing of supplemental curriculum materials. Teachers
reported sourcing supplemental materials from a library about once per month (9.1 times per
year), compared to sourcing from a website dedicated to supplemental materials such as
Teachers Pay Teachers approximately every three days (53.3 times per year). Some of the most
common forms of supplement use happened offline (“already owned” and “created myself”),
underscoring the importance of including supplementation that occurs outside virtual resource
pools in empirical work, as I discussed in Paper 1.
88

Figure 9. How Often Do You Use Each of the Following Sources to Find Supplemental
Materials?

Averaged across teachers, virtually all listed reasons for supplementation factored into
teachers’ decision making to a moderate extent (Figure 10). Increasing student engagement and
addressing academic needs were the reasons that weighed on teachers’ supplementation
decisions to the greatest extent, in line with existing empirical findings (e.g., Marple et al., 2017).
Teacher-related considerations such as price, ease of use, and prep time weighed into
supplementation decisions to approximately the same extent that more academic considerations
such as alignment to standards and assessments and to a greater extent than “real-world”
considerations like cultural relevance or topicality, lending further credence to the idea that
supplementation may be, but is not necessarily, academically motivated.
89

Figure 10. To What Extent Are Each of the Following Reasons a Factor When You Decide to
Use a Supplemental Curriculum Material?

Most of the design features in the survey instrument factored into teachers’
supplementation decision making to between a moderate and large extent (Figure 11). The
features most heavily attended-to were split between those clearly linked to a supplemental
curriculum material’s academic merits (standards alignment, accuracy) and those less obviously
so (visual appeal, clarity, alignment to teaching style). Some of the teachers’ least-considered
features were recommendations about the resource, either from their personal colleagues or from
teachers they didn’t know, a finding consistent with the idea that teachers may approach
curriculum use as a very personal, highly contextualized endeavor (Ball & Feiman-Nemser,
1988; Drake & Sherin, 2006). The only other features that weighed on teachers’ decision making
around supplementation to a less-than-moderate extent, on average, were a resource’s online
implementability and its cultural relevance, suggesting either that teachers did not consider these
to be particularly important features of curriculum or that teachers considered their official
90

curriculum to do relatively well in these areas, so they had less reason to attend to these features
when seeking supplements.
Figure 11. To What Extent Are Each of the Following Features an Important Consideration
When You Look for or Create a Supplemental Material?

Only two items related to the use pattern dimension, and each measured a different use
pattern. Figure 12 displays distributions of responses for each use pattern. On average, half of
teachers’ math materials were supplemental, with a bell-shaped distribution of supplementation
prevalence around that mean. At the extremes, one teacher reported that none of their materials
were supplemental and 3 reported that all of them were. Teachers reported supplementing 287
times per year (about 1.6 times per day, assuming a 180-day school year), although this mean
was affected by a long right tail, with 12 teachers reporting supplementing 720 or more times per
year, or more than once per hour, assuming four hours of academic instruction over the course of
a typical 6.5-hour school day. At the other extreme, only one teacher reported never
supplementing, and just four more reported doing so less often than every two weeks (fewer than
20 times per year).
91

Figure 12. Use Patterns

It was tempting to delve more deeply into these item-level findings by teacher subgroup
(e.g., Which teachers supplemented most? How experienced were those who reported that ease
of use was their most-important reason for supplementing?), but I worried that such an analysis
could give an illusion of more precision and confidence that I had in these data, given the non-
representative nature of this sample. For that reason, I confined my discussion of item-level
findings to the very high-level one above. But addressing item-level variation by teacher
subgroup is an important task for future work with a larger, randomized sample.
I explored co-occurrence among the most salient characteristics of teacher curriculum
supplementation using UpSet plots, ideal tools for visualizing and understanding sets that
intersect across many dimensions. The plots were essentially a way of clearly conveying the
information contained in a Venn diagram with 9 different circles, allowing me to comment on
whether, for example, those who frequently supplement from online sources also frequently
92

supplement for teacher-related reasons and have a high prevalence of supplements and attend to
features related to usability to a particularly large extent.
Co-occurrence of Characteristics Across Dimensions of Supplementation
Table 6 provided some insight into the co-occurrence of supplement sources, reasons, use
patterns, and design features in these data, but because it relied on bivariate correlation, it was
limited to two dimensions. To analyze the intersections of more than two characteristics of
supplementation simultaneously, I generated an UpSet plot (Lex et al., 2014), displayed in
Figure 13. To start, I created indicators for whether each teacher was “high” (above the sample
median) on each of the nine supplementation subscales that emerged in the factor analysis, listed
in Table 5.
The UpSet plot displays sets of teachers who were above the median for intersecting sets
of subscales. The bars at left show how many respondents were “high” on each survey subscale,
ranging from 82 to 127. These numbers vary because differing numbers of teachers fell exactly
on the median for different subscales. The connected dots indicate the different intersections and
the bars across the top indicate how many participants fell into each intersecting set. For
example, eight participants fell above the median on the “features: quality” subscale only (first
column and bar) and another eight fell above the median on all nine subscales (third column and
bar). The UpSet plot in Figure 13 visualizes every intersecting set with more than one member.

93

Figure 13. Intersecting Sets of Supplementation Dimensions and Their Characteristics
94

The subscales related to the source and feature dimensions appeared in more intersecting
sets together than the subscales related to the reason dimension. This indicates that teachers who,
for example, supplemented most frequently from internet sources were reasonably likely to also
be teachers who supplemented most frequently from offline sources, but that teachers who
supplemented for teacher-related reasons to an above-median extent were not as likely to also be
teachers who supplemented for student-related reasons to the same extent.
Teachers who reported supplementing for student-related reasons to a greater extent than
others did not have a tendency for sourcing from online versus offline venues, but tended to lean
on supplement features that spoke to quality to a greater-than-median extent and tended not to be
those with above-median supplement prevalence among their math materials. We might consider
these teachers to be particularly choosy or conscientious supplementers based on their reasons
for supplementation, sourcing, features attended to, and supplement use pattern.
The most frequent supplementers were about evenly split between those who were high
on teacher-related versus student-related reasons for supplementation, but attended to
supplement features related to general usability more than those related to integratability or
academic quality. One potential interpretation of this use pattern / feature / reason intersection is
that these teachers may have supplemented as a way to get something in front of students, but
were less choosy about what that something was.
As noted in discussion of bivariate relationships among subscales (Table 6), teachers who
highly considered supplement features related to integratability also tended to highly consider
those related to general usability. Teachers who considered these features highly also tended to
have teacher-related reasons to supplement and use unofficial internet sources for their
supplemental materials. We might surmise that teachers with this reason / use pattern / feature
95

intersection are those about whom education leaders should worry. Perhaps they tend to
supplement to make their jobs easier and use lower-quality online sources more frequently than
their peers.
Examination of the survey data in this way would flow naturally into an analysis that
attempts to create different supplementation “profiles,” a direction that studies of general
curriculum use have productively pursued in recent years (Kaufman et al., 2020). The above
paragraphs were examples of the types of profiles that might emerge from such an analysis, but
as with the univariate analysis above, I did not continue the analysis beyond this relatively high,
illustrative level due to the nonrandom sample that informs the data.
In a study with representative data and ideally a larger sample, a useful next step would
be to assign teachers to these and other profiles, then explore which profiles were most prevalent
among which teachers. This last step is where the work truly has policy application, since
decision-makers could, for example, target professional learning to those groups of teachers who
tend to supplement in ways they want to change or discourage. This compelling policy
application is precisely why I chose not to continue this analysis on non-representative data—
highlighting some actionable-seeming finding but then caveating it with a warning that one
should neither form a conclusion on it nor act on it due to the nonrandom sample seems a tall
ask, even for the most responsible readers of research.
Limitations and Future Directions
I was limited in my ability to comment on substantive findings from the survey data
presented here, given the nonrandom teacher sample. This nonrandomness was a necessary
constraint given my available resources, but future studies in this area should endeavor to use
teacher samples randomly selected from the population of interest (for this study, U.S. in-service
96

elementary teachers of math) to support valid inferences from survey findings to that population.
Failing that, future work should recruit a large enough samples of teachers that survey responses
can be convincingly weighted for representativeness, at least along observable characteristics.
Between this study and the follow-up interviews discussed in Paper 3, it became clear
that there were small changes I would recommend making to the survey tool for future uses as
well. First, I recommend adding an item to the supplement features section about supplement
usability for parents, which came up in one follow-up interview. Second, I recommend adding an
item to the supplement sources section about sourcing from school supply stores, which came up
in one follow-up interview. Third, as describe in the Results section, I recommend removing the
item about student grouping from the supplement features section. Aside from these three minor
changes, Paper 3’s follow-up interviews suggested that this survey only partially measured the
use pattern’s teachers commonly consider when supplementing their official curriculum
materials in practice. I recommend adding items to measure anticipated supplement use patterns
related to student grouping and timing. Any recommendation of a specific item to this survey
instrument would require another round of cognitive testing (which I have not completed), but
preliminary draft versions of items designed to measure the constructs discussed in this
paragraph are presented in Appendix A.
After making the relatively small changes outlined above, future empirical work using
this survey instrument should focus on generating supplementation “profiles” from commonly
co-occurring combinations of source/reason/use pattern/design features, as discussed in the
Results section. Such work might then relate these profiles to characteristics of the school,
classroom, and teacher to build understanding of what kinds of supplementation tend to occur in
which contexts. Such findings might inform the design of professional learning tools aimed at
97

shifting teachers’ supplementation practices, perhaps moving teachers from profiles deemed less
likely to contribute to student success to those deemed more so. Relatedly, future research might
relate supplementation profiles generated by data from this survey tool to student-level academic
and/or social outcome measures, building understanding of how difference supplementation
styles relate to student success. Such an understanding could inform, for example, district and
school policies designed to encourage certain supplement sources or use patterns and might
provide insight into the topics and exercises that official curriculum materials should strive to
include in future editions. These are just a few suggestions for future research directions; teacher
curriculum supplementation as a phenomenon and a practice is little-explored relative to its
prevalence in U.S. classrooms, so empirical inquiry in virtually any direction could yield useful
insight.
Discussion and Conclusion
The main goal of this paper was to argue that the survey instrument under study can
produce valid inferences about teacher curriculum supplementation. I argued first that teacher
curriculum supplementation is measurable in principle, then showed that the survey instrument
was consistently measuring something. I then used interview data, an exploratory factor analysis,
and bivariate correlations to argue that what the survey instrument measured was, in fact, teacher
curriculum supplementation. I then provided brief evidence that the information it provided was
not redundant, that for future administrations, the survey instrument should stay largely as
administered here. I believe I presented convincing arguments for the validity of inferences
yielded by this instrument.
A secondary goal of the paper was to explore, at a high level, the supplementation
practices of the sample of teachers used to generate validity evidence. I illustrated how these
98

explorations might begin but refrained from carrying them to their full conclusions because I did
not want readers to mentally apply findings based on a nonrandom, non-representative selected
sample to the population from which the sample was selected. Still, these analyses indicated
reasonable variation in teachers’ item-level responses and supported the idea, prominent in the
TCSF, that teachers consider more than just academics in their supplementation decision making.
Analysis across survey subscales indicated that developing supplementation profiles as a way to
study teachers’ curriculum supplementation practices might be a fruitful next step, with clear
policy applications.
As the first theoretically-grounded survey tool designed specifically to measure teacher
curriculum supplementation, I believe this survey instrument could be a boon to the growing
field of research centering this phenomenon. However, another straightforward way of building
understanding about teachers’ curriculum supplementation processes is to simply observe them,
in real time. Compared to survey data, data collected in this way could yield greater insights into
how teachers weigh the various dimensions of supplementation against one another and could
inform the content of potential professional development tools aimed at improving teachers’
supplementation skills. Paper 3 explores these possibilities.

99

Paper 3: Observing Teachers’ Curriculum Supplementation
As outlined in the introduction to these three studies, teacher curriculum supplementation
is a relatively new focus of academic inquiry. Even those studies that focus at least in part on
supplementation tend to employ incomplete conceptions of the practice, leading to ways of
measuring or accounting for supplementation in empirical work that fail to capture all important
dimensions of the practice. These inconsistent conceptions of supplementation as a phenomenon
entail impoverished measures of supplementation as a practice.
I have attempted to provide a more complete conception of teacher curriculum
supplementation as a phenomenon (Paper 1) and to bolster our understanding of supplementation
as a practice (Papers 2 and 3). Paper 2 presented a novel survey measure of supplementation,
since having a common way of measuring real-world supplementation is one important way to
better understand how it occurs in practice. Another way of building understanding of teacher
curriculum supplementation is to observe it occurring firsthand, rigorously and systematically
reporting on those observations. This study adopts that strategy.
To date, most empirical evidence about teacher curriculum supplementation comes from
two strands of studies. First, there are survey studies of supplement consumers (teachers; e.g.,
Kaufman et al., 2020) or supplement producers (teacherpreneurs; e.g., Shelton & Archambault,
2019). These may provide useful insight into their processes, but are necessarily mediated
through the teacher or teacherpreneur respondent. Additionally, many of these studies center
virtual curriculum use as opposed to curriculum supplementation, so relatively little is known
about how teachers supplement when they do not source their materials from large virtual
resource pools, and almost nothing is known about the process when it happens entirely offline.
100

A second strand of supplementation-relevant study analyses the properties of
supplemental materials themselves, typically coding the materials using machine learning (e.g.,
Hu et al., 2018) or qualitative methods (e.g., Hu et al., 2019; Polikoff & Dean, 2019; Sawyer et
al., 2019). These studies generally aim to discuss the educational or cultural value of the
materials or the online marketplace they occupy rather than commenting on teachers’
supplementation processes.
This study combines the aspirations of the first strand with the methods of the second,
setting out to analyze the act of supplementation directly and in-the-moment rather than after the
fact. We recorded 15 in-service public elementary math teacher participants thinking aloud
during an approximately 40-minute session during which they supplemented their officially-
adopted curriculum materials in a simulated lesson planning session, then discussed their thought
process in an interview immediately afterward. Although the lesson planning session in this
study was simulated, participants consistently attested that their decisions during the simulation
closely mirrored their practice outside of simulated situations, as discussed in more detail in the
Results section. We then applied qualitative content analysis with deductive category
assignments based on the Teacher Curriculum Supplementation Framework (TCSF; see Paper 1)
to the recordings of these sessions (Mayring, 2000, 2015) to address two research questions:
1. What considerations do in-service elementary math teachers have as they supplement
their official curriculum, and to what extent are these considerations consistent with
existing theories of curriculum use and supplementation, including the TCSF?
2. What insights do teachers’ supplementation processes yield about teacher curriculum
supplementation as a phenomenon and as a practice?
101

We start by reviewing the supplementation-relevant literature to demonstrate that
supplementation is not a phenomenon exclusively tied to large virtual resource pools, despite
much of our empirical knowledge being tied to these sources due to the methods commonly used
in this literature. This study is designed to account for teachers’ supplementation processes
whether or not they make use of virtual resource pools. We then give a few examples of studies
from the wider curriculum literature that make use of observation methods, which may allow for
deeper insights into individual teachers’ supplementation processes than currently-popular
survey methods and are agnostic as to teachers’ supplement sources, as a proof of concept for
this work. Substantively, this study will expand understanding of the process of teacher
curriculum supplementation and provide an early test of the extent to which the TCSF maps onto
teachers’ day-to-day practice. Methodologically, this study is among the first to use recorded
observations of teachers’ lesson planning processes alongside in-depth, follow-up interviews to
study supplementation, a data source that holds promise moving forward.
Literature Review
This study employs a method not yet in wide use in the supplementation-relevant
literature. We now review the methods in current use across this literature to illustrate how
qualitative content analysis of recorded observations of teachers supplementing could fill gaps in
our current understanding.
Teachers Report Supplementing from Non-Virtual Resource Pool Sources
A number of teacher survey studies focused on curriculum use include questions about
teachers’ sources for curriculum materials. For example, a 2017 survey of 2,148 elementary and
middle school math teachers across ten districts found that 32% reported using official, district-
adopted materials in instruction, but 45% reported using unofficial, teacher-developed materials
102

(Perry et al., 2017). While 14% of the sample reported using teacher-developed materials
sourced from Teachers Pay Teachers and another 5% reported using Pinterest (both large virtual
resource pools), a full 22% reported using materials they developed themselves. Note that these
proportions were not mutually exclusive—the same respondent could have reported using a
district-adopted textbook and self-created materials and materials sourced from Teachers Pay
Teachers.
These findings are consistent with other survey studies. One found that 26% of its sample
of 257 upper elementary math teachers sourced their supplements from online (no distinction
between virtual resource pools and other online sources was made) and 23% (again, not mutually
exclusive) used self-created supplements (Polly, 2017). Another found that more than 90% of
surveyed elementary (97%) and secondary (99%) math teachers reported using “materials I
developed and/or selected myself,” proportions very similar to reported use of “materials
developed and/or selected by my district” (98% elementary and 92% secondary; Opfer et al.,
2017). These studies are useful for highlighting how teachers supplement from a variety of
sources, many of which are not virtual resource pools. However, large-scale surveys tell us little
about teachers’ individual supplementation processes, including their specific considerations as
they supplement. The observation methods employed in this study generate insights into these
processes, which may reveal considerations beyond the scope of a typical survey, such as teacher
beliefs and past experience.
Another limitation of existing survey studies of curriculum use is that many restrict their
investigation to online curriculum, so do not even provide an option for teachers to indicate that
they create materials themselves or source them from other offline sources. These surveys of
online curriculum use have consistently found that large virtual resource pools, while commonly-
103

used, are not the only online source teachers use for supplementation. A survey of 601
elementary math teachers found that 89% reported regularly using Teachers Pay Teachers to find
supplemental materials, but 68% reported just Googling to find materials (Shapiro et al., 2019).
A nationally-representative survey of math teachers found that 60% used Teachers Pay Teachers
weekly or more to find materials compared with 64% who did the same with Google (Kaufman
et al., 2018). Again, these surveys of online supplementation provide useful overviews of what
sources teachers use, but are often limited in scope (e.g., missing offline sources) and tend not to
ask questions designed to build deep understandings of individual teachers’ supplementation
processes.
Many Studies Focus Only on Supplementation from Virtual Resource Pools
Focusing only on large virtual resource pools like Teachers Pay Teachers, Pinterest,
TFANet, and others misses a large swath of teacher curriculum supplementation in practice. In
this section, we summarize what is known about teachers’ supplementation processes from the
current, virtual resource pool-focused literature. For a more comprehensive review of this
literature, see Paper 1.
Survey data indicate that teachers have a wide variety of reasons for seeking
supplemental materials, including those connected to their students (e.g., to meet the needs of
English learners; Sawyer, Dick, Shapiro et al., 2020), to themselves (e.g., to save time in lesson
planning; Polikoff & Dean, 2019) or to their school contexts (e.g., their school is missing some
material necessary to teacher the official curriculum with fidelity; Marple et al., 2017). It is likely
that teachers who supplement in non-virtual resource pool contexts do so for similar reasons,
although we know of no study that has investigated this comparatively. Additionally, even when
survey studies do generate data related to teachers’ supplementation processes (e.g., asking about
104

their common reasons for supplementation), it is unlikely that they ask with enough nuance to,
for example, capture the full range of context-dependent reasons a teacher might supplement. By
observing and then discussing teachers’ supplementation processes in an interview format, the
present study is designed to achieve more of this nuance, possibly yielding insights beyond what
survey methods can easily support.
There is also evidence that, even restricted to research from virtual resource pool sources,
teachers select their source purposefully, in line with what they hope to accomplish by
supplementing. For example, one survey study found that teachers generally use Pinterest to
curate “boards” of curriculum resources, so this source may be especially useful to teachers
looking to maintain a set of supplemental materials for use over time (Schroeder et al., 2019). In
contrast, survey results have indicated that teachers often use Twitter as a sounding board for
resource ideas (Trust et al., 2016), which may be especially useful when teachers want to
generate lots of ideas in a short period of time. Teachers have highlighted Teachers Pay Teachers
as convenient and easy-to-use in retrospective interviews (Irvine, 2015), so this virtual resource
pool may be especially useful when teachers are strapped for time. It is reasonable to imagine,
although not yet documented in the literature, that teachers may choose non-virtual resource pool
sources for specific reasons as well. For example, a teacher might choose to create a material
themselves in cases where they feel like they have special expertise relative to others who might
make materials available for download online.
One study using national survey data found that multiple distinct supplement use patterns
exist, predicted by distinct sets of teacher-, school-, and textbook-level characteristics (Polikoff
& Silver, 2021). Another survey study found that pre-service teachers’ understandings of the
affordances of YouTube affected the ways in which they tried to use it to supplement their
105

lessons (Krauskopf et al., 2011). A critical content analysis of pre-service teachers’ class
assignments found that they tend to be academically and culturally uncritical about materials
they download for use (Rodriguez et al., 2020). In the more general curriculum use literature,
some have used survey data to classify teachers’ approaches to curriculum use in terms of how
prominent a place their official materials occupy in their pedagogy (Kaufman et al., 2020).
There has not been much empirical work that focuses on the design features of
supplemental resources specifically, but there is not much reason to think that such features (e.g.,
presence of images, color versus black and white illustrations) would function differently for
supplemental resources relative to officially-adopted ones. However, some design features are
specific to, or at least probably more prevalent among, supplemental resources. For example,
back-end download data from virtual resource pools indicated that teachers attended heavily to
features like resource metadata (i.e., ratings and comments; Abramovich & Schunn, 2012;
Abramovich et al., 2013). Case studies have found that general visual appeal or “cuteness” of the
resource (Gallagher et al., 2019; Grote-Garcia & Vasinda, 2014) affected decisions about
supplemental resource use. It is currently unclear which design features are most important to
teachers when supplementing from sources other than virtual resource pools, but because this
study is not explicitly focused on virtual resource pools, it may shed light on this.
Teachers Processes of Curriculum Use
The above sections show that our understanding of teachers’ processes of
supplementation is limited, both because the supplementation-relevant literature tends to focus
on virtual resource pools above other sources and because it tends to use methods such as
surveys that do not allow researchers to directly observe supplementation occurring. These
methods are useful for understanding teachers supplement sources, reasons, and use patterns in a
106

broad sense, and those that include nationally- or state-representative samples may be especially
relevant to policy discussion. However, these methods tend not to provide in-depth, granular
understandings of individual teachers’ processes. By relying on observation and interview
methods, this study’s data could contain insights that existing, largely survey-based studies may
not have been set up to detect.
But outside the relatively small body of supplementation-relevant literature, researchers
have used observation methods to study teachers’ general curriculum implementation processes.
It is beyond the scope of this paper to provide a comprehensive review of the curriculum use
literature (see Remillard, 2005 or Kim, 2021 for such reviews), but we will summarize a few of
the most influential of these studies in order to provide a sense of the sorts of understandings that
have been gained from observing teachers interact with their officially-adopted curriculum
materials. This should serve as evidence of the value of the current study, which applies
observation methods to teacher curriculum supplementation.
Brown (2002) observed three middle school science teachers’ curriculum use for ten
weeks, conceiving of curriculum use as a design process characterized by “a dynamic interaction
between elements of the curriculum materials and teachers’ knowledge, goals, and beliefs”
(Brown, 2002, p. 442). Under this conception, teachers showed varying “pedagogical design
capacities,” which accounted for “similarities and differences in teacher practices” (Brown,
2002, p. iv). When interacting with curriculum materials in this study, teachers’ instructional
decisions were informed by specific features of the materials, by teachers’ own reasons for using
the materials, and by their understandings about the materials. These considerations parallel a
few of the dimensions highlighted in the TCSF as essential to understanding teachers’
107

supplemental curriculum use, suggesting that it may be a useful framework for understanding
teachers’ decision making during curriculum supplementation (e.g., Research Question 1).
In a case study of two elementary math teachers, including observations of lesson
planning and instruction, Drake and Sherin (2006) found that the teachers had stable approaches
to curriculum, informed by their own early educational experiences and by their beliefs about
mathematics instruction. This work was later expanded into the Curriculum Strategies
Framework (Sherin & Drake, 2009), which described how teachers’ “reading, evaluating, and
adapting” activities differed before, during, and after delivering instruction. In-depth observation
of teachers’ instructional planning time uncovered these important insights about their use of
official curriculum, and there is no reason to believe in-depth observation of supplementation
would not yield similarly rich insights (e.g., Research Question 2).
Remillard (1996) observed the ways two elementary math teachers interacted with their
textbooks, concluding that differences in what sections the teachers read and their purposes for
reading them during lesson preparation contributed to different student learning opportunities.
This work was expanded into a model for teacher “curriculum development,” which emphasized
how teacher and textbook interactions during three stages—curriculum design, curriculum
construction, and curriculum mapping—can affect the instruction that students ultimately receive
(Remillard, 1999). In contrast to Remillard’s 1996 finding, Choppin (2011) found that that three
middle school math teachers’ purposes for interacting with their textbooks were quite similar,
and attributed differences in instruction to differences in what they noticed about student
thinking and how they responded to it. Although they emphasized aspects of curriculum
enactment, both researchers observed the instructional planning process and found evidence that
considerations related to the textbook, the teacher, and/or the teacher’s context informed
108

instruction. The present study presents evidence gathered from observation of teachers’
curriculum supplementation processes, yielding insights around teachers’ considerations when
supplementing their official curriculum.
Based on the studies summarized above, it is clear that directly observing teacher
interactions with curriculum can yield rich theoretical and practical findings about how, when,
and why teachers make the decisions they do with regards to their officially-adopted textbooks.
This study sets out to do the same, focused on supplemental curriculum materials.
Sample
At the end of the survey that was the focus of Paper 2, we offered participants the
opportunity to provide a contact email address to be a part of this follow-up study, with
compensation of $20 if selected. In efforts to maximize participants’ backgrounds and
supplementation styles, we purposively sampled 15 participants based on their survey responses
to set up a Zoom call to carry out this study (Maxwell, 2013). A sample size of 15 in-service
elementary math teachers is somewhat larger than many of the curriculum use case studies
outlined above, but is in line with other studies in education that employ similar methods
(Glaeser-Zikuda & Mayring, 2003; Glaser-Zikuda & Fuß, 2008).
Our sampling method ensured that we had representation from teachers above and below
the median for: years of experience teaching at their grade level, supplementation frequency,
teacher-related reasons for supplementation, and student-related reasons for supplementation.
We included grade-level experience in our sampling strategy because it has been shown to relate
to supplementation strategies in earlier work (Schroeder et al., 2019). We included the other
three because they are key axes along which teachers differ within the TCSF (Paper 2). We also
attempted to recruit as many non-white teachers as possible, but due to the heavily white sample
109

of participants who provided an email address in the initial study (approximately 81% white),
only two of the fifteen participants (13%) in this study were teachers of color. Although these
proportions of non-white teachers may be small, they are similar to the proportion of non-white
elementary teachers nationally (National Center for Education Statistics, 2019, Table 209.22).
These and other key characteristics are listed by participant in Table 7.
110

Table 7. Participant Characteristics
Participant 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Average
Grade taught 3 2 4 3 3 4 1 3 5 5 5 K 3 1 4 .
Region MW MW NE NE S S NE W S W MW S S W W .
Is white? 1 1 0 1 1 1 1 0 1 1 1 1 1 1 1 87%
Is Female? 1 1 0 1 1 1 1 1 1 0 1 1 1 1 1 87%
Classroom > 50%...
English learners 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 13%
Students with disabilities 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0%
Students of color 0 1 0 0 1 0 0 1 0 0 0 0 0 0 1 27%
At/above full sample median for…
Grade-level experience 1 0 0 1 1 1 1 1 1 1 0 0 0 1 0 60%
Supplementation frequency 1 0 0 0 1 1 0 1 1 1 0 1 1 0 0 53%
Student-related reasons 0 0 1 0 1 0 1 1 0 1 0 1 1 1 1 60%
Teacher-related reasons 1 0 0 0 0 0 0 0 1 0 0 1 1 1 1 40%
CTT scale 1 0 1 1 0 1 0 1 0 1 0 1 1 1 0 60%
CMER scale 0 1 0 1 1 1 0 1 0 1 0 1 0 0 1 53%
OTOM scale 1 0 0 1 0 0 0 0 0 1 0 1 0 1 0 33%
School expects curricular fidelity? 0 0 0 1 0 1 1 0 1 1 1 1 0 1 1 60%
Note: CTT is the Commitment to teaching scale (Ware & Kitsantas, 2007); CMER is the Commitment to Mathematics Education Reform
scale (Ross et al., 2003); OTOM is the Orientation Toward Official Material scale. See Paper 2 for discussion of each.

111

In May and June of 2022, we observed and interviewed at least one teacher in each
elementary (K-5) grade, with the majority (N=11) teaching grades 3-5. The teacher participants
came from each of the four US census regions, with five from the South, four from the West and
three each from the Northeast and Midwest. The majority of teachers were women, and few
taught in classrooms with a majority of English learners (N=2), students with disabilities (N=0),
or students of color (N=4).
The teachers we observed and interviewed were slightly more experienced that the
typical teacher in Paper 2’s full survey sample and had slightly higher scores on the Commitment
to Teaching and Commitment to Mathematics Education Reform scales (from Ware & Kitsantas,
2007 and Ross et al., 2003, respectively; see Paper 2 for more scale details). These teachers
tended to fall above the full survey sample median for supplementation frequency and extent of
student-related reasons for supplementation, and fell below the full-sample median for extent of
teacher-related reasons for supplementation. Nine of the fifteen teachers taught at schools that
expected them to teach with fidelity to the officially-adopted curriculum to a moderate or great
extent, although the majority were cooler-then-median toward their official materials (see Paper
2 for details on the Orientation Toward Official Materials scale).
Taken together, then, the typical teacher participant in this study was a relatively
experienced white female teacher in fairly privileged school setting who supplemented often and
was less-than-enthusiastic about their official curriculum, all compared to the larger sample of
teachers who took the supplementation survey from Paper 2. Although attempting to sum up the
participants like this might provide a useful first glance at this group, there was important
diversity in how teachers approached and practiced curriculum supplementation, as we discuss in
the Results section.
112

Method
Researcher Positionality and Motivation
Daniel Silver is a white male graduate student who taught kindergarten and third grade
for five years to primarily non-white English learner students and supplemented his curriculum
materials heavily during that time, mainly creating his own materials designed to improve his
official materials’ academic quality and to better engage his students. These identities and
experiences afford him unearned privileges and inform his interest in supplemental curriculum
materials and the ways in which teachers enact them in the classroom, with special focus on how
supplementation may or may not promote equity.
Morgan Polikoff is a white male professor whose recent research has centered
supplemental curriculum sources and use patterns. He is interested in the factors that contribute
to teachers’ supplementation strategies and in how curriculum can be harnessed to promote
equity in and across classrooms. He has studied standards implementation for over a decade,
recently authoring a book about standards-based reform policy and the role of curriculum
materials.
Data Collection
We contacted each selected participant to organize a Zoom call, outlining the broad goals
of the study and letting them know the call would include a 20-minute simulated
supplementation activity followed by a discussion about the activity and their practice more
generally. During this initial contact, we previewed that we would record video of their screen
alongside audio of our interaction. The text of this introductory email is available in Appendix E.
For those who did not respond to this initial email within approximately 72 hours, we selected
another teacher with similar supplementation and background characteristics who had provided a
113

contact email address. In total, we contacted 96 teachers who had indicated their willingness to
participate in order to obtain our final sample of 15.
We began each call by reviewing key points from the recruitment email: the teacher
would think aloud during the first part of the call while screen sharing, we would record and take
notes, then we would pause them after 20 minutes so we could have a more typical interview in
the second part of the call. After reviewing this structure, we would then present the teacher with
a simulated task for them to spend up to 20 minutes working on. We emphasized that the priority
was to get as authentic a glimpse into their practice as possible, so it was completely fine if they
did not complete the task before the 20 minutes were up. We had to pause three teachers at 20
minutes, and the other twelve took between 2 and 18 minutes to finish the task. For all teachers,
the task was:
Imagine that your students are struggling with [skill as described in Table 8].
You decide to supplement your official, school-adopted curriculum to help them with this
concept, either by finding or creating resources for your students to use.

In the interest of gathering authentic information about teachers’ supplementation
practices, the academic skill featured in the task varied according to teachers’ grade level. We
did not want teachers to need to familiarize themselves with new content as part of this task. To
facilitate comparability across tasks, though, the academic skill was always taken from the
teacher’s grade-level Numbers and Operations in Base Ten Common Core Standard 1 (NBT.1).
We selected this particular standard for two reasons. First, this standard is considered a
grade-level “anchor” standard, absolutely essential to math at that grade level, by virtually all
schools and districts at all grades because it involves foundational skills related to number sense.
Therefore, the likelihood that any teacher put off teaching content in NBT.1 for time reasons was
extremely small. (None of the fifteen we observed and interviewed had.) Second, the content of
114

this standard across elementary grades lends itself to an especially diverse array of pedagogical
approaches, from “kill and drill” worksheets to real-life scenarios to highly conceptualized
lessons. To free teachers to supplement in their preferred styles, we designed the task to be as
non-prescriptive as possible while remaining realistic. (For example, the 3.NBT.1, the third
grade iteration of this standard, involves rounding to the nearest ten and hundred. To address it,
teachers might provide their students worksheets with numbers to round, emphasize conceptual
understanding of rounding using a number line or base ten blocks, ask real-world questions that
emphasize the utility of rounding—“How many kids do you think go to our school?”—or any
number of other strategies.)
Table 8 provides the official NBT.1 text and our adaptation of that text for the
supplementation simulation task for each K-5 grade level. Although not every state adheres to
the Common Core standards, even states that never adopted Common Core have revised their
standards to be quite similar (Garland, 2016), and those that adopted and later repealed Common
Core tended to leave theirs quite similar, with the repeal more of a rebrand than a deep policy
change (Goldstein, 2019). Even participants in non-Common Core states had no trouble carrying
out the simulated supplementation task.
Table 8. Skills in Supplementation Simulation Prompts, by Grade Level
Grade NBT.1 Text Prompt text
Imagine that your students are
struggling with…
K CCSS.MATH.CONTENT.K.NBT.A.1
Compose and decompose numbers from 11 to 19 into ten ones and
some further ones, e.g., by using objects or drawings, and record each
composition or decomposition by a drawing or equation (such as 18 =
10 + 8); understand that these numbers are composed of ten ones and
one, two, three, four, five, six, seven, eight, or nine ones.

…decomposing numbers
between 10 and 20 into a ten
and some number of ones
1 CCSS.MATH.CONTENT.1.NBT.A.1
Count to 120, starting at any number less than 120. In this range, read
and write numerals and represent a number of objects with a written
numeral.

…representing numbers of
objects between 50 and 120
with written numerals
115

2 CCSS.MATH.CONTENT.2.NBT.A.1
Understand that the three digits of a three-digit number represent
amounts of hundreds, tens, and ones; e.g., 706 equals 7 hundreds, 0
tens, and 6 ones. Understand the following as special cases:
CCSS.MATH.CONTENT.2.NBT.A.1.A
100 can be thought of as a bundle of ten tens — called a
"hundred."
CCSS.MATH.CONTENT.2.NBT.A.1.B
The numbers 100, 200, 300, 400, 500, 600, 700, 800, 900 refer
to one, two, three, four, five, six, seven, eight, or nine
hundreds (and 0 tens and 0 ones).

…understanding that three-
digit numbers are composed of
hundreds, tens, and ones
3 CCSS.MATH.CONTENT.3.NBT.A.1
Use place value understanding to round whole numbers to the nearest
10 or 100.

…rounding whole numbers to
the nearest 10
4 CCSS.MATH.CONTENT.4.NBT.A.1
Recognize that in a multi-digit whole number, a digit in one place
represents ten times what it represents in the place to its right. For
example, recognize that 700 ÷ 70 = 10 by applying concepts of place
value and division.

…recognizing that each place
in a multi-digit number
represents ten times what the
place to its right represents
5 CCSS.MATH.CONTENT.5.NBT.A.1
Recognize that in a multi-digit number, a digit in one place represents
10 times as much as it represents in the place to its right and 1/10 of
what it represents in the place to its left.
…recognizing that each place
in a multi-digit number
represents 1/10 of what the
place to its left represents

While participants thought aloud about their supplementation process, we took notes on
key decision points, thought processes, resources used, and any other element of their practice to
surface in the interview. When the participant expressed that they were finished or after 20
minutes, whichever came first, we carried out a semi-structured interview designed to dig deeper
into their decision making. The interviews were loosely structured according to the protocol in
Appendix E, but we commonly asked probing questions outside the protocol based on teacher-
specific issues or considerations that surfaced during the observation.
Importantly, due to a technological issue, the first observation and interview was not
recorded. We still wrote a detailed memo afterward, so we were able to include information from
the first observation and interview in general assertions about the group (e.g., all fifteen
participants found the simulation to be authentic to their practice). However, the bulk of this
study’s findings are based on the other fourteen coded transcripts. Participant 1 did not differ
much from the other fourteen participants on observable characteristics, as shown in Table 7.
116

Data Analysis
Once an observation was complete, we wrote a short one- to two-page memo containing
immediate reflections to both record those reflections and deepen our analytic understanding of
each teacher’s supplementation process (Maxwell, 2013). We used the automated transcription
service Otter.ai to transcribe the audio from each recording, then extensively reviewed each
transcript to ensure accuracy to the recording, to anonymize any instance where teachers referred
to themselves by name (changing their name to “[NAME]”), and to note (also using square
brackets) instances where teachers’ on-screen actions lent additional meaning to the transcript.
For example, Observation and Interview Participant 2 said “Oh, let’s be me” when testing out a
game, which only made sense with the on-screen context that the game had prompted her to
choose an avatar, at which point she chose one she had already made for herself. We modified
the transcription to, “Oh, let’s be me [chooses avatar].” These modifications were infrequent
since teachers’ think-alouds tended to be quite comprehensive.
Once all observations were completed, we applied qualitative content analysis to the
observation transcripts to address the research questions, as described below. For an overview of
the qualitative content analysis method, see Mayring, 2014 and Mayring, 2015.
Despite its name, qualitative content analysis is a mixed methods approach to
systematically drawing meaning from a text. The method is an effort to preserve some of the
strengths of traditional content analysis, such as the emphasis on systematic and rule-bound
procedures and the focus on categories during analysis, while addressing its shortcomings, such
as being limited to readily-quantifiable text features or aspiring to some objective and context-
neutral “truth” (Mayring, 2015). Qualitative content analysis is especially attractive for this study
because its highly systematized and rule-governed nature allows for ready comparability across
117

all teacher supplementation observations. Additionally, these features make typical qualitative
content analyses more reproducible than analyses using related methods (i.e., discourse analysis),
which is useful since this study is among the first to apply the method in the context of teacher
curriculum supplementation.
The method has been broken down in a number of ways: Elo and colleagues (2014)
describe it in three phases: preparation, organization (analysis), and reporting, whereas Schreier
(2014) breaks the method down into seven steps, beginning with choosing a research question
and ending with presenting findings. Rather than breaking down the method temporally, Mayring
(2014) identifies eight characteristics that should be present in any qualitative content analysis.
In the following paragraphs, we describe each and explain how we kept each present during data
analysis.
Eight Characteristics of Qualitative Content Analysis
First, qualitative content analyses embed the text “within the communicative context”
(Mayring, 2014, p. 39), with the researcher always interpreting the text as a form of
communication from the participant. Concretely, we kept in mind contextual features such as
school friendliness toward supplementation and teacher years of experience at their current grade
level when coding, in an effort to understand both surface and latent meanings from the
supplementation observations.
Second, analyses must follow a rule-bound, though not necessarily immutable, procedure.
These rules appeared concretely in a few ways. Prior to coding, we specified that the coding
units for this analysis (the smallest units to which we would assign a code) was a sentence, the
context unit (the largest unit to which we would assign a code) was a paragraph, and the
recording unit (the amount of text to which we would apply a system of codes at one time) was
118

one transcribed observation and interview. A second way this analysis was rule-bound was in the
way we specified and coded categories. Coding in qualitative content analysis can proceed either
inductively (data-first) or deductively (theory-first; Selvi, 2019), and in this case we followed a
deductive coding procedure, where the TCSF informed the categories we started with. To define
categories and subcategories to code, we followed the four steps described by Selvi (2019): we
named the category, described the category, provided an archetypal example of the category
from the data, and specified decision rules to clarify the edges of the category. The coding frame
was a fluid tool and we updated it quite substantially throughout data analysis. Any time we
updated or altered the coding frame, we reanalyzed all data, ensuring that all observations were
coded consistently. See Appendix F for our starting and final coding frames, annotated with key
modifications made over the course of coding.
Third, coded categories are the focus of a qualitative content analysis. By staying away
from general, holistic impressions of large chunks of text, we improved comparability across
interviews and repeatability of the analysis while ensuring that all conclusions were concretely
supported in the text (Mayring, 2014). Part of the mixed-methods approach to this analysis
shines through here, since once we coded all observations, we used descriptive statistics to
summarize findings across the full set of observations and to compare supplementation styles
between observations.
The fourth characteristic is “object reference,” which maintains that qualitative content
analysis is not simply a series of steps to be applied to any text—the technique must be adapted
to the object of analysis itself. We maintained object references when, for example, we found
that our initial coding frame (which contained about 25 codes organized into three levels) did not
allow us to capture the nuance contained in the transcripts in enough granularity to analyze
119

confidently. We added a fourth layer of approximately 55 additional subcodes nested within the
existing codes and recoded, which allowed us to analyze the data productively.
Fifth, qualitative content analyses arrive at their non-standardized instruments (such as
coding frames aligned to the coded objects) via testing on the data to be analyzed. Mayring
(2015) recommended revising the instruments of analysis after analyzing 10-50% of the texts, so
after completing coding for three observations (20%), we stepped back and revisited all the
coding categories we had used. We made a number of revisions to the coding frame at this stage,
including adding the fourth layer of subcodes. While our method permitted us to update codes at
any time, this formal step ensured that the most important changes happened early in the coding
process.
Sixth, qualitative content analysis must be theoretically-guided. As explained above, our
coding and other analytic decisions were grounded in the TCSF (Paper 1), a framework designed
expressly for understanding teachers’ curriculum supplementation.
Seventh, qualitative content analysis integrates quantitative analyses where doing so is
sensible. As mentioned above, we employed descriptive statistics both to summarize, by
describing frequencies of categories across all observations, and to compare, by describing how
these frequencies of categories differed between observations.
Eighth, qualitative content analyses attend to specific quality criteria to ensure
trustworthiness. To start, we attempted to foreground any biases we may bring to the work by
discussing our own positionalities as researchers at the beginning of the Methods section. We
also carefully documented our method and instruments, including the changes we made to them
throughout analysis, allowing for greater reproducibility and giving readers the opportunity to
judge the validity of our choices. To bolster reliability, the second author independently coded
120

three randomly-chosen transcripts. We agreed on 96% of codes across transcripts, with Cohen’s
Kappa = 0.95, indicating near-perfect agreement (Burla et al., 2008; Landis and Koch, 1977). We
discussed each instance of disagreement in our coding and came to a decision about the best
codes for each sentence, as recommended in Mayring (2014).
Summary of Analysis Plan
The above section broke down our analytic plan in terms of how it accorded with the
characteristics of qualitative content analysis. Here, we also summarize it in the order we carried
it out.
Once we had transcriptions of all recorded observation/interviews, we randomly selected
an order in which to code them and randomly selected three for secondary coding. We began by
watching the video and reading the transcript for each observation/interview in its entirety,
updating the transcript with key on-screen actions and immersing ourselves in the data (Selvi,
2019). We then used the coding frame, grounded in the TCSF and presented in Appendix F, to
assign codes and subcodes to each sentence in the transcript. We added or modified codes as the
data required and noted any changes we made. We went back and recoded earlier transcripts
following changes to the coding frame, so there was never a situation where some transcripts
were coded one way and others were coded another.
After completing this process for the first three transcripts, we stepped back to reflect on
the content-analytic decisions we made related to units of analysis and to categories and
subcategories, updating them as needed. We did not update the unit of analysis (the sentence was
an appropriate coding unit) but made changes to the coding frame, documented in Appendix F.
We then coded the remaining transcripts. We allowed ourselves to continue updating content-
121

analytic rules as needed after the first three, but did not need to make additional changes to the
coding frame or unit of analysis.
After the first author completed primary coding, he sent all relevant materials to the
secondary coder (the second author), and after discussing the coding frame together, the
secondary coder completed coding for the three randomly-selected transcripts using the same
process. We largely agreed on codes (inter-coder reliability = 96%) and discussed areas of
disagreement to arrive at a consensus.
Once coding of all observation and interviews was complete, we described the various
categories and subcategories across and between observations using descriptive statistics and
discussed how the observed categories aligned with existing conceptions of curriculum use and
supplementation (Research Question 1). We then described key patterns about teachers’
supplementation processes that emerged from these data and discussed theoretical and practical
implications of each (Research Question 2).
Results
Before describing specific findings from the observation and interview sessions, it is
important to comment on the trustworthiness of the simulated supplementation task. Instructional
observations have been employed to generate data about teachers’ practices (e.g., Gargani &
Strong, 2014), but observation of the lesson preparation process, let alone the curriculum
supplementation process, is rarer. Although the purpose of this study is not to generate
conclusions generalizable to the overall elementary teacher population, it is important that its
findings do capture teachers’ authentic curriculum supplementation practices. If they did not, this
study would have little to contribute to theoretical or practical understanding of teacher
curriculum supplementation.
122

Credibility and authenticity are two key components of qualitative research quality which
speak to the extent to which a study is “isomorphic to some reality” (Lincoln & Guba, 2005,
p.178) and “describes the reality of the participants” (Toma, 2011, p. 14). All fifteen study
participants attested that the simulation task was quite easy for them because it was similar to
their day-to-day lesson planning practice., underscoring the task’s credibility as a way to gain
insight into teacher curriculum supplementation. A few of their statements about this authenticity
are presented in Table 9. Many participants also mentioned that they would use one or more
resources found during the simulation task in their classrooms, underscoring the task’s
authenticity (also presented in Table 9). The fact that teacher participants found the
supplementation simulation to mirror their own practice so closely suggests that this method
could be productively employed in future study of teacher curriculum supplementation.
Table 9. Selected Statements about the Credibility and Authenticity of the Supplementation Simulation
Participant Statement
2 And so I bookmarked it. I'm not doing any more lesson planning this year. I got two days left, but
for next year.

6 Oh, super easy! When I got your email, I was like, “This is what I do all the time. Like, why
not?” you know?

8 I think because my process is so similar for all subjects, it's natural. It's not a big deal.

11 It was pretty easy and pretty similar. This would be my normal process. I'd done my little
teachers pay teacher's downloaded, printed, hope that it all came out, and carried on.

13 It's very similar to what I usually do. And honestly, I talk to myself all the time during planning
in class, like when my kids are out at specials. So yeah, this is normal.

15 This actually might come in handy next year because you never know, my students might need it.

Research Question 1: Participants’ Considerations While Supplementing
Satisfied that the supplementation simulation produced trustworthy information about
teachers’ supplementation processes, we now address the first research question, about teacher
participants’ considerations during supplementation and the extent to which those considerations
are consistent with the TCSF and other theories of curriculum use. We present numeric
breakdowns of these considerations, using quantitative analysis to better understand qualitative
123

observation and interview data, in accordance with the mixed-methods nature of qualitative
content analysis. We present quotations from individual participants where especially illustrative,
but the focus of this section is on the prevalence of various considerations across the fourteen
coded transcripts. We discuss individuals’ supplementation processes in greater depth during
discussion of Research Question 2.
At a high level, over 90% of teachers’ on-topic statements (i.e., excluding statements
clarifying the task or solving hiccups with technology) across the 14 coded transcripts were
related to one of the dimensions of supplementation outlined in the TCSF (Table 10). The
remaining statements were related to either the teacher’s official material or to the fact that they
had used some resource before. There were no statements that necessitated the formation of
some other dimension beyond the four initially laid out in the TCSF, although it was not
uncommon for a statement to necessitate the expansion of an existing dimension. For example,
we identified more different use patterns in the transcripts (e.g., timing, student grouping) than
we had accounted for in earlier work (Polikoff & Silver, 2021). This suggests that the TCSF is
useful as a comprehensive framework for understanding teacher curriculum supplementation, but
should not be regarded as immutable.
Table 10. High-Level Topic Coverage during Observations and Interviews
Topic Number of Codes % of All Codes
Discussion of any TCSF dimension 1123 93%
Dimension: feature 429 36%
Dimension: reason 381 32%
Dimension: source 207 17%
Dimension: use pattern 106 9%
Dimension: other 0 0%
Discussion of official material 44 4%
Discussion of previously-used resource 32 3%

Teachers discussed their reasons for supplementation and specific features of
supplemental resources much more often than they discussed their supplement sources or
124

anticipated use patterns (Table 10). This suggests that supplement features may be more salient
considerations to teachers than, for example, anticipated use patterns during supplementation.
However, we interpret the fact that participants mentioned each dimension frequently (features
were discussed approximately every 80 seconds during each 40-minute observation and
interview, on average, and even use patterns were discussed about once every five minutes) as
evidence that each dimension was a salient consideration as teachers supplemented their official
curriculum materials.
Considerations about Supplement Features
Teacher participants discussed supplement features related to the supplement’s overall
quality approximately twice as often as features related to a supplement’s usability for
themselves (58% versus 30% of statements about supplement features; see Table 11) and more
than five times as often as features related to a supplement’s usability for their students (11% of
statements about supplement features).
Table 11. Supplement Feature Topic Coverage
Topic Number of Codes % of Feature Codes
Discussion of any design feature 429 100%
Feature: quality 249 58%
Learning 73 17%
Engagement and fun 52 12%
Aesthetics 40 9%
Other quality feature 84 20%
Feature: usability for teacher 128 30%
Time/money/convenience 94 22%
Other teacher usability feature 34 8%
Feature: usability for students 49 11%
Feature: other 1 <1%
Note: Some categories have been combined or collapsed for readability. See Appendix F for
complete list of categories.

Within features related to overall supplement quality, teachers in this study mentioned
supplement features that contribute to student learning 40% more often than features that
contribute to student engagement (Table 11), in contrast with earlier findings that teachers’ most-
125

frequent reason for supplementation is to engage their students (e.g., Webel et al., 2015). There
was a range in how heavily teachers considered such features. Over 20% of Participant 6’s coded
statements highlighted feature that contribute to student engagement, with statements like, “She
has excellent PowerPoints that are engaging, they’re interactive, and they always have a theme
and they’re cute.” But Participant 6 did not once mention supplement features in terms of their
contributions to student learning. On the other hand, Participant 9 did not mention a single
feature as contributing to student engagement, but over 16% of her coded statements highlighted
features that contribute to student learning, with statements like, “It’s not just watching a video
and then doing an activity. It’s walking them through how to do things using the base ten blocks,
and then trying to make that connection that the ones place is one-tenth smaller than the tens
place or one-hundredth smaller than the hundreds place.” These disparities suggest that some
teachers attended mainly to features that contribute to student learning while others attended to
features that contribute to student engagement. Unfortunately, Table 7 suggests little observable
that might explain this difference: both Participants 6 and 9 were relatively experienced white
female teachers in the South with similar classroom compositions who supplemented often but
were relatively warm toward their official curriculum. (For additional selected quotations about
supplement features from all participants spotlighted in this section, see Table 12.)
Table 12. Illustrative Statements about Supplement Features
Features that contribute to
student engagement
A lot of our curriculum stuff is just a
worksheet, just a plain worksheet. And
there's only so much of that that they can
do before they just want to tear their hair
out, you know? So I try to make it more
fun and more engaging for them.
(Participant 6)
And if I was going to teach that over the
course of a week, I might walk them
through one of these worksheets and
then dig around and find something
that's more engaging, like some type of a
flipbook or something where they make
something. (Participant 6)

Features that contribute to
student learning
And usually it has a video, several videos,
and exercise and activity, so that you can
see if they really understand the
prerequisite standard of the grade level.
(Participant 9)
And the other thing I like about it is they
give you hints, math helps to help you
with, and so if I get to a kid that
struggles, granted if they're below grade
level, I wouldn't give them an on grade
level task, I would differentiate for them.
126

But if their “helps” didn’t help, then they
can contact an imagine math teacher.
(Participant 9)

Features that contribute to
teacher convenience
But the game that I saw on Teachers Pay
Teachers, it looked like you just needed
dice and that sheet and all I needed to do
was copy it. So that would be a game that I
would use because I don't have to prep as
much. (Participant 15)

And it's free, so that's like the best part.
(Participant 15)
Aesthetic features Since I've become a TPT seller and not just
a TPT buyer, I have found that your
images and the amount of time that you're
putting into your product is what people
really want to see. (Participant 12)
Well, this is terrible, but the first thing I
look for is it has to look good. If it's not
pretty, then I'm not... You'll find stuff on
there sometimes that’s, like, made in
Word or something, you know? And I'm
like, “No! No, nobody cares about that!”
(Participant 6)

In contrast to existing conceptions of curriculum use that paint supplementation as a
primarily student-driven phenomenon (e.g., Gallagher et al., 2019), teacher participants
considered supplement features that could save them time, money, or simply make their jobs
easier more often than features related to student learning, nearly six times per observation and
interview (Table 11). Again, there was a range in how frequently such features were considered:
over 15% of all of Participant 15’s coded statements related to these types of convenience
features whereas Participant 4 did not consider them at all. In general, teachers with less grade-
level experience attended to convenience features more often, which underscores how important
it is for empirical work to attend to factors about a teacher’s background and experiences to
understand their supplementation practices, consistent with recommendations of the TCSF.
Just under 10% of all statements about supplement features were related to non-
functional aesthetic features (visuals and formatting that contributed to student usability or
academic understanding had separate codes). Other work has warned that supplemental materials
from large online virtual resource pools such as Teachers Pay Teachers may privilege aesthetics
over academic substance (Sawyer et al., 2020), and results suggested that some teachers attended
to aesthetics over academics at times. Between 11% and 12% of all of Participant 6’s and 12’s
127

coded statements related to resources’ non-functional aesthetic qualities (compared to 0% and
4% about features that contribute to student learning, respectively). Both were heavy users of
Teachers Pay Teachers (Participant 12 also sold on the platform) who worked in schools that
encouraged fidelity to the official curriculum material, suggesting that simply mandating that
teachers not use low-quality supplemental materials is not sufficient to ensure that such materials
do not enter the classroom. However, six of the fourteen participants with coded transcripts did
not mention non-functional aesthetic supplement features even once, and many of them were
frequent TPT users as well. Curriculum policies as heavy-handed as mandates or bans are
unlikely to account for the nuanced ways in which teachers attend to supplement features over
the course of curriculum supplementation.
Considerations about Reasons for Supplementation
Besides discussing features of specific supplemental resources, teacher participants also
frequently discussed their general reasons for either supplementing at all or supplementing in a
certain way. In general, these reasons mirrored their discussion of supplement features. Teachers
cited student-related reasons for supplementation (45% of statements about reasons for
supplementation) slightly more often than teacher-related reasons (40%) and much more often
than reasons related to broader contextual factors (13%; Table 13).
Table 13. Reasons for Supplementation Topic Coverage
Topic Number of Codes % of Reason Codes
Discussion of any reason 381 100%
Reason: student-related 172 45%
Learning 100 26%
Engagement and fun 38 10%
Other student-related reason 34 9%
Reason: teacher-related 151 40%
Time/money/convenience 90 24%
Teacher background/identity 25 7%
Other teacher-related reason 36 9%
Reason: context-related 49 13%
128

Reason: other 9 2%
Note: Some categories have been combined or collapsed for readability. See Appendix F for
complete list of categories.

Improving student learning overall was the most-discussed individual reason (26% of
statements about reasons for supplementation), followed closely by convenience (e.g., making
teaching easier, saving time or money; 24%), with improving student engagement a distant third
(10%). Some even presented curriculum supplementation almost as a form of self-care:
Teachers need to start thinking about themselves. I mean, it’s okay to be a little selfish.
You’re gonna give it back, you’re a teacher, you love kids, you’ll give it back. But you
got to know what you’re capable of and what your weak spots are. (Participant 10)

In line with conceptions of teacher curriculum use that highlight highly personal aspects
of the practice (Drake & Sherin, 2006), eight of the fourteen coded transcripts contain at least
one reason for supplementation related to a teacher’s identity or background. Some teachers use
identity to justify supplementation sourcing decisions (Participant 11 shied away from Teachers
Pay Teachers because “I’m not that cutesy and funsy”) and others used their identity as lifelong
learners to motivate their supplementation overall:
If I don’t keep trying new things, I’m not going to keep learning and that goes against
everything I believe in. So I’m not going to use the same materials year after year. I’m
constantly going to look for updated things. (Participant 2)

Although six teacher participants did not mention identity or background as a motivator for their
supplementation, this may indicate a reticence to share pieces of their identity with an
interviewer they had just recently met rather than low salience of identity to their
supplementation processes.
Considerations about Supplement Sources
Just over half of teachers’ statements about supplement sources referred to unofficial
online sources, compared to one third that referred to unofficial offline sources, and the
remainder that referred to “official” (but not school- or district-required) sources such as
129

standards unpacking documents (Table 14). Every teacher mentioned at least one source of
supplementation, underscoring findings that teacher curriculum supplementation is ubiquitous
(Kaufman et al., 2018; Tosh et al., 2020). However, the fact that online sources made up only
about half of teacher participants’ mentioned unofficial online sources despite the vast majority
of supplementation-relevant studies focusing on such sources points to a need for empirical work
to better account for offline supplementation.
Table 14. Supplement Source Topic Coverage
Topic Number of Codes % of Source Codes
Discussion of any source 207 100%
Source: online unofficial 108 52%
Teachers pay teachers 35 17%
Pinterest 2 1%
Other virtual resource pool 13 6%
Worksheet generating website 13 6%
Other unofficial online source 45 22%
Source: offline unofficial 67 33%
Teacher’s own resource pool 26 13%
Self-created 23 11%
Other unofficial offline source 18 9%
Source: “official” 29 14%
Source: other 3 1%
Note: Some categories have been combined or collapsed for readability. See Appendix F for
complete list of categories.

In line with findings about its market dominance (Koehler et al., 2020), teacher
participants cited Teachers Pay Teachers as a supplement source more than any other, with 35
references to the massive virtual resource pool (17% of all statements about supplement sources),
compared to 4 references to Education.com and 3 to Khan Academy, the second- and third-most-
popular virtual resource pools. Despite being one of the most-studied virtual resource pools (e.g.,
Schroeder et al., 2019; Liu et al., 2020), Pinterest was only mentioned twice, and both mentions
were about how the teacher either does not like or no longer uses Pinterest. The popularity of
different supplement sources may vary across time and teacher samples, so studies that restrict
130

analysis to only a single source may risk becoming less-useful in cases where that source falls
out of favor among supplementing teachers.
Participant 6 explained the dominance of Teachers Pay Teachers in terms of the quality
of its resources, “It’s nice that you know teachers are making those materials. So they feel you,
they know what you’re looking for.” Participant 7 highlighted the convenience of “do[ing]
downloads easily” as key to its popularity. Participant 11 praised both the quality of “teacher-
created” resources and the platform’s convenience when she needed a resource quickly: “I’m
like, ‘God bless your free stuff,’ because that saved me I can’t tell you how many times.”
Participant 3 had a warier take on that convenience, though: “I try to save money. So Teachers
Pay Teachers is very addicting, and I’ve been down that road. And I still do use Teachers Pay
Teachers.”
After Teachers Pay Teachers, teacher participants’ next most frequently-mentioned
supplement sources were their own curated pools of resources and self-made resources (13% and
11% of all statements about supplement sources, respectively). Self-made resources were simply
those they created themselves as they supplemented (e.g., “So for this particular strand, I would
probably create some slides,” Participant 3), whereas curated resource pools might include
resources they had downloaded in the past, resources they had previously made themselves and
saved, or resources that were part of an old official curriculum material they had used in past
years (e.g., “Now that I’ve been teaching for a while, I can pull the materials and stuff I have, so
I don’t always have to run to TPT [Teachers Pay Teachers] anymore,” Participant 12). The
prevalence of teachers’ own curated resource pools as a supplement source points to the
importance of empirical work centering curriculum curation (Torphy, Liu et al., 2020), while the
131

prevalence of self-created materials highlights a hole in the empirical literature, discussed in
Paper 1.
Considerations about Supplement Use Patterns
Teacher participants’ most-discussed use patterns by far were timing (e.g., “It could just
be like, while they're having snack or something, playing this in the background,” Participant 7)
and grouping (e.g., “So whenever I'm thinking of a lesson, I'm thinking of first, we're gonna do it
together, [then] they're gonna practice it with partners countless times before they can show me
on their own if they can do it,” Participant 8). Over 90% of all statements related to use pattern
discussed either timing or grouping (Table 15).
Table 15. Supplement Use Pattern Topic Coverage
Topic Number of Codes % of Use Pattern Codes
Discussion of any use pattern 106 100%
Use pattern: timing 51 48%
Use pattern: grouping 47 44%
Use pattern: other 8 8%
Note: Some categories have been combined or collapsed for readability. See Appendix F for
complete list of categories.

Use patterns outlined in earlier empirical work (such as prevalence and frequency;
Polikoff & Silver, 2021) are still important considerations for those studying or making policy
relevant to supplementation (it would be useful to know how often teachers supplement before
regulating it, for example), but their relative absence from the observation/interviews may point
to a relative unimportance to teachers’ implementation of supplemental materials. This makes
intuitive sense: the prevalence of supplements among all materials or the frequency with which a
teacher supplements should have little bearing on teachers’ individual instructional decisions,
whereas when they use a material and how they group their students are more practically-
relevant considerations. It would be productive for future empirical work in this space, including
132

future studies that make use of the survey tool introduced in Paper 2, to better account for
teachers’ decisions around timing and student grouping.
Summary of Research Question 1
The above findings provide a detailed snapshot of teachers’ real-time supplementation
processes, and they support some existing ideas about teacher curriculum use while standing as
evidence against others. Overall, they suggest that the TCSF is a useful conceptual tool for
understanding the multidimensional phenomenon of teacher curriculum, provided it is applied
flexibly. These empirical findings usefully advance theory around teacher curriculum
supplementation, but this study’s rich observation and interview data also contain more
practically-relevant findings, with implications for education leaders and policymakers looking
to better understand their teachers’ supplementation tendencies or design professional learning
experiences to improve teachers’ curriculum supplementation skills.
Research Question 2: Insights from Participants’ Supplementation Processes
To address Research Question 2, about insights generated from observations of and
interviews focused on teachers’ curriculum supplementation processes, we start by detailing
three archetypal orientations that emerged over the course of the observations and interviews.
We then discuss two tentative findings around the roles of gender and experience in teachers’
supplementation processes that beg further investigation.
Three Teacher Orientations Toward Curriculum Supplementation
Across the observations and interviews, three archetypal orientations toward curriculum
supplementation stood out: academics-first, students-first, and aesthetics-first. Importantly, these
archetypes are not profiles; no single teacher fit entirely into any, and the majority of teachers
evinced elements of all three. Still, viewing teachers who supplement as adopting either an
133

academics-first, students-first, or aesthetics-first orientation could be useful both to researchers
looking to better understand teacher curriculum supplementation and to school leaders looking to
better understand their teachers’ instructional decision-making. In the next paragraphs, we
summarize the orientations and discuss one or more teacher participants who stood out as
particularly embodying each.
Academics-First Orientation. A teacher supplementing from the academics-first
orientation is mainly concerned with their students’ mastery of skills. Their reasons for
supplementation tend to be related to student learning, and they attend specially to supplement
features that they believe promote that learning. When making supplementation decisions, they
heavily consider use patterns that may affect learning, such as student grouping. If relatively
experienced, they likely have a deep knowledge of supplement sources and a large, curated pool
of go-to materials, reflecting the wide array of student learning needs and academic levels they
recognize in their classroom. They always ensure that their materials align with state learning
standards.
Participant 9 embodied the academics-first orientation. The four most-common codes in
her transcript, comprising 48% of her coded statements, had to do with the academic features of
supplements, their desire to promote student learning, and to the concrete implementation of
supplements (Table 16). Her ten most-common codes, comprising 75% of her coded statements,
included discussion of virtual resource pools (but not Teachers Pay Teachers and Pinterest,
notably), supplement alignment to state standards and summative assessments, and which
resources she had used previously.

134

Table 16. Participant 9’s Most Common Codes
Code Number
of Codes
% of Participant
9’s Codes
Dimension//Design Feature//Academic Quality//Contributes to Learning 22 16%
Dimension//Reason//Student-Related//Learning and Academics 22 16%
Dimension//Design Feature//Usability for Teacher//Academics 12 9%
Dimension//Use Pattern//Grouping 9 7%
Dimension//Design Feature//Usability for Teacher//Convenience 8 6%
Dimension//Source//Online Unofficial//Other VRPs 7 5%
Dimension//Design Feature//Usability for Teacher//Money 6 4%
Dimension//Design Feature//Academic Quality//Aligns to Tests or Standards 5 4%
Official Material//No Stated OTOM 5 4%
Previously-Used Resource 5 4%

Participant 9 was the only teacher to identify that the simulation task referred to standard
NBT.1 without needing to consult a standards website or document, and within one minute of
seeing the prompt, she had already pulled a self-created anchor slide from her Google Drive and
was explaining expected academic pitfalls related to 5.NBT.1:
Then I would use lots of manipulatives. They would not be virtual manipulatives. But one
thing that is difficult is for my kids to transition [their understanding of] these place value
blocks from whole numbers to decimals. And decimals are also part of the 1/10
understanding. So what I would do first is I would really take it back down a level if
they're not understanding and go back to fourth grade, which has place value from the
10,000’s all the way to the tenths place, but really just focus on the place value of whole
number to begin with.

Participant 9 frequently considered the learning implications of student grouping, with
statements like, “I would independently divvy up my groups as far as who gets it, who doesn’t,
and reteach them. But while the kids are waiting to be pulled, I would either frontload them or
reteach them using Khan Academy.” She explained how certain supplemental materials “are
great” because they “require more thinking. So it’s more problem-solving based” and shared her
wide knowledge of supplement sources she deemed to be high-quality, including a neighboring
district’s website with a free uploaded math curriculum and a website with items aligned
specifically to end-of-year assessments in North Carolina, Participant 9’s state. Participant 9 saw
supplementation as a way to promote learning and inject rigor into her practice.
135

Students-First Orientation. A teacher supplementing from a students-first orientation is
mainly concerned with their students’ wellbeing. They likely feel that their official curriculum
materials fall short of serving their specific students in some way, so supplement to make up for
the shortcoming. These teachers may be more likely to teach high proportions of underserved
student groups such as English learners, students with disabilities, or students of color. When
supplementing, these teachers tend not to source their supplements from “official” sources, and
readily create materials themselves to serve their particular students.
Participants 2, 7, and 8 each embodied elements of the students-first archetypal
orientation. Participant 2 taught mostly students of color and summed up her supplementation in
terms of wanting to teach her students more authentically:
Obviously, you know, the supplemental materials are so important because as thorough
as the boxed curriculum is, I teach in a Title One district, it is not geared towards my
students. It's geared towards white entitled kids. That’s the fairest way to say it. So I have
to teach them that wording and the way they word things in a word problem…we go into
those key words that we're looking for, like, ‘how to learn white math.’ It’s a little
disturbing, but I love my students and I really think it's just not fair.

Participant 2 went on to critique the way her official curriculum was written, complaining that
“my children don’t talk like that. I don’t talk like that. It’s confusing.” When searching for
supplemental materials, she defaulted to Education.com, a large virtual resource pool, but
attended to supplement features that contributed to her students’ emotional wellbeing with
statements like, “I also need to know what it says to them when they get it wrong” and “I need
positive language when they make mistakes, for sure.”
Participant 7 taught first grade and attended specially to the developmental
appropriateness of supplemental materials, including whether a material “look[s] like it’s a lot of
clicks for them,” and whether materials were “not too busy, they’re the right amount of length,
they’re appropriate for first grade.” She supplemented mainly from a pool of materials she had
136

built up over the years and kept organized in her computer’s Documents folder. She was
especially sensitive to the toll that not attending preschool and kindergarten during the COVID-
19 pandemic had taken on student development (“it’s the first year I have barely anyone who can
tie their shoes”), so made an effort to include “fine motor” activities in daily math centers as a
result.
Participant 8 taught mostly English learners, but felt her official curriculum was not
English learner friendly. Because her students “need to see color coding, they need to see
examples, manipulatives” and “know the academic vocabulary,” Participant 8 supplemented
extensively. She tended to use Google searches to find materials with visuals that would clarify
concepts for her students. She also frequently created her own supplemental materials with
ample visual support and shied away from traditional worksheets because “that rote stuff on a
worksheet would not do my students any good.”
Participants 2, 7, and 8 each embodied a students-first orientation toward
supplementation by keeping their students’ specific, but not necessarily purely academic, needs
as students of color, young students, and English learners (respectively) in mind as they sought
supplemental materials. Each of those participants had a cooler-than-median orientation toward
their officially-adopted curriculum materials and supplemented in part to make up for their
perception of their official material’s inability to serve their particular students (Table 7).
Aesthetics-First Orientation. A teacher supplementing from an aesthetics-first
orientation values the look of their supplemental materials first and foremost. They may believe
that good-looking materials drive student engagement or that aesthetically pleasing supplements
constitute “what counts as good enough in teaching” (Pittard, 2017, p. 30), as opposed to
academically rigorous or culturally and developmentally relevant materials. These teachers may
137

look down on the aesthetic quality of their official curriculum materials and lean on Teachers
Pay Teachers for their supplemental materials, since the platform is known for hosting visually
attractive, well-designed resources.
Participants 6 and 12 supplemented from an aesthetics-first orientation, at least in part.
Participant 6 “spen[t] oodles of money on Teachers Pay Teachers” and had accumulated over
1,000 items in her Teachers Pay Teachers purchase history across her ten years of teaching. She
described her official materials as “very dry and boring,” with many lessons “just a plain
worksheet.” She spoke positively of “pretty and cute” materials on Teachers Pay Teachers and
complained that her officially-adopted material taught too many different multiplication
strategies (“If I teach them how to do this one way, and they already know how to come to that
answer, why do we need to spend three weeks teaching them two other ways to do it?”). She
wanted her teaching materials “to look good and look good to the kids,” suggesting that she
valued aesthetically pleasing materials in part to engage her students and in part because she
simply derived satisfaction from them.
Participant 12 was an active seller as well as buyer in the Teachers Pay Teachers
marketplace. She explained that her district had a strictly-enforced mandate to use the official
curriculum before supplementing, but that she used supplemental materials extensively during
math centers time, so approximately half of her students’ math instructional minutes were spent
learning from supplemental materials. She emphasized the importance of materials’ visual appeal
as a seller since “I can pull the standards all day long, but it’s really got to be appealing,
something that meets the standard and looks good” to attract buyers. As a buyer, “if I’m trying to
teach something and the graphics just don’t look good, I’m not going to click on it.” She
complained that some materials provided by her district had images that were “probably from
138

like the 1980s” and were “just ugh. They need to put more time and effort into them to make
them actually pretty.” As someone who valued her materials’ aesthetic qualities so highly,
Participant 12 clearly believed that visually appealing materials were part of being a successful
teacher, ascribing to “the good enough teacher subject position that is both produced and
maintained” by marketplaces like Teachers Pay Teachers (Pittard, 2017, p. 43). But Participant
12 also suggested that this emphasis on visual appeal was grounded in student engagement:
And kids nowadays, because they have so much technology, they can get better graphics
on their iPads and stuff that they have at home. And so they will not pay attention to your
lesson, they will not focus on any games or materials if they do not find that it's worth
their time.

Participants 6 and 12 embodied an aesthetics-first orientation toward supplementation by
attending to the aesthetic features of specific materials during the simulation, highlighting
aesthetics as a major reason for their supplementation in general, and sourcing materials mainly
from Teachers Pay Teachers, which they both felt had attractive materials in addition to being
easy to use and a good value.
No Teacher-First Orientation. Despite the prevalence of statements related to making
lesson preparation faster, cheaper, or more convenient, none of the participants adopted a truly
teacher-first orientation to supplementation, where a major goal was to make their jobs easier.
Statements about convenience often emerged as considerations secondary to academics, student
characteristics, or aesthetics, not as decision-drivers in themselves. For example, Participant 13
explained that her favorite online source for supplemental materials provided “so many different
options for them to practice with. And I feel like each worksheet is organized so well and they
don’t get lost…I know we have to pay for it, but it just feels so much more usable.” Teachers
with fewer years of experience tended to attend more closely to considerations like saving time
139

and money, which makes intuitive sense since salaries and work efficiency tend to increase with
experience.
The fact that worries about saving money and time were common considerations during
the supplementation process (every teacher participant mentioned them at least once), though,
speaks to the gargantuan expectations we place on teachers alongside their typically-modest pay.
Participant 10 summed up the situation well:
One of the reasons why teachers don't get paid enough and why teachers are leaving is
because you have to be a social worker, a psychologist, a math teacher, an engineer, a
chef, a principle, you have to do all these things. And it's extremely challenging. And you
know, we don't get a lot of praise.

Despite often working in physically draining, emotionally demanding situations (Chang, 2009),
teacher participants in this study tended not to put themselves first in their supplementation
decision making. (Of course, effects of social desirability may also be at play here. For better or
for worse, society expects teachers to be almost nonsensically selfless (e.g., Labaree, 2000), so it
is reasonable to surmise that even if a teacher participant did primarily adopt a teacher-first
orientation to supplementation, they may have worked to hide that during their observation and
interview. Other methods such as choice experiments may be useful for teasing out the role of
social desirability in such findings.)
Why These Orientations Matter. Understanding of these archetypal orientations toward
supplementation could prove concretely useful to education decision-makers. For example, if a
school principal felt her teachers were not using rigorous enough instructional materials and
wanted to move them toward a more academics-first orientation toward supplementation,
knowledge of other archetypal orientations would serve her well. For teachers who tended to
supplement from a students-first orientation, the principal might design professional learning
experiences designed to help teachers adapt rigorous materials to better reflect their students’
140

experiences without sacrificing academic quality. On the other hand, this principal might have
conversations with teachers who supplemented from an aesthetics-first perspective about ways
they can keep their rooms and materials beautiful while still engaging students in rigorous
learning experiences. Understanding these orientations could be useful for creators of
supplemental and traditional curriculum materials as well, who might design materials with each
group’s needs in mind.
Men and Women May Have Different Tendencies Toward Self-Created Supplements
Outside the archetypal orientations toward supplementation, teachers’ tendencies toward
self-created supplemental materials seemed to vary somewhat by gender among this study’s
participants. This should be treated as purely an exploratory idea that future research may choose
to investigate, since it is based on findings from only two men (and 12 women) and since gender
differences were not a focus of this study. Tentatively, though, the men in this study seemed
ready to create materials wholesale, based entirely on their own knowledge: one created a set of
Google slides using problems he made up on the spot and the other described home-grown
activities he would lead using cheese sticks and base ten blocks. On the other hand, the women
in this study tended to either avoid self-creating supplemental materials altogether or to self-
create supplemental materials by adapting existing materials: one typed a popular mnemonic
rounding song onto paper, another copied drawings that accompanied a teachers’ blog post, and a
few others copied math problems from worksheets onto their own worksheets or resources. See
Table 17 for these and other examples.

141

Table 17. Approaches to Self-Created Materials, by Gender
Men Women
So for this particular strand, I would probably create some
slides. (Participant 3)
So I took that song and changed the lyrics and
literally typed them out one by one to help kids take
the song onto a piece of paper with the actual steps in
front of them. (Participant 5)

And so, I will usually go off script and do some more
discussing with whiteboards and markers. I tend to do that a
lot. (Participant 3)
And then from there either create a worksheet or
handout that uses different parts of those with some
of the things that I liked (Participant 5)

Typically, with something like this, I would probably use
rubiflex cubes [base ten blocks]…And then I would
demonstrate to them that having all 10 pieces would be our
whole number which would be one. And then we can break
off pieces and see when we have half. That's usually a good
place to start. (Participant 10)

Management is my thing. But creating things from
scratch, I'd probably rather not. (Participant 7)
So one thing that's really fun that you can do is, you might
not believe it, you can get a cheese stick…And you can say,
“I would like you to divide this cheese stick into 10 equal
pieces, and I'm going to show you how to do it” (Participant
10)
So I not only create this as a worksheet for them to
glue into their math journal, but I also make this into
a math center, so I put it inside of clear sheets.
(Participant 8)
I would not need to give them the worksheet itself. I
could just use those numbers to help me come up
with extra practice on the board if I needed to.
(Participant 13)

And then I would just make my own materials based
on these pictures. (Participant 15)

At this stage, this pattern should be considered more a research curiosity than a reliable
finding, although it is consistent with gendered differences in performance of expertise and
confidence from the psychology literature (Niederle & Vesterlund, 2011; Thomas-Hunt &
Phillips, 2004). Importantly, we can claim that the differing approaches by gender did not
actually reflect greater expertise among the men—both made major errors in their
supplementation. (One designed tasks that addressed different standards than the one in the
simulation task and the other misunderstood the offerings present in his official curriculum.)
Future empirical work might incorporate analysis by teacher gender to investigate whether and
how teachers’ socialization as men or women interacts with their supplementation strategies.
142

Teacher Curriculum Supplementation Can Be Learned and Developed
Finally, teacher participants in this study tended to view curriculum supplementation as a
skill they had learned and developed (or would learn and develop, for less-experienced teachers)
over time. Seven of the fourteen transcripts contained at least one reference to greater teaching
experience yielding stronger curriculum supplementation (and none contained anything
contradictory to this idea), presented in Table 18. Participants described curating their own pools
of supplemental resources, getting “a feel” for the types of resources available on Teachers Pay
Teachers and other websites, and gaining greater “perspective” on curriculum, content, and the
standards over years of experience at the same grade level or from more-experienced colleagues.
Table 18. Selected Discussion of the Connection between Teaching Experience and Curriculum Supplementation
Participant Statement
3 For me, it [deciding times when official curriculum falls short] comes after experiencing it at least
once.

5 Originally, a lot of the things that I found were freebies or things that were previously already free
for me. I don't typically go towards these anymore, because I've found ones that I use more
strategically and consistently.

5 If I was a first year teacher, I would have had no clue based off of this standard, that rounding
would have been as relevant and important on an EOG [end of grade test]. So being able to use this
and seeing that they don't even call it necessarily rounding but they call them estimation strategies,
really gives me a clue that it's not a front runner, in the minds of most teachers. Yet, it's still
something that needs to be taught and understood.

7 I’ve been teaching 17 years of first grade, I have a lot of resources. So I'd like to go with what I
have first, and then look out [for other resources].

9 The longer you're in a grade level, the more you familiarize yourself with the content, the more
you're going to know as far as what your kids are going to know, what they need to know, and so
forth.

10 But it's my job to delve into that treasure trove of experience. And lay that on the table for the kids.
I mean, I'm literally trying to lay all my tools out and say “which one is going to work?”

10 I remember my first three years teaching, those file cabinets were empty. Everything I found I
would put in and I don't care what it was.

11 I’ve probably used Teachers Pay Teachers, got to be at least five years, if not longer. And so I think
you just get a feel for the content there and what fits.

14 And I'm lucky enough, I worked in a school where we had teacher collaboration meetings pretty
often. And so I was on a grade level with lots of awesome veteran teachers that were able to give
their perspective and their point of view and the stuff that they do. So it's kind of just like learn as
you do kind of thing.

143

These perspectives suggest that curriculum supplementation is a skill that can develop
and improve over time, much like general curriculum use, pedagogical content knowledge, or
classroom management. If curriculum supplementation can develop over time like these other
skills, it stands to reason that it could be sensitive to some of the same professional learning tools
that are commonly used to develop those skills, such as traditional professional development
sessions, summer workshops, instructional coaching, or discussion in professional learning
communities. Yet not a single teacher participant had ever received any professional learning
related to teacher curriculum supplementation from their school or district. The closest any had
come was being given access to district-sanctioned supplemental materials available online.
(Questions about this came at the end of the interview protocol, so we were not able to ask them
of all participants for time reasons. These results hold for the nine participants who were asked,
and we have no reason to believe the six who weren’t would differ much in their experiences.)
When asked whether a professional learning experience around curriculum
supplementation would be useful to them, participants varied in their responses. A few simply
rejected the question’s premise since their district would never sponsor professional learning
about anything other than fidelity to their officially-adopted material. Others were enthusiastic at
the idea, eager to better understand how to vet supplemental materials and incorporate them into
their instruction. The most common response was a more measured one, where teachers were
interested in the possibility that such an experience could show them high-quality materials they
did not know about already, but were wary or even unenthusiastic about their schools or districts
dictating how to supplement their official curriculum materials, a highly personalized part of
their practice. Teachers generally agreed that such a professional learning experience would be
most useful for early career teachers, whose skills may not have developed as far.
144

The fact that school districts did not seem to offer professional learning opportunities
designed to develop their teachers’ curriculum supplementation skills is a missed opportunity,
given the extent to which virtually all teachers supplement their officially-adopted curriculum
(see Papers 1 and 2). Some teacher participants’ districts seemed to ignore supplementation,
whereas others had adopted a combative stance toward it—in describing how another teacher
was “written up” for not adhering to her official curriculum, Participant 6 explains, “It’s coming
from the top. The top wants the curriculum used because they pay a fortune for it.” Participant 6,
who averaged just over one Teachers Pay Teachers resource download every other day across
her 10-year career, illustrates the effectiveness of such zero-tolerance policies toward
supplementation.
It would make little sense for a district to forbid teachers who struggled with behavior
management from trying to use behavior management techniques, yet district policies that forbid
curriculum supplementation over quality concerns are built on similar logic. If policy at the
district level and above treated curriculum supplementation as just another component of
teaching and dedicated resources to helping teachers build their curriculum supplementation
skills, it is difficult to imagine that instruction would not improve across U.S. classrooms.
Limitations and Future Directions
This study generated useful information about teachers’ curriculum supplementation
processes, but there are a few important limitations that future work can address. First, although
we (successfully) selected this study’s participants to represent a variety of supplementation
styles, due to the heavily white, female population of participants eligible for selection into this
study, we were unable to delve as fully into other teacher characteristics as would have been
ideal. Future observations of elementary teachers’ curriculum supplementation should take
145

efforts to observe more male teachers, teachers of color, teachers in charter schools, teachers
with nonstandard licenses, and more, either by going to greater lengths to recruit these teachers
or by observing a larger number of teachers.
Second, although this study’s observation methodology allowed us to comment on
supplementation outside of virtual resource pools, available resources and ongoing COVID-19
protocols entailed that the observations were conducted remotely, via Zoom. We observed
teachers supplement from offline sources (e.g., by creating their own presentations) but with
Zoom tethering them to their computers, they may have been less likely than otherwise to
supplement in truly analog ways (e.g., by creating a review game on chart paper). Despite the
fact that teachers in this study reported that their practice during the observation was typical
when probed about this, future work using observation methods to study teacher curriculum
supplementation should consider the possibility that in-person observations might bolster study
authenticity.
Third, it is important to keep in mind that the results presented here, particularly those for
Research Question 1, analyzed the salience of various topics in teacher participants’ observation
and interview transcripts, which may not map perfectly onto those topics’ importance. We were
not able to code teachers’ statements with indications of degree or intensity, so it is possible that
some of the less-discussed topics were in fact some of the more important ones to teachers. Still,
in the absence of a reliable way to code statements’ intensity, the frequency with which teachers
discussed various topics provides some insight into their importance to teachers’
supplementation practices.
Fourth and finally, because the observation and interview were conducted in the presence
of a researcher whom the teachers did not know well, it is plausible that social desirability
146

concerns biased what teachers were willing to share of their supplementation practices. He tried
to mitigate this by introducing himself as a former teacher who missed his students and by
adopting an interested, affirming, non-judgmental demeanor throughout observation and
interview sessions, but these steps can only mitigate social desirability concerns, not eliminate
them. Other methods with less interaction between participant and researcher such as discrete
choice experiments might be better suited for assessing the extent to which social desirability
biases come into play in studies of teacher curriculum supplementation.
Discussion and Conclusion
This study was among the first to use live observation methods to study elementary math
teachers’ curriculum supplementation practices, and its findings have implications for future
research and instructional policy related to supplemental curriculum materials. The study’s first
goal was to document a group of in-service elementary school teachers’ considerations as they
supplemented their official math curriculum materials in order to evaluate the extent to which
existing theory accounted for their thought processes. We found that teachers mainly thought
aloud about specific supplement features and their reasons for supplementation, and that student
learning, saving time or money, student engagement, and visual appeal were some of the major
drivers of decision making around whether and how to use supplemental materials. The most-
used source of supplemental materials, by far, was Teachers Pay Teachers. These findings were
consistent with the TCSF, which remains a useful way to conceive of teachers’ decision making
around curriculum supplementation.
Moving forward, these findings could be productively built upon by studies designed to
investigate supplement features and reasons for supplementation more explicitly, perhaps by
interviewing teachers about them in greater depth. We could build our understanding of the
147

dominance of Teachers Pay Teachers by systematically analyzing available resources, perhaps
using unsupervised machine learning methods to cluster resources according to a large set of
characteristics (including those difficult to build into regression models, such as text descriptions
and images) or using supervised learning methods to relate those characteristics to “outcomes”
like resource popularity.
The study’s second goal was to use the rich observation and interview data to generate
insights about the practice of supplementation that might be difficult to generate using other data
collection methods. We found evidence of three archetypal orientations toward supplementation:
academics-first, students-first, and aesthetics-first. This expands the notion of teacher curriculum
supplementation as necessarily students-first or academics-first, a conception commonly-held in
the literature (e.g., Gallagher et al, 2019; Sawyer et al., 2020). Although we found no evidence of
a teacher-first orientation, which would truly run counter to these existing conceptions, we
suggest investigation of teacher curriculum supplementation by methods less susceptible to
social desirability concerns before becoming convinced that a teacher-first orientation to
supplementation does not occur much in practice.
We also found evidence that teachers view curriculum supplementation as a skill that can
be learned and strengthened as well as tentative evidence of differences in the ways men and
women approach creating their own supplemental materials. School leaders could use
understanding of these archetypal orientations to better develop and guide their teachers’
instructional decisions, and curriculum creators could use these same orientations almost as
marketing psychographics, ensuring that their products appeal to teachers who approach
curriculum from academics-first, students-first, and aesthetics-fist perspectives.
148

Perhaps most applicably, we found that schools and districts tend not to provide formal
professional learning experiences dedicated to curriculum supplementation. We hope findings
from this study underscore to education decision makers that (1) teachers will supplement their
official materials regardless of official policies and (2) curriculum supplementation need not be
negative—some teachers may well improve on the educational offerings of their official
curriculum materials by supplementing them. Therefore, it makes sense for districts to develop
their teachers’ capacities for curriculum supplementation as they would any other skill. Given its
ubiquity, curriculum supplementation is a promising and understudied lever for instructional
improvement.

149

References
Abramovich, S., & Schunn, C. (2012). Studying teacher selection of resources in an ultra-large
scale interactive system: Does metadata guide the way?. Computers & Education, 58(1),
551–559. doi:10.1016/j.compedu.2011.09.001
Abramovich, S., Schunn, C., & Correnti, R. (2013). The role of evaluative metadata in an online
teacher resource exchange. Education Tech Research Dev, 61(6), 863–883.
doi:10.1007/s11423-013-9317-2
Achinstein, B. (2002). Conflict amid community: The micropolitics teacher
collaboration. Teachers College Record, 104(3), 421–455. doi:10.1111/1467-9620.00168
Agodini, R., & Harris, B. (2010). An experimental evaluation of four elementary school math
curricula. Journal of Research on Educational Effectiveness, 3(3), 199-253.
doi:10.1080/19345741003770693
Attick, D. (2017). Homo economicus at school: Neoliberal education and teacher as economic
being. Educational Studies, 53(1), 37–48. doi:10.1080/00131946.2016.1258362
Ball, D., & Cohen, D. (1996). Reform by the book: What is—or might be—the role of
curriculum materials in teacher learning and instructional reform?. Educational
researcher, 25(9), 6-14.
Ball, D., & Feiman-Nemser, S. (1988). Using textbooks and teachers' guides: A dilemma for
beginning teachers and teacher educators. Curriculum Inquiry, 18(4), 401-423.
Bartell, T., Cho, C., Drake, C., Petchauer, E., & Richmond, G. (2019). Teacher agency and
resilience in the age of neoliberalism. Journal of Teacher Education 70(4), 302–305.
doi:10.1177/0022487119865216
150

Beahm, L., Cook, B., & Cook, L. (2019). Proceed with caution: Using web-based resources for
instructing students with and at risk for EBD. Beyond Behavior, 28(1), 13–20.
doi:10.1177/1074295619836076
Bennett, C. (2019). Teachers pay teachers: The fast food of education. The educator’s room.
https://theeducatorsroom.com/teachers-pay-teachers-the-fast-food-of-education/
Bhatt, R., & Koedel, C. (2012). Large-scale evaluations of curricular effectiveness. Educational
Evaluation and Policy Analysis, 34(4), 391–412. doi:10.3102/0162373712440040
Bhatt, R., Koedel, C., & Lehmann, D. (2013). Is curriculum quality uniform? Evidence from
Florida. Economics of Education Review, 34, 107-121.
Blazar, D., Heller, B., Kane, T., Polikoff, M., Staiger, D., Carrell, S., Goldhaber, D., Harris, D.,
Hitch, R., Holden, K., & Kurlaender, M. (2019). Learning by the book: Comparing math
achievement growth by textbook in six Common Core states. Research Report.
Cambridge, MA: Center for Education Policy Research, Harvard University.
Boitnott, J. (2015, March 31). 6 companies that are teaching educators what good 'disruption'
means. Entrepreneur. https://www.entrepreneur.com/article/244423
Brown, M. (2002). Teaching by design: Understanding the intersection between teacher practice
and the design of curricular innovations. [Unpublished doctoral dissertation]. Chicago,
IL: Northwestern University.
Buckley, A., & Nzembayie, K. (2016). Teacherpreneurs: From vocation to innovation.
In ICIE2016-Proceedings of the 4th International Conference on Innovation and
Entrepreneurship: ICIE2016 (p. 36).
151

Burla, L., Knierim, B., Barth, J., Liewald, K., Duetz, M., & Abel, T. (2008). From text to
codings: Intercoder reliability assessment in qualitative content analysis. Nursing
Research, 57(2), 113-117. doi: 10.1097/01.NNR.0000313482.33917.7d
Caniglia, J., & Meadows, M. (2018). Pre-service mathematics teachers' use of web
resources. International Journal for Technology in Mathematics Education, 25(3), 17-34.
Carpenter, J. (2016). Teachers at the wheel. Educational Leadership, 73(8), 30-35.
Carpenter, J., & Harvey, S. (2019). “There's no referee on social media”: Challenges in educator
professional social media use. Teaching and Teacher Education, 86, 102904.
doi:10.1016/j.tate.2019.102904
Carpenter, J., & Krutka, D. (2014). How and why educators use twitter: A survey of the field.
Journal of Research on Technology in Education, 46(4), 414–434.
doi:10.1080/15391523.2014.925701
Carpenter, J., & Morrison, S. (2018). Enhancing teacher education… with Twitter?. Phi Delta
Kappan, 100(1), 25-28.
Carpenter, J., Morrison, S., Craft, M., & Lee, M. (2020). How and why are educators using
Instagram?. Teaching and Teacher Education, 96, 103149.
doi:10.1016/j.tate.2020.103149
Cattell, R. (1966). The scree test for the number of factors. Multivariate Behavioral Research,
1(2), 245-276. Doi: 10.1207/s15327906mbr0102_10
Chang, M. (2009). An appraisal perspective of teacher burnout: Examining the emotional work
of teachers. Educational psychology review, 21(3), 193-218.
152

Chapman, S., Wright, P., Pascoe, R. (2019). Criticality and connoisseurship in arts education:
Pedagogy, practice and ‘Pinterest’. Education 3-13, 47(8), 957–968.
doi:10.1080/03004279.2018.1544578
Choppin, J. (2011). Learned adaptations: Teachers’ understanding and use of curriculum
resources. Journal of mathematics teacher education, 14(5), 331-353.
CloudResearch.com. (2021). Pricing. Cloud Research. https://www.cloudresearch.com/pricing/
Coburn, C., & Russell, J. (2008). District policy and teachers’ social networks. Educational
Evaluation and Policy Analysis, 30(3), 203–235. doi:10.3102/0162373708321829
Cochran, K., King, R., & DeRuiter, J. (1991). A tentative model for teacher preparation. [Paper
presentation]. American Education Research Association, Chicago, IL.
Cuban, L. (1984). How Teachers Taught: Constancy and Change in American Classrooms,
1890-1980. Research on Teaching Monograph Series.
Cummings, M. (2015). Why is this marketplace for teachers to sell their lesson plans thriving?
Slate. https://slate.com/human-interest/2015/06/teachers-pay-teachers-why-the-site-for-
teachers-to-sell-their-lesson-plans-is-thriving.html
Davis, J., Choppin, J., Roth McDuffie, A., & Drake, C. (2013). Common Core State Standards
for mathematics: Middle school mathematics teachers' perceptions (Technical report).
University of Rochester, Warner School of Education.
Davis, E., & Krajcik, J. (2005). Designing educative curriculum materials to promote teacher
learning. Educational researcher, 34(3), 3-14.
Desimone, L. (2009). Improving impact studies of teachers’ professional development: Toward
better conceptualizations and measures. Educational researcher, 38(3), 181-199.
153

Desimone, L., & Le Floch, K. (2004). Are we asking the right questions? Using cognitive
interviews to improve surveys in education research. Educational evaluation and policy
analysis, 26(1), 1-22.
Dietiker, L., Males, L., Amador, J., & Earnest, D. (2018). Curriculum noticing: A comprehensive
framework to describe teachers’ interactions with curriculum materials. Journal for
Research in Mathematics Education, 49(5), 521-532. doi:
10.5951/jresematheduc.49.5.0521
Dietiker, L., & Riling, M. (2018). Design (in)tensions in mathematics curriculum. International
Journal of Educational Research, 92, 43-52.
Drake, C., & Remillard, J. (2019). Seeing teacher-designer-curriculum research through an
ergonomic lens: Commentary. International Journal of Educational Research, 95, 227-
236.
Drake, C., & Sherin, M. (2006). Practicing change: Curriculum adaptation and teacher narrative
in the context of mathematics education reform. Curriculum Inquiry, 36(2), 153–187.
doi:10.1111/j.1467-873X.2006.00351.x
Drake, C. (2021). Commentary: New paradigms for understanding elementary teachers’
interactions with instructional materials. The Elementary School Journal, 122(1), 136-
140.
Dupuis, D. (2018). Content-related validity evidence. In B. Frey (Ed.), The SAGE encyclopedia
of educational research, measurement, and evaluation (pp. 399-400). SAGE
Publications, Inc. Thousand Oaks, CA. doi: https://dx.doi.org/10.4135/9781506326139
Elo, S., Kääriäinen, M., Kanste, O., Pölkki, T., Utriainen, K., & Kyngäs, H. (2014). Qualitative
content analysis: A focus on trustworthiness. SAGE open, 4(1), 1-10.
154

Emo, W. (2015). Teachers’ motivations for initiating innovations. Journal of Educational
Change, 16(2), 171–195. doi:10.1007/s10833-015-9243-7
English, R. (1980). The politics of textbook adoption. The Phi Delta Kappan, 62(4), 275-278.
Engman, M., Lemanski, L., & Shepard-Carey, L. (2019, April 5-8). Entrepreneurial sign and
click-worthy teachers in the digital social world. [Paper presentation]. Toronto, Canada:
American Educational Research Association.
Farr, R., & Tulley, M. (1985). Do adoption committees perpetuate mediocre textbooks?. The Phi
Delta Kappan, 66(7), 467-471.
Finkelstein, B. (1970). Governing the young: Teacher behavior in American primary schools,
1820-1880. A documentary history. [Unpublished doctoral dissertation]. New York, NY:
Columbia University.
Finkelstein, L. (1975). Fundamental concepts of measurement: Definition and scales.
Measurement and Control, 8(3), 105-111. Doi: 10.1177/002029407500800305
Follett, R. (1985). The school textbook adoption process. Book Research Quarterly, 1(2), 19-23.
Forbes, C., & Davis, E. (2010). Curriculum design for inquiry: Preservice elementary teachers'
mobilization and adaptation of science curriculum materials. Journal of Research in
Science and Teaching, 47(7), 820–839. doi:10.1002/tea.20379
Fordham Institute. (2004). The mad, mad world of textbook adoption (Technical report).
Washington, DC: Fordham Institute. https://fordhaminstitute.org/national/research/mad-
mad-world-textbook-adoption
Foster, S. J. (1999). The struggle for American identity: Treatment of ethnic groups in United
States history textbooks. History of Education, 28(3), 251-278.
155

Fowler, F. (2008). Designing questions to be good measures. In Survey Research Methods, 4th
Edition (pp. 87-114). Thousand Oaks, CA: Sage Publications.
Frank, K., & Torphy, K. (2019). Social media, who cares? A dialogue between a millennial and a
curmudgeon. Teachers College Record, 121(14).
Fyfield, M., Henderson, M., & Phillips, M. (2021). Navigating four billion videos: teacher search
strategies and the YouTube algorithm. Learning, Media and Technology, 46(1), 47-59.
Gallagher, J., Swalwell, K., & Bellows, M. (2019). "Pinning" with pause: Supporting teachers'
critical consumption on sites of curriculum sharing. Social Education, 83(4), 217-224.
Gargani, J., & Strong, M. (2014). Can we identify a successful teacher better, faster, and
cheaper? Evidence for innovating teacher observation systems. Journal of Teacher
Education, 65(5), 389-401.
Garland, S. (2016, May 26). In Texas, new math standards look a whole lot like Common Core.
The Hechinger Report. https://hechingerreport.org/texas-new-math-standards-look-
whole-lot-like-common-core/
Gay, A., Barry, A., Rothrock, K., & Pelkey, M. (2020). Mathematics student teachers' views and
choices about teaching and textbooks in middle and high school
classrooms. International Journal of Research in Education and Science, 6(1), 120-132.
Ghaderi, I. (2018). Unitary view of validity. In B. Frey (Ed.), The SAGE encyclopedia of
educational research, measurement, and evaluation (pp. 1752-1756). SAGE
Publications, Inc. Thousand Oaks, CA. doi: https://dx.doi.org/10.4135/9781506326139
Gläser-Zikuda, M., & Fuß, S. (2008). Impact of teacher competencies on student emotions: A
multi-method approach. International Journal of Educational Research, 47(2), 136-147.
156

Glaeser-Zikuda, M., & Mayring, P. (2003). A qualitative oriented approach to learning emotions
at school. In P. Mayring & C. V. Rhoeneck (Eds.), Learning emotions: The influence of
affective factors on classroom learning (pp. 103–126). Frankfurt: Lang.
Goldstein, D. (2019, December 6). After 10 years of hopes and setbacks, what happened to the
Common Core? The New York Times.
https://www.nytimes.com/2019/12/06/us/common-core.html
Greene, K. (2016). For sale: Your lesson plans. Educational Leadership, 74(2), 28-33.
Greenhalgh, T., & Peacock, R. (2005). Effectiveness and efficiency of search methods in
systematic reviews of complex evidence: Audit of primary sources. BMJ, 331(7524),
1064-1065.
Grossman, P., & Thompson, C. (2008). Learning from curriculum materials: Scaffolds for new
teachers?. Teaching and Teacher Education, 24(8), 2014–2026.
doi:10.1016/j.tate.2008.05.002
Grote-Garcia, S., & Vasinda, S. (2014). Pinning and practice: Using Pinterest as a tool for
developing pedagogical content knowledge. Texas Journal of Literacy Education, 2(1),
36-45.
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. (2018). A data-
driven analysis of workers' earnings on Amazon Mechanical Turk. In Proceedings of the
2018 CHI conference on human factors in computing systems (pp. 1-14).
Harris, L., Shelton, C., & Archambault, L. (2020). Examining issues of quality on teachers pay
teachers: An exploration of best-selling U.S. history resources. [Paper presentation].
National Council for the Social Studies: College and University Faculty Assembly.
157

Hashim, A., & Carpenter, J. (2019). A conceptual framework of teacher motivation for social
media use. Teachers College Record, 121(14), 1-18.
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning. New York,
NY: Springer.
Herrington, K. (2021). Top Pinterest categories to drive traffic.
https://kimberlyherrington.com/pinterest-management-services/pinterest-categories
Hertel, J., & Wessman-Enzinger, N. (2017). Examining Pinterest as a curriculum resource for
negative integers: An initial investigation. Education Sciences, 7(2).
doi:10.3390/educsci7020045
Hill, J., & Hannafin, M. (2001). Teaching and learning in digital environments: The resurgence
of resource-based learning. Educational Technology Research and Development, 49(3),
37-52.
Hodge, E., Salloum, S., & Benko, S. (2016). (Un)Commonly connected: A social network
analysis of state standards resources for English/language arts. AERA Open, 2(4),
2332858416674901. doi:10.1177/2332858416674901
Hodge, E., Salloum, S., & Benko, S. (2019). The changing ecology of the curriculum
marketplace in the era of the Common Core State Standards. Journal of Educational
Change, 20(4), 425–446. doi:10.1007/s10833-019-09347-1
Hu, S., & Torphy, K. (2020). Social media and education: Curation within and outside the
schoolhouse (Working paper). Michigan State University: Teachers in Social Media.
Hu, S., Torphy, K., Chen, Z., & Eagan, B. (2018). How do U.S. teachers align instructional
resources to the Common Core State Standards. Proceedings of the 9th International
Conference on Social Media and Society. 315–319.
158

Hu, S., Torphy, K., Evert, K., & Lane, J. (2020). From cloud to classroom: Mathematics
teachers' planning and enactment of resources accessed within virtual spaces. Teachers
College Record, 122(6).
Hu, S., Torphy, K., & Opperman, A. (2019). Culturally relevant curriculum materials in the age
of social media and curation. Teachers College Record, 121(14).
Hu, S., Torphy, K., Opperman, A., Jansen, K., & Lo, Y. (2018). What do teachers share within
socialized knowledge communities: A case of Pinterest. Journal of Professional Capital
and Community, 3(2), 97–122. doi:10.1108/JPCC-11-2017-0025
Hunter, L., & Hall, C. (2018). A survey of K-12 teachers’ utilization of social networks as a
professional resource. Education and Information Technologies, 23(2), 633-658.
Hur, J., & Brush, T. (2009). Teacher participation in online communities. Journal of Research on
Technology in Education, 41(3), 279–303. doi:10.1080/15391523.2009.10782532
Hylen, J. (2015). Open educational resources: Opportunities and challenges (Technical report).
OECD’s Centre for Educational Research and Innovation.
Ingram, C. (2019). The edupreneur frontier: Creating a new system of curriculum and
professional development for teachers. Art Education, 72(5), 14–18.
doi:10.1080/00043125.2019.1621629
Irvine, T. (2015). Pinned: A qualitative study of teacher experiences interfacing with online
resources for lesson planning. [Unpublished doctoral dissertation]. Minneapolis, MN:
Capella University.
Jackson, K., & Makarin, A. (2018). Can online off-the-shelf lessons improve student outcomes?
Evidence from a field experiment. American Economic Journal: Economic Policy, 10(3),
226–254. doi:10.1257/pol.20170211
159

Jones, L. (2018). Teacher-powered: The unstoppable community behind TeachersPayTeachers.
Forbes. https://www.forbes.com/sites/lilyjones/2018/07/19/teacher-powered-the-
unstoppable-community-behind-teacherspayteachers/?sh=200fcbbf3c8d
Kaufman, J., Doan, S., Tuma, A., Woo, A., Henry, E., & Lawrence, R. (2020). How instructional
materials are used and supported in U.S. K-12 classrooms (Technical report). Santa
Monica, CA: RAND Corporation.
Kaufman, J., Opfer, V., Bongard, M., & Pane, J. (2018). Changes in what teachers know and do
in the Common Core era: American Teacher Panel findings from 2015 to 2017
(Technical report). Santa Monica, CA: RAND Corporation.
Kim, D., Lee, Y., Leite, W., & Huggins-Manley, A. (2020). Exploring student and teacher usage
patterns associated with student attrition in an open educational resource-supported
online learning platform. Computers & Education, 156.
doi:10.1016/j.compedu.2020.103961
Kim, M. (2021). A systematic review of the design work of STEM teachers. Research in Science
& Technology Education, 39(2), 131-155.
https://doi.org/10.1080/02635143.2019.1682988
Kimmons, R. (2015). OER quality and adaptation in K-12: Comparing teacher evaluations of
copyright-restricted, open, and open/adapted textbooks. International Review of Research
in Open and Distributed Learning, 16(5), 39-57.
Koedel, C., Li, D., Polikoff, M., Hardaway, T., & Wrabel, S. (2017). Mathematics curriculum
effects on student achievement in California. Aera Open, 3(1), 2332858417690511.
160

Koehler, M., Shelton, C., Carpenter, J., & Greenhalgh, S. (2020). Where does all the money go?
Free and paid transactions on teacherspayteachers.com. Teachers College Record.
https://www.tcrecord.org/Content.asp?ContentId=23478
Krauskopf, K., Zahn, C., & Hesse, F. (2011). Leveraging the affordances of YouTube:
Pedagogical knowledge and mental models of technology affordances as predictors for
pre-service teachers’ planning for technology integration. In Society for Information
Technology & Teacher Education International Conference (pp. 4372-4379). Association
for the Advancement of Computing in Education (AACE).
Krutka, D., Manca, S., Galvin, S., Greenhow, C., Koehler, M., & Askari, E. (2019). Teaching
“against” social media: Confronting problems of profit in the curriculum. Teachers
College Record, 121(14), 1-19.
Kurz, A., Elliott, S., Kettler, R., & Yel, N. (2014). Assessing students' opportunity to learn the
intended curriculum using an online teacher log: Initial validity evidence. Educational
Assessment, 19(3), 159-184.
Labaree, D. (2000). On the nature of teaching and teacher education: Difficult practices that look
easy. Journal of teacher education, 51(3), 228-233.
Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American
educational research journal, 32(3), 465-491.
Landis, J., & Koch, G. (1977). The measurement of observer agreement for categorical
data. Biometrics, 33(1), 159-174. doi: 10.2307/2529310

161

Lane, J., Boggs, B., Chen, Z., & Torphy, K. (2019). Conceptualizing virtual instructional
resource enactment in an era of greater centralization, specification of quality
instructional practices, and proliferation of instructional resources. Teachers College
Record, 121(14).
Lex, A., Gehlenborg, N., Strobelt, H., Vuillemot, R., & Pfister, H. (2014). UpSet: Visualization
of intersecting sets. IEEE transactions on visualization and computer graphics, 20(12),
1983-1992.
Lincoln, Y., & Guba, E. (2005). Paradigmatic controversies, contradictions, and emerging
confluences In N. Denzin & Y. Lincoln (Ed.), The SAGE Handbook for Qualitative
Research (3
rd
ed.) (pp. 163-188). Thousand Oaks: Sage Publications, Inc.
Liu, Y., Torphy, K., Hu, S., Tang, J., & Chen, Z. (2020). Examining the virtual diffusion of
educational resources across teachers’ social networks over time. Teachers College
Record, 122(6).
Lohmeier, J. (2018). Criterion-based validity evidence. In B. Frey (Ed.), The SAGE encyclopedia
of educational research, measurement, and evaluation (pp. 423-425). SAGE
Publications, Inc. Thousand Oaks, CA. doi: https://dx.doi.org/10.4135/9781506326139
MacCallum, R., Widaman, K., Zhang, S., & Hong, S. (1999). Sample size in factor analysis.
Psychological Methods, 4(1), 84.
Marcin, T. (2020, June 25). The online lesson plan marketplace boomed when the pandemic hit.
Mashable. https://mashable.com/article/covid-lesson-plan-marketplace-boom/
Mari, L. (2013). A quest for the definition of measurement. Measurement 46(8), 2889-2895. Doi:
10.1016/j.measurement.2013.04.039
162

Mari, L., Carbone, P., Giordani, A., & Petri, D. (2017). A structural interpretation of
measurement and some related epistemological issues. Students in History and
Philosophy of Science Part A, 65-66, 46-56. Doi: 10.1016/j.shpsa.2017.08.001
Marple, S., Bugler, D., Chen-Gaddini, M., Burr, E., & Finkelstein, N. (2017). Why and how
teachers choose to supplement adopted materials (Technical report). San Francisco, CA:
WestEd.
Mason, S., & Kimmons, R. (2018). Effects of open textbook adoption on teachers’ open
practices. International Review of Research in Open and Distributed Learning, 19(3).
Maxwell, J. (2013). Qualitative research design: An interactive approach. Los Angeles, CA:
Sage.
Mayring, P. (2000). Qualitative Content Analysis. Forum Qualitative Sozialforschung, 1(2),
Article 20.
Mayring, P. (2014). Qualitative content analysis: Theoretical foundation, basic procedures and
software solution. SSOAR Open Access Repository. https://nbn-
resolving.org/urn:nbn:de:0168-ssoar-395173
Mayring, P. (2015). Qualitative content analysis: Theoretical background and procedures. In
Approaches to qualitative research in mathematics education (pp. 365-380). Springer,
Dordrecht.
Miller, D. (2015). Pinterest Isn't Pedagogy. Texas Library Journal, 91(3), 101.
Mishra, P., & Koehler, M. (2006). Technological pedagogical content knowledge: A framework
for teacher knowledge. Teachers college record, 108(6), 1017-1054.
Morales, R., & Baker, A. (2018). Secondary students’ perceptions of open science textbooks.
Journal of Interactive Media in Education, 2018(1). doi:10.5334/jime.455
163

Nicol, C., & Crespo, S. (2006). Learning to teach with mathematics textbooks: How preservice
teachers interpret and use curriculum materials. Educational Studies in Mathematics,
62(3), 331–355. doi:10.1007/s10649-006-5423-y
Niederle, M., & Vesterlund, L. (2011). Gender and competition. Annual Review of
Economics, 3(1), 601-630.
Opfer, V., Kaufman, J., & Thompson, L. (2017). Implementation of K-12 state standards for
mathematics and English language arts and literacy: Findings from the American
Teacher Panel (Technical report). Santa Monica, CA: Rand Corporation.
PBS News Hour. (2018, January 9). Why teachers selling lesson plans have sparked debate.
https://www.pbs.org/newshour/show/why-teachers-selling-lesson-plans-have-sparked-
debate
Perry, R., Marple, S., & Reade, F. (2017). Instructional materials: Who makes the choice?
Findings from the annual survey on implementing the Common Core State Standards in
mathematics (Technical report). San Francisco, CA: WestEd.
Pittard, E. (2017). Gettin’ a little crafty: Teachers Pay Teachers, Pinterest and neo-liberalism in
new materialist feminist research. Gender and Education, 29(1), 28–47.
doi:10.1080/09540253.2016.1197380
Polikoff, M. (2015). How well aligned are textbooks to the Common Core Standards in
mathematics?. American Educational Research Journal, 52(6), 1185–1211.
doi:10.3102/0002831215584435
Polikoff, M., Campbell, S., Rabovsky, S., Koedel, C., Le, Q., Hardaway, T., & Gasparian, H.
(2020). The formalized processes districts use to evaluate mathematics textbooks.
Journal of Curriculum Studies, 52(4), 451–477. doi:10.1080/00220272.2020.1747116
164

Polikoff, M., Campbell, S., & Korn, S. (2018). Using quantitative and qualitative methods to
study the content and effects of curriculum materials. In Chad R. Lochmiller (Ed.),
Complementary Research Methods for Educational Leadership and Policy Studies, 193-
212. New York, NY: Springer International Publishing.
Polikoff, M., & Dean, J. (2019). The supplemental curriculum bazaar: Is what's online any
good? (Technical report). Washington, DC: Fordham Institute.
Polikoff, M., & Silver, D. (2021). Identifying and distinguishing among teachers’ supplementary
curriculum use patterns using the lasso. Frontiers in Education.
https://doi.org/10.3389/feduc.2021.722554
Polly, D. (2017). Elementary school teachers’ uses of mathematics curricular resources. Journal
of Curriculum Studies, 49(2), 132–148. doi:10.1080/00220272.2016.1154608
Recker, M., Dorward, J., & Nelson L. (2004). Discovery and use of online learning resources:
Case study findings. Journal of Educational Technology & Society, 7(2), 93-104.
Redmiles, E., Kross, S., & Mazurek, M. (2019). How well do my results generalize? comparing
security and privacy survey results from MTurk, web, and telephone samples. In 2019
IEEE Symposium on Security and Privacy (pp. 1326-1343). IEEE.
Remillard, J. (1996). Changing texts, teachers, and teaching: The role of curriculum materials in
mathematics education reform. (9734181). [Doctoral dissertation, Michigan State
University]. ProQuest Dissertations Publishing.
Remillard, J. (1999). Curriculum materials in mathematics education reform: A framework for
examining teachers' curriculum development. Curriculum Inquiry, 29(3), 315-342.
Remillard, J. (2005). Examining key concepts in research on teachers’ use of mathematics
curricula. Review of educational research, 75(2), 211-246.
165

Reys, B., & Reys, R. (2006). The development and publication of elementary mathematics
textbooks: Let the buyer beware!. Phi Delta Kappan, 87(5), 377-383.
Riling, M., & Dietiker, L. (2018). Given a traditional textbook—now what?. The Mathematics
Teacher, 112(3), 192-199.
Robin, B. R. (2009). Digital storytelling: A powerful technology tool for the 21st century
classroom. Theory into Practice, 47(3), 220-228.
Rodriguez, N., Brown, M., & Vickery, A. (2020). Pinning for profit? Examining elementary
preservice teachers’ critical analysis of online social studies resources about Black
history. Contemporary Issues in Technology and Teacher Education, 20(3).
Ross, J., McDougall, D., Hogaboam-Gray, A., & LeSage, A. (2003). A survey measuring
elementary teachers' implementation of standards-based mathematics teaching. Journal
for Research in Mathematics Education, 34(4), 344-363.
Rothstein, H., & Hopewell, S. (2019). Grey literature. In Cooper, H., Hedges, L., & Valentine, J.
(Eds.), The handbook of research synthesis and meta-analysis. Russell Sage Foundation.
Rowan, B., Camburn, E., & Correnti, R. (2004). Using teacher logs to measure the enacted
curriculum: A study of literacy teaching in third‐grade classrooms. The Elementary
School Journal, 105(1), 75-101.
Sawyer, A., Dick, L., Shapiro, E., & Wilson, H. (2020). Adapting to survive: An exploration of
elementary mathematics teachers’ adaptation of online resources [Manuscript submitted
for publication]. Middle, Secondary, and Mathematics Education Department, College of
Education, James Madison University.
166

Sawyer, A., Dick, L., Shapiro, E., & Wismer, T. (2019). The top 500 mathematics pins: Analysis
of elementary mathematics activities on Pinterest. Journal of Technology and Teacher
Education, 27(2), 235-263.
Sawyer, A., Dick, L., & Sutherland, P. (2020). Online mathematics teacherpreneurs developers
on Teachers Pay Teachers: Who are they and why are they popular? Education Sciences,
10. doi:10.3390/educsci10090248
Sawyer, A., Dredger, K., Myers, J., Barnes, S., Wilson, R., Sullivan, J., & Sawyer, D. (2020).
Developing teachers as critical curators: Investigating elementary preservice teachers’
inspirations for lesson planning. Journal of Teacher Education, 71(5), 518–536.
doi:10.1177/0022487119879894
Sawyer, A., MacNeille, M., & Dick, L. (2020). Comparing the quality of paid vs. free
elementary mathematics activities found online (Conference presentation). Association of
Mathematics Teachers Educators 2020 Conference (held remotely).
Sawyer, A., & Myers, J. (2018). Seeking comfort: How and why preservice teachers use internet
resources for lesson planning. Journal of Early Childhood Teacher Education, 39(1), 16–
31. doi:10.1080/10901027.2017.1387625
Schreier, M. (2014). Qualitative content analysis. In U. Flick (Ed.), The SAGE handbook of
qualitative data analysis (pp. 170–183). London: Sage Publications.
Schroeder, S., Curcio, R., & Lundgren, L. (2019). Expanding the learning network: How
teachers use Pinterest. Journal of Research on Technology in Education, 51(2), 166–186.
doi:10.1080/15391523.2019.1573354
Selvi, A. (2019). Qualitative content analysis. In The Routledge handbook of research methods
in applied linguistics (pp. 440-452). Routledge.
167

SEMrush. (2019): SEMrush Ranking Factors Study 2.0 (Technical report). Boston, MA:
SEMrush.
https://www.thewebconsulting.com/media/media/SEMrush_Ranking_Factors_Study_2_0
.pdf
Shapiro, E., Sawyer, A., Dick, L., & Wismer, T. (2019). Just what online resources are
elementary mathematics teachers using?. Contemporary Issues in Technology and
Teacher Education, 19(4), 670-686.
Shelton, C., & Archambault, L. (2018). Discovering how teachers build virtual relationships and
develop as professionals through online teacherpreneurship. Journal of Interactive
Learning Research, 29(4), 579-602.
Shelton, C., & Archambault, L. (2019a). Who are online teacherpreneurs and what do they do? A
survey of content creators on teacherspayteachers.com. Journal of Research on
Technology in Education, 51(4), 398–414. doi:10.1080/15391523.2019.1666757
Shelton, C., & Archambault, L. (2019b). Teachers paying teachers? Four guidelines for school
leaders. ISTE Blog. https://www.iste.org/explore/professional-development/teachers-
paying-teachers-four-guidelines-school-leaders
Shelton, C., & Archambault, L. (2020). Learning from and about elite online teacherpreneurs: A
qualitative examination of key characteristics, school environments, practices, and
impacts. Teachers College Record, 122(7).
Shelton, C., Archambault, L., & Harris, L. (2020). Lesson plan platforms for teachers have a
racism problem. Slate. https://slate.com/technology/2020/08/teachers-paying-teachers-
racist-materials.html
168

Shelton, C., Koehler, M., Greenhalgh, S., & Carpenter, J. (2021). Lifting the veil on
TeachersPayTeachers. com: An investigation of educational marketplace offerings and
downloads. Learning, Media and Technology, 1-20.
Shelton, C., Schroeder, S., & Curcio, R. (2020). Instagramming their hearts out: What do edu-
influencers share on Instagram?. Contemporary Issues in Technology and Teacher
Education, 20(3), 529-554.
Sherin, M., & Drake, C. (2009). Curriculum strategy framework: Investigating patterns in
teachers’ use of a reform‐based elementary mathematics curriculum. Journal of
Curriculum Studies, 41(4), 467–500. doi:10.1080/00220270802696115
Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational
Researcher, 15(2), 4-14.
Silver, D. (2022). A theoretical framework for studying teachers’ curriculum
supplementation. Review of Educational Research, 92(3), 455-489.
Smarter Balanced. (2020). 10 years of smarter collaboration. https://smarterbalanced.org/our-
vision/our-history/
Smith, A., Westberg, K., & Hejny, A. (2017). Accelerated Reader program: What do teachers
really think. International Journal of Higher Education, 6(3), 138-146.
doi:10.5430/ijhe.v6n3p138
Staudt Willet, K. (2019). Revisiting how and why educators use Twitter: tweet types and
purposes in #Edchat. Journal of Research on Technology in Education, 51(3), 273–289.
doi:10.1080/15391523.2019.1611507
169

Superfine, A., Marshall, A., & Kelso, C. (2015). Fidelity of implementation: Bringing written
curriculum materials into the equation. The Curriculum Journal, 26(1), 164–191.
doi:10.1080/09585176.2014.990910
Takacs, Z., & Bus, A. (2018). How pictures in picture storybooks support young children’s story
comprehension: An eye-tracking experiment. Journal of Experimental Child
Psychology, 174, 1-12.
Tarr, J., Chavez, O., Reys, R., & Reys, B. (2006). From the written to the enacted curricula: The
intermediary role of middle school mathematics teachers in shaping students' opportunity
to learn. School Science and Mathematics, 106(4), 191–201. doi:10.1111/j.1949-
8594.2006.tb18075.x
Thomas-Hunt, M. C., & Phillips, K. W. (2004). When what you know is not enough: Expertise
and gender dynamics in task groups. Personality and Social Psychology Bulletin, 30(12),
1585-1598.
Toma, J. (2011). Approaching rigor in applied qualitative research In C. Conrad & R. Serlin
(Ed.), The SAGE Handbook for Research in Education: Pursuing Ideas as the Keystone
of Exemplary Inquiry (pp. 263-280). Thousand Oaks: Sage Publications, Inc.
Torphy, K., Chen, Z., Yang, X., & Chhikara, D. (2019, October 4). The impacts of teachers’
curation of mathematics resources on students’ achievement [Paper presentation].
Midwest Sociology of Education Conference, Evanston, IL.
Torphy, K., & Drake, C. (2019). Educators meet the fifth estate: The role of social media in
teacher training. Teachers College Record, 121(14).
Torphy, K., Hu, S., Liu, Y., & Chen, Z. (2020). Teachers turning to teachers: Teacherpreneurial
behaviors in social media. American Journal of Education, 127(1), 49-76.
170

Torphy, K., Ishmael, K., Song, J., South, J., Benko, S., Hodge, E., Salloum, S., Mardis, M., &
Morales, R. (2018). Creating systems of sustainability: Four focus areas for the future of
PK-12 open educational resources (Technical report). Washington, DC: International
Society for Technology in Education.
Torphy, K., Liu, Y., Hu, S., & Chen, Z. (2020). Sources of professional support: Patterns of
teachers’ curation of instructional resources in social media. American Journal of
Education, 127(1), 13-47.
Tosh, K., Doan, S., Woo, A., & Henry, D. (2020). Digital instructional materials: What are
teachers using and what barriers exist? (Technical report). Santa Monica, CA: RAND
Corporation.
Tourangeau, R., Rips, L., & Rasinski, K. (2000). The psychology of survey response.
Cambridge, England: Cambridge University Press.
Trust, T., Krutka, D., & Carpenter, J. (2016). “Together we are better”: Professional learning
networks for teachers. Computers & Education, 102, 15–34.
doi:10.1016/j.compedu.2016.06.007
Tyack, D. (1976). Ways of seeing: An essay on the history of compulsory schooling. Harvard
Educational Review, 46(3), 355-389.
https://doi.org/10.17763/haer.46.3.v73405527200106v

171

United States Department of Education, National Center for Education Statistics. (2019). Table
209.22: Number and percentage distribution of teachers in public elementary and
secondary schools, by instructional level and selected teacher and school characteristics:
1999-2000 and 2017-18. In United States Department of Education, National Center for
Education Statistics (Ed.), Digest of Education Statistics (2019 ed.).
https://nces.ed.gov/programs/digest/d19/tables/dt19_209.22.asp
Valentine, J. (2019). Judging the quality of primary research. In Cooper, H., Hedges, L., &
Valentine, J. (Eds.), The handbook of research synthesis and meta-analysis. New York,
NY: Russell Sage Foundation.
Wang, E., Tuma, A., Doan, S., Henry, D., Lawrence, R., Woo, A., & Kaufman, J. (2021).
Teachers’ Perceptions of What Makes Instructional Materials Engaging, Appropriately
Challenging, and Usable. (Technical report). Santa Monica, CA: Rand Corporation.
Ware, H. & Kitsantas, A. (2007). Teacher and collective efficacy beliefs as predictors of
professional commitment. The Journal of Education Research, 100(5), 303-310. doi:
10.3200/JOER.100.5.303-310
Watkins, M. (2018). Exploratory factor analysis: A guide to best practice. Journal of Black
Psychology, 44(3), 219-246. Doi: 10.1177/0095798418771807
Webel, C., Krupa, E., & McManus J. (2015). Teacheres' evaluations and use of web-based
curriculum resources in relation to the Common Core State Standards for mathematics.
Middle Grades Research Journal, 10(2), 49-64.
Webster, M., & Thatcher, D. (2014). Common Core State Standards: Answers to legislators’
frequently asked questions. (Technical report). Washington, D.C.: National Conference of
State Legislatures.
172

Yeigh, M., Villagomez, A., Lenski, S., McElhone, D., & Larson, M. (2017). Teachers’
professional judgment in contrast with policy mandates: Instructional material adoption
and students’ needs. Northwest Journal of Teacher Education, 12(2).
doi:10.15760/nwjte.2017.12.2.5
Zhao, Y., & Frank, K. (2003). Factors affecting technology uses in schools: An ecological
perspective. American Educational Research Journal, 40(4), 807-840.

173

Appendix A

Survey Item-to-Construct Crosswalk

Table A1. Survey Items, Target Constructs, and Subscales
Survey Item Construct Targeted Subscale (if
applicable)
Which of the following best describes the
industry that you, personally, work in?
(Sample definition)
Which of the following best describes your
current profession?
(Sample definition)
What grade(s) do you teach? (Check all
that apply.)
(Sample definition)
(Teacher context)

What subject(s) do you teach? (Check all
that apply. All-subjects teachers (e.g., most
elementary teachers) will check multiple
boxes.)
(Sample definition)
(Teacher context)

Which best describes your school? (Sample definition)
(School context)

Now, think about the official material that
you use the most. This is often, but not
always, a textbook. For how many years
have you used that material, including this
year?
(Teacher context)
*To what extent is this material aligned to
the learning standards for the grade you
teach?
Orientation toward
official material
Orientation toward
official material scale
*To what extent is this material aligned to
your teaching style?
Orientation toward
official material
Orientation toward
official material scale
*To what extent is this material easy for
you to use?
Orientation toward
official material
Orientation toward
official material scale
*To what extent does this material help
your students learn?
Orientation toward
official material
Orientation toward
official material scale
*How likely would you be to recommend
this official material to a colleague?
Orientation toward
official material
Orientation toward
official material scale
*Overall, how satisfied are you with this
official material?
Orientation toward
official material
Orientation toward
official material scale
Materials I already owned personally Source unofficial
offline
Source unofficial
offline
Materials I created myself Source unofficial
offline
Source unofficial
offline
Materials I found at my school(that are not
part of my official curriculum)
Source unofficial
offline

Materials I got from a colleague I know
personally
Source unofficial
offline

174

Materials I got from a Google search Source unofficial
online
Source unofficial
online
Materials I got from a website dedicated to
or heavily featuring curriculum
supplements (e.g.,teacherspayteachers,
betterlesson, pinterest, etc.)
Source unofficial
online
Source unofficial
online
Materials I got from another teacher’s
personal blog or website
Source unofficial
online
Source unofficial
online
Materials from an online open curriculum
(e.g., engageNY)that my school does NOT
already use as official curriculum
Source official
Materials I checked out from a library Source official
†
Materials I bought at a school supplies
store
Source unofficial
offline

I want a material that better aligns to
learning standards
ReasonStudent-based ReasonStudent-
based
I want a material that better aligns to year-
end state tests
ReasonStudent-based ReasonStudent-
based
I want a material that better aligns to unit
tests or interim tests
ReasonStudent-based ReasonStudent-
based
I want a material that better fits my
students' academic characteristics (e.g.,
learning style, EL status, IEP status)
ReasonStudent-based ReasonStudent-
based
I want a material that better fits my
students non-academic characteristics (e.g.,
interests, family income, race/ethnicity)
ReasonStudent-based ReasonStudent-
based
I want a material that is more difficult for
students than my school’s official
curriculum
ReasonStudent-based
I want a material that is less difficult for
students than my school’s official
curriculum
ReasonStudent-based
I want a material that better engages my
students
ReasonStudent-based ReasonStudent-
based
I want a material that is easier for me to
use
ReasonTeacher-based ReasonTeacher-
based
I want a material that requires less prep
time to use
ReasonTeacher-based ReasonTeacher-
based
I want a material that takes students less
time to complete (e.g., I don't have enough
class time to use the official curriculum
materials)
ReasonTeacher-based ReasonTeacher-
based
I want a material that takes students more
time to complete (e.g., I have extra time
after I finish lessons in my official
curriculum)
ReasonTeacher-based
175

I want a material that is more culturally
responsive
ReasonStudent-based ReasonStudent-
based
I want a material that does not cost much
money
ReasonTeacher-based
I want a material that is more topical (e.g.,
more relevant to current events)
ReasonStudent-based ReasonStudent-
based
I have to use a supplemental material
because my school does not provide all the
materials I need to teach the official
curriculum
ReasonContext-based
My school administration encourages me
to use supplemental materials
ReasonContext-based
The material's alignment to learning
standards
Feature Quality
The material's alignment to assessments Feature Quality
The material's overall difficulty Feature Usability
The material's price Feature Usability
The material is recommended by teachers I
know personally
Feature Quality Feature Quality
The material is recommended by teachers I
don't know personally (e.g., website
comments)
Feature Quality Feature Quality
The material is culturally relevant to my
students
Feature Quality Feature Quality
The material is accurate (e.g., free from
math errors)
Feature Quality Feature Usability
The material's images are helpful Feature Usability Feature Usability
The material has a clear layout/formatting Feature Usability Feature Usability
The material is visually appealing overall Feature Fluff Feature Usability
The material is easily modifiable(e.g.,
word document vs. PDF)
Feature Usability Feature Usability
The material is easy to use Feature Usability Feature Usability
The material is aligned to my teaching
style
Feature Fluff Feature Usability
The material is similar to my official
curriculum materials
Feature Usability Feature Integratabilit
y
The material is easy to implement online
(e.g., for remote instruction)
Feature Usability Feature Integratabilit
y
**How the material is intended to be used
(i.e., designed for whole group vs. small
group vs. individual work)
Feature Usability
†
The material is easy for parents (or
whoever helps my students at home) to
understand
Feature Usability
Think about all the materials you use to
teach math across the year. About what
Use
PatternPrevalence

176

proportion of those materials are
supplemental curriculum materials?
†
When you look for supplemental
materials, about how often do you look
specifically for materials you can use in
whole-group instruction?
Use Pattern Grouping
†
When you look for supplemental
materials, about how often do you look
specifically for materials you can use in
small-group instruction?
Use Pattern Grouping
†
When you look for supplemental
materials, about how often do you look
specifically for materials you can use in
one-on-one instruction?
Use Pattern Grouping
†
When you look for supplemental
materials, about how often do you look
specifically for materials that students can
complete on their own?
Use Pattern Grouping
†
How important is the way students will be
grouped (e.g., whole-group, small-group,
individual) when making your decision of
what supplemental materials to use?
Use Pattern Grouping
†
For about how many minutes do you
usually anticipate using any given
supplemental material?
Use Pattern Timing
†
About when in a typical lesson do you
anticipate using supplemental materials?
Use Pattern Timing
†
About when in a typical unit do you
anticipate using supplemental materials?
Use Pattern Timing
*I like to use math problems that can be
solved in many different ways.
Commitment to math
education reform
Commitment to math
education reform scale
*I regularly have my students work
through real-life math problems that are of
interest to them.
Commitment to math
education reform
Commitment to math
education reform scale
*When two students solve the same math
problem correctly using two different
strategies, I have them share the steps they
went through with each other.
Commitment to math
education reform
Commitment to math
education reform scale
*I tend to integrate multiple strands of
mathematics within a single unit.
Commitment to math
education reform
Commitment to math
education reform scale
*I often learn from my students during
math time because my students come up
with ingenious ways of solving problems
that I have never thought of.
Commitment to math
education reform
Commitment to math
education reform scale
*It is not very productive for students to
work together during math time.
Commitment to math
education reform
Commitment to math
education reform scale
177

*Every child in my room should feel that
mathematics is something he/she can do.
Commitment to math
education reform
Commitment to math
education reform scale
*I integrate math assessment into most
math activities.
Commitment to math
education reform
Commitment to math
education reform scale
*In my classes, students learn math best
when they can work together to discover
mathematical ideas.
Commitment to math
education reform
Commitment to math
education reform scale
*I encourage students to use manipulatives
to explain their mathematical ideas to other
students.
Commitment to math
education reform
Commitment to math
education reform scale
*When students are working on math
problems, I put more emphasis on getting
the correct answer than on the process
followed.
Commitment to math
education reform
Commitment to math
education reform scale
*Creating rubrics for math is a worthwhile
assessment strategy.
Commitment to math
education reform
Commitment to math
education reform scale
*In my class, it is just as important for
students to learn data management and
probability as it is to learn basic arithmetic
facts.
Commitment to math
education reform
Commitment to math
education reform scale
*I don't necessarily answer students' math
questions, but rather let them puzzle things
out for themselves.
Commitment to math
education reform
Commitment to math
education reform scale
*A lot of things in math must simply be
accepted as true and remembered.
Commitment to math
education reform
Commitment to math
education reform scale
*I like my students to master basic
mathematical operations before they tackle
complex problems.
Commitment to math
education reform
Commitment to math
education reform scale
*I teach students how to explain their
mathematical ideas.
Commitment to math
education reform
Commitment to math
education reform scale
*Using computers to solve math problems
distracts students from learning basic math
skills.
Commitment to math
education reform
Commitment to math
education reform scale
*If students use calculators, they won't
master the basic math skills they need to
know.
Commitment to math
education reform
Commitment to math
education reform scale
*You have to study math for a long time
before you see how useful it is.
Commitment to math
education reform
Commitment to math
education reform scale
Including this year, how many years have
you been a teacher?
(Teacher context)
Including this year, how many years have
you been teaching math in [grade level]?
(Teacher context)
What type of teaching certification do you
have?
(Teacher context)
What is your age? (Teacher context)
178

How would you describe your racial and
ethnic identity? (Check all that apply.)
(Teacher context)
English language learners

(Classroom context)
Students with individualized education
plans (IEPs) or 504plans

(Classroom context)
Black, Hispanic, and/or Native American
students

(Classroom context)
Students who performed below grade level
on their most recent year-end state math
assessment

(Classroom context)
Students who qualify for free or reduced
lunch programs
(Classroom context)
To what extent does your school have an
expectation (either explicit or implicit) that
teachers teach the school’s official math
curriculum with few or no modifications?
(School context)
How do administrators at your school feel
about teachers using supplemental
curriculum materials?
(School context)
*I am generally satisfied with being a
teacher at my school.
Commitment to
teaching
Commitment to
teaching scale
*I sometimes feel it is a waste of time to
try to do my best as a teacher.
Commitment to
teaching
Commitment to
teaching scale
*If you could go back to your college days
and start over again, would you become a
teacher or not?
Commitment to
teaching
Commitment to
teaching scale
*How long do you plan to remain in
teaching?
Commitment to
teaching
Commitment to
teaching scale
*This item is not “officially” part of the supplementation survey tool. It was included in this study to
generate additional data.
**I recommend removing this item from future version of this survey.
†
This item is a preliminary draft version of an item I recommend adding to future versions of this
survey. (These should undergo cognitive testing before being relied on for valid inferences.)
179

Appendix B

Cognitive Interview Protocol

1. Intro to me:
a. Thanks for participating
b. I’m a former 3
rd
grade teacher, now researcher at USC.
c. Time check: 10min, 15 tops?; check email address for compensation
d. Permission to record
2. Hit record
3. Why we’re here
a. Cognitive interviews common step between writing and opening to full sample
b. You’ll help us ensure that the survey is as clear as possible and that we’re actually
asking about what we intend to be
4. Outline:
a. 2 broad questions about the survey in general
b. 3 questions about specific survey items
i. I can screen share PDF of the survey if it’s helpful to refer to
c. then a few minutes for any thoughts / feedback you have
5. Broad questions
a. If someone else asked you what the survey was about, what would you tell them?
i. If it’s been more than a day or two, “how did you feel this survey did at
gathering information about your curriculum supplementation practices?”
b. Did the survey feel like it was missing anything that you’d have wanted to include
about your curriculum supplementation?
c. (Flipside) Did the survey feel repetitive, like it asked about the exact same thing
multiple times?
6. Item-specific questions
a. Official textbook quality question: Different answer if we asked “how likely to
recommend to peer” vs. “how would you rate overall quality”?
i. What about “how satisfied are you with this material” vs “how would you
rate overall quality”?
b. Slider question: proportion of mats that are supps
i. What was answering this question like? (hard/easy, etc.)
ii. Could have asked about proportion of teaching (not mats) that uses
supplemental materials.
iii. Is one of those easier for you to think about / remember?
iv. Does one of those seem more useful/important if the goal is to understand
how you use supplemental materials?
c. Question toward end: To what extent does your school have an expectation (either
explicit or implicit) that teachers teach the school’s official math curriculum with
few or no modifications?
i. How did you go about answering this question?
ii. Was it easy/straightforward to answer? (If no, why?)
7. Any further feedback / thoughts / questions you have?
8. Outro: thanks again, share dsilver@usc.edu for any follow-up
180

Appendix C

Full Survey Text with Changes Noted

181

182

• The four items under “continue thinking about…” and the two slider items on this page form the
“orientation toward official material” scale.
• The first slider item on this page changed from “How would you rate the overall quality of this material?”
based on cognitive interviews.
• The second slider item on this page was added based on cognitive interviews.
183

184

185

186

• I recommend removing the bottom item under design features (“How the material is intended…”) based on
exploratory factor analysis.
• The 20 items under “Commitment to standards-based…” (from Ross et al., 2003) were added after
cognitive interviews. They form the “commitment to mathematics education reform” scale.
187

188

189

• The last item on this page and the first three on the next (from Ware & Kitsantas, 2007) were added after
cognitive interviews. They form the “commitment to teaching” scale.
190

• The language under “Wrap-up” was updated slightly after cognitive interviews. (“20 minutes” became
“30-40 minutes” and “$10” became “$20.”)
191

Appendix D

Exploratory Factor Analysis Results

Table D2. Supplement Source Rotated Factors
Factor Variance Explained
1 1.23951
2 1.17590
3 0.62765
4 0.57636
Factor retained

192

Table D4. Supplement Reason Rotated Factors
Factor Variance Explained
1 2.04552
2 1.74331
3 1.58979
4 1.48964
5 0.93950
6 0.68445
7 0.08085
8 0.05678
Factor retained

Table D3. Supplement Reason Factor Loadings
Item Factor 1 Factor 2 Factor 3 Factor 4
1: Want material better aligned to standards 0.1730 0.5184 0.4379 0.1190
2: Want material better aligned to state tests 0.2265 0.6974 0.0951 0.1029
3: Want material better aligned to unit tests 0.1674 0.7066 0.2004 0.0985
4: Want material to better fit Ss academic characteristics 0.1339 0.1927 0.6796 0.0806
5: Want material to better fit Ss non-academic characteristics -0.0106 0.1739 0.4402 0.5097
6: Want material that is more difficult 0.1118 0.2773 -0.0229 0.1926
7: Want material that is less difficult 0.2709 0.0904 0.2663 0.1320
8: Want material that better engages Ss 0.1354 0.1271 0.6632 0.2343
9: Want material that is easier for T to use 0.7007 0.1741 0.1717 0.1036
10: Want material that requires less prep time 0.7801 0.1509 0.1084 0.0561
11: Want material that takes Ss less time 0.5223 0.3000 -0.0388 0.2495
12: Want material that takes Ss more time 0.3147 0.2118 -0.0987 0.2427
13: Want material that is more culturally responsive 0.1922 0.0411 0.2558 0.6430
14: Want material that does not cost much money 0.3809 0.0784 0.2001 0.3008
15: Want material that is more topical 0.1597 0.2426 0.0845 0.5863
16: School does not provided all materials for official curriculum 0.3486 0.2100 0.0811 0.1944
17: School administration encourages supplementation 0.1164 0.1467 0.1069 0.2430
Item retained as part of factor
Item did not load onto any retained factor

193

Table D5. Supplement Feature Rotated Factors
Factor Variance Explained
1 3.14181
2 1.36371
3 1.29719
4 0.96993
5 0.85014
6 0.25825
7 0.08540
8 0.06882
9 0.04684
10 0.02712
Factor retained

Table D6. Supplement Feature Factor Loadings
Item Factor 1 Factor 2 Factor 3
1: Alignment to standards 0.3931 0.0277 -0.0198
2: Alignment to assessments 0.1485 0.3117 0.1762
3: Overall difficulty 0.0855 0.0608 0.1503
4: Price 0.1293 -0.0390 0.1579
5: Recommendations from personal colleagues 0.2929 0.1248 0.5873
6: Recommendations from other teachers 0.0207 0.2137 0.6435
7: Cultural relevance 0.2600 0.1928 0.4645
8: Accuracy 0.5379 0.1104 -0.0091
9: Image helpfulness 0.7439 0.0721 0.2102
10: Clarity of layout/formatting 0.7896 0.1393 0.0663
11: Overall visual appeal 0.6087 0.2547 0.1093
12: Modifiability 0.4883 0.3821 0.1719
13: Ease of use 0.4978 0.2288 -0.0283
14: Alignment to teaching style 0.4886 0.2295 0.1821
15: Similarity to official curriculum 0.3029 0.5207 0.1003
16: Ease of online implementation 0.2067 0.6597 0.2630
17: Intended grouping (individual/small/whole) 0.3017 0.3213 0.2003
Item retained as part of factor
Item did not load onto any retained factor

194

Appendix E

Recruitment Email and Observation/Interview Protocol

Recruitment email

Subject: Scheduling a follow-up interview to curriculum survey

Hello,

My name is Dan Silver, and I am a member of the research team behind a survey you took a few
weeks ago about the ways you use curriculum materials in your teaching. I am emailing because
you provided this email address at the end of the survey, to indicate that you would be interested
in helping us learn more about the ways teachers use and supplement their curriculum via a 30-
40 minute Zoom interview. Thank you for doing this!

As a reminder, we are offering a $20 Amazon gift card as compensation for participating in this
follow-up interview. At the bottom of this email, I provide more information about how the
interview will be structured.

I know this can be a busy time, so I want to be as accommodating of your schedule as possible. I
can be available any time between about 7am and 8pm Pacific time on any day for the rest of this
week (June 6-10).

Please reply with a time slot (let’s say 45 minutes, to be safe) when you can meet on any of
these days and I’ll send a zoom link. If none of these times work for you, let me know and we
can find a different time that does.

Thank you,
Dan Silver

Interview details:
This interview is structured a little differently than typical interviews. You can think about it as
having two parts, each of which will take about 15-20 minutes.

In the first part, I will provide you with a hypothetical scenario in which you might want to use
supplemental curriculum materials. It will be something like: “Imagine your students are having
a hard time with [math topic or idea relevant to your grade level] in math and that you decide to
supplement your official, school-provided curriculum to help with this concept, either by finding
or creating resources for your students to use.” For the next 15 or so minutes, you will pretend
you really are looking to help your students with that topic/idea and, as best you can, will think
aloud as you work to either create or find resources. During this process, I’ll ask you to share
your screen using Zoom’s screen share function (it’s easy to use – I’ll help if you have any
trouble with it) so we can get a better sense of how this important work actually unfolds in real
195

time. For example, we are interested in what resources or applications you use, what websites
you visit if any, and anything else you think or do as you work to supplement your curriculum
materials. I’ll be mostly silent during this time, since your think-aloud is what we’re really
interested in.

After about 15 minutes, even if you haven’t finished creating or finding a resource(s), we’ll
move to the second part of the interview, which will run more traditionally. For the second part,
you can stop screen sharing, and I’ll ask you questions about your process and decision-making
both during the simulated activity and in your teaching more generally. This conversation will
last about 15-20 minutes and will dig into your approach to curriculum more deeply than our
survey was able to. Each interview will be recorded for research purposes, with recordings fully
anonymized and stored securely. If you have any further questions, please don’t hesitate to ask.

196

Observation and Interview Protocol

1. Intro to me:
a. Thanks for being a part of this follow-up interview
b. I’m a former 3
rd
grade teacher, now researcher at USC
c. Time check: 40min?
2. Housekeeping
a. Confirm email address for sending compensation
b. Confirm grade worked with
i. [wrote in grade for survey data here]
c. Get screen share up and running
d. Start recording
3. Broad framing
a. We’re interested in learning more about how teachers find and make decisions
about supplemental curriculum materials, anything they use BESIDES what the
school provides. In particular, we want to know more about how teachers make
decisions, what kinds of resources they use, and their overall thought processes.
4. Outline (2 parts: simulation then interview, as outlined in my first email)
a. Simulation:
i. I’ll drop a task in the chat, you’ll spend 15-20 minutes doing it while
thinking your thoughts aloud, stream-of-consciousness style.
ii. I know we’re at a computer—if there are parts of the task for which you
wouldn’t typically use a computer, do your best to describe what you’d do
instead.
iii. I’m interested in your thought process, so I’ll stay mostly silent.
b. Interview:
i. I’ll pause you after 15-20 minutes. Totally fine if you aren’t done.
ii. We’ll talk about this activity AND your decision-making during it and
during your typical lesson planning / supplementation.
iii. There will also be time for you to add any of your own reflections,
thoughts, or questions.
c. Any questions about this?
5. Let’s get started!
a. Paste prompt into chat, clarify prompt as needed
b. Note key decision points, areas of uncertainty or surprising certainty, etc.
i. [took notes during observation here, used them to inform interview and
memo right afterward]
ii.
iii.
iv.
v.

197

6. Interview options (subject to change based on observation):
a. What were you looking for in a supplemental material?
i. Important features of the material?
ii. Thinking of specific misconceptions?
iii. Trying to make things more interesting/fun relative to textbook?
iv. Trying to make things easier to plan/teach?
b. What made you end up choosing [material]?
i. How confident were you in this choice?
ii. Why didn’t you choose [other option]?
c. How might you actually use this material?
i. When in a lesson? // Individual vs group? // Repeated or one-time thing? //
For everyone or just a few students?
d. It seems like you thought a lot about [decision point].
i. What were you considering?
ii. Why did you end up making the decision you did?
1. What were some of the tradeoffs there?
2. Why didn’t you choose [alternative]?
e. Zoom out: What was the task like for you overall? Easy/hard?
i. How did you know what to do?
ii. Have you ever had PD/formal learning about supplementation?
Grade Prompt
K Imagine that your students are struggling with decomposing numbers between 10
and 20 into a ten and some number of ones.
You decide to supplement your official, school-adopted curriculum to help them
with this concept, either by finding or creating resources for your students to use.
1 Imagine that your students are struggling with representing numbers of objects
between 50 and 120 with written numerals.
You decide to supplement your official, school-adopted curriculum to help them
with this concept, either by finding or creating resources for your students to use.
2 Imagine that your students are struggling with understanding that three-digit
numbers are composed of hundreds, tens, and ones.
You decide to supplement your official, school-adopted curriculum to help them
with this concept, either by finding or creating resources for your students to use.
3 Imagine that your students are struggling with rounding whole numbers to the
nearest ten.
You decide to supplement your official, school-adopted curriculum to help them
with this concept, either by finding or creating resources for your students to use.
4 Imagine that your students are struggling with recognizing that each place in a
multi-digit number represents ten times what the place to its right represents.
You decide to supplement your official, school-adopted curriculum to help them
with this concept, either by finding or creating resources for your students to use.
5 Imagine that your students are struggling with recognizing that each place in a
multi-digit number represents one-tenth of what the place to its left represents.
You decide to supplement your official, school-adopted curriculum to help them
with this concept, either by finding or creating resources for your students to use.
198

1. Would you want it?
f. Did this activity differ in important ways from how you might usually supplement
in your day-to-day teaching?
g. Any further reflections / thoughts / questions?
7. Outro: Thanks again, share dsilver@usc.edu for any follow-up
a. If you haven’t gotten compensation in about a week, let me know
199

Appendix F

Coding Frames

Initial Coding Frame, with Selected Edits Noted
Code Definition Example Further Rule(s)
Transcript Coding
Unrelated to
supplementation
1

Statement unrelated to
teacher curriculum
supplementation
(based on data)
Dimension:
Reason
Any of a teacher’s
motivations for searching
for or deciding on a
supplemental material
Subcodes:
2

teacher-related / student-
related / context-related
3
/
other
Dimension:
Source
4

Any website or real-life
entity that a teacher gets a
supplemental material from
Subcodes:
online unofficial / offline
unofficial / official / other
Dimension: Use
Pattern
A teacher’s anticipated (or
actual) usage of the
supplement (e.g., frequency,
timing, prevalence among
official materials, etc.)
Subcodes:
frequency / prevalence
(relative to other mats) /
timing / other
Dimension:
Design Feature
5

Any aspect of a
supplemental material itself
that a teacher mentions
6

Subcodes:
usability for T
7
/ usability
for S
8
/ academic quality /
non-academic quality /
other
Dimension: Other Any factor that seems to be
driving supplementation
decision making but does
not fit one of the
dimensions identified in the
TCSF

Key decision
point
The teacher makes a
decision about what
9

supplement to use or how to
use a supplement
Subcodes:
implicit / explicit
10

On-Screen Coding (Leave codes on transcript in second “layer”)
11

Visits webpage Teacher opens webpage Subcodes:
new supp. site / new page
of same supp. site /
official curriculum site /
other official resource
(e.g., CCSS website) /
other
200

Selected Edits to Initial Coding Frame
1
Don’t code truly unrelated stuff; add “official material” and “generally interesting—revisit”
which would have been coded as this.
2
All subcodes still too general. Need to add a layer of sub-subcodes.
3
Can be school context, can also be larger context. E.g., official material “geared toward white
entitled kids” so need to supplement to teacher her kids what they need to access material.
4
ADD CODE: previously-used resource. Don’t apply to entire websites but to specific
pages/images/etc. that would be set in front of the kids. Do NOT code every new item with a
source; just the actual source of the items.
5
If there’s something that seems like a feature of a supplement source rather than a supplement
itself, it probably fits under “reason” (e.g., “I like education.com because I already pay for the
subscription”)
6
“…that a teacher mentions or factors into their thinking without mentioning” (includes absent
or hypothetical aspects. E.g., “I need to know what it says to them when they get it wrong”)
7
Related specifically to features that make it easier for a teacher to use the resource (e.g.,
adaptability, price, etc.) It may be that a teacher feels that a resource’s prettiness is what makes it
usable for them, but prettiness is related to non-academic quality—the teacher could use the
resource without that prettiness, they just don’t think it would be as good.
8
Specifically related to students’ ability to engage with the resource. (A feature like a resource
being at a student’s academic level is coded as academic quality although it may well also ensure
that the resource is usable to the student.)
9
“…makes a decision that may inform what supplement to use or…”
10
Ts pretty good at making their thinking explicit in think alouds – no need for “implicit”
subcode, so no need for subcodes “Key Decision Point” code isn’t actually giving additional
info—just a bunch of sources and throat-clearing statements coded with this. No one really had
epiphanies while working that seemed notable. (Which is good—means the task just felt like
another day on the job to them, which was intended.)
11
These codes aren’t adding anything; relevant on-screen activity noted in the transcripts
themselves so they got coded alongside everything else

Note: not necessarily
world wide web. Code
applies to shared Google
drives, etc.
Opens word doc Teacher opens word doc (or
Google doc, etc.)

Screen / transcript
mismatch
Whatever is happening on
screen is in conflict /
tension with what teacher is
thinking aloud

Non-computer
description
Teacher describes aspects of
their process for which they
wouldn’t typically use a
computer.

201

Final Coding Frame, At a Glance

L0 Code L1 Code L2 Code L3 Code
Generally Interesting (I created some subcodes but they aren't include . .
Official Material Positive Orientation toward Official Material . .
Negative Orientation toward Official Material . .
No Stated Orientation toward Official Material . .
Previously-used Resource . . .
Dimension Design Feature Academic Quality Aligns to Other Materials
Aligns to Tests or Standards
Contributes to Learning
Multimodality
Visual Support (Deeper Understanding)
Whether On- or Off-Topic
Other
Non-Academic Quality Aesthetics
Appropriateness
Engagement and Fun
Other
Usability for Students Academic Appropriateness
Developmental Appropriateness
Formatting
Visual Support (Accessibility)
Other
Usability for Teacher Academics
Classroom Culture or Management
Convenience
Money
Time
Other
Other .
Reason Context-Related COVID
District or State Expectations
Lack of Essential Materials
Larger Social Contexts
School Expectations
Other
Student-Related Appropriateness
Engagement and Fun
Learning and Academics
Other
Teacher-Related Academics
Aesthetics
Colleagues and Peers
Convenience
Money
Teacher’s Background
Teacher’s Identity
Time
Other
202

Teacher-Related Academics
Aesthetics
Colleagues and Peers
Convenience
Money
Teacher’s Background
Teacher’s Identity
Time
Other
Other .
Source Official (Still a Supplement) Standards and Related Documents
Supplemental Part of Official Material
Supplemental Tool Offered by District
Other
Offline Unofficial Colleagues
Manipulatives not Part of Official Material
Self-Created
Teacher’s Curated Pool of Materials
Other
Online Unofficial Games
Google Search
Infrastructure (Google Forms, etc.)
Other VRPs
Pinterest
Teachers Pay Teachers
Videos and Songs
Worksheet Generators
Other
Other .
Use Pattern Frequency .
Grouping .
Prevalence .
Timing Absolute
Relative
Other
Other .
Other . .
203

Final Codes in Detail

204

205

206

207

208

209

210

211

212

213

214

Abstract (if available)

Abstract It is reasonable to imagine that teachers have supplemented their officially-adopted textbooks with supplemental materials for as long as schools and districts have provided materials to their teachers. National surveys have shown teacher curriculum supplementation to be widely-practiced across U.S. classrooms, but despite a robust research literature on official textbooks, teachers’ supplementation practices have only become a focus of empirical work in the last decade or less. Perhaps because it is a relatively new area of research, teacher curriculum supplementation is often conceived in disparate ways that do not always translate well across research studies, and there are still many open questions about the process of teacher curriculum supplementation, in practice. This dissertation contains three papers designed to deepen our understanding of teacher curriculum supplementation. In the first paper, I systematically review existing supplementation-relevant literature, highlight seven areas of consensus, and offer the Teacher Curriculum Supplementation Framework (TCSF) as a theoretical framework for studying this important practice. In the second paper, I design a survey measure of supplementation grounded in the TCSF, then collect and analyze evidence about the tool’s ability to provide valid inferences about the practice. In the third, my coauthor and I conduct observations of in-service teachers during a simulated supplementation activity, then interview them about their practice, yielding insights about teachers’ in-the-moment tendencies and considerations while they supplement their official curriculum materials. I believe teacher curriculum supplementation is a lever that can be—and often already is—harnessed for instructional improvement, and this dissertation provides a conceptual grounding for future study of the practice alongside new evidence about its nature.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Examining the sociopolitical and racial beliefs of United States history teachers

PDF

A document analysis of the math curriculum on teachers pay teachers

PDF

Disruptions to the traditional textbook narrative: lessons from district leaders and teachers in California

PDF

Sociocognitive and neurophysiological contributors to effective secondary teaching

PDF

Care and social-emotional well-being: organizational conditions in policy and practice

PDF

Participation in higher education diversity, equity, and inclusion work: a relational intersectionality of organizations analysis

PDF

Levels of interest: the effects of teachers' unions on Congress, statehouses, and schools

PDF

Navigating the future of teacher professional development: Three essays on teacher learning in informal and digital context

PDF

Choosing wisely: a three paper dissertation exploring how parents evaluate and choose schools

PDF

Urban teacher residencies and their impact on teacher retention in high-needs school settings

PDF

Tough conversations and missed opportunities: implementing district policies for racial equity

PDF

No place like home: a three paper dissertation on K-12 student homelessness & housing affordability

PDF

From meaning‐making to expansive learning: how contradictions shape teachers' implementation of technology‐based personalized learning

PDF

California teacher residency leaders’ understanding and implementation of the characteristics and evidence of an effective teacher residency

PDF

Examining how teacher pre-service/credential programs prepare teachers to teach BIPOC students

PDF

Building networks for change: how ed-tech coaches broker information to lead instructional reform

PDF

Culturally-relevant, autonomy supportive instruction: toward an integration for enhancing the motivation of racially and ethnically diverse learners

PDF

A study of alignment between the Common Core State Standards in math and an introductory college textbook

PDF

Spatial and temporal expenditure-pricing equity of rail transit fare policies

PDF

Quiet voices hidden curriculum: evaluating culturally responsive curriculum in secondary English textbooks

Asset Metadata

Creator Silver, Daniel Mark (author)

Core Title Teacher curriculum supplementation as phenomenon and process

School Rossier School of Education

Degree Doctor of Philosophy

Degree Program Urban Education Policy

Degree Conferral Date 2022-12

Publication Date 08/23/2022

Defense Date 08/22/2022

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag curriculum,education policy,mixed methods,OAI-PMH Harvest,supplementation,Teachers

Format application/pdf (imt)

Language English

Contributor Electronically uploaded by the author (provenance)

Advisor Polikoff, Morgan (committee chair), Aguilar, Stephen (committee member), Marsh, Julie (committee member), Saavedra, Anna (committee member)

Creator Email dsilver@usc.edu

Permanent Link (DOI) https://doi.org/10.25549/usctheses-oUC111379830

Unique identifier UC111379830

Legacy Identifier etd-SilverDani-11155

Document Type Dissertation

Format application/pdf (imt)

Rights Silver, Daniel Mark

Type texts

Source 20220825-usctheses-batch-975 (batch), University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA

Repository Email cisadmin@lib.usc.edu