Losing the richness of education data: Arts education and teacher research impacts

We at Agora Education Research set out to ask researchers and data users about what the loss of access to NCES data really means for them, their fellow researchers and students, and what the downstream implications might be. 

Today’s installment features excerpts from a conversation with Dr. Ken Elpus, Professor of Music Education and Associate Director for Faculty Affairs & Graduate Studies in the School of Music at the University of Maryland. 

• Dr. Ken Elpus from UMD shares how losing access to rich, national data like NCES’ affects the field of arts education and downstream users of the data.

How did a music educator get involved with large statistical datasets from the National Center for Education Statistics (NCES)?

“I first came to these datasets in their public-use form, in a graduate school class with Larry Hedges.¹ This was quantitative research preparation while I was in my music education graduate studies. At the time, Larry was on the National Assessment Governing Board, NAGB, and the class ended up covering a whole bunch of statistical innovations that were developed specifically because of educational statistical needs. For example, how to handle clustering was really focused on data such as how to account for students within schools, schools within districts, etc. At one point, he had a unit on large scale assessment, and we had to do projects, and I took a deep dive into the frameworks of NAEP (National Assessment of Educational Progress, https://www.nationsreportcard.gov/) and the arts NAEP questions that had been released. Then I found the data, dug further, found the longitudinal studies, and realized that Larry had given me the tools to use that data, downloaded the public-use Education Longitudinal Study of 2002 (ELS:2002)², and did a back-of-the-napkin analysis about kids that had been in music, which formed the spark of my dissertation. That ended up being my first published article, which started as a simple demographic study. It turned into ‘well what happened to these kids…’ … and then that became the start of my dissertation. Because no one had been doing large-scale secondary analyses of arts education in the literature, I got the article in press quickly. The fact that the article was in press got me my job at UMD at the time.”

Work that Elpus pursued as a faculty member went on to further impact the field and shed important light on arts education for policymakers, such as the finding that taking arts coursework in high school does not hinder successful college admissions outcomes, despite the common thinking that students should take as many academic courses as possible.³ Another important finding was that arts students were nearly thirty percent more likely to apply to a postsecondary institution than were non-arts students, even controlling for other characteristics.⁴ His work has also linked arts course-taking in adolescence with significant reductions in the likelihood for disciplinary actions such as out-of-school suspensions.⁵

As a faculty member, Elpus had resources to gain access to the restricted-use data and provide students with the opportunity to gain quantitative skills and experience with large datasets. The field of research on arts education and arts educator pipelines could grow.   

Fast forward to now, what is happening as a result of NCES data becoming harder to access?

“The loss of NTPS⁶ meant we lost the benchmark to understand the population of music teachers. Say I go work on my own studies, locally, as has been suggested. If I’m going to create my own studies, how can I create the weights or design the studies if I don’t know how many music teachers there are? Or know anything about the population of music teachers?” 

Elpus and his lab have have submitted requests to NCES for access to the newest High School and Beyond Longitudinal Study of 2022 (HS&B:22)⁷ and the National Postsecondary Student Aid Study of 2016 (NPSAS:16)⁸ but they have not yet been met. They are waiting to hear about those data requests. One data request would support a project related to college financing of pre-service educators and the competitiveness of merit-based financial assistance. There are questions about more debt for some kinds of pre-service educators. “These are fundamental questions that could inform the field. Their answers might affect philanthropic approaches to generating donations for scholarships, to keep pre-service educators on track.”

When asked about the impact the loss of NCES data access has had on junior researchers or postdocs in his lab and department, he answered, “Part of our lab’s goal is to build capacity for researchers to do this work in arts education. That forward progress has stopped. I don’t want to be one of a small handful of scholars doing this work, or who are able to do this work.”

He has tried to find substitute datasets. “The biggest challenge for arts education research without using NCES data is the loss of the richness. NCES data are collected with ancillary information that were not in the basic study design. For example, complete transcript data are collected in some of the longitudinal studies, so any outcome can be compared between arts and non-arts students.” In studies such as HSLS:09⁹ and NPSAS, he explains, the course and major codes are collected, allowing for comparisons between students’ course-taking pathways. The national representativeness of the studies enables this. “Most other datasets available are not nearly as comprehensive in terms of the participant characteristics. It is much easier to do these studies using NCES data than other resources.” 

What are some downstream implications to data access loss?

There are several perspectives from which to view the impacts, from the national to the very local. Broadly, there’s a great impact on the field in terms of knowledge lost. Elpus says, “Take equity of access – if there are benefits to the access of arts in the schools… then if we are systematically denying some students access to those benefits… why?”

Elpus goes on to explain that there are policymakers and decisionmakers faced with scarce resources and difficult choices every fiscal year. “If some of them need to see a piece of research suggesting why they should continue to invest in arts education programs, and the research on why this is a worthwhile investment is absent, then our policymakers are given incomplete information. And uninformed advocacy is just wishful thinking.”

“Uninformed advocacy is just wishful thinking,” says Dr. Ken Elpus, Professor of Music Education at the University of Maryland.

“We know many decision makers want the data. Even one board member, convinced of the meaningfulness of the investment, could mean hundreds of kids affected by the resulting access.” However, while arts and music educators would like to be able to provide these data on an individual level whenever requested, Elpus says, it’s really not in their purview. The teachers are there to educate the students. 

In terms of the downstream consequences to each student… “think about what choices the student is faced with… their scarcest resource is their time. If someone is advising the student that they shouldn’t pursue their artistic talent because they should double up in a core subject, then that student misses the opportunity to go out and make what could be a meaningful impact on the world. If we believe that school is a place that should nurture the talents of every particular student, then that is something we need to document, and something that we should pay as much attention to as standardized tests.” 

_________________________

1. https://www.ipr.northwestern.edu/who-we-are/faculty-experts/hedges.html ↩︎
2. https://nces.ed.gov/surveys/els2002/ ↩︎
3. Elpus, K. (2018). Estimating the effect of music and arts coursework on college admissions outcomes. Arts Education Policy Review, 119(3), 111-123. https://www.tandfonline.com/doi/abs/10.1080/10632913.2016.1201030 ↩︎
4. Elpus, K. (2014). Arts education as a pathway to college: College admittance, selectivity, and completion by arts and non-arts students. National Endowment for the Arts, 1-56. https://www.arts.gov/sites/default/files/Research-Art-Works-Maryland2-rev.pdf ↩︎
5. Elpus, K. (2013). Arts education and positive youth development: Cognitive, behavioral, and social outcomes of adolescents who study the arts. National Endowment for the Arts, 1-56. https://www.arts.gov/sites/default/files/Research-Art-Works-Maryland.pdf ↩︎
6. The National Teacher and Principal Survey, https://nces.ed.gov/surveys/ntps/ ↩︎
7. https://nces.ed.gov/surveys/hsb ↩︎
8. https://nces.ed.gov/surveys/npsas ↩︎
9. The High School Longitudinal Study of 2009, https://nces.ed.gov/surveys/hsls09 ↩︎