Dr. Amber Northern’s study, Reimagining the Institute of Education Sciences, offers thoughtful guidance on how to improve IES activities. With respect to the report’s attention to the National Center for Education Statistics (NCES) more specifically, there are sentiments that echo long-standing critiques, while other points suggest new directions and priorities. Drawing on my decades at NCES, I wanted to reflect on some key points that I feel are critical to strengthen NCES’s ability to inform education policy decisions and address emerging issues. The key points I want to address are timing, priorities, NCES management, and the broader federal statistical system.
Immediate steps should be taken to preserve core NCES functions and support the work of external groups to guide the development and implementation of longer-term efforts to improve education statistics.
Timing
Some of the report’s most important recommendations involve long-term development activities, such as creating new surveys or retooling surveys to capture essential elements from cancelled survey systems. New surveys can take years to develop given the complexity of the U.S. education system and the need for all states, localities, and education institutions to respond and implement new requirements across their own systems. Some NCES surveys are conducted through postsecondary institutions, as well as state and local education authorities. Before the collection of comparable data can begin, these respondents need time to adapt their own reporting systems to ensure that information is reported consistently across the United States. These longer-term efforts could be expedited through additional funding for state and local data infrastructure or further development of their AI capabilities; however, many challenges remain due to the varying capacities of education authorities to respond to new data reporting requirements. Completely new systems may be feasible in the coming years that seem impractical today if there was more flexibility in public and private sector data sharing. Privacy restrictions that made sense decades ago when laws were enacted are no longer adequate or reflect reality. These evolving concerns and solutions with unknown time dimensions point to the value of an ongoing external NCES advisory body along the lines of what Dr. Northern suggested in her study.
Priorities
Critical institutional knowledge is being lost now and there is a need to release annual data promptly that are essential to local, state, federal, and business decision makers. These releases include year to year changes in such fundamental statistics as student, teacher, and school counts, school expenditures, and postsecondary completions. These basic data from the Common Core of Data (CCD) and Integrated Postsecondary Education Data System (IPEDS), along with NCES projections and processed elements of the Current Population Survey and American Community Survey should be available in accessible formats with basic descriptions as soon as possible. NCES formerly released these data annually in tabular formats that were easily accessed by the general public without need of statistical and programming skills. Merely posting complex data sets online that require the use of statistical packages is not adequate to ensure widespread use of these official statistics and substantially compromises the utility of the data for smaller state and local education agencies which lack the technical and time resources to use such information. Making these data accessible now would fulfill critical data needs, such as tracking increases and decreases in student enrollment, and help inform the dimensions of new or revised surveys to be implemented over the medium and longer term.
Hand-in-hand with the need to differentiate short-term and longer-term implementation strategies is the long-held concern about the timeliness of NCES data. Dr. Northern’s critique of this data weakness has been echoed by a broad range of constituents, with similar public concerns about the statistics produced by other federal agencies. Substantial improvements may be made to the NCES annual data collections and reporting by reducing internal barriers within the review process. Some of the delays have been related to lack of staffing and others to the limited flexibility within the NCES review system. In addition to faster survey processing and reviews, some improvements in timelines could be made for CCD and IPEDS by using more imputed data for nonrespondents in early release files to be followed by final files released at a later time. Other approaches could focus on improving state capacity to report data more quickly for CCD and EDFacts, which would also improve the utility of state- and local- data systems. Some barriers should be addressed by tailoring the NCES review process to specific products, with an emphasis on modernizing the review. Modernization should not be equated with large investments in dedicated new systems. There should be an expectation that rapid advances in AI will enable low-cost adaptations to existing systems, provided they are adequately validated and documented. Modernizing systems for long-term improvements will require some investment in staffing and contract support now.
Substantial improvements may be made to the NCES annual data collections and reporting by reducing internal barriers within the review process.
NCES Management
Dr. Northern’s report makes salient observations about the siloed nature of NCES and IES systems. Within NCES, some responsibility for the siloed nature of activities was related to the reorganization associated with the IES implementation. The NCES commissioner’s office was reduced from a substantial team with budget and technical expertise to essentially just the Commissioner alone. This made it difficult to coordinate and manage some $300 million annually in activities across the agency. The NCES survey divisions were forced to operate nearly autonomously. Over the years, one or two additional staff were incorporated into the commissioner’s office, but lack of internal NCES coordination, support, and oversight was always a shortcoming. The lack of coordination across divisions was compounded by changing the division structure from separate elementary/secondary and postsecondary divisions to having one division defined as an “administrative” and another as a “sample” survey division. This further fostered data silos by separating survey units by statistical methodologies that should have been aligned to their common elementary/secondary or postsecondary constituents. Building up the capabilities of the NCES commissioner’s office would improve the efficiency of budget and staffing management. It would also reduce the siloed nature of survey systems, particularly if NCES was reorganized to better align with the U.S. education system.
Staffing at NCES had been an issue for many years, with a staff of about 100 managing a $300 million budget. Following the reduction in force of last year, the staff is now about a dozen, and most contracts have been cancelled. This level of staffing is not adequate to implement the improvements that Dr. Northern and others have recommended. While increased staffing is required across NCES, an expanded commissioner’s office would also facilitate implementation of external feedback by creating an effective point of contact that had more capacity to respond to external inquiries and implement changes. Thus, the NCES commissioner’s office would be adequately resourced to support and respond positively to the suggestion Dr. Northern made about an NCES advisory group. An agency with the scope of NCES would benefit from long-term feedback from technical and subject matter experts.
Building up the capabilities of the NCES commissioner’s office would improve the efficiency of budget and staffing management.
Federal Statistics Management
It is important to note that NCES is part of the federal statistics system which includes 13 primary statistical agencies, 3 units, and dozens of other federal agencies with statistical activities. The Reimagining IES study makes thoughtful suggestions about reducing duplicative efforts across federal agencies. The Evidence Act also envisions more work in coordinating federal statistics to improve the quality and quantity of data for decision-making. While much additional work needs to be done, NCES and other agencies already coordinate in the production of statistics and development of statistical methodologies and concepts. Other federal agencies partner with NCES to collect data that meet their needs such as the Department of Health and Human Services funding for the NCES Early Childhood Longitudinal Study and NCES funding for the October education supplement for the Current Population Survey. The Census Bureau is the primary contractor for administering many NCES and other federal agency surveys, which eliminates the need for duplicative survey field staff. NCES partners with other agencies to produce reports and studies, such as with the Bureau of Justice Statistics and Interagency Forum on Child and Family Statistics. Obstacles remain in the sharing of sensitive data such as using Internal Revenue Service income data to reduce income questions across federal surveys. Expanding the use of external data sources such as credit bureau data could also augment federal statistics. Improved and interoperable data tools and systems could be fostered by an expanded Office of the Chief Statistician in the Office of Management and Budget (OMB).
More coordinated activities involving the use of AI across the federal statistical system could reduce duplicative statistical development costs and improve the taxpayer return on investment.
The Office of the Chief Statistician, with responsibilities to coordinate and guide the activities of federal statistical agencies, has operated with a small staff that is significantly reliant on short-term temporary staff from other federal agencies. The larger OMB Office of Information and Regulatory Affairs reviews proposed federal surveys to standardize questions and reduce duplicative and unnecessary paperwork burden on the American public. While these OMB offices fulfill these essential roles, their capacity to foster, sustain, and share innovations throughout the federal statistical system is limited. As an example, NCES data tools have been criticized as being primitive and hard-to-use; however, they represent pioneering efforts to enable sophisticated statistical procedures and computation of complex standard errors that are not supported in many other federal data tools. Innovations such as these should be more extensively shared among federal agencies. The rapidly expanding capability of AI offers new opportunities for the efficient development of data tools and documentation.
More coordinated activities involving the use of AI across the federal statistical system could reduce duplicative statistical development costs and improve the taxpayer return on investment. Although AI use has become ubiquitous, there are specific applications within the federal statistical system where major savings in terms of agency resources and user efforts could be made. The production of survey data set documentation and analyses of complex surveys can be very time consuming and expensive. To achieve the maximum cost savings in using AI for these processes, more work is needed to determine how best to structure, implement, review, and validate AI work on survey documentation. At present, full use of many federal data sets requires statisticians and programmers with training on the specific surveys. More work is needed to determine how to array the data sets and documentation to enable future AI software to ingest the information and produce valid responses to user questions. This approach would enable the widest access to federal data and reduce the need for more specialized survey specific data tools. While major challenges for AI enabled applications and processes will be encountered, federal statistical agencies should be supported through interagency teams to resolve challenges and adapt solutions with efficiency and access benefits for all agencies and data consumers.
Summary
There is an immediate need to release more accessible information from ongoing NCES surveys such as CCD and IPEDS and fully staff all current NCES surveys especially the National Assessment of Educational Progress. Staffing within NCES generally and in the Commissioner’s office specifically needs to be brought up to levels that support ongoing surveys and reporting systems and enables the development of longer-term initiatives and new surveys. These longer-term initiatives to improve the efficiency and effectiveness of NCES require thoughtful discussions across the scope of potential beneficiaries including students, families, instructors, decision-makers, researchers, and policymakers. NCES must have the capacity to support these discussions by providing information on statistical parameters, survey procedures, legislative requirements, legal frameworks, and other matters that will guide final decisions that will inevitably involve hard decisions and compromises. I feel that immediate steps should be taken to preserve core NCES functions and support the work of external groups to guide the development and implementation of longer-term efforts to improve education statistics.
Tom Snyder is one of the former NCES staff who joined Agora Education Research as an associate. He spent 40 years at NCES, where he directed the Digest of Education Statistics and oversaw the agency’s annual reports activities.
Leave a Reply