National Education Goals Panel

In this report, the term "significance" refers to statistical significance and indicates that change over time is not likely to have occurred by chance. The majority of indicators in this report are based on samples and not entire populations. For example, mathematics achievement results were obtained by sampling a portion of the nation's 4th, 8th, and 12th graders. This enables the nation and the states to use smaller, cost-efficient samples to predict how the entire student population would have performed on an assessment without testing all of them. This is similar to a public opinion poll that predicts, with a certain degree of confidence, how all individuals would have responded to a set of questions had they all been polled.

It is important to note that any estimate based on a sample contains a small amount of imprecision, or sampling error. The estimate would be slightly higher or slightly lower if a different sample were chosen. Public opinion polls account for this error when they caution that their results are "accurate within plus or minus three percentage points."

If we want to determine whether the nation and the states have made progress over time, we must apply a statistical test to tell us whether there are likely to be differences in actual performance over time in the entire population. The statistical test takes into account not only the difference between the measures, but also the precision of the estimate for each measure. If the test indicates that there are likely to be differences in performance between groups in the entire population, we say that the difference is statistically significant. This means that the differences are not likely to have occurred by chance, and we can be confident that performance has changed over time.

All differences in this report that are termed "statistically significant" are measured at the 0.05 level. For formulas and more detailed technical information, see the following sections on "accuracy of data," "sampling errors," and "non-sampling errors."

Accuracy of Data. The accuracy of any statistic is determined by the joint effects of "sampling" and "nonsampling" errors. Estimates based on a sample will differ somewhat from the figures that would have been obtained if a complete census had been taken using the same survey instruments, instructions, and procedures. In addition to such sampling errors, all surveys, both universe and sample, are subject to design, reporting, and processing errors and errors due to nonresponse. To the extent possible, these nonsampling errors are kept to a minimum by methods built into the survey procedures. In general, however, the effects of nonsampling errors are more difficult to gauge than those produced by sampling variability.

Sampling Errors. The samples used in surveys are selected from a large number of possible samples of the same size that could have been selected using the same sample design. Estimates derived from the different samples would differ from each other. The difference between a sample estimate and the average of all possible samples is called the sampling deviation. The standard or sampling error of a survey estimate is a measure of the variation among the estimates from all possible samples and, thus, is a measure of the precision with which an estimate from a particular sample approximates the average result of all possible samples.

The sample estimate and an estimate of its standard error permit us to construct interval estimates with prescribed confidence that the interval includes the average result of all possible samples. If all possible samples were selected under essentially the same conditions and an estimate and its estimated standard error were calculated from each sample, then: 1) approximately 2/3 of the intervals from one standard error below the estimate to one standard error above the estimate would include the average value of the possible samples and 2) approximately 19/20 of the intervals from two standard errors above the estimate to two standard errors below the estimate would include the average value of all possible samples. We call an interval from two standard errors below the estimate to two standard errors above the estimate a 95 percent confidence interval.

Analysis of standard errors can help assess how valid a comparison between two estimates might be. The standard error of a difference between two independent sample estimates is equal to the square root of the sum of the squared standard errors of the estimates. The standard error (se) of the difference between independent sample estimates "a" and "b" is:

To compare changes in between-group differences (groups "a" and "b") over time (years "1" and "2"), we approximate the standard error of the difference as:

This method overestimates the standard error because it does not account for covariance. Because of this overestimation, the approach is conservative; that is, one is less likely to obtain significant results.

Nonsampling Errors. Universe and sample surveys are subject to nonsampling errors. Nonsampling errors may arise when respondents or interviewers interpret questions differently; when respondents must estimate values; when coders, keyers, and other processors handle answers differently; when persons who should be included in the universe are not; or when persons fail to respond (completely or partially). Nonsampling errors usually, but not always, result in an understatement of total survey error and, thus, an overstatement of the precision of survey estimates. Since estimating the magnitude of nonsampling errors often would require special experiments or access to independent data, these magnitudes are seldom available.

NAEP is a survey of the educational achievement of American students and changes in that achievement across time. Since 1969, NAEP has assessed the achievement of national samples of 9-, 13-, and 17-year-old students in public and private schools. In 1983, it expanded the samples so that grade-level results could be reported.

The assessments, conducted annually until the 1979-1980 school year and biennially since then, have included periodic measures of student performance in reading, mathematics, science, writing, U.S. history, civics, geography, and other subject areas. NAEP also collects demographic, curricular, and instructional background information from students, teachers, and school administrators.

National Assessment Governing Board (NAGB) Achievement Levels. The National Education Goals Panel has set its performance standard at the two highest levels of achievement- Proficient or Advanced- on NAEP. These levels were established by the National Assessment Governing Board in 1992 and are reported by the National Center for Education Statistics (NCES).

Basic: This level, below Proficient, denotes partial mastery of knowledge and skills that are fundamental for proficient work at each grade - 4, 8, and 12. For 12th grade, this is higher-than-minimum competency skills (which are normally taught in elementary and junior high school) and covers significant elements of standard high-school-level work.

Proficient: This central level represents solid academic performance for each grade tested - 4, 8, and 12. It reflects a consensus that students reaching this level have demonstrated competency over challenging subject matter and are well prepared for the next level of schooling. At Grade 12, the Proficient level encompasses a body of subject-matter knowledge and analytical skills, and of cultural literacy and insight, that all high school graduates should have for democratic citizenship, responsible adulthood, and productive work.

Advanced: This higher level signifies superior performance beyond Proficient grade-level mastery at grades 4, 8, and 12. For 12th grade, the Advanced level shows readiness for rigorous college courses, advanced training, or employment requiring advanced academic achievement.

NAGB has established standards for reporting the results of the National Assessment of Educational Progress. This effort has resulted in three achievement levels: Basic, Proficient, and Advanced. The NAGB achievement levels are reasoned judgments of what students should know and be able to do. They are attempts to characterize overall student performance in particular subject matters. Readers should exercise caution, however, in making particular inferences about what students at each level actually know and can do. A NAEP assessment is a complex picture of student achievement, and applying external standards for performance is a difficult task. Evaluation studies have raised questions about the degree to which the standards in the NAGB achievement levels are actually reflected in an assessment and, hence, the degree to which inferences about actual performance can be made from these achievement levels. The Goals Panel acknowledges these limitations but believes that, used with caution, these levels convey important information about how American students are faring in reaching Goal 3. They are also consistent with the Panel's efforts to report such performance against a high-criterion standard.

The percentage of infants at risk is based on whether the infant was born with one or more of four risk factors. Risks are late (in third trimester) or no prenatal care, low maternal weight gain (less than 21 pounds), mother smoked during pregnancy, or mother drank alcohol during pregnancy. However, the National Center for Health Statistics notes that alcohol use during pregnancy is likely to be underreported on the birth certificate.

Source: Nicholas Zill and Christine Winquist Nord of Westat developed the concept of the Children's Health Index. Stephanie Ventura of the National Center for Health Statistics provided the special tabulations of the birth certificate data needed to produce the index.

The Goals Panel reports 1994 data as the baseline year for immunizations. This was the first year for which data were collected using the National Immunization Survey (NIS). In prior years, the Centers for Disease Control and Prevention collected data on immunizations using the National Health Interview Survey (NHIS). The Goals Panel does not compare data from NIS and NHIS, due to methodological differences between the two instruments.

"Two-year-olds" are defined as children 19 to 35 months of age. "Fully immunized" is defined as four doses of diphtheria-tetanus-pertussis vaccine, three doses of polio vaccine, and one dose of measles or measles-mumps-rubella vaccine.

Source: National Immunization Survey, (various years), Centers for Disease Control and Prevention. Morbidity and Mortality Weekly Report.

The population estimates for the National Household Education Survey (NHES) cover 3- to 5-year-old children who are not yet enrolled in kindergarten. "Parents" includes parents or other family members. Figures combine responses of "read to every day" and "told a story three or more times a week."

In the NHES:93, information on daily reading was collected using two approaches with split-half samples. The two approaches did not result in significantly different estimates for daily reading to 3- to 5-year-old preschoolers. A combined measure using both items for NHES:93 is included in this report.

Sources: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey: 1993 School Readiness Interview, unpublished tabulations prepared by Westat, August 1994.

U.S. Department of Education, (various years), National Center for Education Statistics, National Household Education Survey: Parent Interview, unpublished tabulations prepared by Westat.

The population estimates for the NHES cover 3- to 5-year-old children who are not yet enrolled in kindergarten. "Preschool participation" includes children enrolled in any center-based program, including nursery schools, prekindergarten programs, preschools, day care centers, and Head Start.

"High income" is defined as a family income of $50,000 or more. "Low income" is defined as a family income of $10,000 or less.

Sources: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey: 1991 Early Childhood Component, unpublished tabulations prepared by Westat, August 1994.

U.S. Department of Education, (various years), National Center for Education Statistics, National Household Education Survey: Parent Interview, unpublished tabulations prepared by Westat.

The high school completion rates for 18- to 24-year-olds are computed as a percentage of the non-high school enrolled population at these ages who hold a high school credential (either a high school diploma or an alternative credential, such as a General Educational Development (GED) certificate, Individualized Education Program (IEP) credential, or certificate of attendance).

Source: U.S. Department of Commerce, Bureau of the Census, (various years), October Current Population Surveys, unpublished tabulations prepared by the National Center for Education Statistics and MPR Associates, Inc.

In 2000 grade 4 students were assessed at the national level. Students in grades 8 and 12 were not assessed in 2000.

Sources: Donahue, P., Voelkl, K., Campbell, J., & Mazzeo, J. (1999). NAEP 1998 reading report card for the nation and the states. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

U.S. Department of Education, National Center for Education Statistics (2001), The Nations Report Card: 4th Grade Reading 2000. Washington, DC.

During 1999, student achievement levels were established for writing by the National Assessment Governing Board. The percentages of U.S. 4th, 8th, and 12th graders who performed at the two highest levels of achievement - Proficient or Advanced - on the 1998 NAEP writing assessment are presented for the first time in the 1999 Goals Report and Data Volume. This information replaced data that were previously reported from the 1992 NAEP Writing Portfolio Study before the student achievement levels were available.

Source: Greenwald, E., Persky, H., Campbell, J., & Mazzeo, J. (1999). NAEP 1998 writing report card for the nation and the states. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Source: Braswell, J., Lutkus, A., Grigg, W., Santapau, S., Tay-Lim, B., & Johnson, M. (2001) The Nation's Report Card; Mathematics 2000. Washington, DC: National Center for Education Statistics.

Source: Bourque, M.L., Champagne, A., & Crissman, S. (1997). 1996 science performance standards: Achievement results for the nation and states, a first look. Washington, DC: National Assessment Governing Board.

Source: National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 1998 civics assessment, Washington, DC.

Source: Williams, P.L., Lazer, S., Reese, C.M., & Carr, P. (1995). 1994 NAEP U.S. history: A first look. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Source: Williams, P.L., Reese, C.M., Lazer, S., & Shakrani, S. (1995). 1994 NAEP world geography: A first look. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Methodological changes in the 2000 Schools and Staffing Survey required that this indicator be substantially revised. Previously, this indicator reported the percentage of full time secondary school teachers who were certified and/or had a college major (not including second majors or minors) in their main teaching assignment. The assignment categories were: mathematics, science, English, social studies, fine arts and special education.

The revised indicator reports the percentage of full- or part-time secondary school teachers who were certified and had a college major (including second majors and minors) in any subject field taught.

Source: U.S. Department of Education, (various years), National Center for Education Statistics, Teacher Surveys of the Schools and Staffing Survey, 1990-1991, 1993-1994, and 1999-2000 unpublished tabulations prepared by Westat and NCES.

Methodological changes in the 2000 Schools and Staffing survey required that this indicator be substantially revised. Previously, this indicator reported the percentage of teachers who reported participating in 1 or more of 4 professional development topics since the end of the previous school year. The 4 topics were teaching methods, subject field, students assessment and educational technology. The revised indicator reports the percentage participating in 1 or more of 3 topics, which are teaching methods, subject field and student assessment.

Source: U.S. Department of Education, (various years), National Center for Education Statistics, Teacher Survey of the Schools and Staffing Survey, unpublished tabulations prepared by Westat.

For the Third International Mathematics and Science Study (TIMSS), the following countries did not meet international guidelines at Grade 4: Australia, Austria, Hungary, Israel, Kuwait, Latvia (LSS), Netherlands, Slovenia, and Thailand. In England, more than 10% of the population was excluded from testing at Grade 4. In England and Scotland, a participation rate of 75% of the schools and students combined for Grade 4 was achieved only after replacements for refusals were substituted.

The following countries did not meet international guidelines at Grade 8: Australia, Austria, Belgium (French), Bulgaria, Colombia, Denmark, Germany, Greece, Israel, Kuwait, Netherlands, Romania, Scotland, Slovenia, South Africa, and Thailand.

In four countries, more than 10% of the population was excluded from testing at Grade 8: England, Germany, Israel, and Lithuania. In Belgium (Flemish), England, Germany, Latvia (LSS), Switzerland, and the United States, a participation rate of 75% of the schools and students combined for Grade 8 was achieved only after replacements for refusals were substituted.

For the 1999 TIMSS, Israel did not meet international guidelines at Grade 8. Furthermore, Lithuania tested the same cohort of students as other nations, but later in 1999, at the beginning of the next school year.

Data from the 1995 TIMSS are not comparable with data from the 1999 administration of the test. Therefore change information are not presented.

Sources: U.S. Department of Education, National Center for Education Statistics. (1996). Pursuing excellence: A study of U.S. eighth-grade mathematics and science teaching, learning, curriculum, and achievement in international context. NCES 97-198. Washington, DC: U.S. Government Printing Office.

U.S. Department of Education, National Center for Education Statistics. (1997). Pursuing excellence: A study of U.S. fourth-grade mathematics and science achievement in international context. NCES 97-255. Washington, DC: U.S. Government Printing Office.

U.S. Department of Education, National Center for Education Statistics. (1998). Pursuing excellence: A study of U.S. twelfth-grade mathematics and science achievement in international context. NCES 98-049. Washington, DC: U.S. Government Printing Office.

Martin et al. (2000) TIMSS 1999 International Science Report: Findings from IEA's repeat of TIMSS at the eighth grade. Chestnut Hill, MA: Boston College.

Martin et al. (2000) TIMSS 1999 International Mathematics Report: Findings from IEA's repeat of TIMSS at the eighth grade. Chestnut Hill, MA: Boston College.

Data include only U.S. citizens and resident aliens on permanent visas. Degrees awarded by institutions in the outlying areas are included in the U.S. percentages.

Mathematical sciences is the only field of study included in the mathematics category for this report. Fields of study in the science category for this report include: engineering; physical sciences; geosciences; computer science; life sciences (includes medical and agricultural sciences); and science and engineering technologies (includes health technologies). Note that these data have been modified from previously published data - in 2000, the social sciences were retroactively taken out of the sciences field.

Source: Integrated Postsecondary Education Data System, (various years), conducted by the National Center for Education Statistics. The data were analyzed by Westat, using the National Science Foundation's WebCASPAR Database System.

The U.S. Department of Education and the Educational Testing Service (ETS) characterized the literacy of America's adults in terms of three "literacy scales" representing distinct and important aspects of literacy: prose, document, and quantitative literacy. Each of the literacy scales has five levels, with Level 1 being least proficient and Level 5 being most proficient.

Prose literacy, selected as a national indicator for this report, is defined as the knowledge and skills needed to understand and use information from texts that include editorials, news stories, poems, and fiction - for example, finding a piece of information in a newspaper article, interpreting instructions from a warranty, inferring a theme from a poem, or contrasting views expressed in an editorial. The five levels are:

Level 1 - Most of the tasks in this level require the reader to read relatively short text to locate a single piece of information which is identical to or synonymous with the information given in the question or directive. If plausible but incorrect information is present in the text, it tends not to be located near the correct information.

Level 2 - Some tasks in this level require readers to locate a single piece of information in the text; however, several distractors or plausible but incorrect pieces of information may be present, or low-level inferences may be required. Other tasks require the reader to integrate two or more pieces of information or to compare and contrast easily identifiable information based on a criterion provided in the question or directive.

Level 3 - Tasks in this level tend to require readers to make literal or synonymous matches between the text and information given in the task, or to make matches that require low-level inferences. Other tasks ask readers to integrate information from dense or lengthy text that contains no organizational aids such as headings. Readers may also be asked to generate a response based on information that can be easily identified in the text. Distracting information is present, but is not located near the correct information.

Level 4 - These tasks require readers to perform multiple-feature matches and to integrate or synthesize information from complex or lengthy passages. More complex inferences are needed to perform successfully. Conditional information is frequently present in tasks at this level and must be taken into consideration by the reader.

Level 5 - Some tasks in this level require the reader to search for information in dense text which contains a number of plausible distractors. Others ask readers to make high-level inferences or use specialized background knowledge. Some tasks ask readers to contrast complex information.

Source: Kirsch, I.S., Jungeblut, A., Jenkins, L., & Kolstad, A. (1993). Adult literacy in America: A first look at the results of the National Adult Literacy Survey, p. 17. Washington, DC: U.S. Department of Education, National Center for Education Statistics.

Adults 17 years old and older who participated in one or more adult education activities on a full-time, but not on a part-time, basis in the previous 12 months are excluded from both the numerator and denominator in the calculations of adult education participation.

Source: U.S. Department of Education, (various years), National Center for Education Statistics, National Household Education Survey: Adult Education Component, unpublished tabulations prepared by Westat.

The indicator reports the disparities in college entrance rates between non-Hispanic White and minority high school graduates who enroll in 2 or 4 year colleges immediately after high school graduation, and are based on three-year averages (1989-1991 for 1990; 1996-1998 for 1997). College completion rates are based on adults aged 25 to 29. "College" includes junior colleges, community colleges, and universities. "College degree" includes Associate's degrees, Bachelor's degrees, and graduate/professional degrees.

Sources: U.S. Department of Commerce, Bureau of the Census, (various years), October Current Population surveys, unpublished tabulations from the National Center for Education Statistics, prepared by Pinkerton Computer Consultants, Inc.

U.S. Department of Commerce, Bureau of the Census, (various years), March Current Population Surveys, unpublished tabulations from the National Center for Education Statistics, prepared by Pinkerton Computer Consultants, Inc.

Use of any illicit drug includes any use of marijuana, hallucinogens, cocaine, heroin, inhalants, or any use of stimulants or tranquilizers not under a doctor's orders.

Source: Johnston, L.D., O'Malley, P.M., & Bachman, J.G. (various years). Selected outcome measures from the Monitoring the Future Study for Goal 7 of the National Education Goals: A special report for the National Education Goals Panel. Ann Arbor: University of Michigan, Institute for Social Research.

Student victimization involves threats and injuries to students include those made with or without a weapon.

Sources: U.S. Department of Education, National Center for Education Statistics, Fast Response Survey System, Teacher Survey on Safe, Disciplined, and Drug-free Schools, FRSS 42, unpublished tabulations prepared by Westat, August 1994.

U.S. Department of Education, (various years), National Center for Education Statistics, Teacher Survey of the Schools and Staffing Survey, unpublished tabulations prepared by Westat.

Student Reports: Percentage represents responses from students who reported that during an average week, misbehavior by other students interfered with their own learning six times a week or more.

Source: Johnston, L.D., O'Malley, P.M., & Bachman, J.G. (various years). Selected outcome measures from the Monitoring the Future Study for Goal 7 of the National Education Goals: A special report for the National Education Goals Panel. Ann Arbor: University of Michigan, Institute for Social Research.

Teacher Reports: Percentage represents responses from secondary school teachers who "agreed" or "strongly agreed" that student misbehavior interferes with their teaching.

Source: U.S. Department of Education, (various years), National Center for Education Statistics, Teacher Surveys of the Schools and Staffing Survey, unpublished tabulations prepared by Westat.

Survey respondents were principals or their designees. "More than half" included responses of "more than half" and "most or all" combined. Data include only those public schools in which the school reported that it held regularly scheduled schoolwide parent-teacher conferences during the year.

An elementary school was any school where the highest grade identified on the survey questionnaire was 6 or lower. A middle school was any school where the highest grade identified was 7 or 8, and three or fewer grades were served. All other schools (for example, where the highest grade identified was 7 or 8, and more than three grades were served) were not included in the analysis.

Source: U.S. Department of Education, National Center for Education Statistics, Fast Response Survey System, Survey on Family and School Partnerships in Public Schools, K-8, FRSS 58, 1996, unpublished tabulations prepared by Westat.

Survey respondents were principals or their designees. Data include responses of "moderate extent" and "great extent" combined. Policy areas include: allocation of funds; curriculum or overall instructional program; the design of special programs; library books and materials; discipline policies and procedures; health-related topics or policies; monitoring or evaluating teachers; or developing parent involvement activities.

In the NHES:99, data for the three variables included in this report (attendance at a general school meeting, attendance at a school or class event, and acting as a volunteer at the school or serving on a school committee) were collected for a split-half of the sample. The other split-half of the sample included items that were worded slightly differently.

Sources: U.S. Department of Education, National Center for Education Statistics, National Household Education Survey: 1993 School Safety and Discipline Component, unpublished tabulations, National Center for Education Statistics.

U.S. Department of Education, National Center for Education Statistics, National Household Education Survey: 1999 Parent Interview, unpublished tabulations prepared by Westat.

Readers interested in further information from data sources for the national indicators can contact the sponsoring agencies, as follows:

Children's Health Index
Children's Health Index
National Center for Health Statistics (NCHS)
Stephanie Ventura
(301) 436-8500

National Immunization Survey
Immunizations
National Center for Health Statistics (NCHS)
Robert Wright
(301) 458-4595

National Household Education Survey (NHES)
Family-Child Reading and Storytelling, Preschool Participation, Participation in Adult Education, and Parents' Reports of Their Involvement in School Activities
NCES
Chris Chapman
(202) 502-7327

NCES items in the Current Population Survey (CPS)
High School Completion and Participation in Higher Education
NCES
Chris Chapman
(202) 502-7327

National Assessment of Educational Progress (NAEP)
Reading Achivement, Writing Achievement, Mathematics Achievement, Science Achievement, Civics Achievement, History Achievement, and Geography Achievement
NCES
Peggy Carr
(202) 502-7321

Schools and Staffing Survey (SASS)
Teacher Preparation, Teacher Professional Development, Student and Teacher Victimization and Disruptions in Class by Students
NCES
Daniel Kasprzyk
(202) 502-7486

International Education Survey
International Mathematics Achievement and International Science Achievement
NCES
Eugene Owen
(202) 502-7422

Integrated Postsecondary Education Data System (IPEDS)
Mathematics and Science Degrees
NCES
Susan Broyles
(202) 502-7318

National Adult Literacy Survey (NALS)
Adult Literacy
NCES
Andrew Kolstad
(202) 502-7374

Monitoring the Future
Overall Student Drug and Alcohol Use, Sale of Drugs at School, Student and teacher Victimization, and Disruptions in Class by Students
University of Michigan, Institute for Social Research
Lloyd Johnston
(734) 763-5043 Fast Response Survey System (FRSS)
Schools' Reports of Parent Attendance at Parent-Teacher Conferences and Schools' Reports of Parent Involvement in School Policy Decisions
National Center for Education Statistics (NCES)
Edith McArthur
(202) 502-7393

This page is best viewed with 800x600 resolution, using Internet Explorer 4.0 or above or Netscape Navigator 4.0 or above.

[Home] [Search] [Feedback] [Site Overview]
[About NEGP] [Goals] [What's New] [Data] [Publications] [News and Events] [Links]

NATIONAL EDUCATION GOALS PANEL
1255 22nd Street, N.W., Suite 502
Washington, DC 20037
(202) 724-0015 -- (202) 632-0957
NEGP@ed.gov

Technical Notes and Sources for the National Indicators