Why Standardized Achievement test schools are inappropriate and misleading measures of the success of schools, school districts, and educational programs, and their adverse effects on education

Ronald G Corwin, Professor Emeritus

Department of Sociology, Ohio State University

The idea that the success of a school district can be measured with standardized tests comes from an obsolete industrial model that treats schools as factories processing students as raw materials—with test scores reflecting the quality of the product. Of course, that is sheer nonsense, because in the first place, a school does not control most of the so-called production process. The way students perform on tests depends on many extraneous forces, such as parental guidance, peer group influences, job and travel experiences, and the like.

            There is a long history of rancorous debate over standardized tests, commingled with equally fierce disputes about the goals of schooling. Historically, schools were established and then made compulsory in this country primarily to teach basic skills and to instill moral education, especially for low-income, immigrant children. Even today, schools remain saddled with dozens of stated and unstated objectives, from sex education and driver training to cultural appreciation and current events. The advent of standardized testing rode waves of so-called efficiency management and then the search for ability mobilized by frenzied international competition in the 1950. National projects funded by the federal government in the 1970s spawned Project Talent, the National Longitudinal Survey, and the Equal Educational Opportunity Survey, and later there was the High School and Beyond Survey. These studies were premised on the assumption that although cognitive skills represent only one type of educational outcome, there is little disagreement about its relevance. Maybe. But it is also true that the validity of SATs for measuring cognitive ability has never been fully accepted. Teachers, school administrators, and popular writers have vehemently criticized the “tyranny of testing.” Yet, tests are being constructed today in the same manner they were a century ago. There was already a precedent in 1889 when Joseph Meyer Rice surveyed school officials. Tests were used in the early part of the 20th century to demonstrate that large numbers of children were overage for their class placement, and by 1925 they were being used to create tracks and other forms of homogeneous grouping. During World War II tests were used extensively to screen draftees for work assignments.

            A major impetus for testing came in 1958 with the passage of the National Defense Education Act, which provided federal funds to establish and maintain a program for testing and aptitudes and abilities of students in public secondary schools, for the purpose of identifying outstanding students to be enticed in math and science courses. However, it was the Elementary Secondary Education Act of 1965 that was largely responsible for institutionalizing testing in most public schools. That Act mandated evaluation of all Title I programs for children from low-income families. Then, again in 1974, the U.S. Office of Education promoted SATs as a means for uniformly evaluating all Title I programs. Then, of course, there is the recent infamous federal No Child Left Behind Act that mandates standardized testing in most public schools in this country.

            Ironically, even as the use of SATs was becoming universal, many researchers were becoming highly critical of the way they were being used. Two legendary testing specialists, Ralph Tyler an dOatis Burrows, were among the most vocal critics. All of the critics noted that SATs were designed and constructed for selection purposes and hence are poor indicators of instructional program accomplishment. Testing in schools began with purpose of ranking students to select those most likely to benefit from special instruction. Accordingly, standardized tests are constructed to maximize differences among individuals in a school or classroom. Since test items are usually selected to maximize variance among individual test takers, they are inappropriate measures of higher-level organizational units like schools, school districts, and programs. They do not to measure differences between schools or to assess national programs. They simply were not constructed for this purpose. In fact, because of a statistical artifact, when variance increases within classrooms, it declines among schools, districts, and programs. The scores of individuals within classrooms are spread out so much that when they are averaged for each school or program, they tend to be similar for the aggregated units. There is little difference left to explain, as Alexander and Griffin have demonstrated. Aggregating individual test scores to the level of schools, programs, or states inadvertently masks real differences among the units being compared. Researchers have been fussing over inconsequential differences derived from unjustifiable uses of standardized test scores.

            Aside from that, because the objective of SATs is to maximize variance among examinees, items covering important topics or skills get eliminated when all the examinees get the items right or wrong. SATs do not necessarily measure what schools intend to teach. Of course, the solution to that has been to “teach to test,” meaning that to show up well on the test, schools often allow the tests to drive the curriculum. This of course means that teaching complex reasoning skills or subjects like public speaking, music and art get squeezed out of the curriculum in favor of a few subjects that lend themselves to multiple choice answers. And, finally, because they are being used to compare and evaluate schools and school districts, SATs have become vulnerable to political influences. Some states have purposely created easier tests for the purpose of making school districts look better, schools routinely excuse students with learning problems from the tests, and teachers have been caught giving out test answers. For all of these reasons, and more, it is not surprising that the relationship between test performance and out-of- school performance is very weak.