FCAT LIES & FLORIDA'S ECONOMIC DEMISE
Written by PT   
January 21, 2007
If you believe the FCAT and A+ plan glamour stories issued by Florida's gubernatorial press office, their paid cheerleaders, and the Florida Department of Education, be prepared to be disabused. The facts and data collected by the Fl. Lake Mary, FL -- October 16, 2006 -- If you believe the FCAT and A+ plan glamour stories issued by Florida’s gubernatorial press office, their paid cheerleaders, and the Florida Department of Education, be prepared to be disabused. The facts and data collected by the Fl. DOE itself belie the distortions and twisted interpretations created by persons who have pledged to promote the conservative agenda of big business and selected wealthy fiscal donors.

Educational improvement has become a major political issue. It will impact the life-long productivity of existing students and affect the political futures of many state officials. The importance of education reform has caused public-office holders to hire agencies with known political connections to produce reports that distort reality. Those reports are more like cheer-leader chants than scholarly efforts. The cheering agents include the Manhattan Institute, the Koret Task Force of the Hoover Institution and the newly formed Foundation for Florida’s Future.

The truth can be found through an understanding of the FCAT and how such standardized or norm-referenced tests are created and scored. The bottom line can be found in reports of the impact of inappropriate use of the FCAT on school curricula, student learning, school dropout rates, and the eventual educational decline of Florida’s young workforce.

The FCAT

What is the nature of the FCAT? It is a norm-referenced test that best serves to rank order Florida public school students by their ability to answer a limited set of questions. The FCAT Handbook – A Resource for Educators -- is a 125+ page document that provides an overview of the test development and some guidance in the use of test results. As a test instrument, the FCAT has merit and has received national recognition for its good technical qualities. But its use as the primary criterion for major judgments and decisions about students, schools, teachers and school administrators violates most of the ethical and technical guidelines provided in the Code of Professional Responsibilities in Educational Measurement created by National Council on Measurement in Education and recommendations of the Board on Testing and Assessment - Commission on Behavioral and Social Sciences and Education – National Research Council.

Contrary to the frequent claims of the Florida Department of Education, the FCAT is not a criterion-referenced test nor is it a diagnostic test. The FCAT cannot be used to help educators identify specific students strengths and weaknesses or to plan child-appropriate lessons. The test items cover only a part of what students are expected to learn. There are too few items on any specific skill to be useful for instructional planning. Teachers, students, and parents don’t see student responses and have no legal opportunities to review copies of the test.

Too often, the editors and writers of regional newspapers have either blindly or purposively printed the distorted and politically motivated messages about education in Florida, the FCAT, and student achievement. Few newspaper editors or their reporters have a sufficient knowledge of the technical aspects of measurement and statistical processes to identify or challenge the distortions.

A major FCAT lie – grade retention helps students catch-up or get ahead.

One of the most destructive, life altering and arguably criminal lies being told to Florida students, parents, and educators is that grade-level retention of students who fall behind helps the students catch-up and become better learners in subsequent years.

Three papers have been written about the effect of Florida’s law that now mandates grade retentions for grade-three students who score at level one on the FCAT reading assessment. The reports are based on data provided by the Florida Department of Education. The data are as good as is available but slightly flawed. However, the meanings assigned to the data are distortions that violate the statistical standards presented in most all measurement and statistical reference materials.

The authors of the first report (dated December, 2004 and available from the Manhattan Institute) examined scores of students retained in grade 3 at the end of the 2002-2003 school year. The second report (dated Spring 2006 and available from the Florida Department of Education) is a spruced-up version of the first report. The third report (dated September, 2006 and available from the Manhattan Institute) is based on 2005 scores and is a two year follow-up of part of the students studied in the first report.

In all three reports, contrary to the author’s claims, the results closely match the results of the large number of studies on the impact of grade retention that have been reported over the past 50 years. A second or third year in grade 3 resulted in trivial or very small improvements when compared to nearly similar students who were promoted.

Most of the grade-three students retained in 2003 were still behind most of their younger grade peers one year and two years later. The technical details: compared to similar students who were promoted in the previous year, retained students reading scores gained less them 5 percentile points or an effect size of less then .07. All statistical reference materials refer to those gains as trivial. The authors of the first two reports painted those gains as indicating substantial improvements that would permit those students to get ahead, -- a scandalous distortion of the truth.

Although students’ FCAT math scores were not the reason for retention, improvements in math were also examined. Gains in math scores for retained students were a bit larger but were still at the level statistical standards call very small to moderate. Perhaps, the somewhat larger math gains were was due to over-learning of skills already achieved.

The authors of the third report (September, 2006 and based on 2005 scores) used a slightly different methodology but found similar results and made similar distorted claims about the benefits of grade retention. They also failed to note that during the 2003-2004 school year 734 students were required to spend a third year in grade three or that 1,116 students were kept in grade three for a third time during 2004-2005.

Another flaw of the Manhattan institute reports was the authors’ assumption that the developmental scale scores from the FCAT could be equated across grade levels. They assumed that a score of 600 on the grade-three test indicated the same level of ability as a score of 600 on the grade-four test. Chapter 6 of the FCAT Handbook – A Resource for Educators provided a warning against that assumption.

The Drop-Out rate lie

National organizations that attempt to monitor school drop-out rates for every state have reported Florida’s dropout rate as the highest of all 50 states. The Fl. DOE uses their own method of calculation in order to make the facts look less harmful.

The Florida DOE data reveal the pattern in the state. Students who were two or more years older then their grade peers seldom stay in school beyond grade nine or grade ten. Not counting students with disabilities, during 2003-2004 the numbers of Florida students 2 or more years older than grade peers were: 17,332 for grade 8; 41,048 for grade 9; 23,693 for grade 10; 13,478 for grade 11; and 2,539 for grade 12. The number of lost students during high school was similar for the next year. In the 2004-2005 school year the numbers of students two or more years older than grade peers were: 16,848 for grade 8; 32,745 for grade 9; 21,728 for grade 10, 14,310 for grade 11, and 2,239 for grade 12.

The 1999 U.S. Department of Education published the policy document “Taking Responsibility for Ending Social Promotion”. The policy has been wrongly changed to “Ending Social Promotion” by politicians in Florida and other states to encourage marked increases in grade-level retention. In truth, the Federal policy statement strongly discouraged the use of retention as a school improvement strategy. Numerous alternative interventions were suggested, including smaller class sizes, and longer school years. The 2005 Statistical Brief on Florida Non-Promotions by grade level from 1999-2000 to 2003-2004 identified the marked increase in Florida’s non-promotions during 2002-2003.

Researchers have established what Florida parents know. Reduced class size permits teachers to use more effective interventions for improving student achievement. Grade retention imparts a life-long academic and social suffocation to most youth.

Key leaders of Florida’s conservative movement have done their best to distort the reality of the positive potential of reduced class size. Smaller classes coupled with instruction matched to student needs produce some of the most positive results. It reduces school failure and lowers the rate of school drop-outs.


Test score Improvement and Real Learning

Average FCAT scores for Florida elementary school students have increased a bit in the past few years. The questions is: Do the slightly higher FCAT scores indicate a true improvement in student learning? The answer to that question is not easy to document.

When the limited content of the FCAT became the focus of the major part of the school instruction, called teaching to the test, scores improved a bit. But at what was the cost to total learning? What parts of a full curriculum were replaced with endless test drill and practice?

The lost or reduced components of school curricula included: Physical Education – students became less healthy and local Heart Associations asked for funding to run exercise programs for kids; science – the new science part of the FCAT will now require a redirection of school time; music and art – student interest in school and ability to find joy in the arts disappeared; social science and history – loss of an understanding of how society became what it is today; problem solving and cooperative group work – major skills desired by most employers in the world of work and needed for gainful employment.

The lost learning has begun to become evident as FCAT victims finish high school and seek admission to colleges and universities. Only slightly over half of Florida youth now graduate from high school. For those who do graduate, SAT scores have declined and their ability to be admitted to the most prestigious post-high-school institutions has declined.

Grade 3 reading scores

The Foundation for Florida’s Future, a new entity described below has highlighted some data that seems a bit suspicious. They noted that in Spring, 2006 only 25% of Florida’s grade 3 students scored at levels 1 and 2 or below grade level on FCAT reading. In 2001, 43% of grade 3 students had reading scored classified at levels 1 and 2. In 2002, it was 42% with 23% of the total at level 1 and subject to retention. The rates in level 1 and 2 in 2003, 2004 and 2005 were 38%, 35% and 32% respectively. Of the 18% reduction from 2001 to 2006, the year-by-year reductions, in order, were 2%, 3%, 3%, 3%, and 7%. What could have caused the overall and sudden 7% reduction in the number of students reading below grade level from 2005 to 2006?

There are four components to the answer. First, schools did a better and better job of teaching reading. Second, Fl. DOE data show that larger numbers of students were retained in grades K-2, a strategy that helped improve school grades because it helped produce higher average scores for grade-3 students. Third, as any test is used from year to year with only a few items changed each year, there has always been a natural increase in scores caused by “test corruption”, the so called “Lake Wobegone Effect”, or “test score creep”. The gradual increase has been documented for all norm referenced tests. Fourth, there is some suspicion, based on 2006 oral reports of test scorers, that Fl. DOE staff modified the scoring guides in order to help increase test scores. All four of those elements could have had some part in the improved scores.

School Grades and Teacher Bonuses

The number of Florida schools given A and B grades has increased and the count of schools with D and F grades has declined. What does that mean? First, reading has improved to some extent, a good thing. But few people know how the secret grading formulas have been manipulated. The grading system has changed frequently, often yearly, so that no true pattern of improvement is possible. As noted above, student scores on norm referenced tests have no validity as a measure of school quality. The rankings of schools by average test scores is a nearly perfect match with rankings by the average household incomes of students’ families. Parents with moderate or higher household incomes have more time and resources to provide their children with many more life experiences that assist children with school learning.

In an attempt to reward teachers’ efforts, Florida established a bonus program for high performing schools. The bonus plan has increased the staffing problems for schools serving students from lower-income families. As noted during the Seminole County School Board meeting during the week of Oct. 9, 2006, schools serving poorer students have very high annual teacher turnover rates. Because those schools cannot retain teachers, there is a loss of continuity and school commitment. When school board members were asked to approve an incentive program of $500 annually for a teacher retention program, board members noted that $500 might not be effective. Teachers could move to a school serving high-income families and receive a state bonus of $1,000 to $1,500 annually.

Prior attempts to use teacher bonus programs to help improve student learning in Florida and other states have always been found to be counter-productive. Although it is good to improve teacher salaries in order to attract and keep good teachers, a general increase in teacher salaries and bonuses for teaching in schools serving low income families could be more effective. Many people seem to agree, but political considerations have kept that type of plan from being initiated and maintained.

Paid Cheerleaders

Florida residents have seen ads and reports on T.V. and in newspapers that proclaim the benefits of the A+ plan for education and the benefits of grade-3 retentions. What about those reports and who produced them?

The two members of education division of the Manhattan Institute wrote reports on grade-3 retention. They were housed in Florida. The Manhattan Institute is a conservative think tank funded by big business and the wealthy. Their openly stated objective is to advance the conservative agenda by publishing position statements and conducting research or policy analysis studies in an attempt to insert the conservative agenda into federal and state policies. The three reports on the impact of mandated grade-3 retentions described above represent the bias and distortions that result.

The authors of the three reports are now located within the College of Education at the University of Arkansas in a new department purchased by a $20 million gift from the Walton Foundation and related donators. It seems that the goal of the new department is to support the conservative agenda. They support efforts to establish new for-profit private schools that are exempted from accountability and an increased use of voucher programs that use public education dollars for those schools.

A second group of authors produced a nearly 300-page document praising the components of the Florida A+ plan for school improvement. The authors are members of the Koret Task Force on K-12 Education of the Hoover Institute located at Stanford, California. Governor Bush hired the well-known conservative-minded group to produce a policy assessment of his education improvement efforts. To no one’s surprise, the report praised his efforts. As noted by social scientists, when decision makers have made up their minds and stake their reputations on past decisions, no honest report is acceptable. One might wonder if the report could have been written without knowledge of the facts of school-life in Florida.

The third group is new, the Foundation for Florida’s Future. The foundation was established by Jeb Bush with donations from big corporations and wealthy friends. Its purposes are yet to be made public, but one of its first acts was to publicly support the Bush A+ plan for education. The foundation published statements about data that appear positive but were contaminated with the slanted interpretations described above.

Bush’s new foundation is clearly a political organization that is, in part, tied to the longer-term reputation of Mr. Bush as a successful education-governor. Their success will be based on public opinion of Mr. Bush’s education improvement efforts that have become less and less popular with Florida’s parents and voters.

Florida deserves honest studies of its efforts to improve education under the A+ plan and the federal NO Child Left Behind law. It is of little value to rely on paid cheerleaders as they dance about with a roll of dollars in one fist and an A+ pompom in the other fist.

The editors of the Orlando Sentinel, the Ledger from Polk county and similar conservative regional newspapers have a history of publishing stories containing the distorted facts about the impact of the FCAT and the A+ plan. They continue to attempt to persuade their readers to support the twisted perceptions of the meaning of FCAT scores and the use of average scores for assigning grades to schools. The Florida Department of Education has used copies of the stories as appendices to some of the Department’s reports.

Many Florida parents and other residents now believe that the A+ plan has done more damage to public education and the future of current school-age children then could be done by any foreign nation. Perhaps they are correct. The enemy with the most lethal weapons of educational mass destruction my already be among us clothed in “Here to Help You” dress.

Florida’s Economic Demise

The economic future of any state or nation is partly tied to the quality of its education programs. The ability to compete in the world economy requires a constant flow of well-educated people into the workforce. It is clear that Florida has needed to improve its education system if it is to compete with other states and nations. The question is: how to best improve learning for every student?
The July 27,2006 United States Government Accounting Office written Testimony before the House Committee on Education and the Workforce described efforts by the 50 states to measure and assess student academic growth (GAO-06-948T). Only two states have developed methods that attempt an assessment of year-to-year progress of individual children that takes into account each child’s estimated potential. Tennessee’s method was described in limited detail. For the most part, states have had serious problems in validly assessing student’s year-to-year academic progress.
The key to long-term education improvement is to meet the learning needs of every child. Slower learning students must be rewarded, not punished, when they do their best and achieve at an optimal level for their ability. More development is needed create measurement and instructional methods that help achieve that goal. Educators currently have many research-based strategies that are effective, but more strategies are needed.
If Florida is to succeed in worldwide economic competition, it must become more than a state depending on a service and vacation industry supported by low-wage workers. The state must use its education dollars wisely.

According to Fl. DOE reports, in every year since the 2002-2003 school year, approximately 100,000 K- 8 students were not promoted. At the current cost per student year of approximately $7,000, the lost years cost Florida approximately $700 million dollars annually. A similar number of students were not promoted each year in grades 9 to 12. Most of those students reach the age at which they drop out of school and become an economic drain on the state.

The combined K-8 and high school costs of over a billion dollars per year associated with grade retention could be better spent on learning interventions that have been demonstrated to be effective. Small class sizes and innovative technologies that help teachers meet the needs of most all children do exist.

The fiscal cost to Florida tax payers is much greater than the costs of wasted school years. Retention is known to reduce the long-term academic achievement of most retained students. Over-aged k-8 students tend to have high rates of disruptive and antisocial behaviors that heavily contribute to an increased risk of youth and adult criminal behavior.

Crime costs in two ways. There are economic and life style costs for crime victims and high costs of prisons. Florida’s prison systems are currently overcrowded and plan to become much larger as the impact of punitive school policies lead to a need for more prison space.

It is sad that state leaders prefer spending extra dollars on prisons rather than spending increased dollars on education programs that can help every student attain an optimal level of learning. Perhaps Florida’s moneyed power brokers believe that large numbers of school dropouts will provide them with a large source of low salaried workers.

An alternative explanation of Florida’s policies is a desire to use vouchers to switch students from public schools to for-profit private schools that have no accountability requirement. It may be wise to “Follow the money” to see who intends to fiscally benefit.

Robert R. Lange, Retired
Professor of Educational Research, Measurement, Data Analysis & Program Evaluation
Dated: October 16, 2006

Phone: 407 322-6234
Email: This email address is being protected from spam bots, you need Javascript enabled to view it This email address is being protected from spam bots, you need Javascript enabled to view it
Or This email address is being protected from spam bots, you need Javascript enabled to view it This email address is being protected from spam bots, you need Javascript enabled to view it

###
About the Press Release
If you believe the FCAT and A+ plan glamour stories issued by Florida's gubernatorial press office, their paid cheerleaders, and the Florida Department of Education, be prepared to be disabused. The facts and data collected by the Fl.


 
< Prev   Next >