A perfect example of a “place rating” fiasco was published this week in, of all places, theatlanticcities.com, it not only demonstrated an obscene level of ignorance regarding standardized tests and the significance of family income but also illustrated one of the primary flaws in place rating. The headline read “Report: Schools in American Cities Are Still a Mess”. What the report showed was that students in nearly all cities in the United States getter lower scores on standardized tests than the average American student. Putting two and two together and coming up with five what this urban oriented website deduces is that this then indicates that city SCHOOLS are below average.
Not that I expect the writing of a small time blogger like myself to permeate the zeitgeist and alter the worldview of the public at large, but I would think that a pro city website like The Atlantic Cities would have at least a vague idea of what it means to thoughtfully analyze education related data. The first variable one must account for when comparing outcomes on standardized exams is parental income. The positive correlation is so strong that to ignore it would be like ignoring gender when looking at pregnancy related data. The question one asks when looking at test score data is not “What school(s) performed best?”, but rather “After taking into account parental income, what school(s) over-performed relative to the outcomes income would predict?” Anything else is a waste of time.
I will refer new readers to these blogposts (here, here, here, here, and here…oh, and here) to see in detail that that my assertion is correct and backed up both by the evidence and by experts in the field. City schools are not “a mess”, the “green flight” (to use a term stolen from a conversation I had this week) of people of means out of the cities and in to the suburbs has given city schools the more difficult task of educating a disproportionate number of the children of the poor.
Enough on the anti-urban element of the story and on to the place rating disaster. What do you notice? The cities that do occasionally score “above average” on the standardized tests are all from the south. “Oh, my! I do declare, why…I am shocked!” This is, of course, due to the excellent schools the South has been known for for decades and decades. Right? Wrong. It is due to the fact that the “cities” in question are The size of what in the northeast would be “metro areas”, in the case of Charlotte and Austin geographically over 5 times the land area of Boston (for example) and therefore these numbers include the results of the de facto suburbs which change the demographics, particularly in the most significant data point of all, parental income.
Just look at the population densities of these communities:
Charlotte: 2,500 per square mile.
Austin: 2,800 per square mile
Boston: 12,900 per square mile
These numbers don’t even compare to the densities in some parts of suburban Boston
Belmont: 5,300 per square mile
or even smaller cities like my hometown
Springfield: 4,800 per square mile
Once again, as with the previously discussed crime data, cities in the South and West appear to outperform their northeastern counterparts but a better analysis of the data not only discredits this view, but sometimes shows quite clearly that the opposite is true.