The straw man in the standardized testing debate

Frank Bruni wrote  a column in yesterdays  The New York Times, in which he expounded on the virtues of college admission committees decisions to look past marginal test scores in a handful of underprivileged applicants in order to diversify their classes. Depending on your perspective, what Bruni describes  can either be construed as a noble undertaking or the  symptom of a corrupt system that unfairly disadvantages hardworking, middle-class  applicants, but Im actually not concerned with that particular debate here. Rather, my issue with Brunis column is that it  perpetuates a common straw man argument in the debate over college admissions namely, that test scores have traditionally been  the be-all end-all of the admissions game, and that only now are a handful of intrepid admissions officers are willing to look past less-than-stellar scores and consider other aspects of a students application.   Bruni is the author of  Where You Go is not Who Youll Be,  a book that very validly  emphasizes  the questionable  relationship between name-brand colleges and overall success in life, but in terms of actual admissions,  his  authority appears to stem primarily from the  fact that he turned down Yale to attend the University of North Carolina at Chapel Hill and still managed to land a job as a columnist at the  Times. Although hes generally familiar with the field, he is not  actually an expert in admissions the way, say, Paul Krugman is an expert in economics. As a result, its hardly a surprise that he misrepresents some of  the issues at play. In short, what is news to Bruni is a long-established practice known as  holistic admissions a practice that  was, incidentally, introduced in the  1920s, when the Ivies first decided to consider character  in order to limit the number of Jews. Since then, the purpose of evaluating applicants according to factors beyond grades and test scores has  changed again and again, but colleges continue to select  students  according to their particular set of institutional needs be it diversity, donors, athletics, or physics  research   and test scores play a role in that process only insofar as they garner universities freshman classes  with the desired characteristics. For example,  a major reason  for inflating scores on the new SAT was presumably to allow colleges that arent quite ready to go test-optional to admit more  applicants from under-represented demographics without compromising their USNWR rankings. The College Board has danced around this fact with v arious euphemisms about opportunity, but it is difficult not to conclude that  this type of demographic manipulation was not a driving force.   For at least four  decades, though, admissions committees willingness to  give disadvantaged applicants a boost has had a very real effect  on thousands of students life-altering effects, in many cases. When people attack colleges for relying too heavily on test scores, theyre obviously  thinking of all the other thousands of applicants who didnt get that boost. Whats interesting (but not at all surprising), though, is that the other  side of the argument is almost never considered   that is, the students who are given every advantage but who never achieve scores anywhere remotely what they would need to be competitive applicants at top colleges  are rarely mentioned. Yes, the majority of students achieving high scores are well-off, but it does not follow that every well-off student achieves high scores. As Ive pointed out before, the  lowest-scoring students I worked with tended to be  from the wealthiest families.   As a result, I  left Mr. Bruni the following comment,  which can be viewed  here (and which, might I add, was selected as a  Times top pick!): Frank, With all due respect, what you describe in this column is holistic admissions, which has long been the policy at the vast majority of selective private colleges in the United States. Theres a reason that schools do not publish and, to the best of my knowledge, have not ever published official cut-off scores for applicants. Admissions committees are well aware that applicants come from a wide range of socio-economic backgrounds, and that some applicants face far more obstacles than others. That said, what about the opposite end of the spectrum, e.g. a student who has spent 10+ years in a $40K/year Manhattan private school and whose parents have doled out an additional $10K or more for tutoring, but who can barely break 600 on any section of the (old) SAT? These numbers are not exaggerations, by the way: I tutored students in that demographic for a considerable period, and some of them could not in fact achieve scores that would make them even remotely competitive at most top-25 or s o schools. (In case anyone is wondering, money and connections only get you so far). Some of those students were reasonably bright and hardworking, but their scores were also very accurate reflection [sic] of their academic limitations. The fact that there is a correlation between scores and family income does not in itself mean that scores cannot provide an important piece of information when considered in their full context. The real problem is that test scores mean such different things for different applicants. Sometimes they reveal an awful lot, and sometimes very little.   In my experience, scores for the most privileged applicants do tend to be a roughly accurate reflection of what those students know.   A slew of 750+ scores  from a student at a top  private school is  by no means indicative of brilliance, but 500/600-range scores from a student at the same school are usually a sign that there are some real gaps. Thats a  significant  piece of information for an admissions committee to have when evaluating those students against their classmates,  as well  30,000 other applicants.   On the other hand, how is a  committee supposed to judge 500-range test scores in an applicant from an academically marginal  school  and a single-parent household with an income of less than $20,000/year?  It would be obtuse to believe that that applicants scores did not also reveal some gaps (even though  500-range scores are actually quite an achievement in that context); but the question is  what sort of potential other aspects of the application reveal, whether  and to what extent they outweigh the test scores, and whether the college has the resources in place to help that student catch up academically to his or her peers.   The fragmented nature of the American school system and the relationship  between real-estate prices and school quality ensures that these are not apples-to-apples comparisons. Theyre not even apples-to-bananas comparisons. Theyre more like apples-to-skyscrapers comparisons. Scores are not everything. Admissions officers  know this. They struggle with these kinds of calculations every for day, for months, and in the end they just cant take everyone. Exactly  what  role test scores should play in the process  is  up for debate. But to suggest that everyone has  just been  playing a straightforward numbers game all long†¦ well thats just not true.