I recently had occasion to look at some past SAT (College Board) reading comprehension exam questions, and I want to make some harsh observations about them, because I think they are unfair, misleading, and misguided in a number of ways.  First, let me say that I always did well on such exams, as did my children, and so this is not sour grapes. And there is, from time to time, evidence that the exams do not serve the purpose very well for which they are intended -- trying to decide which students will do well in college and get the most out of it. It does not help to have objective tests if the tests do not measure what they are interpreted to show.  (When I was accepted to medical school, I was told I had one of the highest Medical College Admissions Test scores they had seen, and my response was that there must be something wrong with the test then.  When I and others -- who were the ones that tests and grades indicated would be the most successful in it -- dropped out of the program, the medical school began to look for other ways to measure likely success in the program and field. It is my belief the same sort of problem plagues College Board SAT's, and I think colleges and universities need to do something about it, for they are not only squandering much potential talent in students, but they are not helping school systems develop the talent they test for, and they are being unfair to a great many capable students.) 

First and foremost, with regard to the reading comprehension section where passages are given and then questions asked, the tests are unfair because the answer options from which students must choose are often not very good answers.  That is a flaw because it often means that students who read carefully must choose from two or more options that are each wrong in some way, and so the issue becomes a guessing game as to what the author of the question probably had in mind, rather than knowing what the passage really means and being able to give an answer one can support from the passage.  The student rewarded with a perfect score, would be either very lucky, or is one who simply comprehends the passage in the same shallow, mistaken way the author of the question did. 

For example, in one passage, consisting of three or four paragraphs the point being argued is that when something is read out loud to you by someone else, instead of your reading it for yourself, the way that person reads it influences very strongly how you perceive the story, and that it can mislead you or diminish your enjoyment.  The author points out that his own daughter sometimes tells him not to read a bedtime story in the way he is doing it because that isn't right.  But the author of the question, in asking what the point was of the author's including his daughter's objection to his reading, gave two different answer options that were the best of the bunch, and yet neither was real good. The first option was that he was reading without enough expression, but we cannot know this because it may have been that he was reading too expressively to suit her or that he was reading with a different kind of expressiveness than whoever had been reading the story to her previously, in a way she liked. The other possibility was that he used this as an "example of active participation".  The author does say that his daughter is in a position better than the listenerof an audio tape who is "powerless against the taped voice."  Well, his daughter's telling him not to read it the way he is doing is not much more "interactive participation" than the listener to a taped book who can turn off the tape.  Children cannot necessarily, or even likely, get you to read the book a certain way if you cannot do it or don't know what they want.  Moreover, the second passage in the same section, given as a contrasting point of view to the first passage, tells how a woman loves to be read to by her husband because they discuss the passages as he reads them.  That is a clear "example of active participation" in the reading aloud process. The daughter's saying "don't read it that way" hardly rises to that level.  To call it an example of "active participation" seems to be quite a stretch.  So I am not really sure which of these answers is the one the author of the question has in mind, and that the College Board would grade as correct.  Since they put the one about reading with insufficient expression first, my guess is that is a trick and that they prefer the other one, but that is to play amateur psychologist, not to display reading comprehension. 

The above case is really problematic because it is impossible to tell the answer that will be counted correct.  But even in cases where the bad answer is obviously the best, and is the one a good student will likely finally select, its being a bad answer still unfairly slows down the careful and reflective student because s/he is made to waste time going through all the options looking for a better one, and then having to wonder and look again to see whether s/he missed, misread, or misunderstood something either in the story, the question, or the other answer options. It is extremely disconcerting and time-consuming, especially under test conditions, for someone who is trying to be conscientious and do a good job to find only answer options that are not right or reasonable. 

Also, it is my contention that the most efficient way to answer these questions correctly is, not to read each answer option to see whether it makes sense, but to read the question (after having studied the passage) and formulate one's own answer before reading the options given, and then finally to look for that option.  If I am right in this, then when the correct answer does not appear as an option, it creates all kind of unfair, unfortunate, and unnecessary havoc for the more reflective, and perhaps even "better", student. 

There are four other problems with these "passage-and-questions" sections: 
(1) Because the test is timed, students feel under pressure to be quick; but sometimes quickness is often the enemy of efficiency. It is often more efficient to do something slowly and well than to do it quickly and poorly, especially when one knows one is not doing it correctly. When one is under pressure and is impatient because of a feeling of lack of time (even if that feeling is false), one cannot always do one's best work or think or read clearly.  So, regardless of how well they might comprehend a passage normally, if a feeling of panic or impatience has set in, and if the student has not been taught to disregard it and to take time and be careful, that student's scores will be more reflective of impatience and anxiety than of poor reading comprehension. 

It is my guess that long passages are used on College Board tests to create impatience, pressure, and panic, in order to help weed out students and get a wider span of scores, particularly lower scores.  If so, I think that is unconscionable. 

(2) It is a common mistake to try to look at the questions first, as though one were doing a research project or looking up facts specific facts one was asked to find, and either then read the passages with those questions in mind, or, worse, to read the questions and hunt for the answers.  The questions are constructed in such a way that the answers are not on the surface, but require knowing a context or "reading between the lines" -- understanding and interpreting the passage.  I strongly believe that one will be much better off if one reads the passage first very carefully and makes sure one understands the passage and the way it is contructed -- knowing what the point is and what the evidence is for that point, and knowing how each piece of evidence is used.  (I discuss some of this in "Writing Non-Fiction Papers...") If the story is a narrative, one should know what each sentence contributes to the narrative.  These are not things someone needs to memorize, but just to understand as one is reading it. 

In the test question mentioned above, one needed to understand the point of the passage in order to understand why the author of it quoted his daughter.  There is no sentence in the passage to the effect of "I am quoting my daughter because...." The questions themselves even tell you which lines in the passages they they refer to, and that itself ought to be a clue that just knowing where the part is they are asking about does not contain the answer.  The line numbers are given, I presume, so that you can quickly find the context in which they are located, but you must already, if I am correct, understand that context within the passage as a whole; and I think you can only do that by reading carefully from beginning to end, not by hunting piecemeal for an answer. 

Furthermore, the questions are not easy to keep in mind, particularly in a pressure situation, so reading the questions first, instead of reading the passage first really slows you down, and essentially makes you read the questions twice even if you also do not waste time and concentration trying to keep them in mind while you read the story.  Again, students who have not figured out or been taught to ignore the questions before reading the passages will be at a disadvantage to those who have, and that has nothing to do with reading comprehension ability, which the test is supposed to measure. 

(3) Schools, in teaching "reading," predominantly teach fiction, and students concentrate on plot, possible character development, setting, etc. These are often something of surface issues, and fiction lends itself to skimming or reading quickly with only minor reflection about each passage.  However, at least some of the SAT passages, such as the one above, are more in the form of essays or editorial pieces, more or less well or poorly written or argued, and they require reflection as one reads them.  Students who have not had much practice analyzing essays for their logic and their merit, which requires understanding their points and their construction and purpose, are at a serious disadvantage in reading these sorts of passages.  This is an important skill to have in college (and in life), but most schools, teachers, and curricula do not teach it.  Therefore the College Board tests often weed out students who have not had opportunities to learn or practice analyzing and interpreting essays, and making it mistakenly seem they have some lack of ability, rather than lack of exposure and training.  That essentially consigns a large population of students to the educational backwaters rather than helping them get the kind of education they need and from which they actually could learn and benefit. 

While tests can "drive" the curriculum (whether for better or worse), they will not do so if teachers and educators cannot tell which skills the tests actually require.  Hence, often schools teach "to the tests" by giving surface tricks and practice that merely rehearse students for certain limited kinds of content and questions -- those expected to be on the test.  But that only treats symptoms and only gives the illusion of successful teaching and learning.  And it does not improve scores nearly as much as would teaching a broader range of material better and more appropriately. So although the kinds of passages and questions the College Boards use address really important skills, their nature is not sufficiently recognized by most educators and school systems to "drive the curriculum" in the right direction.  The students lose out on the opportunity to go to better schools through no fault of their own, and then are given no, or diminished, opportunity to develop potentials they may have, but which have not been developed prior to these tests.  This is not so much a criticism of the test as it is a criticism of educators and of the way the test scores are interpreted and used. 

(4) The ideas in many of the passages are new or foreign to many students, and when those passages are not as well-written as they might have been, it is extremely difficult for students to comprehend them well enough to be able to answer the questions correctly.  But this is not an issue of "reading" comprehension.  It is an issue of reading assimilation, which is a different matter altogether and usually depends on prior experience or reflection.  Students who have not been exposed to audio books, or who have not seen movies based on books they have read and been disappointed in the portrayals, or who have not heard people read aloud poorly or differently passages they have first read for themselves, would not as likely understand the passage I described above as an example, as would students who have had those experiences.  But that does not distinguish those who read better from those who do not.  It just distinguishes in large part those who have, through luck or circumstance, had certain kinds of experience with the content of particular passages from those who have not. 

One way to see this last point is to read the passages out loud to students (there being some irony in this demonstration, of course, given the passage with which I began as an example).  If they still do not comprehend what is said well enough to answer the questions, then the issue is not one about their reading comprehension but about their comprehension in general, or, more likely, about their prior experience.  Using passages that contain ideas not likely familiar to students is only a step or two short of having them read technically or conceptually difficult material without a proper introduction to it.  If you give the finest readers material with totally unfamiliar content, particularly material that is not well-written or presented, they are not likely to understand it. Again, that has nothing to do with reading skills or (reading) comprehension ability. 

While the above example is not particularly difficult, students often have trouble with ones that are only a bit more difficult, because they are not given instruction and practice in using logical constructions.  They cannot recognize them in print or speech, and they cannot use them in writing papers or in constructing speeches. 

While the College Board SATs test what I am calling the logic of construction in the reading comprehension and writing sections, schools just generally do not teach these things very well.  Moreover, when the College Board gives multiple choice questions with incorrect answers, and no way to appeal one's score, that is not a good measure of student knowledge or ability. 

I believe the College Board SAT's do test some skills that are important for students to have, but they are not the only skills students should have.  So the test has too much importance or weight in the admissions process.  It weeds out "late bloomers" and it weeds out students who have much potential but have had inadequate prior education or experience. 

And the test comes too late for many students because instead of showing them what sorts of skills they need to develop, it only shows them what they have missed developing. And it does not really do that clearly.  There is no set of guidelines from colleges or from the College Board that carefully explain the kinds of skills they want students to have or the kinds of skills they want schools to foster and nurture.  The better schools know what is needed, but all schools should know.  Teachers and others should not have to look at past tests and try to guess what the skills are or figure out ways to "beat" the test.  Colleges and the College Board should make clear to parents, students, and school systems alike, what it is they are really trying to test -- what the skills are that they want students to have.  And it would be helpful, of course, if they can provide some instruction in the best way schools can help students achieve and master those skills. Or at least if they would provide instruction for students to learn and develop on their own (or with the help of mentors) the skills they should have.  If schools are not going to help students develop important college and lifetime skills, students and families at least ought to have the information available to do it themselves.