How Do You Test a Student?

By underlining the right answer in a multiple-choice test the student can show “scholastic aptitudeand a capacity for college work. But how shall the college judge his common sense and what the student can bring to bear from his own experience, both academic and practical? FRANK D. ASHBURN is headmaster of Brooks School, North Andover, Massachusetts, has served in various capacities on the College Entrance Examination Board, and is at present chairman of its committee studying the current achievement testing program of the Board.

1

ONCE upon a time an examination was something which supposedly measured how much you knew (that is, remembered) about something you had studied in the reasonably recent past. It represented a contest between you and the examiner. If you demonstrated that you knew enough to “pass,” all was well; otherwise you failed, even though you had abilities that might make you outstandingly successful in later life. Usually you knew what you would be examined on, and by real effort, either on your part or that of somebody better qualified, you could get a good mark.

Such a concept of an examination is becoming less general. There is a possibility that recent changes in the idea of what an examination should be and do will profoundly affect the course of intellectual and cultural development in the United States. Many teachers feel that an effect is already discernible.

To estimate that statement let us examine the evidence at just one critical point, that of transition from school to college. It is obvious that only a portion of the American school population is involved in such a transfer, and that it is the elite portion academically. But for that very reason it serves as a good illustration of a trend. And in weighing the effects of new types of testing in that portion, let us confine our attention to an outstanding agency involved, which is the College Entrance Examination Board.

The College Board was born fifty years ago, with Nicholas Murray Butler and Charles W. Eliot as godfathers. It was intended to do away with chaos, in a time when every college had its own entrance requirements, by establishing a single set of standards for examinations and reliable machinery for administering and correcting them.

It was never intended that the Board itself would admit or reject any applicant. It has always left the function of selection entirely to the colleges. The Board is merely the agent of interested schools and colleges, both of which are amply represented in its councils, with powers of veto and initiative. This coöperation is itself a significant educational development.

For many years quiet growth took place. Each June and September a series of examinations was given. These papers were usually either two or three hours in length. The aim of the examinee was to get a 60 (pass). If he got an 80 it was an honor and so reported. In due course two “Plans” emerged, with periodic variations. One, the Old Plan (later Plan A) was a pay-as-you-enter scheme. Papers were taken over two or three years as courses were completed. When fifteen “points” were accumulated, sometimes quite late in life, it was generally held a college’s duty, if only as a sporting proposition, to admit the persistent candidate. This arrangement gradually gave way to Plan B, which required a single set of examinations on the work of senior year in high school. The candidate staked his all on these, unless a good school record could be counted on to salvage his collapse in examination week.

The examinations were based on fairly definite requirements. An experienced teacher or tutor could, and usually did, anticipate a considerable part of an examination in any given year. It was a system which favored the private or independent school and capable tutor, while often working against public school candidates, who, though able and in good schools, were handicapped by teachers’ lack of familiarity with the College Board type of examination or by their lack of time and means to prepare for the high standards met there.

Meanwhile a considerable development of objective or multiple-choice tests had taken place. The difference between the subjective examination and the objective test is illustrated by the following oversimplified examples: —

Subjective: Discuss the consequences of the Dred Scott decision.

Objective: The chief justice in the Dred Scott Case was (underline one): —

1) John C. Calhoun 2) Roger B. Taney

3) William Lloyd Garrison 4) Salmon P. Chase

5) Stephen A. Douglas

It will help to keep this comparison in mind.

The development of objective testing came with a rush because of the war. The armed services had to have huge quantities of reliable easily administrable and correctable tests in a hurry. The College Board was outstandingly qualified to do much to meet the need, both by organization and experience. Almost overnight the whole apparatus of objective testing increased enormously. It had a chance to demonstrate its possibilities and to undergo refinements on a prodigious scale, while subjective testing, so far as the College Board went, practically disappeared. Which meant it disappeared so far as college admission went, too. It has reappeared since only in the shyest sort of way.

2

A COLLEGE candidate today is tested in an entirely different manner from his parents. The multiplechoice test, developed by experts, proved a surprisingly successful instrument in measuring ability and, in certain respects, achievement also. For instance, the present achievement tests, which take only an hour, predict more accurately than the former twoor three-hour examinations ability to do college work in most subjects. Why and how? Partly because the tests are constructed with such meticulous care, every item being thoroughly pretested. Since the answers are of fact (to reach them may require real thought, but the answers themselves are factual) there is no question of one examiner’s opinion differing from another. An answer is right or wrong, and the element of human frailty which has always plagued givers of marks is removed. In fact it is removed to such an extent that much, if not most, of the correcting is done by machine, which is characteristically indifferent to opinion.

Of its nature, an objective test cannot measure certain things, such as style, organization of material, or power of coherent argument, but these are in themselves matters of opinion, and opinion is a variable which your true tester loathes like poison. The minute you introduce even the possibility of it you lose reliability, and reliability is what born testers love above all else.

Objective tests have additional virtues. They are economical in time and money. Instead of a week of examinations, a modern college candidate takes all his tests in one day, and the saving in food and lodging is considerable. One day, and the thing is done.

No matter how strongly teachers feel about the disappearance of the essay-type examination (and some of them feel violently), the statistics seem to show that the Scholastic Aptitude Test of the College Board is the best single predictive instrument for foretelling academic success in college. The thing it measures is just what its name implies — scholastic aptitude; that one thing. When a physician places a clinical thermometer in your mouth he learns whether you have a temperature or not. He does not learn why you have a temperature, nor whether your condition is likely to be chronic or fatal, nor what he should do about it. But he does know one important clinical fact about you. In the same way, these objective tests produce certain bits of evidence which are highly significant so far as educational prognosis goes. The medical analogy breaks down in one respect. If you have a temperature the thermometer will show it. You cannot avoid it by not trying. Whereas if you don’t try in taking a test, you can do just as badly as though you were really stupid.

The emphasis is on potential ability in a restricted sense. It is not on finding out whether a candidate has studied certain material or remembers particular facts. It is on finding out what his chances are of doing college work, which is quite different from finding out whether he can pass a course. And since the emphasis in college admission is now on ability, it follows that there is less emphasis on training and on qualities such as stick-toitiveness and common sense. An important further result, since a school can train a boy to handle given material, but cannot make a dull boy bright, is to take away an advantage formerly enjoyed by the independent school on College Boards and bestow it on the much larger group of bright boys in the vast public school population.

Let us see how this works out in practice. A candidate registers for College Board tests at one of the five sessions held during the year. Of these the spring program is much the most commonly required, and the number of candidates now involved in it runs into tens of thousands, a huge increase over a few years ago. In 1950, on the morning of March 11, each candidate presented himself at a testing center. He did his work as a number, not as a name. It is easier to penetrate the inner mysteries of Oak Ridge than to break the security measures of the College Board. Between 9 A.M. and noon he took the Scholastic Aptitude Test, which yielded two scores, one in Verbal, the other in Mathematical, Aptitude. Preparation for either portion of the test is held undesirable and impracticable. It is recognized that these two abilities may differ sharply in an individual, and the distinction between them is carefully preserved. Some colleges require only the aptitude test; all welcome it.

After an interval for lunch and recuperation, most of the candidates returned for three one-hour achievement tests, based on the work of the current year, and chosen from English, mathematics, a language (French, German, Latin, and Spanish for the most part), science (biology, chemistry, physics), social studies (a single paper), and a straight special aptitude test called Spatial Relations, Most colleges like as wide a sampling as possible. The questions seem deceptively simple, but require both knowledge and intelligence.

No mark, in the ordinary sense, is given. There are only scores. There is no such thing as a pass. In each test the “raw” score (that is, the exact number of correct answers) is distributed along a scale of 800. Each score is placed in its relative position on this scale, with a new score, in terms of the scale, now given it and replacing the mark of old days. A score of 500 indicates a number of correct answers which is exactly in the middle of all the raw scores on a given test. Any score of 700 or up means the candidate is in roughly the top 3 per cent of all scores; a score of under 400 means that he is roughly in the bottom 15 per cent. At this stage the candidate resumes his identity. His scores are reported to the colleges and his performance is registered in terms which relate it to the scores of thousands of other candidates. In other words, the present College Board system is strictly cutthroat competition and, to repeat once more, emphasizes scholastic aptitude, which it is able to identify with embarrassing accuracy.

Economy, reliability, simplicity, and one other great thing have been achieved. That other thing is the freeing of schools from any restrictions as to course content, diploma requirements, or pedagogical method. A school may arrange its curriculum as it likes; it may follow its own inclinations to an extent never possible before.

Despite this new freedom for the schools, the greatly increased use of Board tests implies a trend to unity. That trend is further indicated by the formation, two years ago, at the instigation of a committee headed by President Conant, of a superorganization in testing called the Educational Testing Service, to which four large testing agencies (the two biggest of which were the College Board and the Co-operative Test Service, the latter the testing arm of the American Council on Education) voluntarily surrendered capital and numerous prerogatives. The ETS has already moved towards singleness of purpose and administration in the great testing movement, and it could develop into an educational supertrust, in terms of influence, with all its concomitant advantages and dangers.

The College Board, while yielding both revenue and much administrative detail to the ETS, has held on to its main function, which is that of determining the fitness of candidates to move from one particular educational level to another. It now offers tests in many fields above the point of transition from school to college, and these tests are almost wholly multiple-choice. For years the College Board has set the pace; the results it has achieved have been on the educational gold standard, and its influence has been out of proportion to its membership. Its trend may therefore be regarded as pointing the course of American educational practice in the predictable future.

3

WITH so much to be said on the positive side, is there a negative case? It must at least be stated that a degree of uneasiness about the status quo exists in the minds of the members of the Board, some of whose most severe critics are at the same time among its warmest admirers. Many teachers resent the Board’s not checking on whether a student has done any work in a course, and worry that what testing there is takes place long before a course can be finished. This objection rests to some extent on a misapprehension, inasmuch as the Board has no intention of testing specific content and considers such testing entirely up to the schools themselves.

Nevertheless it is true that at the present time there is no College Board test which could be called terminal—that is, which measures the performance of either the student or the teacher in terms of a year’s (or several years’) completed work. The possibility must be faced that the cumulative effect, through several generations, of not having such a measurement, even though ability alone may be thoroughly estimated, may have serious consequences for American teaching and learning.

There is general agreement that the objective tests represent a real educational gain and that they make available a form of evidence and a simplicity of administration hitherto lacking. At the same time, there is an increasing, though as yet minority, feeling that when the essay or subjective type of test was abandoned, there ceased to be any testing by the Board of certain crucially important educational values, and that unless these values are re-emphasized, the long-range effect on American education will be most unfortunate. Many teachers have felt the lack of an educational ingredient which might be defined as the power to deal cogently and lucidly with ideas synthesized from reading, instruction, and experience, both academic and practical. Such teachers believe that the development of this ingredient is the hallmark of a good school and its possession the best indication of good college material and, indeed, of good education at any level. This is not being tested by the College Board at the present time.

Another fear is that the emphasis on ability has resulted in a decline of emphasis on such immeasurables as workage and loafage. Some feel that the brilliant idler is not as good a bet as the sound, reasonably able person who is willing to learn. They hold that character, if it is still valued, can be just as important as naked ability, though it is difficult to test objectively. And they hold that ability thoroughly trained is better than ability untrained.

Then, too, the very virtue of complete freedom for the schools, desirable as it is, raises the question as to whether complete freedom (since it must mean, if complete, freedom from standards) will not lead to something savoring of anarchy, as happened with the originally virtuous elective system. No one wants to return to chaos and old night; everybody wants ability; but many feel that ability steeped in proved values is better than ability alone.

Today’s situation is that the colleges, so far from dictating what the schools should teach, give them a minimum of guidance and restraint. I would hazard the guess that within ten years, if this tendency is unchanged, the colleges will regret their generosity. If schools don’t have to teach grammar, they won’t teach grammar; if they don’t have to teach the geography, simple biographies, and stories which are the vocabulary and grammar of the social sciences, they won’t teach them — which means the colleges will have to. The schools will teach a lot of other things, probably worth while, but they won’t teach such basic simplicities as require hard mental work if they don’t have to. It would be unwise to confuse the needs of the many with reasonable demands for the able minority. We speak of the college level as higher education. Unless it has intellectual integrity it is not very high.

There has never yet been a decisive examination which did not materially affect the work done in preparation for it, as there has never yet been a system of testing for which competent teachers did not eventually find a means of preparation. In theory one of the beauties of the objective test is that one cannot prepare for it. But teachers whose professional reputation and whose bread and butter depend on preparing for it are giving, and of course will give, the matter their best attention. If they can find an answer, it will neglect things not needed for tests. If they cannot find an answer, the tests do not require that very kind of training the lack of which is probably the most serious weakness in American secondary education today.

4

IT is easy to express forebodings as to testing in America. Is it possible to foresee measures which might prevent suggested dangers while guarding proved merits? To underestimate the difficulties of restoring any form of subjective testing would be folly; they are real and weigh heavily with experts who have given their lives to the cause of American education.

The wave of increased enrollment now rushing upon American education suggests the following conclusions: —

1. There must he an enormous increase of schools and colleges. It is already clear that a new type of educational institution, the junior college (a most unhappy name; what is needed is not an ersatz college, but something quite new and terminal and sufficient unto itself) is emerging. It is properly suggested that requirements for and after entrance to such academic levels should he different from those in the four-year colleges. This does not or should not mean that four-year requirements should be lowered. There is no magic in any given number of years or courses, but a college education implies setting and preserving standards.

2. It is inevitable that any such educational inflation as can alone provide for the tremendous expansion which is coming (as heralded by the President’s Commission Report and numerous other prophecies) will result in a lowering of teaching standards and therefore of educational quality in the United States. This condition might be temporary, although the evidence is that devaluation rarely results in revaluation to original worth. The condition will certainly be acute while it lasts, simply because no nation can find quickly the number of adequate teachers we shall need.

Our national duty to provide educational opportunity for all is clear, but it would be a slow intellectual suicide to take care of the many at the neglect of the relatively few really able. These able are there; the public schools are rich in them, and in many cases they are wasted by being fed on scholastic pabulum when they need red meat which requires mental chewing. Gresham’s Law works in education as elsewhere. This is not a matter of private versus public education. One can be for public education, junior colleges galore, believe enthusiastically in the independent schools, and still think that such variegated places as Harvard, Haverford or Swarthmore, the University of North Carolina or the University of Minnesota, have crucial contributions to make to national life.

In aiming at our laudable goal of more education for all, we would do well to avoid falling into the soft fuzziness of making education for everybody easier at a time when what we desperately need is to make it a lot harder for those who are able. There are exceptions, but by and large our school standards are too easy for the outstandingly intelligent. If the conditions I have just mentioned materialize, students will need all the academic starch which can be put into them.

Therefore, while a great testing organization such as the College Board should do all it can to make for better testing of many sorts, one might maintain that it has an obligation to its own traditions and high standards to devise the best testing it can, regardless of time and expense, to find what boys and girls are really fitted for higher education, and to do everything it can to ensure their receiving the best all-round preparation for it.

By fairly common agreement there is merit in both objective and subjective testing. This is not an either-or problem. The ideal is neither alone, but something of both. Such an ideal would permit terminal tests as well.

The essence of the whole matter is whether the current trend in testing is thoroughly wholesome or not. To say it is excellent as far as it goes is not to justify it; nor is it justified by saying that the schools do not prepare for anything better, nor by recognizing frankly that the ideal would be both hard and expensive to attain. Nothing short of the best possible can be considered adequate. If what is adequate can be obtained only at increased expense, it is still a bargain for the commonwealth and for private citizens. To rest content with the partially satisfactory because it saves a few dollars and a lot of trouble is mighty poor economy.

College education is not the same thing as education for all members of a free society. They overlap and interplay, but the average citizen neither needs higher education, nor is capable of it, nor will confer benefits on society because of il; whereas advanced knowledge and power of deep reflection have an indispensable function in any civilized society. All men and women need the education essential for members of a free society; all benefit by making sure that those fitted for it receive a higher education.

A person will have ability to some extent whether trained or not. But the point of any formal educational process is to provide organized training which will eliminate as far as possible the risks of chance, while making sure that the individual benefits by the cumulative experience of the race and by the interpretation of that experience through the spirits of the wise and the learned. And a testing procedure which detects and encourages not only ability, but the trained use of that ability, surely contributes both to wisdom and its use.

It is idle to inveigh against the College Board or similar testing services as entities. No one can work with the officials of the Board without being keenly aware of their high-minded devotion not only to their immediate duty but to the cause of American education. They are agents, not masters, of those interested in testing, and their anxiety to do what is best is constant. Like all good agents they are bound by economic facts, by the most material and practical considerations, by the wishes of those they serve, and by a specific duty to weigh the merits and demerits of all suggested change. They are also professionals and specialists, keenly aware of amateurish vagaries. By rushing from one quick idea to another they could obviously do a great disservice. If real changes are to occur they must come not by the highhanded will of the agents alone, but by the wish and consent of the parties concerned. A great deal of thought has already gone into the problem, and, judging from past experience, changes will take place and remedies will doubtless be found for all these dangers.

Meanwhile, the ambitious college candidate or his equally ambitious parent would do well to realize that things are no longer as they were consule Planco. The modern test painfully, and perhaps wholesomely, makes it evident that not all people who would like to go to college are as smart as they might be when it comes to book learning — not even some very intelligent and useful citizens.