Testing the Teachers: The Dallas Experiment

When schoolteachers say that logical thought is a waste of time, ought one start to worry about the quality of the schools? The people in Dallas worried, and decided to do something about it.

Here is an excerpt from what may prove to be one of the more important unpublished works of 1978. It’s something we’d all like to know more about these days, the credo of a teacher:

I believe a role of a classroom teacher is to a leader of her class. She should be a leader in teaching her class knowledge to its full potentials. She should make sure that the children maximize in their subjects by having materials available for them at all times and that it is appropriate for them (the subject matter.) She should be patience and understanding with the children and know that all the children have different capabilities.

These words were written by a young woman looking for a job in the Dallas Independent School District (DISD). She was not hired, although she has probably taken a job someplace else by now. Maybe she’s teaching your children to maximize in their subjects.

A year earlier she might have had that job in Dallas. However, the people who hire teachers down there have now started doing some things you’re not supposed to do in a public school system. In most other large cities it is only the parents who discover that there are some bad teachers in the schools. In Dallas just about everybody has reached that conclusion. Furthermore, they have figured out how to do something about it.

The man in charge of hiring new teachers in Dallas is John Santillo, a small man, meticulously dressed. He has a serious face and the demeanor of a bank treasurer whose accounts no one would dare to question. In an office orderly enough to be a rebuke, he speaks in quiet, ruminative tones of power—the power to think, the power to seize and express ideas, the power that is given to those who have learned how to use “our two principal symbol systems, words and numbers.” Skill in words and numbers, thinks Santillo, does not alone make a good teacher, but the lack of it will almost surely make a bad one.

Few of the attributes of a good teacher can be specified, never mind measured, but skill in the use of words and numbers can be measured. All applicants for teaching positions in Dallas now take an exam designed to measure it. The Wesman Personnel Classification Test is not a test of intelligence or of knowledge. Many industries and business enterprises use it to predict success in almost any kind of work that isn’t purely physical. They find it satisfactory.

The Wesman test is made up of sixty questions: forty problems of verbal analogy and twenty elementary arithmetical computations. There is no passing or failing score. Those who give the test must decide for themselves what score best predicts the level of performance they want. The DISD has for now chosen a score of 35 correct answers as the lowest it can accept from a would-be teacher. (The Psychological Corporation of New York, the publisher of the Wesman test, suggests that an average score of 31.9 can be expected of “female clerical employees.”)

The verbal questions resemble this one:

-is to night as breakfast is to_ 1 flow 2 gentle 3 supper 4 door A include B morning C enjoy D corner

The first blank is to be filled in with a number, the second with a letter. The correct choices are, of course, “supper” and “morning.” While the subtle mind might object that the relationship between supper and night is not in all respects exactly the same as the relationship between breakfast and morning, the alternatives to the correct answer are obviously preposterous. That’s what “correct” means on the Wesman test—not preposterous. In John Santillo’s opinion, too great a readiness to tolerate the preposterous is undesirable in a teacher.

The twenty problems in arithmetic require only simple computation; no algebra, no trigonometry, no calculus. The most intimidating are the problems of square and cube roots, but they all come out even. Those who have forgotten how to extract roots can solve the problems with nothing more than the multiplication table. Scribbling is permitted. If you can’t figure out that 5x12 plus 3x7 is the same as 60 plus 21, or 81, and that 81 is 9x9, then your trouble is not that you have forgotten how to extract roots. The problems increase in difficulty from simple addition and subtraction to determinations of ratio. For instance, what one number will do for both blanks: 2 is to__as_is to 32?

Every applicant who takes the test is a college graduate holding a lifelong certificate to teach in the state of Texas. Some of them have made scores of 2 or 3 on the Wesman test. A few have scored zero. One applicant says: “I made a 17, which is extremely low, but it wasn’t dealing with my subject. It was dealing with logic, they say, but to me it was just more or less a waste of my time.” In Dallas they are willing to ask in public how much of any subject could have been learned by someone unable to work with ease in the symbol systems of words and numbers. Is there any other way to learn a subject?

William Webster is in charge of research and evaluation for the DISD. Webster is to Santillo as flow is to corner. He is bigger than some of the Dallas Cowboys, tieless, moustached, emphatic; his desk is strewn with work in progress, his office a comfort. He speaks with authority about norms and deviations, but his attack on the problems of the schools is not normal and his deviation from the customary political cautiousness of the administrator is hardly standard. “There are in this country,” he says, “probably only eight or nine school districts that have decent research and evaluation departments.” His is “probably the most efficient research department in existence.” They don’t call that hyperbole in Dallas; it’s just a reasonably modest assertion.

Webster designed a long-term plan for identifying and hiring good teachers. Since the teacher corps in Dallas numbers about seventy-five hundred and seems unlikely to grow by much, and since five or six hundred are replaced every year, Webster reasoned that a significant improvement in the overall quality of instruction could be made in a few years through nothing more than judicious hiring.

“We know that grades don’t mean anything,” says Webster, and Santillo agrees that an applicant’s transcript is almost worthless. No one at the DISD thinks colleges and schools of education can any longer be trusted. Elementary and secondary schools aren’t the only ones with a system of social promotion.

Partly for this reason, the DISD used to ask all of its new teachers to take the National Teacher Examination, a standardized achievement test which surprisingly few teachers’ colleges require of their graduates. Unfortunately, the NTE is given only a few times a year, and the scores are slow in coming back. “We’d hire a teacher in July,” says Santillo, “put him to work in September, and give him time off to take the NTE in October.” When the score proved dismal—and it often did—nothing could be done. By then the teacher had a contract.

The Wesman test was chosen not only because of what it measures but also because it can be given and scored before the contract is signed. But what, exactly, would the scores mean? Would the Wesman test be a useful substitute for the National Teacher Examination? Would the scores show any correlation? Furthermore, would the students of a teacher who had made a 50 on the Wesman learn any more than the students of a teacher who had made a 30? Would the annual principals’ evaluations of teachers, which are anything but quantitative, match the expectations provided by the Wesman scores?

Webster undertook to answer these and related questions through a year-long study of 535 first-year teachers, about two thirds of whom had taken the NTE. He began collecting facts in the late fall of 1977, when all of the new teachers were given the Wesman test. His intent was what he calls “norming.” The members of the test group were to be measured and assessed in many other ways too, so that reasonable expectations could be formed about future teachers.

But what seemed a good idea at the time proved calamitous, and for Webster’s purposes almost fatal. In March of 1978, while Webster was still collecting data, a reporter for the Dallas Times Herald learned that the test had been given. Under the provisions of the Texas Open Records Act, the newspaper sought and eventually effected the public release of the scores. The story became all the more interesting when it was reported that the DISD had fought hard against the release; and Superintendent Nolan Estes’s objections sounded like the stonewalling of any typical bureaucrat. Few believed his claim that premature disclosure might do irreparable damage to a research project designed not to cover up a deficiency but to cure one. But he was, in fact, telling the simple truth.

Inevitably, on July 18, 1978, the Times Herald announced across the top of the front page_ “50% OF DISD TEACHERS FAIL TO PASS TEST.” Since a newly unearthed DISD internal report suggested that no applicant who scored less than 35 ought to be hired in the future, to “fail” meant to make a score of 34 or less. Even worse, the Times Herald gave the test to about twenty students at a Jesuit prep school in North Dallas. Their headmaster says that the students were not selected, they were just volunteers. They did better than the teachers in Dallas. Much better.

It was a big F for the big D. William Webster, however, is not one to waste time complaining about a bad press. In fact, he points out, those teachers who looked so poor on the front page were as a group significantly above the national norm on the NTE. More important, though, was the fact that Superintendent Estes had spoken the truth. The meaning of those scores will be known, if ever, only after the collection of much more evidence—evidence now almost certain to be contaminated. Who, Webster asks, will now provide honest evaluations that are supposed to be confidential, knowing that they may appear in print next week?

“The scientific method,” he adds, “requires that the first step in any kind of problem-solving is to define the problem. If we have to define the problem all over the front page of the newspaper before we can complete our studies, the endeavor is not going to last very long. Furthermore, there are going to be very few institutions that will permit this kind of inquiry, simply because of the danger of uncovering the problems. It’s much easier to sit on our hands and say, ‘Problems? We don’t have any problems. Our teachers are all great. Our administrators are fantastic.’ We’ve got an example of that in all the suburbs around us. Their school districts are no better than Dallas, but they’ve got a whole lot better press because they don’t investigate their problems.”

The schools in the suburbs of Dallas are hardly unusual in their reluctance to investigate their problems. As the story of the tests in Dallas began to spread, many Americans must have concluded that they lived in school districts where the teachers were all great and the administrators were fantastic, but the problems were uninvestigated. No one seemed surprised that some schoolteachers were found to be mentally disabled; what surprised people was that a school system had gone to the trouble to discover that. The DISD had confirmed the growing public suspicion that this new breed of teacher enjoys unmerited trust and respect and has inherited the rank of “professional” only because the legitimate heirs have taken employment elsewhere. (John Santillo says that it was easier to find good math teachers before Texas Instruments came to town.) From all over people began writing and telephoning, asking how they could get their school boards to start testing teachers.

The schoolteacher’s increasingly strident claim to be a professional is much less convincing than it used to be when teachers were more obviously intellectuals. Certainly teachers are the least trained of all the socalled professionals, and their training is the least likely to be scrutinized by anyone other than those who claim to provide it. Its effectiveness cannot be tested in the marketplace; the pay scale is the same for the best and the worst teachers. To be certified as a teacher in Texas, as in almost any state, all one needs to do is to graduate from college with passing grades in a certain number of education courses. As to the worth of those courses and the demands they make upon the students, there is disagreement. There are applicants in Dallas who make very low scores on the Wesman test and on the National Teacher Examination, but who can nevertheless point to transcripts bristling with A’s and B’s in their education courses. In some cases, all their grades seem suspect. “I guess,” says an unsuccessful applicant, “the test made me look like a total idiot, but I am not. One cannot graduate from SMU with honors if they are not academically inclined.”

How is it, people are willing to ask in Dallas these days, that academic inclination at Southern Methodist University seems not to guarantee that pronouns will agree with their antecedents? Even applicants who are honors graduates, therefore, are asked not only to take the Wesman test but to write two paragraphs of prose about the work they hope to do.

Here is an example from a college graduate, a certified teacher, a professional:

As a classroom teacher my role will be to improve the knowledge of the students I wiil have to teach. I will also have to let the students know that I am truly concerned about their well fair, not only in the classroom but outside as well. I will have to teach the basic school subjects to students so that they will be better able to move to higher levels in education. Seeing to the mental well fair of students is also my concern. The years I receive children will be important years in the whole development the children. As the teach I must try to make these times as fore filling and successful as possible.

In the process of teaching science to students there will be a preassessment test to determine what the student already knows. From the finding of the preassessment test objectives, teacher and students will be designed. Concepts to be taught will be listed. If the subject area being covered is weather, than the different aspects of weather will be introduced. There will be worksheets and experiment to prove and evaluate the objectives set for the activity. The activity children will be given activities that they will have to do on their own example collecting date of the formation of clouds and what weather usually follow certain cloud formation. The project must be flexable so that it meets the need of each child and is meaningful for them.

They say in Texas that if you find a turtle up on a stump you know damn well he didn’t get there by hisseff. Bill Hallowed, a member of the Texas House of Representatives, has probably heard that saying often, for he plans to introduce a bill calling for a statewide standardized test for the certification of teachers. He does not intend that teachers currently employed should be tested, of course; even in Texas that would be suicide for a politician. He asks only that “something more than the passing of some courses in education” be used as a measure of a teacher’s potential competence. Even in that modest proposal he can expect some opposition. That turtle didn’t get there by hisseff either.

Many of the applicants in Dallas are graduates of the numerous nearby colleges, but not one dean of education has called John Santillo to ask how his or her students did on the Wesman test. Dale Davis, chairman of the department of education at SMU, is too busy to talk about the testing program in Dallas. Besides, he has been away a lot and has heard hardly anything about it. He does say, though, that “if you want a person’s ability, I would use one of the standardized IQ tests, and that way you would avoid controversy.” (William Webster makes a finer distinction between intelligence and ability; the Wesman is not a test of innate intelligence, but of an ability that can be learned.) Davis adds that some education students at SMU do take IQ tests “on a voluntary basis,” and that “only in one or two cases in the past few years have we had anyone that would fall below the average that we would have to worry about guiding them into something else.” He doesn’t say what that something else might be. Perhaps it’s English.

The National Education Association opposes all testing of teachers. Although the Classroom Teachers of Dallas is an affiliate of the NEA, it has taken a less extreme position. It had to. Many of its members clearly approve of the use of the Wesman test for applicants. They are decently proud of their competence, and they too wonder what’s happening to teachers. Ada Williams, the president of the CTD, has said that her organization will oppose the Wesman, or any test at all, should it be used as “the sole qualification for employment, promotion, demotion, transfer, or retention.” Since that is patently not the case—it is only one of eleven criteria for hiring—the few noises made at the CTD seem intended merely to comfort those first-year teachers who don’t believe Santillo when he says that their scores will be used for statistical purposes only. Ada Williams must believe Santillo, because the testing of current teachers for the purpose of evaluation is something that an NEA organization, even in Texas where unions walk softly, could not tolerate. Anyway, the superintendent has said that such a thing will never happen in Dallas. Indeed, if Webster is right in thinking that five or six hundred new good teachers every year will soon make significant changes in the quality of the teaching in general, it won’t have to happen.

It isn’t only the majority of the current teachers who approve of giving the Wesman test to applicants; most of the applicants seem to approve. In the waiting room of the DISD personnel department one young woman about to be tested admitted that she was nervous, but said that she could see no reason why a teacher should not be expected to demonstrate superior verbal and mathematical abilities. Another had just taken the test. Did she mind? “Why should I? I’m supposed to be a teacher.”

Some do have a different view, however. “I feel like it’s a way of eliminating certain people from the classroom,” said one. “I think it’s a way of eliminating minorities.” It is true that among both first-year teachers and applicants, Anglos (that’s what they call them in Dallas) as a group do better than Mexican-Americans or blacks, although there are very high and very low individual scores in every group. It is also true that culturally slanted questions can be found on some standardized tests, but even the objectors cannot find any on the Wesman. Ada Williams, a onetime music teacher who still speaks with the elegant grace of a singer, is herself black, but she does not see the test as a device for the elimination of minorities. She does see its results as another proof that minorities have traditionally been slighted in the schools, and about that there is no quarrel in Dallas. Indeed, some see those very results as a further argument in favor of the new hiring program. If the black and Mexican-American applicants of the future are to do as well as their white classmates, they’ll need the very best teachers they can have.

The Dallas Independent School District has all the troubles you would expect in any urban school system. Integration, however, while still on the list, is no longer at the top. Herb Cook, the executive director of the Classroom Teachers of Dallas, seems to speak for almost everyone when he says that integration is a matter mostly of “people problems” rather than institutional problems. “We are now,” he adds, “beyond that point where everything is a matter of race.” Like most of the members of the CTD, he is convinced that forthright and vigorous action by the most influential people in the city has provided an unusual measure of racial harmony in the schools. The leading citizens, he explains, made it known that there was no political profit to be derived from noisy, truculent opposition to integration and that the turmoil that took place in Boston and Louisville was not to occur in Dallas. It didn’t.

Private schools and academies did spring up, and some families did move out to the suburbs. Nevertheless, in the fall of 1978, at a time when the school-age population was declining, enrollment in the public schools of Dallas rose slightly. Some parents who sent their children to private schools are beginning to suspect that the public schools have more to offer after all, including better teachers. Out in the suburbs, some parents have decided that their school departments are investing in new buildings rather than in the enriched programs that the process of integration has provided in the city schools.

Furthermore, the administrators of the DISD feel that now that the matter of race has ceased to be the number one problem, it’s time to go to work on building a faculty that is not only integrated in accordance with legally mandated ratios, but is also expert and effective. Superintendent Estes has chosen not to wait around for the latter until the former is fully achieved.

William Webster’s idea of a good affirmative action program is one that will make such a program unnecessary in the future. “Sure,” he says, “we could do what they do in most other cities—get on the phone and say, ‘Send us another truckload of blacks. We don’t care what they’re like, just so long as they’re black.’ ” That, thinks Webster, is the new way of excluding minorities from access to the essential skills of an increasingly technological society, and a guarantee of revolution in the future. He isn’t kidding. “We are going to keep hiring minority teachers,” he says, “the best ones in the country. We have a quota and we’re going to fill it, but we’re going to fill it with the best.”

Where other cities have civic pride, Dallas has what must be called municipal patriotism. In the classrooms of the DISD there are almost as many volunteers as there are teachers. The leading citizens, having once been scolded by a judge for failing to lead the way into integration, have formed a coalition with parents, teachers, and administrators called the Community Network for Public Education. Among its projects is the occasional sponsorship of a survey of public opinion about the schools. In spite of all the publicity about the test scores (could it be because of that publicity?), the latest survey shows some surprising increases in public approval. The percentage of parents who would recommend the schools to newcomers rose in one year from sixty-eight to eighty-one, and among parents in the almost completely black East Oak Cliff section that figure rose from seventy-five to an impressive ninety-one. Even confidence in the ability of the teachers increased.

Nolan Estes likes to say, and he says often, that the school system is the flagship of Dallas; that where it leads the city will follow; that what befalls it befalls all Dallas. In Chicago or Philadelphia the response would be, “Yeah, yeah.” In Dallas they believe it. Someday soon, many feel, the schools in Dallas will be the Cowboys of American public education. Even Webster, who insists that continuous public disclosure will ruin his studies, says that if that should happen, they’ll just have to find some other way to improve the quality of instruction. Like everyone else, he’s sure there’s a way.

If public education is to be saved anywhere, it may be in Dallas. The people have an active and optimistic sense of community, a reasonable and realistic teachers’ organization, an administration with the resources to design the faculty of the future, and the will to do it all.

Wisdom in Dallas is ordained in strange ways. One of the applicants wrote: “If we were working on math for example—If we were playing the bingo game I would reward one person with a reward for bingoing if they could call out their numbers to me or if they couldn’t remember a number I could help them and still they would bingo.” Another: “Cover any questions the students might have & give them a handout of material that would be covered on a test, handout will include the things not to do and how to do them right.”

There’s a message in the mess. These are fore filling times in Dallas, times when teacher and student are designed. If Nolan Estes, John Santillo, William Webster, and the citizens of Dallas can just keep their flagship afloat, they may yet bingo, and the fair may yet be well. At least they do know the things not to do and how to do them right. □