There’s about to be a new entrant in the college-ranking business: the Boeing Company.
The Chicago-based aerospace giant has spent the past year matching internal data from employee evaluations with information about the colleges its engineers attended. It has used that analysis to create a ranking system, which it plans to unveil in the coming month, that will show which colleges have produced the workers it considers most valuable.
With a 160,000-person work force that includes 35,000 engineers worldwide, Boeing may make a mark where the government and others have not — raising the possibility that employers could become a major force for college accountability.
“We want to have more than just subjective information” for evaluating the colleges that Boeing visits to recruit and hire, said Richard D. Stephens, the company’s senior vice president for human resources and administration. “We want to have some concrete facts and data.”
In some ways, Boeing’s approach to evaluating colleges is just a more- sophisticated extension of businesses’ long-standing practice of using historical trends to help them decide which colleges to favor. In simpler terms, said James L. Melsa, a retired dean of engineering at Iowa State University, that has meant practices such as hiring graduates of the same colleges that top company executives attended.
Precision Work
But the Boeing model could have even wider implications, suggesting common ground for educators and policy makers who have been in a heated battle over the best ways of bringing scientific precision to the job of evaluating college performance.
Mr. Stephens himself was a member of the Commission on the Future of Higher Education, a panel formed by U.S. Education Secretary Margaret Spellings in 2005 to determine “how we can get the most out of our national investment in higher education.”
As he sat through meeting after meeting of the commission, listening to long discourses by education professionals about ways to lower the cost and improve the effectiveness of college, Mr. Stephens decided that his company could take action on its own.
He asked Boeing officials to review detailed performance records of the company’s 35,000 engineers, an exercise that revealed differences in quality that could be correlated to workers’ alma maters.
Boeing will now use that data for internal purposes, guiding the company toward colleges it wants to favor in such areas as recruiting and hiring. The performance data also will drive its choices of partners for academic research and its decisions about which colleges it will ask to share in the $100-million that Boeing spends each year on course work and supplemental training for its employees.
The company also plans to share its findings with the colleges, in an effort to help them improve themselves. Within the next few weeks, when Mr. Stephens sends 150 letters to engineering deans notifying them of their colleges’ rankings, he also plans to offer them specific critiques, based on the work records of their graduates.
“It’s really about improving the dialogue on curriculum, performance, and how we can build a stronger relationship between the colleges, universities, and us, because, ultimately, their students become our employees,” the Boeing executive said.
Receptive Audience
Colleges and their engineering-school deans appear receptive to the advice, but many want both Boeing’s advice and the rankings kept private.
“Constructive feedback that allows us to improve the quality of the education we’re providing to our students, to me, is always a good thing,” said Sarah A. Rajala, dean of engineering at Mississippi State University and president of the American Society for Engineering Education. At the same time, deans must be free to put the information in its proper perspective, meaning privately, without the pressure of a public ranking, she said.
Boeing will treat its findings as confidential, Mr. Stephens said. The top-ranked colleges, however, may be less interested in keeping the news quiet. “We’re not under any false pretenses that whatever we tell people they’re going to keep amongst themselves, for those who rank at the top of the list,” he said.
Self-promotion is especially likely because some lesser-known institutions will be revealed as having done an “excellent” job of producing high-performing Boeing engineers, Mr. Stephens said, without identifying any such colleges ahead of their expected notification.
Ranking the nation’s engineering schools may not be entirely without risk. Some colleges have been accused of making cosmetic or even detrimental changes in their operations to bolster their scores on the annual rankings by U.S. News & World Report, for example, without improving the actual quality of their programs.
The Boeing rankings could invite similar attempts to game the numbers, perhaps by steering students into or away from certain specialties, said Mr. Melsa, the former dean at Iowa State.
Any such responses would probably be rare, said Mr. Melsa, who is also a former president of the engineering-education society. “I have more faith in faculty members and deans and department chairmen than to believe they will all of a sudden start doing unnatural things because Boeing either put them at the bottom of their list, or the top of their list, or didn’t have them on their list,” he said.
Definition of Success
The Boeing project also will invite greater examination of its employee-evaluation criteria, which Mr. Stephens has described in general terms as involving both technical and nontechnical skills. Ideally, the company’s measures will reflect a wide-ranging definition of worker success and may thereby spark greater appreciation of well-rounded students, said Carol Geary Schneider, president of the Association of American Colleges and Universities.
The Spellings Commission produced a final report that emphasized the need to get more students into college at a lower cost and to send them back into the world with better job skills. But, Ms. Schneider said, the panel paid little or no attention to strategies for broadening the definition of a successful college graduate in such areas as critical thinking, problem solving, and social responsibility.
Companies such as Boeing can help emphasize their value to students, Ms. Schneider said. “Employers are much better able to call students’ attention to the importance of these skills than their deans are,” she said.
Those skills, including the ability to communicate and work well in teams, are some of the same attributes Boeing seeks in its employees, said Mr. Stephens. “We need engineers who can practically apply what they’ve learned in school to help our customers.”
Boeing’s definitions of a successful worker also could fuel the debate among Washington policy makers over what constitutes a successful college. Diane Auer Jones, president of the Washington Campus, a consortium of university business schools, resigned in May as U.S. assistant secretary for postsecondary education, saying she felt her colleagues in the Bush administration were pressing colleges too hard to make their curricula fit specific corporate needs.
“Higher education prepares individuals not just to be good engineers, but to be good citizens, good parents, good community members, and to appreciate those things that enrich our lives, like art, music and literature,” Ms. Jones said. “Even under the best of circumstances, Boeing would only be able to measure one aspect of higher education.”
Potential Pitfalls
Mr. Melsa also recognizes the dangers in a college’s focusing too much on its rating from Boeing. Iowa State trains engineers not only for Boeing but also for Lockheed Martin, Exxon Mobil, and John Deere, among other companies. Graduates need to be trained broadly enough so they will be able to solve problems not even anticipated with technologies not yet invented, he said.
Boeing’s system of evaluating its engineers doesn’t typically reward innovators, said Paul R. Illian, who spent 22 years as an engineer there, including seven years as an instructor in employee training and development.
The corporate job-performance criteria often emphasized the need to ensure that “the average employee could do an adequate job,” said Mr. Illian, a researcher at Seattle University who left Boeing in 2001 and has also worked for more than a decade on designs for human-powered flight. “The extreme employees, either plus or minus, struggled with this, whereas the ‘Billy Joe Bob’ engineer in the middle flourished under it,” he said.
That’s not necessarily bad, he said. As a passenger on Boeing-built jets, he said, “I was darn glad to have that approach. I didn’t want to be sitting in Row 37D on an airplane where they took a lot of chances in the design.”
Boeing’s approach to evaluating employees also varied with economic conditions, Mr. Illian added, raising the possibility of year-to-year variations in the company’s assessment of academic quality. In calm economic conditions, it would “reward good team players and people who really got along.” During times of stress, it would emphasize “those whose skills were necessary for the organization to keep the company functioning.”
‘Inherently Fallacious’?
Rankings have other potential pitfalls, said Arthur J. Rothkopf, a senior vice president at the U.S. Chamber of Commerce, who served alongside Mr. Stephens on the Spellings Commission. Many college presidents believe it’s “an inherently fallacious concept to start ranking institutions, on whatever measure,” said Mr. Rothkopf, a former president of Lafayette College, who leads the chamber’s programs on education and work-force development.
As with the U.S. News statistics, employer-driven ratings could suffer from biases favoring colleges that already enjoy a strong reputation, since the top-performing students tend to enroll at the most-prestigious institutions, Mr. Rothkopf said.
Smaller colleges might face a further disadvantage, he said, since they would be likely to have too few graduates employed at any single company to generate any statistical significance in the ratings. On the other hand, a smaller institution could emerge as an unheralded star, Mr. Rothkopf said, gaining an industry-specific reputation such as that enjoyed by Williams College in producing art-museum directors.
Boeing is not the only company seeking better ways of choosing its college partners. Others include Microsoft, which directs its most aggressive recruiting toward a group of about 15 colleges, said Warren K. Ashton, the company’s manager of recruiting.
Those relationships, some dating to the 1980s, involve colleges that offer programs important to Microsoft and have a track record of providing high-quality workers, he said. Microsoft, however, hasn’t dropped any colleges from its priority list over the years, and it might find a Boeing-type model useful for bringing more precision to its recruiting efforts, he said.
Like Boeing, Microsoft seeks a combination of abilities in potential workers, including technical abilities, skills in teamwork and problem solving, and “business savvy,” Mr. Ashton said. But the company is more concerned with identifying parts of the world that are most promising for recruiting than in selecting individual colleges, he said.
Change often threatens those on top. Iowa State already enjoyed strong recruiting connections at Boeing, in part because its graduates include Thornton Arnold Wilson, a former Boeing chairman and chief executive, and Vance D. Coffman, a retired chairman and chief executive at Lockheed Martin.
The alumni preference conferred upon Iowa State by Mr. Wilson may not have been as scientifically sound as Boeing’s new process, but it was based on the real-world understanding among executives that the colleges they attended do a good job of producing successful graduates, Mr. Melsa said.
“We like the system” as it existed, admitted the former dean at Iowa State. “There’s nothing wrong” with it.
For their part, Education Department officials praise the initiative of Boeing and any other companies that might help colleges as they struggle for ways — other than the standardized tests that many of them oppose — to demonstrate their worth to students.
Boeing’s project undoubtedly costs the company more time and money than most businesses could spend, said Cheryl A. Oldham, acting assistant secretary for postsecondary education. But for colleges receiving Boeing’s employee analysis, said Ms. Oldham, who was executive director of the Spellings Commission, “it’s exactly what they would want to know about how well they are preparing their students to go out into the business world.”
http://chronicle.com Section: Money & Management Volume 55, Issue 4, Page A1
