Report says data don’t give a true picture of success
Colloquy Live: Read the transcript of a live, online discussion with Clifford Adelman, a senior research analyst at the U.S. Education Department, about how graduation rates are an unreliable measure of a college’s performance, even though lawmakers are weighing proposals that would penalize or reward colleges based on those rates.
Chart: Graduation Rates and Student MobilityBy STEPHEN BURD
April found her calling during her sophomore year at Oklahoma State University. While taking a course in limnology, the science of lakes, ponds, and streams, she decided that she wanted to pursue a degree in marine biology.
There was a hitch, though. Oklahoma State, almost a 10-hour drive away from any major body of water, did not offer a major in the subject.
So, with the encouragement of a professor, April transferred to the University of Rhode Island, where she could take a 30-minute bike ride to the Atlantic Ocean whenever she pleased. At Rhode Island she excelled, graduating at the top of her class.
A success story? For April, yes. But for the two universities involved, no -- at least not in the eyes of the federal government.
Under U.S. Education Department regulations, Oklahoma State had to count April as a dropout in calculating its official graduation rate because she did not earn her degree there. She failed to show up in the University of Rhode Island’s rate, however, because she had not begun her studies there.
April is the invention of Clifford Adelman, a senior research analyst with the department, who likes to tell the story to illustrate a key point in a new report he has written: Graduation rates are an unreliable measure of a college’s performance.
In the study, Mr. Adelman analyzed the college transcripts of students who graduated from high school in 1992. What he found was that a growing number of students, like April, jump from one institution to another during their college years. For example, he discovered that one in five students who had earned a bachelor’s degree received it from a four-year college other than the one in which they first enrolled. Under federal rules, those students would be counted as dropouts.
For colleges, this is not just an academic matter. With Congress preparing to renew the Higher Education Act, the law governing federal student-aid programs, the Bush administration and Republican lawmakers have been weighing proposals that would penalize or reward campuses based on their graduation rates.
Mr. Adelman does not believe that individual colleges should be judged by those rates. “Love, homesickness, comfort level, change of major, change of interest -- these are some of the major reasons why people change colleges,” he says. “And this raises the question, Why should institutions be judged for choices, made by students, that are beyond their control? College students are legal adults, after all.”
A Fine Line
For Mr. Adelman, this may be dangerous ground to tread. As an Education Department employee, he ultimately works for Bush-administration officials who have said repeatedly that they believe that colleges should be held more accountable for their graduation rates.
But he is no stranger to controversy. Over his 24-year career as a researcher at the department, Mr. Adelman has often had to walk a fine line, reporting the evidence of his research findings while staying off the toes of politically-minded superiors. In an interview on his new study, he sidestepped questions about the Bush administration’s positions, noting that as a federal worker he could not comment on them.
Many college lobbyists, who have had run-ins with the Bush administration over accountability issues, are embracing Mr. Adelman’s findings. “Transcripts are the DNA of higher education; they don’t lie,” says Terry W. Hartle, senior vice president for government and public affairs at the American Council on Education. “This report presents the most complete picture that we have of what students are doing today.”
But some higher-education researchers, even those who tend to agree with much of Mr. Adelman’s research, disagree with him over the importance of monitoring graduation rates. They argue that the government needs to pay close attention to the rates at which financially needy students graduate. Mr. Adelman’s own study, they point out, shows that less than 20 percent of the 1992 high-school graduates from the lowest-income families had earned bachelor’s degrees by 2000, compared with over 70 percent of those from the wealthiest families.
“I wouldn’t want policy makers to pick up this report and say, ‘Gee, maybe things aren’t as bad as we thought,’” says Donald E. Heller, an associate professor of education at Pennsylvania State University’s Center for the Study of Higher Education. “We still have a problem, and that problem is the large gap in graduation rates between rich and poor students.”
A ‘More Honest’ Accounting
Mr. Adelman is a storyteller at heart.
He has spent much of his career combing through academic transcripts so he could tell stories about students -- who they are, how they perform, why they make the decisions they do, and what social and policy dynamics lead to their successes and failures.
The story he tells about higher education in his latest report, “Principal Indicators of Student Academic Histories in Postsecondary Education, 1972-2000,” is a much more positive tale than those that have been heard on Capitol Hill or from other researchers lately.
Mr. Adelman studied the college transcripts of about 9,000 students who had been followed by department researchers over an eight-and-a-half-year period, starting in 1992. As part of the study, he compared the progress of those students with that of similar groups who had graduated in 1972 and 1982.
In making the comparisons, he found that suggestions that students were dropping out of college in ever-greater proportions were wrong. In fact, his report shows that despite a 50-percent expansion in the number of students going to college over the past 25 years, bachelor’s-degree attainment rates for traditional-age students remained stable. The proportion of students who received bachelor’s degrees by the time they were in their middle to late 20s had remained at about 50 percent since the 1970s.
In getting that result, Mr. Adelman focused on “nonincidental students” -- those who had earned at least 10 credits in postsecondary education. He excluded students who had earned fewer credits, he says, because many of them went to college to take a course or two without ever intending to get a credential.
But the statistics don’t tell the whole story, he says. To get a “more honest” picture of bachelor’s-degree attainment rates, he narrowed the group to include only those who had earned at least 10 credits in postsecondary education and received “any credits from a four-year college at any time.” Doing so was necessary, he explains, because without that limitation, the sample would include “students from cosmetology schools and local-area vocational institutions” who had never set foot in a four-year college.
In that pool, Mr. Adelman found that two out of three students had earned bachelor’s degrees while in their 20s.
“We’ve been presented with an image of higher education in which students don’t graduate or finish in a reasonable amount of time, and that’s not true,” he says. “When you really start looking at it, students are doing a lot better than we think.”
Some higher-education experts, however, question Mr. Adelman’s assumptions. Frank Newman, a visiting professor of public policy and sociology at Brown University who is director of a project examining the future of higher education, says it is true that some students go to community colleges to take only a course or two. But “it’s more often the case,” he says, that students “who have had a difficult time academically in high school” enroll in a couple of classes to see “if they have a shot” at pursuing a higher education.
“It is the job of community colleges to capture these students and help them succeed,” he says. “In some sense, Cliff is letting institutions off the hook” by writing off those students.
Mr. Newman also notes that while the data in the report show a huge gap in graduation rates by socioeconomic status, Mr. Adelman writes little about it. “Higher education may be doing a better job than people think, but there are some distinct problems,” Mr. Newman says. “We should not be understating them.”
Mr. Adelman responds that examining the difficulties faced by financially needy students was not the point of his report. “I’m not focusing on the margins, high or low,” he says. “What I set out to look at was the vast majority of students who are in higher education. I think it is about time we acknowledged that these students persist and do well.”
Keeping Track of Students
In 1990 Congress approved legislation requiring colleges to make public their graduation rates. It took Education Department officials five years to develop regulations on how colleges would calculate those rates.
One of the main sticking points was whether transfer students would be included. The department decided that including them would be too complicated. Instead, colleges were to follow the progress only of full-time students who had entered as freshmen and stayed on to graduate within six years.
Many states, interested in devising their own accountability systems, have tried to do a better job tracking students. But they, too, have faced obstacles. For the most part, higher-education officials can keep track of only those transfer students who move between public colleges in a given state. According to the American Association of State Colleges and Universities, only three states -- Ohio, Oklahoma, and Virginia -- have systems in place that allow them to follow students who transfer between public and private colleges in those states. No states have the capability to determine what happens to students who leave the state to study elsewhere.
Mr. Adelman’s study shows that the inability of state and federal government officials to track students across state lines makes it nearly impossible for the federal government to get an accurate count of graduation rates. In his report he notes that of the 20 percent of students who earned a bachelor’s degree from a different four-year college than the one in which they started, almost half moved to institutions in other states.
Some researchers have recommended that the government create a national database to keep track of all students. Mr. Adelman has not taken a position on that issue. “It’s up to the higher-education community to arrive at a proposal if they’re interested in tracking student movements across institutions and state lines,” he says.
Mr. Hartle, of the American Council on Education, says putting together a national database “is essential if you want to get accurate information.” But “the creation of a new database, with tens of millions of records organized around students’ Social Security numbers, is guaranteed to be complex and controversial,” he says.
Another idea, Mr. Hartle says, is for the government to expand the Education Department’s National Student Loan Data System, which keeps track of tens of millions of students who receive federal student aid each year. Some college lobbyists, however, are concerned by reports that the database is rife with inaccurate data.
No matter what federal officials decide, Mr. Adelman does not believe that the government should penalize colleges for failing to graduate students.
To explain his position, he returns to April.
“Should the professor at Oklahoma State have discouraged her from pursuing her dreams because, God forbid, she would leave the university?” he asks. “No, that would not have been responsible of him. In this case, he did the right thing. God bless him.”
| GRADUATION RATES AND STUDENT MOBILITY Having analyzed the college transcripts of students who graduated from high school in 1972, 1982, and 1992, Clifford Adelman, a senior researcher at the U.S. Education Department, reports that bachelor’s-degree attainment rates have remained stable for over 30 years. Of students who earned more than 10 college credits, the proportion who received bachelor’s degrees: Of students who earned more than 10 college credits and any credits from a four-year college at any time, the proportion who received bachelor’s degrees: |
|
|
|
| He also found that the complexity of tracking students from one institution to another makes it nearly impossible to accurately determine colleges’ graduation rates. Of those 1992 high-school graduates who earned bachelor’s degrees within eight and a half years: SOURCE: “Principal Indicators of Student Academic Histories in Postsecondary Education, 1972-2000,” Clifford Adelman, U.S. Education Department, March 2004
|
http://chronicle.com Section: Government & Politics Volume 50, Issue 30, Page A1