Fayetteville State University rarely crops up in the national conversation about educational quality.
Described as a second-tier regional university by U.S. News & World Report, the institution accepts nearly two-thirds of its applicants and struggles to graduate most of them in six years.
But the historically black college is also doing something right in the classroom, according to this year’s National Survey of Student Engagement, which was released on Thursday.
Researchers for Nessie, as the survey is known, took a stab at identifying educational quality on the institutional level, an attribute that is as important to higher education as it is hard to define. The survey collected data from 355,000 freshmen and seniors at 622 institutions in the spring.
Nessie researchers, who are based at Indiana University at Bloomington, created two indicators for quality. One, student-faculty interaction, asked students how often they talked with faculty members about career plans, course topics, or other ideas outside class, among other questions.
The other measure, effective teaching practices, distilled student perceptions of how often their instructors clearly explained course goals and requirements, taught in an organized way, used examples to illustrate difficult points, or provided feedback.
The results were surprising, especially when they were grouped based on how selective a college is.
Past Nessie reports have averaged the responses for disparate colleges, or organized them according to categories like their Carnegie classification. This year, however, researchers analyzed the measures of interaction and teaching according to selectivity, as defined by Barron’s Profiles of American Colleges.
The average student, the researchers found, experienced widely different degrees of educational quality in different colleges within the same category of prestige. And, in all but a few cases, the categories of selectivity had no meaningful relationship to the indicators of teaching and interaction.
Fayetteville State, for example, belonged to a broad category of “competitive” institutions that accept as many as three-quarters of their applicants, many of whom had B-minus averages in high school. The university had higher scores on the measures of teaching and interaction than many “most selective” institutions, which routinely turn away at least two-thirds of those who apply.
Fayetteville State’s freshmen scored their student-faculty interactions at 32 on a 60-point scale. Among the most-selective institutions, a category including places like Bryn Mawr and Harvey Mudd Colleges, the average score was 23.
“Conventional wisdom says that the more selective an institution is, the better it is going to be,” Alexander C. McCormick, director of Nessie, said in an interview. “That’s not systematically true with these two measures.”
But Mr. McCormick, who is an associate professor of educational leadership and policy studies at Bloomington, cautioned against concluding that selectivity doesn’t matter. And he emphasized that the level of variability of responses within colleges tended to be far greater than between them.
The results this year, though, do help achieve one of the goals behind Nessie’s founding in 2000—to shift the conversation about college quality away from assumptions about status, reputation, and wealth. By focusing on the sorts of activities that research suggests lead to robust learning, said Mr. McCormick, Nessie reflects something closer to what educational quality really is.
The Right Concept
Nessie’s findings support a contention often made by faculty members at less-prestigious institutions, that a college off the beaten path can still provide an excellent education.
The emphasis on the interaction between students and faculty members also aligns with recent studies and scholarship, including a Gallup-Purdue Index survey released this year. It found that graduates were three times as likely to report thriving in their sense of well-being if they had connected with a professor during college.
Similarly, Daniel F. Chambliss, a Hamilton sociologist, wrote in How College Works that students who took courses with just one or two faculty members whom they considered to be exceptional judged their entire undergraduate education to be excellent.
Even one of the most vocal critics of Nessie agrees that the two measures it identifies as indicating quality do have value.
But there is a much larger problem, said Stephen R. Porter, a professor in the department of leadership, policy, and adult and higher education at North Carolina State University. Nessie isn’t really measuring what it purports to, he said.
Students don’t have a common frame of reference to answer, for instance, how often they’ve learned something that changed their understanding of an issue over the past year, and few of them agree about what terms like “very much” or “quite a bit” mean. And then, he added, there are the questions themselves. Can students really judge whether their instructors are teaching in an organized way?
The range in the results among different levels of prestige, he said, is probably not evidence of the variability of experiences at the institutions; it’s statistical noise.
It struck Mr. Porter as bizarre to argue that attending a deep-pocketed college with a renowned faculty and a rich array of educational programs would be equal to going to an underfunded college where most students commute. “This does not seem at all plausible,” he said, “given what we know about wealth and selectivity in higher education.”
Quality Control
Mr. McCormick disputed Mr. Porter’s critique. If the students’ answers didn’t truly reflect their experiences, he said, the patterns Nessie found wouldn’t be seen on an institutional level.
The Nessie researchers’ emphasis on organized and clear teaching as indicators of educational quality also reflects recent findings from other scholarship. The Wabash National Study of Liberal Arts Education revealed a relationship between students’ feedback on these facets of teaching and their growth on a standardized test of critical thinking.
For some colleges, like the University of Pikeville, an open-access institution in Appalachian Kentucky, the Nessie data can help quantify a squirrelly concept like educational quality. Meg Wright Sidle, director of institutional research and effectiveness there, said Pikeville uses the results to evaluate and retool programs and to make sure the university is achieving its goals.
On Nessie’s 60-point scale, student-faculty interactions at Pikeville were 25.6 for freshmen and 32.8 for seniors, better than scores at even the most-competitive institutions. But it is also a very small institution, with a student-faculty ratio of 17 to 1.
Exceptional results on such measures probably won’t allow Pikeville to start poaching students from better known institutions in the state like Centre College or Transylvania University, Ms. Sidle said. “We’re still third tier on U.S. News.” But the numbers are valuable nonetheless. “We can say we’re meeting our mission.”
At Fayetteville State, John I. Brooks III, dean of the University College, said Nessie data offer another piece of evidence, along with scores on the Collegiate Learning Assessment, to help officials gauge quality.
While such data might help persuade students, parents, and policy makers that a Fayetteville State education is better than conventional measures and narratives might suggest, Mr. Brooks also struck a realistic note. Reshaping notions of an institution’s prestige or reputation take time.
“It’s harder,” he said, “to tell stories about quality than about quantity.”
Clarification (11/20/2014, 10:45 a.m.): This article has been updated to more accurately describe the lack of statistically meaningful differences in indicators of teaching and interactions among categories of selectivity.
The 2014 National Survey of Student Engagement: a Snapshot
The National Survey of Student Engagement asks students each year to answer a raft of questions: about their conversations with other students, how supported they feel by their campus, how often they engage in higher-order thinking, how many of them study abroad, and so on. Here are some of this year’s most interesting results.
Advising matters but does not always succeed. The more often that freshmen met with their academic advisers, the more supportive they thought the institutional culture was. But about a third of freshmen either never met with their advisers or did so only once. While the vast majority of faculty advisers thought it was “important” or “very important” that they listen closely to student concerns, make themselves available to students, and help fix academic problems, students said their advisers did so far less often. For example, 94 percent of faculty members thought it was important to discuss career interests and postgraduate plans, but only half of students said their advisers actually did so.
Demography is not destiny. Minority students generally reported having lower-quality interactions with advisers, faculty and staff members, and their peers. But there were important exceptions; black students at over 40 percent of the highest-performing institutions in this category reported interactions that were equal to or better than those of their white peers.
Time spent improving teaching means less lecturing. Instructors who dedicated more time to bettering their teaching were also more likely to use discussions, small-group activities, student performances and presentations, and experiential-learning opportunities.
Social media divert many students’ attention. While social media helped connect many students to extracurricular groups, about 40 percent of freshmen and a third of seniors said such tools “substantially distracted” them from course work.
