What Should a Major Teach? ‘Adrift’ Authors Offer Answers

Jay Premack for The Chronicle

A new book from the authors of Academically Adrift: Limited Learning on College Campuses seeks to answer one of the main critiques of that widely read 2011 study.

One knock on that book, by Richard Arum and Josipa Roksa, is that they relied largely on scores from a standardized test to support their conclusions that many students don’t learn much during college. The test they used, the Collegiate Learning Assessment, or CLA, measures critical thinking as a generalized trait, one that’s not attached to a specific discipline.

But is it fair, critics asked, to test students on a generalized skill when the most substantive learning they do takes place in a discipline? Shouldn’t we be paying attention to what students are learning in their majors?

Mr. Arum and Ms. Roksa agreed. Improving Quality in American Higher Education (Jossey-Bass), which they edited with Amanda Cook, a program manager at the American Association of State Colleges and Universities, describes the work of the Measuring College Learning Project.

It is a faculty-led effort to define the core ideas and fundamental practices underpinning biology, business, communications, economics, history, and sociology, which collectively account for more than 35 percent of American bachelor’s degrees. Scholars met over two years and were guided by past efforts in their field. For each discipline, a dozen or so faculty members (70 total) staked out lists of two things, concepts and competencies, a framework that emerged from the biologists’ group. An example of an essential concept from that discipline is evolution; a competency is arguing from evidence.

In a conversation with The Chronicle, Mr. Arum, who is leaving New York University this month to become dean of the School of Education at the University of California at Irvine, and Ms. Roksa, an associate professor of sociology and education at the University of Virginia, discussed the need for disciplinary consensus, how faculty members might use the fruits of the project, and how this work might result in better tests. This interview has been edited for length and clarity.

Q. What need do you think the Measuring College Learning Project fills?

Mr. Arum. There were multiple needs. One of the responses to Academically Adrift was a critique from colleagues that a generic measure of critical thinking, complex reasoning, and writing did not do justice to what faculty within disciplines were working to develop in their students. Another reason is that, like everyone working in higher education, we look around us at the efforts to measure higher-education outcomes with what Josipa and I see as inadequate and wrongheaded approaches. That is, often looking at distal labor-market outcomes.

Q. Meaning things like first-year earnings?

Mr. Arum. Yeah, or earnings six years out, labor-market outcomes that colleges and universities have a very imperfect and weak influence over, rather than outcomes that, in fact, they have responsibility for helping their students to achieve.

Q. I was struck by the way that learning was deeply bound up in discipline in this new book as opposed to how it was treated in Academically Adrift, through the CLA. I wonder if you could talk a little bit about the virtues and flaws of each of these measures, those that are nested in a discipline and those that are generalized.

Ms. Roksa. We think you really need to measure both. There is a link between the competencies that each discipline articulates and the more generic ones: analysis, problem solving, writing. While the concepts are discipline-specific, the competencies we started to think of as discipline-specific manifestations of these more general ideas.

There’s a lot of debate about the extent to which what you learn in a discipline can really transfer. Is critical thinking a disciplinary skill or a generic skill? If you wanted to disentangle those, you’d have to measure those. When faculty think they’re teaching critical thinking, complex reasoning, or analysis, they are teaching them in their disciplines. So that gave us a reason to really focus on disciplines and disciplinary approaches to teaching and learning.

Q. What commonalities in the competencies struck you the most?

Ms. Roksa. There are differences, but there is really this emphasis on higher-order thinking: analysis, evaluation, examination. How you approach a historical document versus how you approach an experiment is different, but it assumes you have a certain content knowledge and then you’re using that to apply, analyze, draw conclusions, and make inferences. It’s not a lower-level of, Do you remember and do you know certain things? They tried to engage students at a much higher level.

Q. You both mention that one thing you learned is that faculty members readily agree on what students should learn, and that this might surprise some readers. Did it surprise you?

Mr. Arum. Going in, we expected that that would be the most difficult part of the project. But we found that in each of these six fields there was a long history of scholars’ working to define learning outcomes, and so people went at this task very systematically and deliberately. Faculty were very quickly able to define what were the essential elements of their major and discipline that you’d expect any student to master. We found there was lots of consensus.

Q. What would be a good way for faculty members to look at these white papers and at these competencies and concepts? How might they use them in their teaching?

Ms. Roksa. One way to think of this is on a department level. If sociologists agree that these are the core concepts and competencies that students ought to master by the time they graduate, it leads to a question of curriculum. It allows us to say, If we agree that these are the core competencies and concepts, where are they in our curriculum? Who’s teaching them? What classes are they covered in?

Mr. Arum. Another use for them, and something I did at NYU when I was teaching an introduction-to-sociology class, was to share them with the students. I handed out the table of concepts and competencies in sociology and said, This is what a national group came together and articulated as essential learning outcomes. Many of you in an introductory course have a question of, What is the discipline, and what would I potentially get out of a major? What’s the goal here? We think that this is particularly useful today when people are asking questions about what the inherent value is of pursuing a particular course of study.

Q. I imagine it will be key, going forward, to make sure that the things staked out in this book are actually happening. How do you see that unfolding?

Mr. Arum. This has to be faculty-led and voluntarily adopted, be iterative, and be part of a larger holistic assessment program. We don’t want one set of standard measures. What we’d like to do is develop some of these tools and bring faculty together with assessment firms that produce these products and see if we can come up with 21st-century assessments. There already are these standardized assessments in the field. The trouble is, they’re not very good, and faculty weren’t at the forefront of designing and developing them.

Ms. Roksa. We understand the concerns one could have with trying to identify outcomes and trying to measure them. The precedents, in K-12 and higher education, aren’t encouraging. The measures are already out there, and what is out there is not compelling. Departments complain, but with accrediting and other pressures, they feel they have to use something, so they use whatever’s available. It’s really important for us as faculty to be proactive in defining what students should learn and defining how that could be measured.

Dan Berrett writes about teaching, learning, the curriculum, and educational quality. Follow him on Twitter @danberrett, or write to him at dan.berrett@chronicle.com.