Christopher A. Conte can hardly stand to utter the word “rubric.”
He finds the grading grid, which has recently captivated many of his faculty colleagues in the history department here at Utah State University, entirely too rigid.
The scoring guide maps out specific learning goals, giving detailed explanations of how a professor will judge a student’s mastery of each. The jargony concept is the talk of his history department now, prompted by a new national project designed to more precisely define the knowledge and skills that go into a degree.
The Lumina Foundation for Education is supporting faculty-led, discipline-specific discussions within several states that seek to articulate what a student should know and be able to do by graduation. The meeting-intensive project is at times clouded with the kinds of bureaucratic buzzwords, like “rubric,” that can turn off educators and obscure the ways it might lead to tangible change.
But proponents of the effort say it brings clarity to the work of academe and makes a stronger case for the value of a college education. At Utah State, history professors say the process of articulating more-detailed expectations for its graduates has prompted faculty members to become more deliberate about what they teach and students to become more cognizant of how they learn.
Mr. Conte, who teaches undergraduate history courses and is the department’s director of graduate studies, has come around to seeing the worth of being more explicit. He’s still never going to talk about rubrics with his students. But he has begun to take time in his classes to spell out the department’s goals for the knowledge, thinking, and skills that history students should be able to demonstrate.
“It’s a useful tool in class,” Mr. Conte says, “so students have a good idea what to aim for and so they understand what a historian does.”
Aurora Wallis Durfee, a senior history major at Utah State, likes the detailed guidance she is getting and is a big fan of the rubric. She writes her essays with the two-page grid sitting beside her computer. It prompts her to ask herself such things as: Does this demonstrate a clear mastery of the concept? “I read my own paper from a more objective viewpoint,” she says. “It allows you to fine-tune your work.”
With its project, Lumina wants states to zero in on these kinds of nitty-gritty details about classroom learning and agree on the specific ingredients that make up a high-quality college education. The statewide conversations could help colleges respond to demands for accountability by providing a guarantee to students, parents, businesses, and lawmakers about what their graduates will know.
“Quality in higher education is best represented by what students learn,” says Marcus Kolb, a program officer at Lumina. “We hope this will elevate student learning to the center of the conversation.”
Indeed, statewide and national debates about higher education often focus on who gets into college and who gets out but not as much about what happens in between. Lumina is pursuing the question of what a degree means as part of its focus on increasing the number of Americans with college credentials.
The foundation has set a goal of getting 60 percent of Americans to hold “high-quality” postsecondary degrees or credentials by 2025, a goal similar to President Obama’s. About 39 percent of U.S. residents hold associate degrees or higher, a level at which the country has been stuck for four decades.
With so much emphasis on improving college completion, some people worry that the pressure to award more and more degrees could create a perverse incentive to move people through higher-education systems without much regard for the rigor of the diplomas being handed out. And if their meaning is unclear, what good does it do a nation to focus on broadening its citizenry of degree-holders?
To try to answer these concerns, Lumina looked to Europe, which has spent more than a decade working to synchronize its systems of higher education through the Bologna Process, named for the location where a declaration starting the effort was signed in Italy, in 1999. A part of that effort, called Tuning, has gathered academics, discipline by discipline, to try to reach agreement on common learning goals, with the idea of promoting transparency, coordination, and quality assurance across borders.
In the United States, Lumina has sought to recreate those conversations. In the first round of its Tuning USA project, the group last year awarded $150,000 each to Indiana, Minnesota, and Utah, guiding teams of faculty members and student representatives in each state to spell out expectations for graduates at every degree level of certain disciplines. The groups surveyed alumni, business leaders, and others for input. Indiana professors worked on the fields of chemistry, elementary education, and history; Minnesota participants met about biology and graphic arts; and Utah officials discussed history and physics.
The Lumina project faced skeptical faculty members who have seen numerous efforts to improve accountability and measure quality in higher education come and go. Those processes often are antagonistic, pitting academics against state lawmakers or other outsiders, whose proposed measurements sometimes lack nuance, which in turn limits buy-in from educators. But many professors have ended up embracing the Lumina project, particularly when it became clear that faculty were driving it. Foundation officials say they see the enthusiasm that many professors developed for the process as a key strength of the project.
Susan O. Shapiro, an associate professor of history and classics at Utah State, says she was wary when she heard about the Lumina project and its call for historians in her state to articulate learning objectives. “We do this anyway, and this is just an added thing to do,” she says of her first reaction. “I’m the expert, I know what I do, and I think I do it pretty well.”
But she says she has since come to appreciate the Lumina project. It allows flexibility for individual professors to adapt statewide learning goals to their specific courses and prompts introspection that has led to greater clarity of purpose in the classroom. The Lumina project has improved students’ understanding of what professors are asking them to do, Ms. Shapiro says. And it has helped faculty to get out in front of accountability conversations.
“It gives us something to say to all these people who have assessment on their minds, and we get to define it,” she says. “There is a lot of pressure to defend ourselves. This helps us begin to make that case.”
Different States, Different Outcomes
The Lumina project so far has been working better in some states and some disciplines than in others. In history, a field in which professors are typically accustomed to making the case for the skills its majors gain, faculty groups in Indiana and Utah have relatively easily agreed on learning objectives. But elementary-education professors in Indiana struggled, with project participants noting that their profession is already guided by hundreds of existing standards and is often subject to other forces of change. The state’s education department, for example, recently announced significant revisions to teacher-licensing requirements, which may lead to new standards at schools of education.
Many of the statewide conversations have yet to result in significant changes to the way colleges, departments, or individual faculty members do business. But participants in the Lumina project commonly say the process has led to important dialogue across their states and laid the groundwork for progress in areas such as student transfer.
The most change has happened in Utah, a state that was particularly ripe for the Lumina project. Higher-education leaders here have long been engaged in detailed statewide meetings the system coordinates about learning, including through annual gatherings of professors major by major and a yearly conference titled “What Is an Educated Person?” that is designed to hash out expectations of general-education curricula.
In addition, the state participates in a program of the Association of American Colleges and Universities, called Liberal Education and America’s Promise, which establishes broad goals for a liberal education and develops ways to assess student work. Statewide consensus on learning outcomes also may be easier to achieve in Utah than in other parts of the country because of the state’s relatively small size and its centralized system of public higher education.
Higher-education and business officials in Utah both see improving education as key to the state’s prosperity. Utah’s Board of Regents and the state’s higher-education commissioner set a goal this fall for Utah to have, by 2020, 55 percent of its adults with at least associate degrees and 11 percent with postsecondary certificates that lead to a livable wage. Only 39 percent of Utah residents have earned at least associate degrees, and 66 percent of all jobs in Utah are projected to require a postsecondary education by 2018.
“Education is the greatest single asset we have in society today,” Mark Bouchard, chairman of the Salt Lake City Chamber of Commerce’s effort to improve the education of Utah’s work force by 2020, said at this year’s “What Is an Educated Person?” meeting. More than a decade ago, when Goldman Sachs weighed whether to relocate some of its operations to Utah, the company’s real-estate committee led the process, evaluating prospects for facilities, said Mr. Bouchard, who is senior managing director of the Salt Lake City office of CB Richard Ellis, a global real-estate business.
Today, companies’ relocation committees are instead led by human-resources officials, he said. “They want to know about the quality of the work force.”
Curriculum Changes
The statewide discussions the Lumina project has fostered about how to ensure high-quality graduates have already spurred concrete changes in programs in the state, particularly here at Utah State, a land-grant university.
“The project has forced us to think about the structure of the curriculum and the way students move through it, and the ways of creating a much more intentional curriculum,” says Norman L. Jones, chairman of the university’s history department and a leader of the state’s participation in the Lumina project.
The statewide history team adopted general learning outcomes in areas of historical knowledge, historical thinking, and historical skills that largely mirror those laid out by the American Historical Association. The Utah team borrowed from those guidelines to spell out a set of specific skills and general competencies that students taking history courses should have, such as demonstrating the capacity to deal with differences in interpretation of history and being able to recognize and analyze change over time and space.
Using learning outcomes outlined through the Lumina project, Utah State’s history department took a closer look at how well students were performing. On a number of research skills, students weren’t doing as well as the department would have liked in upper-level capstone courses.
Looking closely at the deficiencies of students in those courses, faculty inserted basic-skills instruction earlier in students’ coursework. A Greek-history professor, for instance, began holding some class sessions at the campus library after she discovered that many students had never checked out a book there. She taught students about how knowledge is categorized and how to discern credible sources from less-reliable ones.
The department this year also began to require students who declare an interest in majoring in history to enroll in a pre-major program. That program is designed to set a clearer path of study and to better prepare students for advanced history courses. It directs students toward a set of broad-based humanities and social-science courses that are most relevant to building skills in the discipline, rather than leave the students to pick from a wider array of general-education courses.
The pre-major program is also designed to help the history department deal with budget pressures. Over the past two years, the department has lost close to one-third of its faculty, including many who took incentives to retire early. At the same time, the number of history majors continues to grow. The department hopes to winnow majors to people who are serious about the subject and not simply signing up as a default when they don’t get into other programs.
Across the state, participants in Utah’s Lumina project are now working on how to document and measure students’ progress in mastering the learning objectives that have been set through the project. One idea is to use electronic portfolios, a concept that has been adopted by a number of colleges across the country.
In Utah, Salt Lake Community College is pioneering the approach, in which students are required to create an online collection of their classwork. Students can use the portfolio to highlight signature assignments, write about their educational and career goals, talk about how the college’s learning outcomes are relevant to those goals, and demonstrate their progress over time.
The community college began to require students in general-education courses to create electronic portfolios in May. David Hubert, dean of general and developmental education at the college, says the institution began considering use of electronic portfolios in part because its accreditor had criticized it, in 2004, for establishing learning outcomes but not assessing them. He says he hopes the students’ portfolios also will help them demonstrate their skills and knowledge to potential employers, or to other colleges they may attend.
Expansion Plans
For all of the dialogue the Lumina project has facilitated in Utah and elsewhere about student learning, the process has its limits. The European origin of the idea has sometimes muddled the conversation, forcing participants to sort out Tuning’s complex history before understanding what parts are most relevant to U.S. higher education. The meetings the project requires are time-consuming. Skepticism among faculty who did not participate in the project remains a hurdle to putting agreed-upon learning objectives into practice on some campuses. And so far the project has reached only a handful of states and a limited number of disciplines.
In Europe, too, the Tuning process is still unfolding, with much about its bottom-line effect on quality remaining unclear. There, as in the United States, some disciplines have found it easier to forge consensus on learning outcomes than others. Chemistry, history, math, and physics have been among the most successful, says Tim Birtwistle, professor of law and policy of higher education at Leeds Law School at Leeds Metropolitan University, in Britain, and an expert on Tuning. But others, like archaeology, have proven difficult. Professors in that field, he said, have had a hard time even agreeing on whether it is an art, a science, a social science, history, or some combination.
Some of the process’s proponents opened a Tuning Academy this fall at the University of Groningen, in the Netherlands, and the University of Deusto, in Spain. The academy’s goal is to serve as a focal point for continuing Tuning discussions, and to further develop learning outcomes and plans for how to achieve them.
In the United States, Lumina is expanding its work. Texas and Kentucky have been added to the project, and the foundation plans to involve more states over time. It expects to extend the geographic boundaries of some of the discussions, facilitating consensus on learning goals in some subjects across regions and eventually across the nation. The foundation also plans to collect data to assess the project’s effect on learning, perhaps in collaboration with Europeans.
Supporters of the Lumina project say it holds the promise of turning educational assessment from a process that some academics might view as a threat into one that holds a solution, while also creating more-rigorous expectations for student learning.
Mr. Jones, the Utah State history-department chairman, recounted in an essay published in the American Historical Association’s Perspectives on History how he once blithely told an accreditation team that “historians do not measure their effectiveness in outcomes.”
But he has changed his mind. The Lumina project, and others, help define what learning is achieved in the process of earning a degree, he said, moving beyond Americans’ heavy reliance on the standardized student credit hour as the measure of an education.
“The demand for outcomes assessment should be seized as an opportunity for us to actually talk about the habits of mind our discipline needs to instill in our students,” Mr. Jones wrote. “It will do us a world of good, and it will save us from the spreadsheets of bureaucrats.”
It may not save students from the spreadsheets of professors like Mr. Jones, an advocate of the rubric.
But the faculty-produced grids give students a formula that bases grades on the ingredients of learning. And that shifts the conversation, Mr. Jones and his peers say, from what counts in a course to why it matters.
