Editor’s Note: In a series of columns, “From Doctoral Study to …,” the Career Talk columnists explore nonfaculty options for Ph.D.s who want to work in higher education.
•••
C ampus assessment has been taking a bit of a beating lately. A recent essay in The Chronicle, “An Insider’s Take on Assessment: It May Be Worse Than You Thought,” reflected the widespread faculty resentment toward assessment. A fall 2017 article in Intersection, the journal of the Association for the Assessment of Learning in Higher Education, examined the “inherent” methodological flaws of assessment. And just a few weeks ago, The New York Times weighed in with an essay called “The Misguided Drive to Measure ‘Learning Outcomes.’”
So this might not seem the best of times to write about careers in campus assessment and evaluation programs. Yet the demands for accountability in higher education aren’t going to disappear, and neither are these programs. In fact, they offer a growing alternative career path for Ph.D.s and graduate students.
Some people confuse institutional research (IR), which we wrote about last summer, with assessment, but they are indeed two different, yet related, fields in academic administration. To learn more about the latter, we spoke with people in campus assessment offices about their backgrounds, their work, and how they are seeking to improve their relationships with skeptical, resistant faculty members.
Julie: First we asked our interviewees to explain the differences between IR and assessment. Jessica Sperling, head of evaluation and engagement at Duke University’s Social Science Research Institute, said IR offices tend to focus more centrally on institutional data — such as student enrollment, institutional or administrative trends, data reporting for accreditation and governmental entities — while assessment offices tend to concentrate on evaluating student learning and academic programs.
Not surprisingly, academics are far more resistant to assessment of their teaching. Faculty members see the assessment of individual teaching as something that’s ‘not meant to help you.’
Reporting and assessment requirements in higher education are likely to increase, not decrease. “I don’t see the requirement to evidence student learning going away anytime soon, as accreditors, governing bodies, boards, and other stakeholders continue to demand evidence that things are working,” said Diana Leilani Karafin, assistant vice provost for academic program review and assessment at New York University. “The issue is: how to respond to this in a way that avoids wasteful, bureaucratic paper-shuffling.”
Jenny: She and other administrators we interviewed say they are well aware of the faculty perception of assessment as a waste of time and energy. But, they noted, assessment comes in many forms, some of which are welcomed by professors and some not.
Julie: For example, Sperling at Duke said her office does not assess what is going on in the classroom. Instead it evaluates co-curricular programs — for example, trying to understand whether an internship program is successful. Faculty want such programs to succeed, she said, and “we can work collaboratively to grow and improve a program through evidence-based practice. Why are you effective? Why not? For whom are you effective? You don’t have to be perfect. In fact, being imperfect allows you to see how you can get better.”
Not surprisingly, academics are far more resistant to assessment of their teaching. “My sense,” she says, is that faculty members see the assessment of individual teaching as something that’s “not meant to help you. They feel it’s not constructive, it’s forced on you, and it’s not a good indicator of what happens in the classroom.”
Jenny: At NYU, Karafin has encountered just such skepticism, as she seeks to help faculty members evaluate what is going on in their classrooms. Her approach is to take the time to have a conversation and try to understand their perspective and clear up misunderstandings. Sometimes a solution can be devised that meets faculty needs and complies with institutional policies.
“We ask what they would find useful (to know) about their students, the curriculum, how students are performing,” Karafin said, “and of course there are lots of things faculty want to know. We help them find ways to pursue those questions in an assessment framework, and to devise simple strategies that meet our reporting requirements.” That can be as simple as providing a copy of the minutes of a meeting where faculty members reviewed and discussed the assessment findings, rather than asking them to submit documentation that requires additional work.
Julie: Sometimes faculty members are already doing assessment work but do not know how to translate their results “for the purposes of reporting, and they don’t have the time to figure it out,” Karafin said. In such cases, her office looks for a solution “that works for them and for us. We emphasize flexibility with reporting style and seek to avoid a standardized approach. We encourage faculty to take an approach that makes sense for their discipline and their department.”
Jenny: Drew Allen, founding executive director of the Initiative for Data Exploration and Analytics (IDEAS) for Higher Education, at Princeton University said it’s important to take the time to understand faculty resistance because the reasons for it vary and many concerns are justified.
Some academics, he said, “are resistant to the idea of evaluation/assessment, some are concerned about how findings will be interpreted/used (and how it will affect their work), and others are more resistant because of the process (i.e., lack of transparency, inclusion, etc.).” He looks for genuinely meaningful ways to involve faculty members, from the initial planning through the analysis and communication of findings. “Faculty can add incredible insight and value to the process, especially if there’s buy-in and trust.”
Julie: The narrative about assessment should be changed, Allen said, from “someone is looking over my shoulder” to a more collaborative approach that shows how it will benefit students, faculty, and the institution. “Keeping a clear eye on the ultimate goal, while admitting that not everything about the process will make all stakeholders happy all the time, can sometimes help,” he said.
Jenny: The administrators we interviewed obviously believe in the value of their work. How did their graduate studies lead them to a career in assessment? For some, it grew out of a longstanding interest in applied research. Many got started in the field working part-time in assessment offices while in graduate school.
Jaclyn Kelly, a doctoral student in political science at the City University of New York Graduate Center and a senior research associate at its Center for Urban Research, said: “I got into this field by acting as a research assistant to professors as an undergraduate, and then as a young professional. You can’t be picky about what you’re going to work on, and you definitely get the grunt work, but having the opportunity to watch and work with experts in an apprentice-like role was extremely helpful to my skill development.”
Julie: For others, assessment and evaluation emerged as a compelling alternative to the faculty track. “I was one year into my first tenure-track position out of grad school,” said Karafin of NYU, “and I had a sinking feeling as the year passed that this wasn’t the right choice for me. By midway through Year 2, I knew I had to move on. I felt really guilty because of the tremendous support my advisers had given me, the doors they helped open for me, and all the work and time I had dedicated to get to that point. I came to peace with the decision and recognized I had to pursue a path that would be fulfilling for me.”
She spotted an opening at NYU for an assistant director of academic assessment — to establish a system to track, grow, and improve program-level student learning outcomes. “I was intrigued,” she said. “I thought my background in quantitative and qualitative social-science research methods might be a great fit.”
Jenny: For those in doctoral programs interested in building career options in this field, our interviewees recommended a few things:
- Seek strong training in a variety of research methods.
- Take an eclectic approach to the questions you ask.
- Conduct informational interviews.
- Take advantage of short-term and temporary roles. (Graduate school is a great time to explore, Sperling noted, “even if doing so slows you down a little bit.”)
- Get to know the assessment literature, both scholarly and applied, as well as the various studies on whether students are getting much out of college, and the critiques of those studies.
Julie: Where to start if you are interested in campus assessment careers? All of our interviewees recommended the American Evaluation Association as an excellent source of information, and suggested attending one of its conferences. Other sources include: the National Institute for Learning Outcomes Assessment; the assessment work of the Association of American Colleges & Universities; the Association for the Assessment of Learning in Higher Education; the Assess Listserv; the National Institute for Learning Outcomes Assessment; the regional accreditation conferences; and the Association for Institutional Research and its regional branches.
Jenny: The administrators we interviewed are now in a position to hire so we asked them what they look for in candidates. Princeton’s Allen said he doesn’t always look for a specific skill set but rather for someone who is curious, has a solid grounding in the research process, and is a good writer. “So much of the evaluation process depends on thoughtful and clear communication,” Allen said. “Demonstrating your writing skills in your cover letter goes a long way.”
Whether he looks for specific training depends on the project — certain statistical skills for quantitative work, for example, or “experience conducting interviews, focus groups, and observational skills” for qualitative projects.
Job opportunities in assessment and evaluation are available at all types of colleges and universities, in both the nonprofit and for-profit sectors. Generally there are two kinds of positions: (1) practitioner-based (working directly with faculty members on assessment) and (2) leadership-oriented (overseeing an assessment system and being responsible for reporting and compliance).
Asked about the hiring process in assessment, Marie Burrage, director of program initiatives and assessment in the dean’s office at Boston University’s business school, said there were usually several interview stages, one of which includes a presentation. That bodes well for Ph.D.s and graduate students interested in this expanding career field, given their experience teaching and presenting.
Julie: Another common question candidates have is about the opportunities for advancement. That tends to be very specific to each institution and to the candidates. At NYU, Karafin was able to move from an assistant-director position to director, and then assistant vice provost within the same institution. Of her experience, she noted, “Learning how to grow an assessment strategy within an institution, how to operate with all the politics, how to manage and use data, how to gain credibility and success, and build a system that works are all great experiences for someone interested in leadership in higher ed.”
Jenny: In a very difficult tenure-track market, assessment and evaluation offers Ph.D.s a path to pursue careers that involve research, analysis, teamwork, and communication — skills and abilities they already have, to some degree. Institutions facing increased external and internal demands for information, Karafin said, need “innovators and thinkers who can help devise strategies that are useful.”
Jennifer S. Furlong is director of the office of career planning and professional development at the Graduate Center of the City University of New York. Julie Miller Vick retired as senior career adviser of career services at the University of Pennsylvania. With Rosanne Lurie they are the authors of The Academic Job Search Handbook (University of Pennsylvania Press), 5th edition.
