Opening the Black Box: Analytics and Admissions

In a guest post today, Chris Peterson describes how technology is changing the admissions process. Mr. Peterson, who directs digital strategy and communications for the Massachusetts Institute of Technology’s admissions office, is a researcher at MIT’s Center for Civic Media.

One morning, shortly before we released admissions decisions for the Class of 2016, I received an e-mail from an applicant.

“No one from MIT checked my link included in the application,” it read. “I just checked my Google Analytics account. No visits from Boston [or] Cambridge. I am sure that I have been rejected. Feeling hopeless and helpless.”

Every year an increasing proportion of our increasing applications contain a link to some digital supplement: a project tumblr, a YouTube video, a Flickr album of artwork. The contents of those supplements often round out the student, adding dimensions that our very flat applications lack. But while we gladly accept the supplements because of the insight they add to the applicant, the analytics that often come embedded in the supplements also add insight into our process.

As admissions officers, we are accustomed to reading applications; now, applications are reading us.

Suppose applicant Alice creates a Web site to accompany her college application. It contains pictures and videos from her two chief extracurricular activities: figure skating and First Robotics. She publishes the Web site on a freely available platform such as WordPress, and includes, perhaps by default, a freely available snippet of Google Analytics in her code.

At Whatsamatta U., Oliver the admissions officer opens Alice’s application. He reads her excellent essays, sees her superb scores, and loves her letters of recommendation. Intrigued, he accesses her Web site and spends some time clicking around, looking at photos, watching videos, and reading her posts.

By doing so, Oliver sends signals to Alice—signals recorded by the Web analytics. She can see that someone visited her Web site, from an IP address associated with a given geographic location (be it Cambridge or Cincinnati), and spent so much time on each page of her site. She can tell if someone visited her site briefly, or if there was a flurry of activity, perhaps from different computers clustered in the same geographic area in the weeks leading up to decision day, perhaps when admissions committees are meeting. Or, more depressingly, she can see if no one visited her site at all.

Alice will inevitably draw inferences from her analytics. Her inferences may be accurate or inaccurate, precise or imprecise, but she will still draw them. By doing so, she will develop indirect insights into her progress through the process, like a spy peeking around a corner with a pocket mirror. The picture she sees may be distorted, even incorrect, but it will still guide her subsequent actions and reactions.

Oliver, like all of us, has not had to deal with this before. For a long time admissions was a black box, the opacity of which shielded the ambiguities of the process. All admissions officers know that some applicants are competitive and some are not; some will have more time and effort expended on them, and others less. But the applicants never had any way of knowing where they fell on this spectrum—until now.

The problem, from the admissions perspective, is that there’s nothing you can do to avoid sending such signals to applicants equipped with analytics. Click only the interesting links, and you accurately inform applicants whether or not you find those links interesting. Click all the links, and you still end up signaling based on frequency and time spent. Don’t click any links, and you lose the truly important information contained in some supplements. Pretend not to click any links (there are technical methods to block or obfuscate some, though not all, analytics), and you send misleading signals of cold indifference to everyone.

No matter what you do, you end up sending signals; there is no neutral stance to take.

We can’t turn back the technological clock. We aren’t driving this train—we’re just along for the ride. During the current admissions cycle, I’ve seen more messages than ever before from applicants stressing out about the signals they think we’ve sent.

Last year a colleague at another college was threatened with a lawsuit by an angry parent. The father had hired a consultant to develop and track an online supplement for his child, and was upset by what he perceived as insufficient attention paid to the site according to his (incorrect) interpretation of the analytics.

More threats of that kind could shift policies not to what best serves the best applicants but to what minimizes liability to admissions offices. Sunlight may be the best disinfectant, but, in excess, it burns.

So what do we do? There are no easy answers. Admissions offices can no longer choose to not send signals. Instead, they must decide which signals they want to send. They will need to be prepared to explain and justify those signals to their applicants, who are now more like constituents, armed with unprecedented insight into the processes that decide their fate.

Without realizing it, as a profession we’ve long been rehearsing safely backstage. Now we must step out into the light.

Return to Top