To Track a Predator: How the Police Identify Threats on Social Media

John Locher, Associated Press

Last year James Fitch, chief of the campus police at North Central Texas College, patrolled several buildings during an emergency-alert test to see how faculty members and students responded. Later that day his office noticed that a student had posted a warning on Facebook: A man was meandering through a campus hallway, the student reported, with a loaded gun.

She was actually describing Mr. Fitch. Though no actual threat had been posed, the incident resonated with him.

“Instead of calling 911, instead of notifying the instructors, instead of doing anything, she posted on Facebook,” Mr. Fitch says.

That’s when he realized the integral role social media could play in campus safety. In the past year dozens of colleges have faced online messages that appear to threaten violence, especially after a shooter at Umpqua Community College killed 10 people in October. The attack was preceded by a vague post on 4chan, an anonymous message board, warning students in the Northwest: “Don’t go to school tomorrow.”

It later became clear that the shooter, Chris Harper-Mercer, was not responsible for the post. But online threats remain matters of deep concern. And many students use social media to react to and discuss those threats — including after the most recent campus shooting, when a former student killed a professor, then himself, last week at the University of California at Los Angeles. (The shooter, Mainak Sarkar, had previously aired grievances and disparaged his victim, his mentor William S. Klug, on a blog.)

Finding and analyzing social-media messages is a relatively new but constant duty for campus police departments. It’s often challenging because of the vast amount of information — most of it irrelevant — that must be weeded through. Since so much of college students’ lives takes place on social media, the police recognize that they need to have an established presence there. But they are still deciding how best to screen and analyze the wealth of data. Many departments are trying to manage the work in-house, but others are turning to outside companies for help.

‘Extra Manpower’

Mr. Fitch decided on the latter option. He signed a contract with Social Sentinel, one of several companies that have popped up in recent years to monitor social media for threats. The company scans 850 million public posts a day from sites like Twitter, Facebook, Tumblr, and YouTube, to isolate potential threats directed toward their clients, says Gary Margolis, the company’s president. Social Sentinel also works with private companies, schools, and government agencies. Colleges and universities make up 35 percent of the company’s clientele, says Mr. Margolis.

Social Sentinel’s software sifts through the posts for terms cataloged in the company’s “library of harm,” a database of coded keywords, patterns, and phrases previously identified as threatening by linguistics researchers. (Mr. Margolis says he employs a librarian to continually study the language of harm on the internet and update the system.) After flagging a suspect post, the software reads it a second time to determine its context. If the post still seems threatening in context — if it seems to refer to a real shooting as opposed to, say, a golf shot — the company sends it to the client.

Mr. Fitch has chosen to separate his alerts: Posts flagging the use of alcohol or narcotics get routed to his email. “Otherwise it would probably just wear my phone out,” he says. Urgent notifications come as texts, and Mr. Fitch gets roughly eight to 10 per week.

At first the department was flooded with erroneous alerts because North Central’s abbreviation, NCTC, happens to be the same as that of the National Counterterrorism Center, an institution beset by online harassment, says Mr. Fitch. But Social Sentinel filtered out those instances, and the number of false positives dropped significantly. Now Mr. Fitch tailors the software by entering the names of faculty members and events, like graduation, into the system, to predict scenarios that might draw increased threats.

Mr. Fitch has yet to receive a post threatening mass violence. But he was once alerted to a female student’s complaining on Twitter that she was being stalked. Based on the tweets, he located the student, told officers nearby to keep an eye on her, and attempted to identify the man in question.

North Central Texas College has five campuses that stretch across four counties, so being able to assess so many posts, regardless of their location, is “like having extra manpower,” says Mr. Fitch.

The University of California at Davis also decided during the last year to use Social Sentinel to screen social media because of the sheer volume of data it had to wade through, says Matthew Carmichael, the university’s police chief. Previously the department assigned an employee to track activity online, but it proved impossible for one person to review every channel. The police did not want to risk missing key information, Mr. Carmichael says.

Both he and Mr. Margolis describe public-media sifting as a tool — a data point — that feeds into systems already in place on college campuses. When officials are notified of a potentially troubling post, it’s up to them to investigate the claim, establish its credibility, and take action if necessary.

“Our job is to give you something that you can look at and go, Oh yeah, we probably need to look into this,” Mr. Margolis says.

Constant Collaboration

Many institutions don’t use social-media-scanning software. Temple University, for example, relies on constant collaboration with local and federal watchdogs to combat online threats, says Charles Leone, Temple’s executive director of campus-safety services.

When a threat against colleges in the Philadelphia region was posted on 4chan last October, Temple was alerted to it by the Bureau of Alcohol, Tobacco, Firearms, and Explosives. Campus police officials then consulted with surrounding institutions like Drexel University and the University of Pennsylvania, the local police, the Federal Bureau of Investigation, and other federal agents to gauge the potential danger, says Mr. Leone.

Temple’s police department has a team of eight detectives who assess threats by deciphering the credibility of the language, considering the context of the message, and looking for similar posts elsewhere, says Mr. Leone. If a threat is deemed credible, the police involve computer experts and seek subpoenas for social-media sites to gather more information, he says.

Aside from the detective team, Temple’s police department has roughly 130 sworn officers and 300 other employees patrolling and monitoring the campus, says Mr. Leone. The department also engages with students on Twitter and maintains an anonymous tip line.

Mr. Leone says he does not see a need to turn to an outside company for social-media monitoring. That technology is still new, he says, and it may need more work before it can distill threats effectively from the enormous pool of online chatter.

“You’re talking about quite a bit of data coming in,” Mr. Leone says. “We still have to find out the best ways to manage that.”

As colleges and universities reckon with the real hazards virtual platforms can pose to campus safety, they also appreciate the insight those platforms can provide into the lives of students. Mr. Carmichael says he does not scan social media just for active-shooter scenarios. He also wants to be clued in to the mental-health climate on the campus, especially to the harm students often inflict on themselves.

Two weeks after setting up Social Sentinel’s software, the UC-Davis police were notified that a student was posting online about harming himself. From those posts, public-safety officials located the student that day and offered him mental-health resources.

Before the internet, cries for help were “letters or maybe a comment made to somebody in passing,” says Mr. Carmichael. Now social media fulfills that role, he says, and public-safety officials should view it as another resource to make campuses safer.