Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Virtual Events
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    Hands-On Career Preparation
    An AI-Driven Work Force
    Alternative Pathways
Sign In
Publishing

The Journal That Couldn’t Stop Citing Itself

By Tom Bartlett September 23, 2015
In a four-paragraph editorial published in 2014, the “Journal of Criminal Justice” made 47 citations, all of other pieces that had appeared in the same publication.
In a four-paragraph editorial published in 2014, the “Journal of Criminal Justice” made 47 citations, all of other pieces that had appeared in the same publication.

The Journal of Criminal Justice has been on a roll. Once considered a somewhat middling publication — not in the same league as top journals like Criminology and Justice Quarterly — it is now ranked No. 1 in the field according to its impact factor, which measures the average number of citations a journal receives and is meant to indicate which titles are generating the most buzz.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

The Journal of Criminal Justice has been on a roll. Once considered a somewhat middling publication — not in the same league as top journals like Criminology and Justice Quarterly — it is now ranked No. 1 in the field according to its impact factor, which measures the average number of citations a journal receives and is meant to indicate which titles are generating the most buzz.

Rocketing to No. 1 is even more impressive when you find out that in 2012 the Journal of Criminal Justice was way back in 22nd place. That’s quite a leap!

Predictably, that sharp uptick made some researchers in a field devoted to misdeeds a tad, shall we say, suspicious. Among them was Thomas Baker, an assistant professor of criminal justice at the University of Central Florida. So Mr. Baker did what good researchers in all fields do: He took a hard look at the data. Then, after emailing it to a few friends, he decided to publish what he found in the field’s widely read newsletter, The Criminologist.

What he found was this: Much of the rise in the journal’s impact factor was due to citations in articles published in the Journal of Criminal Justice itself.

That impact factors can be gamed is news to no one who has paid any attention to academic publishing in the last, oh, couple of decades. For instance, in 2012 Thomson Reuters, which publishes the rankings, dropped 51 journals from its list for trying to artificially inflate their statuses. In one instance, several medical journals were banned after apparently colluding in a kind of you-cite-my-articles, I’ll-cite-yours arrangement.

Looking back over the last two years of articles published in the Journal of Criminal Justice, Mr. Baker noticed that many of the citations had appeared in editorials and articles written or co-written by Matt DeLisi, the editor in chief, who is a professor of criminal justice at Iowa State University. Of the 328 citations made to the journal in 2012 and 2013, 157 appeared in the pages of the journal itself, and 90 of those 157 papers had Mr. DeLisi’s name at the top.

In the most eyebrow-raising instance, one four-paragraph editorial, published in 2014, didn’t take up even a single page yet managed to have 47 citations, all to the Journal of Criminal Justice. Notably, all but three of the citations were from 2012 and 2013, the years used to calculate the most recent impact factor. (All three of the 2011 citations were to articles on which Mr. DeLisi was an author.)

Without those self-citations, the Journal of Criminal Justice would still have improved significantly under Mr. DeLisi’s stewardship, moving from 22nd place to 10th, according to Mr. Baker’s calculations. But it wouldn’t be leading the pack.

‘Questionable Editorial Decisions’

So the journal is clearly, brazenly gaming the system, right?

No, says Mr. DeLisi, who took over as editor in 2010. In an interview, he explains that he was citing research that was relevant to the articles, and nothing else. “If someone writes an editorial, of course there are going to be citations,” Mr. DeLisi says. His critics “seem to think that the only reason one does editorials is for citations,” he says.

ADVERTISEMENT

As for citing so many of the papers in his own journal, Mr. DeLisi says this was a way of further recognizing the authors, not pumping up the stats.

Mr. DeLisi notes that Eric Baumer, the editor of the Criminologist newsletter, is also a co-editor of the journal Criminology, which was knocked down a peg by the rise of the Journal of Criminal Justice. “There are all kinds of interesting conflicts of interest,” Mr. DeLisi says. He is now working on a written response to Mr. Baker’s piece.

What this will mean for the journal’s reputation is unclear. If Thomson Reuters decided the journal was, in fact, gaming its impact factor, it could be banned from the rankings. (A call to Thomson Reuters went unreturned.)

One well-known scholar, Bob Bursik, a professor of criminology and criminal justice at the University of Missouri at St. Louis, thinks the damage has already been done. “Intentional or not, in my mind, there is no question that the meteoric rise in the ranking of JCJ was due mostly to highly questionable editorial decisions,” he writes in an email. Those decisions, he says, render the impact-factor ranking “misleading and meaningless.”

ADVERTISEMENT

Skirmishes like this are not uncommon in the curious little world of academic research. What’s a bit unusual here is that Mr. Baker actually made the effort to crunch the numbers and figure out what was going on. At first he wasn’t so sure he wanted to disseminate what he had uncovered. He is, as he says, “very untenured,” and getting a reputation as a rabble-rouser might not be the best move. Some senior professors advised him against going forward. “I guess I’m strong-headed that way,” he says.

There’s a philosophical angle here, too. As editor, Mr. DeLisi has shifted the focus of the Journal of Criminal Justice from a more traditional approach to one that emphasizes biosocial factors — that is, looking at the environmental and biological influences that affect criminal behavior. There’s a divide in the field between those for and against a biosocial view.

But Mr. Baker is all for biosocial theory. He praises the journal’s change in focus and its tendency to publish pieces that would have trouble finding a home elsewhere. Several years ago, under Mr. DeLisi’s editorship, Mr. Baker even published an article in the Journal of Criminal Justice. “That’s the article of mine with the most citations,” he says. “Of course, some of those cites are from DeLisi.”

Tom Bartlett is a senior writer who covers science and other things. Follow him on Twitter @tebartl.

A version of this article appeared in the October 2, 2015, issue.
We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Scholarship & Research
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
About the Author
Tom Bartlett
Tom Bartlett is a senior writer who covers science and ideas. Follow him on Twitter @tebartl.
ADVERTISEMENT
ADVERTISEMENT

More News

Marva Johnson is set to take the helm of Florida A&M University this summer.
Leadership & governance
‘Surprising': A DeSantis-Backed Lobbyist Is Tapped to Lead Florida A&M
Students and community members protest outside of Coffman Memorial Union at the University of Minnesota in Minneapolis, on Tuesday, April 23, 2024.
Campus Activism
One Year After the Encampments, Campuses Are Quieter and Quicker to Stop Protests
Hoover-NBERValue-0516 002 B
Diminishing Returns
Why the College Premium Is Shrinking for Low-Income Students
Harvard University
'Deeply Unsettling'
Harvard’s Battle With Trump Escalates as Research Money Is Suddenly Canceled

From The Review

Glenn Loury in Providence, R.I. on May 7, 2024.
The Review | Conversation
Glenn Loury on the ‘Barbarians at the Gates’
By Evan Goldstein, Len Gutkin
Illustration showing a valedictorian speaker who's tassel is a vintage microphone
The Review | Opinion
A Graduation Speaker Gets Canceled
By Corey Robin
Illustration showing a stack of coins and a university building falling over
The Review | Opinion
Here’s What Congress’s Endowment-Tax Plan Might Cost Your College
By Phillip Levine

Upcoming Events

Ascendium_06-10-25_Plain.png
Views on College and Alternative Pathways
Coursera_06-17-25_Plain.png
AI and Microcredentials
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Virtual Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin