Skip to content
ADVERTISEMENT
Sign In
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
  • More
  • Sections
    • News
    • Advice
    • The Review
  • Topics
    • Data
    • Diversity, Equity, & Inclusion
    • Finance & Operations
    • International
    • Leadership & Governance
    • Teaching & Learning
    • Scholarship & Research
    • Student Success
    • Technology
    • Transitions
    • The Workplace
  • Magazine
    • Current Issue
    • Special Issues
    • Podcast: College Matters from The Chronicle
  • Newsletters
  • Events
    • Virtual Events
    • Chronicle On-The-Road
    • Professional Development
  • Ask Chron
  • Store
    • Featured Products
    • Reports
    • Data
    • Collections
    • Back Issues
  • Jobs
    • Find a Job
    • Post a Job
    • Professional Development
    • Career Resources
    • Virtual Career Fair
    Upcoming Events:
    College Advising
    Serving Higher Ed
    Chronicle Festival 2025
Sign In
A Post-Truth Syllabus

The Fine Art of Sniffing Out Crappy Science

By Steve Kolowich January 16, 2017
Carl T. Bergstrom (left) and Jevin West, of the U. of Washington, want to teach students how to survive the avalanche of false or misleading data shaken loose by shifts in media, technology, and politics.
Carl T. Bergstrom (left) and Jevin West, of the U. of Washington, want to teach students how to survive the avalanche of false or misleading data shaken loose by shifts in media, technology, and politics.Stephen Brashear

Facts and figures are like cow pastures. Unless you squint, you can’t always tell how full of bullshit they are.

Carl T. Bergstrom and Jevin West, a pair of scientists at the University of Washington, think it’s time to arm students with boots and shovels. They have published the outline of a course, titled “Calling Bullshit,” which would try to teach how to spot bad data and misleading graphs at a time when bending statistics has become a popular art form.

To continue reading for FREE, please sign in.

Sign In

Or subscribe now to read with unlimited access for as low as $10/month.

Don’t have an account? Sign up now.

A free account provides you access to a limited number of free articles each month, plus newsletters, job postings, salary data, and exclusive store discounts.

Sign Up

Carl T. Bergstrom (left) and Jevin West, of the U. of Washington, want to teach students how to survive the avalanche of false or misleading data shaken loose by shifts in media, technology, and politics.
Carl T. Bergstrom (left) and Jevin West, of the U. of Washington, want to teach students how to survive the avalanche of false or misleading data shaken loose by shifts in media, technology, and politics.Stephen Brashear

Facts and figures are like cow pastures. Unless you squint, you can’t always tell how full of bullshit they are.

Carl T. Bergstrom and Jevin West, a pair of scientists at the University of Washington, think it’s time to arm students with boots and shovels. They have published the outline of a course, titled “Calling Bullshit,” which would try to teach how to spot bad data and misleading graphs at a time when bending statistics has become a popular art form.

“Pending approval from the administrative powers-that-be at the University of Washington, we hope to offer the seminar in the near future,” they wrote on a website they built for the course. “In the meantime, connoisseurs of bullshit may enjoy the course syllabus, readings, and case studies that we have lovingly curated.”

The Chronicle caught up with Mr. Bergstrom, a biologist, and Mr. West, an information scientist, to talk about their course. The following interview has been edited and condensed.

Q. In the wake of the election season, your course and its title arrive with a sense of urgency. But you don’t just implicate politicians in the spreading of bullshit. You also call out advertisers, the techno-optimists of start-up culture, and administrators of all stripes. How are you defining bullshit?

Carl Bergstrom: There’s a lot of discussion of how to define bullshit in the philosophy literature, rather to one’s surprise. Harry Frankfurt defines it as producing verbiage without concern whatsoever for the truth. And that’s bullshitting, according to him. There are others who argue bullshit is broader than that, and would include deliberate attempts to deceive.

What we’ll do in the course is teach people a set of tools and techniques for assessing quantitative evidence that will allow them to make claims about that evidence that will generally be accepted by the scientific community as valid.

How we define it is: “language, statistical figures, graphics, and other forms of presentation intended to persuade by impressing and overwhelming a reader or listener with a blatant disregard for truth and logical coherence.”

Q. So something that has that seductive, kind of musical quality of truth but which is unreliable when subjected to scrutiny.

ADVERTISEMENT

Mr. Bergstrom: Yes, or kind of impressing you and making you feel like you cannot challenge it because I dump a bunch of numbers at you and say, Look, here are the statistics, and we ran the latest machine-learning algorithm on it, and here’s a fancy data visualization.

Q. The theater of rigor.

Mr. Bergstrom: Yes, exactly. And that could be done verbally, but more and more we see it done quantitatively with figures, data graphics, and with appeal to algorithms that generate results but which no one really understands.

Q. Bullshit is, I think we can agree, both pervasive and often cleverly disguised. How do we know this class is not also bullshit?

ADVERTISEMENT

Mr. Bergstrom: That’s a good question … [long pause] ... Do you have a witty, snap answer to that, Jevin?

Jevin West: After the students take the course, hopefully they’ll be able to assess whether that’s true or not. The class isn’t a set of lectures only produced from our brains. We’re using readings and real material that you find in texts that are about this very topic. We’re aggregating for the students this kind of information that’s around and sort of gluing it together, so hopefully by the end they could assess that. But it’s a great question.

Mr. Bergstrom: And, you know, unfortunately, good bullshit might do all of the things that Jevin just mentioned. I guess I would say, ‘Trust us.’

Q. Well, that’s ultimately what so much of it comes down to, right? Politicians are successful, as we’ve recently seen, not because the rigor of their arguments checks out according to some refereed system of bullshit identification, but because enough people trust them that whatever data and data graphics and extras they’re bringing to the table amount to something credible.

ADVERTISEMENT

Mr. Bergstrom: That’s right. Let me give you a serious answer to the question of whether the course is bullshit: What we’ll do in the course is teach people a set of tools and techniques for assessing quantitative evidence that will allow them to make claims about that evidence that will generally be accepted by the scientific community as valid.

Truth Issue
The Post-Truth Issue
“Post-truth—adjective; relating to or denoting circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief.” So says Oxford Dictionaries, announcing their 2016 word of the year. If we really have entered a post-truth era, as so many have written, what does that mean for the scholar and the student? For the citizen and the state? In our special issue, we wrestle with these and other urgent questions.
  • Facebook and Falsehood
  • Google and the Misinformed Public
  • Our Graduates Are Rubes
  • When Truth Becomes a Commodity
  • The Plots Against America
  • Fighting Fake

Mr. West:
Just by putting students in a room addressing the topic — even if we were bullshitting them with the lectures and the content that we give them — just by having them talk about this three hours a week, that’s putting their antennas up. And they can hone those skills for themselves going forward.

Q. I went to college not too long ago, and I remember learning skills for identifying bullshit. I also remember learning how to bullshit. And I’m sure as professors you’ve read papers, possibly submitted at the 11th hour ...

Mr. Bergstrom: No, we’ve never seen anything like that. [They laugh.]

ADVERTISEMENT

Q. … and said, ‘This student is bullshitting right now.’ And then, sure enough, you take out your grading pen and say, ‘But, they bullshitted well enough to pass this paper.’ Do you think that colleges are, at some level, complicit in teaching students how to become effective bullshitters?

Mr. Bergstrom: I think that what we’re teaching in the classroom, at least in the sciences, is how to cut through numerical bullshit of various sorts. But I do concede that the assessment structure may favor those who practice the art on their own time.

Q. Strictly speaking, bullshit — like, literal bull shit — can be used as a fertilizer to grow things that are necessary for survival. Similarly, political bullshit can be used to inspire good acts. Start-up bullshit can help raise money for useful products. Administrative bullshit, um … can, uh …

Mr. Bergstrom: It must be useful for something. [They laugh.]

ADVERTISEMENT

Mr. West: Maybe to generate funds to build a new building or something.

Q. Yeah, OK, there you go. So, is bullshit always bad?

Mr. West: Since our focus will be on number or algorithmic or graphic manipulation, I think in that case it is all bad. What you’re talking about has more of a social context, and I think in that case, you’ve couched it pretty well — maybe there are circumstances where it’s OK.

Mr. Bergstrom: You have to distinguish a little bit between the types of bullshit you’re talking about. I mean, we’ve got to repeatedly, every day, say, ‘Hey it’s great to see you,’ even when we’re thinking, ‘Oh God, I wish I’d ducked around the corner.’ So, there are these white-lie, minor acts of bullshit that are essential to lubricate social function.

ADVERTISEMENT

Q. Is there such a thing as a white lie in statistical bullshit?

Mr. Bergstrom: I don’t believe so. I believe the whole purpose of statistics is to impose a sort of rigor on messy data and make strong claims, or formally specify how strong of claims we can make, given the data. Some people will say, ‘Well, the statistics I used was actually not sufficiently conservative and so my confidence interval is probably a little broader — but, it’s close enough.’ And I really dislike that.

Q. The full title of your course is “Calling Bullshit in the Age of Big Data.” How does big data exacerbate the problem, and in what ways can it be used as an antidote?

Mr. Bergstrom: In the past, you had very small data sets, because that’s all you could get. If you had a hypothesis that was wrong, and you tested it, the data would tell you you’re on the wrong track. But once you’re looking at a data set with a hundred million observations, almost anything you test is going to come out to be statistically significant. You think you’ve found this strong signal, and it turns out that it’s all the result of some spurious correlation in the data.

I want to challenge myself to think well of actors who have put together bullshit, rather than saying, ‘Oh, this person is working for this agency I don’t like, so they’re probably trying to deceive me.’

Mr. West: In my field we spend a lot of time building software tools, visualization tools, algorithmic tools. So more and more of the population is engaging with this kind of data. More and more beautiful graphs are being produced. Which is great! But with that come lots and lots of mistakes in arguments and the way they’re presenting the data. We see more and more of this stuff, and more kinds of bullshit in graphical representation and in numbers and stats.

ADVERTISEMENT

Q. In my experience, calling bullshit often will cause other people call you cynical. Cynicism isn’t necessarily a bad trait to cultivate, but is there a way that calling bullshit can go wrong?

Mr. Bergstrom: I expect there’s all kinds of ways that calling bullshit can go wrong, and I expect we’ll find out most of them pretty soon. One thing I do want to think about is that I want to be very good-spirited in everything I put together in the class. I want to challenge myself to think well of actors who have put together bullshit, rather than saying, “Oh, this person is working for this agency I don’t like, so they’re probably trying to deceive me.”

I think the effort to humanize the text you’re looking at is very important. We often do too little of that in the academy, and people are very quick to say, ‘Oh these people are just stupid,’ ‘Oh, that discipline is garbage.’ There’s a difference between being a hard-minded skeptic and being a domineering jerk.

Q. So alongside the tools students can use to identify and call out bullshit, you’re also teaching the discretion to know when it’s appropriate to do so?

ADVERTISEMENT

Mr. Bergstrom: It’s very, very important in interdisciplinary work. Because when you sit down and listen to somebody from another field, they are bringing a whole different set of assumptions, possibly a different jargon. You’re thinking, “None of this can be right — this is bullshit!” And then you realize, “No, what they’re saying is completely consistent. They’re just working off different assumptions than the ones you were making.”

It’s very important to have that generosity of spirit … [laughs] at least, until you’re pretty sure it’s not merited.

Mr. West: We’ll be making fun of some of the issues we’ve had in our own work and our research, just because we know humans are fallible.

Mr. Bergstrom: My first case study attacks my own textbook. Just to say, “There’s bullshit in all kinds of forms, and here, I put some in my textbook, and it’s interesting to look at how that happened to me.”

ADVERTISEMENT

Something I think I need to work on is learning how to call bullshit on a claim without attacking a person. I’d love to get better at that, and help other people learn how to do that as well.

Steve Kolowich writes about how colleges are changing, and staying the same, in the digital age. Follow him on Twitter @stevekolowich, or write to him at steve.kolowich@chronicle.com.

We welcome your thoughts and questions about this article. Please email the editors or submit a letter for publication.
Tags
Teaching & Learning
Share
  • Twitter
  • LinkedIn
  • Facebook
  • Email
About the Author
Steve Kolowich
Steve Kolowich was a senior reporter for The Chronicle of Higher Education. He wrote about extraordinary people in ordinary times, and ordinary people in extraordinary times.
ADVERTISEMENT
ADVERTISEMENT

Related Content

Anatomy of a Hoax

More News

Illustration showing a letter from the South Carolina Secretary of State over a photo of the Bob Jones University campus.
Missing Files
Apparent Paperwork Error Threatens Bob Jones U.'s Legal Standing in South Carolina
Pro-Palestinian student protesters demonstrate outside Barnard College in New York on February 27, 2025, the morning after pro-Palestinian student protesters stormed a Barnard College building to protest the expulsion last month of two students who interrupted a university class on Israel. (Photo by TIMOTHY A. CLARY / AFP) (Photo by TIMOTHY A. CLARY/AFP via Getty Images)
Campus Activism
A College Vows to Stop Engaging With Some Student Activists to Settle a Lawsuit Brought by Jewish Students
LeeNIHGhosting-0709
Stuck in limbo
The Scientists Who Got Ghosted by the NIH
Protesters attend a demonstration in support of Palestinian activist Mahmoud Khalil, March 10, 2025, in New York.
First-Amendment Rights
Noncitizen Professors Testify About Chilling Effect of Others’ Detentions

From The Review

Vector illustration of a suited man with a pair of scissors for a tie and an American flag button on his lapel.
The Review | Opinion
A Damaging Endowment Tax Crosses the Finish Line
By Phillip Levine
University of Virginia President Jim Ryan keeps his emotions in check during a news conference, Monday, Nov. 14, 2022 in Charlottesville. Va. Authorities say three people have been killed and two others were wounded in a shooting at the University of Virginia and a student is in custody. (AP Photo/Steve Helber)
The Review | Opinion
Jim Ryan’s Resignation Is a Warning
By Robert Zaretsky
Photo-based illustration depicting a close-up image of a mouth of a young woman with the letter A over the lips and grades in the background
The Review | Opinion
When Students Want You to Change Their Grades
By James K. Beggan

Upcoming Events

07-31-Turbulent-Workday_assets v2_Plain.png
Keeping Your Institution Moving Forward in Turbulent Times
Ascendium_Housing_Plain.png
What It Really Takes to Serve Students’ Basic Needs: Housing
Lead With Insight
  • Explore Content
    • Latest News
    • Newsletters
    • Letters
    • Free Reports and Guides
    • Professional Development
    • Events
    • Chronicle Store
    • Chronicle Intelligence
    • Jobs in Higher Education
    • Post a Job
  • Know The Chronicle
    • About Us
    • Vision, Mission, Values
    • DEI at The Chronicle
    • Write for Us
    • Work at The Chronicle
    • Our Reporting Process
    • Advertise With Us
    • Brand Studio
    • Accessibility Statement
  • Account and Access
    • Manage Your Account
    • Manage Newsletters
    • Individual Subscriptions
    • Group and Institutional Access
    • Subscription & Account FAQ
  • Get Support
    • Contact Us
    • Reprints & Permissions
    • User Agreement
    • Terms and Conditions
    • Privacy Policy
    • California Privacy Policy
    • Do Not Sell My Personal Information
1255 23rd Street, N.W. Washington, D.C. 20037
© 2025 The Chronicle of Higher Education
The Chronicle of Higher Education is academe’s most trusted resource for independent journalism, career development, and forward-looking intelligence. Our readers lead, teach, learn, and innovate with insights from The Chronicle.
Follow Us
  • twitter
  • instagram
  • youtube
  • facebook
  • linkedin