The National Science Foundation says not enough grant applicants and peer reviewers are taking seriously — or even know about — an 11-year-old requirement that they evaluate the “broader impacts” of proposed studies.
So in a recent “Dear Colleague” letter, the agency reminds applicants and reviewers of the rule, which is meant to encourage researchers to pursue socially useful results. Those can include, for example, recruiting members of minority groups as scientists and improving the public’s understanding of science.
In 1997 the NSF simplified to two the agency’s principal criteria for the merit-based review of grant applications: intellectual merit and broader impacts. The NSF was reacting to pressure from Congress and the White House to show it was using federal tax dollars for more than just ivory-tower, fundamental research.
Ever since, many academic researchers have been perplexed and annoyed by the broader-impacts requirement, often called “Criterion 2.” Although the NSF has stressed that it gives equal weight to both criteria, some scientists have said that intellectual merit should count for more. They also point out that socially useful applications of fundamental research can be difficult to predict and take years to appear.
By some measures, compliance with Criterion 2 seems to be improving. Ninety-two percent of all external peer reviews commented on applications’ ideas for broader impacts in 2004, up from 84 percent in 2002. What’s more, the NSF has reported that in the 2005 fiscal year, the agency returned to applicants only 176 research-grant applications, out of more than 30,000, without reviewing them because their summary sections did not discuss broader impacts, as required.
More applicants ignored the NSF’s request for details in the separate project-description section of applications, but that is not grounds for rejection before review, said Luis Echegoyen, director of the agency’s division of chemistry, who wrote the letter on behalf of the entire agency. And even when grant applicants do elaborate about the broader impacts, some of the NSF’s external peer reviewers, who are drawn from the ranks of NSF grant recipients, remain silent on that part of the applications, he said.
The NSF has previously published examples of how, apart from future technological applications, scientists can describe the potential broader impacts of their work. The new letter pointedly calls for “rigorous” and “innovative” proposals. It says, for example, “a simple listing of outreach activities, or reference to inclusion of research personnel who are members of underrepresented groups, falls short of the rigor required to satisfactorily address this criterion.”
The NSF is expected to say more about Criterion 2 soon. Congress directed the agency last year to report by August on how it was carrying it out. —Jeffrey Brainard