Share Tweet Share Email

Science or bunk: How to tell the difference

January 18, 2011

by Suzanne E. Willis, NIU Department of Physics

Suzanne E. Willis

Suzanne E. Willis

In class one day, I was discussing pseudoscience and the fact that it can be a waste of taxpayers’ hard-earned money, when a student asked, “Why do you care if we waste our money?”

Well, I do care. We all should. If society wastes money on junk science, we all lose. The public is fed misleading information, or worse, misinformation. Scientific projects with great potential might go untested in favor of the false promises of junk science. And sometimes pseudoscience can actually be harmful.

I was reminded of all this recently after reading a widely reported story on a 1998 study purporting to find a relationship between the MMR (measles/mumps/rubella) vaccine and autism. Turns out, the study was not only fatally flawed, it was actually fraudulent. But some parents took it to heart and did not vaccinate their children to protect them from sometimes fatal childhood diseases such as measles and whooping cough, which have seen a resurgence.

In the media nowadays, we’re hit with a constant barrage of scientific studies, and the case of the MMR vaccine underscores the importance of training students to be critical thinkers who can recognize the real thing from a fake.

So, how can a non-scientist figure out what’s what? It’s good to first remember that science usually follows a process:

  • Get curious.
  • Ask a question.
  • Decide how to figure out an answer.
  • Do an experiment. Make sure you understand what you are doing.
  • Figure out what the results of the experiment mean. Have you answered your question? If you made a hypothesis, was it confirmed or refuted?
  • Share your results with other people. Can they replicate your experiment? Do they get the same results?
  • Keep on going.

Pseudoscience, on the other hand, often sounds “scientific” but is missing one or more key ingredients, such as thoughtful testing by experienced scientists, peer review by other experienced scientists, or consistency with known scientific laws and facts. The more missing ingredients, the more skeptical one should become—and the tighter you should grasp your wallet.

Robert L. Park, a physics professor and author of “Voodoo Science: The Road From Foolishness to Fraud,” wrote of the seven “warning signs of bogus science” in The Chronicle of Higher Education. I adapt his warning signs here:

  • The discoverer pitches his or her claim directly to the media, bypassing scientific review. Particularly beware of paid infomercials.
  • A powerful establishment is supposedly trying to suppress the work.
  • The scientific effect is at the limit of detection (and stays there despite decades of improvement in scientific methods).
  • Evidence for a discovery is anecdotal. The plural of “anecdote” is not “data”; data need to be collected in a thoughtful, reproducible way.
  • The discoverer claims a belief is credible because it has endured for centuries.
  • The claim requires new laws of nature, often in serious contradiction to well-established knowledge.
  • The person making the claim is trying to sell you something.

I’ll add a few recommendations of my own as well. First, see what other scientists are saying. Go to original sources as much as possible. When conducting research on the Internet, .gov and .edu sites are the most trustworthy sources of information, followed by .info and .net.

Second, if someone makes a claim, ask “How do you know?” Learn how to evaluate the response.

Third, beware of “scientifical” or “sciency” arguments. Especially beware of anything that invokes vibrations, frequencies or energy flow, as well as anything with “quantum” in the title that doesn’t involve partial differential equations.

And, finally, if it sounds too good to be true, it usually is. A number of useful web sites that specialize in debunking bunk are listed below.

Dr. Suzanne E. Willis is a professor and assistant chair in the NIU Department of Physics.

Related: Useful web sites

Health
Quackwatch
Information is Beautiful

Energy
Popsci.com
Wired Science

General
Mythbusters

Blogs
Respectful Insolence 
Bad Astronomy  
Pharyngula 
Scientopia blogs
Science blogs