How do you know whether a scientific idea is a good one or not?

The only sure way is to study it in detail and know all the technical ins and outs. But good ideas and bad ideas behave differently over time, and this can provide clues as to which ones are solid; useful if you’re a non-expert trying to evaluate a field, or a junior researcher looking for a career.

Today’s ideas are the basis for tomorrow’s experiments. A good idea will lead to experiments which provide interesting results, generating new ideas, which will lead to more experiments, and so on.

Before long, it will be taken as granted that it’s true, because so many successful studies assumed it was. The mark of a really good idea is not that it’s always being tested and found to be true; it’s that it’s an unstated assumption of studies which could only work if it were true. Good ideas grow onwards and upwards, in an expanding tree, with each exciting new discovery becoming the boring background of the next generation.

Astronomers don’t go around testing whether light travels at a finite speed as opposed to an infinite one; rather, if it were infinite, their whole set-up would fail.

Bad ideas generate experiments too, but they don’t work out. The assumptions are wrong. You try to explain why something happens, and you find that it doesn’t happen at all. Or you come up with an “explanation”, but next time, someone comes along and finds evidence suggesting the “true” explanation is the exact opposite.

Unfortunately, some bad ideas stick around, for political or historical reasons or just because people are lazy. What tends to happen is that these ideas are, ironically, more “productive” than good ideas: they are always giving rise to new hypotheses. It’s just that these lines of research peter out eventually, meaning that new ones have to take their place.

As an example of a bad idea, take the theory that “vaccines cause autism”. This hypothesis is, in itself, impossible to test: it’s too vague. Which vaccines? How do they cause autism? What kind of autism? In which people? How often?

The basic idea that some vaccines, somewhere, somehow, cause some autism, has been very productive. It’s given rise to a great many, testable, ideas. But every one which has been tested has proven false.

First there was the idea that the MMR vaccine causes autism, linked to a “leaky gut” or “autistic enterocolitis”. It doesn’t, and it’s not linked to that. Then along came the idea that actually it’s mercury preservatives in vaccines that cause autism. It doesn’t. No problem – maybe it’s aluminium? Or maybe it’s just the Hep B vaccine? And so on.

At every turn, it’s back to square one after a few years, and a new idea is proposed. “We know this is true; now we just need to work out why and how…”. Except that turns out to be tricky. Hmm. Maybe, if you keep ending up back at square one, you ought to find a new square to start from.