I haven’t been keeping up with neutrino blogging since the first two weeks after the big OPERA announcement last September. To recap: the OPERA collaboration (Oscillation Project with Emulsion-tRacking Apparatus) released results showing neutrinos arriving about 60 nanoseconds too soon, meaning they were somehow travelling faster than the speed of light. The ensuing public narrative mixed good science with bad, hyperbole with sense. Some declared that Einstein’s relativity was dead, others declared that this was hard evidence for extra dimensions beyond the standard four, and yet others (myself included) warned that it was too early to say one way or the other.

So my first reaction upon hearing today that the ICARUS experiment had found results consistent with relativity was an immediate sense of relief. Physics as we know it is correct! We don’t have to worry about what went wrong at OPERA (if the results didn’t hold up), or what new physics is necessary to explain the OPERA results (if they did).

However, my first reaction was wrong and unscientific. To see why, let’s go into the ICARUS paper a little more deeply. (Here’s the actual technical paper; see also Sean Carroll’s take on it, and John Timmer’s report for Ars Technica.) The ICARUS collaboration is the left hand to OPERA’s right hand: they are both detectors in Italy that measure neutrinos produced by high-energy reactions at CERN. Both experiments rely on precision timing facilitated by satellite. Although they don’t work in exactly the same way, they operate at about the same level of experimental precision, so any effect spotted by one should also be spotted by the other—if said effect is real.

The most important figure from the ICARUS paper released today is reproduced at the right, along with the OPERA results (on page 13 of the PDF file, for those following along at home). The data consist of event rates: how many neutrinos land in each detector in a given amount of time. The horizontal axis is the difference between the amount of time a photon would take to traverse the distance from CERN to the detectors and how much time was actually measured, so a value of zero represents a neutrino moving at the speed of light.

Two aspects of this plot are immediately obvious to me. First, the ICARUS result is still consistent with neutrinos that move faster than light, so it’s premature to say everything is back to normal as far as relativity is concerned. Second, the ICARUS and OPERA data do not overlap at all: at least one set of results must simply be wrong. In fact, they may both be out of sync with reality, though I personally put low odds on that possibility! The key to interpreting the plot is to look not just at where the highest number of counts are (the tallest rectangles), but how spread out each set of data are.

The wider spread for ICARUS means it has a larger error—which we shouldn’t think of in the colloquial sense of “mistake”, but rather the inevitable inaccuracies involved in any measurement process. However, this larger error is why it overlaps 0, meaning while it is possible neutrinos in the ICARUS data are moving faster than light, it’s a lot less likely. OPERA’s errors are smaller, which is why their original press announcement and paper said the results definitely showed faster-than-light neutrinos: no neutrinos were found moving slower than light in those data.

My initial reaction was the same as Sean Carroll’s: the new ICARUS results are definitive and the OPERA results are wrong. However, a few minutes’ reflection made me back away from that reaction. Just as it was too facile to say right off in September that the OPERA results are correct, it’s too easy to say right now that the ICARUS results are correct—and it’s all the more important for me to urge caution in my own reactions because ICARUS confirms my prejudices. Yes, relativity has held up since 1905, so ICARUS being consistent with that is what most scientists would expect…but until we know what happened with OPERA for sure, we run the risk of jumping to the wrong conclusion. Just because a result confirms our prior expectations is no reason to think it’s correct.

Once again, I do think the OPERA results are likely to be wrong, albeit for subtle reasons. (Similarly, if ICARUS is wrong, it will also be for subtle reasons!) Simply providing another set of data from which we can infer different conclusions isn’t enough to say OPERA is wrong. As scientists, we have to know why and how, and that will require further experiments. The pace of science may seem sloth-like at times, but with enough patience we will find which—if either—of these results is correct.