stevepostrel, it’s impossible to say. Stephan Lewandowsy managed to do what you describe by applying tests inappropriate for his data. The tests he used require the data have a normal distribution. If that’s not true, the tests produce incorrect results. In Lewandowsky’s case, it produced the spurious correlations you refer to.
For part of his work, Dan Kahan applied simple correlation tests which also assume the data is normally distributed. His data does not have a normal distribution. That means those test were inappropriate in the same way Lewandowsky’s tests were.
That doesn’t mean the result of using an inappropriate test is the same though. Using data with a non-normal distribution can produce all sorts of strange results. Without being able to see the data itself, I don’t know how the inappropriateness affected the results. It could have had any number of impacts. I know what Kahan did, and it is the same thing Lewandowsky did, but until Kahan publishes his data (something he refuses to do), I can’t say how it affected his results.
What I can say is Kahan is incredibly obtuse about this issue. I pointed out the issue, saying:
the r scores show you making the same sort of mistake Stephan Lewandowsky and Michael Wood made.
Kahan asked if the simple correlation scores he reported were what I was referring to. I responded by saying (in part):
The only thing I could have possibly been referring to were the “simple correlations” you refer to now, ones you can find by searching for “r =0.” in the document.
Clearly acknowledging I was referring to what he asked if I was referring to. His response was baffling:
So in other words you won’t tell me which correlations you think are invalid. Okay. I’m sure you are very busy– just thought since you’d made the effort to read, I’d see if I could get some useable insight from you.
I don’t understand this response. How can someone be snarky based on the idea I refused to answer a question when I specifically acknowledged the answer was what he thought it was. I even gave a search string so one could find what I was referring to without having to look through the whole document being discussed.
That wasn’t the only time he flat-out made things up about what I said. And like the other times, he simply ignored the issue when I pointed out his error. If that’s not him being willfully obtuse, I don’t know what it is.
![\scriptstyle\rule[2.25ex]{0.01pt}{0.01pt}\,\boldsymbol{\overset{\scriptstyle\circ\wedge\circ}{\smile}\,\heartsuit\,{\displaystyle\text{\bfseries!!!}}\,\heartsuit\,\overset{\scriptstyle\circ\wedge\circ}{\smile}}\ \rule[-0.25ex]{0.01pt}{0.01pt}](http://s0.wp.com/latex.php?latex=%5Cscriptstyle%5Crule%5B2.25ex%5D%7B0.01pt%7D%7B0.01pt%7D%5C%2C%5Cboldsymbol%7B%5Coverset%7B%5Cscriptstyle%5Ccirc%5Cwedge%5Ccirc%7D%7B%5Csmile%7D%5C%2C%5Cheartsuit%5C%2C%7B%5Cdisplaystyle%5Ctext%7B%5Cbfseries%21%21%21%7D%7D%5C%2C%5Cheartsuit%5C%2C%5Coverset%7B%5Cscriptstyle%5Ccirc%5Cwedge%5Ccirc%7D%7B%5Csmile%7D%7D%5C+%5Crule%5B-0.25ex%5D%7B0.01pt%7D%7B0.01pt%7D&bg=bbffcc&fg=0055ff&s=2 "\scriptstyle\rule[2.25ex]{0.01pt}{0.01pt}\,\boldsymbol{\overset{\scriptstyle\circ\wedge\circ}{\smile}\,\heartsuit\,{\displaystyle\text{\bfseries!!!}}\,\heartsuit\,\overset{\scriptstyle\circ\wedge\circ}{\smile}}\ \rule[-0.25ex]{0.01pt}{0.01pt}")
