Ole Rogeberg produced a pretty thoughtful critique of the Dunedin Longitudinal Survey group’s finding that marijuana use reduced IQ. And, he subsequently wrote a very nice summary of the dispute, along with some musings about whether scientific progress is helped or hindered by media sniping.
Rogeberg’s critique wasn’t based on replication of the Dunedin study. Indeed, Rogeberg couldn’t even get some summary stats out of the Dunedin group. He writes:
When I originally started looking into this last August, I sent an e-mail to the corresponding author asking for a couple of tables with information on “pre-treatment” differences between the exposure groups. I did not receive this. This is quite understandable, given that they were experiencing a media-blitz and most likely had their hands full. I therefore turned to past publications on the Dunedin cohort to see if I could find the relevant information there.
Do read the whole thing. It has a whole lot of very substantial critique. But this bit above piqued my interest. Because he was criticized by Dunedin’s Prof Poulton for, well, you read it:
Rogeberg said the political approach to cannabis could change depending on whether there was a change in young smokers’ IQ because of the drug itself, or because of living conditions.He said his study did not mean the results of the Dunedin study were discredited, but it was fair to say the New Zealand study’s methodology was flawed and the results premature.But Professor Richie Poulton, co-author of the Dunedin research, said Rogeberg’s data was not taken from real people.“Rogeberg’s challenge is based on simulations, but we used actual data on 1000 people to carry out the analyses he suggested,” he said. [emphasis added]
I’m going to take this as Poulton offering to share his data with Rogeberg. Because it would be rather, well, gauche to criticize somebody for failing to use data you’ve refused to share.
I completely understand the Dunedin Longitudinal folks’ unwillingness to share data given the rather high costs they’ve incurred in collecting it; it just seems off to critique somebody for having used simulations rather than data after you’ve refused to share the data.