They say that fish rot from the head, so could the drop in Australia’s performance in education be due to the quality of some of the academics holding tenured positions at universities rather than what happens in primary and secondary school?
If credentialled, well-funded and tenured tertiary institution staff are capable of dishing-up research which should fail an undergraduate, what chance have lower echelons.
Which brings me to the case of Professor Stephan Lewandowsky, who has been recently making a career trying to link what he calls “climate denial” with delusion.
His latest project is a study entitled MOTIVATED REJECTION OF SCIENCE and sub-titled “NASA faked the moon landing|Therefore (Climate) Science is a Hoax: An Anatomy of the Motivated Rejection of Science”.
It could have been sub-titled “The man who polled his prejudices and mistook them for the facts”. Prof Lewandowsky’s paper is certainly motivated, but it is not scientific or competent.
I’ve labelled this Part 1, as there is so much to analyse, and I know I will have to serialise.
I first came across this survey when approached by Jo Nova to have a look at the questionnaire two years ago. The study does not include the questionnaire, but I kept screen shots, and yes, I am one of the quoted sample of n>1200, but as we will soon see, n is actually, for the headline results <10. I’ve put the links to the screen shots at the bottom of this page.
The first thing to note is that there are 40 questions in the questionnaire, but a number of them are not referenced in the paper. I’m not sure why this is the case. One of them is also not present in the dataset that Professor Lewandowsky provided to me. That question is:
7. The Iraq War in 2003 was launched for reasons other than to remove Weapons of Mass Destruction (WMD) from Iraq
This question was also removed, but this was explained in the paper, effectively on the basis that it should never have been in the questionnaire. Which raises the question as to why it was in the questionnaire.
20. The claim that the climate is changing from emissions due to fossil fuel emissions is a hoax perpetrated by corrupt scientists who wish to spend more taxpayer money on climate “research”.
Questions 28 to 32 have also been removed. It was a mystery to me why they were in there in the first place as they ask how I feel about my life. That they play no part in the paper suggests that whatever theory was being investigated using them failed to pan out. If this is the case it is bad practice, and basically dishonest, not to report the failure.
There are two substantial findings in Lewandowsky’s paper. One is that people who believe in the free market are likely to reject science for ideological reasons. The second is that people who believe in particular conspiracy theories are even more likely to reject science for ideological reasons.
I’m going to conclude this first post by looking at how large his “n” really is.
For the free market question there were 6 questions that purported to define it (next blog for why “purported”). The other 5 questions were used in the model. The number of respondents who agreed with all 5 was 80. The graph below shows the number of respondents for each question.
On the basis of this, his n, when talking about free markets is somewhere between 80 and 244, depending. This is a long way short of the >1200.
Things get worse when you look at the conspiracy figures. There were only three correspondents (thank goodness) who agreed with all the propositions, and as this graph shows, generally only small numbers agreed with any of the propositions.
The conspiracy which gave its name to the subtitle in fact had only 10 adherents (and only 3 of these thought global warming was a hoax).
So the n in this case is somewhere between 3 and 289. Again, a long way short of >1200.
Questionnaire images:
Thanks for the analysis look forward to read your follow ups. I have linked to this post on this thread where the paper is also discussed
http://www.bishop-hill.net/blog/2012/9/5/more-lew.html#comments
One point of interest is that it seems at least two Skeptic blogs where offered a similar survey at the time but with the questions 28-32 appearing in the first 1-5 places.
Comment by tlitb1 — September 6, 2012 @ 3:53 am
As you are no doubt aware, this “study” is generating comment and questions all over the blogosphere. What you are saying here is very relevant and I look forward to the further posts. The screen captures you have made are of particular interest as there has been some speculation that different versions of the same survey may have been circulated. Bearing that in mind, could you fix the links to questions 22-27 and 35-42?
Comment by LaurieChilds — September 6, 2012 @ 3:55 am
As tlitb1 points out at Bishop Hill, the paper failed to report any answers for the WMD in Iraq question.
I realized today that there is one very popular conspiracy theory that the survey leaves out entirely. Something like
“Climate skepticism only exists as a result of secret funding by big oil and tobacco interests”
Surely Lewandowsky, or his assistants, would have come across this in researching climate blogs?
Comment by Manicbeancounter — September 6, 2012 @ 4:58 am
Great analysis.
You might wish to point out that some of the ‘Free Market’ questions were reverse scored so that all point in the same direction (i.e. low score = pro-free-market).
Comment by DR_UK — September 6, 2012 @ 6:35 am
Could you amend the link for Questions 22-27 (I get a page not found error)? I think it’s a mistyped I and .)
Thanks!
Comment by DR_UK — September 6, 2012 @ 6:52 am
[…] Jo Nova has a great summary here, and writes about one Australian investigator who was invited to take the survey questions two years ago, kept screen shots of it, and did an analysis. She wrote: Graham from OnlineOpinion was so struck by the study he’s written a post titled: Fish rot from the head Part 1. […]
Pingback by Stephan Lewandowsky’s slow motion social science train wreck | Watts Up With That? — September 6, 2012 @ 8:18 am
Sorry about the links. I thought I had them sorted, but it was late last night that I did this. I’ve been sitting on my hands trying to find time to write.
DR_UK I don’t think anything hinges on the reverse scoring. I’m not sure why they did that in the analysis but as long as it is consistent there is no problem with it.
Comment by Graham — September 6, 2012 @ 10:47 am
Well done Graham. Lewandowsky has surpassed himself; the ramifications of this ‘effort’ at ‘research’ by him should have employment ramifications but I would content myself with knowing what sort of fish you think Lewandowsky is.
Comment by cohenite — September 6, 2012 @ 1:56 pm
So let me get this straight.
10 respondents believed the moon landing were a hoax
3 of those believed Global Warming is a hoax
therefore 7 out of the 10 do not believe Global Warming is a hoax.
70% of people who believe the moon landings were a hoax “believe in Global Warming”.
Shouldn’t the papers title be:
“NASA faked the moon landing|Therefore (Climate) Science is NOT a Hoax: An Anatomy of the Motivated Rejection of Science”.?
Comment by Geckko — September 6, 2012 @ 7:50 pm
If these researchers were actually good at what they do, they’d do the same study investigating the alarmist side. It might be entitled “IPCC faked climate alarm – therefore (climate) science is true: An anatomy of the motivated acceptance of science.”
Somehow I doubt they will investigate climate alarmism in the same way they did climate skepticism. I doubt they would have the balls.
cheers
Comment by klem — September 6, 2012 @ 11:34 pm
Thanks Graham – sorry I didn’t make myself clear.
I don’t think reverse scoring is a problem either. I meant that your first graph has some qs that would have been agreed with, and others that would have been disagreed with by ‘free-marketeers’, and reverse scoring sorts that out. I thought it might confuse your readers. But probably it didn’t!
Comment by DR_UK — September 7, 2012 @ 6:45 am
The poor performance in Australian education has a great deal to do with Left wing Australian academics—and they are responsible for what happens in Primary and Secondary schools.
Australian education is firmly in the grip of the Left wing academics who do the ‘research’ and design the curriculum—the same academics who snow the Left wing bureaucrats—the same Left wing bureaucrats who run the Boards of Studies or equivalent—-the same Left wing bureaucrats in the boards of studies , who easily snow the politicians of both sides—-the same politicians who decide on policies as espoused by their Boards of studies, who design the testing if the Left wing decides to permit testing, and who oversee the mainly Left wing teachers who are ‘looked after’ by the Left wing education unions, and the same Left wing teachers who find it very very easy to snow the parents.
That’s why we are on a slide and why it never changes—because only the easily-snowed politicians change.
The curriculum is the thing—given competent teachers—because it doesn’t matter how superior a teacher might be if he/she is teaching a rubbish curriculum that leaves children without the fundamental knowledge in literacy including a comprehensive knowledge of grammar , and in numeracy—— by the end of Primary School [ crucial].
All of their further education, in secondary and tertiary studies relies on that fundamental literacy and numeracy education—and for decades, the Left in Australia and the US and UK—-has provided ideological indoctrination instead.
The names of the Left wing academics whose ideas and policies are responsible for the decline are Dale Spender and the late Garth Boomer, who demanded an end to teaching of grammar, Brian Cambourne [ whole language guru], Mary Kalantzis , Bill Cope, Alan Luke, Ian Lowe and back in the 80s, Michael Pusey and Professor Ted Wheelwright, to name a few.
Some words of Wheelwright, commenting on his fellow Leftist colleague Pusey’s ideas…..
[ ‘In short’, comments Wheelwright, ‘Pusey is telling us that the educational aims and objectives are now so diverse and diffuse that they amount to nothing less than total personality development.’
‘ Hence these goals have become completely indeterminate, and thus inimical to bureaucracy and all forms of organisational control; they are in a word , resistant to rationalisation.
And this is a most excellent thing.
Poorly defined and diffuse goals are the enemy of bureaucratic administration.
The goals of education must therefore be kept as ill-defined and diffuse as possible, for how can you assess efficiency and productivity unless you know what you are supposed to be producing?’ ]
And so, of course , parents have not known for decades whether their children are achieving the outcomes they want for them or not.
They are not meant to know.
These Leftists believe it’s not the business of the parents to know what their children are being deprived of.
Comment by truth — September 7, 2012 @ 4:05 pm
The link for 22-27 is actually the image for 28-32.
Comment by Martyn — September 17, 2012 @ 8:58 am
While it is sensible to look beneath the bonnet of research, your refrain that numbers fall far short of the n= 1200 is laughable. N always has and always will refer to the number of people who participated in the study, not answered a particular question. We are not clones, so the Graham N will never equal the actual N. I think the fact that the study was conducted on the net and random sampling (necessarily) wasn’t possible was more of a problem…but all research has its faults. Even rolled gold cancer research often doesn’t properly account for the placebo effect. I would describe this study as MOTIVATED REJECTION OF RESEARCH STUDY…BECAUSE I DON’T LIKE WHAT IT IS SAYING
Comment by Olav Muurlink — September 18, 2012 @ 12:18 pm
Olav, that is nonsense, and an academic should know that. The “n” value refers to the number of people in the sample you are dealing with. If you have taken a subset of your sample then the “n” is not the same as for the whole sample. The major purpose of the “n” is so that the reader has a good idea of the sampling error in each case and how reliable the result is. If this is typical of what comes out of psychology departments then they should all be wound up. Results of research must be very flaky indeed. Some of Lewandowsky’s conclusions are based on samples where the “n” is 10, and are simply rubbish.
Comment by Graham — September 23, 2012 @ 10:40 am
I suspect you haven’t done first year stats, Graham. Psychologists do four years of statistics before they even graduate. It’s too complex to explain here, but the full power of the 1200 sample was in fact used to show statistical significance in relation to the ‘n’ (as you call it) of 10. It’s perfectly normal and valid procedure…trust me, I’m a psychologist 😉 To reiterate, I have problems with the sampling procedure…but then, sampling procedure problems are rife in all forms of scientific research, partly for good, ethical and practical reasons.
Comment by Olav Muurlink — September 26, 2012 @ 10:41 am
You can’t have a statistical significance that is much above meaningless noise in a sample of 10 Olav. You should have learned that in your first year, assuming that you attended the stats classes.
By your claim that he used the 1200 to show the statistical significance of the 10 I assume you are referring to his claim that in answering the questions that he didn’t report on, which were how you felt about your life, this sample was similar to another that he had done, and therefore his sample hadn’t been gamed. That would seem to be a pretty useless procedure too, but as he didn’t actually provide any data on that, difficult to definitively comment.
Comment by Graham — September 30, 2012 @ 1:49 pm
[…] the paper’s “Anthropogenic warmist nonsense,” suggest that the paper is “not scientific or competent,” and describe it as “an ad hom[inem] argument taken to its absurd extreme,” an […]
Pingback by Climate Change Denial Linked To Conspiratorial Thinking : 13.7: Cosmos And Culture : NPR – Taking Science to the People — December 12, 2012 @ 5:37 am
[…] the paper’s “Anthropogenic warmist nonsense,” suggest that the paper is “not scientific or competent,” and describe it as “an ad hom[inem] argument taken to its absurd extreme,” an […]
Pingback by Why publishers should explain why papers disappear: The complicated Lewandowsky study saga | Retraction Watch — March 29, 2013 @ 6:03 am
@12: For a change, this paper is not the result of the Australian school system. Though the author Lewandowsky did attend Australia school for some time, he did study psychology at the University of Saarbruecken, Germany, and one co-author is from Zuerich, Switzerland. I am deeply embarrassed by this, because the University of Saarbruecken is also my Alma mater and I was always under the impression that we did get a great deal of training in statistics.
Comment by Marlene — April 1, 2013 @ 4:41 am
[…] http://www.ambitgambit.com/2012/09/06/fish-rot-from-the-head-part-1/ […]
Pingback by Lew’s Third Table | Geoffchambers's Blog — August 9, 2013 @ 8:25 pm