Pew Research Center: Religious Knowledge in the US

Assalamu aleikum/G'day/Gutentag/Konnichiwa/你好,

Not to long ago the Pew Research Center conducted a survey in which 3,412 Americans were asked to answer 32 questions on religion. The survey found that:

" On average, Americans correctly answer 16 of the 32 religious knowledge questions on the survey by the Pew Research Center’s Forum on Religion & Public Life. Atheists and agnostics average 20.9 correct answers. Jews and Mormons do about as well, averaging 20.5 and 20.3 correct answers, respectively. Protestants as a whole average 16 correct answers; Catholics as a whole, 14.7. Atheists and agnostics, Jews and Mormons perform better than other groups on the survey even after controlling for differing levels of education."

Predictably, the blogosphere was soon filled with significant amounts of self-congratulatory chatter. For instance, in response to an entry on the Guardian website about this topic (which contained a link to a 15-question sample survey), reader comments such as the following could be found:

"well of course the atheists know their stuff - we're the ones who have actually applied some thought and enquiry to the concept!"

 and

"Atheists do better in this quiz because we are better educated and better informed. We are interested in other people's points of view and take the trouble to learn about them before making decisions. As an atheist, I got 14\15."

Now, one of the fundamental concepts in academia is the acknowledgement of the difference between the hard data itself, and the interpretation of the data. It is very natural to infer from the results of the survey that the survey itself is a manifestation of a truth that atheists and agnostics benefit from inherent superiority in the field of intellect. However, such a conclusion fails to take into account the fact that the basic finding of this 32-question survey of 3,412 Americans was exactly that - that of a group of 3,412 Americans asked 32 questions, atheists and agnostics on average got the most number of questions right out of the 32 questions asked.

It is worth asking ourselves:

1. What exactly was the nature of these questions? Is this significant?
2. Of what significance is the number of questions asked?
3. Of what significance is the sample size?
4. How was it that the atheists/agnostics got the most questions right on average? What is the meaning of such a result?
5. It is stated that "Data from the survey indicate that educational attainment – how much schooling an individual has completed – is the single best predictor of religious knowledge." So, what is the relationship between lack of religion and educational attainment? Is there a relationship? And what kind of relationship is it anyway?
6. What reflection is such a high score of an individual's intelligence and wisdom?

So, do people like the writers of the above two comments have it in the bag? Or are they merely victims of the likes of confirmation bias and illusory correlation? After all, can't one also suggest that the study only goes to show the atheistic tendency for gathering of superficial trivia at the expense of any real sort of understanding?

Blind men, here is your elephant!