EMT and the difference between Sceptics and Believers Part II

There is an old expression heard today in many fields; “There is nothing more dangerous than a first-year (freshman) somethingorother”. I have often said there is nothing more dangerous than a first-year programmer and I have heard that there is nothing more dangerous than a first-year med student or law student.

I’m about to prove that there’s nothing more dangerous than a first-year psychology student. Keep that in mind; this is not a proven psychological phenomenon, but rather, these are more musings on my hypothesis.

A few weeks ago I wrote about my thoughts on the link between Belief or Scepticism and the types of errors in Error Management Theory. I mentioned that I thought that this was a testable idea so I’m writing this to “put up” rather than “shut up”.

What I would need to support my claim is a test which would show a statistically significant correlation between high levels of belief in the paranormal and a propensity for Type II errors. We all know that correlation does not equal causation, but the correlation in this case may hint towards a common cognitive process. There are two parts to this test; measure a subject’s rating on a paranormal belief scale and then measure the number of times the subject makes Type I or Type II errors in controlled situations.

The Paranormal Belief Scale Test

Fortunately, this is the easiest part of the test to design. Dr. Jerome Tobacyk from Louisiana Tech designed just such a standardised test (the RPBS) which was published in The International Journal of Transpersonal Studies in 2004. This revised test is based on the earlier Paranormal Belief Scale test (Tobacyk & Milford, 1983) and both of these standard paranormal belief scales have been used in many studies seeking correlations between paranormal belief and other phenomena. For some examples, have a browse through the articles in the latest Cortex (tip of the hat to Mind Hacks via Ionian Enchantment for bringing this edition to my attention).

Propensity of Error Types Test

Unfortunately, for this part of the test, I have no short-cuts available to me. I can’t find any standard psychological tests which establish a subject’s tendency towards acceptance of a hidden agent (if it doesn’t exist) or the subject’s implicit rejection of an unknown agent (if it does exist). Whenever I thought about designing such a test I would repeatedly fall into the trap of affirming the consequent, begging the question or using a kind of circular logic because to design a test about hidden agents we could ask about the subject’s acceptance of paranormal phenomena. But that was established in the RPBS test and cannot be used to prove the second part of the hypothesis. Intuitively it sounds like they are intimately related by definition, but cognitively, there isn’t necessarily this implicit similarity.

Initially, my thoughts about such a test involved a simulation, a type of haunted house virtual reality world where unexplained things could happen and the subject would solve the problem by identifying the cause of the unexplained, but this was flawed in that it plays-off a person’s preconceived notions about haunted houses (an alternative way of testing the RPBS again).

I also considered the simple experiment that Derren Brown (the mentalist magician) used in the season finalé of his latest Trick or Treat (Season 2) TV specials. In the test (which was partly based on B.F Skinner’s classic “supersition in pigeons” experiment from 1947) Derren Browne had set-up a room with hundreds of strange and unrelated objects and told the participants that they could share a prize money if the counter reached 100 before a time limit expired. There was no mention that their actions would have any bearing on the counter increasing, but that is the conclusion the subjects reached when the counter would coincidentally increase while they were doing something with the objects in the room. Heated discussion and logical arguments ensued whenever the counter increased unexpectedly or failed to increase when expected to. In actual fact, the counter was being incremented whenever a goldfish in a fish tank (which was outside of the experiment area) passed a line drawn on the side of the tank. At the end of the experiment some of the subjects had said that they thought they were very close to understanding the pattern of actions which triggered the increasing counter while the others had said that there was no way they could decipher if there even was such a pattern in principle.

This is also very close to the conclusion drawn by a recent study into the human tendency to search for patterns which really didn’t exist in situations when the subject feels that the outcome of a situation is out of their control. We are primed to search for meaning when we lose control, even when that meaning doesn’t exist.

Together, these experiments have made me think that there is perhaps a way to study the tendency towards Type II error by designing a simple game with an array of unmarked buttons and switches where the subject’s input is completely ignored but the game continues as though it was receiving some kind of input. It would then appear that the subject’s commands were sometimes executed and (rather vexingly) ignored at other times. Any subject who ceased “playing” the game and watched the game unfold (having correctly guessed at the reason for the anomolies in gameplay) would be removed from the results. A questionnaire about the game would be provided where the subject would rate a number of inconsequential aspects (enjoyment, visual appeal, age suitability, etcetera) and crucially ask for open-ended feedback. Those who report the game cheating, the rules changing, something working against them would be rated as having a higher Type II tendency.

To test for Type I error propensity, a similar game might be designed which actually worked against the subject, cheating and even changing the rules at times when the subject was doing well (but not in an overt way). The subject would then complete a questionnaire which, again, had a number of inconsequential questions but also asked the subject to rate their own performance in the game. Those who correctly identify that the game was cheating would have a low Type I error propensity while those who stated that their own inadequacy was the reason for their failure would have a high Type I error propensity.

Game Design: The Crucial First Step

The games in this experiment must be carefully designed so that they don’t “give the game away”, so to speak. It may be that I have not solved the testability problem but only moved it into another domain, I don’t think that I have, it sounds possible to me but let me know if you think there are flaws in this experiment design, as a student of psychology I find the feedback invaluable.


~ by James on 24 December 2008.

2 Responses to “EMT and the difference between Sceptics and Believers Part II”

  1. Interesting. But… I suspect creating a really tight protocol that everybody can agree on will be nearly impossible. Psychological experimental results are almost always open to multiple interpretations (given the large number of variables) and the problem increases as you move to more abstract concepts like “type II error”.

    Keep thinking about it though…

  2. I will definitely keep thinking about it. Thanks for the feedback. 🙂 I also thought yesterday that there are flaws inherent in computer-based error experiments – it is more likely that the subjects would respond that the program is buggy rather than actively working against them. I have worked in IT for many years and it is incredible the number of people who have a distrust of computers for no good reason. Yet more to think about.

Comments are closed.

%d bloggers like this: