Wetenschapsforum

 

Psychological theory and replication.


 Aug 7, 2016
 
Psychological science has been under serious scrutiny lately. Academic psychology suffers from a ‘replicability crisis’. This seemed to hit social psychology, and priming studies, in particular (cf. Kahneman 2012), but other psychological fields are not immune to it (Open Science Collaboration 2015). What is the problem? Many psychological experiments have been unable to yield the same results upon repetition. Some of these experiments have resulted in theories that appeared in well-known journals and well-cited publications. Some have become part of psychological theorising for several decades. When a particular finding cannot be repeated, many theorising based on it may be lacking a foundation. How can it happen that a scientific result turns out to have been a one-off result?

A psychological theory aims to understand, and ultimately predict, human mental activity and behaviour. As these are probably among the most complex phenomena occurring, there is ample opportunity for uncontrolled variables to influence the results of studies into these phenomena. The complexity of human activity arises from a myriad of causes, many escaping control of the experimenter. The behaviour (and mental activity) of a human subject can be caused by current and past behaviour, perceptions, ideas, surrounding and physiology (to name a few).
In order for a psychological theory to be valid and reliable, many aspects of behaviour and mental activity ideally are captured by it. Particularly the many potential causes of, and influences on, the phenomena to be explained should be acknowledged to play a role in the construction of the theory. Overlooking them may result in theories that falter in replication research.
For above mentioned reasons I suggest that a psychological theory should be ‘HEPI’ (Holistic, Embodied, Phenomenological and Implicit):

1. Holistic:
   
   A theory should include as many ecologically valid aspects as viable. For sensory perception this means the aspects are based on multiple sensory inputs. All sensory systems interact, with each other but also with top down processes like expectations (cf. Ellingsen et al. 2016, Piqueras-Fiszman and Spence 2016). The stimulation employed to study, propose or test a specific theory should be close to the situation that the theory is supposed to work in. This means that a non-holistic (reductionist) approach will increase the chance of a theory faltering in subsequent experiments, where some part of the situation may be different yet (in hindsight) important. The effect of top down processes implies that the information that subjects receive –or may infer- becomes part of the experiment, and should ideally be under rigorous control.

Although this seems to argue pro strictly isolated (reductionist) laboratory studies –to control as much unwanted influences as possible-, such studies are at the same time un-holistic. This dilemma is certainly a dilemma of applied psychological scientists. They often find themselves working in noisy environments, trying to control the noise. If they obtain an effect it was large enough to stand out from the noise, and valid (from the point of view of their applied research question) as it appears under realistic circumstances. Studies in, low noise, laboratories may yield significant, but small, effects that can be relevant for a specific theory (a laboratory, reductionist one), but outside the laboratory (or in another laboratory) the effect may not appear. A practical way out is performing a ‘proof-of-principle’ or ‘pilot’ study first, followed by larger studies and finally by field studies (cf. Woods et al. 2010, Le Berre et al. 2011, 2013, Dijksterhuis et al. 2014).

1. Embodied:
   
   A psychological theory should recognise that cognition is hardly an abstract and isolated mental activity but that it is grounded, primarily in the body (cf. Barsalou 1999, 2010). Psychological activity, be it thinking, perceiving, or behaving cannot be seen separate from the interaction of a subject with its body (Semin and Smith 2008, Shapiro 2014). In addition the state this body is in must be taken into account as it can have serious effects on cognition. This can be seen g. in decision making situations (Bechara et al. 1997), where a relation to Somatic Marker theory (Damasio 1996) may exist (cf. Reimann et al. 2012), but perhaps this should be called embodied emotion rather than cognition. We will not expand on this link here.

Although a general physiological state of the body (hunger, fear, fatigue, etc.) is not what is commonly meant in ‘embodied cognition’, it can exert an influence on cognition and behaviour. For this reason it is mentioned here.

1. Phenomenological:
   
   A psychological theory should be phenomenological, e. it should refer to what an individual experiences, instead of to some description of what is presented to the subject. External (physical or semantic, sometimes called ‘objective’) descriptions of stimuli can have an unclear relation to the percepts the subject experiences. Percepts could arise as a result of an illusion or of an interaction between information and sensory stimulation. This potentially destroys the relation of an external (‘objective’) description of the stimulation to a subject’s experience.
   
   Although the stimulation presented to subjects in a psychological experiment should be described as completely as possible in ‘external’ terms, it should be clear that there can be, and often is, is a difference between the presentation of this stimulation and the experience of the subject as a result from that stimulation.
2. Implicit:
   
   A psychological theory should take into account the fact that most psychological processes remain inaccessible to the individual itself. This means that a theory should not rely on self-reports. There is a difference between ‘reasons’ and ‘causes’, just like correlation do not imply causation. The ‘snare of psychology’ (James 1890) refers to the fact that the state that a reporting subject is in, is different from the state the subject is (trying to) report on. The act of thinking about a question, interpreting it and formulating an answer is likely to distort that what is under investigation. This may in particular affect questionnaire based studies where a theory is based on a Factor Analysis of a list of items. Posing an ‘n-Factor model’ as a theory in a specific field, based on questionnaires alone is likely to yield a theory that is not easily replicated under (even slightly) different circumstances.
   
   There are exceptions, g. when it is the aim of the theory to find out how a subject reports, or what a subject can explicitly relate w.r.t. a certain stimulation or situation. There can also be questionnaires that are especially designed to not alert subjects to the subject matter under investigation. They will yield more valid ‘n-Factor-models’, than the questionnaires with mainly explicit questions about the matter under research.

 
References
Barsalou, L. (1999). Perceptual symbol systems. Behavioral and Brain Sciences (1999) 22, 577–660.
Barsalou, L. (2008). Grounded Cognition. Annu. Rev. Psychol. 2008.59:617-645.
Bechara, A., Damasio, H., Tranel, D., Damasio, A.R. (1997). Deciding Advantageously Before Knowing the Advantageous Strategy. Science 275, 1293-1295.
Damasio, A.R. (1996). The somatic marker hypothesis and the possible functions of the prefrontal cortex, Philosophical Transaction: Biological Sciences, 351: 1413–1420.
Dijksterhuis, G.B., Boucon, C., Le Berre, E., (2014). Increasing saltiness perception through perceptual constancy created by expectation. Food Quality and Preference. 34, 24–28.
Ellingsen, D-M., Leknes, S., Guro, L., Wessberg J., Olausson, H., (2016). The Neurobiology Shaping Affective Touch: Expectation, Motivation, and Meaning in the Multisensory Context. Frontiers in Psychology, 01986. doi: 10.3389/fpsyg.2015.01986.
Kahneman, D. (2012).  A proposal to deal with questions about priming effects. Letter/e-mail.
Le Berre, E., Woods, A.T., Boucon, C., Dijksterhuis, G.B. (2011). The effect of expecting homogeneity on taste perception. Presentation at the 9th Pangborn Sensory Science Symposium. September 2011, Toronto, Canada.
Le Berre, E., Boucon, C., Knoop, M., Dijksterhuis, G.B. (2013). Reducing bitter taste through perceptual constancy created by an expectation. Food Quality and Preference. 28, 370-374.
James, W. (1890). The Principles of Psychology. (Chapter VII. The methods and snares of psychology.).
Semin, G.R., Smith, E.R. (2008). Embodied Grounding. Social, Cognitive, Affective, and Neuroscientific Approaches. Cambridge: Cambridge University Press
Piqueras-Fiszman, B., Spence, C. (2016). Multisensory Flavor Perception: From Fundamental Neuroscience Through to the Marketplace. Woodhead Publishing.
Shapiro, L. (2014). The Routledge Handbook of Embodied Cognition. London: Routledge.
Open Science Collaboration (2015). Estimating the reproducibility of psychological science. Science, 349, 6251, 943.
Reimann, M., Feye, W., Malter, A.J., Ackerman, J.M., Castano, R., Garg, N., Kreuzbauer, R., Labroo, A.A., Lee, A.Y., Morrin, M., Nenkov. G.Y., Nielsen, J.H., Perez, M., Pol, G., Rosa, J.A., Yoon, C., Zhong, C.-B. (2012). Embodiment in Judgment and Choice. Journal of Neuroscience, Psychology, and Economics , Vol. 5, No. 2, 104–123
Woods, A.T., Poliakoff, E., Lloyd, D.M., Dijksterhuis, G.B., Thomas, A.T. (2010). Flavor Expectation: The Effect of Assuming Homogeneity on Drink Perception. Chemosensory Perception. 3: 174-181.