Mental chronometry: long past, bright future
DOI:10.1093/acprof:oso/9780199228768.003.0016
Abstract and Keywords
This chapter looks at changes and developments in the field of mental chronometry during the past fifty years. In 1950, E. G. Boring recognized that the subtractive method for measuring mental operations never worked and argued that total processes are not compounded of elements with separate times but rather were due to differential preparation between the tasks. This chapter discusses the use of purely behavioural methods designed to partition overall response time into component processes and measurement of brain activity.
Keywords: mental chronometry, E. G. Boring, subtractive method, mental operations, behavioural methods, brain activity
In his History of Experimental Psychology, E. G. Boring (1950) labelled the late nineteenth century as the ‘period of mental chronometry’. However, Boring argued that the subtractive method (Donders 1868) for measuring mental operations never worked. He believed that reaction times were too unreliable and their differences even more so. Moreover, he agreed with Kulpe (1985), who argued that total processes are not compounded of elements with separate times but rather were due to differential preparation between the tasks.
Despite Boring's prestige as professor of psychology at Harvard, his view turned out to be far off the mark. From the middle of the twentieth century to the present day, mental chronometry, originally based on reaction time and its derivatives and later incorporating the measurement of electrical activity and haemodynamic changes, has been developed as a means of tracing information flow in the human nervous system. This chapter first discusses the use of purely behavioural methods designed to partition overall response time into component processes, and then turns to measurement of brain activity, finally discussing the use of all these methods in combination as a current and future research direction.
Reaction time and its derivatives
Shannon and Weaver (1949) developed a general mathematical theory for the measurement of information. In subsequent years these ideas were widely applied to psychology (Attneave 1959) and they helped to revive the use of mental chronometry. It was found that the time to respond to a stimulus was a function of the amount of information transmitted from the stimulus to the response. This finding showed that reaction time (RT) was not dependent on the physical stimulus alone, but was influenced by what might have occurred, or, roughly speaking, of the person's expectation. Information transmitted (p.208) allowed psychologists to combine the number of possible stimuli, their probabilities, and the error rate in a single correlation that sometimes accounted for more than 90% of the variance in particular laboratory tasks (Hick 1948; Hyman 1953). The Hick-Hyman Law was a major contribution to chronometric measurement. However, the slope of the function relating RT to information transmitted could vary widely depending on the compatibility between stimulus and response. When the fingers rested on vibrating keys and the person pressed the vibrator (Mowbray and Rhoades 1959), or when a vocal response to a written word was required, the slope was zero. When the response was highly compatible with the stimulus so that no conflict resulted, it appeared that response time did depend on the physical stimulus and not on the number of possibilities (Posner 1966). In most novel situations in which the stimulus to response mapping was unfamiliar, the person transmitted no more than 20–30 bits per second. Statistical models with a criterion for the occurrence of a response and a random walk based on the strength of the stimulus response combination were able to account for most of the data (Stone 1960). However, it was clear that people could concentrate their attention on one S-R association and modify the rate of information accumulation. Thus, both top-down and bottom-up factors influenced the RT.
Fitts' Law (Fitts 1954) related the time to move from a home button to a target (movement time) to the amount of information generated by the movement. This law continues to be applied in industrial situations. In some situations the Hick–Hyman and Fitts laws could account quite well for the time to begin and end a movement to a target. The amount of information that an aircraft instrument generates proved to be a good predictor of how long the pilot will spend examining it (Senders 1964). Despite these empirical successes, the idea of an infinite capacity to transmit information implied by a zero slope between information and reaction time led Neisser (1967) to declare that the era of information theory was dead. Mental chronometry would need more than a single mathematical tool that summarized input-output relations in order to describe the processing capacity of humans.
In a paper published in Science, Saul Sternberg (1966) showed that it took about 30 milliseconds (ms) per item to scan a list of digits in memory. Posner and Mitchell (1967) published a Psychological Review article using a version of Donders' subtractive method to show that it took about 80 ms to derive the name of a letter from its visual form. Both of these measurements were reliable and neither depended upon comparing separate tasks for which people could differentially prepare. Rather, they both embedded the measurement within a single random block of trials, so that predictions about what would occur were impossible.
(p.209) At a meeting in honour of Donders' centenary, Sternberg (1969) presented the general additive factors method. His idea was to assume that overall RT in a task was the sum of a set of independent processing stages that summed to the total time. If this is true, he showed that the effect of any independent variable could be located within the sequence of processing stages by looking for a pattern of additivity and interaction between the unknown variable and variables chosen to influence particular stages. It was a very important contribution that greatly generalized the utility of Donders' approach, and has since been exploited in a wide number of experimental investigations (Sanders 1998).
A natural objection to Sternberg's method is to suppose that many processes can occur in parallel. For example, one can try to name the ink colour of the word RED and, while attempting to do so, also to acquire the word meaning in parallel (MacLeod 1991). In fact, brain processes are often carried out in parallel. However, it is rather easy to design tasks in which one stage must be completed before any useful information is available for the next state, and in such serial tasks the additive factors method remains a very useful tool.
Even if two processes occur in parallel it is still possible to demonstrate that they maintain their independence by showing that one independent variable can influence the time for stage X without affecting the time for stage Y, whereas another variable may influence the time for Y but not for X. In the book Chronometric Explorations of Mind (Posner 1978), this method was applied to the separation of visual and name codes of a single letter. Differences in colour and brightness between stimuli influenced the time for visual matches but not for name matches, whereas rhyming and other auditory changes influenced name matches but not visual ones. [See the section on Haemodynamic imaging and Sternberg (2004) for the same method used with functional magnetic resonance imaging (fMRI).]
Another chronometric approach is to separate those processes that occur in serial order from those that can be carried out in parallel (Sigman and Dehaene 2005). This model assumes that sensory and motor processes can be carried out in parallel, but that a central workspace constitutes a bottleneck, such as has been studied by the psychological refractory period. The serial processes are fixed in time while the parallel processes vary in the time required for integration of input. In a simple arithmetic task, determining whether an input digit is above or below 5, the major variance in times is attributed to numerical distance from 5, whereas the other processes contribute little variance and can be carried out in parallel with a secondary task. In a further study (Sigman and Dehaene 2006), it was shown that even the central stage cannot be an entirely passive bottleneck but can be (p.210) employed strategically, based on features of the task (see also Meyer and Kieras 1997).
The link between speed and accuracy for rapid responses is a very close one. A given person may be slow and accurate or fast with lowered accuracy. This fact has been used to develop a method for tracing the time course of information processing based on error probability (Meyer et al. 1988). Subjects are given a task (e.g. ‘Was the word DOG in a recent study list?’). They are then required to respond even before they are ready. One method is to train them to respond in synchrony with a tone. Sometimes the tone comes well before the person has any idea of the correct answer, and their response may be at chance levels. The accuracy (often measured as d′, as taken from signal detection theory) is plotted as a function of time between the target and response signal. The resultant function shows the time to go from chance to perfect performance, and indicates that the time course of processing can be measured in terms of error rate. As RT and accuracy are both fundamental measures of performance the speed accuracy function provides a very fundamental way of linking them.
Chronometric methods have been used to test general models of cognitive processes. Major connectionist (Rumelhart and McClelland 1986) and symbolic (Meyer and Kieras 1997; Newell 1990) models of cognition have used RT and its derivatives to provide precise tests of models of human performance.
Although most of mental chronometry has involved the analysis of aspects of performance common to everyone, an important application has been to individual and group differences (Posner and Rueda 2002). Differences in RT in normal development and in a wide variety of mental disorders have been examined with the goal of developing diagnostic methods for the disorder or designing rehabilitative strategies. Although much of the work has used changes in overall RT as a measure, some studies have tried to locate the particular stage that might be damaged using additive factors methods. Recently it has also been possible to examine the genetic influences on these individual differences. For example, a number of dopamine and serotonin genes (Posner et al. 2007) have been shown to be related to a central executive control system measured by the degree of conflict in the Attention Network Task (Fan et al. 2002) that presumably involves some or all of the same anatomy discussed above as being related to the central bottleneck in dual-task performance (e.g. Sigman and Dehaene 2005).
Electrical potentials
In 1965, Samuel Sutton and colleagues (1965) reported a large positive wave in the scalp recorded event-related potential (ERP) at about 300 ms after input (p.211) that they called the P300, which was elicited by low probability or surprising events. This raised the possibility of using scalp electrodes as a physical basis for tracing information processes in the brain.
Over the ensuing years a number of components of the event-related cortical potential were shown to reflect psychological functions. With auditory stimuli it was possible to record components that proved to come from brainstem and thalamic generators (Mokotoff et al. 1977) as well as cortical potentials related to events that abruptly mismatched prior input (Naatanen et al. 1978). In the case of brainstem potentials, these influences could be well validated because they could be recorded from implanted electrodes in animals. The brainstem and cortical mismatch negativity can be used to examine auditory and language processing in infants and non-verbal organisms, making it possible to examine hearing and language very early in life.
When a visual stimulus is presented the first cortical potentials occur from striate areas at 50 ms after input and from pre-striate areas at 90 ms after input. When attention was drawn to the location of these stimuli within the visual field the electrical potentials were increased in size, suggesting that attention influenced visual or components at least by 80–90 ms after input. While striate potentials can also be influenced by attention, this appears to arise due to feedback from pre-striate areas (Martinez et al. 2001). These findings raised a difficulty for some serial models of RT because they showed clearly that attentional influence could change the amplitude of the ERP of early cortical potentials. Although the influence of attention was early in time, determination of the anatomy of these effects depended upon methods that could provide a better link with the anatomy (see below).
Another important development from electroencephalography (EEG) that influenced mental chronometry was the recording of ERPs linked to the motor response (Coles et al. 1995). The lateralized readiness potential (LRP) was recorded as a DC shift on the side of the brain opposite the responding hand. Recordings of the LRP allowed the researcher to trace the time course of the build-up of information about the response well before any motor output. One of the important findings was that often the conflict between multiple responses could be seen by a build-up of information on the wrong side that was replaced by the correct response. It was also possible to show that many psychological tasks involved the start of motor output in the LRP a long time before the sensory processing was over, and thus provided some support for a continuous transfer of information to output rather than a transfer only after the sensory stage was over.
Electrical recording was also able to trace brain activity associated with an error in responding. Within about 70 ms after an error, there was increased (p.212) negativity recorded above the frontal midline in comparison with the same response when correct (Bechtereva et al. 1990; Gehring et al. 1993). It was reported in the 1960s that people slowed following an error trial when they had either detected the error or received feedback that they had made an error (Rabbitt 1968). Rabbitt argued that error detection was computed at about the same time as the correct response. The EEG sign was confirmation of the rapid detection of error, and the finding that this influenced the next trial showed how important this self-monitoring was to human performance. The error negativity was one of the first components to be localized to a particular brain structure, the anterior cingulate gyrus, first by the use of algorithms that sought the best fitting generator (Dehaene et al. 1994) and also by experiments that used fMRI to localize the error effect (van Veen and Carter 2002). The linking of ERPs and haemodynamic imaging is discussed further in the final section of this chapter.
Haemodynamic imaging
In 1988 haemodynamic imaging became a tool for the localization of mental processes in brain tissue (Petersen et al. 1988). The use, first of positron emission tomography (PET) and later of fMRI, provided ample evidence that even in high-level cognitive tasks, such as word association, there was common localization of mental operations so that the activations could be averaged across subjects. Most of these studies were consistent with the view that cognitive and emotional tasks involve a small set of often widely scattered areas of activity, which must be brought together to perform the task.
The subtractive method of Donders, additive factors method of Sternberg, and independent manipulability methods discussed above in conjunction with RT have all had their counterparts in studies of imaging. The subtractive method applied to imaging is of special interest; although it depends on an assumption of linearity, it moves beyond RT both because one can make subtractions in both directions and because the effects of many different brain areas can be assayed. In most cases more complex activity brings in areas of activation in addition to the ones found for the simple control task. However, if the tasks are not well chosen, they may have quite different areas of activity. The ability to design a control task that includes most but not all of the components of the experimental task is difficult, because people may apply different strategies based on small changes between tasks. Despite these problems and the assumptions involved, the subtractive method has yielded relatively consistent results in many domains. The convergence between imaging, behavioural, and lesion methods suggests that, although many of the (p.213) criticisms of the subtractive method are correct the use of imaging can greatly enrich our understanding.
The findings of imaging in many different task domains are consistent with the idea that networks of specific neural areas underlie human abilities. Imaging methods have been developed to image the structural and functional connectivity between these brain areas. Diffusion tensor imaging is a method for viewing white matter connections between neural areas. Effective connectivity uses the correlations in activity among areas to establish the flow of information between them. Causal modelling can provide good hypotheses about the relative direction of information flow between active areas (Posner et al. 2006).
The time course of haemodynamic change is rather slow, lagging several seconds behind the neuronal activity whose activation it reflects. Methods have been developed to examine the time course of activity in these areas in the range of several tens or hundreds of milliseconds. Although this level of temporal resolution may not be sufficient for many cognitive tasks that differ by 100 ms or less, it has had important applications. When tasks are relatively slow it is possible to obtain a good account of the order of the various modules involved and for how long each of the modules is active (Formisano and Goebel 2003).
Combining methods
The network idea that stems originally from Hebb (for a discussion of this history see Posner and Rothbart 2007) calls for a chronometric account of the sequences of anatomical operations and their functions in accomplishing a cognitive task. At present this requires a combination of behavioural, electrical, and haemodynamic methods (Bledowski et al. 2006). The behavioural methods provide a task analysis in terms of the component operations involved and their timing; the haemodynamic images show which brain areas are involved and can suggest an order and direction of information flow; the EEG methods provide a detailed time course and can also test the temporal frequency of synchronization of the different neural area involved in performing the task.
Several methods have been designed to relate scalp signatures based on electrical recording to the underlying generators as found in fMRI studies (Pascualmarqui et al. 1994; Scherg and Picton 1991). In simple tasks, such as processing a click or flash, that involve only one or a very small number of generators, considerable validation of these algorithms has already taken place (Martinez et al. 2001). In their studies, Hillyard and colleagues have (p.214) systematically moved stimuli to various locations in the visual field and shown how the ERP and fMRI generators converge on similar areas of the primary visual cortex (Martinez et al. 2001). They found that the earliest primary visual cortex activation is not influenced by attention but that, later, attention does modify primary visual cortex activity via feedback from pre-striate regions. For more complex tasks the algorithms seemed to work when the subtraction used allows for only one or a very small number of widely separated generators to be active at any one time. When, for example, RT for reading a visual word aloud is subtracted from generating the use of the word, it has been possible to show that frontal (area 47) activity occurs about 200 ms after input and that Wernicke's area is active at about 500 ms, whereas the overt response does not occur until about 1100 ms (Abdullaev and Posner 1998) and coherence analysis using EEG suggests transfer of information from the frontal to posterior areas at about 450 ms (Nikolaev et al. 2001).
Many cognitive tasks involve extensive re-entry into a particular anatomical area at different temporal periods after input. To be able to tell how many times and when in the sequence a particular area is active can be important for interpreting the performance of the network (Bledowski et al. 2006; Debener et al. 2006; Posner et al. 2006). For tasks such as generating the use of a word, the long time taken to produce an answer provides ample opportunity for extensive re-entrant processes, illustrating how important temporal information is in understanding network function. Finding activity in the visual system for an auditory task may mean that there are direct connections between audition and vision that were not expected, or it may mean that higher centres act on the visual system by attention or by developing a visual image. In order to determine what is the case, one needs to know the order and timing of the anatomical modules involved. Thus, mental chronometry has gone beyond the measurement of RT to play an important role in understanding the neural networks that underlie human performance.
In the years ahead we may expect further application of mental chronometry to the analysis of neural networks. The degree of synthesis so far possible between behavioural, electrical–magnetic, and haemodynamic measures suggests that each will play an important role in this task. Moreover, it would be foolish to suppose that there will be no new methods at all levels of analysis. Improved links between human and animal models will also allow the chronometric paradigm to be pushed into more precise analysis, as cellular activity like EEG can be a source of precise temporal information (Pouget et al. 2005). The range of new methods and questions makes the achievements and future of mental chronometry even brighter than many of us would have thought half a century ago.
No comments:
Post a Comment