Attention

Goals

Today we will discuss a psychological topic: attention. In this lecture, I consider three aspects of attention.

• Psychological models of attention and the history of their development.
• Physiological studies of attention that have employed Event Related Potentials (ERPs) and fMRI.
• The emerging role of the thalamus in the deployment of attention.
• The anatomy of attention and attentional control systems.

This lecture depends upon a group of methods used to measure human brain activity, including scalp-recorded EEG (electroencephalogram), ERPs (event-related potentials), MEG (magnetoencephalography), and functional MRI (fMRI). You should review the notes I have prepared on these techniques before lecture.

Topic slide

Steven Hillyard (b. 1942) received his Ph.D. in Psychology at Yale. He, and his many students and collaborators, has studied visual and auditory attention using ERPs (event-related potentials, or evoked potentials) and fMRI.

Anne Triesman (1935-2018) was a professor at Princeton University and long has been a leading figure in experimental psychological studies of attention. She is perhaps best known for her work on Feature Integration Theory. Professor Triesman died in 2018, you can read about her life in this New York Times obituary.

Reading

• Reading: Chapter 29
• Review lecture notes 03.02

History

Most discussions of attention begin with the presumption that there is a limitation (or 'bottleneck') in the brain's processing capacity, i.e., not every sensation engaging the senses can be fully processed. Attention suggests that resources can be strategically deployed (voluntary attention) or summoned by an exogenous cue (involuntary attention) to improve processing of some items at the cost of processing other items. Attention, thus, has both costs and benefits.

Some theorists consider circumstances for which attention is required or not required. Preattentive processes are early perceptual processes that occur in an automatic or obligatory manner that are presumed not to require attention, and are distinguished from perceptual processes that do require attention to be accomplished. This distinction is most notable in Triesman's Feature Integration Theory which will be discussed further below.

A brief review of major historical concepts in attention

The polymath Hermann von Helmholtz performed one of the first studies of attention. He showed that individuals could accurately report the identity of a small group of letters within a much larger array of letters when attention was covertly drawn to the location of the small letter group. However, they could not recall those letters when attention was not directed to that location.

Marisa Carrasco reviewed the field of visual attention in 2011. Here are her comments on Helmholtz:

Hermann von Helmholtz is considered to be the first scientist to provide an experimental demonstration of covert attention (ca. 1860) (Helmholtz, 1896), cited in (Nakayama & Mackeben, 1989). Looking into a wooden box through two pinholes, Helmholtz would attend to a particular region of his visual field (without moving his eyes in that direction). When a spark was lit to briefly illuminate the box, he found he got an impression of only the objects in the region he had been attending to, thus showing that attention could be deployed independently of eye position and accommodation.

Early selection

British investigators in the 1950-60s including Cherry, Broadbent, Triesman, and others used dichotic listening and shadowing tasks to study attention. A subject would hear simultaneous messages in the left and right ear, and (in a shadowing task) repeat the message in one ear only. Experiments showed that the subject learned nothing about the message in the non-shadowed (non-attended) ear.

• These studies emphasized a bottle-neck of attention – at some point, stimuli that were not-attended do not pass through the bottleneck. This was the basis of filter theory.
• The filter was presumed to occur early in processing, at the perceptual level, and so it was also known as an early-selection theory.
• Triesman showed that some highly salient stimuli (such as one’s name) did get through on the non-shadowed message, suggesting that the filter was not perfect.
  • Triesman proposed a filter-attenuation theory, whereby stimuli in an unattended channel were attenuated, but not completely eliminated. Highly salient stimuli were able to penetrate the filter.

Late Selection

Deutsch and Deutsch proposed a model whereby stimuli were all processed at a perceptual level, but then the selection occurred at a later stage when responses were selected. In their view, you could process many sources of information, but you could only make a single response. Thus, the bottle-neck was in responding to stimuli. This was a late-selection theory. A comparison of early and late models of attention can be found here.

Top down, Bottom up

Most contemporary theories do not insist of attention occurring early or late, but rather posit that attention can operate at many different processing levels.

A distinction is often drawn between top-down and bottom-up modes of attention. Top-down is roughly analogous to voluntary attention – you direct your attention voluntarily to the task on-hand. Bottom-up is roughly analogous to involuntary attention – something captures or commands your attention to a location or object (such as a flash of light, or change in motion).

Feature Integration Theory

Triesman and her collaborators considered the distinction between perceptual processes that are automatic, or pre-attentive (acting before attention is deployed), and those processes that require attention. Triesman used a visual search paradigm to show that pre-attentive features (such as color, e.g., a green 'X' among a field of red 'X's, or form, such as a 'S' among a field of 'E's) can 'pop out' of backgrounds. Pop-out means that the time to detect target items in different colors or form are independent of the number of items present in the display. This means that the subject does not exhaustively examine each item in the display to determine if it is the target. The process of feature detection is presumed to be automatic and to not require attention.

However, if a subject is required to detect conjunctions of features (a single green 'X' among a field of field of red 'X's and red and green 'O's, the task is much more difficult and search time is dependent upon the number of elements of the display.

Here is an example of two visual displays used by Triesman that illustrate automatic feature search on the left, and effortful (and attention demanding) conjunctive search on the right.

In her Feature Integration Theory, Triesman argued that attention was required to bind features together into an object. Without attention, the color and forms present in the display were not bound together into an object. Other studies demonstrated the occurrence of illusory conjunctions, whereby the subject reported seeing letters or shapes in the wrong colors.

Physiological studies of attention.

Event-related potentials (ERPs)

Most early neuroscience studies of attention focused on cognitive models then in currency – i.e., was there an attentional ‘filter’ and did it operate ‘early’ or ‘late’. Many of these early studies used neurophysiological recording in humans and animals. The first fMRI studies were performed in 1992, so this was not possible. However, even it if had, fMRI doesn’t have the temporal resolution necessary for discriminating early from late.

Hernandez-Peon showed (in many different experiments) that evoked potentials to irrelevant stimuli were greatly reduced when an animal was attending to a highly salient stimulus.

• I provided an example of a cat attending to mouse in jar. An auditory evoked-response was reduced when the cat attended to the mouse.
• Hernandez-Peon's studies did not employ rigorous controls (e.g., perhaps the cat turned its head away from the appropriately for head movements of the cat

The 'Hillyard Task'

The ‘Hillyard’ task has been used in numerous attention studies, particularly those in which ERPs are measured, but also in fMRI. The task requires the subject to attend to one visual field for an entire run consisting of many stimuli. It can be used with stimuli in any modality.

In a typical study using visual stimuli, the subject is asked to fixate the center of a screen while small flashes of light (the 'standards') appear asynchronously and at high presentation rates (1-2 stimuli per sec) in the left and right VF. The subject is asked to detect the occurrence of a 'target' stimulus which might be a slightly dimmer flash, which occurs very infrequently. On different runs, the subject attends the left VF or the right VF.

The critical comparison occurs across runs – for example, the ERPs evoked by standard flashes presented to the left VF when the left VF was attended will be compared to the ERPs evoked by standard flashes in the left VF when the right VF was attended. That is, ERPs are compared for identical stimuli – only the direction of attention is different.

The typical results of the Hillyard task in the visual modality is as follows:

• The visual ERP is larger for attended standard flashes than non-attended standard flashes. This is most evident at the ‘P1’ visual ERP that occurs at about 100 msec after flash onset, but later ERPs are similarly affected.
• This result establishes that voluntary attention influences visual processing as early as 100 msec. This result was interpreted as evidence for early selection.

The Hillyard task has also been used extensively in auditory evoked potential studies. The task is very similar to the visual task, just substitute left or right ear for left or right VF. The effects of attention are significant at about 20-50 msec after tone onset. Attentional effects are also very prominent at 100 msec (N100). These auditory results even more strongly support an early selection component to attention, as an auditory stimulus is just reaching auditory cortex at 20 msec.

The 'Posner Task'

ERPs have also been measured during the ‘Posner’ attentional-cuing task. This task can be modified to study both voluntary (top-down) and involuntary (bottom-up) attention.

Voluntary Attention variation of Posner task

In this version of the Posner task, a central cue orients attention to the left or right visual field. A target then appears in the attended field (a 'valid target') or in the unattended field (an 'invalid target').

• Reaction times decrease and detection accuracies increase for valid targets. This is the benefit of attention.
• Reaction times increase and detection accuracies decrease for invalid targets, when measured against a neutral cue condition that does not direct attention to either VF. This is the cost of attention.
• Similar results are found for ERPs in the Posner task as in the Hillyard task.

Involuntary Attention variation of Posner task

The Posner task can be modified to study involuntary attention. Here, the attention cue is NOT a central arrow, but rather a brief presentation of an outlined box (or some similar stimulus) in the left or right VF with equal probability. A target stimulus then occurs within the box previously presented (valid) was compared to a target stimulus presented in opposite VF in which the box was NOT presented. In this modified task, the subject’s attention is involuntarily drawn to the site of the flashed box.

When studied with ERPs, the results have been as follows:

• The visual ERPs show a P100 enhancement to the target appearing in the attended compared to unattended location. This is similar to the result of the Hillyard task.
• However, this P100 enhancement only occurs for short intervals between the cue and target. With longer intervals, the effect reverses and the unattended target evokes a larger P1.
  • This reflects the phenomena of Inhibition of Return (IOR). The involuntary shift of attention to a location in space only lasts for a short while, and then there is a cost to returning to that previously attended location. Presumably, this makes visual search more efficient (i.e., you stop returning to look where you’ve already looked).

Mismatch negativity (MMN)

Mismatch negativity (MMN) is another ERP related to attention. MMN is evoked by a dissimilar stimulus among a train of similar stimuli, when attention is NOT directed to that stimulus train. It is similar to an involuntary shift of attention, but the involuntary shift is caused by a stimulus that deviates from the sequence of stimuli that preceded it. It is usually studied in the auditory modality (indeed, there is some controversy as to whether MMN exists in the visual modality).

Studies of inter-modal (i.e., between two or more modalities) attention performed in my lab have used MMN in a visual-motor tracking tasks that can be made difficult or easy. While performing the task, the subject hears a train of irrelevant auditory stimuli, with an occasional ‘deviant’ (different tone pitch). When visual-motor tracking is easy, the deviant tone evokes a large MMN. When the visual-motor tracking task is difficult, and presumably consumes more attentional resources, the deviant tone evokes a small MMN. This result suggests that at least some attentional resources are fungible – can be deployed across different modalities.

This study also showed that auditory cortex activation, as measured by fMRI, was reduced when the subject was engaged in the difficult visual-motor tracking task. This suggests that attentional resources are 'fungible' and that the involuntary allocation of attention to irrelevant auditory tones is reduced when visual processing demands are increased.

Attention studies of thalamus

As we have discussed, each sensory modality (excepting olfaction) routes sensory information from peripheral receptors (and some intermediary nuclei) through a sensory-specific thalamic nucleus to the primary sensory cortex. For example, the visual thalamus (LGN, or lateral geniculate nucleus) projects to layer 4 of primary visual cortex (V1). These are feed forward projections.

However, about 30% of all input to the LGN are feedback projections from layer 6 of primary visual cortex (V1). Indeed, perhaps as little as 5-10% of LGN input is actually from the optic nerve. The rest comes from V1, thalamic reticular nuclei, brainstem projections, LGN interneurons, etc.

• I provided an example in lecture showing responses of LGN neurons with and without V1 feedback. Feedback ‘sharpened’ the responses.

Thus, control of sensory input to the LGN, and other sensory thalamic nuclei, through feedback and other non-sensory projections can influence sensory input – perhaps this is one mechanism of attention.

Studies using a Hillyard Task adapted from fMRI have shown increased activation of the lateral geniculate nucleus (LGN – visual thalamus) corresponding to the attended VF. This suggests that an early selection attentional filter might operate in some simple attentional studies.

However, with fMRI, one is never sure of when the activation occurred in the sequence of neuronal events. It is possible that the LGN effects occurred as a consequence of feedback from higher visual areas (such as V1, or V4). This is the concept referred to as re-entrance.

Rodent studies of thalamic control of attention

An intriguing series of studies have recently been published from Michael Halassa’s MIT lab concerned with thalamic control of attention. These studies are remarkable in that they utilize optogenetics to isolate the neurons involved in attentional control.

In the first of these studies, Wimmer and colleagues showed that prefrontal cortex (PFC) controlled attention in mice through its control of sensory cortices, and that this control was exerted through PFC influence over sensory thalamus. By using optogenetic methods to disrupt PFC control, they further demonstrated that disruption caused mice to be unable to select between the competing visual and auditory modalities. The PFC control appeared to be exerted through the visual thalamic reticular nucleus, which in turn affected the ‘gain’ of visual thalamic input to cortex by feedforward inhibition.

In a subsequent study from this same lab, Schmitt and colleagues showed that dorsomedial thalamus (which is connected widely to PFC) plays an important role the imposition of ‘rules’ for processing by amplifying local PFC connectivity. We will return to this study in my next lecture on executive processing and the frontal lobes.

FMRI studies of attention in Auditory Cortex

I described a recent study where tonotopic regions of A1 (primary auditory cortex) were identified, and voxels (volume elements of the image) were identified that were responsive to two different frequencies. The fMRI activation was increased in the attended frequency, but not the unattended frequency. This provides more evidence for an early selection effect of attention.

Attention to stimulus categories

Thus far, we have considered attention to simple stimulus qualities such as intensity, frequency, and spatial location. A fMRI study used a more complex stimulus consisting of a superimposition of a face and a house. When the face was attended, the fusiform face area showed increased activation. When the house was attended, the parahippocampal place area showed increased activation. This study is a good demonstration that attention can be deployed to categories of stimuli and that attention engages category-specific regions of cortex.

Anatomy of attention.

Hemispatial Neglect

Patients with hemispatial neglect frequently have lesions in the right temporoparietal junction (TPJ) including the right angular gyrus. Hemispatial neglect has elements of agnosia, and I introduced this topic two lectures ago using the example of a simultanagnosic who could easily report when a single object was present, but not when two objects were present. That is, the patient had awareness of one object at a time. This phenomenon has been interpreted as a fundamental deficit in attention.

• If the two objects are joined together (e.g., two balls are connected by a line and thus become a barbell), then the simultanagnosic can see this entire joined object.

Patients with hemispatial neglect from right hemisphere lesions may show the following:

• Cancel lines on the right side of a page, but not the left side
• Draw a clock with all of the numbers crammed into the right side
• Draw a picture (or self-portrait) and leave out most of the left side
• Report accurate spatial memories on the right side of a scene (the Italian Piazza study), but not the left. If they are asked to then reposition themselves across the piazza, they now report details for the right side of the scene (that were previously unreported when those details were on the left side from the original perspective).
• Show simultaneous extinction
  • This is a roughly synonymous for simultanaganosia itself – e.g., a patient will report a single object in the good or ipsilesional hemifield (right) and in the bad or contralesional (left) hemifield. However, when both items are presented, the item in the bad (left) hemifield is not reported.

There are other syndromes that are frequently comorbid with hemispatial neglect:

• Anosagnosia: Denial of illness, in the case of hemispatial neglect – it refers to denial of the paralysis of (usually) left side. Patients will sometimes confabulate.
• Somatoparaphrenia: Denial that the left side hand or arm is his/her own hand or arm. Patient may explain that it belongs to another. “It was left here by my brother when he was visiting.” This bizarre behavior usually does not persist beyond the acute stage after the stroke. However, it is an interesting parallel to the idea of the ‘interpreter’ that we discussed with respect to split-brain patients.
  • I read example case studies of somatoparaphrenia.

Experimental studies of hemispatial neglect using the Posner attentional task

Studies of individuals with hemispatial neglect show that there is a large cost (relative to healthy controls) in disengaging attention when cued to the good, or ipsilesional, hemifield and moving it to the bad, or contralesional, field. So, perhaps patients with hemispatial neglect have difficulty in disengaging from the good VF.

Feature Integration Theory and Hemispatial Neglect

In 1991, Cohen and Rafal tested a patient with hemispatial neglect on a simplified version of Triesman's letter/color task to test one of the basic tenants of Feature Integration Theory. Cohen and Rafal reasoned that if attention is required to bind together color and form features into an object, then a subject who could not allocate attention to her contralesional field would report many illusory conjunctions to color/letter stimuli appearing in her contralesional compared to ipsilesional visual field. Their results confirmed this prediction.

Attentional control networks

FMRI studies by Hopfinger and Mangun compared activation patterns in voluntary and involuntary attentional cuing paradigms (i.e., Posner tasks). Remember that we already discussed that both tasks evoke a visual ERP difference at P1 (100 msec). Here, the focus was on the activity evoked by the attentional cues themselves (and not the targets per se). These studies showed that cues evoke widespread activity in visual and other cortices with a large activation in inferior parietal lobe (IPL) and dorsolateral prefrontal cortex (DLPFC).

Corbetta synthesized the work of Mangun and Hopfinger with considerable work by himself and others, and proposed the existence of two attentional networks. These networks are predominantly in the right hemisphere.

The dorsal attentional network is presumed activated when attention is deployed voluntarily. It incorporates the inferior parietal lobe (IPL) and a part of the frontal lobe called the frontal eye fields (FEF). This is similar to what Hopfinger described above.

The ventral attentional network is activated by novel or unexpected stimuli – i.e., it is activated when exogenous stimuli attract attention. Corbetta used the metaphor of a circuit breaker – this attentional network disrupts on-going processing when unexpected novel or salient events occur in the environment.

The ventral attentional network is located more inferiorly (or ventral) to the dorsal attentional system. It includes the temporal-parietal region and the lateral inferior frontal region.

• The ventral system corresponds roughly to where lesions (in the right hemisphere) causes hemispatial neglect.

Videos

Prerecorded lecture for Fall 2020

Previously recorded live lectures

The lecture embedded below was recorded fall, 2019, and closely adheres to the notes from above. (Note that the title slide incorrectly states 2018F).

The lecture embedded below was recorded in fall, 2018.