SyKronix.com - pharmaceutical marketing research portal
  | Home | About | Fair Use | Privacy | Contact | SyKronix.net services | SEARCH SyKronix
  | White Papers | MR Forum | SyKwest | mousEye usability | Tyman-Space Online College

TRACKING ATTENTION
DURING TELEVISION COMMERCIALS

--o--
How to Use the RT-probe Technique

Robert S. Owen, CET, Ph.D.
rsowen@sykronix.com
www.sykronix.com


The Implication of Attention Through Physical Measures

The implication of attention-related constructs through observable physical measures has a long history. Welch (1898), for example, constructed a device ("dynamograph") which could detect changes in the amount of force that a subject maintained on a hand grip. Changes in grip on this device would be magnified by a lever and recorded on a revolving drum. The chief source of error in maintaining a constant force on the device was found to be "lack of attention." The device could therefore be used to provide a physical measure that could be taken to imply the amount of attention that the subject was paying to some other task, or how this second task interfered with the ability to maintain a constant muscular force. Jastrow (1892) studied the ability of a subject to maintain finger tapping rhythm as a measure of the amount of mental processing required by some accompanying second task. Swift (1892) found that reaction times to a stimulus increased when in the presence of disturbing sounds, with differences in reaction time taken to suggest differences in the ability of the subject to maintain attention to some particular task. This is the fundamental basis of using the "RT-probe" in the measurement of attention-related constructs - that quantitative changes in the performance of one task (reaction time to the "secondary task" of an occasional beep or flash of light) can be taken to indicate qualitative changes in an attention-related process.

Attention Sharing and Attention Interference

The idea of the studies mentioned above was based on the general hypothesis that, "the falling away from clear consciousness must be due to interference and inhibition by other conscious elements" (Slaughter 1900-01). Many of the studies of attention at that time were "dual task" or "secondary task" studies (e.g., Bryan & Harter 1899; Downey & Anderson 1915; Jastrow 1892; Solomons & Stein 1896; Swift 1892; Welch 1898), interested in understanding why concurrently performed tasks seemed to interfere with each other less and less with continued practice. In all cases, a direct measure of mental processing could not be made; changes in "attention" and its related constructs were only implied through the measure of some other physically countable measure, such as changes in the ability to maintain the grip of a weight, changes in the ability to maintain a finger tapping pattern, or changes in reaction time to a secondary stimulus - all physical measures of an observable secondary task (see further discussion in Owen 1991).

Psychologists lost interest in research on "conscious" mental processing constructs such as attention to the study of behaviorism for a few decades, but renewed interest emerged again in the 1950s (e.g., Bahrick, Noble, & Fitts 1954; Broadbent 1954, 1957; Garvey & Knowles 1954; Garvey & Taylor 1959). Much research over the next three decades was focused on very practical issues associated with attention as a "processing capacity" sort of construct, addressing apparent processing limitations with respect to the work of military pilots (Gabriel & Burrows 1968; Gopher & Kahneman 1971; North & Gopher 1976), professional bus drivers (Brown 1968; Kahneman, Ben-Ishai, & Lotan 1973), and automobile drivers (Brown 1965, 1967; Brown, Tickner, & Simmonds 1969; Finkelman, Zeitlin, Filippi, & Friend 1977).

The past two decades of attention research has seen a return to the sorts of methods that were practiced a century earlier. In a manner similar to Jastrow (1892), Friedman, Polson, & Dafoe (1988) used the maintenance of finger-tapping patterns as a quantitative measure of attention to a reading task. Using the "RT-probe" technique of Posner & Boies (1971; see discussion of this method in Owen, Lord, & Cooper 1995), whereby attention to a task is taken as a quantitative measure of the speed of reaction time to an occasional flash of light or a beep or a clicking sound, marketers have measured attention to television advertising (Lord & Burnkrant 1993; Lord, Burnkrant & Owen 1989; Thorson, Reeves, & Schleuder 1985, 1987), audio advertising (Moore, Hausknecht, & Thamodarin 1986), and magazine advertising (Unnava & Burnkrant 1991). (The latter study used equipment and programs constructed by the present author.)

Methods used in the studies of the above discussion have relied primarily on a capacity or interference sort of definition of attention. An increase in the amount of "attention" (or "processing effort," or "involvement," or other such attention-related constructs) that is being devoted to the reading of a passage of text is taken from, say, a quantitative measure of changes in concurrent finger-tapping performance; an increase in "attention" that is being devoted to a television show can be taken from a quantitative measure of changes in reaction times to an occasional "beep" sound during the show. Lord & Burnkrant (1993), for example, used the "RT-probe" technique (constructed by the present author) to find apparently "high involvement" and "low involvement" segments within a suspenseful television program. Longer RTs (reaction times) to an occasional audible beep during the show were taken as an indication of higher viewer "involvement" with particular points within the program story.

How the RT-Probe is Used

In using the RT-probe technique, subjects are asked to press a hand-held button in response to an occasional flash of light or an audible click or beep sound. An increase in reaction time (RT) to an occasional beep (the secondary task) while reading an advertisement or watching a television commercial (the primary task) is taken as a quantitative measure of an increase in attention or mental effort being devoted to the primary task. Points where Rts dramatically increase above normal are taken to indicate that the primary task is consuming attentional resources approaching the threshold of processing system capacity. Changes in secondary task performance, then function as a probe into attentional resource consumption by the primary task.

Lord, Burnkrant, & Owen (1989) found that longer Rts continued throughout television advertisements embedded within high involvement segments, but remained normal throughout the same advertisements positioned within low involvement segments. This measure was taken to suggest, then, that processing resources were still being consumed by program elaboration throughout advertisements positioned within high involvement program segments. After the program was shown to subjects, attitude and recall measures were taken wth regard to the target commercials. The results suggest that there was indeed some processing interference associated with the commercials positioned within the high involvement program segments. The RT probe technique was used, then, to first identify parts of the television program that were high and low involvement. After these segments were identified, commercials were placed in the high and low segments. The RT probe technique was then used to probe the level of processing activity during the commercials, with the finding commercials placed in the high involvement segments were affected by elaboration about the program.

Making Stimuli, Collecting Data, and Analyzing Data in the Lord Studies

A computer was used to dub the "beep" stimuli onto the television show. These beeps, a pleasant "bing" sound, had to occur at random intervals to ensure that subjects did not become accustomed to a regular rhythm. The computer generated beeps that were separated by three to nine seconds, for an average of ten beeps per minute. They were recorded on one of the two stereo tracks so that on playback, the computer only heard the beep track. The two tracks were mixed to be played through the single TV speaker to the subjects. On playback of the video tape to subjects, a computer received the beeps and received switch button inputs from each of ten subjects. Subjects were to press a hand held thumb switch whenever they heard the beep sound while watching the show. The computer began counting beeps from the start of the show; thereby identifying the position of the tape. At the onset of each beep, a millisecond resolution counter was started. Whenever a switch button press was received from subjects, the subject identification number, time on the millisecond counter, and beep number were saved to the data file.

Of interest in analysis is the identification of points in the data where subjects have relatively long reaction times. A subject will normally respond to the beep stimuli in around 250-300 milliseconds, and these response times will be relatively consistent (i.e., within that range). During the portions of the show when greater processing resources are being consumed (presumed to represent the construct, "high involvement"), the data will show markedly longer reaction times, around 1500 milliseconds, and these reaction times will be very inconsistent. Additionally, there will be many "misses," when the subject does not press the switch button at all, and there will be many "false alarms," when the subject presses the button a second time before any beep occurred. The differences are so marked that no special analysis is required - these points can be clearly seen by simply eyeballing the data. (This is a testament to just how limited is human processing capacity; it is relatively easily swamped, making the R-probe technique possible.)

Some Cautions

The RT-probe technique is a low-cost, easy to use method to detect attention related constructs during a variety of tasks. It can be used with the reading of text, with listening to the radio, with watching TV, or even while viewing a Web browser. We must be cautious in using it on a number of fronts, however. One important issue in using it has to do with its underlying theoretical assumptions. Its use presumes the existence of a single, general purpose, limited capacity processor in the human processing system, and assumption is not true (see further discussion of this and related issues in Owen 1991). We have to be careful that the primary task and the secondary task are tapping the same limited resource, whatever that might even be. Another important problem has to do with the instrumentation that is used to take the physical measurements.

A problem with using computers to collect reaction time data is that researchers are emboldened whenever numerical data is returned by the machine. People who might normally be very cautious and meticulous with regard to issues of validity and reliability seem to cast all caution aside whenever physical measures are taken to imply the detection mental constructs. If a computer returns data that a subject reacted to a stimulus in 130 milliseconds, what is the validity and reliability associated with that physical measure? There are a number of issues associated with the latencies of the response switch, with starting a timer precisely with the onset of stimuli, and such, but the biggest problem has to do with the resolution of the counter that is used to keep time. The Lord study used a counter that could keep time with a resolution of around ten microseconds, so that all sources of random and systematic error, contained wholly within the last significant digit, were dropped with the significant digits that were tossed in recording time in milliseconds. Unfortunately, most examples that can be found in the literature used counters (timers) which were much more crude.

Most studies that are reported have used the interrupt timers on an IBM-PC platform or on a Macintosh platform. The interrupt timer on a Mac ticks at a rate of nominally 60 times per second; the interrupt timer on a PC ticks at a rate of nominally 18.2 times per second. When generating data, however, many researchers are using methods that report results in decimalized form to three significant digits. This gives the impression that millisecond level resolution is being used, when in fact the resolution is 55 times more crude on a PC (1/18.2 seconds = 55 milliseconds). (For an example of this issue, see the discussion of the Javascript and Java timers elsewhere on this site; see Owen & Cooper 1990 for technical discussion.) Although such resolution is acceptable for some studies (if reported as ticks, rather than falsely in decimalized form), it is far too crude for detecting differences in reaction times that are less than one unit of resolution (again, see Owen & Cooper 1991 for reasons as to why averaging across a large sample size does not solve this problem).

The author has found examples in the literature in which controversial results could be explained by this problem. In one article submitted for editorial review, all three reviewers demanded that examples be added to support such a claim. Examples were given, including two studies that had reported opposite results by the same researcher - resulting in further research by others in an attempt to explain these results. Although the two studies reported results with millisecond resolution, a third article reported on methods used to collect data using an interrupt counter, suggesting an important source error in backlash, or "slop," due to how close actual responses might be to the front or back of a single tick of resolution. When examples were included in the revision of the article, the editor wrote back after a very long period of time that two of the reviewers would not respond to the revised article which included examples as requested. Oops! (Unfortunately, I was a doctoral student trying to get through comps at the time, and never got around to submitting a less embarrassing revision.)

If you collect data using a "tick" counter, then analyze data as raw ticks. Note that such a level of resolution is not necessarily a limitation. Moore, Housknecht, & Thamodaran (1986), for example, used the sixtieth-second timer of an Apple computer in studying attention to audio commercials. Be very careful of using any application or programming language which returns timer data in decimal form; run a loop to ensure that the device is capable of incrementing a single millisecond at a time. See, for example, the Javascript loop on this site that will return different resolutions depending on whether the client machine is a Macintosh, a PC running MS Windows, or is a PC running Linux. Relevant reviews of limitations in resolution when using higher level languages and when using monitors and keyboards as stimulus and response devices relevant to using microcomputers can be found in Beringer 1992; Gabrielsson & Jarvella 1990; Owen & Cooper 1990; and Segalowitz & Graves 1990.

References

Bahrick, Harry P., Merrill Noble, & Paul M. Fitts (1954), "Extra-task Performance as a Measure of Learning a Primary Task, "Journal of Experimental Psychology, 48(4), 298-302.

Berringer, Jorg (1992), "Timing Accuracy of Mouse Response Registration on the IBM Microcomputer Family," Behavior Research Methods, Instruments, & Computers, 24(3), 486-490.

Broadbent, D.E., (1954), "The Role of Localization in Attention and Memory Span," Journal of Experimental Psychology, 47, 191-196.

__________ (1957), "A Mechanical Model for Human Attention and Immediate Memory," Psychological Review, 64, 205-215,

Brown, I.D. (1967), "Measurement of Control Skills, Vigilance, and Performance on a Subsidiary Task during 12 Hours of Car Driving," Ergonomics, 10, 665-673.

__________ (1967), "Effect of a Car Radio on Driving in Traffic," Ergonomics, 8,475-479.

__________, A.H. Tickner, & D.C. Simmonds (1969), "Interference Between Concurrent Tasks of Driving and Telephoning," Journal of Applied Psychology, 53, 419-424.

Bryan, W.L. & N. Harter (1899), "Studies on the Telegraphic Language. The Acquisition of a Heirarchy of Habits," Psychological Review, 6, 345-375.

Downey, J.E., & J.E. Anderson (1915), "Automatic Writing," The American Journal of Experimental Psychology, 26, 161-19

Finkelman, Jay M., Lawrence R. Zeitlin, John A. Filippi, & Michael A. Friend (1977), "Noise and Driver Performance," Journal of Applied Psychology, 62, 713-718.

Friedman, Alinda, Martha Campbell Polson, & Cameron G. Dafoe (1988), "Divided Attention Between the Hands and Head: Performance Trade-offs Between Rapid Finger Tapping and Verbal Memory," Journal of Experimental Psychology: Human Perception and Performance, 14, 60-68.

Gabriel, Richard & Alan A. Burrows (1968), "Improving Time-sharing Performance of Pilots Through Training," Human Factors, 10, 33-40.

Gabrielsson, Jan & Robert J. Jarvella (1990), "How to Obtain Millisecond Precision on the IBM PC with Visual Stimuli Not Exceeding One Printed Line," Behavior Research Methods, Instruments, & Computers, 22(6), 520-525.

Garvey, W.D. & W.B. Knowles (1954), "Response Time Pattern Associated with Various Display-control Relationships," Journal of Experimental Psychology, 47, 315-322.

__________ & F.V. Taylor (1959), "Interaction Among Operator Variables, System Dynamics, and Task-induced Stress," Journal of Applied Psychology, 43, 79-85.

Gopher, Daniel & Daniel Kahneman (1971), "Individual Differences in Attention and the Prediction of Flight Criteria," Perception and Motor Skills, 33, 1335-1342.

Jastrow, Joseph (1892), "The Interference of Mental Processes - A Preliminary Survey," American Journal of Psychology, 4, 219-223.

Kahneman, Daniel, Rachen Ben-Ishai, & Michael Lotan (1973), "Relation of a Test of Attention to Road Accidents," Journal of Applied Psychology, 58, 113-115.

Lord, Kenneth R. & Robert E. Burnkrant (1993), "Attention Versus Distraction: The Interactive Effect of Program Involvement and Attentional Devices on Commercial Processing," Journal of Advertising, 22(March), 47-60.

__________, __________, & Robert S. Owen (1989), "An Experimental Comparison of Self-report and Response Time Measures of Information Processing," in AMA Educators' Proceedings: Enhancing Knowledge Development in Marketing, Paul Bloom et al., eds. Chicago: American Marketing Association, 196-200.

Moore, Danny L., Douglas Hausknecht, & Kanchana Thamodarin (1986), "Time Compression Response Opportunity in Persuasion," Journal of Consumer Research, 13(June), 85-99.

North, Robert A. & Daniel Gopher (1976), "Measures of Attention as Predictors of Flight Performance," Human Factors, 18, 1-14.

Owen, Robert S. (1991), "Clarifying the Simple Assumption of the Secondary Task Technique," Advances in Consumer Research, 18, 552-557.

__________, Kenneth R. Lord, & Martha C. Cooper (1995), "Using Computerized Response Time Measurement for Detecting Secondary Distractions in Advertising," Advances in Consumer Research, 22, 84-88.

Posner, Michael I. & Stephen J. Boies (1971), "The Components of Attention," Psychological Review, 78(5), 391-408.

Segalowitz, Sidney J. & Roger E. Graves (1990), "Suitability of the IBM XT, AT, and PS/2 Keyboard, Mouse, and Game Port as Response Devices in Reaction Time Paradigms," Behavior Research Methods, Instruments, & Computers, 22(3), 283-289.

Slaughter, J.W. (1900-01), "The Fluctuations of Attention in Some of their Psychological Relations," American Journal of Psychology, 12, 313-334.

Solomons, L. & G. Stein (1896), "Normal Motor Automatism," Psychological Review, 3, 492-512.

Swift, Edgar James (1892), "Disturbance of the Attention During Simple Mental Processes," American Journal of Psychology 5(1), 1-19.

Unnava, H. Rao & Robert E. Burnkrant (1991), "Effects of Repeating Varied Ad Executions on Brand Name Memory," Journal of Marketing Research, 28(November), 406-416.

Welch, Jeanette C. (1898), "On the Measurement of Mental Activity through Muscular Activity and the Determination of a Constant of Attention," American Journal of Physiology, 1(3), 283-306.



  See my vita for a listing of research publications with links to abstracts.