Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The Pupillary Response to the Unknown: Novelty Versus Familiarity:

The Pupillary Response to the Unknown: Novelty Versus Familiarity: Object recognition is a type of perception that enables observers to recognize familiar shapes and categorize them into real-world identities. In this preregistered study, we aimed to determine wheth- er pupil size changes occur during the perception and recognition of identifiable objects. We com- pared pupil size changes for familiar objects, nonobjects, and random noise. Nonobjects and noise produced greater pupil dilation than familiar objects. Contrary to previous evidence showing greater pupil dilation to stimuli with more perceptual and affective content, these results indicate a greater pupil dilation to stimuli that are unidentifiable. This is consistent with the relative salience of novelty compared to familiarity at the physiological level driving the pupil response. Keywords object, form, recognition, pupillometry, pupil, dilation, constriction, novelty, familiarity Date received: 4 April 2019; accepted: 12 August 2019 Corresponding author: Steve Beukema, McGill University, 845 Sherbrooke St W, Montreal, QC H3A 0G4, Canada. Email: steve.beukema@mail.mcgill.ca Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 i-Perception 10(5) Introduction Pupil size changes are reflexive; the pupil not only grows and shrinks unconsciously in response to varying levels of light (Ellis, 1981), but it also changes in size depending on the content of the stimulus. This cognitive influence on the pupillary response has been long established (Beatty & Lucero-Wagoner, 2000; Loewenfeld, 1958), but its precise nature is less clear (Mathoˆ t, 2018). In this article, we focus on the pupillary responses to object recogni- tion, in order to deepen our understanding of the cognitive influences on the pupillary response as well as on object recognition itself. Object recognition is an important part of human perception, necessary not only for determining the content of the visual world but also to establish links between familiar identities in our past, present, and future. One issue that engages researchers using pupill- ometry to study object recognition concerns the process of distinguishing familiar from nonfamiliar objects (DiCarlo, Yoccolan, & Rust, 2012; Grill-Spector, 2003; Kafkas & Montaldi, 2015). For example, Kafkas and Montaldi (2015) showed subjects a range of familiar objects during a training phase and then measured the pupil responses to a set of test objects, some of which had been presented in the training phase. They found that the test objects that had been seen during the training phase produced a greater pupil dilation than the ones not previously seen. In a similar study by Naber, Frassle, Rutishauser, and Einhauser (2013), subjects viewed images of everyday scenes and were asked to memorize half of them. After a distraction phase, the images were presented again but intermixed with novel images, and the subjects indicated whether the images were familiar or novel and how confident they were in their judgement. Consistent with the results of Kafkas and Montaldi (2015), a greater pupil dilation was observed for the memorized scenes, while the novel scenes produced a pupil constriction that became larger as a function of confidence. Unlike Kafkas and Montaldi (2015), however, Naber et al. (2013) tested the pupil response during both retrieval and encoding. Interestingly, a greater constriction response was observed during the encoding phase for scenes that were later remembered compared to the scenes that were later forgotten. Then, during the retrieval phase, the novel (i.e., unfa- miliar) scenes produced a greater constriction compared to the familiar (i.e., remembered) scenes. The authors thus showed that under the task of retrieving a memory, stimulus novelty could produce a pupil constriction not dilation. This highlights not only a potential for separate pupil mechanisms during encoding versus retrieval but also the importance of the particular experimental context for determining whether pupil constriction or dila- tion occurs. Although both the Kafkas and Montaldi (2015) and Naber et al. (2013) studies mentioned earlier are consistent in showing greater pupil dilation to familiarity during an encoding task, there are exceptions to this finding. In Laeng et al. (2007), three patients were tested using a similar memory paradigm as the aforementioned studies—with encoding and retrieval phases. Interestingly, even though the patients lacked explicit awareness of having seen the images before (failure of encoding due to hippocampal lesions), they still produced a greater pupil dilation to the novel foil stimuli during retrieval compared to previously seen stimuli, contrary to the pupillary response observed in the studies examining healthy partic- ipants. This further supports the notion that separate neural processes underpin encoding versus retrieval, and if damaged, the pupil will respond accordingly based only on the infor- mation sustained by the remaining pathways. Many other studies have shown that stimulus novelty is capable of inducing a pupil dilation response (Ferrari et al., 2016; Kamp & Donchin, 2015; Kloosterman et al., 2015; Liao, Yoneya, Kidani, Kashino, & Furukawa, 2016). For example, Ferrari et al. (2016) Beukema et al. 3 found a greater dilation response for emotional compared to neutral natural scenes but only when they were novel; emotional content did not induce a greater pupil dilation when the scenes were repeatedly presented during the experiment. This highlights how a physiological response to novelty versus familiarity can be affected by other factors, such as arousal. In two studies concerning illusory phenomena, Kloosterman et al. (2015) and Beukema, Olson, Jennings, and Kingdom (2017) show that physically present and static stimuli can cause a pupil dilation when the images are perceived to vanish or move respectively, contrary to the physical reality. The perception of illusory phenomena is a novel perception that has close ties with the concept of surprise—the betrayal of an individual’s level of certainty. Two studies that explore this concept more closely employ a gambling task while measuring participants’ pupil responses. In an auditory gambling task, participants placed bets on which of two cards (ranging in value from 1 to 10) would be of a higher numerical value. After the first card was announced, a level of certainty was established, and a greater pupil dilation response is observed for more certain outcomes (i.e., if the first card is 2, certainty is high that the second card will be of higher value). After the second card was announced, a level of surprise was established, and greater pupil dilation is observed for more surprising outcomes (i.e., if the first card was 2, but the second card was 1; Preuschoff, Marius, Einh€auser, & Nieuwenhuis, 2011). In another study, participants took part in a visual Iowa Gambling Task during pupil measurements. Similar to Preuschoff et al. (2011), Lavın, Martın, and Jubal (2014) showed that pupil dilation changes were modulated by learned uncertainty and surprise. These findings are relevant in the context of object recognition and familiarity because some level of uncertainty is inherently present when viewing unfamiliar forms (par- ticularly nonobjects) when intermixed with other identifiable objects—even in a passive task- independent paradigm such as ours. Pupillometry studies using sounds rather than visual patterns have also revealed a role for novelty in producing pupil dilation. When listening to a series of tones (high vs. low pitch) presented repetitively, conscious reports of deviations to these sound patterns are accompa- nied by a pupil dilation, presumably brought on by the novelty of the mismatched tone. On the other hand, if a deviation to the pattern occurred but was not consciously reported by the participant, pupil dilation was not observed (Quirins, Marois, Valente, & Seassau, 2018). This is arguably in contradiction to the aforementioned Naber et al.’s (2013) study, in which forgotten scenes that were previously encoded produced a constriction not dilation response. That is to say, one study shows that consciousness is important in order for novelty to produce a pupil dilation (Quirins et al., 2018), while another posits that when novelty is simply a result of forgetting it produces a pupil constriction (Naber et al., 2013). The aim of this study is to test whether familiarity or novelty per se elicits a change in the pupil response. Given that unfamiliar 3D structures have been shown using functional mag- netic resonance imaging to activate the same lateral-occipital networks that common objects do (Malach et al., 1995), one strategy for determining how familiar and unfamiliar objects are processed is to compare them in the absence of any task-specific encoding requirements or restrictions. The pupillometry studies discussed earlier examined the memory of encoded objects and scenes, but the stimuli in their experiments had a baseline level of familiarity prior to their encoding. That is, even unknown objects and scenes are familiar enough simply due to their relation to a wider semantic network of identifiable items and places. Our goal is to test the perception of identifiable objects as a marker of familiarity and unidentifiable nonobjects (and noise) as a marker of unfamiliarity. An important methodological issue arises when one wishes to compare familiar with nonfamiliar objects: Which stimulus features should be common to both types of objects 4 i-Perception 10(5) Figure 1. Example stimuli used in the experiments. in order to make the comparison valid? When we encounter new objects in daily life, they usually possess certain common features with objects that we are already familiar with, such as straight edges, curved contours, textured surfaces, or familiar colors. If one thinks of object recognition as the end point of a hierarchy of visual processing from low-level to high- level features (Riesenhuber & Poggio, 1999), one ideally wants to match the two types of object as closely as possible to equate their responses throughout the hierarchy until one reaches the stage at which the critical differences that distinguish the familiar from the nonfamiliar are processed. Unfortunately, the literature on object recognition is insufficient to determine exactly where this point lies, so we made an educated guess. While many studies have attempted to control stimuli as best as possible to achieve this end, conventional methods have typically destroyed the most basic of visual properties, for example, when phase-scrambling the image, which involves randomizing the phases of the image’s Fourier components (Malach et al., 1995). This makes it difficult to determine whether the observed differences between objects and nonobjects is relevant to object recognition per se, as opposed to relevant to only the initial stages of visual processing (Stojanoski & Cusack, 2014). The stimuli used here, examples of which are shown in Figure 1, are designed to make as valid-as-possible a comparison between the pupil responses to familiar and unfamiliar objects. For example, an obvious prerequisite of any study using pupillometry to assess cognitive differences between conditions is that the stimuli across all conditions are equal in mean luminance; otherwise, one might simply be measuring the pupillary reflex to differ- ing light levels. Our stimuli fulfil this requirement through the use of direct current-balanced Gabor micropatterns. The use of a fixed number of micropatterns also equates the stimuli for another low-level feature: average contrast content. The other manipulations aimed at matching the familiar with unfamiliar objects are described in the next section. Based on findings that the pupil response is affected by perceptual content, arousal, and cognitive load (Bradley, Miccoli, Escrig, & Lang, 2008; Kloosterman et al., 2015; Peysakhovich, Dehais, & Causse, 2015), we expect to find pupil size differences between our familiar objects, the unfamiliar nonobjects, and the noise conditions. On the one hand, familiar objects evoke a semantic and possibly affective response, which previous studies suggest should cause an increase in pupil size (Partala & Surakka, 2003). On the other hand, more cognitive effort might be expended in processing nonfamiliar objects pre- cisely because they are not familiar, or because they evoke more curiosity and arousal, and this might predict increased pupil size for nonfamiliar objects (Klingner, Tversky, & Beukema et al. 5 Hanrahan, 2011; Naber et al., 2013). The aim of this study is to test between these possi- ble outcomes. Methods The methods reported here are similar to those reported by Beukema et al. (2017).The procedure and analyses were preregistered online (see: https://osf.io/2wf3d/). Participants Thirty healthy volunteers at McGill University were recruited as observers. No participants were excluded. The participants ranged in age from 18 to 40 years (median¼ 21.5; 19 females and 11 males). All participants had normal or corrected-to-normal visual acuity and did not possess any abnormalities of the eye. Demographic information about participants’ sex, age, ethnicity, handedness, eye color, and vision correction were collected. Participants gave informed consent prior to the study, and the research protocol was approved by the Research Ethics Board of the Research Institute of the McGill University Health Centre. Furthermore, this research adhered to the relevant tenets of the Declaration of Helsinki. Equipment Stimuli were generated within the MATLAB Psychophysics Toolbox 3.0 and were displayed on a 1920 by 1080 pixel ASUS high-definition monitor. An EyeTribe pupilometer was used to collect eye gaze and pupil size data via infrared reflections using the EyeTribe toolbox for MATLAB (version 0.0.3; Dalmaijer, 2014). The EyeTribe sampled pupil size data at 60 Hz, collecting 540 data points per trial. Viewing distance to the monitor was 60 cm. Participants were placed in a chin and head rest to remove motion artifacts. Eye gaze and fixation were calibrated prior to each experimental block. Stimuli The stimulus set was taken from a previous experiment investigating luminance inputs to mid- and high-level vision (Jennings & Martinovic, 2014). Jennings and Martinovic (2014) created their object stimulus set by choosing nameable line drawings of familiar objects from various stimulus sets (Alario & Ferrand, 1999; Bates et al., 2003; Hamm & McMullen, 1998) and digitally replacing the lines with Gabor patches that defined the outline shapes (Figure 1, first panel). The nonobjects were created by manipulating the distribution of the object-line drawings using image processing software and then applying the same shape-defining pro- cedure as mentioned earlier to the subsequent Gabor placements (Martinovic, Mordal, & Wuerger, 2011). Importantly, the manipulation of objects into nonobjects attempted to preserve image complexity (same number of elements), aspect ratio (field of view), and closed outer shape contour (global structure), so that the resulting unfamiliar shapes retained comparability to their former objecthood (Figure 1, middle panel). All of the objects used in the current study were accurately identified with an accuracy rating of over 75% from all participants in Jennings and Martinovic (2014). A third stimulus group composed of random clusters of Gabor patches acted as a control (Figure 1, third panel). All conditions contained the same number of local Gabor patches, and all images in one condition were paired to an image of equal number of Gabors in the other conditions. 6 i-Perception 10(5) Figure 2. Experimental design. The three phases of a trial were the baseline fixation, the stimulus phase (noise, object, or nonobject conditions), and the blink fixation (3,000 milliseconds between each phase). Participants were instructed to fixate throughout the experiment and to attempt to isolate any blinks to the blink fixation phase. Example stimulus enlarged for clarity. All phases were 3 seconds in duration, for a trial length of 9 seconds, and a block length of 4.5 minutes. Procedure Participants were shown sample trials from all three conditions prior to the experiment to ensure they could quickly and accurately identify whether the stimulus resembled an object, a nonobject, or a random array of Gabor elements. The actual experiment consisted of 50 images from each condition (150 total), broken up and randomly distributed into five 30-trial blocks to reduce dry eyes from visual fatigue. Each trial consisted of three phases: baseline, stimulus, and blink phases, as shown in Figure 2. The baseline phase served as a break from the stimulus to collect data for baseline correction. The phase of interest to the investigation is the stimulus phase during which pupillary differences were expected. Following this, participants were encouraged to blink a few times to reduce dry eyes, as blinks during the stimulus phase were discouraged in order to prevent data loss during the period of interest. Relatively long presentation times were used to ensure the perception of the stimulus was properly registered and ruminated, the after-image from the stimulus vanished, and that participants had ample time to blink. To avoid any effect of eye-movements on pupil size, participants were given a head rest for their chin and forehead and asked to remain fixated on the center of the screen for the entire length of the experiment. Data Processing and Analysis Our data transformations mirror those of our previous study (Beukema et al., 2017, p. 6). The participants’ data were grouped and all trials were subject to postexperimental Beukema et al. 7 processing. Due to the high amount of noise inherent in pupillometry data, and the large amount of data collected at the 60 Hz sampling rate (540 samples per trial), samples within trials were excluded from the analysis according to two criteria: overly large fixational eye movements and blinks. To remove samples involving overly large fixational eye movements, coordinates outside of a 100-pixel radius from the fixation point for at least 15 samples (0.25 seconds) were rejected. To remove blink trials, if more than 10 contiguous samples (0.17 seconds) contained no data, a blink was assumed and the trial was excluded. Finally, if the stimulus phase contained fewer than 100 samples (1.7 seconds), the whole trial was excluded due to insufficient data. After all exclusions, 70.17% of the samples remained; all data are available online (https://osf.io/ve8kr/). Data were baseline-corrected to the median pupil size within the last 200 milliseconds of the baseline phase to normalize the data. Statistical analyses focused on the segment of the stimulus phase occurring after the re-orienting pupil constriction caused by the onset response due to the sudden change in spatial frequency from blank fixation (Ebitz & Moore, 2019); analysis of the pupil time-course occurred from minimum pupil size in the stimulus phase (4,000 milliseconds) to the start of the blink phase (6,000 milliseconds). We predicted that pupil size would vary by condition. To test this, we used linear regres- sion to compare the average baseline-corrected pupil sizes for each condition, block, and participant. Using t tests with contrasts, we compared all three possible pairs of conditions. Tests were nondirectional with a Type I error rate of 0.05 and no family-wise error control. Assumptions were reasonable for all tests. For effect sizes, we use d , a robust version of Cohen’s d, which shows standardized mean differences (Algina, Keselman, & Penfield, 2005). Square brackets throughout denote 95% bootstrapped confidence intervals. Data processing and analysis were done using R version 3.3.2 with packages multcomp, bootES, and ggplot2. All data are available online. Results Figures 3 and 4 show, respectively, the pupil responses across the full trial duration (0–9,000 milliseconds) and across a portion of the trial duration (3,000–6,000 milliseconds). When averaged across the full stimulus phase (4,000–6,000 milliseconds), there was no difference in pupil size when viewing objects (M¼ 22.852 arbitrary units), nonobjects (M¼ 22.92), and random noise (M¼ 22.948). In line with previous studies measuring the pupil responses, we conducted a post hoc analysis over a narrower time window (4,000–4,500 milliseconds) corresponding to the fast reaction times found for discriminating static visual target stimuli (Barbur, Wolf, & Lennie, 1998; Kafkas & Montaldi, 2015; Naber et al., 2013; Schroger € & Widmann, 1998). Within this stimulus range following the pupillary light reflex, pupil size was smallest when viewing objects (M¼ 22.689 arbitrary units) and of equal size when viewing nonobjects (M¼ 22.849) and random noise (M¼ 22.847; Figures 3 and 4). Discussion The majority of studies investigating cognitively mediated pupil responses show a dilation response. Factors that have been shown to influence pupil dilation include anxiety, stress, recollection, cognitive effort, perceptual load, physical attraction, general emotional arousal (positive or negative), and novelty (Sirois & Brisson, 2014). As a result, the field of pupill- ometry is currently evolving to determine how the magnitude of pupil dilation is affected by these various factors. 8 i-Perception 10(5) Stimulus Baseline Blink Condition noise non−object object 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 Time (ms) Figure 3. Averaged raw data showing pupil size by condition across the full trial. Pupil size is plotted as a function of time in milliseconds. Vertical lines represent onset and offset of phases outlined in Figure 2: baseline fixation from 0 to 3,000 milliseconds, stimulus (object, nonobject, noise) fixation phase from 3,000 to 6,000 milliseconds, and then blink fixation from 6,000 to 9,000 milliseconds. The small gray surrounds represent 95% confidence intervals. Lines show GAM-smoothed curves, as in subsequent plots. In the present study, we investigated the effect of object form on pupil response, of which two competing hypotheses were proposed: First, that a greater dilation to familiar objects would be observed (given previous results showing pupil dilation to affective factors such as those elicited by familiar objects), and second, that a greater dilation to unfamiliar stimuli would be observed (given previous results showing pupil dilation to novelty). Our results showed that the unfamiliar stimuli (nonobject and noise conditions) provoked the greatest dilation when compared to the familiar object stimuli. The effect of interest was significant only during a shorter time-window in a post hoc analysis (Table 1). This is not terribly surprising given that the methods were borrowed from our previous paper where the effect of interest was an illusion that physically persisted for the entire 3-second stimulus phase (Beukema et al., 2017). Our current stimuli, however, are more in line with other pupillometry studies employing a static visual discrimination task (Kafkas & Montaldi, 2015; Naber et al., 2013); therefore, there is nothing of interest (like motion) to perceive beyond the first half of a second, and this is in keeping with the rapid pupillary response to stimulus onset (Barbur et al., 1998; Schroger € & Widmann, 1998). The paradigm was kept the same both for methodological consistency as well as to provide a precautionary window in case the pupillary light reflex needed more time to resolve. A generic problem with pupillometry is that one can never be sure that the cause of pupil dilation is due to the stimulus manipulation itself rather than simply a low-level arrangement of the stimulus parts. Our combined use of stimuli made from Gabor micropatterns, as well as the inclusion of the random noise condition, enabled us to make a comparison between familiar and unfamiliar objects that minimized the effects of low-level stimulus factors. Pupil size (arbitrary units) Beukema et al. 9 1.015 1.010 Condition noise 1.005 non−object object 1.000 0.995 3000 4000 5000 6000 Time (ms) Figure 4. Baseline-corrected pupil size by condition. Expanded and processed data from the middle portion of Figure 3 (after baseline correction and artifact rejection), revealing the differences between conditions. The object stimulus (blue line) constricts pupils more than the noise (red line) and nonobject (green line) following the pupillary light reflex after stimulus onset. The gray surrounds represent 95% confidence intervals. For example, there were inevitably differences in the number of contours between the non- object and object conditions that could potentially cause the difference in pupil dilation. However, if the number of contours was critical, we would expect a difference between the nonobject and random-noise conditions, as the latter contain no contours; we found no difference between these two conditions. The current results are intriguing due to the type of comparison being made. In Kafkas and Montaldi’s (2015) study, which showed greater pupil dilation in participants observing old objects previously seen compared to novel objects, all of the stimuli were selected from a familiar class of objects even though the specific instance of the object was novel. In our study, on the other hand, a true familiarity response was assessed as we compared familiar objects to never-before-seen nonobjects, even though the nonobjects comprised basic object properties. There are at least two possibilities for why our unfamiliar conditions produced more dilation than the familiar objects, and they are not mutually exclusive: novelty and cognitive effort. Regarding novelty, the less interesting explanation is that the unfamiliar nonobjects and the random noise were simply more interesting than the familiar objects. However, this explanation is unlikely because objects inevitably elicit physiological arousal. Indeed, it may well be the case that our object condition elicited an arousal response, but that this response was mitigated by the fact that cognitive demands take priority over arousal in affecting the pupillary response (Stanners, Coulter, Sweet, & Murphy, 1979); that is, a pupil dilation response reflecting arousal is more likely when the cognitive demands are minimal. Regarding cognitive effort, familiar objects may have a rich semantic network relevant to cognitive processing; however, recognition of the object itself is effortless. That is, it is likely Pupil size (relative to baseline) 10 i-Perception 10(5) Table 1. Pairwise Contrasts of Baseline-Corrected Pupil Size Between Conditions. Range Comparison Difference tp dR 4,000–6,000 milliseconds Nonobject vs. Object 0.00202 0.736 .461 –0.06 [–0.14, 0.03] Noise vs. Object 0.00273 0.999 .318 –0.05 [–0.13, 0.04] Noise vs. Nonobject 0.00071 0.259 .795 0.01 [–0.07, 0.10] 4,000–4,500 milliseconds Nonobject vs. Object 0.00629 2.575 .010 –0.11 [–0.20, –0.02] Noise vs. Object 0.0058 2.384 .017 –0.09 [–0.17, –0.01] Noise vs. Nonobject –0.00049 –0.2 .841 0.03 [–0.06, 0.12] Note. There was no difference when averaging across the full-time range (4,000–6,000 milliseconds, confirmatory test), but pupil size was smallest when viewing objects in a particular time range (4,000–4,500 milliseconds, post hoc test). d shows robust Cohen’s d with bootstrapped 95% confidence intervals. that under the passive viewing conditions, participants viewed the objects more reflexively due to their high familiarity, while the unfamiliar conditions required a heightened level of cognitive effort before dismissing the stimuli as nonobjects. This theory would fit under the tonic activation profile of the locus coeruleus associated with task disengagement and encouraging exploration (Gilzenrat, Nieuwenhuis, Jepma, & Cohen, 2010). It is also worth considering that the result we are observing is due to the specific context of our experimental protocol, in which we compare three types of stimuli: familiar-object, unfamiliar-nonobject and unfamiliar-noise. For example, if the experiment were to test between two unfamiliar conditions, we might see a dilation to one over the other based on the stimulus properties. However, despite not giving the participants a task, there is likely still an implicit task with identifying objects from the other conditions and cognitive effort might therefore be invoked during the unfamiliar conditions in this experiment. The current investigation outlines the importance of context within an experimental par- adigm. Alone, the words familiarity and novelty are not enough to assign to a pupillary mechanism. In previous studies, these words were used to describe the stimuli within the experiment, categorizing remembered objects as familiar, and new or forgotten objects as novel. In our study, we use these terms more generally, categorizing all objects as familiar on the basis that they are recognizable despite never being observed before, and instead only using the word novel for unrecognizable forms with no identity. The difference in observed results between these two classifications further demonstrates a need for careful clarification in the study of cognitively mediated pupil size responses. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a FRQNT Fellowship #259463 (given to S. B.), a Joseph-Armand Bombardier Scholarship from the Social Sciences and Humanities Research Council of Canada (given to J. O.), and a Canadian Institute of Health Research grant #MOP 123349 (given to F. K.). Beukema et al. 11 ORCID iD Steve Beukema https://orcid.org/0000-0002-7603-0287 References Alario, F. X., & Ferrand, L. (1999). A set of 400 pictures standardized for French: Norms for name agreement, image agreement, familiarity, visual complexity, image variability, and age of acquisi- tion. Behavior Research Methods, Instruments, & Computers : A Journal of the Psychonomic Society, 31, 531–552. doi:10.3758/BF03200732 Algina, J., Keselman, H. J., & Penfield, R. D. (2005). An alternative to Cohen’s standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods, 10(3), 317–328. doi: 10.1037/1082-989X.10.3.317 Barbur, J. L., Wolf, J., & Lennie, P. (1998). Visual processing levels revealed by response latencies to changes in different visual attributes. Proceedings of the Royal Society B: Biological Sciences, 265, 2321–2325. doi:10.1098/rspb.1998.0578 Bates, E., D’Amico, S., Jacobsen, T., Sze´ kely, A., Andonova, E., Devescovi, A., . . . Tzeng, O. (2003). Timed picture naming in seven languages. Psychonomic Bulletin and Review, 10, 344–380. doi:10.3758/BF03196494 Beatty, J., & Lucero-Wagoner, B. (2000). The pupillary system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (pp. 142–162). New York, NY, US: Cambridge University Press. Beukema, S., Olson, J. A., Jennings, B. J., & Kingdom, F. A. A. (2017). Pupil dilation to illusory motion in peripheral drift images: Perception versus reality. Journal of Vision, 17, 1–13. doi:10.1167/17.8.1 Bradley, M. B., Miccoli, L. M., Escrig, M. a, & Lang, P. J. (2008). The pupil as a measure of emotional arousal and automatic activation. Psychophysiology, 45, 602–607. doi:10.1111/j.1469- 8986.2008.00654.x Dalmaijer, E. (2014). Is the low-cost EyeTribe eye tracker any good for research? PeerJ Preprints. doi: 10.7287/peerj.preprints.585 DiCarlo, J. J., Yoccolan, D., & Rust, N. C. (2012). How does the brain solve visual object recognition? Neuron, 73, 415–434. doi:10.1016/j.neuron.2012.01.010. Ebitz, R. B., & Moore, T. (2019). Both a gauge and a filter: Cognitive modulations of pupil size. Frontiers in Neurology, 9, 1–14. doi:10.3389/fneur.2018.01190 Ellis, C. J. (1981). The pupillary light reflex in normal subjects. British Journal of Ophthalmology, 65, 754–759. doi:10.1136/bjo.65.11.754 Ferrari, V., Cesarei, A. D., Mastria, S., Lugli, L., Baroni, G., Nicoletti, R., & Codispoti, M. (2016). Novelty and emotion: Pupillary and cortical responses during viewing of natural scenes. Biological Psychology, 113, 75–82. Gilzenrat, M. S., Nieuwenhuis, S., Jepma, M., & Cohen, J. D. (2010). Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cognitive, Affective and Behavioral Neuroscience, 10, 252–269. doi:10.3758/CABN.10.2.252 Grill-Spector, K. (2003). The neural basis of object perception. Current Opinion in Neurobiology, 13, 159–166. doi:10.1016/S0959-4388(03)00040-0 Hamm, J. P., & McMullen, P. A. (1998). Effects of orientation on the identification of rotated objects depend on the level of identity. Journal of Experimental Psychology: Human Perception and Performance, 24, 413–426. doi:10.1037/0096-1523.24.2.413 Jennings, B. J., & Martinovic, J. (2014). Luminance and color inputs to mid-level and high-level vision. Journal of Vision, 14, 9. doi:10.1167/14.2.9 Kafkas, A., & Montaldi, D. (2015). The pupillary response discriminates between subjective and objec- tive familiarity and novelty. Psychophysiology, 52, 1305–1316. doi:10.1111/psyp.12471 Kamp, S. M., & Donchin, E. (2015). ERP and pupil responses to deviance in an oddball paradigm. Psychophysiology, 52, 460–471. doi:10.1111/psyp.12378 12 i-Perception 10(5) Klingner, J., Tversky, B., & Hanrahan, P. (2011). Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology, 48, 323–332. doi:10.1111/j.1469- 8986.2010.01069.x Kloosterman, N. A., Meindertsma, T., van Loon, A. M., Lamme, V. A. F., Bonneh, Y. S., & Donner, T. H. (2015). Pupil size tracks perceptual content and surprise. European Journal of Neuroscience, 41, 1068–1078. doi:10.1111/ejn.12859 Laeng, B., Waterloo, K., Johnsen, S. H., Simonsen, S. S., Bakke, S. J., & La, T. (2007). The eyes remember it : Oculography and pupillometry during recollection in three amnesic patients. Journal of Cognitive Neuroscience, 19, 1888–1904. Lavın, C., Martın, R. S., & Jubal, E. R. (2014). Pupil dilation signals uncertainty and surprise in a learning gambling task. Frontiers in Behavioral Neuroscience, 7, 1–8. doi:10.3389/fnbeh.2013.00218 Liao, H. I., Yoneya, M., Kidani, S., Kashino, M., & Furukawa, S. (2016). Human pupillary dilation response to deviant auditory stimuli: Effects of stimulus properties and voluntary attention. Frontiers in Neuroscience, 10, 43. doi:10.3389/fnins.2016.00043 Loewenfeld, I. E. (1958). Mechanisms of reflex dilatation of the pupil—Historical review and experi- mental analysis. Documenta Ophthalmologica, 12, 185–448. doi:10.1007/BF00913471 Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., . . . Tootell, R. B. (1995). Object-related activity revealed by functional magnetic resonance imaging in human occip- ital cortex. Proceedings of the National Academy of Sciences, 92, 8135–8139. doi:10.1073/ pnas.92.18.8135 Martinovic, J., Mordal, J., & Wuerger, S. M. (2011). Event-related potentials reveal an early advantage for luminance contours in the processing of objects. Journal of Vision, 11, 1. doi:10.1167/11.7.1 Mathoˆ t, S. (2018). Pupillometry: Psychology, physiology, and function. Journal of Cognition, 1, 1–23. doi:10.5334/joc.18 Naber, M., Frassle, S., Rutishauser, U., & Einhauser, W. (2013). Pupil size signals novelty and predicts later retrieval success for declarative memories of natural scenes. Journal of Vision, 13, 11. doi:10.1167/13.2.11 Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human Computer Studies, 59, 185–198. doi:10.1016/S1071-5819(03)00017-X Peysakhovich, V., Dehais, F., & Causse, M. (2015). Pupil diameter as a measure of cognitive load during auditory-visual interference in a simple piloting task. Procedia Manufacturing, 3, 5199–5205. doi:10.1016/j.promfg.2015.07.583 Preuschoff, K., Marius, B., Einh€auser, W., & Nieuwenhuis, S. (2011). Pupil dilation signals surprise : Evidence for noradrenaline’s role in decision making. Frontiers in Neuroscience, 5, 1–12. doi:10.3389/fnins.2011.00115 Quirins, M., Marois, C., Valente, M., & Seassau, M. (2018). Conscious processing of auditory regu- larities induces a pupil dilation. Scientific Reports, 8, 14819. doi:10.1038/s41598-018-33202-7 Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2, 1019–1025. doi:10.1038/14819 Schroger, € E., & Widmann, A. (1998). Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology, 35, 755–759. doi:10.1017/S0048577298980714 Sirois, S., & Brisson, J. (2014). Pupillometry. Wiley Interdisciplinary Reviews: Cognitive Science, 5, 679–692. doi:10.1002/wcs.1323 Stanners, R. F., Coulter, M., Sweet, A. W., & Murphy, P. (1979). The pupillary response as an indi- cator of arousal and cognition. Motivation and Emotion, 3, 319–340. doi:10.1007/BF00994048 Stojanoski, B., & Cusack, R. (2014). Time to wave good-bye to phase scrambling: Creating controlled scrambled images using diffeomorphic transformations. Journal of Vision, 14, 6. doi:10.1167/14.12.6 How to cite this article Beukema, S., Jennings, B. J., Olson, J. A., & Kingdom, F. A. A. (2019). The pupillary response to the unknown: Novelty versus familiarity. i-Perception, 10(5), 1–12. doi:10.1177/ http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png i-Perception SAGE

The Pupillary Response to the Unknown: Novelty Versus Familiarity:

Download PDF
12 pages

Loading next page...
 
/lp/sage/the-pupillary-response-to-the-unknown-novelty-versus-familiarity-ia0v9XCgpa

References (56)

Publisher
SAGE
Copyright
Copyright © 2022 by SAGE Publications Ltd unless otherwise noted. Manuscript content on this site is licensed under Creative Commons Licenses
ISSN
2041-6695
eISSN
2041-6695
DOI
10.1177/2041669519874817
Publisher site
See Article on Publisher Site

Abstract

Object recognition is a type of perception that enables observers to recognize familiar shapes and categorize them into real-world identities. In this preregistered study, we aimed to determine wheth- er pupil size changes occur during the perception and recognition of identifiable objects. We com- pared pupil size changes for familiar objects, nonobjects, and random noise. Nonobjects and noise produced greater pupil dilation than familiar objects. Contrary to previous evidence showing greater pupil dilation to stimuli with more perceptual and affective content, these results indicate a greater pupil dilation to stimuli that are unidentifiable. This is consistent with the relative salience of novelty compared to familiarity at the physiological level driving the pupil response. Keywords object, form, recognition, pupillometry, pupil, dilation, constriction, novelty, familiarity Date received: 4 April 2019; accepted: 12 August 2019 Corresponding author: Steve Beukema, McGill University, 845 Sherbrooke St W, Montreal, QC H3A 0G4, Canada. Email: steve.beukema@mail.mcgill.ca Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (http://www.creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 i-Perception 10(5) Introduction Pupil size changes are reflexive; the pupil not only grows and shrinks unconsciously in response to varying levels of light (Ellis, 1981), but it also changes in size depending on the content of the stimulus. This cognitive influence on the pupillary response has been long established (Beatty & Lucero-Wagoner, 2000; Loewenfeld, 1958), but its precise nature is less clear (Mathoˆ t, 2018). In this article, we focus on the pupillary responses to object recogni- tion, in order to deepen our understanding of the cognitive influences on the pupillary response as well as on object recognition itself. Object recognition is an important part of human perception, necessary not only for determining the content of the visual world but also to establish links between familiar identities in our past, present, and future. One issue that engages researchers using pupill- ometry to study object recognition concerns the process of distinguishing familiar from nonfamiliar objects (DiCarlo, Yoccolan, & Rust, 2012; Grill-Spector, 2003; Kafkas & Montaldi, 2015). For example, Kafkas and Montaldi (2015) showed subjects a range of familiar objects during a training phase and then measured the pupil responses to a set of test objects, some of which had been presented in the training phase. They found that the test objects that had been seen during the training phase produced a greater pupil dilation than the ones not previously seen. In a similar study by Naber, Frassle, Rutishauser, and Einhauser (2013), subjects viewed images of everyday scenes and were asked to memorize half of them. After a distraction phase, the images were presented again but intermixed with novel images, and the subjects indicated whether the images were familiar or novel and how confident they were in their judgement. Consistent with the results of Kafkas and Montaldi (2015), a greater pupil dilation was observed for the memorized scenes, while the novel scenes produced a pupil constriction that became larger as a function of confidence. Unlike Kafkas and Montaldi (2015), however, Naber et al. (2013) tested the pupil response during both retrieval and encoding. Interestingly, a greater constriction response was observed during the encoding phase for scenes that were later remembered compared to the scenes that were later forgotten. Then, during the retrieval phase, the novel (i.e., unfa- miliar) scenes produced a greater constriction compared to the familiar (i.e., remembered) scenes. The authors thus showed that under the task of retrieving a memory, stimulus novelty could produce a pupil constriction not dilation. This highlights not only a potential for separate pupil mechanisms during encoding versus retrieval but also the importance of the particular experimental context for determining whether pupil constriction or dila- tion occurs. Although both the Kafkas and Montaldi (2015) and Naber et al. (2013) studies mentioned earlier are consistent in showing greater pupil dilation to familiarity during an encoding task, there are exceptions to this finding. In Laeng et al. (2007), three patients were tested using a similar memory paradigm as the aforementioned studies—with encoding and retrieval phases. Interestingly, even though the patients lacked explicit awareness of having seen the images before (failure of encoding due to hippocampal lesions), they still produced a greater pupil dilation to the novel foil stimuli during retrieval compared to previously seen stimuli, contrary to the pupillary response observed in the studies examining healthy partic- ipants. This further supports the notion that separate neural processes underpin encoding versus retrieval, and if damaged, the pupil will respond accordingly based only on the infor- mation sustained by the remaining pathways. Many other studies have shown that stimulus novelty is capable of inducing a pupil dilation response (Ferrari et al., 2016; Kamp & Donchin, 2015; Kloosterman et al., 2015; Liao, Yoneya, Kidani, Kashino, & Furukawa, 2016). For example, Ferrari et al. (2016) Beukema et al. 3 found a greater dilation response for emotional compared to neutral natural scenes but only when they were novel; emotional content did not induce a greater pupil dilation when the scenes were repeatedly presented during the experiment. This highlights how a physiological response to novelty versus familiarity can be affected by other factors, such as arousal. In two studies concerning illusory phenomena, Kloosterman et al. (2015) and Beukema, Olson, Jennings, and Kingdom (2017) show that physically present and static stimuli can cause a pupil dilation when the images are perceived to vanish or move respectively, contrary to the physical reality. The perception of illusory phenomena is a novel perception that has close ties with the concept of surprise—the betrayal of an individual’s level of certainty. Two studies that explore this concept more closely employ a gambling task while measuring participants’ pupil responses. In an auditory gambling task, participants placed bets on which of two cards (ranging in value from 1 to 10) would be of a higher numerical value. After the first card was announced, a level of certainty was established, and a greater pupil dilation response is observed for more certain outcomes (i.e., if the first card is 2, certainty is high that the second card will be of higher value). After the second card was announced, a level of surprise was established, and greater pupil dilation is observed for more surprising outcomes (i.e., if the first card was 2, but the second card was 1; Preuschoff, Marius, Einh€auser, & Nieuwenhuis, 2011). In another study, participants took part in a visual Iowa Gambling Task during pupil measurements. Similar to Preuschoff et al. (2011), Lavın, Martın, and Jubal (2014) showed that pupil dilation changes were modulated by learned uncertainty and surprise. These findings are relevant in the context of object recognition and familiarity because some level of uncertainty is inherently present when viewing unfamiliar forms (par- ticularly nonobjects) when intermixed with other identifiable objects—even in a passive task- independent paradigm such as ours. Pupillometry studies using sounds rather than visual patterns have also revealed a role for novelty in producing pupil dilation. When listening to a series of tones (high vs. low pitch) presented repetitively, conscious reports of deviations to these sound patterns are accompa- nied by a pupil dilation, presumably brought on by the novelty of the mismatched tone. On the other hand, if a deviation to the pattern occurred but was not consciously reported by the participant, pupil dilation was not observed (Quirins, Marois, Valente, & Seassau, 2018). This is arguably in contradiction to the aforementioned Naber et al.’s (2013) study, in which forgotten scenes that were previously encoded produced a constriction not dilation response. That is to say, one study shows that consciousness is important in order for novelty to produce a pupil dilation (Quirins et al., 2018), while another posits that when novelty is simply a result of forgetting it produces a pupil constriction (Naber et al., 2013). The aim of this study is to test whether familiarity or novelty per se elicits a change in the pupil response. Given that unfamiliar 3D structures have been shown using functional mag- netic resonance imaging to activate the same lateral-occipital networks that common objects do (Malach et al., 1995), one strategy for determining how familiar and unfamiliar objects are processed is to compare them in the absence of any task-specific encoding requirements or restrictions. The pupillometry studies discussed earlier examined the memory of encoded objects and scenes, but the stimuli in their experiments had a baseline level of familiarity prior to their encoding. That is, even unknown objects and scenes are familiar enough simply due to their relation to a wider semantic network of identifiable items and places. Our goal is to test the perception of identifiable objects as a marker of familiarity and unidentifiable nonobjects (and noise) as a marker of unfamiliarity. An important methodological issue arises when one wishes to compare familiar with nonfamiliar objects: Which stimulus features should be common to both types of objects 4 i-Perception 10(5) Figure 1. Example stimuli used in the experiments. in order to make the comparison valid? When we encounter new objects in daily life, they usually possess certain common features with objects that we are already familiar with, such as straight edges, curved contours, textured surfaces, or familiar colors. If one thinks of object recognition as the end point of a hierarchy of visual processing from low-level to high- level features (Riesenhuber & Poggio, 1999), one ideally wants to match the two types of object as closely as possible to equate their responses throughout the hierarchy until one reaches the stage at which the critical differences that distinguish the familiar from the nonfamiliar are processed. Unfortunately, the literature on object recognition is insufficient to determine exactly where this point lies, so we made an educated guess. While many studies have attempted to control stimuli as best as possible to achieve this end, conventional methods have typically destroyed the most basic of visual properties, for example, when phase-scrambling the image, which involves randomizing the phases of the image’s Fourier components (Malach et al., 1995). This makes it difficult to determine whether the observed differences between objects and nonobjects is relevant to object recognition per se, as opposed to relevant to only the initial stages of visual processing (Stojanoski & Cusack, 2014). The stimuli used here, examples of which are shown in Figure 1, are designed to make as valid-as-possible a comparison between the pupil responses to familiar and unfamiliar objects. For example, an obvious prerequisite of any study using pupillometry to assess cognitive differences between conditions is that the stimuli across all conditions are equal in mean luminance; otherwise, one might simply be measuring the pupillary reflex to differ- ing light levels. Our stimuli fulfil this requirement through the use of direct current-balanced Gabor micropatterns. The use of a fixed number of micropatterns also equates the stimuli for another low-level feature: average contrast content. The other manipulations aimed at matching the familiar with unfamiliar objects are described in the next section. Based on findings that the pupil response is affected by perceptual content, arousal, and cognitive load (Bradley, Miccoli, Escrig, & Lang, 2008; Kloosterman et al., 2015; Peysakhovich, Dehais, & Causse, 2015), we expect to find pupil size differences between our familiar objects, the unfamiliar nonobjects, and the noise conditions. On the one hand, familiar objects evoke a semantic and possibly affective response, which previous studies suggest should cause an increase in pupil size (Partala & Surakka, 2003). On the other hand, more cognitive effort might be expended in processing nonfamiliar objects pre- cisely because they are not familiar, or because they evoke more curiosity and arousal, and this might predict increased pupil size for nonfamiliar objects (Klingner, Tversky, & Beukema et al. 5 Hanrahan, 2011; Naber et al., 2013). The aim of this study is to test between these possi- ble outcomes. Methods The methods reported here are similar to those reported by Beukema et al. (2017).The procedure and analyses were preregistered online (see: https://osf.io/2wf3d/). Participants Thirty healthy volunteers at McGill University were recruited as observers. No participants were excluded. The participants ranged in age from 18 to 40 years (median¼ 21.5; 19 females and 11 males). All participants had normal or corrected-to-normal visual acuity and did not possess any abnormalities of the eye. Demographic information about participants’ sex, age, ethnicity, handedness, eye color, and vision correction were collected. Participants gave informed consent prior to the study, and the research protocol was approved by the Research Ethics Board of the Research Institute of the McGill University Health Centre. Furthermore, this research adhered to the relevant tenets of the Declaration of Helsinki. Equipment Stimuli were generated within the MATLAB Psychophysics Toolbox 3.0 and were displayed on a 1920 by 1080 pixel ASUS high-definition monitor. An EyeTribe pupilometer was used to collect eye gaze and pupil size data via infrared reflections using the EyeTribe toolbox for MATLAB (version 0.0.3; Dalmaijer, 2014). The EyeTribe sampled pupil size data at 60 Hz, collecting 540 data points per trial. Viewing distance to the monitor was 60 cm. Participants were placed in a chin and head rest to remove motion artifacts. Eye gaze and fixation were calibrated prior to each experimental block. Stimuli The stimulus set was taken from a previous experiment investigating luminance inputs to mid- and high-level vision (Jennings & Martinovic, 2014). Jennings and Martinovic (2014) created their object stimulus set by choosing nameable line drawings of familiar objects from various stimulus sets (Alario & Ferrand, 1999; Bates et al., 2003; Hamm & McMullen, 1998) and digitally replacing the lines with Gabor patches that defined the outline shapes (Figure 1, first panel). The nonobjects were created by manipulating the distribution of the object-line drawings using image processing software and then applying the same shape-defining pro- cedure as mentioned earlier to the subsequent Gabor placements (Martinovic, Mordal, & Wuerger, 2011). Importantly, the manipulation of objects into nonobjects attempted to preserve image complexity (same number of elements), aspect ratio (field of view), and closed outer shape contour (global structure), so that the resulting unfamiliar shapes retained comparability to their former objecthood (Figure 1, middle panel). All of the objects used in the current study were accurately identified with an accuracy rating of over 75% from all participants in Jennings and Martinovic (2014). A third stimulus group composed of random clusters of Gabor patches acted as a control (Figure 1, third panel). All conditions contained the same number of local Gabor patches, and all images in one condition were paired to an image of equal number of Gabors in the other conditions. 6 i-Perception 10(5) Figure 2. Experimental design. The three phases of a trial were the baseline fixation, the stimulus phase (noise, object, or nonobject conditions), and the blink fixation (3,000 milliseconds between each phase). Participants were instructed to fixate throughout the experiment and to attempt to isolate any blinks to the blink fixation phase. Example stimulus enlarged for clarity. All phases were 3 seconds in duration, for a trial length of 9 seconds, and a block length of 4.5 minutes. Procedure Participants were shown sample trials from all three conditions prior to the experiment to ensure they could quickly and accurately identify whether the stimulus resembled an object, a nonobject, or a random array of Gabor elements. The actual experiment consisted of 50 images from each condition (150 total), broken up and randomly distributed into five 30-trial blocks to reduce dry eyes from visual fatigue. Each trial consisted of three phases: baseline, stimulus, and blink phases, as shown in Figure 2. The baseline phase served as a break from the stimulus to collect data for baseline correction. The phase of interest to the investigation is the stimulus phase during which pupillary differences were expected. Following this, participants were encouraged to blink a few times to reduce dry eyes, as blinks during the stimulus phase were discouraged in order to prevent data loss during the period of interest. Relatively long presentation times were used to ensure the perception of the stimulus was properly registered and ruminated, the after-image from the stimulus vanished, and that participants had ample time to blink. To avoid any effect of eye-movements on pupil size, participants were given a head rest for their chin and forehead and asked to remain fixated on the center of the screen for the entire length of the experiment. Data Processing and Analysis Our data transformations mirror those of our previous study (Beukema et al., 2017, p. 6). The participants’ data were grouped and all trials were subject to postexperimental Beukema et al. 7 processing. Due to the high amount of noise inherent in pupillometry data, and the large amount of data collected at the 60 Hz sampling rate (540 samples per trial), samples within trials were excluded from the analysis according to two criteria: overly large fixational eye movements and blinks. To remove samples involving overly large fixational eye movements, coordinates outside of a 100-pixel radius from the fixation point for at least 15 samples (0.25 seconds) were rejected. To remove blink trials, if more than 10 contiguous samples (0.17 seconds) contained no data, a blink was assumed and the trial was excluded. Finally, if the stimulus phase contained fewer than 100 samples (1.7 seconds), the whole trial was excluded due to insufficient data. After all exclusions, 70.17% of the samples remained; all data are available online (https://osf.io/ve8kr/). Data were baseline-corrected to the median pupil size within the last 200 milliseconds of the baseline phase to normalize the data. Statistical analyses focused on the segment of the stimulus phase occurring after the re-orienting pupil constriction caused by the onset response due to the sudden change in spatial frequency from blank fixation (Ebitz & Moore, 2019); analysis of the pupil time-course occurred from minimum pupil size in the stimulus phase (4,000 milliseconds) to the start of the blink phase (6,000 milliseconds). We predicted that pupil size would vary by condition. To test this, we used linear regres- sion to compare the average baseline-corrected pupil sizes for each condition, block, and participant. Using t tests with contrasts, we compared all three possible pairs of conditions. Tests were nondirectional with a Type I error rate of 0.05 and no family-wise error control. Assumptions were reasonable for all tests. For effect sizes, we use d , a robust version of Cohen’s d, which shows standardized mean differences (Algina, Keselman, & Penfield, 2005). Square brackets throughout denote 95% bootstrapped confidence intervals. Data processing and analysis were done using R version 3.3.2 with packages multcomp, bootES, and ggplot2. All data are available online. Results Figures 3 and 4 show, respectively, the pupil responses across the full trial duration (0–9,000 milliseconds) and across a portion of the trial duration (3,000–6,000 milliseconds). When averaged across the full stimulus phase (4,000–6,000 milliseconds), there was no difference in pupil size when viewing objects (M¼ 22.852 arbitrary units), nonobjects (M¼ 22.92), and random noise (M¼ 22.948). In line with previous studies measuring the pupil responses, we conducted a post hoc analysis over a narrower time window (4,000–4,500 milliseconds) corresponding to the fast reaction times found for discriminating static visual target stimuli (Barbur, Wolf, & Lennie, 1998; Kafkas & Montaldi, 2015; Naber et al., 2013; Schroger € & Widmann, 1998). Within this stimulus range following the pupillary light reflex, pupil size was smallest when viewing objects (M¼ 22.689 arbitrary units) and of equal size when viewing nonobjects (M¼ 22.849) and random noise (M¼ 22.847; Figures 3 and 4). Discussion The majority of studies investigating cognitively mediated pupil responses show a dilation response. Factors that have been shown to influence pupil dilation include anxiety, stress, recollection, cognitive effort, perceptual load, physical attraction, general emotional arousal (positive or negative), and novelty (Sirois & Brisson, 2014). As a result, the field of pupill- ometry is currently evolving to determine how the magnitude of pupil dilation is affected by these various factors. 8 i-Perception 10(5) Stimulus Baseline Blink Condition noise non−object object 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 Time (ms) Figure 3. Averaged raw data showing pupil size by condition across the full trial. Pupil size is plotted as a function of time in milliseconds. Vertical lines represent onset and offset of phases outlined in Figure 2: baseline fixation from 0 to 3,000 milliseconds, stimulus (object, nonobject, noise) fixation phase from 3,000 to 6,000 milliseconds, and then blink fixation from 6,000 to 9,000 milliseconds. The small gray surrounds represent 95% confidence intervals. Lines show GAM-smoothed curves, as in subsequent plots. In the present study, we investigated the effect of object form on pupil response, of which two competing hypotheses were proposed: First, that a greater dilation to familiar objects would be observed (given previous results showing pupil dilation to affective factors such as those elicited by familiar objects), and second, that a greater dilation to unfamiliar stimuli would be observed (given previous results showing pupil dilation to novelty). Our results showed that the unfamiliar stimuli (nonobject and noise conditions) provoked the greatest dilation when compared to the familiar object stimuli. The effect of interest was significant only during a shorter time-window in a post hoc analysis (Table 1). This is not terribly surprising given that the methods were borrowed from our previous paper where the effect of interest was an illusion that physically persisted for the entire 3-second stimulus phase (Beukema et al., 2017). Our current stimuli, however, are more in line with other pupillometry studies employing a static visual discrimination task (Kafkas & Montaldi, 2015; Naber et al., 2013); therefore, there is nothing of interest (like motion) to perceive beyond the first half of a second, and this is in keeping with the rapid pupillary response to stimulus onset (Barbur et al., 1998; Schroger € & Widmann, 1998). The paradigm was kept the same both for methodological consistency as well as to provide a precautionary window in case the pupillary light reflex needed more time to resolve. A generic problem with pupillometry is that one can never be sure that the cause of pupil dilation is due to the stimulus manipulation itself rather than simply a low-level arrangement of the stimulus parts. Our combined use of stimuli made from Gabor micropatterns, as well as the inclusion of the random noise condition, enabled us to make a comparison between familiar and unfamiliar objects that minimized the effects of low-level stimulus factors. Pupil size (arbitrary units) Beukema et al. 9 1.015 1.010 Condition noise 1.005 non−object object 1.000 0.995 3000 4000 5000 6000 Time (ms) Figure 4. Baseline-corrected pupil size by condition. Expanded and processed data from the middle portion of Figure 3 (after baseline correction and artifact rejection), revealing the differences between conditions. The object stimulus (blue line) constricts pupils more than the noise (red line) and nonobject (green line) following the pupillary light reflex after stimulus onset. The gray surrounds represent 95% confidence intervals. For example, there were inevitably differences in the number of contours between the non- object and object conditions that could potentially cause the difference in pupil dilation. However, if the number of contours was critical, we would expect a difference between the nonobject and random-noise conditions, as the latter contain no contours; we found no difference between these two conditions. The current results are intriguing due to the type of comparison being made. In Kafkas and Montaldi’s (2015) study, which showed greater pupil dilation in participants observing old objects previously seen compared to novel objects, all of the stimuli were selected from a familiar class of objects even though the specific instance of the object was novel. In our study, on the other hand, a true familiarity response was assessed as we compared familiar objects to never-before-seen nonobjects, even though the nonobjects comprised basic object properties. There are at least two possibilities for why our unfamiliar conditions produced more dilation than the familiar objects, and they are not mutually exclusive: novelty and cognitive effort. Regarding novelty, the less interesting explanation is that the unfamiliar nonobjects and the random noise were simply more interesting than the familiar objects. However, this explanation is unlikely because objects inevitably elicit physiological arousal. Indeed, it may well be the case that our object condition elicited an arousal response, but that this response was mitigated by the fact that cognitive demands take priority over arousal in affecting the pupillary response (Stanners, Coulter, Sweet, & Murphy, 1979); that is, a pupil dilation response reflecting arousal is more likely when the cognitive demands are minimal. Regarding cognitive effort, familiar objects may have a rich semantic network relevant to cognitive processing; however, recognition of the object itself is effortless. That is, it is likely Pupil size (relative to baseline) 10 i-Perception 10(5) Table 1. Pairwise Contrasts of Baseline-Corrected Pupil Size Between Conditions. Range Comparison Difference tp dR 4,000–6,000 milliseconds Nonobject vs. Object 0.00202 0.736 .461 –0.06 [–0.14, 0.03] Noise vs. Object 0.00273 0.999 .318 –0.05 [–0.13, 0.04] Noise vs. Nonobject 0.00071 0.259 .795 0.01 [–0.07, 0.10] 4,000–4,500 milliseconds Nonobject vs. Object 0.00629 2.575 .010 –0.11 [–0.20, –0.02] Noise vs. Object 0.0058 2.384 .017 –0.09 [–0.17, –0.01] Noise vs. Nonobject –0.00049 –0.2 .841 0.03 [–0.06, 0.12] Note. There was no difference when averaging across the full-time range (4,000–6,000 milliseconds, confirmatory test), but pupil size was smallest when viewing objects in a particular time range (4,000–4,500 milliseconds, post hoc test). d shows robust Cohen’s d with bootstrapped 95% confidence intervals. that under the passive viewing conditions, participants viewed the objects more reflexively due to their high familiarity, while the unfamiliar conditions required a heightened level of cognitive effort before dismissing the stimuli as nonobjects. This theory would fit under the tonic activation profile of the locus coeruleus associated with task disengagement and encouraging exploration (Gilzenrat, Nieuwenhuis, Jepma, & Cohen, 2010). It is also worth considering that the result we are observing is due to the specific context of our experimental protocol, in which we compare three types of stimuli: familiar-object, unfamiliar-nonobject and unfamiliar-noise. For example, if the experiment were to test between two unfamiliar conditions, we might see a dilation to one over the other based on the stimulus properties. However, despite not giving the participants a task, there is likely still an implicit task with identifying objects from the other conditions and cognitive effort might therefore be invoked during the unfamiliar conditions in this experiment. The current investigation outlines the importance of context within an experimental par- adigm. Alone, the words familiarity and novelty are not enough to assign to a pupillary mechanism. In previous studies, these words were used to describe the stimuli within the experiment, categorizing remembered objects as familiar, and new or forgotten objects as novel. In our study, we use these terms more generally, categorizing all objects as familiar on the basis that they are recognizable despite never being observed before, and instead only using the word novel for unrecognizable forms with no identity. The difference in observed results between these two classifications further demonstrates a need for careful clarification in the study of cognitively mediated pupil size responses. Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by a FRQNT Fellowship #259463 (given to S. B.), a Joseph-Armand Bombardier Scholarship from the Social Sciences and Humanities Research Council of Canada (given to J. O.), and a Canadian Institute of Health Research grant #MOP 123349 (given to F. K.). Beukema et al. 11 ORCID iD Steve Beukema https://orcid.org/0000-0002-7603-0287 References Alario, F. X., & Ferrand, L. (1999). A set of 400 pictures standardized for French: Norms for name agreement, image agreement, familiarity, visual complexity, image variability, and age of acquisi- tion. Behavior Research Methods, Instruments, & Computers : A Journal of the Psychonomic Society, 31, 531–552. doi:10.3758/BF03200732 Algina, J., Keselman, H. J., & Penfield, R. D. (2005). An alternative to Cohen’s standardized mean difference effect size: A robust parameter and confidence interval in the two independent groups case. Psychological Methods, 10(3), 317–328. doi: 10.1037/1082-989X.10.3.317 Barbur, J. L., Wolf, J., & Lennie, P. (1998). Visual processing levels revealed by response latencies to changes in different visual attributes. Proceedings of the Royal Society B: Biological Sciences, 265, 2321–2325. doi:10.1098/rspb.1998.0578 Bates, E., D’Amico, S., Jacobsen, T., Sze´ kely, A., Andonova, E., Devescovi, A., . . . Tzeng, O. (2003). Timed picture naming in seven languages. Psychonomic Bulletin and Review, 10, 344–380. doi:10.3758/BF03196494 Beatty, J., & Lucero-Wagoner, B. (2000). The pupillary system. In J. T. Cacioppo, L. G. Tassinary, & G. G. Berntson (Eds.), Handbook of psychophysiology (pp. 142–162). New York, NY, US: Cambridge University Press. Beukema, S., Olson, J. A., Jennings, B. J., & Kingdom, F. A. A. (2017). Pupil dilation to illusory motion in peripheral drift images: Perception versus reality. Journal of Vision, 17, 1–13. doi:10.1167/17.8.1 Bradley, M. B., Miccoli, L. M., Escrig, M. a, & Lang, P. J. (2008). The pupil as a measure of emotional arousal and automatic activation. Psychophysiology, 45, 602–607. doi:10.1111/j.1469- 8986.2008.00654.x Dalmaijer, E. (2014). Is the low-cost EyeTribe eye tracker any good for research? PeerJ Preprints. doi: 10.7287/peerj.preprints.585 DiCarlo, J. J., Yoccolan, D., & Rust, N. C. (2012). How does the brain solve visual object recognition? Neuron, 73, 415–434. doi:10.1016/j.neuron.2012.01.010. Ebitz, R. B., & Moore, T. (2019). Both a gauge and a filter: Cognitive modulations of pupil size. Frontiers in Neurology, 9, 1–14. doi:10.3389/fneur.2018.01190 Ellis, C. J. (1981). The pupillary light reflex in normal subjects. British Journal of Ophthalmology, 65, 754–759. doi:10.1136/bjo.65.11.754 Ferrari, V., Cesarei, A. D., Mastria, S., Lugli, L., Baroni, G., Nicoletti, R., & Codispoti, M. (2016). Novelty and emotion: Pupillary and cortical responses during viewing of natural scenes. Biological Psychology, 113, 75–82. Gilzenrat, M. S., Nieuwenhuis, S., Jepma, M., & Cohen, J. D. (2010). Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cognitive, Affective and Behavioral Neuroscience, 10, 252–269. doi:10.3758/CABN.10.2.252 Grill-Spector, K. (2003). The neural basis of object perception. Current Opinion in Neurobiology, 13, 159–166. doi:10.1016/S0959-4388(03)00040-0 Hamm, J. P., & McMullen, P. A. (1998). Effects of orientation on the identification of rotated objects depend on the level of identity. Journal of Experimental Psychology: Human Perception and Performance, 24, 413–426. doi:10.1037/0096-1523.24.2.413 Jennings, B. J., & Martinovic, J. (2014). Luminance and color inputs to mid-level and high-level vision. Journal of Vision, 14, 9. doi:10.1167/14.2.9 Kafkas, A., & Montaldi, D. (2015). The pupillary response discriminates between subjective and objec- tive familiarity and novelty. Psychophysiology, 52, 1305–1316. doi:10.1111/psyp.12471 Kamp, S. M., & Donchin, E. (2015). ERP and pupil responses to deviance in an oddball paradigm. Psychophysiology, 52, 460–471. doi:10.1111/psyp.12378 12 i-Perception 10(5) Klingner, J., Tversky, B., & Hanrahan, P. (2011). Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology, 48, 323–332. doi:10.1111/j.1469- 8986.2010.01069.x Kloosterman, N. A., Meindertsma, T., van Loon, A. M., Lamme, V. A. F., Bonneh, Y. S., & Donner, T. H. (2015). Pupil size tracks perceptual content and surprise. European Journal of Neuroscience, 41, 1068–1078. doi:10.1111/ejn.12859 Laeng, B., Waterloo, K., Johnsen, S. H., Simonsen, S. S., Bakke, S. J., & La, T. (2007). The eyes remember it : Oculography and pupillometry during recollection in three amnesic patients. Journal of Cognitive Neuroscience, 19, 1888–1904. Lavın, C., Martın, R. S., & Jubal, E. R. (2014). Pupil dilation signals uncertainty and surprise in a learning gambling task. Frontiers in Behavioral Neuroscience, 7, 1–8. doi:10.3389/fnbeh.2013.00218 Liao, H. I., Yoneya, M., Kidani, S., Kashino, M., & Furukawa, S. (2016). Human pupillary dilation response to deviant auditory stimuli: Effects of stimulus properties and voluntary attention. Frontiers in Neuroscience, 10, 43. doi:10.3389/fnins.2016.00043 Loewenfeld, I. E. (1958). Mechanisms of reflex dilatation of the pupil—Historical review and experi- mental analysis. Documenta Ophthalmologica, 12, 185–448. doi:10.1007/BF00913471 Malach, R., Reppas, J. B., Benson, R. R., Kwong, K. K., Jiang, H., Kennedy, W. A., . . . Tootell, R. B. (1995). Object-related activity revealed by functional magnetic resonance imaging in human occip- ital cortex. Proceedings of the National Academy of Sciences, 92, 8135–8139. doi:10.1073/ pnas.92.18.8135 Martinovic, J., Mordal, J., & Wuerger, S. M. (2011). Event-related potentials reveal an early advantage for luminance contours in the processing of objects. Journal of Vision, 11, 1. doi:10.1167/11.7.1 Mathoˆ t, S. (2018). Pupillometry: Psychology, physiology, and function. Journal of Cognition, 1, 1–23. doi:10.5334/joc.18 Naber, M., Frassle, S., Rutishauser, U., & Einhauser, W. (2013). Pupil size signals novelty and predicts later retrieval success for declarative memories of natural scenes. Journal of Vision, 13, 11. doi:10.1167/13.2.11 Partala, T., & Surakka, V. (2003). Pupil size variation as an indication of affective processing. International Journal of Human Computer Studies, 59, 185–198. doi:10.1016/S1071-5819(03)00017-X Peysakhovich, V., Dehais, F., & Causse, M. (2015). Pupil diameter as a measure of cognitive load during auditory-visual interference in a simple piloting task. Procedia Manufacturing, 3, 5199–5205. doi:10.1016/j.promfg.2015.07.583 Preuschoff, K., Marius, B., Einh€auser, W., & Nieuwenhuis, S. (2011). Pupil dilation signals surprise : Evidence for noradrenaline’s role in decision making. Frontiers in Neuroscience, 5, 1–12. doi:10.3389/fnins.2011.00115 Quirins, M., Marois, C., Valente, M., & Seassau, M. (2018). Conscious processing of auditory regu- larities induces a pupil dilation. Scientific Reports, 8, 14819. doi:10.1038/s41598-018-33202-7 Riesenhuber, M., & Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nature Neuroscience, 2, 1019–1025. doi:10.1038/14819 Schroger, € E., & Widmann, A. (1998). Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology, 35, 755–759. doi:10.1017/S0048577298980714 Sirois, S., & Brisson, J. (2014). Pupillometry. Wiley Interdisciplinary Reviews: Cognitive Science, 5, 679–692. doi:10.1002/wcs.1323 Stanners, R. F., Coulter, M., Sweet, A. W., & Murphy, P. (1979). The pupillary response as an indi- cator of arousal and cognition. Motivation and Emotion, 3, 319–340. doi:10.1007/BF00994048 Stojanoski, B., & Cusack, R. (2014). Time to wave good-bye to phase scrambling: Creating controlled scrambled images using diffeomorphic transformations. Journal of Vision, 14, 6. doi:10.1167/14.12.6 How to cite this article Beukema, S., Jennings, B. J., Olson, J. A., & Kingdom, F. A. A. (2019). The pupillary response to the unknown: Novelty versus familiarity. i-Perception, 10(5), 1–12. doi:10.1177/

Journal

i-Perception – SAGE

Published: Sep 7, 2019

Keywords: object; form; recognition; pupillometry; pupil; dilation; constriction; novelty; familiarity

There are no references for this article.