Revisiting the looking at nothing phenomenon: Visual and semantic biases in memory search

ABSTRACT When visual stimuli remain present during search, people spend more time fixating objects that are semantically or visually related to the target instruction than looking at unrelated objects. Are these semantic and visual biases also observable when participants search within memory? We removed the visual display prior to search while continuously measuring eye movements towards locations previously occupied by objects. The target absent trials contained objects that were either visually or semantically related to the target instruction. When the overall mean proportion of fixation time was considered, we found biases towards the location previously occupied by the target, but failed to find biases towards visually or semantically related objects. However, in two experiments the pattern of biases towards the target over time provided a reliable predictor for biases towards the visually and semantically related objects. We therefore conclude that visual and semantic representations alone can guide eye movements in memory search, but that orienting biases are weak when the stimuli are no longer present.

Visually perceiving an object invokes the activation of various types of representation, from its visual features such as shape and colour to the semantic category it belongs to. However, as our cognitive capacities are limited, not all objects can be processed at the same time, and we selectively attend to certain objects over others. An important question is therefore which types of representation are available for prioritizing certain objects.
There is now substantial evidence that selection of visual objects is driven by both visual and semantic representations. Part of this evidence comes from the visual search task, in which people are instructed to search for a specific target item amongst other objects. A number of studies have shown that observers can select targets on the basis of both pictorial and word cues (e.g., Schmidt & Zelinsky, 2009Wolfe, Horowitz, Kenner, Hyle, & Vasan, 2004). Moores, Laiti, and Chelazzi (2003) investigated the influence of semantics more directly and found that objects that were semantically related to a verbal target instruction received more initial fixations than the other objects and slowed down responses on target absent trials (see also Meyer, Belke, Telling, & Humphreys, 2007;Telling, Kumar, Meyer, & Humphreys, 2010). Other evidence comes from the field of psycholinguistics, especially from the visual world paradigm. Here a visual display of multiple objects is presented before a spoken utterance. In the crucial displays, some objects in the display have a relationship with a specific word in the spoken utterance. Results show that people spend more time fixating related than unrelated objects, whether this relationship is semantic (e.g., Huettig & Altmann, 2005;Yee & Sedivy, 2006;Yee, Overton, & Thompson-Schill, 2009) or visual in nature (e.g., Dahan & Tanenhaus, 2005;Dunabeitia, Aviles, Afonso, Scheepers, & Carreiras, 2009;Huettig & Altmann, 2007;Huettig & Altmann, 2011;Rommers, Meyer, & Huettig, 2015;Rommers, Meyer, Praamstra, & Huettig, 2013). In de Groot, , we directly compared visual and semantic biases in the visual search and the visual world paradigm. When pictures were being previewed (as in visual world studies), we found that orienting biases towards semantically and visually related pictures arose around the same time (see also Huettig & McQueen, 2007), but when the pictures were not previewed (as in visual search studies) semantic biases were delayed, and less strong, compared to visual biases. Overall, these studies show that both the visual appearance and the semantics of objects can influence visual orienting.
In the above studies, the pictures were always present during search. 1 However, as has been shown in many studies (e.g., Gaffan, 1977;Sands & Wright, 1982;Wolfe, 2012), people can also search their memory for pictures that are no longer present. An interesting question is therefore whether these visual and semantic orienting biases would still be observable in memory search. A large number of studies has demonstrated that orienting biases towards the target object are observable in memory search: People make eye movements towards locations previously occupied by target objects, even though this was unnecessary for the task (e.g., Altmann, 2004;Dell'Acqua, Sessa, Toffanin, Luria, & Jolicoeur, 2010;Hoover & Richardson, 2008;Johansson & Johansson, 2014;Laeng & Teodorescu, 2002;Richardson & Kirkham, 2004;Richardson & Spivey, 2000;Spivey & Geng, 2001;Theeuwes, Kramer, & Irwin, 2011). These "looks at nothing" indicate that observers have formed episodic memory traces in which the visual object identities are bound to their respective locations. Referring to the target object then also leads to retrieval of its original location, which in turn triggers an eye movement (Ferreira, Apel, & Henderson, 2008;Richardson, Altmann, Spivey, & Hoover, 2009). In the current study, we investigated whether remembered objects that are either visually or semantically related to the target instruction, but are not the target, could also trigger "looks at nothing".
Two theories have been proposed that explain why people make eye movements towards visually and semantically related pictures when the visual stimuli are present during search. The first is the cascaded activation model of visual-linguistic interactions (Huettig & McQueen, 2007). According to this model, the word activates visual and semantic information in parallel, while the visual stimuli activate first a visual representation (as these are visual in nature) and only later a semantic representation. So whether the semantic or the visual aspects of objects are prioritized depends on the timing of the different stimulus types. This model successfully predicted the results of de Groot, . Although the cascaded activation model assumes lingering representations at least for the verbal input (otherwise there would be no biases in a standard visual search task where the word is presented before the onset of the pictures), no claims are being made about the strength of the representations associated with the pictures, once these are removed (i.e. memory search). Huettig, Olivers, and Hartsuiker (2011) proposed another general cognitive framework to explain the visual-linguistic interactions in standard conditions where the visual stimuli are still present. According to this model the basis of these interactions lies in stable knowledge networks in long-term memory, where all visual, semantic and phonological information related to the spoken target instruction and pictures is stored. The visual environment however variesthat is, the locations of the visual stimuli in visual world and visual search studies are typically randomand the binding between the long-term information of the target instruction and the spatial location of the pictures should only be temporary. The authors therefore proposed that the locus of binding between long-term identity and arbitrary location information is working memory. This model therefore predicts that when one source of information is activated, all temporarily related information will be activated too (similar to Ferreira et al., 2008;Richardson et al., 2009). Thus, according to this hypothesis a match on only a semantic or visual level would in principle also activate the associated location information, but whether this activity is sufficient to trigger an eye movement remains to be seen.
Here we ran three experiments investigating whether people are biased towards semantically or visually related objects in memory search. We included trials where the target was absent, but where one object was visually (but not semantically) related, one object was semantically (but not visually) related and one object was unrelated to the target instruction. If there are visual or semantic biases in memory search equivalent to those found in conditions where the pictures remain on the screen during search, then we should observe that people spend more time fixating the previous locations of the visually or semantically related objects than the location previously occupied by the unrelated object. Target present trials were inspected as well to assess whether previous findings could be replicated, i.e., people spend more time looking at locations previously occupied by targets (e.g., Altmann, 2004;Hoover & Richardson, 2008;Spivey & Geng, 2001). Furthermore, as will become evident later, the target present trials proved to be useful in predicting the time course of the visual and/ or semantic biases. Experiment 1 directly compared a condition where the visual stimuli were present during search to a condition where they were absent (memory search). In two follow-up experiments (Experiments 2 and 3), we tested the hypothesis that biases may need more time to emerge in memory search, and hence we manipulated the preview time of the pictures, and the interstimulus intervals (ISIs) between picture offset and word onset. Figure 1 illustrates the procedure. People saw three pictures for 250 ms before they received the target instruction. During search, pictures could either be present or absent (memory search). The experiment included target absent and target present trials. The target absent trials contained two objects that were related to the target instruction: either semantically or visually. In the target absent trials we were interested whether participants would spend more time fixating the (previous) locations of the related pictures than the (previous) location of the unrelated picture. In the target present trials we expected people to spend more time fixating the (previous) location of the target object than the (previous) locations of the non-targets.

Participants
In this experiment a planned number of 24 Dutch native speakers (eight males, aged 18-34, average 23.1 years) participated for course credits or money. No participant was replaced. All reported not to suffer from dyslexia (or any other language disorder) and/or colour blindness. None had participated in one of our earlier experiments. This study was conducted in accordance with the Declaration of Helsinki and approved by the Scientific and Ethical Review Board of the Faculty of Behaviour and Movement Sciences at the Vrije Universiteit Amsterdam.

Stimuli and apparatus
There were 240 trials in this experiment: 120 target present and 120 target absent trials. Each trial contained three objects. On target absent trials, two of the three objects were related to the target Figure 1. An example of a target absent trial in Experiment 1 to illustrate the experimental design. Here the target instruction was the word "banana" (presented in Dutch as "banaan"). The monkey is the semantically related picture, the canoe is the visually related picture, and the tambourine is the unrelated picture. In one condition the pictures were present during search, whereas in the other they were absent. instruction: either semantically or visually. We used the same stimuli as in de Groot,  but the original size of the pictures (in pixels) was now scaled with factor 0.6 instead of 0.5. The different object types were matched on several linguistic and visual parameters and the visual and semantic relationships were established by independent raters (de Groot, . Appendices A and B list the target absent and target present trials, respectively. All pictures were presented on a grey background (RGB: 230,230,230) with an equal distance from fixation cross (distance from the middle of the pictures to fixation cross: 210 pixels, 4.5°). The exact location of each picture was randomized, but pictures were always presented 120 degrees apart. Each object was shown only once, as neither trials nor pictures were repeated during the experiment.
Two participants were tested on a HP Compaq 6300 Pro SFF computer, whereas the other 22 were tested on a HP DC7900CMT computer. Stimuli were presented on a Samsung Syncmaster 2233RZ monitor (refresh rate of 120 Hz and a resolution of 1680*1050). The distance from the screen to chin rest was 75 cm. OpenSesame (version 2.8.2 for two participants and 2.8.3 for the others; Mathôt, Schreij, & Theeuwes, 2012) was used to present the stimuli. The left eye was tracked with an Eyelink 1000 Desktop Mount with a temporal and spatial resolution of 1000 Hz and 0.01°, respectively. Words were presented through headphones (Sennheiser HD202) connected to the computer with a USB Speedlink soundcard.

Design and procedure
The study used a 2 by 2 within-subject design with Trial type (target absent and target present trials) and Visual stimuli presence (present and absent) as factors. In total there were four blocks of 60 trials. Trial type was mixed within blocks (50% each), whereas Visual stimuli presence was blocked. Participants were told before each block whether the pictures would remain present during search, or would disappear before search (memory search). Two practice trials were presented prior to the experiment. During the experiment, participants did not receive feedback about their performance. Stimuli were counterbalanced and randomized in such a way that per two participants stimuli appeared in each condition (i.e., whereas for participant A the stimuli were shown in the condition where the pictures were present during search, the same stimuli were shown for participant B in the condition where the pictures were absent).
Each trial started with a drift correction that was triggered by pressing the space bar. After the response a blank screen was presented for 600 ms, followed by a display of three objects. After 250 ms a word was presented through headphones. In one condition the pictures were removed just prior to word onset, whereas in the other condition the pictures stayed on the screen during word presentation and search. The participant indicated whether the target was present or absent by pressing respectively "J" or "N" on the keyboard. After the response participants heard a soft click sound and the same screen stayed on for another 1000 ms while eye-tracking continued. In both conditions this was followed by a blank screen. A new trial started after 600 ms (see also Figure 1).

Eye movement analyses
For all experiments, we defined a region of interest (ROI), a circular area with a radius of 180 pixels (4°) centred on each picture. Note that ROIs could contain pictures (Experiment 1 in the condition where the visual stimuli remained present during search) or could be empty (memory search condition of Experiment 1, Experiment 2 and Experiment 3). Thus, we will refer to fixations towards ROIs instead of towards objects. The dependent measure was the proportion of time that people spent fixating an ROI within the critical time period, running from word onset until the average reaction time of each condition (from now on called proportion of fixation time). Since the average reaction times differed somewhat for Trial type and Visual stimuli presence, this led to slightly different time periods across conditions. Eye movement data was included for those trials where observers responded correctly. Greenhouse-Geisser corrected values are reported when Mauchly's sphericity test was violated. Confidence intervals (95%, two-tailed) for within-participants designs were also calculated (as described in Cousineau, 2005;Morey, 2008).

Manual responses
In all analyses Greenhouse-Geisser corrected values are reported when Mauchly's sphericity test was violated. A repeated measures analysis of variance (ANOVA) on search times of the correct trials, with Trial type (target absent and target present trials) and Visual stimuli presence (present and absent) as factors revealed an effect of both Trial type, F(1,23) = 41.283, p < .001, h 2 p = 0.642, and Visual stimuli presence, F(1,23) = 4.646, p < .05, h 2 p = 0.168, but no interaction, F(1,23) = 2.409, p = .134. Search was faster on target present (M = 1200 ms, SD = 179) than on target absent trials (M = 1370 ms, SD = 265), and was also faster when the pictures were present (M = 1235 ms, SD = 262) than when they were absent (M = 1340 ms, SD = 228).
Eye movement data Overall mean proportion of fixation time. Figure 2 displays the overall mean proportion of fixation time ("P(fix)") on the different types of ROIs. First, a repeated measures ANOVA on the overall mean proportion of fixation time with Trial type (target absent and target present trials) and Visual stimuli presence (present and absent) as factors revealed that observers spent more time fixating the ROIs in the condition where the pictures were present during search than in the condition where they were absent, F(1,23) = 29.988, p < .001, h 2 p = 0.566. This was to be expected. There was no effect of Trial type, F(1,23) = 0.829, p = .372, nor an interaction, F(1,23) = 0.471, p = .499. Our main interest was in the condition where the pictures were removed prior to search (memory search), and the condition where the pictures were present only served as a manipulation check. These conditions were therefore analysed separately. The target absent trials were the trials of interest as they contained the semantically and visually related objects. We also analysed the target present trials to see whether we could observe biases towards the target.
For target absent trials a repeated measures ANOVA was done on the mean proportion of fixation time Figure 2. The overall mean proportion of fixation time ("P(Fix)") within the time period from word onset until the average RT of each condition in Experiment 1. At the top data is shown for when pictures were present during search (A and B), whereas at the bottom it is displayed for when they were absent (C and D). On the left are the target absent trials (A and C), while on the right are the target present trials (B and D). Errors bars are the confidence intervals (95%, two tailed) for within-participants designs (Cousineau, 2005;Morey, 2008).
with ROI type (semantically related, visually related and unrelated) as factor. In the condition where the pictures were absent during search, there was no effect of ROI type, F(2,46) = 0.120, p = .887. People did not spend a larger proportion of time fixating semantically or visually related ROIs than unrelated ROIs (see Figure 2C). In the condition where the pictures were present, however, there was an effect of ROI type, F(2,46) = 29.217, p < .001, h 2 p = 0.560. Figure 2(A) and follow-up t-tests showed that participants spent a larger proportion of time fixating both visually and semantically related ROIs, when compared to unrelated ROIs, respectively, t (23) = 7.150, p < .001, r = 0.83, and t(23) = 2.913, p < .01, r = 0.519. Moreover, proportion of fixation time was higher for visually related pictures than for semantically related pictures, t(23) = 4.883, p < .001, r = 0.713. So, whereas we could observe semantic and visual biases in the condition where the pictures were presentreplicating earlier work of de Groot,  and Huettig and McQueen (2007) these appeared to be absent in the memory search.
For the target present trials, we took the average of the two non-target sets, as non-targets were randomly placed in set 1 or 2 (Appendix B). However, paired ttests on the proportion of fixation time confirmed that there was no difference between the two nontarget sets, in both conditions, where the objects were present during search, t(23) = 1.932, p = .066, and where the objects were absent, t(23) = 0.994, p = .330. When the pictures were present during search, we found that observers fixated target ROIs considerably more than non-target ROIs, t(23) = 12.911, p < .001, r = 0.937 ( Figure 2B). Importantly, a highly reliable similar effect was also found in the condition where the pictures were absent during search, t (23) = 5.521, p < .001, r = 0.755 ( Figure 2D). Thus, replicating previous "looking at nothing" studies, we observed clear orienting biases towards the previous locations associated with the target objects. Orienting biases can thus arise in memory search condition. Moreover, the overall absence of the semantic and visual biases in memory search was not caused by a lack of eye movements per se, as the time spent in total on ROIs was comparable between the target present (0.53) and target absent trials (0.53).

Time course analyses.
There is the possibility that taking the overall mean proportion of fixation time obscures underlying information. Specifically, as a single, rather static, group measure, it may not capture information available in the dynamics of fixation patterns. Figure 3 shows the biases towards the target ROIs (on target present trials) and towards the visually and semantically related ROIs (on target absent trials) relative to the neutral distractor baseline (i.e., difference scores in proportion of fixation time) as a function of time into the trial from 0 to 1400 ms after word onset, in 200 ms time bins. We chose 1400 ms since on average most responses had been given by then. Note that for target absent trials the neutral distractor baseline consisted of the proportion of fixation time on the unrelated pictures, and for the target present trials it consisted of the proportion of fixation time on the non-targets.
In the following steps we assessed whether the time course patterns merely reflected noise, or actually carried information about visual and semantic orienting biases. The idea behind these analyses is that the time course of the orienting bias towards the target (on target present trials) provides a reliable predictor of the biases towards visually and semantically related ROIs on target absent trials. Moreover, if the patterns for visual and semantic biases across time indeed reflect non-random biases, then the time course of the visual bias may also be predictive of the time course of the semantic bias. To this end, we computed the correlation between the target bias, visual bias, and semantic bias across time bins, using a bootstrapping procedure. In 10,000 iterations, the bootstrapping procedure randomly resampled the time bin data, with participant as index. The correlation of the average time series data for target and visual bias, target and semantic bias, and visual and semantic bias was then computed for each bootstrap sample, using Pearson correlations (r). The ensuing distributions of r-values was then Fisher Z-transformed (using hyperbolic arctangent transformation) to correct for skewedness. From this transformed distribution the two-tailed 95% confidence intervals were computed, which were inverse transformed back to the original r-space (-1 to 1). We report the median r together with these confidence intervals, and consider the correlation significant when 0 was not included in the interval. A significant r then means that the time course of one condition was reliably predicted by the time course of another condition.
When the pictures remained on screen during search, target fixation biases were indeed predictive of biases towards visually related ROIs, r = 0.927 (CI: 0.864; 0.963), as well as towards semantically related ROIs r = 0.519, which was approaching significance under the two-tailed test (CI: −0.033; 0.843; significant under a one-tailed test). Biases towards visually related ROIs were also predictive of biases towards semantically related ROIs, r = 0.771 (CI: 0.309; 0.946). The pattern was different for the memory search condition. Here biases towards the target did not reliably predict biases towards visually related ROIs, r = 0.193 (CI: −0.878; 0.925), nor towards semantically related ROIs, r = −0.463 (CI: −0.957; 0.758). Neither did the time courses of visual biases and semantic biases correlate significantly, r = 0.315 (CI: −0.710; 0.925).
We conclude that participants stored episodic memories of the objects including their locations, as is evident from target present trials in the memory search condition. However, such episodic memories are insufficient to trigger strong overall eye movement biases on the basis of visual or semantic similarity alone.
Additional eye movement measures. Tables 1 and 2 show, for the target absent and target present trials respectively, the gaze duration (i.e., how long did people fixate an ROI between first entering and first leaving it) and proportion of total fixations. A fixation was only included when it was made in the time period from word onset until the reaction time of that specific trial. For these analyses one participant was excluded, for a lack of sufficient numbers of eye movements in this time period.
For the target absent trials a repeated measure ANOVA on gaze duration with ROI type (semantically related, visually related and unrelated) as a factor showed a significant effect for when the pictures were present during search, F(2,44) = 20.774, p < .001, h 2 p = 0.486, but not when the pictures were absent during search, F(1.564,34.405) = 0.718, p = .462. Post-hoc ttests revealed that when the pictures were present during search, participants fixated in the first pass the visually related ROIs longer than the unrelated ROIs, t (22) = 5.822, p < .001, r = 0.779, but that this was not the case for the semantically related ROIs, t(22) = 0.681, p = .503. The same analyses on the proportion of total fixations revealed a significant effect when the pictures were present, F(2,44) = 23.530, p < .001, h 2 p = 0.517, but not when they were absent F(1,22) = 0.566, p = .460. Here post-hoc t-tests showed that when the pictures were present during search, the semantically and visually related ROIs received more fixations than the unrelated ROIs, t(22) = 2.284, p < .05, r = 0.438 and t(22) = 7.347, p < .001, r = 0.843, respectively.
For the target present trials a paired t-test showed that when the pictures remained on the screen during search, the gaze duration was higher for the target than for the average of the non-targets, t(22) = 9.542, p < .001, r = 0.897. In the condition where the pictures were removed prior to search, there was a trend towards significance in the same direction, t(22) = 1.795, p = .086. In both conditions the proportion of total fixations was higher on the target than on the average of the non-targets, t(22) = 5.355, p < .001, r = 0.752 and t(22) = 27.076, p < .001, r = 0.985 for when the pictures were absent and present, respectively.

Experiment 2
In Experiment 1 we observed no semantic or visual biases in the memory search condition when assessing overall mean proportion of fixation time or the time course patterns. However, we always presented the word immediately after picture offset, at an ISI of 0 ms, as we a priori thought that the representations would be the strongest directly after the pictures were removed. But the reverse may also be true, in that representations build up over time, and/ or need time to be consolidated (e.g., Nieuwenstein & Wyble, 2014). Therefore, in Experiment 2 we repeated the procedure, but added two extra ISIs (750 ms and 3000 ms). If the memory representations are indeed becoming stronger over time, then we would expect semantic and visual biases to appear at longer ISIs. As our stimulus set was limited, we dropped the visual stimuli present condition. The target present trials still served as a manipulation check, and as a predictor for the time course patterns.

Participants
Twenty-four people (three males, aged 18-27, average 21.1 years) participated in Experiment 2 for course credits or money. The same requirements as Experiment 1 were applied. One participant was replaced because this person reported multiple times to be extremely tired. Another participant showed an extremely high error rate, and was excluded from the analyses. Including this participant did not alter the conclusions. 2

Stimuli, apparatus, design and procedure
The methods were the same as the condition in Experiment 1 where the pictures were removed prior to search, except for the following. We varied the timing between picture offset and word onset. The word was either presented directly after picture offset (ISI of 0 ms) or with a delay (ISI of 750 ms or 3000 ms). This led to different stimulus onset asynchronies (SOAs) of 250, 1000 and 3250 ms. The study thus used a 2 by 3 within subject design with Trial type (target absent and target present trials) and ISI (0, 750 and 3000 ms) as factors. Both Trial type and ISI were mixed within each block (each condition presented equally often within a block). Stimuli were randomized and counterbalanced per six participants (i.e., after six participants every stimulus set had been shown equally often in each condition). All participants were tested on a HP Compaq 6300 Pro SFF computer and stimuli were presented using OpenSesame version 2.8.2 (Mathôt et al., 2012). of 750 ms also differed from the condition with 3000 ms, t(22) = 3.375, p < .01, r = 0.584. The interaction was also significant, F(2,44) = 5.873, p < .01, h 2 p = 0.211. In both target present and target absent trials the proportion of errors increased with higher ISIs but this increase was stronger for target present than for target absent trials.

Manual responses
Eye movement data Overall mean proportion of fixation time. Figure 4 displays the overall mean proportion of fixation time for target absent and target present trials, for different ROI types. For the target absent trials a repeated measures ANOVA on mean proportion of fixation time with ROI type (visually related, semantically related and unrelated picture) and ISI (0, 750 and 3000 ms) as factors showed no effect of ISI, F(2,44) = 4.65, p = .631, nor an interaction, F(4,88) = 0.176, p = .950. There was however an effect of ROI type, F (2,44) = 5.101, p < .05, h 2 p = 0.188. Figure 4(A) and subsequent t-tests show that, across the board, participants spent slightly more time fixating semantically related ROIs than visually related ROIs and unrelated ROIs, respectively, t(22) = 2.542, p < .05, r = 0.476 and t(22) = 3.031, p < .01, r = 0.543, suggestive of a semantic bias. There was however no evidence for a visual bias, t(22) = 0.126, p = .901.
Like Experiment 1, the time spent in total on ROIs was comparable between the target present (0.65) and target absent trials (0.63). For the target present trials, t-tests showed no difference between the two non-target sets in each ISI, respectively, for an ISI of 0, 750 and 3000 ms, t(22) = 1.161, p = .258, t(22) = 0.147, p = .885 and t(22) = 1.453, p = .160, therefore we took the average of the non-targets as in Experiment 1. A repeated measures ANOVA on proportion of fixation time with ROI type (target and average of the non-targets) and ISI (0, 750 and 3000 ms) as factors showed no effect of ISI, F(2,44) = 0.091, p = .913, nor an interaction, F(2,44) = 0.207, p = .814. There was however an effect of ROI type, F(1,22) = 10.318, p < .01, h 2 p = 0.319. As Figure 4(B) shows, people spent more time fixating the target than non-targets.
Time course analyses. We performed the same correlation analyses as in Experiment 1 to assess whether the time course of the bias towards the target (on target present trials) is predictive of the biases towards visually and semantically related ROIs on target absent trials. To increase power we collapsed across ISIs, as the number of fixations underlying each time bin would otherwise be quite low.   semantically related, visually related, and target ROIs, implying that information on semantic and visual relationships between the word and the pictures must have been available.
Note that here biases towards the target ROIs predicted biases towards the visually and semantically related ROIs, whereas in Experiment 1 there was no such relationship. We have no satisfactory explanation for this discrepancy, other than that the current experiment had more power (collapsed across ISIs), and/or that the task context of Experiment 1 may have invited observers to try and avoid looking at related ROIs. Such a tendency may have been induced by the condition where the pictures remained visible and observers noticed being drawn towards related ROIs. Either way, Experiment 3 served to see whether the pattern of Experiment 2 could be replicated.
Additional eye movement measures. As in Experiment 1 we examined the gaze duration and the proportion of total fixations towards the different ROIs on both the target absent and target present trials (see Tables 1 and 2). For these analyses we collapsed all ISIs together. For the target absent trials the repeated measures ANOVA on gaze duration with ROI type (semantically related, visually related and unrelated) as factor revealed no significant effect, F(2,44) = 1.351, p = .270. The same ANOVA on proportion of total fixations also showed no significant effect, F(2,44) = 1.565, p = .22. For the target present trials, paired ttests showed no effect on gaze duration, t(22) = 1.554, p = .134, but did show an effect on the proportion of total fixations, t(22) = 3.092, p < .01, r = 0.55.

Experiment 3
To further promote orienting biases towards memorized semantically and/or visually related ROIs, we increased the presentation time of the pictures from 250 ms to 2000 ms in Experiment 3. The idea was that with a longer preview, people would have more time to encode the pictures resulting in stronger memory representations, which in turn would potentially lead to stronger biases. We again varied the ISIs between picture offset and word onset.

Participants
Twenty-four people (nine males, aged 18-28, average 21.0 years) participated in Experiment 3 for course credits or money. The same requirements as Experiment 1 were applied. Two participants were replaced: one participant because of not following both general and specific task instructions (specifically: using the phone during testing), whereas the other one reported having been unable to understand many of the words presented (despite reporting to be a Dutch native speaker). Accidentally two additional participants were run. Including these participants in the analyses would lead to counterbalancing problems. We therefore repeated all analyses but now with two participants replaced with the additionally ran participants. This did not alter the conclusions. 3

Stimuli, apparatus, design and procedure
The methods were the same as Experiment 2, except that the pictures were now presented for 2000 ms instead of 250 ms. As the same ISIs were used, this led to SOAs of 2000, 2750 and 5000 ms.

Manual responses
The same repeated measures ANOVA as Experiment 2 was conducted on search RTs. This showed an effect of Trial type, F(1,23) = 26.407, p < .001, h 2 p = 0.534. Search was faster on target present than on target absent trials (respectively, M = 1172 ms, SD = 197 and M = 1301 ms, SD = 192). There was no effect of ISI nor an interaction, Eye movement data Overall mean proportion of fixation time. Figure 6 shows the mean proportion of fixation time for the different ROI types on target absent and target present trials. For target absent trials ( Figure 6A), the same repeated measures ANOVA as in Experiment 2 revealed an effect of ISI, F(1.471,33.827) = 6.905, p < .01, h 2 p = 0.231. Follow-up paired t-tests showed that the condition with an ISI of 0 ms (M = 0.28, SD = 0.03) differed significantly from the ISI of 750 ms (M = 0.26, SD = 0.04) and 3000 ms (M = 0.25, SD = 0.05), t (23) = 3.031, p < .01, r = 0.534 and t(23) = 2.864, p < .01, r = 0.513, respectively. There was no difference between the conditions with an ISI of 750 ms and 3000 ms, t(23) = 1.304, p = .205 (see also Figure 6C). More importantly, there was no effect of ROI type, nor an interaction, F(2,46) = 0.563, p = .573 and F (4,92) = 0.058, p = .994, respectively.
Time course analyses. We performed the same correlation analyses as in Experiments 1 and 2. Like Experiment 2, we collapsed across ISIs. Figure 5(B) shows the difference scores in proportion of fixation time for the target ROIs, visually related ROIs, and semantically related ROIs, across time. The graph again Figure 6. The overall mean proportion of fixation time ("P(Fix)") within the time period from word onset until the average RT of each condition in Experiment 3. At the top data is shown when collapsed across ISIs (A and B), whereas at the bottom it is displayed for each ISI separately (C and D). On the left are target absent trials (A and C), while on the right are target present trials (B and D). Errors bars are the confidence intervals (95%, two tailed) for within-participants designs (Cousineau, 2005;Morey, 2008).
suggests a similarity in time course. Indeed, target biases reliably predicted biases towards visually related ROIs, r = 0.873 (CI: 0.025; 0.988), as well as towards semantically related ROIs, r = 0.943 (CI: 0.593; 0.992). Biases towards visually related ROIs were also predictive of biases towards semantically related ROIs, r = 0.918 (CI: 0.119; 0.994). Thus, the time course of the proportion of fixation time was again similar for semantically related, visually related, and target ROIs, implying that visual and semantic representations must have been available.
Additional eye movement measures. The same analyses were done as in Experiment 2. For the target absent trials the repeated measures ANOVA on gaze duration with ROI type (semantically related, visually related and unrelated) as a factor revealed no significant effect, F(2,46) = 0.812, p = .450. Neither did the ANOVA on the proportion of total fixations, F(2,46) = 0.138, p = .872. For the target present trials, paired ttests showed a trend towards significance for gaze duration, t(23) = 1.736, p = .096, and a significant effect for proportion of total fixations, t(23) = 6.580, p < .001, r = 0.808 (see Tables 1 and 2 for the averages and standard deviations).

General discussion
In three experiments we explored eye movement behaviour in memory search. In all experiments we found that people spent overall more time fixating locations previously occupied by the target than locations previously occupied by non-targets. This matches earlier work showing that people "look at nothing" when retrieving memories (e.g., Altmann, 2004;Hoover & Richardson, 2008;Spivey & Geng, 2001). Here the main question was whether participants would also spend more time fixating previous locations of objects that only matched the target instruction in one aspect (either semantically or visually). Two of the three experiments showed neither semantic nor visual biases in overall mean proportion of fixation time. We therefore conclude that semantic and visual biases are often too weak to generate measurable increases in the overall mean proportion of fixation time. However, Experiment 2 suggests that occasionally biases may be strong enough to also show in the overall analyses. Moreover, Experiments 2 and 3 showed that the pattern of fixations towards the target across time was predictive of the pattern of fixations towards both visually and semantically related objects (with the visual time course pattern also being predictive of the semantic time course pattern). This can only occur when visual and semantic information was available during memory search. We thus conclude that target-related visual and semantic representations are being activated, and form the basis for dynamic fixation patterns towards and away from visually and semantically related objects, but that such patterns are more complex than can be captured by a single overall statistic such as mean fixation bias.
The question is then why we did not observe semantic and visual orienting biases when looking at the overall mean proportion of fixation time, while at the same time there was evidence for such biases when taking the time courses into account. As noted earlier, the overall mean obscures subtle differences that may appear in the time course. For example, two participants may each show a bias, but with a time course that is different, leading to a partial cancelling out. In fact, even within a participant, an early bias towards an object may be cancelled out by a late bias away from that object. Such a bias away could represent the fact that after having looked at the location of the related object, only the remaining locations are left. It may also represent a more strategic effort of trying to avoid retrieving distracting information, a mechanism that might also explain the lack of any bias in Experiment 1. Future research could look more into the influence of task demands, for example exploring the influence of alternating a memory search condition with a condition where the visual stimuli were present (like Experiment 1).
Another interesting question is why people make more eye movements towards semantically and visually related objects when the pictures remain present during search. Earlier, it has been proposed that "looks at nothing" are actually strategic (e.g., Johansson & Johansson, 2014;Laeng & Teodorescu, 2002). That is, fixating previous target locations during retention period or when the target instruction is presented helps to retrieve the correct object, which will improve task performance. Indeed, researchers have shown that memory performance is better when eye movements are directed to locations congruent with the utterance, compared to when they are incongruent (Johansson & Johansson, 2014). Note that strategically moving the eyes is only helpful with the target object, and not with objects that are only semantically or visually related to the target instruction. So maybe it is just much harder to ignore the stimuli or to suppress an eye movement when the images are actually there. Measures of covert attention and suppression, such as the N2pc and Pd components of the EEG signal (Eimer, 1996;Hickey, Di Lollo, & McDonald, 2009;Luck & Hillyard, 1994) could be a more sensitive way of testing semantic and visual biases in memory search.
Although a number of questions remain, we believe our study has implications for the literaturefirst of all for the cascaded activation model of visual-linguistic interactions. Earlier we have found that the lingering semantic and visual representations of the word alone are strong enough to trigger eye movements . So far, the model has not made any predictions with regard to what would happen to the different types of representations when the pictures are removed from the screen. The current data suggests that semantic and visual representations remain available in a location-specific fashion for at least a few seconds, leading to a dynamic pattern of eye movement biases that are similar to what has been shown when visual stimuli remain present. At the same time our data puts boundary conditions on this model as it suggests that the lingering semantic and visual representations of the pictures alone are not enough to trigger stable overall biases in eye movements. Instead, the model would need to account for more subtle time course patterns of looking towards and looking away from related objects, and how this may be affected by different strategies. Secondly, in the past researchers have investigated why we see orienting biases towards objects in memory, but so far researchers have focused on the target object. Here we show that activation of only a visual or semantic representation level cannot always be captured in a single summary statistic like the overall mean. However, with the correlation analyses we do offer a promising method that appears to be more sensitive to information hidden in the eye movement patterns towards visually and semantically related objects. This method exploits the differences in dynamics of eye movement biases, rather than being hindered by such variability, and reveals that while overall biases remain weak, the eye movement system is sensitive to visual and semantic relationships even in memory search.