Visual acuity response when using the 3D head-up display in the presence of an accommodation-convergence conflict

Visual discomfort and fatigue due to accommodation-convergence (AC) conflict in stereoscopic displays has been widely reported, but little is known about its impact on visual acuity, particularly when automotive three-dimensional (3D) head-up displays (HUDs) are involved. This paper presents a study on the visual acuity response when an indigenously developed 75% transparent retroreflective screen is used as a windshield 3D HUD. The simulated optical collimation technique was used to provide the virtual content at a farther depth (i.e. on the road while driving). Two user test experiments were performed. The first test was performed under the see-through condition, where the real scene (i.e. roadside view) was perceived through the 3D HUD, while the second test was performed under the simulated collimation condition, where a stereo-collimated virtual content was projected on the HUD at a farther depth. The results showed a slightly declining trend (from 20/20 to 20/25) in visual acuity response when the HUD screen was placed between the viewer and the scene. An inverse relation between the amount of AC conflict and visual acuity was observed under the simulated collimation condition. The >100 cm user-to-screen distance was found to be comfortable, providing the highest acuity response.


Introduction
See-through screens and augmented reality (AR) displays have been extensively explored as automotive head-up displays (HUDs) to provide navigation and other driver assistance information without causing distraction. Different types of HUD have been developed, but most of them provide a virtual image at a fixed depth. The simplest HUD is based on a liquid crystal display (LCD) lying flat on the dashboard, and a partially reflective beam combiner oriented at a 45°angle. More sophisticated HUDs use viewing optics to provide a virtual image ahead of the car, but the field of view (FOV) is limited (typically < 10°) due to the small étendue of the projection system. The waveguide-type combiners and self-emissive high-brightness displays developed of late also show promise for HUDs [1]. Holographicoptical-elements-based HUDs have also been demonstrated to provide an asymmetric FOV [2]. On the other hand, the development of three-dimensional (3D) HUDs is usually limited to lenticular-screenbased multi-view displays [3], bandpass-filter-based CONTACT  wavelength-division-multiplexed systems [4], and transparent-screen-based stereo 3D HUDs [5,6]. The visual discomfort and fatigue in stereoscopic 3D displays and its relationship with the accommodationconvergence (AC) conflict has been studied in various ways. In [7], a new stereoscopic display was developed, and disparity and focus cues were provided to study the visual effect and level of fatigue due to the absence of focus cues. The effect of the viewing distance and direction of a stereoscopic display on visual discomfort has also been investigated [8]. The importance of accommodation and presence of AC conflict in head-mounted displays is also presented in the literature [9]. The impact of 3D HUDs on visual acuity in the presence of an AC conflict, however, is unknown.
In this study discussed in this paper, the impact of a large-size 3D HUD on the human vision was investigated and evaluated by observing the visual acuity response under see-through and simulated colimitation conditions. To the best of these authors' knowledge, this has been the first study related to the visual acuity response of large-size 3D HUDs. The optical collimation feature inherent in the typical HUDs was provided by employing simulated collimation, where the parallax between the stereo views was adjusted to display imagery at a defined depth. The procedure and results of the two different user tests conducted using a see-through retroreflective HUD screen are presented [10]. The results are extendable to any type of stereo 3D HUD where the virtual imagery is displayed at a fixed distance from the user. Section II discusses the background of stereo 3D displays and the AC conflict. Section III presents the see-through retroreflective screen as a 3D HUD, and simulated collimation. Section IV explains the procedures of the user tests, and section V discusses the results of and key findings from the test experiments.

Stereo 3D displays
The primary source of the human brain's depth extraction and 3D perception is the fusion of two images each acquired by only one eye. The amount of horizontal displacement (disparity) for each object appearing in such two images is used in a variety of stereo display technologies to provide depth perception. The first stereoscopic device (stereoscope) was introduced by Charles Wheatstone in 1832. The stereo 3D displays used today range from direct-view stereoscopic displays, which require eyewear, to glassless stereo 3D (autostereoscopic) displays. The eyewear-based stereo displays mainly use one of three different techniques for view separation: (1) color multiplexing; (2) polarization multiplexing; and (3) time multiplexing [11]. A color-multiplexed stereo display separates the two images by color-coding the left and right views, which are then perceived by the viewer wearing the matchingcolor filter-based glasses. The anaglyph color-codingbased glasses and displays are the best examples of color-multiplexed displays, which were developed in the early twentieth century. In the polarization-multiplexed stereo displays, the polarization directions of light for the stereo views are made mutually perpendicular while the screen is viewed with polarization-matched glasses, which transmit the light (image) intended for a particular eye only. The time-multiplexed displays, on the other hand, use high-refresh-rate displays to display stereo views in the sequence, which are then separated using the active-shutter-based synchronized eyewear. The polarized and active-shutter-glasses-based 3D displays are widely used for the consumer-grade display systems (i.e. 3D TVs and projectors) and in the movie industry ( Figure 1).

Accommodation-convergence conflict
All stereoscopic 3D displays are affected by the AC conflict. The accommodation is the ability of the eye lens to change its focal length and to focus on different objects at different depths to draw a sharp image on the retina [7][8][9][10][11][12]. On the other hand, convergence is the rotation of both eyeballs in the inward direction to converge on an object (fixed depth) and to produce a brain-fused single image [13,14]. Under the natural viewing conditions, the accommodation and convergence are coupled and change together synchronously when the viewer looks at different objects at different depths, as shown in Figure 1(a). In the stereoscopic 3D displays, the AC relation is disturbed when the images are displayed on the screen at a fixed distance, but the amount of convergence varies with the level of disparity, as seen in Figure 1(b). The mismatch or conflict forces the eyes to focus on the screen distance but to converge at a different distance, resulting in the reduced sharpness of the perceived image and in visual discomfort. The amount of discomfort can increase further when head-mounted displays are involved [9]. The amount of AC conflict is calculated as the difference in the accommodation and convergence distances in dioptres, which can be expressed as: where D S is the screen distance (displayed image distance) and D C is the convergence distance (perceived depth distance) from the viewer. The amount of conflict is negative when the perceived image is in front of the screen, and positive when the perceived image is inside the screen, as shown in Figure 1(b).

3d head-up display using a transparent retroreflective screen
A retroreflective surface contains microstructures that refract and/or reflect an incoming light ray, such that it is turned back towards the source with narrow-angle scattering and provides high brightness to the viewer standing close to the light source, as illustrated in Figure 2(a). The retroreflective materials were primarily developed for road safety applications and airport runways [15,16], but since the past few decades, they have also been explored as light-efficient personalized screens for wearable/head-worn projection display applications [17,18]. The retroreflective property of the surfaces enables the use of low-power display sources (i.e. pico projectors) to provide very bright imagery. A significantly transparent version of such reflective screens was developed and demonstrated for AR display applications [19,20]. The transparent retroreflective screen was fabricated by partially patterning an optically clear glass with retroreflective microspheres. By controlling the filling factor of retroreflective microspheres, the screen provides 50%-75% transparency and up to 10 times the brightness provided by a regular non-transparent diffused screen [19]. Figure 2(b) shows a 75% transparent screen displaying a virtual dashboard imagery. The stereo views projected by the two pico projectors are separated through the retroreflective property of the screen, while the stereo crosstalk is minimized by the polarization maintainability of the retroreflective microspheres.

Simulated collimation
A typical HUD superimposes the virtual information on top of the real-world scene, such that the user/driver is not required to have additional accommodation and convergence efforts to perceive the virtual content. To provide such depth effect, the optical collimation technique is used where the additional collimation optics is used together with a projector or an image source. The collimation optics creates an image display at a farther depth, but it usually limits the FOV, eyebox, and brightness of the systems. The optical-collimation-based HUDs are still widely used in aircrafts, the military, and the automotive industry. An alternative way to provide such viewing effect is to use the stereoscopic simulated optical collimation technique [5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22]. Simulated collimation is based on the amount of parallax between stereo views. In a typical vehicle driving scenario, when the parallax between the two images is set close to the interpupillary distance (IPD), the displayed image appears away from the viewer, and the driver's eyes already focused and converged on the roadside scene perceive a single image through the windshield screen, as illustrated in Figure 3(a). The real focus of the projected imagery, however, lies on the windshield screen plane, which generates the AC conflict. Simulated collimation has been explored in the past using transparent (dot-patterned) diffused projection  screens [21], LCD screens with partially coated glasses [6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22], and computer-graphics-based simulated environments [23].

User study
To investigate and understand the effect of the AC conflict due to simulated collimation and screen transparency on the visual acuity of the viewer, a subjective study was conducted to determine the visual acuity response using the Snellen visual acuity chart [24,25], which is widely used for the said purpose.

Apparatus
The test setup consisted of an in-house-designed and -fabricated 60 × 40cm 2 75% see-through retroreflective screen coupled with two pico projectors positioned close to the viewer's head, as shown in Figure 3(b). The pico projectors are based on the MEMS scanner and five laser sources that provided a 60 Hz refresh rate and 30 lumen power. The laser-sourced projectors provided a focusfree image at a variable distance. Each projector has a 1280 × 720 (720p HD) pixel resolution and a covered 42°× 24°FOV [10].
The screen consisted of square-shaped micropatterns of half-shell aluminium-coated retroreflective microspheres deposited on a clear acrylic sheet/glass using a perforated steel mask (stencil) [19]. Custom-3D-printed projector housings and mirrors were used to adjust the distance between the eye and the projector. A pair of linear polarizer films was also used in front of the eyes and projectors to block the part of the projected light diffused from the screen and to eliminate the stereo crosstalk, where each eye/projector used mutually perpendicular polarization orientations [10]. The brightness of the virtual content was > 150 cd/m 2 per eye. The previous results showed that the stereo crosstalk is higher ( > 10%) when polarizers are not used [10]. The placement of polarizers in front of the eyes can be seen as a limiting factor when considering the actual HUD systems. The light-diffusing characteristic of the screen is due to the incorrect orientation of a fraction of the microspheres (seen as dark microspheres in the Figure 2(b) inset) during the fabrication of the screen. The light-diffusing characteristic of the screen can be further reduced by refining the fabrication process of the screen and ensuring the proper orientation of all the microspheres, which can also eliminate the necessity of placing polarizers in front of the eyes.

Participants
Fifteen volunteers (10 male and 5 female) were recruited from the University Campus (students and staff). The participants were aged between 22 and 33 years. The minimum number of participants was determined as per the ITU recommendations for the subjective assessment of television screens [26]. The participants were of different nationalities, but all reported high English proficiency. Sixty five percent of the participants reported 20/20 bareeye vision while the remaining participants used their vision correction glasses during the test. The main criteria for the selection of participants was 20/20 visual acuity (with or without prescription glasses) and familiarity with driving.

Procedure
The participants signed an informed consent form before the start of the experiments. After receiving instructions, the participants were asked to stand close to the projectors, at the viewing position shown in Figure 3(b). The height and IPD of the projectors were adjusted by the researcher to match them with those of each participant.

Experiment 1: see-through condition
In this experiment, the Snellen acuity chart was placed at a 700 cm distance, and the screen was placed at a 75 cm distance from the user, as illustrated in Figure 4(a). The chart was printed in black and white and was scaled for a 700 cm calibrated viewing distance. The chart represented the text symbols of different sizes corresponding to 20/200-20/20 (3-30 cycles/degree) visual acuity.
The test for the see-through condition was divided into two parts. In the first part, the participants were directed to observe the acuity chart with a see-through retroreflective screen in between, and to interpret the text symbols on the chart corresponding to 20/200-20/20 acuity. In the second part of the experiment, the see-through retroreflective screen was removed, and the users were asked to look at the chart directly, and to interpret the text symbols. The performed task was similar to the one conducted by a general ophthalmologist for checking the visual acuity, except in the first part of the experiment, where the screen was placed between the participants and the acuity chart. Both parts of the experiment were repeated for each participant, and the responses of the participants were simultaneously recorded by the researcher. To prevent the learning effect, the recorded data were not shown to the users during the test.
HUD screens are always placed in a slanted position, which can influence both the brightness of the screen and the clear vision (transparency) through the screen. For slanted or tilted retroreflective screens, the image brightness is defined by the entrance angle. The entrance angle of the transparent retroreflective screen that was used in the test setup in the present study had been experimentally measured [19] and showed a higher acceptance angle (about ± 75°) with only 10% brightness variations. The higher acceptance angle further indicated that the brightness of the screen would be unaffected if the screen was placed in a slanted position. Moreover, the retroreflective microspheres were patterned as square-shaped patches, and screen transparency was achieved by having optically clear regions in between the microsphere's patches (as shown in the Figure 2(b) inset). Therefore, the tilting of the screen had a negligible effect on the see-through quality of the screen.

Experiment 2: simulated collimation condition
In the simulated collimation condition, the text symbols from the Snellen chart with a pre-defined acuity measure were displayed on the 3D HUD. The stereoscopic views for a line of five text symbols corresponding to specific visual acuity levels were displayed on the screen. The acuity scale from 20/200 to 20/50 (3-12 cycles/degree) was used. The polarizers were mounted together with projectors as shown in Figure 3(b), and as such, the users did not have to wear separate polarized glasses. The parallax between the views was adjusted such that the symbols appeared at a 700 cm depth distance from the viewer, as illustrated in Figure 4(b). The stereo parallax and size of the symbols at a specific distance were determined by calculating the angular size of the projected pixel. The subjects were instructed to keep both their eyes open during the test, to converge and focus their eyes on the distant real scene (at 700 cm), and to interpret the text symbols from the projected text. The responses of the participants were simultaneously recorded by the researcher.

Visual acuity response to the real scene through the transparent screen
The visual acuity response when looking through the screen was determined from the see-through condition. The response was evaluated by calculating the number of text symbols correctly resolved by each participant when reading the acuity chart through a transparent retroreflective screen, and were compared to the acuity response when a screen was not present. Figure 5 shows the average percentage of text symbols read/resolved by the participants on the scale of visual acuity/spatial frequency. As all the text symbols larger than 20/50 were interpreted with 100% accuracy (with and without having a see-through screen in between), the results were plotted from the acuity scale of 20/50 to 20/20 corresponding to the spatial frequency of 12-30 cycles/degree, respectively.
An average human eye can resolve the spatial frequency of around 30 cycles/degree, which corresponds to 20/20 visual acuity. As all the participants reported normal visual acuity (with/without prescription glasses) before the experiment, the character size corresponding to 20/20 visual acuity were recognized with an average accuracy rate of 90% when a screen was not placed in between the user and the acuity chart. The results showed a drop in visual acuity when the Snellen chart was viewed through the screen, where the average percentage of correct responses was reduced to 40% for the 20/20 acuity symbols. Similarly, the average response of the larger text symbols was also slightly lower than that of the direct view. The decrease in acuity response was mainly due to the existence of retroreflective micropatterns on the screen (as shown in the Figure 2(b) inset) and the reduced amount of light transmitted through the screen. The placement of repeated micropatterns affects the point spread function of the transmitted light, which results in loss of acuity for the small symbols. On the other hand, a 25% drop in the amount of transmitted light reduces the overall contrast of the scene on its other side, which also plays a role in acuity response reduction. Figure 5 also shows that a considerably better acuity response ( > 75%) is achieved for the visual acuity of < 20/25 ( < 24 cycles/degree). As the visual acuity standard for driving in USA and UK is 20/40 [27,28], the measured acuity response suggests that the transparentretroreflective-screen-based 3D HUD can be used as an automotive display for a comfortable driving experience, provided that the normal visual acuity of the driver/user is 20/20 before the placement of the HUD screen.  The result of the placement of the screen in front of the viewer can also be observed from the transmission modulation transfer function (MTF) shown in Figure 5(b). The transmission MTF was measured using the slantededge technique, where a slanted edge was displayed on an LCD monitor placed on the other side of the seethrough retroreflective screen, and was captured through a camera [29]. The result showed a low MTF value for the see-through screen compared to the direct view. The drop in transmission MTF particularly for the higher spatial frequencies follows the decreased acuity response results observed in the see-through condition test.

Visual acuity response of the content displayed on the screen
The visual acuity response of the content displayed on the screen using the simulated collimation technique was determined by examining the results of the simulated collimation condition. The visual acuity was determined by calculating the number of displayed text symbols correctly resolved by the participants for different acuity scales (20/200 to 20/100) at each screen distance (50, 75, 100, and 125 cm). Figure 6(a) shows the average percentage of correctly recognized text symbols for the different visual acuity scales. The result shows poor visual acuity when the distance between the screen and the viewer was shorter (50 cm). The poor acuity particularly for the small text symbols was due to the amount of blur in the perceived content. In a typical non-transparent stereo 3D display, such higher AC conflict develops fatigue and doubleimage perception. In the case of the see-through screen, however, the user perceives the real scene together with virtual stereo imagery. Therefore, the user's eyes are converged at a depth distance (700cm), and a single image is perceived even when the AC conflict is higher.
Further, the figure shows that the large text symbols (20/200 to 20/100) were resolved correctly for all the screen distances, except for the 50 cm screen distance. A significant improvement in acuity was observed when the user-to-screen distance was increased. This acuity improvement was mainly due to the AC conflict reduction as the screen distance increased. Figure 6(b) shows the accurate response rate for the 20/50-and 20/70-sized text symbols as a function of the AC conflict in diopters, calculated using equation (1). As the convergence distance of the projected stereo images was always 700 cm, the change in the amount of AC conflict was thus due only to the screen distance (accommodation distance). The result clearly shows the inverse relation between the visual acuity and the amount of AC conflict, where a higher AC conflict (closely positioned screen) results in poor visual acuity while a lower AC conflict (distant screen) provides a better acuity response. Due to the dynamic behavior of the human eye, it requires several hundred milliseconds to refocus. In this study, however, the viewer's eyes were always focused and converged on the real scene (roadside view) at a farther distance from the HUD screen. Therefore, the viewer/driver was not required to refocus at the screen distance. The purpose of changing the screen distance was only to change the level of AC conflict.

Difficulty level and discomfort
In the post-test questionnaire, the participants were asked to report the relative difficulty for each screen distance under the simulated collimation condition. All the participants reported the close-distance screen (50 cm) as the most difficult, while the 125 cm distance was reported as the least difficult. Collimation was simulated using stereoscopy, which may cause discomfort and fatigue, such as eye strain, nausea, and headache. The participants were asked to report their experiences of such conditions during the test experiments. About 40% of the participants reported eye strain when they used the screen at a 50 cm distance, while no discomfort or other effects were reported for the remaining parts of the tests.

Key findings of the study
The first experiment of the user tests showed that the placement of a see-through retroreflective screen as a 3D HUD reduces the visual acuity of the user/driver, where the 75% transparent screen reduces the visual acuity from 20/20 to 20/25. Considering the international eyesight rules for driving, however, a transparent screen can be installed as a windshield 3D HUD for drivers with normal visual acuity (20/20).
The second part of the experiment showed an inverse relation between AC conflict/screen distance and acuity response, which suggests that the > 100 cm screen distance ( < 0.85 dioptre AC conflict) is the optimal value for the HUD applications. The results showed poor acuity at a close-screen distance (higher AC conflict). It was also observed, however, that the amount of AC conflict or the screen distance has a minimal effect when large objects are displayed on the HUD screen, which also provides guidelines for the optimal size of objects while designing the virtual interfaces for automotive display applications. Although the motivation for the design and use of the typical HUD screens is to provide both accommodation and convergence at a farther distance, the simulated collimation technique used in this study does not solve the accommodation problem but provides correct convergence of the displayed contents at a farther depth. Therefore, this study contributed to the understanding of the visual acuity loss in the stereoscopic HUD scenarios where convergence coincides with the real-world view across the windshield screen but where there is an AC conflict.

Conclusion
The visual acuity response of a three-dimensional (3D) head-up display (HUD) based on the retroreflective seethrough screen was evaluated. The stereo parallax was used to simulate the optical collimation so as to display virtual imagery at a farther depth (700 cm). Two different user tests were conducted to evaluate the effect of the reduced light transmission (compared to a clear windshield glass) of the retroreflective screen as well as stereoscopic simulated optical collimation on the visual acuity response. Additionally, the relationship of the AC conflict and screen distance with the acuity response was examined in the simulated collimation mode. A 75% transparent screen developed in-house was used, coupled with two pico projectors, as the test setup. The results showed a minor reduction in visual acuity (20/20 to 20/25), when the clear vision of the user was covered with a retroreflective HUD screen. Under the simulated collimation condition, the results showed an opposite relation between the acuity response and the amount of AC conflict (user-to-screen distance), where the average accuracy rate for recognizing 20/50-(12 cycles/degree)sized text symbols decreased from 86% to 16% when the screen distance changed from 125 cm (0.65 D) to 50 cm (1.85 D). While the see-through performance of HUD is dependent on the type of screen and its transmission characteristics, the presented test results for the simulated collimation (stereo collimation) condition can be extended to other 3D HUDs and wearable AR glasses, where the virtual image is displayed at a fixed distance, and the depth cue is provided by adjusting the stereo parallax. The participants in the current study, however, used the display only briefly (15-20 min), and the study can be extended to assess the visual performance when the 3D HUD is tested for longer durations.