Optical see-through head-mounted display including transmittance-variable display for high visibility

This paper presents an optical see-through head-mounted display (OST-HMD) that can provide a highly immersive augmented reality (AR) environment. Because visibility of objects is degraded by external light and because projection optics cannot represent black color, the reality of AR content is reduced. To solve this problem, we adopted an additional transparent display to adjust the transmittance of exterior light. This adjustment is achieved using a guest–host liquid crystal (GHLC) to provide transparency in the initial state and opacity when voltage is applied. To verify the display’s usefulness, we measured ambient contrast ratio (ACR) and color gamut under various ambient light conditions after attaching the GHLC panel to the existing OST-HMD (Microsoft HoloLens). Under typical office illumination, use of the GHLC panel increased the ACR by 4.67 times, and the color gamut by 2.87 times compared to the OST-HMD without it. Under high illumination the panel increased the ACR by 1.54 times and the color gamut by 16.16 times. The proposed method significantly improved the ambient contrast ratio and color expression, and yielded a flexible way to customize existing OST-HMDs by adding a detachable display.


Introduction
Optical see-through head-mounted displays (OST-HMDs) allow overlay of computer-generated images on reality, provide real-time interactivity, and enable 3D registration [1,2]. An OST-HMD should show virtual information and real-world scene concurrently, and consequently requires an optical combiner to merge them. Several OST-HMDs have been commercialized [3], but the development of an OST-HMD with a wide field of view (FoV) and good image quality has been a considerably challenging area for decades, and the process of designing an OST-HMD system requires an interdisciplinary approach. Several papers described the goal of consumer OST-HMDs as supplying a cost-effective solution for multiple optical design challenges: resolution, transparency, eye box, and FoV [4,5]. Rendering of mutual occlusion relationships between digital and physical objects in 3D space is one of the main problems in the development of OST-HMDs [6]. If a real object is fully opaque, it should occlude a virtual object behind it, and a virtual object should do the same for a real object. When the locations of the real and virtual objects are known, the occlusion of a virtual object by a real object can be achieved simply by not rendering the occluded parts of the virtual object. However, occlusion of a real object by a virtual one is difficult because it requires blocking of light in the real scene. Consequently, a black virtual object can seem transparent [7][8][9][10][11]. State-of-theart OST-HMDs typically achieve occlusion by using a beam splitter to uniformly blend the light from the real scene with the virtual objects. This method cannot selectively prevent light of the real world from reaching the eye. Therefore, digitally rendered virtual objects viewed through OST-HMDs typically appear 'ghost-like.' This artifact is related to one of the most significant problems of the OST-HMD systems that use projection optics; i.e. that the environment influences the visibility of the virtual objects. In existing OST-HMDs, strong external light (e.g. sun-light) acts as optical noise on the image of a virtual object. This results in degraded color correctness and reduced ambient contrast ratio of the image, so the visibility of virtual objects is reduced. This reduction can cause perceptual errors while information is being obtained from a virtual object, so viewers may have a difficulty understanding or identifying the image [12][13][14][15]. The color fidelity of the displays is vulnerable, especially in bright environments, and this matter is more critical with an OST-HMD that uses optical projection than one that uses video see-through technology [16]. Specifically, degradation of the image quality increases as the strength of the external light increases. External illumination can degrade the effective resolution, the ambient contrast ratio, and the color perception [17,18]. Over the past few years, several papers have reported methods to counter the effects of external illumination, such as global dimming [19,20], local dimming [21] and occlusion-capable LCD layer [22][23][24][25][26][27][28][29]. These approaches have their own advantages and disadvantages. Dimming is a method of installing a neutral density filter or display in front of the OST-HMD to uniformly reduce the brightness of the external light. This method can reduce the ghost-like effect of virtual content, particularly in bright environments. If the unit on the front is adjusted globally, it is called global dimming, and if the area is selectively adjusted, it is called local dimming. This approach may be useful in some scenarios especially under very bright illumination, but the additional filter makes the device excessively bulky and inconvenient to use. Occlusioncapable layer is a method of placing an additional spatial light modulator (SLM) optical path between the optical combiner. The user's eye can be used to selectively block light from the real world scene. However, this method creates a blurred image since the occlusion SLM is out of focus with respect to the virtual image. This approach also requires a bulky optical system. For this reason, these methods have been suggested for the conceptual verification by installing optical components on the breadboard, making it difficult to find examples that have been applied to commercialized OST-HMDs. This paper describes an OST-HMD with a transmittance-variable display with two states. (i.e. 'on' and 'off'). This display can block penetration of external light into the OST-HMD and thereby minimize the problem of the virtual object seeming transparent. This attachment increases image quality and sense of immersion, and consequently boosts the ambient contrast ratio (ACR) and the color gamut as well.

Theoretical background
Most commercialized OST-HMDs offer a contrast ratio that is appropriate for use in a dark room. However, external light greatly degrades black virtual images in OST-HMDs, so the ACR must be evaluated in a practical environment. ACR is the contrast ratio between the luminance values of the black L K and white L W states where the environmental influences are considered: where L B is the background luminance of the external environment and L R is the luminance due to reflection or glare in the optical system [30][31][32][33]. L B has the largest influence on ACR. ACR decreases rapidly as the intensity of the external light increases ( Figure 1).

Method
The operating mode should be determined first when a secondary display is used. In the proposed method, the initial state must be transparent to keep the external image as clear as possible, whereas a sufficiently low transmittance should be available under the highest applied voltage to minimize the influence of external light on the virtual object. For this purpose, we chose a guest-host liquid crystal (GHLC) since it meets these requirements.
A GHLC cell consists of host liquid crystals (LCs) and guest dichroic dyes, which are aligned parallel to the LCs (Figure 2(a)). Due to their dichroism, dye molecules absorb incident that is polarized parallel to the absorption axis. They transmit the light that is polarized perpendicular to the absorption axis. The transmittance of a GHLC cell depends on the alignment of the dye molecules, the absorption coefficient of the dye, the cell gap (i.e. the thickness of the GHLC layer) and the dye concentration. By increasing the cell gap or dye concentration, transmittance in the opaque state can be reduced, but transmittance in the transparent state also decreases.
We fabricated a GHLC cell in which the LC is a mixture of a negative LC ( n = 0.039, ε = −3.7) and a chiral dopant (S-811, Merck) with a pitch 0.75 um. We added 2wt% of black dichroic dye (X12, BASF) to the LC mixture to increase light absorption. A vertical alignment layer was coated on each substrate to vertically anchor the LCs. After the baking process, the vertical alignment material that was spin-coated on each substrate was rubbed with cotton. The rubbing direction of substrates was set to be orthogonal, while the cell gap was maintained at 6.5 um using silica spacers. The GHLC cell has two LC states (Figure 2(a)), i.e. a vertically aligned mode and an inverse super-twisted nematic (inverse-STN) mode [34]. The LC cell can be switched between the transparent and haze-free opaque states by applying an electric field (Figure 2(b)). In the initial transparent state, the LC and dye molecules are aligned perpendicular to the substrates so that most of the incident light passes through the LC cell without being scattered or absorbed. The transmission spectra of the fabricated LC cell measured using a spectrometer (MCPD 3000, Photal) in the range of 400 − 700nm (Figure 2(c)) have average transmittances of 60.0% and 27.2% in the transparent and opaque states, respectively. The measured absorption coefficients were 0.01856 μm −1 and 0.20176 μm −1 in the transparent and opaque states, respectively. The background objects can be clearly seen (Figure 2(d)). When the applied voltage is increased, the vertically aligned negative LCs and dye molecules tend to align parallel to the substrates and form a twisted structure by the chirality due to the chiral material and alignment conditions. The dye molecules can absorb the incident light without light scattering in this state, regardless of its polarization direction. At an applied voltage of 4V, the transmittance was reduced to 25.7%, and correspondingly, the photograph showed a dark image (Figure 2(e)).

System configuration
To implement the proposed method, we fabricated a GHLC cell with an active area of 15 × 20 mm. We used a DC power supply (Keysight U8031A power supply) to activate the cell, and attached it in front of a Microsoft HoloLens. The light visor of the HoloLens was removed to measure the effectiveness of the proposed method. The display quality of the OST-HMD was evaluated using a spectrometer (GL Spectis 1.0 Touch, GL Opti Probe 5.0 luminance) with an optical probe. Optical properties such as CIE 1931 chart, color spectrum, luminance, and color coordinates were extracted using a GL Spectrosoft. Numerical calculations and graph representation were performed using MATLAB. We constructed an experimental environment (Figure 3) to test the method. We attached the GHLC cell in front of the display area of the OST-HMD.

Experimental results
To verify the effectiveness of the proposed method, the optical characteristics were measured under three illumination conditions: dark room (0 lux), office environment (600 lux) and bright environment similar to outdoors (6000 lux). The ambient illuminance was controlled using an LED flood light projector (D65, ST-TK-LED-150W, 250 lm), while the illuminance fluctuation was kept within ±1%. We prepared the intensity spectra and CIE 1931 based color charts for each experimental environment ( Figure 4).
Red (R, G, B = 255, 0, 0), green (0, 255, 0) and blue (0, 0, 255) colors were displayed in the OST-HMD, and luminance values were obtained using the spectrometer. The optical characteristics for the seven measurement conditions were subsequently evaluated and compared (Table 1), and the color characteristics, spectrum graphs and photographs of the background images were obtained (Figure 4). Without the GHLC cell, ACR was 338.6 in a dark room, but only 6.62 in the office environment, which is a 98% reduction. When the GHLC cell was used, the luminance values of white and black images decreased in a similar ratio in the 'off' state of the cell (i.e. transparent state under application of 0 V). The ACR was 7.56, which is close to that of the result without the GHLC cell. In the 'on' state (i.e. opaque state under application of 4 V) of the GHLC cell, the luminance of the black image was significantly reduced and the ACR  became 31.07. This is 4.69 times higher than the corresponding value without the GHLC cell. In the outdoor environment, the ACR was 1.5 without the GHLC cell, and it increased 1.54 times when the cell was in the 'on' state. Even though the luminance of the bright state was decreased ∼ 51% due to the finite transmittance of the GHLC cell, and that of the black state was reduced by ∼ 68%, so the ACR was improved. The low transmittance of the OST-HMD with the GHLC cell under bright illumination may even be desirable because the cell can act as a neutral-density filter against bright luminance. The transmittance of the 'on' state GHLC cell can be reduced by modifying the dichroic dye material and optimizing the doping concentration. These changes will further increase the ACR. The GHLC cell had different influences on the color gamut under various illumination conditions. The color coordinates obtained using CIE 1931 of the red (255, 0, 0), green (0, 255, 0) and blue (0, 0, 255) images were collected under the seven conditions mentioned above. The color gamut values were then derived  using NTSC 72% (Table 1). The measurement results in a dark room environment confirmed that the optical characteristics of OST-HMD used in the experiment had high intensity of red and green, and that blue had lower intensity and slightly lower expression than other colors ( Figure 5(a)). Without the GHLC, the color gamut was 76.16% in the dark room, and decreased 74% to 19.45% in the office condition. In the office and outdoor environments, the overall area of color gamut decreased in all states. This is because 4100 K of lighting was used in the office environment of the external light constituting the experimental environment, while 6500 K of lighting was used in the outdoor environment, so the image from the OST-HMD was additive-mixed with white light (Figure 5(b, c)). When the GHLC cell was used, the color gamut was 18.27% with the secondary display in the 'off' state, and this is similar to the case without the secondary display cell. Then with the GHLC cell in the 'on' state, the gamut improved 2.87 times to 55.91%.
The increase was mainly due to the improvement in black quality with the 'on' state GHLC cell. In the office environment, color gamut was improved in the visible region. It was confirmed that many improvements were made, especially in blue, which was not well expressed due to external light, and each wavelength had a similar color expression. In addition, it was confirmed that the reason why red improvement occurs less than other colors in the office environment is that the GHLC transmittance in the opaque state increases at the corresponding wavelength.
Under the outdoor condition, the OST-HMD without the GHLC cell had a color gamut of 0.54%, but this rose to 8.73% when the GHLC was used, which is 16.16 times better. In the outdoor environment, white light generated by strong external light is dominant, so color gamut improved and color expression is almost impossible in without state and 'off' state. In the 'on' state however, color can be expressed in all visible regions.
These results indicate that the additional GHLC cell could improve color characteristics of the OST-HMDs in the office and outdoor illumination conditions, compared to the bare OST-HMDs without the GHLC cell. In particular, this method has been confirmed improved color representation in an outdoor environment where with existing OST-HMDs, it is hard to recognize colors due to the severe decrease in color expression.

Conclusion
We proposed a method to improve the display quality of an OST-HMD by using an additional transmittanceadjustable display, i.e. a GHLC cell, on its front side. This cell can reduce the degradation of ACR and of color accuracy by external light. To verify the improvement experimentally, the GHLC cell was attached to the existing OST-HMD product. Measurements were subsequently performed under seven evaluation conditions. The GHLC cell substantially increased the ACR and color gamut in the various illumination environments. This method can be applied to existing OST-HMDs by adding the GHLC cell in a detachable form. Our future research will focus on improving the GHLC cell and developing an additional auto-brightness control (ABC) to optimize the ACR and color gamut under an arbitrary illuminance condition. We will also improve the ACR and color gamut by using a pixel-driven GHLC cell as well as an accommodation-dependent eye model.

Disclosure statement
No potential conflict of interest was reported by the author(s).

Funding
This research project was supported by the Sports Promotion Fund of the Seoul Olympic Sports Promotion Foundation from the Ministry of Culture, Sports and Tourism (The revolutionary athlete data integration technology for national football performance innovation). This work was also supported by the BK21 FOUR project through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Korea. He was a research engineer at LG Display for 26 years, and is currently an associate professor at the Pohang University of Science and Technology (POSTECH) in Pohang, Republic of Korea. His research interests cover traditional displays, near-eye displays for AR and VR, and future displays. He also has research activities related to wave array technologies such as LC-based antennas, meta-surface applications, and ultrasound arrays for various applications.