Seamless holographic image generation for a multi-vision system

This paper investigates the optical seamlessness of a holographic multi-vision display. The purpose of the multi-vision approach is to scale up the resolution of a holographic display without requiring a single extremely high-resolution spatial light modulator. Optical experiments reveal that a holographic multi-vision system with 16M pixel resolution successfully generates a seamless virtual reference plane with the essential optical characteristics of a holographic display, such as the accommodation effect and binocular parallax. It is realized from a matrix of eight (4 2) high-resolution transmission-type liquid crystal displays. Theoretical analysis and numerical simulations are presented for the proposed seamless holographic multi-vision system.


Introduction
Holography is generally considered as an ultimate threedimensional (3D) display technology because holographic displays can provide essential 3D cues such as accommodation-vergence matching and continuous motion parallax [1]. However, many challenges still remain in the commercial development of holographic displays, including the limited capability of spatial light modulators (SLMs), which are necessary for modulating the optical wave field. Most SLMs have insufficient spacebandwidth product (SBP) [2] in terms of information required for the application of holographic 3D displays. A more stringent approach to improving the SBP is to fabricate an extremely large-scale high-resolution SLM.
Despite this practical limitation, several approaches to improving SBP have been proposed in recent decades. One of these approaches is a holographic display system that makes use of multiple SLMs. For example, Fukaya et al. demonstrated a holographic display that eliminates vertical parallax and extends the horizontal viewing angle by tiling a number of SLMs in the horizontal direction [3,4]. Hahn et al. also proposed a holographic display system with a wide viewing angle using a curved array of multiple SLMs [5], in which three sub-SLMs were diagonally positioned in contact with each other and their viewing field was continuously extended without interruption. Kozacki et al. proposed a holographic display with an extended viewing angle achieved by arranging multiple tilted SLMs in a circular configuration [6]. In the curved array configuration of the SLMs, a curved lens array was used to generate a dynamic holographic stereogram with a wide viewing angle. Meanwhile, Sakai et al. reported a novel method for realizing full parallax video holography by using 16 (4×4) reflection-type SLMs in a planar configuration [7,8]. Here, tiling multiple reflection-type SLMs limits their number due to the multiple bulky beam splitters required to illuminate and combine the SLMs. On the other hand, the multivision system is commonly used in the display industry to realize large-sized display [9]. Our multi-vision system is composed of multiple transmission-type liquid crystal displays (LCDs). Transmission-type SLMs are more suitable for scaling up holographic displays because complex optics are not required between the multi-vision system and the observer.
A common problem found in holographic multivision systems is the annoying seams between adjacent SLMs. In a multi-vision configuration, the bezel of each SLM represents an obstacle in producing a continuous image. Several strategies have been proposed in order to remove the seams. For example, directly applying cylindrical lens pairs to the seams can conceal the seam region and generate a continuous holographic image [10]. However, using a refractive optical element, such as a cylindrical lens, to remove the seams causes image distortion, and it is only useful for specific viewing angles. Other approaches include combining a digital holographic display consisting of multiple SLMs with a large-area directional diffractive element composed of pixelated gratings [11] and effectively using multiple SLMs to increase the size of the reconstructed image [12].
In this paper, we propose a simple method for eliminating the annoying seams and generating seamless holographic images on the virtual reference plane of a transmission-type holographic multi-vision system. Holographic technique is expected to reduce the visibility of the seam between SLMs; however, to our knowledge, there have been no studies to clarify the requirements for creating the seamless multi-vision system. We not only define a minimum distance of a seamless virtual reference plane by using the uniformity of intensity contrast of the seam in the retina of the eye, but we also mathematically describe the relationship between the minimum distance of the seamless virtual reference plane and the viewing distance. We present a 16M pixel resolution holographic multi-vision system consisting of a matrix of eight (4×2) high-resolution transmission-type LCDs. A seamless computer-generated hologram (CGH) image is synthesized on the virtual plane between the SLM plane and the viewer. Section 2 describes the proposed transmission-type binocular holographic multivision system and CGH synthesis algorithm for the generation of binocular full color holographic images. Section 3 outlines an experiment for the analysis and verification of the generation of seamless holographic 3D images with a natural accommodation effect. Section 4 presents the conclusion and future directions.

Transmission-type binocular holographic multi-vision system
For the practical implementation of the proposed binocular holographic multi-vision system, a tiling-type multivision approach that spatially arranges several commercial LCDs within the same plane is used in order to increase the SBP. In our experimental setup, the multivision system is composed of a 4×2 matrix, and eight driving boards that control each LCD are mounted outside the active area.
This system employs a 5-inch transmission-type commercial LCD with full color FHD resolution and opaque bezels including the gate driving circuit and metal electrodes ( Figure 1[a]). These bezels create rectangular seam lines inside the multi-vision configuration ( Figure 1[b]), thus leading to discontinuous images and lower image quality of the resulting multi-vision display. It is difficult to completely remove the seam in the display image of a multi-vision system. The experimental setup is presented in Figure 1(c). We use an RGB light-emitting diode (LED) system to illuminate the multi-vision system. The system parameters are shown in Table 1. The central wavelengths of the RGB LED system are 640, 525, and 450 nm, respectively. The multi-vision system consists of eight SLMs, has a diagonal size of 13.13 in., and has a total of approximately 16M pixels. The size of the viewing window generated by this system is 28×28mm 2 . Figures 1(d), (e), and (f) present the 4×2 LCD multivision section, the RGB LED light sources, and the optical components of the system (i.e. the parabolic mirror and beam splitter). The parabolic mirror used as a field lens has a focal length of 1.8 m, and the viewing distance in the multi-vision system is 3.6 m, which is twice the focal length [13]. A dichroic prism is used to combine the RGB beams.
A CGH synthesis algorithm is developed for the binocular full color holographic multi-vision system [14], and the reconstruction characteristics are analyzed. The CGH multiplexes left and right holographic views in order to produce a binocular parallax. Each view provides an accommodation effect for a single eye. Figure 2 displays a schematic diagram of the forward and backward cascaded Fresnel transform model considering both eyes of the observer in a holographic multivision system. The CGH wave field, G Bi (ξ 1 , η 1 ), on the SLM plane propagates to the retina plane, obtaining the wave fields on the retina plane of the left eye, F L (ξ 2 , η 2 ), and right eye, F R (ξ 2 , η 2 ). The binocular CGH pattern, G Bi (ξ 1 , η 1 ), is a multiplexed hologram of the left 3D perspective wave field, G L (ξ 1 , η 1 ), and right 3D perspective wave field, G R (ξ 1 , η 1 ). The mathematical relationship between a wave field on the SLM plane, G L(R) (ξ 1 , η 1 ) and a wave field on the retina plane, F L(R) (ξ 2 , η 2 ), is described by the forward cascaded and inverse cascaded Fresnel transforms [15,16] (Appendix).
We focus on the compact multiplexing of G L (ξ 1 , η 1 ) and G R (ξ 1 , η 1 ) into G Bi (ξ 1 , η 1 ). The left and right stereoscopic holographic images are multiplexed in a square viewing window by using the pupil space division method [17]. The binocular CGH pattern on the SLM plane is expressed as Here, α L(R),x(y) denotes carrier frequency along the x(y) axis for left (right) view. For the synthesis of G Bi (ξ 1 , η 1 ), the carrier wave field patterns, exp[j2π(α L,x ξ 1 + α L,y η 1 )] and exp[j2π(α R,x ξ 1 + α R,y η 1 )], are multiplied to G L (ξ 1 , η 1 ) and G R (ξ 1 , η 1 ) in order to synthesize the left and right images within the square viewing window. The size of the viewing window in the pupil plane, W x × W y , can be obtained as where λ is the wavelength of the light, d 0 is the observation distance from the SLM, and p x and p y denote the pixel pitch of the LCD in the horizontal and vertical directions, respectively. The vector components of the appropriate carrier wave field patterns are  given by In an ideal environment, the viewing window should be at least larger than the size of the human pupil in order to display a holographic image without losing optical information. Since we are using an amplitude-only SLM, the size of the viewing window should be at least twice the size of the pupil in order to exclude the inevitable conjugate area. In addition, the left and right directional RGB LED sources generate two independent viewing windows. The inter-pupil distance, PD, and the horizontal length of viewing window are used to determine the interval between the left and right light sources, . Since the distance from the SLM plane to the light sources and the distance from the SLM plane to the pupil plane are equal to d 0 , the interval between the right and left light sources, , is given by The inter-pupil crosstalk (IPCT) effect should also be considered in the design of the binocular holographic multi-vision system. The IPCT effect is caused by the higher-order diffraction of the pixel structure. Higherorder diffraction patterns are heavily influenced by the shape of the SLM pixel. Square pixels generate a rightangle cross-shaped higher-order diffraction distribution. This distribution for the left view has a strong possibility of spreading across the pupil plane and interfering with the right view ( Figure 3[a]). In contrast, pixels with an asymmetric parallelogram shape generate a scissors-like diffraction pattern in the pupil plane ( Figure 3[b]). This special diffraction pattern deviates the right eye from the high-energy region, thus minimizing IPCT. Figure 3(c) displays the diffraction pattern distribution including the higher-order terms for the left view in the pupil plane. The two circles in Figure 3(c) represent the left and right eyes with a pupil diameter of 5.5 mm, an inter-pupil distance of 65 mm, and a distance between the pupil and the SLM of 3.6 m. The intensity distribution of the diffraction pattern along the x-axis is compared in Figure 3(d).
The numerical simulations in Figures 3(c) and (d) show that the asymmetric pixel architecture differentiates the bright and dark regions, thus allowing the finite right eye pupil to reject the higher-order diffraction crosstalk and reducing the IPCT effect. The same results are observed for the right view. An LCD with an asymmetric parallelogram shape pixel (Figure 3[b]) was used to set up the experimental testbed for IPCT-free binocular full color holographic multi-vision system, as presented in Figure 1.
The numerical and optical experiment results for the binocular full color holographic image produced by the multi-vision configuration are shown in Figure 4. A

Seamless holographic image generation
It is possible to observe a seamless holographic image in our proposed multi-vision system because the holographic image is reconstructed on the object plane via the CGH on the SLM plane, where the seams are physically present. In our multi-vision configuration, the wave field pattern, G Bi (ξ 1 , η 1 ), on the SLM plane is represented by a complex field with the 4×2 seam mask pattern, M seam (ξ 1 , η 1 ) as follows: and Here, T x , T y indicate the horizontal and vertical size of the active area of the SLM, g denotes the width of the seam, and rect(·) is the rectangular function. The main screen is separated from the physical SLM array, and the defocusing effect of the SLM can be used to specify a virtual plane for representing seamless holographic 3D images. The numerical observation results for a reconstructed holographic image on the object plane for various propagation distances in the proposed multi-vision configuration are shown in Figure 5. Furthermore, the CGH patterns generate an in-focus image of the letter A on each virtual screen, while dark seam lines are apparent in the numerically reconstructed holographic image. However, the contrast of the seam lines becomes blurred with the distance between the SLM plane and the object plane, d obj . When the propagation distance is relatively short, as shown in Figures 5(a) and (b), the seam in the numerically reconstructed holographic image is apparent, and it is reduced when the propagation distance is 1.5 m, as shown in Figure 5(c). Finally, the holographic image 2.5 m away from the SLM plane is completely seamless in Figure 5(d).
The horizontal and vertical intersections across the seam line patterns are plotted in Figures 6(a) and (b) with respect to d obj . The relative luminance distribution of the reconstructed holographic image when the distance to the object plane is 0, 0.5, 1.5, and 2.5 m is shown in Figures 6(a) and (b). The defocusing effect caused by long distance wave propagation blurs the seam lines on the virtual screen at d obj . For quantitative analysis, we define Uniformity, which measures the intensity contrast of the seam, as.
where I max and I min are the maximum and the minimum intensities of the holographic image at the retina plane. Uniformity 50 refers to the distance of the holographic image from the SLM plane, d obj , at which the value of Uniformity is 0.5. At Uniformity 50 , the eye perceives the seam region of the reconstructed holographic image as starting to blur. Uniformity 20 refers to the distance of holographic image from the SLM plane, d obj , at which the value of Uniformity is 0.2. At Uniformity 20 , the eye does not perceive the seam of the reconstructed holographic image. Uniformity decreases drastically as d obj increases from 1 m to 3 m, as shown in Figure 6(c). When d obj is about 1.5 m, the value of Uniformity is 0.5, and when d obj is about 2.4 m, the value of Uniformity is 0.2. The seam of the holographic image is noticeable when d obj is less than 1.5 m. As d obj increases past 1.5 m, the seam area gradually vanishes. As d obj is more than about 2.4 m, the seam is not perceived because of Uniformity 20 . The numerical results in Figure 5(c) and (d) indicate that the definitions for Uniformity 50 and Uniformity 20 are reasonable. Therefore, in the proposed multi-vision system the seamless virtual reference plane has a minimum distance of 2.4 m and a seamless holographic 3D image is successfully generated beyond that distance.
The threshold of a seamless virtual reference plane is calculated by using the Uniformity of intensity contrast on the retina of an eye. If the wave field generated by the SLM reconstructs a point on the object plane and the eye focuses on that point, then the entire wave field by the SLM does not enter the eye, as shown in Figure 7(a). The pixel region that reconstructs the wave field entering the eye is called the circle of contribution and its diameter is given by Here, ρ is the radius of the pupil. In addition, the intensity of light diffracted by the pixels within the circle of contribution follows a distribution in the form of a sinc function, denoted by sin(π x) πx . The circle of contribution has a circular shape, and a circular mask function to limit the region occupied by the circle of contribution is given by So, the maximum and minimum intensities at the retina are respectively given by where I 0 denotes the central value of the intensity profile diffracted from one pixel. The distance of the object point, d obj , that satisfies Uniformity = 0.2 in Equation (7) is called the minimum distance of seamless virtual reference plane, d min . Since the viewing distance is extremely large compared to the size of the SLM, paraxial approximation is applicable. The minimum distance of seamless virtual reference plane is computed by where Figure 7(b) shows the relationship between the viewing distance, d 0 , and the minimum distance of seamless virtual plane, d min . In other words, if the viewing distance decreases by using a field lens with a small focal length, the minimum distance of seamless virtual plane also decreases. The required pixel pitch is obtained from Equation (2) because the viewing window size should remain the same even as the viewing distance decreases. A viewing window size of 28 mm is specified in our system. For a given pixel pitch of 57 μm, the viewing distance is 3.6 m and the minimum distance of the virtual reference plane is 2.4 m. If the pixel pitch is reduced to 34 μm, the viewing distance and the minimum distance of the virtual reference plane become 2.2 and 1.45 m, respectively.
To synthesize a full color CGH, a perfectly matched full color holographic field is initially set up on the retinal plane, and the optical field in the retinal plane is transmitted using the ICdFr algorithm to the SLM plane for the object space [15,18]. A depth-map CGH for each color is calculated independently by using ICdFr at the corresponding wavelength. The calculated light distribution fields in object space accumulate in the SLM plane to generate a full color composite hologram. Since no optical filter is used to remove unwanted noise in a direct view configuration, the CGH signal is placed off-axis in order to avoid DC noise. However, for full color CGHs, the signals for each color are distributed in different positions within the viewing zone according to the R/G/B wavelengths. DC noise is present on the optical axis in the viewing zone, while the hologram signal for the red wavelength is farthest from the optical axis and that for the blue wavelength is closest to the optical axis, thereby resulting in axial misalignment. In order to resolve this misalignment, it is necessary to independently calculate the R/G/B carrier waves to adjust the signal for each color in the same viewing zone.
The experimental results for the proposed CGH synthesis algorithm for the generation of seamless full color holographic images are presented in Figure 8. The 3D objects are designed with full color letters of different depths. A, B, and C are located 2.2, 1.3, and 0.4 m in front of the SLM plane, respectively (Figure 8[a]). The experimental results in Figures 8(b), (c), and (d) demonstrate the holographic accommodation effect. When the observer focuses on C at a distance of 0.4 m from the SLM plane (Figure 8[b]), A and B are defocused. When the observer focuses on B at a distance of 1.3 m from the SLM plane (Figure 8[c]), A and C are defocused. Similarly, when the observer focuses on A at a distance of 2.2 m from the SLM plane ( Figure 8[d]), B and C are blurred. When the observer looks at C, the seams are clearly visible, but when the observer looks at B, the seams are less apparent. In addition, the seams are barely noticeable when they look at A. In this experiment, when the distance between the object plane and the SLM plane is 2.2 m, the seam of the holographic image is barely discernible, and the measured value of uniformity is 0.23. Therefore, the experimental results show that the numerical analysis indicating that the minimum distance of the virtual reference plane is 2.4 m is valid. It is expected that the experimental result is slightly smaller than the numerical analysis result because the diffraction angle is wider based on the fill factor of the pixel. Furthermore, the diffracted light distribution is not symmetrical due to the asymmetric shape of the pixel.

Conclusion
In conclusion, we propose a holographic multi-vision display with a 16M pixel resolution made of eight (4×2) high-resolution transmission-type LCDs and investigate the seamlessness of the proposed architecture. A pixel structure in an asymmetric parallelogram generates a special diffraction pattern in the pupil plane. This pattern minimizes IPCT caused by a conventional symmetric pixel shape. The change of seam according to the distance of the object plane of the holographic image is theoretically analyzed, and a seamless virtual reference plane is defined in the multi-vision system. Optical experiments show that the proposed system successfully creates a full color seamless holographic 3D image on the virtual reference display plane and achieves the essential optical characteristics for a holographic display, including the accommodation effect and binocular parallax. The findings are consistent with the supportive numerical simulation results. A seamless holographic image is reconstructed at 2.4 m or more from the multi-vision display in the proposed holographic multi-vision system. Therefore, it is possible to provide a natural, seamless virtual scene at a distance of less than 1 m from the observer. In the current system, the backlight part of the multi-vision, such as the light source and the beam shaping part, becomes very bulky when using a reflective optical system. In the future, we plan to incorporate a waveguide, an optical component that uses total internal reflection to fold an optical path, into this system so as to make the system compact by reducing the illumination distance.

Disclosure statement
No potential conflict of interest was reported by the author(s).