Image-brightness prediction model for wide-color-gamut displays based on Helmholtz–Kohlrausch (H–K) effect

A wider array of color gamut and higher luminance are increasingly offered by displays to provide users with immersive content. However, in the latest displays available, it is difficult to predict image brightness because the extensively used color appearance model is made based on a color patch whose color appearance is not the same as that of an image on the display. Therefore, in this study, a method of predicting image brightness on a display depending on its color gamut by quantifying the Helmholtz–Kohlrausch (H–K) effect is proposed. A method for accurately measuring image luminance is also presented. To verify the performance of the proposed model, a brightness matching experiment is performed under two different color gamut sizes. The experimental results confirm that the image appears brighter with the increase in color gamut due to the H–K effect, demonstrating that the proposed prediction model exhibits desirable performance. The proposed brightness prediction model, which includes the image experience in the color range, delivers better image quality to users.


Introduction
The brightness of a color is affected not only by its luminance, but also by its chroma. An example of such a phenomenon is the Helmholtz-Kohlrausch (H-K) effect in which two colors with the same luminance but different chroma in the same hue appear different [1]. The H-K effect occurs not only when viewing a color patch but also when viewing an image. Consequently, it has been suggested that the H-K effect should be considered for predicting image brightness because the chroma increases with the expansion of the color gamut of the display, causing the image to appear brighter [2]. This implies that the brightness of the display presented to users depends on the color gamut of the display. However, the H-K effect has been extensively investigated using color patches, and few studies have investigated the H-K effect of the image under a color gamut narrower than the digital cinema initiatives (DCI)-P3 color gamut published by the Society of Motion Picture and Television Engineers (SMPTE) [3].
To predict image brightness, it is important to measure the luminance of the image with precision. However, unlike a liquid crystal display (LCD) that emits light using a backlight, a self-emissive display emits light using each pixel to increase the contrast and color range. This characteristic causes the loading effect, where the luminance increases with the reduction in the image signal loaded on the screen [4,5]. Moreover, the latest displays control image luminance to efficiently provide high luminance [6]. Thus, the luminance of the image is not the same as that of the color patch created through the traditional measurement method using a neutral color background or without a background on the full screen. Therefore, it is necessary to find a method for accurately measuring the image luminance and develop a brightness prediction model based on the H-K effect. This study proposes a new prediction model based on the H-K effect as well as a method for accurately measuring the image luminance. The proposed model is evaluated through a brightness matching experiment under two different color gamut sizes at high luminance.

Proposed brightness prediction model for images on displays
The color appearance of an image is not the same as that of a simple patch on a display. The latest displays available provide a broad color range to offer a new experience for viewers. On displays with a wide color gamut, the image appears brighter because of the H-K effect. We defined this experienced color range as XCR. To predict image brightness on a display, 'XCR Q' was defined as the brightness of an image in the experienced color range.
The steps involved in the proposed brightness prediction model are described below.

Step 1: convert the RGB of each pixel to XYZ based on the color on-pixel ratio
The average pixel level (APL) has been applied in measuring the luminance of images on displays considering the loading effect [7][8][9]. However, the APL does not consider the relationship between the RGB values and luminance, even though it is not linear. Therefore, in this study, the color on-pixel ratio (C-OPR) was proposed to measure the tristimulus values of images regardless of the loading effect. The C-OPR is calculated as the average of the RGB values to which the power function is applied for the display electro-optical transfer function (EOTF) as follows: where n is the number of pixels, γ is gamma value of the display, and R i , G i , B i are the RGB values of the ith pixel. Thus, C-OPR is the average value of linear RGB having a linear relationship with luminance.
To measure the tristimulus values of the red, green, and blue colors, three-color patches were generated and measured. The color patch size was set to 1% of the image size, while the background color was set such that the C-OPR values were the same as those of the image. For the measured tristimulus values of the red, green, and blue patches, the RGB channel level was converted to XYZ for each pixel, based on the gain-offset-gamma (GOG) model. This measurement method represents the image experience of the color range but does not represent the characteristics of the display itself. Therefore, to measure the display characteristics, it is necessary to define and select a standard image.

Step 2: estimate the equivalent luminance considering the Helmholtz-Kohlrausch (H-K) effect
According to Withouck et al., the equivalent luminance defined by CIE (L eq_CIE200 ) provides the most desirable performance for the related color among the five prediction models in terms of compensating for the H-K effect [10]. Consequently, it was used in this study to quantify the H-K effect [11]. To predict the equivalent luminance considering the H-K effect, the L eq_CIE200 of each pixel was calculated from the XYZ of each pixel. Based on the L eq_CIE200 of each pixel, XYZ eq having the same chromaticity as XYZ and the same luminance as L eq_CIE200 was calculated. Note that as the degree of the H-K effect is calculated for each pixel, it differs according to the chromatic influence of the pixel. Kim et al. proposed the color appearance model for predicting brightness under high-luminance [12]. This model was developed for high-luminance-level displays, such as self-emissive displays that include various color appearance effects such as the Hunt effect, Stevens effect, and simultaneous contrast effect, but not the H-K effect. Therefore, the new brightness prediction model was developed by combining the equivalent luminance for quantifying the H-K effect and the brightness prediction model for the image appearance at high-luminance levels. XYZ eq , calculated in step 2 to compensate the H-K effect, was used as input to brightness prediction. Finally, XCR Q was calculated as the mean value of the brightness, Q, for each pixel using the color appearance model proposed by Kim et al. In summary, this study proposes a new XCR Q prediction method based on the H-K effect. To measure the tristimulus values of the image, the C-OPR concept was developed. Figure 1 depicts the process involved in the proposed brightness prediction model for the XCR Q of the image.

Brightness matching experiment
To verify the performance of the proposed brightness prediction model, a psychophysical experiment was conducted using a two-alternative forced choice method (2AFC). Fifteen subjects with normal color vision participated in this experiment. The participants were seated 2.3-m away from the display. After adapting to a dark room for 2 min, the participants were asked to evaluate whether the test stimulus was brighter or darker than the reference stimulus in binary.
To display the test stimulus and reference stimulus side-by-side, a 65-inch self-emissive display was used. The display's resolution was 3840 × 2160 pixels, while the resolution of the test stimulus and reference stimulus was set to 40% of the display size. The field of view (FoV) of the reference and test stimuli was controlled at 28°. Four images with colorful content, including the International  Organization for Standardization (ISO) image, landscape images, and an image of people with a black background, were used to generate the stimuli. Figure 2 depicts the experimental environment and the color gamut of the test and reference stimuli (the white text in Figure 2(a) was used for describing the experimental environment and was not shown on the display during the experiment).
The peak white of the reference stimulus was controlled at 700 cd/m 2 , whereas that of the test stimulus was set to 24 levels from 316-912 cd/m 2 with the same log steps. The color gamut of the reference stimulus was simulated as DCI-P3, whereas that of test stimulus was set to the maximum color gamut of the self-emissive display itself, which is 32.5% wider than DCI-P3 based on CIE 1976 u'v' chromaticity diagram. Color gamut areas for simulating the test stimulus and reference stimulus were 0.108 and 0.081, respectively, based on CIE 1931 xy chromaticity diagram, and 0.185 and 0.152 respectively, based on CIE 1976 u'v' chromaticity diagram. The white points of the test and reference stimuli were controlled to have the same chromaticity as that of the display itself, with 2.2 gamma for display electro-optical transfer function (EOTF) applied to both stimuli.
To find the luminance of the test stimulus having equal brightness as the reference stimulus, 24 luminance levels were repeated 10 times for each participant. The locations of the test and reference stimuli were varied: half of the reference stimulus was shown on the left, whereas the other half was shown on the right. Thus, the experiment included eight sections (4 images × 2 locations), and the order of the luminance level of the test stimulus was randomly selected in each section. A total of 960 responses (4 images × 2 locations × 24 test luminance levels × 5 repetitions) were obtained from each participant.

Results and model performance
The responses from the 15 participants were used to extract the luminance value of the test stimulus having equal brightness as the reference stimulus, using a logistic psychometric function. Figure 3 displays the experimental results, where the x-axis indicates the mean luminance of the reference stimulus and the y-axis indicates the mean luminance of the test stimulus at equal brightness. The dashed line is a 45°line, which indicates that the luminance of the reference stimulus is the same as that of the test stimulus at equal brightness. The maximum luminance and mean luminance were calculated by summing the luminance of each pixel and dividing it by the number of pixels in the image. The luminance of each pixel was calculated using the GOG model based on the measured XYZ of the red, green, and blue color patches, which were generated based on the C-OPR values. The luminance of the white patch was measured based on the C-OPR value. As depicted in Figure 3, the maximum luminance of the image and the luminance of the white patch with a C-OPR background (peak white) exhibit considerable image dependency. Although the mean luminance of the image shows image dependency, there is a positive relationship between the mean luminance of the reference stimulus and that of the test stimulus whose brightness is equal to that of the reference stimulus. Moreover, these results show that having equal brightness, the luminance of the test stimulus is lesser than that of the reference stimulus in terms of the maximum luminance, luminance of the white patch, and mean luminance. On average, the mean luminance of the reference stimulus is 22%, 26%, and 23% higher than that of the test stimulus having equal brightness in terms of the maximum luminance, luminance of the white patch, and mean luminance, respectively.
This implies that the test stimulus, which has a wider color gamut than the reference stimulus, appears brighter than the reference stimulus even though they have the same peak white because of the H-K effect. However, the degree of the H-K effect differs depending on the chroma and hue. As shown in Figure 4, the histograms of the chroma and hue differ depending on the image content.
Therefore, it is important to quantify the H-K effect for images, depending on the color characteristics of the    image. According to the proposed model, the degree of the H-K effect and the equivalent luminance are calculated for each pixel based on CIE200. Thus, the brightness based on the H-K effect differs depending on the chromatic influence of the image. Figure 5 depicts the brightness calculation process for image 1. Figure 6 shows the calculated XCR Q based on the proposed brightness prediction model. The x-and y-axes represent the predicted XCR Q of the reference stimulus and that of the test stimulus having equal brightness. Compared to the luminance results (Figure 3(c)), the predicted brightness, XCR Q, of the reference stimulus is similar to that of the test stimulus having the same brightness. The experimental results indicate that the XCR Q performance is good. Figure 7 compares the prediction error of CIECAM02 Q and XCR Q. Prediction error is the difference in the predicted brightness between the reference stimulus and test stimulus having equal brightness. CIECAM02 Q for each pixel was predicted using XYZ calculated based on the measured XYZ of the red, green, and blue color patches on a full screen. Thus, CIECAM02 does not consider the loading effect or the H-K effect. In Figure 7, the zero value on the y-axis indicates that the model predicts the brightness perfectly, and the eight circles and triangles depict the results of CIECAM02 and XCR Q, respectively, for each section. On average, XCR Q exhibits desirable performance with a prediction error of 0.54, compared to CIECAM02 Q with a mean prediction error of 3.99.

Conclusion
This study proposed a new brightness prediction model for images on a display. The concept of C-OPR was designed to measure the tristimulus of the images on a display regardless of the luminance change in accordance with the on-pixel ratio. Based on the measured XYZ of the red, green, and blue colors using a C-OPR patch, the equivalent luminance defined by CIE was calculated to quantify the H-K effect for the images shown on the display. The image brightness was subsequently estimated considering the mean value of the pixel brightness, which was predicted using the color appearance model for high luminance. To verify the performance of the proposed model, a brightness matching experiment was conducted under two different color gamut sizes. The image on the display with a wider color gamut appeared brighter due to the H-K effect. The results of the psychophysical experiment involving 15 participants demonstrated the excellent performance of the proposed model. The findings of this study indicate that it is necessary to quantify the real image experience of the color range, including the loading effect and H-K effect, for predicting the brightness of images on displays. The proposed brightness prediction model, which includes the image experience in the color range, offers better image quality for users.

Disclosure statement
No potential conflict of interest was reported by the author(s).