Color saturation control by modulating spectral power distribution of illumination using color enhancement factors

Abstract. Color enhancement factors are spectral components that modulate the spectral power distribution (SPD) of illumination for interactive color saturation control. They can enhance more than one target color simultaneously or independently while maintaining the current color appearance. However, the color enhancement factors obtained in our previous work depended on the lighting system. Moreover, the effects of color muting, such as decreasing saturation of the target color, were limited because the color enhancement factors were optimized to increase color saturation. We overcame these two issues and succeeded in implementing color enhancement factors that can both enhance and mute color saturation. First, we conducted detailed simulations of color saturation enhancement and muting to determine parameters for calculating the color enhancement factors (e.g., the center wavelength and full-width at half-maximum) and assessed their potentiality. The results showed that maximum variations of color muting became almost the same as those of color enhancement. We also found that SPDs of illumination for increasing and decreasing color saturation of red, green, and blue had three peaks, respectively, and that they appeared for enhancement and muting alternately. In addition, the color enhancement factors for enhancing and muting color saturation were roughly symmetric. We then performed experiments using a newly designed 12-color LED lighting system, and the results showed color enhancing and muting effects. Finally, we demonstrated color restoration of discolored cultural heritage and coloring simulation of industrial products.

deformation conducted in projection mapping. However, in general, when one color is enhanced by illumination with a controlled SPD, other colors, especially white, are also changed by the illumination. This destroys the total color balance of the scene and affects an observer's impression as well as the image recognition accuracy of computer vision systems. Overcoming these issues requires a method that can optimize the SPD of illumination for enhancing a target color while maintaining the color appearance of other colors.
Here our goal is to adjust color enhancement variations of a target color interactively by optimizing the SPD of illumination like it can with photoretouching software. In this paper, we make the following contributions.
(1) We introduce color enhancement factors and achieve interactive color saturation control.
Color enhancement factors are spectral components and are used to modulate the SPD of illumination. (2) We also introduce color enhancement factors optimized for decreasing color saturation and achieve that the maximum variations of color muting become almost the same as those of color enhancement. When color enhancement factors were optimized for increasing color saturation, color muting effects were smaller than those for color enhancement because of the constraint that the wavelength element of the SPD of illumination light must take positive values.
Once color enhancement factors are obtained, the calculations for modulating the SPD of illumination are the weighted summations of both spectral components. The costs of these calculations are small enough for real-time processing, which enables us to control color saturation interactively. Color saturation can be varied gradually by changing the weights of color enhancement factors in small steps. In addition, the number of control parameters becomes equal to the number of target colors, for instance, three when red, green, and blue are the target colors. The color enhancement factor also enables us to enhance more than one target color simultaneously or independently while maintaining the color appearance of other colors.
In a previous conference paper, 11 this concept was presented briefly only for color enhancement, and the possibility of interactive color saturation control was shown. In this paper, we describe the details of this idea mathematically and present a successful implementation of color enhancement factors for both color enhancement and muting using a combination of LEDs whose center wavelengths are different.
First, we conducted detailed simulations to find the best combination of LEDs for increasing and decreasing color saturation. Red, green, and blue patches of X-Rite ColorCheker™ were used as target colors. The center wavelength and the full-width at half-maximum (FWHM) of LEDs were varied independently, and color variations of target and non-target colors were evaluated. In addition, constraints of color shifts of non-target colors were varied, and their effects to enhance target color were evaluated. Next, according to these results, we calculated color enhancement factors optimized for increasing and decreasing color saturation. We then conducted experiments using a newly designed 12-color LED lighting system. Color enhancement factors were calculated using SPDs of its LEDs, and their effects were evaluated using X-Rite ColorChecker™. In addition, we demonstrated color restoration of discolored cultural heritage and coloring simulation of industrial products.
The remainder of this paper is organized as follows. Section 2 covers related work and describes issues to overcome. Section 3 describes what a color enhancement factor is and how it is implemented. Section 4 presents detailed simulations we conducted to find optimized SPDs of illumination for color enhancing and muting, from which we determined the color enhancement factors. Section 5 describes experiments using a 12-color LED lighting system. Section 6 first summarizes this paper and then mentions some applications and a limitaion of our method, as well as future work.

Related Work
There are various LEDs for enhancing visual color impressions of vegetables, fish, and meats. 12 The color temperature of LEDs for vegetables and fish is 3500 K; for meats and cuts of raw fish it is 2700 K. These LEDs are often used in supermarkets. Some of them are customized white LEDs, and their SPD of around 580 nm is tuned to enhance color saturation with a moderating yellowish tint. In addition, a reddish object's color becomes more saturated by shifting the peak wavelength from around 600 to 630 nm or longer. However, these LEDs cannot enhance the colors of vegetables and meats at the same time. Moreover, they should not be used in adjacent areas for color enhancement, because doing so causes differences in the appearances of white, which confuses our color perception.
Simonian et al. 13 developed a lighting system using 12 color channels and achieved color enhancement while maintaining the reference light's color temperature. In their approach, an iterative optimization calculation is conducted by evaluating the error between the objective and replicated SPDs using feedback from the spectral sensor. The objective function consists of terms of color temperature and the object's color, and the combination of each color channel for color enhancement can be obtained. Resultant combinations of each color channel for various applications are memorized and switched to fit the selected application. However, their experimental results showed that increasing the saturation of green caused enhancement of red and magenta, and decreasing the saturation of green caused enhancement of yellow and muting of red.
Nakauchi et al. 14 designed the SPD of illumination to discriminate the colors of blueberry jam and foreign substances. They implemented it using a tunable multi-wavelength light source, which can control light intensities of all wavelengths and generate light with various SPDs. This lighting system has good flexibility for designing SPDs, but the need to determine the light intensity at all wavelengths makes the calculation cost high. In addition, this lighting system is too expensive for practical use. Ito et al. 15 enhanced the color differences between the skin and blood vessels using three types of LEDs to emphasize them. However, the color rendering properties of light synthesized using such a combination of LEDs are not sufficient for accurate color reproduction. This is because the half bandwidth of an LED is narrow and thus has little energy in some parts of the visible wavelength spectrum. Tsuchida et al. 16 achieved accurate color reproduction and color enhancement with metameric white using a multi-color LED lighting system. However, they were unable to achieve interactive color saturation control.
These previous systems [14][15][16] require optimization of the combination of LEDs every time the target color for enhancement is changed even when the color saturation varies a little. In addition, many parameters, such as the brightness of each LED, are needed to control the SPD of illumination light. These issues make it difficult to adjust color enhancement variations interactively because of the high calculation cost. Note that our method has few differences from these previous systems from the viewpoint of optimizing SPDs of illumination. However, there are no comparative methods for interactive control of color saturation with metameric white maintained.

Model of Object Surface Reflection
The spectrum of reflection from an object's surface is modeled using the SPD of incident light (e.g., the illumination spectrum), wðλÞ, and spectral reflectance of a point on the object's surface, rðλÞ where C t ðλÞ represents the CIE color matching function, and v t represents CIE-XYZ trichromatic values. When the object's color is white, rðλÞ takes constant values for all wavelengths: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 2 ; 1 1 6 ; 6 7 7 Generally, when we want to enhance a certain target color on an object, we usually try to control the SPD wðλÞ in Eqs. (1) and (2). However, this affects not only the target color but also other colors, especially white, which destroys the color balance of the scene and makes the scene unnatural. Color appearances other than that for a target color should be maintained during and after target color enhancement.

Introduction of Color Enhancement Factors
Let us introduce color enhancement factors to enhance only a target color while maintaining other color appearances. A color enhancement factor is a spectral component corresponding to a variation in the color saturation of a target color and is calculated using metameric conditions among several colors.
Metamerism is a perceived matching of colors with different SPDs, and it can be represented mathematically using Eq. (1) as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 3 ; 1 1 6 ; 4 6 4 Let us consider the wavelength range from 380 to 780 nm in view of the human eye's sensitivity. Let C X , C Y , and C Z ∈ R 401 be CIE color matching functions for X, Y, and Z, respectively, r ¼ ½r 380 ; · · · r 780 T ∈ R 401 be spectral reflectance of an object, w ¼ ½w 380 ; · · · w 780 T ∈ R 401 be SPD of reference illumination, and x ¼ ½x 380 ; · · · x 780 T ∈ R 401 be SPD of illumination for color enhancement. Equation (3) is rewritten as a matrix formulation as follows: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 4 ; 1 1 6 ; 3 5 8 Let Lð Þ be an operator for calculating CIE a Ã b Ã values of a given spectrum and kLð Þk be an operator for calculating fða Ã Þ 2 þ ðb Ã Þ 2 g 1∕2 . Once the effect of color enhancement against a target color is known, the color difference between before and after enhancing a color of an object is evaluated on the CIE a Ã b Ã color space. Figure 2 shows the relationship between an object's color under the reference illumination: LðCRwÞ and its enhanced color; LðCRxÞ on the CIE a Ã b Ã color space. Let us compare the distances of each color from the white point: ða Ã ; b Ã Þ ¼ ð0;0Þ. The object's color is enhanced when kLðCRxÞk > kLðCRwÞk. On the other hand, it is muted when kLðCRxÞk < kLðCRwÞk. Therefore, when the object's color is maximally enhanced, the SPD of illumination for enhancing the object's color is obtained by finding x that maximizes the distance between LðCRwÞ and ðCRxÞ on the CIE a Ã b Ã color space under the constraint kLðCRxÞk > kLðCRwÞk. This can be represented as where Gð Þ represents an operator for calculating the ratio of a Ã and b Ã .
The process for finding x is conducted under the constraint that the enhanced color is plotted on a line defined by a white point, e.g., ða Ã ; b Ã Þ ¼ ð0;0Þ, and the object's color under the reference illumination w is used for obtaining the color enhancement factor. The color enhancement factor of the color i is defined as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 6 ; 1 1 6 ; 5 2 4 where x i represents SPD of illumination which maximizes kLðCRwÞ − LðCRxÞk. Some wavelength elements of the color enhancement factor may have negative values. The SPD of illumination x i for controlling color saturation of the object's color i is then represented as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 7 ; 1 1 6 ; 4 4 5 under the restriction that every wavelength elements of x i must has positive values. The color saturation of the object's color i can be controlled continuously by varying parameter k i . When k i becomes larger than 1.0 or smaller than −1.0, several wavelength elements of x i become negative, which makes the metameric white condition disappear. Using Eqs. (6) and (7), the chromatic value of illumination for enhancing the target color i Cx i is represented as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 8 ; 1 1 6 ; 3 4 2 w and x i are satisfied with the metameric white condition under the assumption of Eq. (5). Then Eq. (8) shows that x i is also metameric white with w. This color enhancement method can be easily extended for controlling several colors simultaneously. Let us consider the situation where the number of object's colors for enhancement is N. Let spectral reflectance of the t'th object's color be R t ¼ diagðr t Þ. The process for finding x is conducted according to E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 0 9 ; 1 1 6 ; 2 3 9 maximize x X N t¼1 kLðCR t wÞ − LðCR t xÞk; s:t: kLðCR t xÞk > kLðCR t wÞk; t ¼ 1;2; · · · ; N:

Implementation
Color patches of a color chart whose spectral reflectance is known can be used to obtain color enhancement factors. Let us consider the case where "red" is the target color to be enhanced: s:t: kLðCR red xÞk > kLðCR red wÞk kLðCR green wÞ − LðCR green xÞk < δ green ; where U 0 ð Þ and V 0 ð Þ represent operators for calculating CIE u 0 and v 0 , C Y represents the color matching function of Y, and δu 0 white , δv 0 white , δY white , δ blue , and δ green represent error margins of differences in each color. Then the color enhancement factor of red e red is obtained as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 1 ; 1 1 6 ; 5 6 9 When the SPD of illumination can be modeled as a linear combination of the SPD of LEDs n k , x red is represented as E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 2 ; 1 1 6 ; 5 1 3 where α k and N represent intensity parameters of each LED and the number of LEDs, respectively. Note that α k is not limited only in the range [0, 1], because kn k k takes different values for each LED and α k corresponds to the number of each LED in a real system. x red is determined by finding α k for all LEDs satisfying Eq. (10).
To obtain color enhancement factors of green and blue, the same procedures from Eqs. (10)- (12) for each color are conducted. In the simulations and experiments described below, color enhancement factors for red, green, and blue were calculated. That is to say, the optimization procedure was carried out only six times. Only illumination spectra optimized for maximum enhancement and muting of target colors were required, so there was no need to calculate intermediate spectra. Calculation cost can be reduced compared to conventional methods, even though the calculation includes the optimization with non-linear constrains and objectives. This is an important feature of our proposed method.
Target colors to be enhanced are not limited to red, green, and blue. Other colors such as bluish green, yellow, or orange can be enhanced using this framework. Note that the number of target colors is not limited to one. The color saturation of red and blue can be controlled at the same time.

Optimizations of SPD of Illumination for Color Enhancing and Muting
Artificial solar lighting (XELIOS, SERIC Ltd.) was used as the reference illumination in the simulations described in this section. Figure 3 shows the SPD of the artificial solar light. Simulations were conducted under the following conditions: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 3 ; 1 1 6 ; 1 9 2 fðδu white Þ 2 þ ðδv white Þ 2 g 1∕2 < 0.004; E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 4 ; 1 1 6 ; 1 4 8 E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 5 ; 1 1 6 ; 1 2 6 δ blue ; δ green ; δ red ≤ 3; Equations (13) and (14) correspond to color differences on the CIE u 0 v 0 color space and the difference in brightness on CIE Y between their values under the reference and color enhancement illuminations. The threshold in Eq. (13) was determined by experimental trial and error. Equation (15) corresponds to color differences in color patches on the CIE a Ã b Ã color space between their values under the reference and color enhancement illuminations. The threshold was determined according to the criterion that the CIE dEa Ã b Ã color difference perceptible to the human eye at a glance was under 3. The range of wavelengths for calculating spectral components was 400 to 700 nm for luminous efficiency.

Color Enhancement
Color enhancement factors for red, green, and blue were calculated using the spectral reflectance of color patches of X-Rite ColorChecker™. Figure 4 shows the spectral reflectance of red (no. 15), green (no. 14), and blue (no. 13) patches.

Enhancement of color patches of red, green, and blue
First, SPDs of illumination were optimized for enhancing target color patches [e.g., x in Eq. (10) for red]. They were represented as a linear combination of Gaussian functions [see Eq. (12)]. Let us consider cases where the FWHM of the Gaussian function are 10, 20, and 40 nm to confirm the possibility of implementing the proposed procedure in a multi-color LED lighting system. The peak position of the Gaussian function ranges from 440 to 660 nm, and its interval is 10 nm. Let the difference in the hue of the target color (i.e., the error of gradient in CIE a Ã b Ã color space) be <0.1: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 6 ; 1 1 6 ; 3 3 5 q t ¼ fGðR t xÞ − GðR t wÞg GðR t wÞ < 0.1 deg t∶red; green; blue; (16) where Gð Þ represents an operator for calculating the ratio of a Ã and b Ã , and t represents an object's color (i.e., red, green, or blue). The threshold was determined by experimental trial and error. The combinations of Gaussian functions were determined using the generalized reduced gradient (GRG) method 17 with a multi-start search algorithm. The multi-start search algorithm  ensures that the local optimal solution is closer to the global solution. The reduced gradient method was developed based on a simple variable elimination technique for equality-constrained problems. The GRG method is an extension of the reduced gradient method to accommodate non-linear inequality constraints. In this method, a search direction is found such that for any small move, the current active constraints remain precisely active. If some active constraints are not precisely satisfied because of non-linearity of the constraint functions, the Newton-Raphson method is used to return to the constraint boundary. Thus the GRG method can be considered somewhat similar to the gradient projection method. Note that other algorithms such as sequential quadratic programming or a genetic algorithm can be used instead of the GRG method. A full search to find the combinations was also conducted in simulations, and the solutions were very close to the results obtained with the GRG method. Note that there are no novelties and originalities in our use of such algorithms.
The procedure for optimizing illumination spectra should be conducted for each color enhancement factor. Only illumination spectra optimized for maximum enhancement and muting of target colors are required; calculating intermediate spectra is unnecessary. Calculation cost can be reduced compared with conventional methods, even when the calculation includes the optimization with non-linear constrains and objectives. This is another important feature of our proposed method. In this work, three color enhancement factors, i.e., for red, green, and blue, were obtained. That is to say, the optimization procedure was carried out only six times for color enhancement and muting.
Let us evaluate the enhancement effect for each color using variations on the CIE a Ã b Ã color space between colors under the artificial solar light and under the optimized illumination. Let the color variations of red, green, and blue be D red ¼ kLðCR red wÞ − LðCR red x red Þk, D green ¼ kLðCR green wÞ − LðCR green x green Þk, and D blue ¼ kLðCR blue wÞ − LðCR blue x blue Þk. Figure 5 plots a Ã and b Ã when the FWHM of the Gaussian function is 20 nm.  D green were 6.36, 5.96, and 6.06, respectively. Although the enhancement effect of color saturation was weaker than for red, the effect was observed.
On the other hand, D blue values were 3.69, 3.30, and 3.26, which indicate that the enhancement effect was not observed for the blue patch. This might be because of the restriction of the error of hue [Eq. (16)].
Then D red , D green , and D blue were calculated again without the restriction. Figure 6 shows the results. D red were 38.7, 37.9, and 30.3 (q red were 0.17, 0.17, and 0.14), D green were 26.0, 22.7, and 14.1 (q green were 0.40, 0.37, and 0.25), D blue were 14.2, 12.7, and 10.3 (q blue were 0.76, 0.75, and 0.71). Without the restriction, the variety of every color became larger. The hue of blue moved slightly to reddish, but the enhancement effect of color saturation was observed. However, D blue was less than half of D red and D green .
To enlarge D blue , restrictions on the color difference of the red and green [δ red , δ green in Eq. (14)] were eased. Here let us consider the cases where δ red , δ green ≤ 5 and 10. Figure 7 plots a Ã and b Ã of each case. D blue were 18.7, 17.3, 14.8 when δ red , δ green ≤ 5 and 30.0, 28.5, 24.9 when δ red , δ green ≤ 10. These results show that the more the blue was enhanced, the more the saturation of red was lost and the more yellowish the green became.
In terms of actually increasing the saturation of the color, the FWHM of LEDs should be made smaller with their center wavelength maintained. However, in the proposed method, the LEDs were switched to LEDs whose center wavelength was near that of the LEDs used before color enhancement, or the proportion of brightness of those LEDs was changed instead of changing their FWHM. That is why the hues of green and blue changed. On the other hand, the shape of spectral reflectance was smooth (see Fig. 4), and the FWHM of the LEDs were narrow enough. This enhanced the saturation with the hue of the color before color enhancement maintained in some cases (e.g., red in simulation results shown above). In addition, the improvement of color enhancement effects is due to a greater degree of orthogonality between two spectrally adjacent LEDs, i.e., the overlap of the SPD. On the other hand, the smaller FWHM of LEDs is, the larger becomes number of each LED for obtaining the same brightness as the reference illumination. In addition, a sparse or spiky spectral distribution of a light source makes the color rendering index low, i.e., the ability of the light source to accurately render the colors of the objects is reduced. In such cases, target colors become enhanced, but others might lose their color appearance. Therefore, a balance between the orthogonality of spectrally adjacent LEDs and density of the SPD of the total output is also important.

Variations of color on the color chart
Using SPDs of optimized illuminations described above, we verified variations of the color of 15 other patches on the color chart (except for gray patch nos. 19 to 24) on the a Ã b Ã color space. Here optimized illuminations calculated without the restriction of the error of hue [see Eq. (15)] were used for red and green. To enhance blue, we used the illumination calculated under the condition δ green , δ red ≤ 10. Figure 8 shows variations of each color plotted on the a Ã b Ã color space when the FWHM of the Gaussian function was 20 nm. The color of each dot on the graphs corresponds to the color of each patch. Colors under the artificial solar light are represented as gray dots. The triangle defined by red, green, and blue is used to show trends in color variations. The blue mesh represents the color under the artificial solar light; the red mesh represents that under the optimized illumination.
First, let us compare the case of enhancing red with the case of enhancing green. In both cases, colors whose a Ã was positive increased their value, and colors whose a Ã was negative decreased their value. Focusing on yellow and orange, we see that the enhancement effect was larger when green was enhanced than when red was.
Next, let us compare the case of enhancing blue when δ green , δ red ≤ 3 and δ green , δ red ≤ 10. For colors other than blue, green, and red, the color variation seemed to exhibit the same trends in both conditions. The more the color enhancement effect of blue increased, the more a Ã of orange decreased as it did for red. Figures 9 and 10 show SPDs of illuminations optimized for enhancing red, green, and blue in the simulations described above. For blue, they show the results obtained under the condition δ green , δ red ≤ 3, 5, 10.
First, let us compare the case of enhancing red with the case of enhancing green, focusing on the wavelengths of 610 and 660 nm. The spectrum with a peak at 610 nm affected the enhancement of green, whereas that with a peak at 660 nm affected the enhancement of red. Although both spectra correspond to red, the wavelength of 610 nm affected orange much more than it did red. This indicates the behavior of the color of orange on the a Ã b Ã color space.
Next, let us compare the case of enhancing blue. Illumination spectra for enhancing blue under the conditions δ green , δ red ≤ 3, 5, and 10 are shown in Fig. 10. Let us focus on the wavelengths of 500 and 610 nm. The more blue was enhanced, the more the power of the spectrum with a peak at 500 nm decreased. In addition, the spectrum with a peak at 660 nm decreased first, and the spectrum with a peak of 610 nm decreased next, which made the saturation of red decrease. These spectral behaviors are consistent with color variations on the a Ã b Ã color space. Figure 11 shows color enhancement factors of red, green, and blue calculated using the spectra in Fig. 8 (FWHM of the Gaussian function of 20 nm; δ blue , δ red , δ green , δ red ≤ 5) and the artificial solar light in Fig. 3.
In following sections, let us consider two situations: muting of the saturation of a target color (i.e., k i < 0) and simultaneous enhancement of red, green, and blue.

Color Muting
First, let us consider the case where saturation of a target color was decreased. Note that k i can take a negative value only while the spectral element of the optimized illumination takes a positive one. Some of the spectral elements of the color enhancement factors had positive values, which made the ranges of negative k i smaller than those of positive k i . Actually, the minimum values of k red , k green , and k blue were −0.04, −0.36, and −0.34, and D red , D green , and D blue in this case were 1.96, 8.17, and 9.69, respectively. Variations of each color were smaller than in the case of k red , k green , k blue ¼ 1.
Then SPDs of illumination for decreasing saturation of red, green, and blue were calculated in the same way they were for enhancing the color saturation. Figure 12 shows a Ã and b Ã of each  Variations of the color of 18 patches plotted on the a Ã b Ã color space are shown in Fig. 13. Light blue triangles are defined by red, green, and blue under the standard illumination, and light pink triangles are defined by red, green, and blue under the generated illumination. In the case of muting red, variations in reddish patches (nos. 9, 15, and 17) are large. In the case of muting green, variations in green (no. 14), bluish-green (nos. 6 and 18), and orangish patches (nos. 7 and 12) are large. In the case of muting blue, variations in bluish and bluish-green patches (nos. 6, 14, and 18) are large. In addition, those in light green (no. 11), yellow (no. 16), and orangish patches (nos. 7 and 12) are relatively large. This is caused by easing the restriction on variations of red and green (δ red δ green < 10). Figure 14 shows SPDs of generated illumination for enhancing and muting red, green, and blue. Gray lines represent the artificial solar light, colored dash lines represent the generated illumination for color enhancement, and colored solid lines represent the generated illumination for color muting. Under the designed illuminations for red and green, the SPDs have three peaks, and they appear for enhancement and muting alternately. This also applies to the designed illuminations for blue, except for peaks between 600 to 650 nm.

Enhancing Several Colors Simultaneously
Next, let us consider the case where red, green, and blue are enhanced at the same time. The two approaches described below were taken for this purpose. In the first approach, color enhancement factors of the three colors were combined: E Q -T A R G E T ; t e m p : i n t r a l i n k -; e 0 1 7 ; 1 1 6 ; 1 1 9 e red þ e green þ e blue ¼ x red þ x green þ x blue − 3w: (17) According to the brightness invariance of the illumination and non-negative condition of the spectral component, the variation of each color became one third of that when each color was enhanced separately:   ðe red þ e green þ e blue Þ∕3 þ w ¼ ðx red þ x green þ x blue Þ∕3: In the second approach, illumination x satisfying the following equation was calculated: s:t: kLðCR red wÞk < kLðCR red xÞk kLðCR green wÞk < kLðCR green xÞk kLðCR blue wÞk < kLðCR blue xÞk: (19) Figure 15 plots aÃ and bÃ of 18 color patches. Most of the color of patches were enhanced. D red , D green , and D blue were 36.7, 23.9, and 23.0, which are almost the same as when each color was enhanced separately. The hue of green was better maintained than when green was enhanced independently.
The SPDs for the optimized illumination are shown in Fig. 16, along with SPDs of illuminations optimized for enhancing red, green, and blue (dashed lines) for comparison with the obtained illumination. The SPD of the obtained illumination has three peaks. The peak at the shortest wavelength is the same as the green enhancement illumination, and the peak at the longest wavelength is the same as the red enhancement illumination. The peak at the middle  wavelength exists between that of green enhancement illumination and that of blue enhancement illumination. This peak exists on the longer wavelength side than that of green enhancement, which contributed to keeping the green hue.

Experiments
A 12-color LED lighting system ( Fig. 17; customized LEDCube™, THOUSLITE) was used to generate and control the SPDs of illumination. The LED-controlling software has a graphical user interface to increase and reduce color enhancement factors of red, green, and blue continuously. The SPDs of each LED are shown in Fig. 18(a). Center wavelengths of the LEDs are 430,  450, 460, 465, 480, 505, 525, 535, 595, 610, 620, and 660 nm. Most of the LEDs have a 15-or 20-nm FWHM, but three LEDs concerned with green have wider FWHMs, which are 30, 35, and 40 nm. These LEDs were chosen from a catalogue of products to satisfy two conditions: one is that each LED would, as far as possible, provide almost the same maximum intensity of light; the other was that the chromaticity of each LED on CIE u 0 − v 0 space would, as far as possible, be at even intervals. Figure 18(b) shows each LED's color on a u 0 − v 0 chromaticity diagram. The SPD of the artificial solar lighting in the simulation was generated using the 12 LEDs and used as a reference illumination. The weights of each LED were set to <0.5 to increase the saturation of target colors in the case of color enhancement. Figure 19 shows the SPD of the generated reference illumination and weights of each LED.  Spectral reflectances of X-Rite ColorChecker™ were used to calculate color enhancement factors and evaluate color variations by our proposed method. A spectroradiometer (SR-3, TOPCON) was used to measure the spectral radiance of color patches on the color chart and the SPDs of illumination. Resultant images were captured using a six-band imaging system 18 because general RGB cameras cannot reproduce the actual colors of objects under an illumination having spiky SPDs such as those shown in Fig. 14.

Implementation on 12-Color LED Lighting System
SPDs of illumination for enhancing red (no. 15), green (no. 14), and blue (no. 13) were obtained by calculating combinations of the SPDs of LEDs. The combination of the 12 LEDs and their brightness were determined using the GRG method. Note that 19 spectral data (∈ R 401 ) were used for determining intensity coefficients of the 12 LEDs: SPDs of each LED, the SPD of the reference illumination, spectral reflectance of three color patches, and the color matching function for obtaining CIE XYZ values. This shows that calculation cost is practical. The GRG method was implemented on Solver, a Microsoft Excel add-in program, and was used through-  factors for increasing and decreasing the saturation of target colors, respectively. For red and green, color enhancement factors for increasing and decreasing saturation are roughly symmetric. For blue, the color enhancement factor at wavelengths shorter than 570 nm shows the same trend as red and blue, but the factors at wavelengths longer than 600 nm have the same values in the case of both increasing and decreasing color saturation. This affects color variations when parameter k i defined in Eq. (4) is changed. Using the calculated color enhancement factors, we evaluated variations of each color by varying parameter k i from −1 to 1. Figure 21 shows resultant images of the color chart with enhanced and muted red, green, and blue. It can be seen that each target color was well controlled and how the controlled colors affected the other color patches. Color variations in the case of enhancing and muting red, green, and blue plotted on the a Ã − b Ã color space are shown in Fig. 22.
First, let us look at the result for red [ Fig. 22(a)]. The hue of red was almost completely maintained while k red was varied. Maximum enhancement of color saturation (k red ¼ 1) resulted in dEa Ã b Ã of 17.1; maximum muting of color saturation (k red ¼ −1) gave dEa Ã b Ã of 29.0. The maximum color saturation was smaller than in the simulation (dEa Ã b Ã ¼ 38.5 in Fig. 6). This is due to the limitation of the brightness of the LED system. The maximum value of the weight modulating the brightness of each LED was 1.0 in this experiment (see Fig. 19). On the other hand, the maximum weight in the simulation result in Fig. 8 was 6.3.
Next, let us look at the result for green [ Fig. 22(b)]. The hue of green was almost completely maintained while k green was varied. Maximum enhancement of color saturation (k green ¼ 1) gave dEa Ã b Ã of 14.6; maximum muting of color saturation (k green ¼ −1) gave dEa Ã b Ã of 7.1. The color differences for both the maximum and minimum color saturations were much smaller than in the simulation (dEa Ã b Ã ¼ 22.9 and 19.8, see Figs. 5 and 11). This was due to the FWHM of the LEDs, which was 20 nm in the simulations and ranged from 30 to 40 nm for green in the experiments.
Finally, let us look at the result for blue [ Fig. 22(c)]. The hue of blue changed the same as it did in the simulation. On the other hand, the direction of the change in hue was almost maintained while k blue was varied. Maximum enhancement of color saturation (k blue ¼ 1) gave dEa Ã b Ã of 13.6; maximum muting of color saturation (k green ¼ −1) gave dEa Ã b Ã be 12.0. These results are almost the same as those of the simulations.
Variations of color whose appearance should remain fixed were within dEa Ã b Ã ≤ 2.3. In addition, these results show that each target color can be enhanced or muted.

Color Restoration of Discolored Cultural Heritage
Our proposed method can be applied to the virtual restoration of discolored cultural heritage. Our restoration results can be seen by observing the actual target object directly instead of images on a monitor. Therefore, the material appearance of target objects is well maintained. In addition, unlike projection mapping, our method does not require any image capturing or processing. Figure 23 shows results of a color manipulation of an old discolored woodblock print (printed in the 19th century): original image (left top), enhanced red (right top), enhanced blue (left bottom), and enhanced green (right bottom). Effects of color enhancement can be observed well on the reddish jacket and bluish pants with the green texture. Figure 24 shows color variations of the jacket, pants, and green texture on the a Ã -b Ã color space. In the case of red, green, and blue enhancements, their color varied as dEa Ã b Ã ¼ 6.02, 7.34, and 4.44, respectively.

Coloring Simulation of Industrial Products
Our method can also be applied to coloring simulation of industrial products. It can enhance or mute saturation of a target color on a mock-up model. Figure 25 shows results of a color adjustment simulation of car models: enhanced red (right), muted red (left), enhanced blue (top), and muted blue (bottom). Figure 26 shows color variations of the red and blue cars on the a Ã -b Ã color space. For the red car, the color variation was dEa Ã b Ã ¼ 16.21 in the case of enhancement and 16.16 in the case of muting. For the blue car, the color variation was dEa Ã b Ã ¼ 10.00 in the case of enhancement and 17.45 in the case of muting. These results show the possibility of reducing the need for mock-up models for delicate color adjustments.

Summary
Control of color saturation by modulating the SPD of illumination was achieved. Color enhancement factors were introduced, which, by varying a few parameters, enable us to easily modulate the SPD of illumination while retaining metameric white and to gradually control the strength of color enhancement effect. SPDs of illumination optimized for enhancing and muting color saturation were designed by simulation. Their effects were evaluated on a CIE a Ã −bÃ color space by varying parameters for synthesizing illumination and restrictions on the metameric condition.
In addition, simulation results indicated that color enhancement factors for increasing and decreasing saturation were roughly symmetric. In experiments, our method was implemented on a 12-color LED lighting system. Color shift variations were evaluated using color patches of a color chart. The results indicate the possibility of applying this method to virtual restoration of cultural heritage and coloring simulation of industrial products. We demonstrated these applications using an old discolored woodblock print  Tsuchida, Kimura, and Harada: Color saturation control by modulating spectral power distribution. . . and car models and showed the effectiveness of the proposed method. Our method can be applied not only to multi-color LED lighting systems but also to tunable multi-wavelength light sources that control light intensities of all wavelengths.
Our method has potential for not only to industrial uses but also for applications, such as in-home medical care. Optimized illumination can enhance the color differences between the skin and underlying blood vessels for, for instance, early detection of bleeding under the skin while maintaining the natural skin's color. This capability will be helpful in making decisions to see a doctor. Other applications include enhancing the colors of vegetables, seafood, and meat simultaneously to visually accent their freshness and color shading of paints for maintenance and aesthetics.  One shortcoming of our method is that colors with a similar hue, such as orange and red, cannot be enhanced. This is because their spectral reflectance has overlaps in most of the wavelength range, so illumination cannot affect only one of the colors. This restricts the number of target colors. The number should be under three when target colors are selected from color patches on a color chart like we did in our experiments. One way to overcome the limitation is to calculate the color enhancement factors of some combination of three target colors beforehand and switch the combination of color enhancement factors case by case. Another is to use a color chart with more highly saturated colors to calculate color enhancement factors. These two solutions will be experimentally assessed in our future work. We also plan to investigate how enhancing (or muting) color rendering affects human perception of objects or scenes and perceived quality.
Masaru Tsuchida received his BE, ME, and PhD degrees from Tokyo Institute of Technology, Tokyo, Japan, in 1997, 1999, and 2002, respectively. Since 2002, he has been at the NTT Communication Science Laboratories. From 2003 to 2006, he worked at the National Institute of Information and Communication Technology as a researcher. Since 2011, he has been a visiting researcher at Ritsumeikan University, Kyoto, Japan. His current research interests include computer vision, color measurement, and multiband image processing.
Akisato Kimura received his BE, ME, and PhD degrees in communications and integrated systems from Tokyo Institute of Technology, Tokyo, Japan, in 1998, 2000, and 2007, respectively. Since 2000, he has been at the NTT Communication Science Laboratories, where he is currently the leader of the Recognition Research Group. His research interests include pattern recognition, multimedia content analysis, machine learning, and computer vision.
Noboru Harada received his BS and MS degrees in computer science from the Department of Computer Science and Systems Engineering, Kyushu Institute of Technology, Fukuoka, Japan, in 1995 and 1997, respectively, and his PhD in computer science from the Graduate School of Systems and Information Engineering, University of Tsukuba, Ibaraki, Japan, in 2017. His current research interests include acoustic signal processing and machine learning for acoustic event detection, including anomaly detection in sound.