JP7521322B2 - IMAGE PROCESSING APPARATUS, PROGRAM, AND IMAGE PROCESSING METHOD - Google Patents

Description

The present invention relates to an image processing device, program, and image processing method for converting a color gamut.

When sending and receiving image data between different devices, gamut mapping is performed to convert the colors into colors that can be reproduced in the color gamut of the output device. The color conversion method used in gamut mapping can be selected from a variety of methods depending on policies, preferences, etc.

Here, the color conversion method will be explained with reference to FIG. 10. FIG. 10 is a diagram explaining various color conversion methods in general gamut mapping. FIG. 10 is a graph extracting the saturation and brightness of a color space, with the horizontal axis representing saturation and the vertical axis representing brightness. FIG. 10 shows an example in which the color gamut of an input device, indicated by light-colored lines, is converted to the color gamut of an output device, indicated by dark-colored lines, by color conversion using gamut mapping.

Figure 10(a) is an example of color conversion that emphasizes brightness. Figure 10(b) is an example of color conversion that emphasizes vividness. Figure 10(c) is an example of color conversion that emphasizes fidelity. Figure 10(d) is an example of color conversion that emphasizes tonality. Note that the color conversion method in gamut mapping may be a method other than that shown in Figure 10.

In order to improve the color reproducibility of an output device, a technique is known that performs gamut mapping by combining various color conversion methods. For example, Japanese Patent Publication No. 5715442 (Patent Document 1) discloses a technique in which a color gamut is divided into multiple regions and color conversion is performed using a different method for each region.

However, in the conventional techniques including Patent Document 1, there is a problem that gradation inversion may occur as explained in the following Fig. 11. Fig. 11 is a diagram showing an example of gamut mapping by dividing a color gamut in the conventional technique. Fig. 11 shows an example of gamut mapping performed by dividing the color gamut into two regions, upper and lower, at the position of lightness _Ls , applying a color conversion method that emphasizes gradation as shown in Fig. 10(d) to the first region and applying a color conversion method that emphasizes lightness as shown in Fig. 10(a) to the second region.

In conventional technology, as shown in the dotted circular area in Figure 11, gradation inversion occurs in the color gamut of the output device, and color reproducibility is sometimes impaired. For this reason, there has been a demand for technology to improve color reproducibility in color conversion using gamut mapping.

The present invention has been made in consideration of the problems with the above-mentioned conventional technology, and aims to provide an image processing device, program, and image processing method that improve color reproducibility.

That is, according to the present invention,
a conversion means for dividing a color gamut into at least two regions and performing gamut mapping on control points related to color conversion of the color gamut by a conversion method set for each divided region,
The conversion means is
An image processing device is provided, characterized in that when a control point occurs in the gamut mapped color gamut whose gradation characteristics are outside a predetermined reference value, the conversion result for the control point is combined with the conversion result when the control point is color converted using a conversion method set for an area other than the area including the control point.

The present invention provides an image processing device, program, and image processing method that improve color reproducibility.

FIG. 1 is a diagram showing a schematic hardware configuration of an entire system according to a first embodiment. FIG. 1 is a diagram showing a hardware configuration included in an image processing apparatus according to a first embodiment. FIG. 2 is a block diagram of software included in the image processing apparatus according to the first embodiment. 4 is a flowchart showing a color conversion process according to the first embodiment. 5A and 5B are diagrams illustrating tone characteristics according to the first embodiment. 5A to 5C are diagrams for explaining an example in which tone characteristics do not satisfy a standard in gamut mapping according to the first embodiment. 5A to 5C are diagrams showing an example in which gradation characteristics are improved in the first embodiment. 10 is a flowchart showing a color conversion process according to a second embodiment. 13A to 13C are diagrams showing an example in which gradation characteristics are improved in the second embodiment. 1A and 1B are diagrams for explaining various color conversion methods in general gamut mapping. FIG. 1 is a diagram showing an example of gamut mapping by dividing a color gamut in a conventional technique.

The present invention will be described below with reference to an embodiment, but the present invention is not limited to the embodiment described below. In addition, in each of the figures referred to below, the same reference numerals will be used for common elements, and their description will be omitted as appropriate.

Figure 1 is a diagram showing a schematic configuration of the overall hardware of a system 100 in the first embodiment. In Figure 1, a system 100 is shown to which a personal computer terminal 110a and a monitor 110b are connected as an example of an image processing device 110 equipped with an image output means.

For example, consider the case where an image created on personal computer terminal 110a is displayed on monitor 110b. The image output means of personal computer terminal 110a, which is the input display device, and the image output means of monitor 110b, which is the output display device, do not necessarily have the same color gamut characteristics. Therefore, in the first embodiment, when an image is displayed on monitor 110b, gamut mapping is performed to convert the color gamut on the personal computer terminal 110a side to the color gamut on the monitor 110b side so as not to impair color reproducibility. Furthermore, in the gamut mapping in the first embodiment, the color gamut is divided into multiple regions and a conversion method selected for each region is applied, thereby improving color reproducibility.

Note that, in the following, an image processing device 110 having the configuration shown in FIG. 1 will be described as an example, but this is not intended to limit the embodiment. Therefore, as long as it outputs an image having a predetermined color gamut, it may be an image processing device other than the personal computer terminal 110a or the monitor 110b, and may be, for example, an image processing device with a print function such as an MFP.

Next, the hardware configuration of the image processing device 110 will be described. FIG. 2 is a diagram showing the hardware configuration included in the image processing device 110 of the first embodiment. The image processing device 110 includes a CPU 210, a RAM 220, a ROM 230, a storage device 240, a communication I/F 250, a display 260, and an operation device 270, and each piece of hardware is connected via a bus.

The CPU 210 is a device that executes programs that control the operation of the image processing device 110 and performs predetermined processing. The RAM 220 is a volatile storage device that provides an execution space for the programs executed by the CPU 210, and is used for storing and expanding programs and data. The ROM 230 is a non-volatile storage device that stores the programs and firmware executed by the CPU 210.

The storage device 240 is a readable/writable non-volatile storage device that stores the OS and various software that operate the image processing device 110, configuration information, various data, and the like. Examples of the storage device 240 include a hard disk drive (HDD) and a solid state drive (SSD).

The communication I/F 250 connects the image processing device 110 to other devices, enabling communication with the other devices via a network. Communication via the network may be either wired communication or wireless communication, and various types of data can be sent and received using a specific communication protocol such as TCP/IP.

The display 260 is a device that displays various data and the status of the image processing device 110 to the user, and an example of such a device is an LCD (Liquid Crystal Display). The operation device 270 is a device that enables the user to operate the image processing device 110, and an example of such a device is a keyboard or a mouse. Note that the display 260 and the operation device 270 may each be separate devices, or may be a device that has the functions of both, such as a touch panel display.

The above describes the hardware configuration included in the image processing device 110 of the first embodiment. Next, the functional means executed by each piece of hardware in the first embodiment will be described with reference to FIG. 3. FIG. 3 is a software block diagram included in the image processing device 110 of the first embodiment. The image processing device 110 in the first embodiment is configured to include a data acquisition unit 310, a conversion method setting unit 320, a color conversion unit 330, a gradation evaluation unit 340, and a display unit 350. Each functional block will be described in detail below.

The data acquisition unit 310 is a means for acquiring data related to gamut mapping. The data acquisition unit 310 acquires, for example, via the communication I/F 250, an input signal indicating the color space of the input device and setting information for the color conversion method. Furthermore, when gamut mapping is performed by dividing the color gamut into multiple regions, the setting information includes information indicating the divided regions.

The conversion method setting unit 320 is a means for setting a conversion method for color conversion in gamut mapping for the color gamut related to the input signal based on the data acquired by the data acquisition unit 310. The conversion method setting unit 320 can set methods such as color conversion that emphasizes vividness, color conversion that emphasizes vividness, color conversion that emphasizes fidelity, and color conversion that emphasizes gradation. In addition, when gamut mapping is performed by dividing the color gamut into multiple regions, the conversion method setting unit 320 can set a conversion method for each region.

The color conversion unit 330 is a means for gamut mapping the input color gamut to convert it to the output color gamut using the color conversion method set by the conversion method setting unit 320. When the color gamut is divided into multiple regions, the color conversion unit 330 of this embodiment can convert to the output color gamut by performing gamut mapping for each region and combining the conversion results of each region. The color conversion unit 330 can also synthesize multiple color conversion results based on the evaluation results of the gradation characteristics by the gradation evaluation unit 340, which will be described later.

The gradation evaluation unit 340 is a means for evaluating the gradation characteristics of the conversion result by the color conversion unit 330. The gradation evaluation unit 340 evaluates whether the gradation characteristics are within a predetermined range based on the control points related to the color conversion of the input side color gamut (hereinafter referred to as "input side control points") and the control points constituting the output side color gamut color-converted in response to the input side control points (hereinafter referred to as "output side control points"). The gradation characteristics can be calculated from the brightness difference, saturation difference, or hue difference between adjacent control points on the color gamut, or a combination of these.

The display unit 350 is a means for controlling the display 260 and displaying images.

The above-mentioned software blocks correspond to functional means realized by the CPU 210 executing the program of this embodiment and causing each piece of hardware to function. Furthermore, the functional means shown in each embodiment may be realized entirely in software, or some or all of them may be implemented as hardware that provides equivalent functions.

Next, the series of processes executed by each of the functional means described above will be explained with reference to FIG. 4. FIG. 4 is a flowchart showing the color conversion process in the first embodiment. In the following explanation of FIG. 4, FIG. 5 to FIG. 7 will also be referenced as appropriate.

The image processing device 110 starts processing from step S1000. In step S1001, the data acquisition unit 310 acquires an input signal indicating the color gamut of the input device related to gamut mapping and color conversion setting information.

Next, in step S1002, the conversion method setting unit 320 divides the color gamut of the input device into multiple regions based on the acquired setting information, and sets a color conversion method for each region.

In the next step S1003, the color conversion unit 330 performs gamut mapping for each region using the conversion method set in step S1002, and converts the colors to the output gamut. Here, the color conversion unit 330 combines the results of the gamut mapping for each region to convert the colors to the output gamut.

Then, in step S1004, the gradation evaluation unit 340 calculates gradation characteristics for the results of the gamut mapping in step S1003. The gradation characteristics are calculated for the signal direction in which the color gamut is divided. For example, when the gamut is divided in the lightness direction, the gradation characteristics are calculated for control points of the same hue in the input signal. Here, the gradation characteristics in the first embodiment will be explained with reference to FIG. 5. FIG. 5 is a diagram explaining the gradation characteristics in the first embodiment.

Fig. 5(a) is a graph showing the color gamut on the output side, and the black circles in the figure indicate the output side control points. In Fig. 5(a), L _i represents the lightness of the control point P _i , L _w represents the lightness of white, and L _b represents the lightness of black. The gradation characteristic value T _i in the first embodiment can be calculated, for example, as shown in the following formula (1).

The gradation evaluation unit 340 calculates the gradation characteristics for each output control point using the above formula (1). Fig. 5B is a graph in which the gradation characteristic value T _i calculated from the above formula (1) is plotted for each output control point. In Fig. 5B, k _i indicates the gradient of the gradation characteristic value between the output control points P _i and P _i-1 . The gradation evaluation unit 340 evaluates the gradation of each output control point P _i based on the ratio R _i of the gradient of the gradation characteristic value calculated, for example, from the following formula (2).

The ratio R _i of the gradient of the gradation characteristic value corresponds to a value indicating the change in gradient of the gradation characteristic at an arbitrary output side control point P _i . The gradation evaluation unit 340 can evaluate the gradation based on R _i calculated from the above formula (2).

Returning to FIG. 4 for the explanation, after evaluating the gradation based on the above formulas (1) and (2) in step S1004, in step S1005, the process branches based on whether the gradation characteristics are within a predetermined reference value. The criteria for evaluating the gradation characteristics are not particularly limited and can be set arbitrarily. As an example, if the allowable change in slope is set to within 50%, it can be determined whether the gradation characteristics are within the reference value based on whether 0.5≦R _i ≦1.5 is satisfied. The reference value can be experimentally determined based on color reproducibility, and is preferably set to about 0≦R _i ≦2.0 in order to prevent gradation inversion and gradation skipping.

If the gradation characteristics are within the standard in step S1005 (YES), it is assumed that color conversion has been performed with appropriate color reproducibility, and the process proceeds to step S1007, where the process ends. On the other hand, if the gradation characteristics are not within the standard in step S1005 (NO), the process proceeds to step S1006 to improve the gradation characteristics.

In step S1006, the color conversion unit 330 performs a process of combining the current color conversion result for the output side control points whose gradation characteristics do not satisfy the criteria with the color conversion result obtained when color conversion is performed using the color conversion method set in the other area. Details of the process in step S1006 will now be described with reference to Figures 6 and 7. Figure 6 is a diagram illustrating an example in which the gradation characteristics do not satisfy the criteria in gamut mapping in the first embodiment. Figure 7 is a diagram illustrating an example in which the gradation characteristics are improved in the first embodiment.

Fig. 6(a) shows a case where the input side color gamut is color converted by a method that emphasizes gradation. Also, Fig. 6(b) shows a case where the input side color gamut is color converted by a method that emphasizes lightness. Here, consider a case where the input side color gamut is divided by lightness _Ls , and input side control points included in a first region with lightness equal to or greater than _Ls are color converted by a method that emphasizes gradation, and input side control points included in a second region with lightness less than _Ls are color converted by a method that emphasizes lightness.

In such a case, the gamut mapping result shown in Fig. 6(c) is output by combining the output control points _P1a , _P2a , and _P3a obtained by color-converting the input control points included in the first region in Fig. 6(a) with the output control points _P4b , _P5b , and _P6b obtained by color-converting the input control points included in the second region in Fig. 6(b). In the gamut mapping result shown in Fig. 6(c), gradation inversion occurs near the output control points _P3a and _P4b in the output gamut. Therefore, the gradation evaluation unit 340 determines in step S1005 of Fig. 4 that the gradation characteristics do not satisfy the criteria.

The color conversion unit 330 then performs a process of combining the current color conversion results with the color conversion results of other adjacent areas for output control points whose gradation characteristics do not satisfy the criteria (step S1006 in FIG. 4). The combination of color conversion results will be described with reference to FIGS. 7(a) and (b).

FIG. 7A shows an example of changing the color conversion method of the input control point _P3 . The input control point _P3 is included in the first region, so it is color converted by a method that emphasizes gradation. However, since the gradation characteristics after color conversion do not meet the criteria, the color conversion results are synthesized. Therefore, the color conversion unit 330 of the first embodiment synthesizes the current color conversion result of the input control point _P3 by averaging the color conversion result when it is color converted by the color conversion method set in the other region. In the example of FIG. 7A, the color conversion unit 330 synthesizes _P3a , which is color converted by emphasizing gradation, and _P3b , which is color converted by emphasizing brightness. By synthesizing the color conversion results of each region in this way, an output control point such as _P3ab is obtained.

FIG. 7B shows an example of changing the color conversion method of the input control point _P4 . The input control point _P4 is included in the second region, so it is color converted using a method that emphasizes lightness. However, since the gradation characteristics after color conversion do not meet the criteria, the color conversion results are synthesized. Therefore, the color conversion unit 330 of the first embodiment synthesizes the current color conversion result of the input control point _P4 by averaging the color conversion result when it is color converted using the color conversion method set in the other region. In the example of FIG. 7B, the color conversion unit 330 synthesizes _P4a , which has been color converted with emphasis on gradation, and _P4b, which has been color converted with emphasis on lightness. By synthesizing the color conversion results of each region in this way, an output control point such as _P4ab is obtained.

By combining the color conversion results as shown in Figure 7(a) and (b), the gamut mapping result shown in Figure 7(c) is obtained. In the example of Figure 7(c), the gradation inversion shown in Figure 6(c) is eliminated, and an output color gamut with improved color reproducibility can be obtained. In addition, the gradation characteristics shown in Figure 7(d) are obtained with a mitigated inversion of the slope k as shown in Figure 5(b).

Let's return to FIG. 4 for further explanation. In step S1006, after compositing the color conversion results for the output control points whose gradation characteristics do not satisfy the criteria, the image processing device 110 ends the process in step S1007.

The process shown in Figure 4 enables gamut mapping that improves the color reproducibility of the output color gamut.

Note that, although the above-mentioned FIGS. 4 to 7 show examples of evaluating gradation characteristics based on the brightness difference of the output side control points, this is not intended to limit the scope of the embodiment. Therefore, gradation characteristics may be evaluated by methods other than those described in FIGS. 4 to 7. Here, as another method of evaluating gradation characteristics, evaluation may be based on the cumulative color difference obtained by adding up the color differences between adjacent control points, as shown in the following formula (3).

In the above formula (3-1), ΔE is the color difference between adjacent control points. Also, in the above formula (3-2), N is the number of output control points to be evaluated. In the case of the hue direction, control points with the same lightness in color space are extracted from the input signal, and the lightness can be converted to hue for calculation.

Furthermore, the evaluation of the gradation characteristics may be based on a standard generated by the learning effect of machine learning. Here, machine learning is a technology for enabling a computer to acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates an algorithm required for judgments such as data identification from learning data that is previously loaded, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods; any learning method for machine learning is acceptable.

So far, the first embodiment has been described. Below, the second embodiment will be described. The image processing device 110 in the second embodiment can further improve color reproducibility by repeating the evaluation of gradation characteristics and the synthesis of color conversion results in the first embodiment. Note that the hardware configuration and software block configuration in the second embodiment are the same as those shown in Figures 1 to 3, so detailed description will be omitted.

Figure 8 is a flowchart showing the color conversion process in the second embodiment. The image processing device 110 starts the process from step S2000. Note that the processes in steps S2001 to S2006 shown in Figure 8 are similar to the processes in steps S1001 to S1006 in Figure 4, so the description will be omitted.

In the second embodiment, unlike the first embodiment, the image processing device 110 returns to the process of step S2004 after executing the process of step S2006, and calculates the tone characteristics again. Then, if the tone characteristics of the gamut mapping executed in step S2006 are not within the standard (NO in step S2005), the conversion results are synthesized again. That is, in the second embodiment, the current color conversion result is repeatedly synthesized with the color conversion result in the area where another conversion method is set until the tone characteristics satisfy a predetermined condition. As a result, the image processing device 110 of the second embodiment can further improve the color reproducibility of the gamut mapping compared to the first embodiment in which the color conversion results are synthesized only once.

Figure 9 is a diagram showing an example of improved gradation characteristics in the second embodiment. The upper parts of Figures 9(a) to (c) show the results of gamut mapping the input gamut to the output gamut, and the lower parts of Figures 9(a) to (c) show the gradation characteristics of each gamut mapping result. Also, Figure 9(a) shows the color conversion result of step S2003, Figure 9(b) shows the result of combining the color conversion results once (first step S2006), and Figure 9(c) shows the result of combining the color conversion results twice (second step S2006).

Consider the case where gradation inversion occurs near P _ia and P _(i+1)b in the first gamut mapping in which different color conversion methods are set for each region, as shown in Fig. 9(a). In such a case, since the gradation characteristics do not satisfy the criteria, in step S2006, P _ia and P _(i+1)b are combined with the color conversion results in other regions. Note that although the following description focuses on the processing for P _ia , it should be noted that similar processing can also be performed on other output side control points such as P _(i+1)b .

In step S2006 in the second embodiment, the color conversion unit 330 obtains an output side control point such as P _iab by combining P _ia and P _ib as shown in the upper part of FIG. 9B. In the second embodiment, the gradation evaluation unit 340 evaluates the gradation characteristics again after step S2006. As a result, the gradation characteristics shown in the lower part of FIG. 7B are obtained. The gradation characteristics shown in the lower part of FIG. 7B show an improvement from the gradation characteristics in FIG. 7A, but the slope is inverted and the improvement is not sufficient. Therefore, when the gradation characteristics are evaluated by returning to steps S2004 and S2005, the criterion is not met, and the image processing device 110 further performs the process of step S2006.

Furthermore, when performing the process of step S2006, as shown in the upper part of FIG. 9C, the color conversion unit 330 combines the current color conversion result P _iab combined as described above with the result P _ib of color conversion using the conversion method set for another region. By combining the color conversion results in this manner, the output side control point P _iab ' in FIG. 9C can be obtained. Furthermore, by performing such a process, the gradation characteristics are further improved from those shown in FIG. 9B, as shown in the lower part of FIG. 9C. Therefore, the image processing device 110 in the second embodiment can improve color reproducibility.

As described in the second embodiment, the image processing device 110 can further improve color reproducibility by repeating the synthesis of color conversion results and the evaluation of the gradation characteristics of the synthesized gamut mapping results until a specified condition is met.

According to the embodiments of the present invention described above, it is possible to provide an image processing device, a program, and an image processing method that improve color reproducibility.

The functions of the above-mentioned embodiments of the present invention can be realized by a device-executable program written in C, C++, C#, Java (registered trademark), etc., and the program of this embodiment can be distributed by being stored on a device-readable recording medium such as a hard disk drive, CD-ROM, MO, DVD, flexible disk, EEPROM (registered trademark), or EPROM, and can also be transmitted over a network in a format that can be used by other devices.

Each function of the above-described embodiments can be realized by one or more processing circuits. In this specification, the term "processing circuit" includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array), and conventional circuit modules designed to execute each function described above.

Although the present invention has been described above with reference to an embodiment, the present invention is not limited to the above-mentioned embodiment, and as long as the function and effect of the present invention are achieved within the scope of the embodiment that a person skilled in the art can imagine, it is included in the scope of the present invention.

100...system, 110...image processing device, 210...CPU, 220...RAM, 230...ROM, 240...storage device, 250...communication I/F, 260...display, 270...operation device, 310...data acquisition unit, 320...conversion method setting unit, 330...color conversion unit, 340...gradation evaluation unit, 350...display unit

Patent No. 5715442

Claims (7)

a conversion means for dividing a color gamut into at least two regions and performing gamut mapping on control points related to color conversion of the color gamut by a conversion method set for each divided region,
The conversion means is
When a control point occurs in the color gamut where the gamut mapping is performed, the gradation characteristics of which fall outside a predetermined reference value, the image processing device combines a conversion result relating to the control point with a conversion result obtained when the control point is color-converted using a conversion method set for an area other than the area including the control point. The image processing device according to claim 1, wherein the gradation characteristics are calculated based on the brightness difference, saturation difference, or hue difference, or a combination thereof, between adjacent control points in the gamut mapped color gamut. 3. The image processing apparatus according to claim 1, wherein said conversion means repeats the synthesis of the conversion results until the condition that said gradation characteristic falls within said predetermined reference value is satisfied. A program executed by an image processing device, the image processing device comprising:
Dividing the color gamut into at least two regions, and functioning as a conversion means for performing gamut mapping on control points related to color conversion of the color gamut by a conversion method set for each divided region;
The conversion means is
When a control point occurs in the gamut mapped color gamut whose tone characteristics fall outside a predetermined reference value, a conversion result relating to the control point is combined with a conversion result obtained when the control point is color converted using a conversion method set for an area other than the area including the control point. The program according to claim 4, wherein the gradation characteristics are calculated based on the brightness difference, saturation difference, or hue difference, or a combination thereof, between adjacent control points in the gamut mapped color gamut. 6. The program according to claim 4, wherein said conversion means repeats the synthesis of the conversion results until a condition that said gradation characteristic falls within said predetermined reference value is satisfied. A step of dividing a color gamut into at least two regions and performing gamut mapping on control points related to color conversion of the color gamut by a conversion method set for each divided region;
evaluating a result of the color transformation in the gamut mapping step;
and when a control point whose gradation characteristics are outside a predetermined reference value in the gamut of the gamut mapped is generated as a result of evaluating the gradation characteristics in the evaluating step, combining a conversion result related to the control point with a conversion result obtained when the control point is color-converted using a conversion method set for an area other than the area including the control point.