JP7613009B2 - IMAGE PROCESSING APPARATUS, IMAGE PROCESSING SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM - Google Patents

Description

The present invention relates to an image processing device, an image processing system, an image processing method, and a program.

In the past, various technologies have been proposed to convert the pixel values of an image captured by an imaging device into colorimetric values (coordinate values in a color space) that indicate the color of the subject, thereby making the color of the subject in the captured image closer to the appearance of the actual subject.

For example, the following Patent Document 1 discloses a technology that calculates a transformation matrix for converting pixel values into colorimetric values based on image data of a standard plate on which multiple color chips with different spectral reflectances are arranged, and then uses the transformation matrix to convert image data of a subject into colorimetric data.

Patent No. 4715288

However, in conventional technology, color conversion that converts a captured image into a colorimetric image is applied uniformly to the entire captured image, making it impossible to narrow the range of possible color characteristics, resulting in reduced accuracy in conversion from pixel values to colorimetric values. The accuracy of conversion from pixel values to colorimetric values increases the narrower (more specific) the range of color characteristics of the subject.

In view of the above-mentioned problems, the object of the present invention is to provide an image processing device, an image processing system, an image processing method, and a program capable of converting a captured image into a colorimetric image with high accuracy.

An image processing device according to one embodiment of the present invention includes an identification unit that identifies the color characteristics of the subject in an area included in an captured image captured by an imaging device that images the subject based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence between pixel values of the captured image and colorimetric values of the subject based on the color characteristics identified by the identification unit; an allocation unit that assigns the conversion information acquired by the acquisition unit to an area included in the captured image; and a conversion unit that converts the pixel values of the captured image to the colorimetric values based on the conversion information assigned to the area included in the captured image.When the subject is a printed matter, the design information is a platemaking image, and the identification unit identifies the position of the material of the subject included in the captured image and the color characteristics of the material by aligning the captured image with the platemaking image.
An image processing system according to one embodiment of the present invention has an imaging unit that images a subject, an identification unit that identifies the color characteristics of the subject in an area included in an image captured by the imaging unit based on design information indicating color characteristics for each material of the subject, an acquisition unit that acquires conversion information indicating a correspondence between pixel values of the image and colorimetric values of the subject based on the color characteristics identified by the identification unit, an assignment unit that assigns the conversion information acquired by the acquisition unit to an area included in the image, a conversion unit that converts the pixel values of the image into colorimetric values based on the conversion information assigned to the area included in the image, and a display unit that displays a display image based on the colorimetric values, and when the subject is a printed matter, the design information is a platemaking image, and the identification unit identifies the position of the material of the subject included in the image and the color characteristics of the material by aligning the image and the platemaking image.
An image processing method according to one embodiment of the present invention includes an identification unit identifying color characteristics of the subject in an area included in an captured image captured by an imaging device that images the subject based on design information indicating color characteristics for each material of the subject; an acquisition unit acquiring conversion information indicating a correspondence between pixel values of the captured image and colorimetric values of the subject based on the color characteristics identified by the identification unit; an assignment unit assigning the conversion information acquired by the acquisition unit to an area included in the captured image; and a conversion unit converting the pixel values of the captured image to the colorimetric values based on the conversion information assigned to the area included in the captured image, wherein when the subject is a printed matter, the design information is a platemaking image, and the identification unit aligns the captured image with the platemaking image to identify the position of the material of the subject included in the captured image and the color characteristics of the material.
A program according to one aspect of the present invention causes a computer to function as an identification unit that identifies color characteristics of the subject in an area included in an captured image captured by an imaging device that images the subject based on design information indicating color characteristics for each material of the subject; an acquisition unit that acquires conversion information indicating a correspondence between pixel values of the captured image and colorimetric values of the subject based on the color characteristics identified by the identification unit; an allocation unit that assigns the conversion information acquired by the acquisition unit to an area included in the captured image; and a conversion unit that converts the pixel values of the captured image into the colorimetric values based on the conversion information assigned to the area included in the captured image, and when the subject is a printed matter, the design information is a platemaking image, and the identification unit identifies the position of the material of the subject included in the captured image and the color characteristics of the material by aligning the captured image with the platemaking image.

The present invention makes it possible to convert a captured image into a colorimetric image with high accuracy.

1 is a diagram illustrating an example of a configuration of an image processing system according to a first embodiment. 1 is a block diagram showing an example of a configuration of an image processing apparatus according to a first embodiment. 4 is a diagram showing an example of a configuration of a profile table in a profile DB according to the first embodiment; FIG. 5 is a flowchart showing an example of a flow of conversion of a captured image according to the first embodiment. FIG. 11 is a diagram illustrating an example of identification information according to the second embodiment. FIG. 11 is a diagram showing an example of the configuration of a profile table in a profile DB according to the second embodiment; FIG. 13 is a diagram illustrating an example of identification information according to the third embodiment. FIG. 13 is a diagram showing an example of the configuration of a profile table in a profile DB according to the third embodiment; FIG. 13 is a diagram showing an example of a relationship between a captured image and design information according to the fourth embodiment. FIG. 13 is a diagram showing an example of the configuration of a profile table in a profile DB according to the fourth embodiment; 13 is a flowchart showing an example of a flow of conversion of a captured image according to a fourth embodiment.

The present invention relates to an image processing system capable of converting pixel values of an image of a subject into colorimetric values of the subject with high accuracy. In the following embodiments, an example in which the subject is a printed matter will be described as an example. Note that the subject is not limited to this example.
Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings.

<<1. First embodiment>>
A first embodiment of the present invention will be described with reference to Figures 1 to 4. In the first embodiment, an example will be described in which color characteristics of a printed material are specified based on information preset by a user.

<1-1. Configuration of image processing system>
The configuration of an image processing system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of an image processing system according to the first embodiment.
As shown in FIG. 1, the image processing system 1 includes an imaging device 10, an image processing device 20, and a display device 30.

The imaging device 10 and the image processing device 20 are connected to each other so that they can communicate with each other. For example, the imaging device 10 and the image processing device 20 are connected by a local area network (LAN). The LAN may be realized by a wired connection, or may be connected by a wireless connection such as Bluetooth (registered trademark) or Wi-Fi (registered trademark). The image processing device 20 and the display device 30 are also connected to each other so that they can communicate with each other.

The imaging device 10 is a device that captures an image of a subject. One example of the imaging device 10 is a camera. As shown in FIG. 1, the imaging device 10 of this embodiment captures an image of a printed matter 51 placed on a shooting stand 50. The printed matter 51 is an example of a "subject." The printed matter 51 is illuminated by a light 40. The imaging device 10 transmits the captured image obtained by capturing an image of the printed matter 51 to the image processing device 20.

In this embodiment, the printed matter 51 is a medium such as paper on which letters, numbers, symbols, figures, images, etc. are printed in ink. Printed area 52a, printed area 52b, printed area 52c, printed area 52d, and printed area 52e in the printed matter 51 indicate areas printed in ink.

The image processing device 20 is a device that performs image processing. Examples of the image processing device 20 include a PC (Personal Computer), a server, a smartphone, and a tablet terminal. The image processing device 20 of this embodiment performs a conversion process for converting pixel values into colorimetric values for the captured image received from the imaging device 10. In this way, the image processing device 20 converts the captured image into a colorimetric image. Here, the colorimetric image may be an image in which pixel values directly indicate colorimetric values such as XYZ or CIELAB, or an image in a color space in which the correspondence between pixel values and colorimetric values is uniquely determined, such as sRGB or AdobeRGB. The image processing device 20 transmits to the display device 30 a display image that has been converted from the colorimetric image obtained by the conversion process according to the display characteristics of the display device 30.

The display device 30 is a device that displays a display image based on color measurement values. The display device 30 is a display unit in the image processing system 1. An example of the display device 30 is a display. The display device 30 may be a display included in the image processing device 20, or may be an external display connected to the image processing device 20 by wire or wirelessly. When the display device 30 is an external display, the location where the display device 30 is provided is not particularly limited. For example, the display device 30 may be provided in a remote location different from the location where the subject is imaged.

<1-2. Configuration of image processing device>
The configuration of the image processing device 20 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the image processing device 20 according to the first embodiment.
As shown in FIG. 2, the image processing device 20 includes a communication unit 200 , a control unit 210 , and a storage unit 220 .

(1) Communication unit 200
The communication unit 200 communicates with other devices. The communication unit 200 transmits and receives various information through the communication. For example, in communication with the imaging device 10, the communication unit 200 receives an image from the imaging device 10. In communication with the display device 30, the communication unit 200 transmits a colorimetric value image to the display device 30.

(2) Control unit 210
The control unit 210 has a function of controlling the overall operation of the image processing device 20. The control unit 210 is realized by causing a CPU (Central Processing Unit) provided as hardware in the image processing device 20 to execute a program.
As shown in FIG. 2, the control unit 210 includes an identifying unit 2102 , an acquiring unit 2104 , an allocating unit 2106 , a converting unit 2108 , and an output processing unit 2110 .

(2-1) Specification section 2102
The identifying unit 2102 identifies color characteristics of the printed matter 51 in an area included in the captured image. The color characteristics of the printed matter 51 are, for example, the spectral reflectance of the material of the printed matter 51. The color characteristics of the printed matter 51 can vary depending on the combination of the type of ink, printing method, etc. Examples of the type of ink include special ink, process ink, etc. Examples of printing methods include letterpress printing, offset printing, gravure printing, screen printing, and inkjet printing. Hereinafter, information indicating the combination of the ink type, printing method, etc. of the printed matter 51 is referred to as "color characteristic information." It is also called.

In this embodiment, the identification unit 2102 identifies the color characteristics of the printed matter 51 by referring to color characteristic information preset by the user. The identification unit 2102 inputs the identified color characteristics of the printed matter 51 to the acquisition unit 2104.

(2-2) Acquisition unit 2104
The acquisition unit 2104 acquires a profile (conversion information). The profile is information for converting a captured image into a colorimetric image, and is used to determine the correspondence between pixel values of the captured image and the colorimetric values of the printed matter 51. The correspondence between the pixel values of the captured image and the color measurement values of the printed matter 51 depends on, for example, the color characteristics of the printed matter 51. For this reason, the profile is It will be prepared.

The correspondence between the pixel values of the captured image and the colorimetric values of the printed matter 51 may also depend on the characteristics of the imaging device 10. The characteristics of the imaging device 10 (e.g., spectral sensitivity) may vary depending on the type of imaging device 10, for example. Furthermore, the characteristics of imaging devices 10 may vary from device to device even if they are of the same type. For this reason, it is preferable that a profile is prepared according to the characteristics of the imaging device 10. A profile prepared according to the characteristics of the imaging device 10 is hereinafter also referred to as a "camera profile." There may be as many camera profiles as there are combinations of the characteristics of the imaging device 10 and the color characteristics of the printed matter 51.

The correspondence between pixel values and colorimetric values indicated by the camera profile is obtained based on pixel values of an image of a color chart captured by the imaging device 10 in advance in the same imaging environment as the imaging environment in which the imaging device 10 captures the printout 51. The camera profile may include correspondence between pixel values and colorimetric values that are not included when the color chart is captured. When a CLUT (Color Look-Up Table) is used as the format of the camera profile, interpolation using a model formula may be performed on the colorimetric values at each RGB grid point. When color conversion is performed, the interpolation process is performed by referring to the values in the CLUT.

When the colorimetric image is displayed on the display device 30, the color of the displayed colorimetric image may depend on the characteristics of the display device 30. Therefore, if the colorimetric image converted by the camera profile is displayed on the display device 30 as is, a difference may occur between the appearance of the printed matter 51 in the displayed colorimetric image and the appearance of the actual printed matter 51. Therefore, by further preparing a profile according to the characteristics of the display device 30 and applying the profile to the colorimetric image before displaying it on the display device 30, the difference can be reduced. The characteristics of the display device 30 may differ, for example, depending on the type of display device 30. Furthermore, the characteristics of the display device 30 may differ for each device even if they are the same type. Therefore, it is preferable that the profile is prepared according to the characteristics of the display device 30. The profile prepared according to the characteristics of the display device 30 is hereinafter also referred to as a "display profile". There may be as many display profiles as there are characteristics of the display device 30.

In this embodiment, each profile is prepared in advance by the user and stored in the storage unit 220 described later. The acquisition unit 2104 acquires a profile from the storage unit 220 based on, for example, color characteristics input from the identification unit 2102. The acquisition unit 2104 inputs the acquired profile to the allocation unit 2106.

(2-3) Allocation unit 2106
The allocation unit 2106 allocates a profile to a target image. For example, when a camera profile is input from the acquisition unit 2104, the allocation unit 2106 allocates the camera profile to an area included in the captured image. The allocation unit 2106 allocates the display profile input from the acquisition unit 2104 to the colorimetric image. The allocation unit 2106 inputs the image to which each profile is allocated to the conversion unit 2108. The allocation unit 2106 may allocate one camera profile to the entire captured image, or may allocate a camera profile to each area in the captured image. When the allocation unit 2106 allocates a camera profile to each area in the captured image, the allocation unit 2106 may allocate different camera profiles to each area, or may allocate the same camera profile to a part of each area. Moreover, the area in the captured image may include the entire captured image.

(2-4) Conversion unit 2108
The conversion unit 2108 performs image conversion processing based on the profile. For example, when a captured image to which a camera profile is assigned is input, the conversion unit 2108 converts pixel values of the captured image into colorimetric values based on the camera profile assigned to an area included in the captured image. In this way, the conversion unit 2108 converts the captured image into a colorimetric value image.

The conversion unit 2108 converts the colorimetric values of the colorimetric image into colorimetric values suitable for the display device 30 based on the display profile assigned to the colorimetric image. In this way, the conversion unit 2108 converts the colorimetric image into a display image. The conversion unit 2108 inputs the display image to the output processing unit 2110.

By performing conversion processing using the profile, the conversion unit 2108 can reproduce the colors of the subject in the image closer to the appearance of the actual subject. When the display device 30 displays an image captured by the imaging device 10 as in this embodiment, the image processing device 20 performs conversion processing on the captured image using a camera profile and conversion processing using a display profile. This allows the display device 30 to display an image of the printed matter 51 that takes into account the color characteristics of the imaging device 10 and the display device 30.

(2-5) Output Processing Unit 2110
The output processing unit 2110 controls the output of the display image. For example, the output processing unit 2110 transmits the display image input from the conversion unit 2108 from the image processing device 20 to the display device 30 via the communication unit 200. When the image processing device 20 has a display unit, the output processing unit 2110 may display the display image input from the conversion unit 2108 on the display unit.

(3) Storage unit 220
The storage unit 220 is configured by a storage medium, for example, a hard disk drive (HDD), a flash memory, an electrically erasable programmable read only memory (EEPROM), a random access read/write memory (RAM), a read only memory (ROM), or any combination of these storage media. The storage unit 220 can be, for example, a non-volatile memory.

The storage unit 220 has a function of storing various information. For example, the storage unit 220 stores various profiles. The various information is stored in a DB (database). For example, the various profiles are stored in a profile DB.

Now, referring to FIG. 3, an example of a profile stored in the profile DB will be described. FIG. 3 is a diagram showing an example of the configuration of a profile table in the profile DB according to the first embodiment. As shown in FIG. 3, a record in the profile table includes a profile name, color characteristics, pixel values, and colorimetric values. Note that the information included in the record in the profile table is not limited to this example.

The profile name is information indicating the name of the profile for each imaging device 10 or display device 30. The color characteristics are information indicating the color characteristics of the printout 51. The pixel value is information indicating the pixel value in the captured image and is indicated by (R, G, B). The colorimetric value is information indicating the value after conversion of the pixel value of the captured image and is indicated by (X, Y, Z). Note that the R, G, and B pixel values correspond to the X, Y, and Z colorimetric values, respectively. Figure 3 shows camera profile 1 and camera profile 2, which are profiles according to the characteristics of the camera, as examples of profiles.

Camera profile 1 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristics of printed matter 51 are "ink 1" and "printing method 1." This correspondence relationship indicates that pixel values (R _1-1 , G _1-1 , B _1-1 ) are converted to colorimetric values (X _1-1 , Y _1-1 , Z _1-1 ), and pixel values (R _1-2 , G _1-2 , B _1-2 ) are converted to colorimetric values (X _1-2 , Y _1-2 , Z _1-2 ).

Camera profile 2 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristics of printed matter 51 are "ink 2" and "printing method 2." This correspondence relationship indicates that pixel values (R _2-1 , G _2-1 , B _2-1 ) are converted to colorimetric values (X _2-1 , Y _2-1 , Z _2-1 ), and pixel values (R _2-2 , G _2-2 , B _2-2 ) are converted to colorimetric values (X _2-2 , Y _2-2 , Z _2-2 ).

<1-3. Processing flow>
A flow of captured image conversion according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing an example of a flow of captured image conversion according to the first embodiment.

As shown in FIG. 4, first, the user places the printed matter 51 on the photographing stand 50 (S102).

Next, the image processing device 20 acquires a captured image of the printed matter 51 placed on the imaging stand 50 (S104). The printed matter 51 is captured, for example, by the user operating the imaging device 10.

Next, the identification unit 2102 of the image processing device 20 identifies the color characteristics of the printed matter 51 included in the captured image (S106). For example, the identification unit 2102 identifies the color characteristics set by the user as the color characteristics of the printed matter 51.

Next, the acquisition unit 2104 of the image processing device 20 acquires a profile from the storage unit 220 based on the identified color characteristics (S108). For example, if the color characteristics are "ink 1" and "printing method 1", the acquisition unit 2104 acquires camera profile 1 from the profile DB of the storage unit 220.

Next, the assignment unit 2106 of the image processing device 20 assigns the acquired profile to the captured image (S110).

Next, the conversion unit 2108 of the image processing device 20 performs conversion processing based on the assigned profile (S112).

Finally, the display device 30 displays the display image obtained by the conversion process of the conversion unit 2108 (S114).

<<2. Second embodiment>>
The first embodiment of the present invention has been described above. Next, a second embodiment of the present invention will be described with reference to Figs. 5 and 6. In the second embodiment, an example will be described in which the color characteristics of a printed matter included in a captured image are identified based on identification information associated with the color characteristics of the printed matter. Note that the identification information in the second embodiment is identification information (first identification information) that is assigned to the printed matter. Note that, in the following, descriptions that overlap with the first embodiment described above will be omitted.

Now, with reference to FIG. 5, an example of the identification information according to the second embodiment will be described. FIG. 5 is a diagram showing an example of the identification information according to the second embodiment. As shown in FIG. 5, identification information 53 is printed (applied) on the printed matter 51. Examples of the identification information printed on the printed matter 51 include a one-dimensional code, a two-dimensional code, etc. that can be read by the imaging device 10. The one-dimensional code is, for example, a one-dimensional barcode. The two-dimensional code is, for example, a two-dimensional barcode. The identification information is associated with the color characteristics of the printed matter 51. As an example, the identification information is associated with the color characteristics of each material of the printed matter 51 and the position of the material in the captured image. Here, the material refers to the medium or ink used in the printed matter 51.

2-1. Image processing system configuration
The configuration of the image processing system according to the second embodiment is similar to the configuration of the image processing system 1 according to the first embodiment described above, and therefore detailed description will be omitted. Hereinafter, the same reference numerals as those in the image processing system 1 according to the first embodiment will be used for the configuration of the image processing system according to the second embodiment.

<2-2. Configuration of image processing device>
The configuration of the image processing device 20 according to the second embodiment is similar to the configuration of the image processing device 20 according to the first embodiment described above. Hereinafter, the same reference numerals as those of the image processing device 20 according to the first embodiment will be used for the configuration of the image processing device 20 according to the second embodiment. Note that in the image processing device 20 according to the second embodiment, part of the functions of the identification unit 2102 and part of the configuration of the profile table of the profile DB are different from those of the first embodiment. The differences from the first embodiment will be described below.

The identification unit 2102 of the second embodiment identifies the color characteristics of the printed matter 51 included in the captured image based on the identification information 53 included in the captured image. For example, the identification unit 2102 identifies the color characteristics associated with the identification information 53 as the color characteristics of the printed matter 51. If the color characteristics of each material of the printed matter 51 are associated with the identification information 53, the identification unit 2102 can identify the color characteristics of each material of the printed matter 51. Furthermore, if the position of the material of the printed matter 51 is associated with the identification information 53, the identification unit 2102 can identify the position of the material of the printed matter 51.

The identification unit 2102 identifies the color characteristics for each material of the printed matter 51, and the acquisition unit 2104 can acquire a profile for each material of the printed matter 51. Furthermore, the assignment unit 2106 can assign a profile for each material of the printed matter 51 to the captured image. This allows the conversion unit 2108 to convert the pixel values of the captured image for each material in accordance with the color characteristics of the material. This allows the conversion unit 2108 of the second embodiment to perform conversion processing with higher accuracy than the conversion unit 2108 of the first embodiment.

Now, with reference to FIG. 6, an example of a profile stored in the profile DB will be described. FIG. 6 is a diagram showing an example of the configuration of a profile table in the profile DB according to the second embodiment. As shown in FIG. 6, a record in the profile table includes a profile name, identification information, color characteristics, pixel values, and colorimetric values. Note that the information included in the record in the profile table is not limited to this example.

The identification information in the profile table is information that indicates the identification information associated with the color characteristics. In FIG. 6, camera profile 3 and camera profile 4, which are profiles according to the characteristics of the camera, are shown as examples of profiles.

The camera profile 3 is associated with the identification information "2D barcode 1", the color characteristics "ink 3" and "printing method 3".
Furthermore, camera profile 3 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristics of printed matter 51 are "ink 3" and "printing method 3." This correspondence relationship indicates that pixel values (R _3-1 , G _3-1 , B _3-1 ) are converted to colorimetric values (X _3-1 , Y _3-1 , Z _3-1 ), and pixel values (R _3-2 , G _3-2 , B _3-2 ) are converted to colorimetric values (X _3-2 , Y _3-2 , Z _3-2 ).

The camera profile 4 is associated with the identification information "2D barcode 2", the color characteristics "ink 4" and "printing method 4".
Furthermore, camera profile 4 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristics of printed matter 51 are "ink 4" and "printing method 4." This correspondence relationship indicates that pixel values (R _4-1 , G _4-1 , B _4-1 ) are converted to colorimetric values (X _4-1 , Y _4-1 , Z _4-1 ), and pixel values (R _4-2 , G _4-2 , B _4-2 ) are converted to colorimetric values (X _4-2 , Y _4-2 , Z _4-2 ).

<2-3. Flow of captured image conversion>
The flow of conversion of a captured image according to the second embodiment is similar to the flow of conversion of a captured image according to the first embodiment described above, and therefore a detailed description thereof will be omitted.

<<3. Third embodiment>>
The second embodiment of the present invention has been described above. Next, a third embodiment of the present invention will be described with reference to Figs. 7 and 8. In the third embodiment, an example will be described in which the color characteristics of a printed matter included in a captured image are identified based on identification information associated with the color characteristics of the printed matter. Note that the identification information in the third embodiment is identification information (second identification information) that is assigned to the imaging environment in which the imaging device 10 captures the printed matter 51. An example of the imaging environment is a shooting stand. Note that, in the following, descriptions that overlap with the first and second embodiments described above will be omitted.

Now, with reference to FIG. 7, an example of the identification information according to the third embodiment will be described. FIG. 7 is a diagram showing an example of the identification information according to the third embodiment. In the example shown in FIG. 7, the shooting stand 50 is divided into an area 58 and an area 59. As an example, a label 56 indicated as "printed matter storage area" is attached to the area 58 as identification information. A printed matter 51 is placed in the area 58 to which the label 56 is attached. As an example, a label 57 indicated as "sample storage area" is attached to the area 59 as identification information. A sample of the printed matter 54 is placed in the area 59 to which the label 57 is attached.

The identification information in the third embodiment associates the color characteristics of the subject with the position of the subject in the captured image. The subject includes both the printed matter 51 and the sample 54 shown in FIG. 7. Therefore, when a captured image includes multiple subjects, the image processing device 20 can distinguish between the subjects based on the identification information and identify the color characteristics corresponding to each subject.

<3-1. Configuration of image processing system>
The configuration of the image processing system according to the third embodiment is similar to the configuration of the image processing system 1 according to the first and second embodiments described above, and therefore detailed description thereof will be omitted. Hereinafter, the same reference numerals as those of the image processing system 1 according to the first and second embodiments will be used for the configuration of the image processing system according to the third embodiment.

<3-2. Configuration of image processing device>
The configuration of the image processing device 20 according to the third embodiment is similar to the configuration of the image processing device 20 according to the first and second embodiments described above. Hereinafter, the same reference numerals as those of the image processing device 20 according to the first and second embodiments will be used for the configuration of the image processing device 20 according to the third embodiment. Note that in the image processing device 20 according to the third embodiment, part of the function of the identification unit 2102 and part of the configuration of the profile table of the profile DB are different from those of the first and second embodiments. The differences from the first and second embodiments will be described below.

The identification unit 2102 of the third embodiment identifies the position and color characteristics of the subject included in the captured image based on the identification information (second identification information) included in the captured image. For example, in the example shown in FIG. 7, the label 56 is associated with the color characteristics of the printed matter 51 and the position of the printed matter 51 (e.g., the area 58 to which the label 56 is attached). This allows the image processing device 20 to recognize the label 56 included in the captured image, determine that the subject placed in the area 58 to which the label 56 is attached is the printed matter 51, and identify the color characteristics of the printed matter 51.

In the example shown in FIG. 7, the label 57 is associated with the color characteristics of the sample 54 and the position of the sample 54 (e.g., the area 59 to which the label 57 is attached). This allows the image processing device 20 to recognize the label 57 included in the captured image, determine that the subject placed in the area 59 to which the label 57 is attached is the sample 54, and identify the color characteristics of the sample 54.

In the third embodiment, a captured image including both the print 51 and the sample 54 may be divided into an image including only the print 51 and an image including only the sample 54, and then the color characteristics of each of the print 51 and the sample 54 may be identified by the above-described method. The conversion process may also be performed on each image.

Now, with reference to FIG. 8, an example of a profile stored in the profile DB will be described. FIG. 8 is a diagram showing an example of the configuration of a profile table in the profile DB according to the third embodiment. As shown in FIG. 8, a record in the profile table includes a profile name, identification information, position information, color characteristics, pixel values, and colorimetric values. Note that the information included in the record in the profile table is not limited to this example.

The position information in the profile table is information that indicates the position of the subject in the captured image. Figure 8 shows camera profile 5 and camera profile 6, which are profiles according to the characteristics of the camera, as examples of profiles.

The camera profile 5 is associated with the identification information "printed matter storage location label" (label 56), the location information "area 58", and the color characteristics "ink 5" and "printing method 5". This indicates that the color characteristics of the subject placed in "area 58" to which the "printed matter storage location label" is attached are "ink 5" and "printing method 5".
Furthermore, camera profile 5 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristics of the subject are "Ink 5" and "Printing Method 5." This correspondence relationship indicates that pixel values (R _5-1 , G _5-1 , B _5-1 ) are converted to colorimetric values (X _5-1 , Y _5-1 , Z _5-1 ), and pixel values (R _5-2 , G _5-2 , B _5-2 ) are converted to colorimetric values (X _5-2 , Y _5-2 , Z _5-2 ).

The camera profile 6 is associated with the identification information "sample location label" (label 57), the location information "area 59", and the color characteristics "ink 6" and "printing method 6". This indicates that the color characteristics of the subject placed in "area 59" to which the "sample location label" is attached are "ink 6" and "printing method 6".
Furthermore, camera profile 6 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristics of the subject are "ink 6" and "printing method 6." This correspondence relationship indicates that pixel values (R _6-1 , G _6-1 , B _6-1 ) are converted to colorimetric values (X _6-1 , Y _6-1 , Z _6-1 ), and pixel values (R _6-2 , G _6-2 , B _6-2 ) are converted to colorimetric values (X _6-2 , Y _6-2 , Z _6-2 ).

<3-3. Flow of captured image conversion>
The flow of conversion of a captured image according to the third embodiment is similar to the flow of conversion of a captured image according to the first and second embodiments described above, and therefore a detailed description thereof will be omitted.

<<4. Fourth embodiment>>
The third embodiment of the present invention has been described above. Next, a fourth embodiment of the present invention will be described with reference to Figs. 9 to 11. In the fourth embodiment, an example will be described in which the color characteristics of a subject included in a captured image are identified based on design information indicating the color characteristics of each material of the subject. When the subject is a printed matter 51, the design information is, for example, a platemaking image of the printed matter 51. Note that the design information of the subject is not limited to this example. Note that, hereinafter, descriptions that overlap with the first to third embodiments described above will be omitted.

Now, referring to FIG. 9, an example of the relationship between the captured image and design information according to the fourth embodiment will be described. FIG. 9 is a diagram showing an example of the relationship between the captured image and design information according to the fourth embodiment. FIG. 9 shows a platemaking image 70 of the printed matter 51 as an example of design information.

The platemaking image 70 shows the printing area when printing the printed matter 51. The printing area is associated with, for example, the color characteristics of the material used in the printing area when printing the printed matter 51. As an example, as shown in FIG. 9, the printing area 71a of the platemaking image 70 is associated with spot ink 1, the printing area 71b with spot ink 2, the printing area 71c with process ink 3, the printing area 71d with process ink 2, and the printing area 71e with process ink 3.

The image processing device 20 aligns the captured image 60 including the printed matter 51 with the platemaking image 70, and, assuming that the color characteristics at corresponding positions are the same, identifies the color characteristics for each material of the printed matter 51. For example, in the example shown in Fig. 9, the printing area 52a, the printing area 52b, the printing area 52c, the printing area 52d, and the printing area 52e of the printed matter 51 correspond to the printing area 71a, the printing area 71b, the printing area 71c, the printing area 71d, and the printing area 71e of the platemaking image 70, respectively.
Therefore, the image processing device 20 can identify that the color characteristics of printing area 52a, printing area 52b, printing area 52c, printing area 52d, and printing area 52e are, respectively, the color characteristics of spot ink 1, the color characteristics of spot ink 2, the color characteristics of process ink 1, the color characteristics of process ink 2, and the color characteristics of process ink 3.

4-1. Image processing system configuration
The configuration of the image processing system according to the fourth embodiment is similar to the configuration of the image processing system 1 according to the first to third embodiments described above, and therefore a detailed description thereof will be omitted. Hereinafter, the configuration of the image processing system according to the fourth embodiment will be denoted by the same reference numerals as those of the image processing system 1 according to the first to third embodiments.

<4-2. Configuration of image processing device>
The configuration of the image processing device 20 according to the fourth embodiment is similar to the configuration of the image processing device 20 according to the first to third embodiments described above. Hereinafter, the same reference numerals as those of the image processing device 20 according to the first to third embodiments are used for the configuration of the image processing device 20 according to the fourth embodiment. Note that in the image processing device 20 according to the fourth embodiment, part of the function of the identification unit 2102 and part of the configuration of the profile table of the profile DB are different from those of the first to third embodiments. The differences from the first to third embodiments will be described below.

The identification unit 2102 of the fourth embodiment identifies the color characteristics of the subject included in the captured image based on the platemaking image of the printed matter 51. For example, the identification unit 2102 aligns the printed matter 51 included in the captured image 60 with the platemaking image 70 to identify the position of the material of the printed matter 51 and the color characteristics of the material.

The identification unit 2102 identifies the color characteristics for each material of the printed matter 51, and the acquisition unit 2104 can acquire a profile for each material of the printed matter 51. In addition, the identification unit 2102 also identifies the position of each material of the printed matter 51 in the captured image 60, so the allocation unit 2106 can assign a profile with high accuracy to each material of the printed matter 51 in the captured image based on the position. This also improves the accuracy of the conversion process in the conversion unit 2108.

Now, referring to FIG. 10, an example of a profile stored in the profile DB will be described. FIG. 10 is a diagram showing an example of the configuration of a profile table in the profile DB according to the fourth embodiment. As shown in FIG. 10, the records of the profile table include a profile name, a print area, color characteristics, pixel values, and colorimetric values. Note that the information included in the records of the profile table is not limited to this example.

The position information in the profile table is information that indicates the position of each material in the platemaking image 70. Figure 10 shows camera profile 7, camera profile 8, camera profile 9, camera profile 10, and camera profile 11, which are profiles according to the characteristics of the camera, as examples of profiles.

The camera profile 7 is associated with the position information "printing area 71a" and the color characteristic "spot ink 1." This indicates that the color characteristic of the "printing area 71a" of the platemaking image 70 is "spot ink 1."
Furthermore, the camera profile 7 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristic is "spot color ink 1." This correspondence relationship indicates that pixel values (R _7-1 , G _7-1 , B _7-1 ) are converted to colorimetric values (X _7-1 , Y _7-1 , Z _7-1 ).

The position information "printing area 71b" and the color characteristic "spot ink 2" are associated with the camera profile 8. This indicates that the color characteristic of the "printing area 71b" of the plate making image 70 is "spot ink 2."
Furthermore, the camera profile 8 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristic is "spot color ink 2." This correspondence relationship indicates that pixel values (R _8-1 , G _8-1 , B _8-1 ) are converted to colorimetric values (X _8-1 , Y _8-1 , Z _8-1 ).

The position information "print area 71c" and the color characteristic "process ink 1" are associated with the camera profile 9. This indicates that the color characteristic of the "print area 71c" of the plate making image 70 is "process ink 1."
Furthermore, the camera profile 9 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristic is "process ink 1." This correspondence relationship indicates that pixel values (R _9-1 , G _9-1 , B _9-1 ) are converted to colorimetric values (X _9-1 , Y _9-1 , Z _9-1 ).

The position information "print area 71d" and the color characteristic "process ink 2" are associated with the camera profile 10. This indicates that the color characteristic of the "print area 71d" of the platemaking image 70 is "process ink 2."
Furthermore, the camera profile 10 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristic is "process ink 2." This correspondence relationship indicates that pixel values (R _10-1 , G _10-1 , B _10-1 ) are converted to colorimetric values (X _10-1 , Y _10-1 , Z _10-1 ).

The position information "print area 71e" and the color characteristic "process ink 3" are associated with the camera profile 11. This indicates that the color characteristic of the "print area 71e" of the plate making image 70 is "process ink 3."
Furthermore, the camera profile 11 is associated with a correspondence relationship between pixel values and colorimetric values when the color characteristic is "Process Ink 3." This correspondence relationship indicates that pixel values (R _11-1 , G _11-1 , B _11-1 ) are converted to colorimetric values (X _11-1 , Y _11-1 , Z _11-1 ).

<4-3. Flow of captured image conversion>
A flow of captured image conversion according to the fourth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing an example of a flow of captured image conversion according to the fourth embodiment.

As shown in FIG. 11, first, the user places the printed matter 51 on the photographing stand 50 (S202).

Next, the image processing device 20 acquires a captured image of the printed matter 51 placed on the imaging stand 50 (S204). The printed matter 51 is captured, for example, by the user operating the imaging device 10.

Next, the identification unit 2102 of the image processing device 20 aligns the captured image 60 with the platemaking image 70 (S206). Based on the alignment result, the identification unit 2102 identifies the color characteristics for each printing area of the printout 51 included in the captured image (S208).

Next, the acquisition unit 2104 of the image processing device 20 acquires a profile from the storage unit 220 based on the identified color characteristics (S210). For example, if the color characteristic is "spot color ink 1", the acquisition unit 2104 acquires camera profile 7 from the profile DB of the storage unit 220.

Next, the assignment unit 2106 of the image processing device 20 assigns the acquired profile to the captured image (S212).

Next, the conversion unit 2108 of the image processing device 20 performs conversion processing based on the assigned profile (S214).

Finally, the display device 30 displays the display image obtained by the conversion process of the conversion unit 2108 (S216).

<<5. Modifications>>
The fourth embodiment of the present invention has been described above. Next, modified examples of the present invention will be described. Note that the modified examples described below may be applied in place of the configurations described in the respective embodiments of the present invention, or may be applied in addition to the configurations described in the respective embodiments of the present invention.

In the above-described embodiments, the color characteristics of the printed matter 51 in the area included in the captured image are identified by the function of the identification unit 2102, but the present invention is not limited to such an example. For example, the color characteristics of the printed matter 51 may be identified based on machine learning. An example of an algorithm used for machine learning is semantic segmentation. Semantic segmentation is a deep learning algorithm that associates labels and categories with all pixels in an image. By using this algorithm, for example, when multiple subjects exist in an area included in the captured image, it is possible to simultaneously perform area division for each subject and identification of the material of each subject. In addition, by using this algorithm, it is possible to convert pixel values to colorimetric values with high accuracy even in captured images of clothing materials or captured images in a room where various subjects may exist.

The above describes the modified examples of the present invention.
As described above, the specifying unit 2102 of the image processing device 20 according to each embodiment specifies color characteristics of the print 51 in an area included in a captured image captured by the imaging device 10 that captures the print 51 .
The acquisition unit 2104 of the image processing device 20 acquires a profile (conversion information) indicating the correspondence between the pixel values of the captured image and the colorimetric values of the subject, based on the color characteristics identified by the identification unit 2102 .
The allocation unit 2106 of the image processing device 20 allocates the profile acquired by the acquisition unit 2104 to an area included in the captured image.
The conversion unit 2108 of the image processing device 20 converts pixel values of the captured image into colorimetric values based on a profile assigned to an area included in the captured image.

With this configuration, the image processing device 20 according to each embodiment can specify (narrow) the range of color characteristics of the subject, the printed matter 51, and then convert the pixel values of the captured image into colorimetric values. As a result, the accuracy of the conversion from pixel values to colorimetric values in the present invention is improved compared to the accuracy of conventional methods in which color conversion is applied uniformly to the entire captured image.

Therefore, the image processing device 20 according to each embodiment can convert a captured image into a colorimetric image with high accuracy.

Each embodiment of the present invention has been described above. Note that each of the above-described embodiments may be implemented as an embodiment of the present invention alone, or may be implemented in combination as one embodiment of the present invention.

The image processing device 20 in each of the above-mentioned embodiments may be realized by a computer. In that case, a program for realizing this function may be recorded in a computer-readable recording medium, and the program recorded in the recording medium may be read into a computer system and executed to realize the function. Note that the term "computer system" here includes hardware such as an OS and peripheral devices. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. Furthermore, the term "computer-readable recording medium" may include a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and a medium that holds a program for a certain period of time, such as a volatile memory inside a computer system that is a server or client in such a case. Furthermore, the above-mentioned program may be a program for realizing part of the above-mentioned function, or may be a program that can realize the above-mentioned function in combination with a program already recorded in the computer system, or may be a program that is realized using a programmable logic device such as an FPGA (Field Programmable Gate Array).

The above describes in detail an embodiment of the present invention with reference to the drawings, but the specific configuration is not limited to the above, and various design changes can be made without departing from the spirit of the present invention.

REFERENCE SIGNS LIST 1 Image processing system 10 Imaging device 20 Image processing device 30 Display device 40 Lighting 50 Photography stand 51 Printed matter 53 Identification information 54 Sample 56 Label 57 Label 60 Captured image 70 Plate making image 200 Communication unit 210 Control unit 220 Storage unit 2102 Identification unit 2104 Acquisition unit 2106 Allocation unit 2108 Conversion unit 2110 Output processing unit

Claims (10)

an identification unit that identifies color characteristics of the subject in a region included in a captured image captured by an imaging device that images the subject, based on design information indicating color characteristics for each material of the subject;
an acquisition unit that acquires conversion information indicating a correspondence relationship between pixel values of the captured image and colorimetric values of the subject based on the color characteristic identified by the identification unit;
an allocation unit that allocates the conversion information acquired by the acquisition unit to an area included in the captured image;
a conversion unit that converts the pixel values of the captured image into the colorimetric values based on the conversion information assigned to an area included in the captured image;
Equipped with
When the subject is a printed matter, the design information is a platemaking image.
The identification unit identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image. The specifying unit specifies the color characteristic for each of a plurality of regions to be converted that are included in the region of the subject,
the acquisition unit acquires the conversion information for each of the plurality of regions based on the color characteristics identified for each of the plurality of regions;
the allocating unit allocates the conversion information acquired for each of the plurality of regions to each of the plurality of regions;
the conversion unit converts the pixel values of the captured image into the colorimetric values based on the conversion information assigned to each of the plurality of regions.
The image processing device according to claim 1 . The identification unit identifies color characteristics of the subject included in the captured image based on identification information associated with the color characteristics of the subject.
The image processing device according to claim 1 . the identification information is first identification information given to the subject,
The identification unit identifies color characteristics of a subject included in the captured image based on the first identification information included in the captured image.
The image processing device according to claim 3 . the identification information is second identification information assigned to an imaging environment in which the imaging device images the subject,
The identification unit identifies color characteristics of a subject included in the captured image based on the second identification information included in the captured image.
The image processing device according to claim 3 . the identification information is further associated with a color characteristic of the object and a position of the object in the captured image;
The identification unit identifies color characteristics and a position of a subject included in the captured image based on the identification information.
The image processing device according to any one of claims 3 to 5 . the identification unit identifies a set of spectral reflectances to which the material of the subject belongs as the spectral reflectance of the material of the subject.
The image processing device according to claim 1 . An imaging unit that captures an image of a subject;
an identification unit that identifies color characteristics of the subject in a region included in a captured image captured by the imaging unit based on design information indicating color characteristics for each material of the subject;
an acquisition unit that acquires conversion information indicating a correspondence relationship between pixel values of the captured image and colorimetric values of the subject based on the color characteristic identified by the identification unit;
an allocation unit that allocates the conversion information acquired by the acquisition unit to an area included in the captured image;
a conversion unit that converts the pixel values of the captured image into the colorimetric values based on the conversion information assigned to an area included in the captured image;
a display unit that displays a display image based on the colorimetric values;
having
When the subject is a printed matter, the design information is a platemaking image.
The identification unit identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image. The identification unit identifies color characteristics of the subject in a region included in a captured image captured by an imaging device that images the subject, based on design information indicating color characteristics for each material of the subject;
an acquisition unit acquires conversion information indicating a correspondence relationship between pixel values of the captured image and colorimetric values of the subject based on the color characteristic identified by the identification unit;
an allocating unit allocating the conversion information acquired by the acquiring unit to an area included in the captured image;
a conversion unit converting the pixel values of the captured image into the colorimetric values based on the conversion information assigned to an area included in the captured image;
Including,
When the subject is a printed matter, the design information is a platemaking image.
The identification unit identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image. Computer,
an identification unit that identifies color characteristics of the subject in a region included in a captured image captured by an imaging device that images the subject, based on design information indicating color characteristics for each material of the subject;
an acquisition unit that acquires conversion information indicating a correspondence relationship between pixel values of the captured image and colorimetric values of the subject based on the color characteristic identified by the identification unit;
an allocation unit that allocates the conversion information acquired by the acquisition unit to an area included in the captured image;
a conversion unit that converts the pixel values of the captured image into the colorimetric values based on the conversion information assigned to an area included in the captured image;
Function as a
When the subject is a printed matter, the design information is a platemaking image.
The identification unit is a program that identifies a position of a material of a subject included in the captured image and color characteristics of the material by aligning the captured image with the platemaking image.

Images