JP6544004B2 - Color sample creating apparatus and color sample creating method, and image processing system using color sample - Google Patents

Description

  The present invention relates to a color sample, a color sample creating apparatus and a color sample creating method, and an image processing system using a color sample.

  A color swatch as a color chart imitates the appearance of a color on an object or the like, and a conventional color swatch is constituted by a single-color patch in which an area of uniform color is printed in a rectangular shape (for example, Patent Document 1).

  Such a color sample is originally used as a sample for coloring such as painting of a print or a color of an object, or used for checking or comparing color reproducibility after coloring on a print or an object.

  For objects that are actually colored, a check of color reproducibility is performed in real space. However, the actual color of the object is, for example, the illumination condition such as direct illumination or indirect illumination, the material of the object, the uneven shape of the surface of the object, or the position of the illumination-object-checker's viewpoint etc. It has a spatial distribution of relationships and is not necessarily uniform.

  In particular, when the printed matter is a photograph of a photographed object or a scene containing an object in real space, even if the object is uniformly colored, the color of the object is spatially nonuniform due to the photographing conditions and the like. It is usually

  Therefore, it is difficult to reproduce an impression received from the appearance of a color having a spatial distribution, which is not uniform depending on the spatial distribution condition, in a monochrome patch such as a conventional color sample.

  Under such circumstances, color samples, color sample creating apparatuses and color sample creating methods, and image processing systems using color samples are currently proposed. In the color sample, all the pixels included in a predetermined area of the original drawing (original image) are rearranged based on the lightness, saturation, and hue angle, and each pixel after rearrangement is arranged in the predetermined area in the color sample. It is arranged in the corresponding area. As such, in the color sample, it is possible to approximately reproduce the impression received from the appearance of the color under the actual environment.

  However, in the color sample, pixels of a similar color may be arranged in a dispersed manner in the pixel group in the area. When the pixels of the similar color are arranged in a dispersed manner, the pixels of the similar color are easily perceived by the observer, and the granularity is increased. The higher the granularity, the easier it is for the viewer to feel granular. Since graininess (graininess) is perceived as noise for the observer, it is desirable to suppress graininess.

  The present invention has been made to solve the problems as described above, and it is possible to approximately reproduce an impression received from the appearance of color in an actual environment, and to create a color sample and a color sample with further reduced granularity. An apparatus, a color sample creation method, and an image processing system using the color sample.

  A color sample according to the present invention includes a second area in which a second pixel group generated from a first pixel group constituting a first area in a first image is arranged, and a second pixel group For each pixel constituting the first pixel group, the first rearrangement is performed based on the lightness, and for each pixel after the first rearrangement, the first arrangement is performed. Assuming a row in the second region, a second rearrangement is performed for each row based on the hue angle, and for each pixel after the second rearrangement, the arrangement is based on a color A third sort to replace is performed and arranged in the second area.

  According to the present invention, it is possible to approximately reproduce an impression received from the appearance of color in an actual environment, and to use a color sample, a color sample creating apparatus and a color sample creating method, and a color sample with further reduced granularity. An image processing system can be provided.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the outline of the image processing system which concerns on embodiment of this invention. It is a block diagram showing an example of composition of an image processing device in an image processing system concerning an embodiment of the invention. It is a block diagram showing an example of functional composition of an image processing system concerning an embodiment of the invention. It is a figure showing an example of case information used for modification processing in an image processing system concerning an embodiment of the invention. It is a figure shown about the retouching process by the standard type of the image in the image processing system which concerns on embodiment of this invention. It is a figure shown about the correction process by the transfer type of the image in the image processing system which concerns on embodiment of this invention. It is a figure explaining a color sample example. It is a figure which shows the 1st example of an image used for creation of a color sample. FIG. 6 is a diagram for explaining an example of a first pixel group copied to a RAM 23; It is a figure which shows the example of a state of the 1st pixel group after performing 1st rearrangement. It is a figure which shows the example of a state of the 1st pixel group after performing 2nd rearrangement. It is a figure explaining the example of a method of adding the pixel to the 1st pixel group after performing the 2nd rearrangement. It is a figure explaining the example of the 2nd pixel group in front of performing 3rd rearrangement. It is a figure explaining the example of the 2nd pixel group after performing the 3rd rearrangement. It is a figure which illustrates the example of the 2nd pixel group after performing the 3rd rearrangement more extensively. It is a figure explaining the example of the 2nd pixel group after performing the 4th rearrangement. It is a figure which shows the created example of a color sample. It is a flowchart which shows the preparation procedure of the color sample image which concerns on the 1st Example of this invention. It is a figure explaining the restriction | limiting method of the movable range of a pixel. It is a flowchart which shows the preparation procedure of the color sample image which concerns on the 2nd Example of this invention.

  Hereinafter, an embodiment according to the present invention will be described with reference to the drawings.

  FIG. 1 shows an exemplary configuration of an image processing system according to an embodiment of the present invention.

  As shown in FIG. 1, the image processing system 100 includes, for example, an image processing device 2 and a server device 3. In the image processing system 100, the image processing device 2 and the server device 3 are connected to each other via the network 4.

  The image processing apparatus 2 is an information processing apparatus capable of performing a retouching process of an input image (image before retouching) to be a retouching object such as a photograph by using a color sample (second image) described later. It is. The information processing apparatus is, for example, a personal computer (PC) or a tablet terminal.

  Further, the image processing apparatus 2 inputs an original image (hereinafter referred to as “first image” because it corresponds to the first image) which is an image used to create a color sample (second image). And, a color sample can be created from the input first image. The color sample creation device according to the present embodiment is realized in the form of being mounted on the image processing device 2.

  The server device 3 is an information processing device that can store a first image and a color sample created using the first image as image data. Although only one server device 3 is shown here, a plurality of server devices 3 may exist. That is, the number of server devices 3 is not particularly limited. The number of image processing apparatuses 2 is not limited to one.

  The network 4 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, or a network group including two or more of them. The network 4 may be a cable that directly connects the image processing device 2 and the server device 3. The image processing device 2 and the server device 3 may be connected wirelessly.

  In the image processing system 100 configured as described above, the image processing device 2 can receive various data including an image (image data) from the server device 3 via the network 4, and transmits various data to the server device 3. can do. In the present embodiment, it is assumed that the server device 3 functions as a database server device that supplies various data required by the image processing device 2 and stores various data generated by the image processing device 2.

  FIG. 2 is a block diagram showing a configuration example of the image processing apparatus 2. The configuration example shown in FIG. 2 is an example, and the configuration of the image processing apparatus 2 is not limited to the configuration example shown in FIG.

  As shown in FIG. 2, the image processing apparatus 2 includes a CPU 21, a flash memory 22, a RAM 23, a display device 24, an input device 25, two or more I / Fs (Interfaces) 26 and 27, and parts 21 to 27. It has a bus 28 to be connected.

  Here, the flash memory 22 is described as including storage such as a solid state drive (SSD). The flash memory 22 stores various programs and various data. The program refers to a program (hereinafter referred to as “color sample creation program”) 22a that enables creation of a color sample and an image to be modified (hereinafter referred to as “pre-modification image”) with reference to the color sample. A program (hereinafter, referred to as an “image modifying program”) 22b that enables modification and generation of a modified image which is a pre-repairing image after the modification is included.

  The RAM 23 is mainly used as a work for the CPU 21 to execute various programs. The various programs stored in the flash memory 22 are read out to the RAM 23 and executed.

  The display device 24 is, for example, a liquid crystal display device. The display device 24 displays various data under the control of the CPU 21.

  The input device 25 is an input device group including, for example, a keyboard and a pointing device. Various instructions from the user and various data inputs can be performed through an operation on the input device 25.

  The I / F 26 enables communication via the network 4. Thereby, the image processing apparatus 2 communicates with the server apparatus 3 via the I / F 26.

  The I / F 27 is, for example, a connection device that enables connection with an external device. The external device is, for example, a storage device such as a medium drive device for driving a recording medium such as a scanner or an optical disk, a hard disk drive, or a semiconductor memory (for example, a USB (Universal Serial Bus) memory).

  The I / Fs 26 and 27 can be used for input and output of various data including images. However, here, for convenience of explanation, it is assumed that only the server device 3 is an input destination and an output destination of various data including an image.

  Similarly to the image processing device 2, the configuration of the server device 3 is not limited. From this, for convenience, it is assumed that the server device 3 is also the configuration example shown in FIG. 2, and the reference numerals of the respective parts use the same ones as the image processing device 2.

  FIG. 3 is a block diagram showing an example of the functional configuration of the image processing system according to the embodiment of the present invention. Here, the functional configuration examples of the image processing device 2 and the server device 3 are shown separately.

  As shown in FIG. 3, the image processing apparatus 2 is equipped with a color sample creation device 210 and an image modification device 230.

  The color sample creating apparatus 210 is a color sample creating apparatus according to the present embodiment that can create a color sample with reference to the first image. The color sample creating apparatus 210 is realized by the CPU 21 executing the color sample creating program 22 a stored in the flash memory 22.

  On the other hand, the image modifying device 230 is a device capable of modifying an uncorrected image with reference to a color sample and creating a modified image obtained by the modification. The image modifying device 230 is realized by the CPU 21 executing the image modifying program 22 b stored in the flash memory 22.

  First, the image modifying device 230 will be described in detail. As shown in FIG. 3, the image modifying device 230 includes, as functional components, a data acquisition unit 231, an image input unit 232, a case selection unit 233, a modification unit 234, and a data output unit 235.

  The data acquisition unit 231 is a function of acquiring necessary data from the server device 3 via the network 4. Data is acquired from the server device 3 under the control of the CPU 21 that executes the image modification program 22 b stored in the flash memory 22. From this, the data acquisition unit 231 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the I / F 26, and the bus 28.

  The various data acquired from the server device 3 include a color sample, an image before modification, and case information.

  The case information is information for specifying modification content for the pre-repair image, and includes modification reference information and case identification information. The modification time reference information is information for specifying a modification method, and the case identification information is information representing a case to be referred to in performing modification.

  FIG. 4 is a diagram for explaining a configuration example of case information.

  Examples of modifying an uncorrected image with reference to a color sample are roughly classified into a standard type that refers to two color samples and a transfer type that refers to one color sample. FIG. 4A shows standard type case information, and FIG. 4B shows transfer type case information.

  The case identification information includes type information, case ID (Identifier) information, image A information, image A tone information, image B information, image B tone information, impression representation information, selected area shape information, and selected area information. .

  Type information is information indicating the type of case, that is, whether the case is a standard type or a transcription type. Thus, as the contents of the information, “standard type” is shown in FIG. 4A and “transfer type” is described in FIG. 4B.

  The case ID information is information uniquely representing a case. As the contents of the information, “0001” is shown in FIG. 4A, and “0001-2” is shown in FIG. 4B.

  The image A information is information representing a color sample (hereinafter also referred to as “image A”) to be referred to as a case image before modification. As content of the information, “img0001.jpg” representing a file name is described in FIG. In the transfer type, since the image A is not referred to, “NULL” is written in FIG. 4B to indicate that no information exists.

  The tone information of the image A is information representing the tone of the entire image A. In this embodiment, it is assumed that each pixel has data of three elements of L * (Elster), a * (Aester), and b * (Bearster) color system. From this, in FIG. 4A, “(50, 12, 15)” representing the average Lab value is described as the content of the information. The type of data representing a pixel is not particularly limited.

  Image B information is information representing a color sample (hereinafter also referred to as “image B”) to be referred to as a case image after modification. In FIG. 4A and FIG. 4B, “img0002.jpg” is described as the contents of the information. In order to distinguish the image B between the standard type and the transfer type, the image B in the transfer type is also referred to as a "reference image".

  The tone information of the image B is information representing the tone of the entire image B. As the contents of the information, “(34, 11, 14)” is described in FIGS. 4 (a) and 4 (b).

  The impression expression information is information representing an impression that the observer receives from the images A and B or the image B. In FIG. 4A and FIG. 4B, “dark and smooth” is described as the contents of the information.

  The selected area shape information is information representing the shape of the area (second area) to be referred to in the images A and B or the image B. As the contents of the information, “circle” is described in FIG. 4 (a) and FIG. 4 (b).

  The selected area information is information representing an area to be referred to in the images A and B or the image B. As the contents of the information, “center (50, 50), radius 50” is described in FIGS. 4 (a) and 4 (b).

  The selected area shape information and the selected area information whose contents are described in FIG. 4A and FIG. 4B are assumed to be a color sample as shown in FIG. There are a plurality of rows in which pixels are arranged in the selection area (second area), and a pixel group (second pixel group) arranged in the selection area is divided in line units.

  The modification reference information includes image processing operator information and image processing parameter information.

  The image processing operator information is information representing an operation method for modification. As the contents of the information, “tone curve (L tone curve)” is shown in FIG. 4A and “transfer operator” is shown in FIG. 4B.

  The tone curve specifies an operation content for modifying the entire image. Image parameter information is information used to perform an operation specified by the image processing operator information. From this, in the standard type, control point information used for creating a tone curve is registered as image parameter information.

  The transfer operator designates an operation for correcting the uncorrected image toward the image B, with the image B (reference image) as a target. From this, in the transfer type, the image parameter information is "NULL (see image B)".

  The configuration example of the case information illustrated in FIG. 4 is one example, and the configuration of the case information is not particularly limited. The method of describing the case information is also not limited. The case information may be described using, for example, XML (Extensible Markup Language).

  Here, it is assumed that the server device 3 stores a plurality of pieces of case information as shown in FIG. 4A or 4B and stores a plurality of uncorrected images.

  In the above assumption, the image input unit 232 is a function of acquiring the specified uncorrected image from the server device 3 in accordance with the user's instruction. The user's instruction is performed by operating the input device 25, and the pre-modification image is acquired from the server device 3. From this, the image input unit 232 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the input device 25, the I / F 26, and the bus 28.

  The case selection unit 233 is a function that presents, for example, case information that can be acquired from the server device 3 to the user and causes the user to select desired case information. Acquisition of data from the server device 3 by the data acquisition unit 231 is controlled by the case selection unit 233. The case selection unit 233 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the display device 24, and the bus 28.

  The modifying unit 234 is a function of modifying the specified uncorrected image with reference to the case information selected by the user. The modification unit 234 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, and the bus 28.

  The image retouching performed by the retouching unit 234 will be specifically described with reference to FIGS. 5 and 6.

  FIG. 5 is a diagram for explaining an example of a method of correcting an image before correction by standard type case information.

  In the modification based on the standard type case information, the image A (FIG. 5 (a)) and the image B (FIG. 5 (b)) specified by the case information are used. The tone curve (FIG. 5C) specified by the image processing operator information is created to correct the tone of the image A toward the image B. The uncorrected image (FIG. 5 (d), referred to as "input image" in this case) is converted using the created tone curve to generate a corrected image (FIG. 5 (e)). In the example shown in FIG. 5, the skin of a person existing as an object in a pre-repairing image (for example, a photo), the clothes of the person, the sky and mountains as the background of the person, and the like are modified.

  FIG. 6 is a view for explaining an example of a method of correcting an image before correction based on transfer type case information.

  In the modification based on the transfer type case information, only the image B (FIG. 6A) specified by the case information is used. In this transfer type, an uncorrected image (FIG. 6 (b), here referred to as “input image”) is corrected using a known method such as color transfer to create a corrected image (FIG. 6 (c)). . Also in the example shown in FIG. 6, the skin of a person existing as an object in the pre-repairing image (for example, a photo), the clothes of the person, the sky and mountains as the background of the person, and the like are modified.

  The data output unit 235 illustrated in FIG. 3 is a function that can output a modified image. As a specific output destination of the modified image, in addition to the server device 3, there is a display device 24 shown in FIG. From this, the data output unit 235 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the display device 24, the I / F 26, and the bus 28.

  Here, a functional configuration example of the server device 3 connected to the image processing device 2 will be specifically described.

  As illustrated in FIG. 3, the server device 3 includes a control unit 31, a communication unit 32, a first storage unit 33, and a second storage unit 34 as functional components.

  The communication unit 32 enables communication via the network 4. Communication via the network 4 is enabled by the CPU 21 executing a network OS (Operating System) stored in the flash memory 22 or an application program for providing a service. From this, the communication unit 32 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the I / F 26, and the bus 28.

  The first storage unit 33 is used, for example, for storing an uncorrected image, a first image, case information, a corrected image, and the like. The first storage unit 33 corresponds to, for example, the flash memory 22 and the RAM 23.

  The second storage unit 34 is used to store, for example, a color sample used as the image A or the image B. The second storage unit 34 also corresponds to, for example, the flash memory 22 and the RAM 23.

  The control unit 31 controls the entire server device 3. This control is realized by the CPU 21 executing a network operating system (OS) stored in the flash memory 22 or an application program for providing a service. From this, the control unit 31 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, and the bus 28.

  The image processing system 100 according to the present embodiment is realized by the image modifying device 230 of the image processing device 2 and the server device 3.

  Assuming that the image modifying device 230 inputs an uncorrected image from the server device 3 and modifies the uncorrected image, the image processing system 100 causes the server device 3 to save the uncorrected image to be corrected. That means that you have acquired the pre-retouched image. The acquired uncorrected image is stored in the first storage unit 33. From such a thing, the image acquisition part of the image processing system 100 is implement | achieved by the control part 31, the communication part 32, and the 1st memory | storage part 33, for example.

  The color sample used as the image A or the image B is stored in the second storage unit 34, and is transmitted to the image processing apparatus 2 as needed. From this, the image storage means of the image processing system 100 includes at least the second storage unit 34.

  The image modification device 230 performs the modification on the pre-repaired image. The image modifying device 230 acquires the pre-repairing image and various data necessary for the modification from the server 3 in order to modify the pre-repairing image. Because of this, the modification unit of the image processing system 100 includes, for example, the control unit 31 of the server device 3, the communication unit 32 and the first storage unit 33, and the data acquisition unit 231 of the image modification unit 230 and the image input unit 232. And the modification unit 234.

  The image modifying device 230 receives various necessary data including the pre-repairing image and the color sample (the image A and the image B or the image B) from the server device 3 and reconstitutes the received pre-repairing image. The image modifying device 230 temporarily stores the data received from the server device 3 even if it is temporary. From this, the image modifying device 230 itself also functions as one image processing system 100. This means that the image processing system 100 can be realized by only one information processing apparatus.

  When the image modifying device 230 is regarded as one image processing system, the image acquisition unit corresponds to, for example, the image input unit 232. The image storage unit corresponds to, for example, a data acquisition unit 231 and a modification unit. The modification means is realized by, for example, the data acquisition unit 231, the case selection unit 233, and the modification unit 234.

  Next, the color sample creation device 210 will be described in detail. As shown in FIG. 3, the color sample creation device 210 includes, as functional components, a data acquisition unit 211, a layout determination unit 212, a color sample creation unit 213, and a registration unit 214.

  The data acquisition unit 211 is a function of acquiring necessary data including the first image from the server device 3 via the network 4. Necessary data is acquired from the server device 3 under the control of the CPU 21 that executes the color sample creation program 22a stored in the flash memory 22. A user who visually recognizes the information displayed on the display device 24 operates the input device 25 to designate the first image acquired from the server device 3. From this, the data acquisition unit 211 is a function corresponding to an image acquisition unit, and is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the display device 24, the input device 25, the I / F 26, and the bus 28.

  The arrangement determination unit 212 is a function of determining a pixel group (second pixel group) to be arranged in the second area of the color sample, with reference to the first image specified by the user. The arrangement determination unit 212 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, and the bus 28.

  The arrangement determination unit 212 includes a first sorting unit 221, a second sorting unit 222, and a third sorting unit 223. The first sorting unit 221, the second sorting unit 222, and the third sorting unit 223 are respectively a first sorting unit, a second sorting unit, and a third sorting unit. It corresponds to

  In the first image, a first area as shown in FIG. 8 is set in advance or set by the user. This first area is an area to be referred to when creating a second pixel group arranged in the second area of the color sample.

  The arrangement determining unit 212 copies the pixel group (first pixel group) constituting the first area to the RAM 23.

  FIG. 9 is a diagram for explaining the first pixel group copied to the RAM 23. In FIG. 9, the RAM 23 is described as "memory".

  The rectangular frames shown in FIG. 9 each represent a pixel. Thus, in FIG. 9, the first pixel group copied to the RAM 23 is represented by an array of pixels in one column.

  The first reordering unit 221 performs a first reordering that rearranges the copied first pixel group based on the lightness (in this case, L * value here). By this rearrangement, the first pixel group shown in FIG. 9 is changed as shown in FIG. In the present embodiment, the first rearrangement based on the lightness is performed in such a manner that the pixels are rearranged in the descending order of the lightness. The first rearrangement may be performed in reverse.

  In FIG. 9, the lightness is expressed by the color depth. The lighter the color, the higher the lightness.

  There may be a plurality of pixels having the same lightness in the first region. In that case, as shown in FIG. 10, in the first pixel group after the first rearrangement, two or more pixels having the same lightness and one or more continuous portions exist. . In this embodiment, in this case, the first rearrangement unit 221 sets saturation (= √ (a * × a * + b * × b *)), which is an attribute based on human color perception different from lightness. A second rearrangement (corresponding to the “fourth rearrangement” described in the claims) is performed to rearrange the basis.

  In the present embodiment, the second rearrangement based on saturation is performed in the form of rearrangement in descending order of saturation. The object to be subjected to the second rearrangement is only a portion where a plurality of pixels having the same lightness are continuous. By performing such second rearrangement, the first pixel group is changed from the state shown in FIG. 10 to the state shown in FIG. If a plurality of pixels having the same lightness do not exist, the second rearrangement is not performed.

  Even when the first rearrangement and the second rearrangement are performed, the number of pixels in the first pixel group does not change. If the number of pixels in the first pixel group is smaller than the number of pixels in the second pixel group, it is necessary to add the missing pixels to the first pixel group. In this embodiment, the addition (generation) of the pixels is performed using linear interpolation. This addition of pixels is also performed by the first rearrangement unit 221. By the addition of this pixel, the first pixel group after the second rearrangement is changed from the state shown in FIG. 11, for example, to the state shown in FIG. 12, and has the number of pixels of the second pixel group. become.

  The addition of the pixel is the same as the N-th pixel from the head of the first pixel group after the first rearrangement or the second rearrangement after the n-th pixel from the head of the second pixel group It is carried out by linear interpolation to achieve color values. When the number of pixels in the first pixel group is N1 and the number of pixels in the second pixel group is N2, N corresponding to n is obtained, for example, by N = N1 × (n / N2).

When N is an integer, the pixel value V, which is data of the pixel to be added as the n-th pixel, is the pixel value V of the N-th pixel in the first pixel group N after rearrangement Interpolate. When N is not an integer, the pixel value V uses the pixel value Va of the ath pixel which is the largest integer not exceeding N and the pixel value Vb of the bth pixel which is the smallest integer not less than N V = Va + (Vb-Va) x (N-a) (1)
It asks by.

  Each pixel on which the linear interpolation as described above is performed has data of three elements of color systems of L * (Elster), a * (Aester), and b * (Bearster). Therefore, the calculation of the pixel value V is performed for all of the lightness L * value, the chromaticness index a * value and the b * value.

  Thus, in the present embodiment, when the number of pixels in the first pixel group is smaller than the number of pixels in the second pixel group, the necessary number of pixels is increased (added). For this reason, it is possible to create a color sample in which all the pixels are arranged in the second region without being limited to the number of pixels in the first pixel group. Note that the method of increasing the pixel may use a method other than linear interpolation. The number of pixels to be increased may be a number such that the number of pixels in the first pixel group is equal to or greater than the number of pixels in the second pixel group.

  The interpolation as described above can be applied even when the number of pixels in the first pixel group is larger than the number of pixels in the second pixel group. In this case, since there are pixels that can not be arranged in the second area, it is not necessary to perform interpolation.

  As described above, the first rearrangement unit 221 generates a first pixel group having the same number of pixels as the number of pixels in the second pixel group. The first pixel group is output as a second pixel group to perform the subsequent operation.

  The second reordering unit 222 performs a third reordering process for reordering the pixels based on the hue angle with respect to the second pixel group output by the first reordering unit 221 (see the claims. Equivalent to the second sort).

  The third rearrangement is performed row by row, assuming a row in which pixels are arranged in the second region, in order to consider the positional relationship of the pixels actually arranged in the second region.

  FIG. 13 is a diagram for explaining an example of a second pixel group immediately before performing the third rearrangement.

  The numerical values “8”, “14” and “16” shown in FIG. 13 represent, for example, the pixel numbers assigned to the respective pixels constituting the second pixel group. A frame (pixel) in which any numerical value is written in the vicinity is a pixel whose color is similar or whose lightness, saturation, and hue angle are all similar, that is, each pixel within a predetermined allowable range It is a pixel whose value or the value calculated from each pixel value matches. Here, in order to facilitate understanding, the color of the frame (pixel) in which any numerical value is described in the vicinity is made different from the color of the other frame (pixel).

  In the second pixel group generated by the first rearrangement unit 221, the pixels are rearranged in accordance with the lightness (in some cases, saturation). However, when the second pixel group is simply disposed in the second region, as shown in FIG. 13, there is a possibility that pixels with similar hue angles are spaced apart.

  A pixel that is so spaced away is relatively likely to appear as a color that is significantly different than its surrounding pixels. For example, if the color of the pixel placed apart is red (or green) and the color of the surrounding pixels is green (or red), the hue angle will be 180 degrees or an angle close to 180 degrees. Because of this, when pixels with approximate hue angles are separated, it is easy to give the viewer a granular feeling. Therefore, in the present embodiment, the third rearrangement based on the hue angle is performed to further suppress giving the graininess to the observer.

  FIG. 14 is a diagram for explaining an example of a second pixel group after the third rearrangement.

  As shown in FIG. 14, by performing the third rearrangement, the pixels arranged at the position where the pixel number is 8 in FIG. 13 are moved to the positions where the pixel number is 7, and the pixel number is 7 It becomes a pixel. Similarly, in FIG. 13, the pixels arranged at positions where the pixel numbers are 14 and 16 move to positions where the pixel numbers are 16 and 17 and become pixel numbers 16 and 17 respectively. In this way, pixels close in hue angle are collected in the vicinity, that is, within a narrow range, so that the degree of graininess is not given to the observer, or even if graininess is given, the degree is further suppressed become. Therefore, the created color sample also has less noise and feels more natural.

  By the third rearrangement, pixels adjacent in hue angle can be collected in adjacent rows. However, pixels whose colors are similar or whose lightness, saturation, and hue angle are all similar (each pixel value within the allowable range, or a pixel whose value calculated from each pixel value is the same. Note that “approximated pixels” may exist in distant rows. The possibility is particularly high when the ratio of the number of pixels for each hue angle in the pixel group of each row is not within a narrow range.

  FIG. 15 is a diagram for more extensively describing the second pixel group example after the third rearrangement. In FIG. 15 as well as in FIGS. 13 and 14, the approximate pixel of interest is expressed by making the color different from that of the other pixels. Thus, in FIG. 15, it is represented that the approximate pixel having the pixel number 43 is disposed at a position distant from the group including the three approximate pixels having the pixel numbers 7, 16, and 17.

  Even if the approximate pixels are arranged at distant positions in the overlapping direction of the rows, it is easy to give the observer a granular feeling for the same reason as described above. Because of this, in the present embodiment, a fourth rearrangement is performed in which the pixels are rearranged based on the color with respect to the second pixel group after the third rearrangement. (Corresponding to the third sort) to further suppress giving the sense of graininess to the observer. As a result, by performing the fourth rearrangement, the color sample created is more natural.

  The third rearrangement unit 223 performs the fourth rearrangement on the second pixel group after the third rearrangement. This rearrangement is performed in such a manner as to concentrate pixels arranged at distant positions in the overlapping direction of the rows.

  FIG. 16 is a diagram for explaining an example of a second pixel group after the fourth rearrangement.

  As shown in FIG. 16, by performing the fourth rearrangement, the approximate pixels arranged at the position where the pixel number is 43 in FIG. 15 are moved to the positions where the pixel number is 29 and the pixel number is 29 The pixel of In FIG. 15, the pixel arranged at the position where the pixel number is 29 moves to the position where the pixel number is 43 and becomes a pixel whose pixel number is 43. With such replacement, more color-approximating pixels are arranged within a narrow range, so it is possible to create a color sample that does not give a more granular feeling (noise).

  By performing the fourth rearrangement, the arrangement of all the pixels in the second region is determined. The creation unit 213 arranges pixels in an area other than the second area to create a color sample. From this, the creation unit 213 corresponds to an image creation unit. The creation unit 213 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, and the bus 28.

  FIG. 17 is a diagram showing an example of a created color sample.

  As described above, the first rearrangement based on the lightness is performed in such a manner that the pixels are rearranged in the descending order of the lightness. Thereby, as shown in FIG. 17, the pixels with high lightness are arranged in the upper part of the second region. This is because in general space, objects that become light sources such as the sun or a light are often present at the top of objects that are not light sources, and thus the upper part is brighter in many images. . In other words, by arranging the pixels in the second region in accordance with the features common to many images, it is to create a color sample that gives a more natural impression.

  The first pixel group organized by rearrangement based on lightness (and saturation) is placed at an undesirable position in the first pixel group due to human visual characteristics, paying attention to the hue angle and color. Rearrangement of pixels is performed. The relocation moves the pixel whose position is not desirable and moves one or more pixels according to the movement (FIG. 14) or replaces it with the pixel to which the pixel is moved (FIG. 16). By performing such rearrangement, in the second area of the color sample created, the second pixel group is organized so that the pixels are aligned so as to make the human visual characteristics feel extremely natural. It will be arranged more reliably.

  In addition, when the pixel arrange | positioned to area | regions other than a 2nd area | region is determined beforehand, preparation part 213 can be made unnecessary. In this case, the third rearrangement unit 223 or the arrangement determination unit 212 including the third rearrangement unit 223 creates a color sample.

  The registration unit 214 is a function that transmits the color sample created by the creation unit 213 and causes the server device 3 to register (store) the color sample. When registration of a color sample is performed according to a user's instruction, the registration unit 214 is realized by, for example, the CPU 21, the flash memory 22, the RAM 23, the display device 24, the input device 25, the I / F 26, and the bus 28.

  The image modifying device 230 mounted on the image processing device 2 modifies the uncorrected image as the color sample created as described above as the images A and B or as a reference image (image B). The created color sample more closely reproduces the impression on the color of the subject when viewed under the actual environment. Therefore, the image modifying device 230 creates a modified image in which the color of the object on the uncorrected image is modified to the actual color of the impression received by the observer who directly sees the object or a color close to the actual color. be able to.

  Below, the Example which produces a color sample using a 1st image is described concretely.

(First embodiment)
FIG. 18 is a flowchart showing the procedure of creating a color sample image according to the first embodiment. The creation procedure whose flowchart is shown in FIG. 18 is realized by the CPU 21 executing the color sample creation program 22a. In FIG. 18, for convenience, after the user designates a first image and instructs creation of a color sample, the CPU 21 determines the arrangement of pixels in a second region included in the color sample to be created. Only the control procedure to be executed is shown. Thus, the arrangement determining unit 212 is realized by the CPU 21 executing the control procedure shown in FIG. In FIG. 18, the RAM 23 is described as "memory".

  First, the CPU 21 copies the pixels present in the first area in the first image received from the server device 3 to the area secured in the RAM 23 (hereinafter referred to as “first area”) (step S1). ).

  Next, the CPU 21 performs first rearrangement on the copied pixels, that is, the first pixel group (step S2). By this first rearrangement, the first pixel group copied as shown in FIG. 9 is changed as shown in FIG.

  Next, the CPU 21 confirms a location where pixels of equal brightness continue in the first pixel group after the first rearrangement, and if the location is confirmed, the pixel at the location Are rearranged in the order of saturation (step S3). By this second rearrangement, the first pixel group as shown in FIG. 10 is changed as shown in FIG.

  Thereafter, the CPU 21 secures in the RAM 23 an area (hereinafter referred to as a “second area”) for storing the second pixel group arranged in the second area (step S4).

  Next, when the number of pixels in the first pixel group is smaller than the number of pixels in the second pixel group, the CPU 21 adds a missing pixel as shown in FIG. 12 using linear interpolation. The pixel value of the added pixel is determined (step S5). Thereby, the second pixel group as shown in FIG. 13 is stored in the second area.

  The above steps S2 to S5 are procedures relating to the first rearrangement unit 221. When realizing the function of the first rearrangement unit 221, the CPU 21 executes the procedures of steps S2 to S5.

  Next, the CPU 21 performs a third rearrangement in which the pixels are rearranged in the hue angle order in row units existing in the second area (step S6). Thus, the second pixel group as shown in FIG. 13 is changed as shown in FIG. 14 or FIG. This step S6 is a procedure related to the second rearrangement unit 222.

  Next, the CPU 21 performs fourth rearrangement on the second pixel group so as to closely arrange pixels of approximate colors dispersed in the overlapping direction of the rows (step S7). Thereby, the second pixel group as shown in FIG. 15 is changed as shown in FIG. Thereafter, the control procedure shown in FIG. 18 ends. This step S7 is a procedure relating to the third sorting unit 223.

  The fourth rearrangement in step S7 may be performed as necessary as described above. As an index used to automatically determine the necessity of performing the fourth rearrangement, for example, granularity can be mentioned.

  For calculation of the granularity, the power spectrum of L *, a *, b * obtained by FFT (fast Fourier transform) of the second region for each value of L *, a *, b * of the pixel is According to the spatial frequency characteristics (contrast sensitivity characteristics (CSF)) of color vision of the eye, methods such as taking linear sums can be used (for example, K. Kagitani, H. Makoto, AND S. Imakawa, "Image noise evaluation method for color hardcopy ", in Proc. IS & T NIP12, 1996, pp. 173-76.). The determination as to whether or not to perform the fourth rearrangement may be made based on whether the calculated granularity value is equal to or more than a predetermined value. The predetermined value may be, for example, 0.1 if the granularity is calculated by the method described in the example.

  The user may select whether or not to perform the fourth rearrangement. The selection by the user may be performed visually by causing the display device 24 to display a color sample in which the second pixel group after the third rearrangement is arranged in the second region.

  The fourth permutation itself may be performed by self-organizing maps (SOM). As a method of self-organization, an algorithm proposed by T. Kohonen is known (for example, T. Kohonen, "Self-Organizing Maps", Springer, 2000, etc.).

  The self-organizing mapping for the fourth rearrangement may be performed, for example, by the following procedure.

  First, as procedure 1, a color sample in which the second rearranged second pixel group after the third rearrangement is arranged in the second area is created in an area secured in the RAM 23 (hereinafter referred to as “third area”). Hereinafter, the created color sample is referred to as “third image”, and the second region in the third image and the second pixel group arranged in the second region It is written as “third pixel group”.

  Next, as procedure 2, the color difference between the pixel number of the second region (second pixel group) and the pixel with the pixel number i (hereinafter referred to as “pixel i”) is minimum, and the pixel i in the third region is The pixel I closest in positional relationship with the pixel I is searched. The positional relationship here is one in the case where the second area and the third area are evaluated assuming the same space.

  Next, as procedure 3, each value of L *, a *, b * of the pixel I and each pixel J in the vicinity of the pixel I in the third region is changed.

  The above procedures 2 and 3 are performed on all the pixels i in the second area, and this is repeated for a predetermined maximum number of repetitions t.

The values L (t + 1), a (t + 1), and L (t) to be set next to L *, a *, b * after changing t times L (t), a (t), and b (t) b (t + 1) may be calculated, for example, as follows.
L (t + 1) = L (t) + h (t) · dL (J, i) (2)
a (t + 1) = a (t) + h (t) · da (J, i) (3)
b (t + 1) = b (t) + h (t) · db (J, i) (4)
However, dL (J, i), da (J, i), and db (J, i) are the L * value and the a * value between the pixel i in the second area and the pixel J in the third area, respectively. , B * value difference, h (t) is a function of the distance between the pixel I and the pixel J.

h (t) is, for example, the following function.
h (t) = α (t) · exp (d ² / (2 · (σ (t) ² )) (5)
Here, α (t) is a monotonically decreasing function with respect to the number of repetitions t, d is the distance between the pixel I and the pixel J, and σ (t) is a constant at each number of repetitions. α (t) is a function such as α (t) = 1−t / (T + 1), for example.

  As the differences dL (J, i), da (J, i), and db (J, i), for example, a CIE 1976 color difference expression represented by Euclidean distance between two points in CIE (1976) L * a * b * space The value calculated by can be used. A value calculated by another color difference formula (such as CIE 1994 color difference formula) may be used.

  In Kohonen's self-organizing mapping, it is common to first secure another pixel group having the same number of pixels as the second pixel group, and assign the initial value of each secured pixel by random numbers. However, the first to third permutations are all means for organizing the pixel group. From this, in the fourth rearrangement by the self-organizing mapping, the procedure as described above is adopted so that the organized state of the second pixel group can be more maintained.

Second Embodiment
In the first embodiment, the fourth rearrangement is performed with the entire second region as the movable range of the pixels. In the second embodiment, the movable range of the pixel is limited.

  FIG. 19 is a diagram for explaining a method of limiting the movable range of the pixel.

  In the second embodiment, as shown in FIG. 19, the second area is virtually divided into a plurality of partial areas, and the movement of the pixel is limited to the partial area in which the pixel exists. Thus, the fourth rearrangement is performed for each partial area.

  When the second area is divided into a plurality of partial areas, the number of pixels in each partial area is reduced, so that the search for the movement destination of the pixel to be moved can be performed at high speed. Even if the fourth rearrangement is performed on all the partial areas, the processing can be performed at high speed as compared to the case where the fourth rearrangement is performed on the entire second area.

  The second area is divided into a plurality of partial areas because, when the third rearrangement is performed, even if the pixels of the similar color are dispersed, those pixels exist within a relatively narrow area. It is because it is often done. From this, even if the fourth rearrangement is performed for each partial area, the pixel to be moved can be appropriately moved.

  When the second area is divided into a plurality of partial areas, the fourth rearrangement determines the necessity for each partial area from the viewpoint of further speeding up of the process, and the judgment is made individually according to the judgment result. It is desirable to do. The automatic determination as to whether or not to perform the fourth rearrangement on the second region is, for example, calculating the granularity for each partial region, and calculating the average value of the calculated granularity or a statistical value such as the maximum value Is further calculated, and may be performed using the calculated statistic. The determination may be made by the user.

  The division of the second area may be performed according to the setting by the user. Also, the degree of change in color of the entire second area is checked, and the size or shape of the partial area is automatically set according to the check result to divide the second area. good.

  FIG. 20 is a flowchart showing the procedure of creating a color sample image according to the second embodiment. The creation procedure whose flowchart is shown in FIG. 20 is also realized by the CPU 21 executing the color sample creation program 22a. Again, FIG. 20 shows only the control procedure executed by the CPU 21 to determine the arrangement of pixels in the second region after the user designates the first image and instructs creation of a color sample. There is. In FIG. 20, the RAM 23 is described as "memory".

  In FIG. 20, procedures the same as or substantially the same as in the first embodiment are assigned the same reference numerals as in FIG. Thus, in the description of FIG. 20, attention is paid to the procedures given with reference numerals that are not present in FIG.

  In the second embodiment, after the procedure of step S6 is performed, the process proceeds to step S11. In the step S11, the CPU 21 virtually divides the second area into a plurality of partial areas as shown in FIG. 19, for example.

  Next, the CPU 21 calculates granularity for each partial region, and extracts the maximum granularity (maximum granularity) among the calculated granularity (step S12).

  Next, the CPU 21 determines whether the maximum granularity is equal to or less than a predetermined value (step S13). If the maximum granularity is equal to or less than the predetermined value, the determination in step S13 is yes, and the control procedure illustrated in FIG. 20 ends here.

  On the other hand, when the maximum granularity exceeds the predetermined value, the determination in step S13 is no, and the process proceeds to step S14.

  In step S14, the CPU 21 performs the fourth rearrangement in units of partial areas using the self-organizing map. The fourth rearrangement may be performed in units of partial areas, for example, by comparing the granularity of each partial area with a predetermined value different from the predetermined value referred to in step S13. After the fourth rearrangement, the control procedure shown in FIG. 20 ends.

  As described above, according to the present embodiment, in the color sample, all the pixels included in the predetermined area of the original image (first image) are abstractly organized and rearranged in the second area. By this, it is possible to more closely reproduce the impression received from the appearance of color under the actual environment.

  That is, the color sample creation device 210 mounted on the image processing device 2 rearranges the pixel group arranged in the first area in the first image as the original drawing based on the lightness. Perform at least a second reordering, reordering based on hue angle, and a third reordering, reordering based on color, on a row-by-row basis, assuming a line in reordering 1 and a second area As a result, the pixel group obtained is arranged in the second region.

  Thereby, even if the uncorrected image is a picture, the color sample creation device 210 assumes an actual picture and organizes the pixels in an abstract manner, and in the visual characteristics of the human being, the position which is not desirable in that organization. It is possible to create a color sample to be created that has in the second area (selected area) a group of pixels in which the pixels arranged in are arranged at optimal positions. Therefore, the impression of the color sample can be made very close to that of the object existing on the uncorrected image as seen in the actual environment.

  Therefore, when such a color sample is used to perform a pre-modification image (input image) modification process, such as a photograph, it is possible to compare the shapes according to the human visual characteristics. As a result, the retouching to the uncorrected image can be performed so as to more closely reproduce the impression received from the appearance of colors under the actual environment.

  Also, by using such a color sample, the image processing system 100 enables the user to make case selection more intuitively. As a result, the operational convenience is improved, and the user can obtain a desired retouched image more easily and quickly.

  Although a photo is taken as an example of the pre-modification image, the pre-modification image is not limited to the photo. The type of object on the pre-modification image is also not particularly limited.

  The color sample may not be rectangular. The shape of the second region is also not limited to a circle.

  As described above, the present invention has the effect of being able to more closely reproduce the impression received from the appearance of colors in an actual environment, and a color sample creation device, a color sample creation method, and image processing using a color sample Useful for the whole system.

2 Image Processing Device 3 Server Device 21 CPU
22 flash memory 22a color sample creation program 22b image modification program 24 display device 25 input device 26, 27 I / F
31 control unit 32 communication unit 33 first storage unit 34 second storage unit 100 image processing system 210 color sample creation device 211 data acquisition unit 212 arrangement determination unit 213 creation unit 214 registration unit 221 first rearrangement unit 222 first Second sorting unit 223 third sorting unit 230 image modification device 231 data acquisition unit 232 image input unit 233 case selection unit 234 modification unit 235 data output unit

JP, 2001-074556, A

Claims (9)

An image acquisition unit that acquires at least a first pixel group disposed in a first region of a first image;
A first reordering means for performing a first reordering for reordering based on lightness for each pixel constituting the first pixel group acquired by the image acquisition means;
For each pixel after the first sorting means has performed the first sorting, assuming a row in a second area included in a color sample to be created, for each row, Second reordering means for performing second reordering according to hue angles;
A third reordering means for performing a third reordering for reordering based on color for each pixel after the second reordering means has performed the second reordering;
A color sample including: an image creating unit that creates a color sample in which each pixel after the third sorting unit performs the third sorting is disposed in the second area; Creation device. When there are a plurality of continuous pixels having the same lightness among the pixels after the first rearrangement performed by the first rearrangement unit, the saturation is applied to the plurality of pixels in the area. The color sample creating apparatus according to claim 1 , further comprising a fourth rearranging means for performing a fourth rearranging based on the first color. When the number of pixels of the first pixel group is smaller than the number of pixels of the second pixel group, the second rearrangement means determines the number of pixels of the first pixel group as the second rearrangement. The color sample creation device according to claim 1 or 2 , which is performed after the number of pixels of the second pixel group is increased.   The third sorting by the third sorting means is a range in which pixels having the same color within a predetermined tolerance range among the pixels subjected to the second sorting are narrower than the original range The color swatch creation device according to any one of claims 1 to 3, wherein the color sample creation device is disposed in the inside.   The color sample creation device according to any one of claims 1 to 4, wherein the third sorting by the third sorting means is performed using a self-organizing map.   The third sorting by the third sorting means is performed when a granularity is calculated and the calculated granularity exceeds a predetermined value. The color sample creation device according to any one of the above. 7. The color sample creation method according to claim 6 , wherein the third rearrangement by the third rearrangement means is performed by virtually dividing the second area into a plurality of partial areas. apparatus. Obtaining at least a first group of pixels arranged in a first region of a first image;
A step of performing a first rearrangement on the basis of lightness for each of the pixels forming the first pixel group acquired, and a creation target for each pixel after the first rearrangement Performing a second reordering step of reordering on the basis of hue angles for each row assuming a row in a second area included in the color sample;
Performing a third rearrangement on the basis of color for each pixel after the second rearrangement;
Creating a color sample in which the pixels after the third rearrangement are arranged in the second area. An image acquisition unit that acquires an uncorrected image that is an image to be corrected;
Reordering based on lightness, hue angle, and color with respect to the first pixel group disposed in the first region of the first image as a case image to be referred to in performing the retouching on the uncorrected image An image storage unit storing a color sample in which a pixel group generated by at least
An image processing system comprising: modifying means for modifying the uncorrected image with reference to a color sample stored in the image storage means.