JP6267656B2 - Image processing method, image processing apparatus, and program - Google Patents

Description

  The present invention relates to a technique for extracting a component of a pattern from an image showing a pattern based on the intensity distribution of irradiation light.

  When light is collected on a different surface by refraction or reflection of light on a certain surface, the brightness of that portion becomes brighter than the others. The pattern generated by this luminance modulation is called a light condensing pattern (caustic pattern). Here, the surface that causes refraction and reflection that causes the condensing pattern is referred to as a condensing surface, and the two-dimensional plane on which the condensing pattern is formed is referred to as a projection surface. The condensing pattern generated on the projection surface varies depending on the three-dimensional shape and refractive index that form the condensing surface. Conventionally, mainly in the graphics field, the shape design of a transparent object for generating an arbitrary condensing pattern (for example, see Non-Patent Document 1), and conversely, a condensing pattern is easily generated from the shape of a transparent object. A method for this has been proposed (for example, see Non-Patent Document 2).

Yue, Y., Iwasaki, K., Chen, B., DoBashi, Y., & Nishita, T., “Poisson-Based Continuous Surface Generation for Goal-Based Caustics,” ACM Transactions on Graphics (TOG), 33 ( 3), (2014), 31. Gutierrez, D., Lopez-Moreno, J., Fandos, J., Seron, F., Sanchez, M., Reinhard, E., “Depicting Procedural Caustics in Single Images,” ACM Transactions on Graphics (Proc. Of SIGGRAPH Asia), 27 (5), (2008), 120.

  However, when a condensing pattern is generated on the projection surface, a method for extracting the condensing pattern from the projection surface has not been established. This is not limited to the condensing pattern. That is, a method for extracting a component of a pattern from an image showing a pattern based on the intensity distribution of irradiation light has not been established.

  The subject of this invention is providing the technique which extracts the component of the said pattern from the image in which the pattern based on the intensity distribution of irradiation light appeared.

  An image in which a pattern based on the intensity distribution of the irradiation light appears on the image surface including the first image component in the first wavelength region and the known second image component in the second wavelength region is represented by the first wavelength region and the second wavelength region. The component derived from the intensity component in the second wavelength region is extracted from the detection image obtained by detection so that it can be identified.

  Since the image pattern of the second image component is known, the component of the pattern can be extracted from the image showing the pattern based on the intensity distribution of the irradiation light.

FIG. 1 is a conceptual diagram for illustrating an apparatus configuration of an embodiment. 2A and 2B are block diagrams for illustrating details of the information processing unit. 3A is a diagram for illustrating the configuration of the projection plane, FIG. 3B is a diagram for illustrating a captured image and a part of the captured image, and FIG. 3C is a G channel extracted from a part of the captured image. It is a figure for demonstrating the removal image component from which the pattern based on the intensity distribution of a component, R channel component, and irradiated light was removed.

Embodiments of the present invention will be described below.
[Overview]
The image pattern derived from the texture reflects the color characteristics of the texture. On the other hand, the pattern based on the intensity distribution of light appearing in the texture reflects the color characteristics of the light itself. Using such optical characteristics, a component of the pattern is extracted from an image showing a pattern based on the intensity distribution of irradiation light.

First, a “pattern based on the intensity distribution of irradiation light” appears on the “image surface (texture surface)” including the “first image component in the first wavelength region” and the “second image component in the second wavelength region”. A “detected image” of “image” is obtained. The “first wavelength range” and the “second wavelength range” may be a wavelength range of visible light, or may include a wavelength range other than visible light (ultraviolet rays or infrared rays). The “first wavelength range” and the “second wavelength range” are wavelength ranges that can be distinguished from each other. In other words, the “first wavelength range” and the “second wavelength range” do not overlap each other. The “first wavelength range” and the “second wavelength range” are only different single wavelengths (for example, RGB tristimulus values (the three primary colors of red, green, and blue) are the wavelengths of a single stimulus value). It may contain, and may contain a plurality of wavelength components. The image pattern of the “second image component” is known, and for example, all pixel values thereof are predetermined. Usually, it is desirable that the image pattern of the “second image component” is uniform or nearly uniform from the viewpoint of simplifying the weighting calculation (described later). In other words, it is desirable that the horizontal and vertical spatial frequencies of the “second image component” image pattern are zero (0 Hz) or near zero. However, this does not limit the present invention. Depending on the application, the image pattern of the “second image component” may be biased. “Near α” means belonging to a range of α−β ₁ or more and α + β ₂ or less. beta ₁ and beta ₂ are positive values, it may be a beta ₁ = beta _2, may not be β ₁ = β _2. The image pattern of “first image component” may be unknown or may be known. For example, the “first image component” image pattern and the “second image component” image pattern are different from each other. An example of the image pattern of “first image component” is an image pattern including a negative or positive spatial frequency component (non-zero spatial frequency component). The “image plane” includes at least two image components in two wavelength regions, “first image component in the first wavelength region” and “second image component in the second wavelength region”, but in other wavelength regions. An image component may be included. “Irradiated light” is light applied to the “image plane”. An example of “irradiation light” is, for example, light emitted from a light source passing through a plurality of “light transmission media” having different light refractive indexes and irradiating an “image plane”. In this case, a condensing pattern corresponding to the three-dimensional shape that forms the boundary surface (condensing surface) of the “light transmitting medium” appears on the “image surface”. Another example of the “irradiation light” is that the “image plane” is irradiated through the “light transmission medium” in which the “light shield” is arranged in a part of the region. In this case, a shadow pattern corresponding to the shape of the “light shield” appears on the “image plane”. That is, examples of the “pattern based on the intensity distribution of the irradiation light” are a condensing pattern and a shadow pattern appearing on the “image plane”. The “light transmission medium” and the “light shield” may be fluid or solid. Examples of the “light transmission medium” are air, water, oil, glass, plastic and the like. Examples of the “light shield” are a human body, a floating material of a light transmission medium, smoke, and the like. The “irradiation light” is, for example, light including a component of “second wavelength range” or light including components of “first wavelength range” and “second wavelength range”. The “light emitted from the light source” may be visible light or not visible light. The “light emitted from the light source” is a light (“first wavelength range” and “second wavelength” in which the intensities of the “first wavelength range” and the “second wavelength range” are equal (for example, white light) or substantially equal. It is desirable that the energy ratio to “range” is 1 or the vicinity thereof. The “detected image” is detected so that the “first wavelength region” and the “second wavelength region” can be distinguished. For example, a “detected image” is obtained by capturing an “image” with an image sensor that detects the “first wavelength region” and the “second wavelength region” separately. The “first wavelength range” is a wavelength range of any one of the RGB tristimulus values (for example, G (green) in the RGB color system), and the “second wavelength range” is another single stimulus. In the wavelength range of the value (for example, R (red)), the “detected image” is obtained by capturing the “image” with an image sensor (such as a normal digital camera or video camera) that can identify the RGB tristimulus values. Can be obtained.

  Next, “a component derived from the intensity component in the second wavelength region” is extracted from the “detected image”. “Δ derived from γ” can be restated as “δ obtained from γ” and “δ corresponding to γ”. δ = γ or δ = f (γ) may be used. However, f (γ) is a function value of γ. Examples of “components derived from intensity components in the second wavelength range” include the intensity component in the second wavelength range, the value obtained by multiplying the intensity component in the second wavelength range by a predetermined weighting factor, and the intensity component in the second wavelength range. A value obtained by subtracting the second reference value from the value obtained by subtracting the second reference value (intensity change, intensity level change amount), a value obtained by multiplying the intensity component in the second wavelength range by a predetermined weighting factor, and other second wavelengths It is a function value of the intensity component of the region. An example of “second reference value” is an average value of intensity components of the second image component, and an example of “intensity component of second wavelength range” is a pixel value (intensity level) of a color corresponding to the second wavelength range. ).

  The “component derived from the intensity component in the second wavelength region” extracted as described above is derived from the “image pattern of the second image component” and the “pattern based on the intensity distribution of the irradiation light”. Since the “image pattern of the second image component” is known, the intensity component of “pattern based on the intensity distribution of the irradiation light” can be extracted from the “component derived from the intensity component in the second wavelength region”. For example, it can be considered that the “image pattern of the second image component” is uniform and the “component derived from the intensity component in the second wavelength region” does not include a component based on the “image pattern of the second image component”. In this case, the “component derived from the intensity component in the second wavelength range” can be set as the intensity component of “pattern based on intensity distribution of irradiation light”. Alternatively, the intensity component based on the “image pattern of the second image component” is extracted from the “component derived from the intensity component of the second wavelength region” by known weighting and filtering corresponding to the “image pattern of the second image component”. May be.

  Image processing using a “component derived from an intensity component in the second wavelength region” may be performed. For example, the “component derived from the intensity component in the first wavelength range” is further extracted from the “detected image”, and the “component derived from the intensity component in the first wavelength range” or a function value thereof is used. An operation including subtraction of a “component derived from an intensity component” or a function value thereof may be performed. In the example of “component derived from the intensity component in the first wavelength range”, “second wavelength range” in the example of “component derived from the intensity component in the second wavelength range” is replaced with “first wavelength range”. It is a thing. Thereby, an intensity component obtained by removing or suppressing the “pattern based on the intensity distribution of the irradiation light” from the image of the “image pattern of the first image component” can be obtained. Alternatively, an image in which “a component derived from the intensity component in the second wavelength region” or a function value thereof is added to an arbitrary image component may be generated. As a result, it is possible to obtain an image in which a “pattern based on the intensity distribution of irradiation light” or a function value thereof is integrated into an arbitrary image component. “Adding” means, for example, “compositing”, “superimposing”, “integrating”, “adding”, “reflecting”, “incorporating”. These image processes can be performed by an operation including at least one of weighted subtraction, weighted addition, and weighted multiplication, for example. Specific examples of these will be described later.

[First Embodiment]
In the first embodiment, using the technique described in the overview, a pattern based on the intensity distribution of irradiation light transmitted through two light transmission media having different refractive indexes (for example, a light collection pattern and a shadow pattern) is removed. Information (texture deformation information) of “image surface (texture surface)” that appears to be deformed by light refraction is extracted. In the following, a layer of “transparent liquid (for example, water)” is present on the image plane, and a layer of “transparent gas (for example, air)” having a refractive index different from that of the layer of “transparent liquid” is provided on this “transparent liquid” layer. The case where it exists is illustrated. In this case, the way in which light is refracted changes according to the movement of the “transparent liquid” and “transparent gas”, and the image on the image surface seen from the layer side of the “transparent gas” is optically deformed. At the same time, a “pattern based on the intensity distribution of irradiation light” such as a light condensing pattern and a shadow pattern may be formed on the “image surface”. By using this technique, it is possible to remove the “pattern based on the intensity distribution of irradiation light” from such an image and extract the texture deformation information of the “image plane”.

<Configuration>
As illustrated in FIG. 1, the system of this embodiment includes an information processing apparatus 11, an image sensor 12, a light source 13, a container 14, and light transmission media 151 and 152.

  The information processing apparatus 11 includes a memory 111, an extraction unit 112, and an information processing unit 113. As illustrated in FIG. 2A, the information processing unit 113 includes a subtraction unit 1131 and a weighting unit 1132. The information processing apparatus 11 is, for example, a general-purpose or dedicated computer including a processor (hardware processor) such as a CPU (central processing unit) or a memory such as random-access memory (RAM) or read-only memory (ROM). Is an apparatus configured by executing a predetermined program. The computer may include a single processor and memory, or may include a plurality of processors and memory. This program may be installed in a computer, or may be recorded in a ROM or the like in advance. In addition, some or all of the processing units are configured using an electronic circuit that realizes a processing function without using a program, instead of an electronic circuit (circuitry) that realizes a functional configuration by reading a program like a CPU. May be. In addition, an electronic circuit constituting one device may include a plurality of CPUs.

  The image sensor 12 is an apparatus that detects an image so that the first wavelength region and the second wavelength region can be distinguished. In this embodiment, as an example, an image sensor 12 (a normal digital camera, a video camera, or the like) that can identify RGB tristimulus values (R channel, G channel, and B channel) is used. The light source 13 is a device that emits light. In this embodiment, the light source 13 that emits visible light (for example, white light) is used as an example.

  An image surface 141 (projection surface) is provided on the inner bottom surface of the container 14. A texture image 141T appears on the image plane 141. The texture image 141T includes “a first image component in the first wavelength region” and “a second image component in the second wavelength region”. The “image pattern of the second image component” in this embodiment is known and is a spatially uniform pattern (all pixel values are the same) or a pattern close thereto. That is, the spatial frequency of the “image pattern of the second image component” is zero or near zero. On the other hand, the “image pattern of the first image component” in this embodiment is a spatially non-uniform pattern, unlike the “image pattern of the second image component”. In the present embodiment, an example is shown in which the texture image 141T is configured with two stimulus values (G and R) of RGB tristimulus values. As illustrated in FIG. 3A, the R channel component 141r based on the stimulus value R (red) of the texture image 141T is a “second image component in the second wavelength region” and has a uniform intensity pattern. The G channel component 141g based on the stimulus value G (green) of the texture image 141T is the “image pattern of the first image component”, and is a pattern with non-uniform intensity. The texture image 141T may be printed on the image surface 141 by a normal color printer, may be drawn by paint or the like, or may be the color of the material itself. Alternatively, the image surface 141 may display the texture image 141T. For example, the bottom surface of the container 14 may be a transparent plate, a display facing the inside of the container 14 may be disposed outside the transparent plate, and the texture image 141T may be displayed from the display. The displayed texture image 141T is transmitted through the bottom surface of the container 14 and appears on the image surface 141. The display is preferably capable of displaying colors with spatially uniform intensity. In addition, the display surface itself may display the texture image 141T.

  A light transmission medium 151 is disposed inside the container 14. The container 14 in which the light transmission medium 151 is disposed is disposed in another light transmission medium 152, and the upper surface of the light transmission medium 151 is in contact with the light transmission medium 152. The light transmission medium 151 and the light transmission medium 152 have different light refractive indexes. In this embodiment, an example in which the light transmission medium 151 is a transparent liquid (for example, water) and the light transmission medium 152 is a transparent gas (for example, air) is shown.

<Processing>
The light emitted from the light source 13 passes through the light transmission medium 152 and further passes through the light transmission medium 151, and is irradiated onto the image surface 141 on which the texture image 141T appears. As a result, a “pattern based on the intensity distribution of irradiation light” that changes in accordance with the movement of the light transmission medium 151 appears on the image surface 141. The “pattern based on the intensity distribution of irradiation light” and the “image” based on the “texture image 141T” are transmitted through the light transmission medium 151 and further transmitted through the light transmission medium 152. Even in this process, the “image” optically changes according to the movement of the light transmission medium 151. This “image” is an example of “an image in which a pattern based on the intensity distribution of irradiation light appears on an image surface including a first image component in the first wavelength range and a second image component in the second wavelength range”. It is detected (photographed) by the image sensor 12 arranged in the transmission medium 152. The obtained captured image 121 is stored in the memory 111 so that the R channel component and the G channel component included in the captured image 121 can be identified (FIG. 3B).

The extraction unit 112 cuts out a detection image 1211 that is a part of a two-dimensional area of the captured image 121 from the memory 111 (FIG. 3B). The detection image 1211 is, for example, an image from which an area not including the image of the texture image 141T is removed. Further, the image may be an image in which the region of the light image totally reflected at the boundary surface between the light transmission medium 151 and the light transmission medium 152 is removed. The detection image 1211 is an example of “a detection image obtained by detection so that the first wavelength region and the second wavelength region can be distinguished”. Here, the component I (x, y) derived from the intensity component at the coordinates (x, y) of the detected image 1211 is expressed as follows (formula (1)).
_{I (x, y) = G} I (x, y) + R I (x, y) (1)
However, x represents a horizontal position and y represents a vertical position. Let the lower limit and upper limit of the horizontal position be x _min and x _max (x _min <x _max ), and the lower limit and upper limit of the vertical position y _min and y _max (y _min <y _max ). x and y satisfy x _min ≦ x ≦ x _max and y _min ≦ y ≦ y _max . G _I (x, y) is a G channel component of I (x, y), and is an example of “a component derived from an intensity component in the first wavelength region”. In the example of G _I (x, y), the pixel value G (x, y) and the pixel value G (x, y) of the G channel of the coordinates (x, y) of the detected image 1211 are multiplied by a predetermined weight coefficient. value, pixel value G (x, y) value obtained by subtracting the reference value a _G from (variation of intensity level), a pixel value G (x, y) a predetermined weight coefficient to a value obtained by subtracting the reference value a _G from This is a function value of the multiplied value and other pixel values G (x, y). An example of “reference value A _G ” is an average value of G channel pixel values G (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) of the detected image 1211. On the other hand, R _I (x, y) is an R channel component of I (x, y), and is an example of “a component derived from an intensity component in the second wavelength region”. In the example of R _I (x, y), the pixel value R (x, y) and the pixel value R (x, y) of the R channel at the coordinates (x, y) of the detected image 1211 are multiplied by a predetermined weight coefficient. value, the pixel value R (x, y) value obtained by subtracting the reference value a _R of (the amount of change in intensity level), the pixel value R (x, y) a predetermined weight coefficient to a value obtained by subtracting the reference value a _R from It is a function value of the multiplied value and other pixel values R (x, y). An example of “reference value A _R ” is an average value of R channel pixel values R (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) of the detected image 1211.

The extraction unit 112 separates and extracts G _I (x, y) and R _I (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) from the detected image 1211 (FIG. 3C). ). Here, the component caused by “texture image 141T” in G _I (x, y) of G channel is G _IT (x, y), and the component caused by “pattern based on intensity distribution of irradiation light” is G _IC ( x, y). G _I (x, y) is expressed as follows (formula (2)).
G _I (x, y) = G _IT (x, y) + G _IC (x, y) (2)
Here, G _IT (x, y) reflects the image pattern (FIG. 3A) of the G channel component 141g. On the other hand, G _IC (x, y) mainly reflects the color characteristics of the “pattern based on the intensity distribution of irradiation light” itself, and is hardly affected by the image pattern of the G channel component 141g.

In addition, the component caused by “texture image 141T” in R _I (x, y) of the R channel is R _IT (x, y), and the component caused by “pattern based on intensity distribution of irradiation light” is R _IC (x , Y). R _I (x, y) is expressed as follows (formula (3)).
R _I (x, y) = R _IT (x, y) + R _IC (x, y) (3)
Here, R _IT (x, y) reflects the image pattern (FIG. 3A) of the R channel component 141r. On the other hand, R _IC (x, y) mainly reflects the color characteristics of the “pattern based on the intensity distribution of irradiation light” itself, and is hardly affected by the image pattern of the R channel component 141r. Since the R channel component 141r of the texture image 141T of this embodiment is uniform, R _IT (x, y) can be regarded as a constant R _IT (known) (formula (4)).
R _I (x, y) = R _IT + R _IC (x, y) (4)
That is, R _I (x, y) corresponds to the intensity based on the “pattern based on the intensity distribution of the irradiation light”. In particular, when R _I (x, y) = R (x, y) −A _R (where A _R is the average value of the R channel pixel values R (x, y) of the detected image 1211), R _IT (X, y) = R _IT = 0. In this case, R _I (x, y) can be expressed as follows (formula (5)).
R _I (x, y) = R _IC (x, y) (5)

G _I (x, y) and R _I (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) are sent to the information processing unit 113 (FIG. 2A). The information processing unit _113, G I _(x, y) from _R I (x, y) weighted values of _{w × R I (x, y} ) is subtracted (weighted subtraction). This is an example of “a calculation including subtraction of a component derived from the intensity component in the second wavelength region or a function value thereof from a component derived from the intensity component in the first wavelength region or a function value thereof”. For example, R _I (x, y) is input to the weighting unit 1132, and the weighting unit 1132 calculates and outputs a weighting value w × R _I (x, y). w is a constant representing the weight (where w ≠ 0), and satisfies the following relationship (Equation (6)), for example.
0 = G _IC (x, y) −w × R _IC (x, y) (6)
The value of w in Equation (6) represents the ratio between G _IC (x, y) and R _IC (x, y). G _IC (x, y) and R _IC (x, y) mainly reflect the color characteristics of the “pattern based on the intensity distribution of the irradiation light” itself. Therefore, the value of w is mainly a sensitivity ratio in the G channel and R channel (“first wavelength region” and “second wavelength region”) of the image sensor 12 and “pattern based on intensity distribution of irradiation light”. It depends on the intensity ratio of the G channel component and the R channel component (“first wavelength region” and “second wavelength region”). Since the sensitivity ratio of the image sensor 12 and the light emitted from the light source 13 are known, the value of w can be set in advance. In particular, when it is assumed that the “pattern based on the intensity distribution of irradiation light” is incident on the image sensor 12 as an ideal equal energy white light, the sensitivity ratio between the G channel and the R channel of the image sensor 12 can be set to w. . In this case, an appropriate w can be set easily. Therefore, it is desirable that the light emitted from the light source 13 is white light or light close thereto. Since G _IT (x, y) and R _IC (x, y) are hardly affected by the image pattern, w may be the same for each coordinate (x, y). Further, it is not necessary to strictly satisfy the expression (6), and the right side of the expression (6) may be in the vicinity of 0. Furthermore, w may be adjustable.

G _I (x, y) and w × R _I (x, y) are input to the subtraction unit 1131, and the subtraction unit 1131 calculates the intensity component I ′ (x, y) as follows (formula (7)) ).
I ′ (x, y) = G _I (x, y) −w × R _I (x, y) (7)
Here, the expressions (2), (4), (6) to (7) can be modified as follows (expression (8)).
I ′ (x, y) = {G _IT (x, y) + G _IC (x, y)} − w × {R _IT + R _IC (x, y)} = G _IT (x, y) −w × R _IT (8)
In particular, when R _IT (x, y) = R _IT = 0, the following is obtained (formula (9), FIG. 3C).
I ′ (x, y) = G _IT (x, y) (9)
As shown in equation (8) and (9) _are assigned the appropriate weight w from G I (x, y) by subtracting the _R I (x, y), based on the intensity distribution of the "irradiation light An intensity component I ′ (x, y) (texture deformation information of the image plane 141) from which a component based on the “pattern” has been removed can be obtained (FIG. 3C). In addition, since w × R _IT in Expression (8) is known, a value obtained by adding w × R _IT to Expression (8) may be I ′ (x, y). More generally, a value obtained by changing the base level of the formula (8) or the formula (9) may be set as I ′ (x, y).

The intensity component I ′ (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) may be used as the pixel value of the G channel coordinates (x, y) as it is, The pixel value of the coordinates (x, y) of other R or B channels may be used, the pixel value of the coordinates (x, y) of all RGB channels, or the coordinates (x, y). May be set as the luminance value. By using such an image, it is possible to observe the fluctuation of the texture image 141T through the light transmission medium 151 without being affected by the “pattern based on the intensity distribution of the irradiated light” such as a light condensing pattern or a shadow pattern. In addition, the configuration of the present embodiment can be realized by a simple facility such as a texture image configured by two color channels and a digital camera.

  Instead of using w that satisfies Equation (6), w that can be arbitrarily set may be used. In this case, the presence / absence and density of the “pattern based on the intensity distribution of the irradiation light” can be manipulated by changing the value of w. Thereby, the impression of an image can be changed.

Only the extracted component of “pattern based on intensity distribution of irradiation light” may be output. For example, a value obtained by changing the base level of R _I (x, y) or R _I (x, y) in Expression (4) or Expression (5) may be output as I ′ (x, y). Further, in this _embodiment, G I (x, y) from the _{w × R I (x, y} ) by subtracting the I '(x, y) is to give the weights back to _G I (x, y) I ′ (x, y) may be obtained by subtracting R _I (x, y) from the value attached with. Alternatively, I ′ (x, y) may be obtained by subtracting a value weighted to R _I (x, y) from a value weighted to G _I (x, y). That is, what kind of calculation is performed as long as it is a "calculation including subtraction of a component derived from the intensity component in the second wavelength region or a function value thereof from a component derived from the intensity component in the first wavelength region or a function value thereof". It may be done.

  The texture image 141T described above is merely an example, and the R channel component may be a “second image component in the second wavelength range” and the B channel component may be the “first image component in the first wavelength range”. The G channel component may be the “second image component in the second wavelength region”, and the B channel component may be the “first image component in the first wavelength region”. Further, the roles of the channels may be switched.

[First Modification of First Embodiment]
The “image pattern of the second image component” has only to be known and does not necessarily have to be a uniform pattern or a pattern close thereto. In this modification, a method that can be applied even when the “image pattern of the second image component” is not uniform or close to the pattern will be described. In the following, differences from the first embodiment will be mainly described, and the same reference numerals will be used for portions common to the first embodiment to simplify the description.

<Configuration>
As illustrated in FIG. 1, the system of this modification is obtained by replacing the information processing apparatus 11 of the system of the first embodiment with an information processing apparatus 11 ′. The information processing apparatus 11 ′ includes a memory 111, an extraction unit 112, and an information processing unit 113 ′. As illustrated in FIG. 2A, the information processing unit 113 ′ includes a subtraction unit 1131 and weighting units 1132 and 1134 ′. The information processing apparatus 11 ′ is an apparatus configured by the above-described computer executing a predetermined program, for example.

<Processing>
The difference from the first embodiment is only the processing of the information processing unit 113 ′. Only the processing of the information processing unit 113 ′ will be described below. The information processing unit 113 _'subtracts the G I (x, y) from _R I (x, y) of the weighted value _{w × R I (x, y} ), according to further "image pattern of the second image component" The weight w ₂ (x, y) for each coordinate (x, y) (where x _min ≤ x ≤ x _max , y _min ≤ y ≤ y _max ) is added.

That is, first, the subtraction unit 1131 calculates the intensity component I ″ (x, y) from the input G _I (x, y) and R _I (x, y) as follows (formula (10)) ).
I ″ (x, y) = G _I (x, y) −w × R _I (x, y) (10)
Here, Expressions (2), (3), and (6) to Expression (10) can be modified as follows (Expression (11)).
I ″ (x, y) = {G _IT (x, y) + G _IC (x, y)} − w × {R _IT (x, y) + R _IC (x, y)} = G _IT (x, y) y) −w × R _IT (x, y) (11)
The intensity component I ″ (x, y) is input to the weighting unit 1134 ″.

Next, the weighting unit 1134 ″ adds the weight w ₂ (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) to the intensity component I ″ (x, y). Thus, the intensity component I ′ (x, y) is obtained and output (formula (12)).
I ′ (x, y) = I ″ (x, y) + w ₂ (x, y) (12)

Here, it is assumed that w ₂ (x, y) satisfies the following relationship (formula (13)).
w ₂ (x, y) = w × R _IT (x, y) (13)
Then, from the equations (11) to (13), the intensity component I ″ (x, y) is as follows (equation (14)).
I ′ (x, y) = G _IT (x, y) (14)
Therefore, the formula satisfy _{(13) w 2 (x,} y) or its approximate _w 2 (x, y) is set to the first embodiment and the same intensity component I '(x, y) to obtain Can do. The R channel component (image pattern of the second image component), the image sensor 12, and the light source 13 of the texture image of this embodiment are known. Therefore, it is easy to set such w ₂ (x, y). For example, w ₂ (x, y) may be obtained by multiplying the pixel value P _R (x, y) of the R channel component of the texture image by a constant w ₃ corresponding to the image sensor 12 and the light source 13 (formula (15) )).
w ₂ (x, y) = w ₃ × P _R (x, y) (15)
Alternatively, w ₂ (x, y) in Expression (15) may be determined with w ₃ as an adjustable value. Others are the same as the first embodiment.

[Second Embodiment]
In this embodiment, R _I (x, y) (component derived from the intensity component in the second wavelength range) extracted as in the first embodiment or its modification 1 or a function value thereof is added to an arbitrary image component. .

<Configuration>
As illustrated in FIG. 1, the system of this modification is obtained by replacing the information processing apparatus 11 of the system of the first embodiment with an information processing apparatus 21. The information processing device 21 includes a memory 111, an extraction unit 112, and an information processing unit 213. As illustrated in FIG. 2B, the information processing unit 213 includes an integration unit 2131 and a weighting unit 2132. The information processing apparatus 21 is an apparatus configured by, for example, the above-described computer executing a predetermined program.

<Processing>
The difference from the first embodiment is only the processing of the information processing unit 213. Only the processing of the information processing unit 213 will be described below. R _I (x, y) is input to the weighting unit 2132 of the information processing unit 213 (FIG. 2B). Weighting section _2132, R I _(x, y) values were subjected to weighted _{R I} '(x, y) the obtained output (equation (16) or formula (17)).
R _I ′ (x, y) = w ₄ × R _I (x, y) (16)
R _I ′ (x, y) = w ₄ (x, y) × R _I (x, y) (17)
However, Expression (16) is a case where the R channel component of the texture image (image pattern of the second image component) is uniform or close to that (when R _I (x, y) extracted as in the first embodiment is used). ). w ₄ is the same in all coordinates (x, y) (where x _min ≤ x ≤ x _max , y _min ≤ y ≤ y _max ). It w ₄ may be identical to the above-mentioned w, may be other values. When w ₄ = 1, the weighting unit 2132 may be omitted, and R _I ′ (x, y) = R _I (x, y) may be set. On the other hand, Expression (17) represents processing when the R channel component of the texture image (image pattern of the second image component) is not necessarily uniform or close thereto. w ₄ (x, y) may be the same as w ₂ (x, y) or w ₃ (x, y) described above, or may be other values.

R _I ′ (x, y) is input to the integration unit 2131. The integration unit 2131 further adds R _I ′ (x to the arbitrary image P (x, y) = {P _r (x, y), P _g (x, y), P _b (x, y)}} that is _input. , Y), and an intensity component I ′ (x, y) (where x _min ≦ x ≦ x _max , y _min ≦ y ≦ y _max ) is obtained and output. Here, P _r (x, y), P _g (x, y), and P _b (x, y) represent an R channel component, a G channel component, and a B channel component of P (x, y), respectively. For example, the integration unit 2131 performs weighted addition or weighted multiplication of R _I ′ (x, y) on each of P _r (x, y), P _g (x, y), and P _b (x, y). Let I ′ (x, y). As a result, an image with a “pattern based on the intensity distribution of irradiation light” added to an arbitrary image P (x, y) can be obtained, and an impression close to liquid can be given.

[Other variations]
The present invention is not limited to the embodiment described above. The various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  When the above-described information processing configuration is realized by a computer, processing contents of functions that the information processing configuration should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

  This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, this computer reads a program stored in its own recording device and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer. The processing according to the received program may be executed sequentially. The above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good.

  In addition, a data structure having data representing a component derived from the intensity component in the second wavelength range extracted from the “detected image” may be distributed. However, the “detected image” is an image in which a pattern based on the intensity distribution of the irradiation light appears on the image surface including the first image component in the first wavelength region and the known second image component in the second wavelength region. It is obtained by detecting so that the wavelength range and the second wavelength range can be distinguished. This distribution may be performed by distributing the data structure via the Internet or the like, or by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM that records the data structure. It may be broken. The device receiving this data structure inputs data representing a component derived from the intensity component in the second wavelength region to the image processing unit, and the image processing unit uses the component derived from the intensity component in the second wavelength region. Image processing.

11, 11 ', 21 Information processing apparatus

Claims (7)

An image in which a condensing pattern and / or a shadow pattern based on an intensity distribution of irradiation light appears on an image surface including a first image component in a first wavelength region and a known second image component in a second wavelength region is the first image component. An extraction step of extracting a component derived from an intensity component of the second wavelength region from a detection image obtained by detecting the first wavelength region and the second wavelength region so as to be distinguishable;
An image processing method comprising: an image processing step in which a component derived from an intensity component in the second wavelength region or a function value thereof is added to an arbitrary image component. An image in which a condensing pattern and / or a shadow pattern based on an intensity distribution of irradiation light appears on an image surface including a first image component in a first wavelength region and a known second image component in a second wavelength region is the first image component. From a detection image obtained by detecting the first wavelength region and the second wavelength region so as to be distinguishable, derived from the intensity component in the first wavelength region and the intensity component in the second wavelength region An extraction step for extracting components to be performed;
An image processing step for performing an operation including subtraction of a component derived from the intensity component of the second wavelength band or a function value thereof from a component derived from the intensity component of the first wavelength band or a function value thereof;
The image processing method includes a step of performing an operation including at least one of weighted subtraction, weighted addition, or weighted multiplication. An image in which a condensing pattern and / or a shadow pattern based on an intensity distribution of irradiation light appears on an image surface including a first image component in a first wavelength region and a known second image component in a second wavelength region is the first image component. An extraction step for extracting a component derived from an intensity component of the second wavelength region from a detection image obtained by detecting the first wavelength region and the second wavelength region so as to be distinguishable;
An image processing method, wherein a spatial frequency of an image pattern of the second image component is zero or near zero. An image in which a condensing pattern and / or a shadow pattern based on an intensity distribution of irradiation light appears on an image surface including a first image component in a first wavelength region and a known second image component in a second wavelength region is the first image component. An extraction unit that extracts a component derived from an intensity component of the second wavelength region from a detection image obtained by detecting the first wavelength region and the second wavelength region so as to be distinguishable ;
An image processing apparatus comprising: an image processing unit that adds a component derived from an intensity component in the second wavelength region or a function value thereof to an arbitrary image component .   An image in which a condensing pattern and / or a shadow pattern based on an intensity distribution of irradiation light appears on an image surface including a first image component in a first wavelength region and a known second image component in a second wavelength region is the first image component. From a detection image obtained by detecting the first wavelength region and the second wavelength region so as to be distinguishable, derived from the intensity component in the first wavelength region and the intensity component in the second wavelength region An extraction unit for extracting components to be
  An image processing unit that performs an operation including subtraction of a component derived from the intensity component in the second wavelength band or a function value thereof from a component derived from the intensity component in the first wavelength band or a function value thereof;
  The image processing unit is an image processing apparatus that performs an operation including at least one of weighted subtraction, weighted addition, or weighted multiplication.   An image in which a condensing pattern and / or a shadow pattern based on an intensity distribution of irradiation light appears on an image surface including a first image component in a first wavelength region and a known second image component in a second wavelength region is the first image component. An extraction unit for extracting a component derived from an intensity component of the second wavelength region from a detection image obtained by detecting the first wavelength region and the second wavelength region so as to be distinguishable;
  An image processing apparatus, wherein a spatial frequency of an image pattern of the second image component is zero or near zero. Program for executing the steps of any of the image processing method of claims 1 to 3, to the computer.