JP7106287B2 - Image processing device, imaging device, image processing method, image processing program, and storage medium - Google Patents

Description

The present invention relates to an image processing device that classifies pixels of a captured image based on color information.

Farmers need to observe the color (leaf color) of rice plants to determine the degree of growth when diagnosing the growth of paddy rice and managing its fertilization. For example, a farmer determines the color of rice plants by comparing the leaf color of the plant to a color sample. Such a color sample is called a leaf color color scale, and generally has seven colors, with color numbers 1 to 7 assigned in order from bright green to dark green. In this way, farmers judge the growing condition of rice plants by comparing the color of rice plants with the color of the leaf color scale in paddy fields, etc., and decide the method of fertilization. However, with this method, individual differences in judging the growth status of rice plants are large, making objective evaluation difficult.

Therefore, Japanese Patent Laid-Open No. 2002-100001 discloses a method of performing color determination that is less susceptible to sunlight by simultaneously photographing a leaf color scale and plant leaves in the shade with a digital camera.

Japanese Unexamined Patent Application Publication No. 2017-3491

However, the method of Patent Literature 1 requires a large-scale facility because it is necessary to create a shaded environment and photograph a leaf color color scale.

Therefore, the present invention provides an image processing apparatus, an imaging apparatus, an image processing method, an image processing program, and an image processing program capable of performing color evaluation that is not easily affected by brightness such as sunlight without using large-scale equipment. The purpose is to provide a storage medium.

An image processing apparatus as one aspect of the present invention includes acquisition means for acquiring image information including image data related to plants , normalization means for normalizing the pixel value of each pixel in the image data for each color component, and and classification means for classifying each pixel in the image data into a plurality of color patterns relating to the leaf color of the plant based on the pixel values normalized by the normalization means.

An imaging device as another aspect of the present invention has an imaging element and the image processing device.

An image processing method as another aspect of the present invention includes the steps of obtaining image information including image data related to plants , normalizing the pixel value of each pixel in the image data for each color component, and and classifying each pixel in the image data into a plurality of color patterns relating to the leaf color of the plant based on the obtained pixel values.

An image processing program as another aspect of the present invention causes a computer to execute the image processing method.

A storage medium as another aspect of the present invention stores the image processing program.

Other objects and features of the invention are illustrated in the following examples.

According to the present invention, an image processing apparatus, an imaging apparatus, an image processing method, an image processing program capable of performing color evaluation that is not easily affected by brightness such as sunlight without using large-scale equipment, and , can provide a storage medium.

1 is a block diagram of an image processing device in each embodiment; FIG. 4 is a flow chart of an image processing method in each embodiment. It is an example of pixel values before normalization in Examples 1 and 2. FIG. It is an example of pixel values after normalization in Examples 1 and 2. FIG. FIG. 2 is an explanatory diagram of a neural network in Example 1; FIG. 4 is an explanatory diagram of a leaf color scale used when learning the neural network in Example 1; FIG. 11 is an explanatory diagram of a neural network in Example 2; FIG. 10 is an explanatory diagram of a leaf color scale used when learning the neural network in Examples 2 and 3; It is an example of the classification result of each pixel in Example 2. FIG. FIG. 11 is an explanatory diagram of a neural network in Example 3; FIG. 11 is a configuration diagram of an image processing system in Example 4; FIG. 11 is a block diagram of an imaging device in Example 5;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, an image processing apparatus and an image processing method according to a first embodiment of the present invention will be described with reference to FIGS. 1 and 2. FIG. FIG. 1 is a block diagram of an image processing apparatus 10 according to this embodiment. FIG. 2 is a flow chart of the image processing method in this embodiment. Each step in FIG. 2 is executed by each unit of the image processing apparatus 10 .

As shown in FIG. 1, the image processing apparatus 10 includes an image information acquisition section (acquisition means) 11, a normalization processing section (normalization means) 12, a classification processing section (classification means) 13, and an output section (output means) 14. As shown in FIG. 2, first, in step S1, the image information acquisition unit 11 acquires (inputs) image information (information including photographed image data) to be evaluated. Image information is information including various conditions at the time of photographing in addition to image data (captured image data). The shooting conditions include, but are not limited to, the date and time of shooting, weather information at the time of shooting, and the like.

Subsequently, in step S2, the normalization processing unit 12 normalizes (normalizes) the image data (color components). That is, the normalization processing unit 12 normalizes the pixel value of each pixel in the image information acquired in step S1 for each color component. Normalization of image data is a process for reducing the influence of fluctuations (vertical fluctuations) in image data values caused by differences in shooting environment on the accuracy of pixel classification. As an example, a method of normalizing color components for each pixel will be described below. Here, the pixel values of R, G, and B (color components) at the coordinates (i, j) of each pixel in the image data are R(i, j), G(i, j), and B(i, j), respectively. ). At this time, the R, G, and B pixel values Rn(i, j), Gn(i, j), and Bn(i, j) after normalization can be expressed as follows.

Rn(i,j)=R(i,j)/(R(i,j)+G(i,j)+B(i,j))
Gn(i,j)=G(i,j)/(R(i,j)+G(i,j)+B(i,j))
Bn(i,j)=B(i,j)/(R(i,j)+G(i,j)+B(i,j))
As described above, the normalization processing unit 12 of this embodiment normalizes the pixel value by dividing each of the R, G, and B color components by the sum of the R, G, and B color components for each pixel. become

FIG. 3 is an example of pixel values R(i,j), G(i,j), and B(i,j) before normalization, and is a graph of a plurality of sample points. Figures 3(a) to (g) correspond to leaf color numbers 1 to 7, respectively, and Figures 3(h) and (i) correspond to ground and sky, respectively. In FIG. 3, 1, 2, and 3 on the horizontal axis correspond to pixel values R(i,j), G(i,j), and B(i,j), respectively, and the vertical axis is a dimensionless quantity (pixel value, luminance value).

FIG. 4 shows an example of pixel values Rn(i, j), Gn(i, j), and Bn(i, j) after normalization, and is a graph of a plurality of sample points. Similar to FIG. 3, FIGS. 4(a) to (g) correspond to leaf color numbers 1 to 7, respectively, and FIGS. 4(h) and (i) correspond to ground and sky, respectively. In FIG. 4, 1, 2, and 3 on the horizontal axis correspond to Rn(i,j), Gn(i,j), and Bn(i,j), respectively, and the vertical axis is a dimensionless quantity (pixel value, luminance value). Note that the normalization process may be any method as long as it calculates the ratio of the color components (R, G, B) of the pixel value (for pixels at the same coordinates) of each pixel, and the method described in this embodiment is an example. However, other methods may be used for the normalization process. The pixel values Rn, Gn, and Bn after normalization are concentrated in a narrow range of luminance values (vertical axis), and differences between image data are small. Therefore, the influence of vertical fluctuations in luminance value for each image data during pixel classification is also reduced.

Subsequently, in step S3 of FIG. 2, the classification processing unit 13 classifies each pixel based on the image data (pixel value of each pixel) normalized by the normalization processing unit 12 in step S2. That is, the classification processing unit 13 classifies into one group (color pattern) determined according to the pixel value of each pixel from among a plurality of groups (color patterns). In this example, color classification is performed using a leaf color scale (color sample) and a trained neural network (NN). Here, the NN may be selected from among a plurality of types of learned NNs and used.

Here, an example of the neural network NN in this embodiment will be described with reference to FIG. FIG. 5 is an explanatory diagram of the neural network NN in this embodiment. When learning the neural network NN, the input is normalized color information at an arbitrary pixel position in a specific color region, that is, pixel values Rn(i,j), Gn(i,j), Bn(i,j ). In general, an image pickup device that acquires a digital image has R pixels, G pixels, and B pixels. An image sensor having B pixels is used). Therefore, in FIG. 5, the input layer value "3" shown on the leftmost side indicates the value for each of R, G, and B colors. Note that when an image sensor having infrared pixels (IR pixels) is used, the value of the input layer is 4 with IRn(i, j) added. Also, the output layer value "7" shown on the rightmost side of FIG. 5 corresponds to leaf color numbers 1 to 7 of the leaf color scale, which is a color sample.

Now, with reference to FIG. 6, the leaf color scale (color sample) used when learning the neural network NN in this embodiment will be described. FIG. 6 is an example of a leaf color scale. As shown in FIG. 6, the outline of a general leaf color scale is arranged with leaf color numbers 1 to 7 and color samples corresponding to each number. This leaf color scale is created based on the respective R, G, and B reflectance properties of plant leaves.

As shown in FIG. 5, the neural network NN of this embodiment comprises an input layer, three intermediate layers and an output layer. The number of units in each layer is 3, 100, 50, 20 and 7 respectively. A method of classifying pixels using the neural network NN will be described below. First, using three values of R, G, and B and coefficients (weight coefficients, coefficient data) α1 _N1 , α2 _N1 , α3 _N1 (N1=1 to 100) of the three values, A _N1 =α1 _N1 Calculate R+α2 _N1 G+α3 _N1 B. At this time, similar calculations are repeated while changing each of the coefficients α1 _N1 , α2 _N1 and α3 _N1 . In this embodiment, 100 sets of coefficients α1 _N1 , α2 _N1 , α3 _N1 (N1=1 to 100) are prepared, and the value A _N1 is calculated for each of the 100 coefficient sets. The first intermediate layer value "100" shown to the right of the input layer is calculated using the input values (R, G, B) and each coefficient while changing the set of coefficients as described above. This indicates that 100 types of values are calculated.

The second intermediate layer value "50" shown to the right of the first intermediate layer indicates that 50 values B _N2 are calculated using 100 values A _N1 of the first intermediate layer. showing. 100 types of values A _N1 (N1=1 to 100) of the first intermediate layer and coefficients (weight coefficients, coefficient data) of the values A _N1 (weight coefficients, coefficient data) β1 _N2 , β2 _N2 , . . . β100 _N2 (N2=1 to 50) to calculate B _N2 =β1 _N2 A ₁ +β2 _N2 A ₂ +, . . . +β100 _N2 A ₁₀₀ .

The third intermediate layer value "20" shown to the right of the second intermediate layer indicates that 20 types of values _CN3 are calculated using the 50 types of values B _N2 of the second intermediate layer. showing. 50 types of values B _N2 (N2=1 to 50) of the second intermediate layer and coefficients (weight coefficients, coefficient data) of the value B _N2 (weight coefficients, coefficient data) γ1 _N3 , γ2 _N3 , . . . γ50 _N3 (N3=1 to 20) are used to compute C _N3 =γ1 _N3 B ₁ +γ2 _N3 B ₂ +, . . . +γ50 _N3 B ₅₀ .

In FIG. 3, the value "7" of the output layer shown on the rightmost side indicates that 7 types of values _DN4 are calculated using 20 types of values _CN3 of the third intermediate layer. 20 types of values C _N3 ( _N3 =1 to 20) of the third intermediate layer and coefficients (weight coefficients, coefficient data) of the values C _N3 (weight coefficients, coefficient data) δ1 _N4 , δ2 _N4 , . , D _N4 = _{δ1 N3} C ₁ + _{δ2 N4} C ₂ +, . As a result of such neural network processing, seven types of values D ₁ to D ₇ are obtained. The values D ₁ -D ₇ correspond to leaf color numbers 1-7, respectively, on the leaf color scale described with reference to FIG.

In this embodiment, the learning of the neural network NN includes the coefficients α1 _N1 , α2 _N1 , α3 _N1 , β1 _N2 , β2 _N2 , _{β100 N2} , γ1 _N3 , _{γ2 N3} , . To determine the values of δ2 _N4 , . . . , δ20 _N4 . Specifically, a pixel signal represented by an arbitrary set of R, G, and B values is input, and the color that matches the input pixel signal among leaf color numbers 1 to 7 on a leaf color scale that is a color sample is selected. Each coefficient is learned (determined) so that the leaf color number corresponding to is output.

In this embodiment, a ReLU (Rectified Linear Unit: normalized linear function) is used as the activation function of the neural network NN, but it is not limited to this. As the activation function, for example, functions such as sigmoid and tanh may be used. When learning the neural network NN, an image of a leaf color scale taken with a digital camera (imaging device) is used. For the leaf color scale image used for learning, an image taken with a commercially available digital camera can be used. The neural network NN can learn based only on leaf color scale images acquired using a single digital camera, or based on leaf color scale images captured using multiple types of digital cameras. .

When used in an outdoor environment, the lighting conditions change greatly as the altitude of the sun changes depending on the time of day. Therefore, the accuracy can be further improved by learning the neural network NN for each time slot. In addition, based on positional information such as GPS, information on the date and time of shooting, the altitude and direction of the sun, weather (weather), etc. Selective use of learned NNs can also improve classification accuracy. Further, image information such as weather information may be automatically acquired or manually input by the user to select the NN. Also, if a digital camera with a very large number of pixels is used, the classification process may take a long time. In such a case, the color classification process may be performed after performing the image reduction process. The capture time can be identified in real time, or stored Exif information may be used.

Subsequently, in step S<b>4 in FIG. 2 , the output unit 14 outputs the result of classification by the classification processing unit 13 . Classification results can be output in various ways. It may be configured to output the number of pixels assigned to leaf color numbers 1 to 7 in detail as a histogram, to output the leaf color number that accounts for the largest percentage of the total, or to output the average value. The output result (classification result) may be added to the header area of the image or output to a file.

Next, an image processing method in Example 2 of the present invention will be described. In this embodiment, as in the first embodiment, the pixel classification method relating to the growth of paddy rice will be described as an example. This embodiment differs from the first embodiment in that, in step S3 (pixel classification step) in FIG. is different. Since other configurations and methods are the same as those of the first embodiment, description thereof will be omitted.

The background in this embodiment is a region having a color other than green of the plant. In this embodiment, two backgrounds, the ground and the sky, which are often included in an image at the time of photographing, are used. (each pixel is classified into one of nine groups).

In this embodiment, the normalization processing unit 12 uses pixel values R(i,j), G(i,j), and B(i,j) at coordinates (i,j) in the image data to normalize R, G, and B pixel values Rn(i, j), Gn(i, j), and Bn(i, j) after conversion are calculated as follows.

Rn(i,j)=R(i,j)/G(i,j)
Gn(i,j)=1
Bn(i,j)=B(i,j)/G(i,j)
As described above, the normalization processing unit 12 of this embodiment sets the G color component to 1 for each pixel, and divides each of the R and B color components by the G color component, thereby obtaining a pixel value of Normalize. The normalization process in this embodiment has the feature that it does not require a reference area, and is different from the white balance of a general digital camera.

Here, an example of the neural network NN in this embodiment will be described with reference to FIG. FIG. 7 is an explanatory diagram of the neural network NN in this embodiment. As shown in FIG. 7, the neural network NN of this embodiment comprises an input layer, three intermediate layers and an output layer. The number of units in each layer is 3, 100, 50, 20 and 9 respectively. When learning the neural network NN, the input is normalized color information at an arbitrary pixel position in a specific color region, that is, pixel values Rn(i,j), Gn(i,j), Bn(i,j ). In a typical digital image, pixel data includes R, G, and B values. Therefore, in FIG. 7, the input layer value "3" shown on the leftmost side indicates the value for each of R, G, and B colors. These points are the same as those of the first embodiment described with reference to FIG.

In this embodiment, the output layer value "9" shown on the rightmost side of FIG. 7 corresponds to leaf color numbers 1 to 9 of the leaf color scale, which is a color sample. That is, one of the values corresponding to leaf color numbers 1 to 7 and background numbers 8 and 9 on the leaf color scale is output by the neural network processing of this embodiment.

Now, with reference to FIG. 8, the leaf color scale (color sample) used when learning the neural network NN in this embodiment will be described. FIG. 8 is an example of a leaf color scale. As shown in FIG. 8, the outline of a general leaf color scale has leaf color numbers 1 to 7, ground background number 8, sky background number 9, and color samples corresponding to each number. ing.

As described above, in this embodiment, the classification processing unit 13 divides the specific color regions (pixels) into a plurality of groups (leaf color numbers 1 to 9) including color information corresponding to plant leaves and color information corresponding to non-plant leaves. ) into one group (one of the leaf color numbers 1 to 9). According to this embodiment, by adding a plurality of types of background regions and learning them by the NN, clear background regions can be removed and regions corresponding to leaf colors can be extracted and classified.

FIG. 9 is an example of the result of classifying a rice image in a paddy field for each pixel without photographing a leaf color scale. Of the grayscale values in FIG. 9, 1-7 correspond to leaf color numbers 1-7, respectively, and 8 and 9 correspond to ground and sky, respectively.

Next, an image processing method in Embodiment 3 of the present invention will be described. In this embodiment, as in the first embodiment, the pixel classification method relating to the growth of paddy rice will be described as an example. This embodiment uses near-infrared (IR) pixels in addition to R, G, and B pixels, which is different from the first and second embodiments that use only R, G, and B pixels. Since other configurations and methods are the same as those of the first embodiment, description thereof will be omitted.

In this embodiment, two backgrounds, the ground and the sky, which are often included in the image at the time of photographing, are used, and in addition to the leaf color numbers 1 to 7, the ground background number 8 and the sky background number 9 are used for a total of nine color patterns. (each pixel is classified into one of nine groups). Therefore, in this embodiment, the NN shown in FIG. 10 is used and the leaf color scale shown in FIG. 8 is used for learning. When it is possible to acquire an IR image as in this embodiment, the normalization processing unit 12, in addition to the R, G, and B color components (R pixels, G pixels, and B pixels), A normalization process is performed in consideration of the component (IR pixel). Specifically, the normalization processing unit 12 uses the pixel values R (i, j), G (i, j), B (i, j), and IR (i, j) to convert the normalized R , G, B and IR pixel values Rn(i,j), Gn(i,j), Bn(i,j) and IRn(i,j) are calculated as follows.

Rn(i,j)=R(i,j)/(R(i,j)+G(i,j)+B(i,j)+IR(i,j))
Gn(i,j)=G(i,j)/(R(i,j)+G(i,j)+B(i,j)+IR(i,j))
Bn(i,j)=B(i,j)/(R(i,j)+G(i,j)+B(i,j)+IR(i,j))
IRn(i,j)=IR(i,j)/(R(i,j)+G(i,j)+B(i,j)+IR(i,j))
In this manner, the normalization processing unit 12 of the present embodiment divides each of the R, G, B, and IR color components by the sum of the R, G, B, and IR color components for each pixel, so that the pixel Normalize values.

Here, an example of the neural network NN in this embodiment will be described with reference to FIG. FIG. 10 is an explanatory diagram of the neural network NN in this embodiment. As shown in FIG. 10, the neural network NN of this embodiment comprises an input layer, three intermediate layers and an output layer. The number of units in each layer is 4, 100, 50, 20 and 9 respectively. When learning the neural network NN, the input is normalized color information at an arbitrary pixel position in a specific color region, that is, pixel values Rn(i,j), Gn(i,j), Bn(i,j ), IRn(i,j).

Next, with reference to FIG. 11, an image processing system (image processing apparatus) according to Embodiment 4 of the present invention will be described. FIG. 11 is a configuration diagram of an image processing system 90 in this embodiment. The image processing system 90 includes an image processing device (hardware as an information processing device) 91 , a display device (monitor) 92 , and an input device (input means such as a keyboard) 93 . In this embodiment, an image processing system (computer software) 90 is used to operate any one of the image processing methods of Embodiments 1 to 3. FIG. The image processing device 91 of this embodiment has the functions of the image information acquisition unit 11, the normalization processing unit 12, the classification processing unit 13, and the output unit 14 of the image processing device 10 described with reference to FIG. .

First, in order to operate the image processing method of any one of Embodiments 1 to 3 in the image processing apparatus 91, software (image processing program) for executing the image processing method is installed in the image processing apparatus (personal computer) 91, and an image is processed. Stored in a storage unit (not shown) of the processing device 91 . Also, data relating to the neural network NN is stored in a predetermined folder in the storage section provided in the image processing device 91 . Data related to the neural network NN includes data related to the structure of the neural network NN (structural data) and data related to weighting factors (coefficient data). Software and data relating to the neural network NN can be installed from media (storage media) 94 such as CD-ROM and DVD-ROM. Alternatively, software and data relating to the neural network NN may be downloaded and installed via a network (Internet).

Next, the image processing device 91 activates the installed software and performs image processing on the captured image. The image processing device 91 automatically acquires information about various shooting conditions (shooting conditions) from Exif information or the like, and performs image processing. The software is configured so that various settings can be made, such as area selection such as performing image processing only on a predetermined target area of an image, a method of extracting a specific color area, and selection of a neural network NN. good too. It is preferable that the user can change various settings while checking the image after image processing on the display device 92 . A detailed description of the image processing is omitted because it has been described in the first to third embodiments.

Next, with reference to FIG. 12, a schematic configuration of the imaging apparatus according to the fifth embodiment of the present invention will be described. FIG. 12 is a block diagram of the imaging device 100 in this embodiment. The imaging apparatus 100 includes an image processing unit (image processing apparatus) 104 capable of executing any of the image processing methods of the first to third embodiments.

In the imaging device 100, an object (not shown) forms an image on an imaging device 102 via an imaging optical system 101 (optical system) including a diaphragm 101a (or light blocking member) and a focus lens 101b. In this embodiment, the imaging optical system 101 is an interchangeable lens (lens device) that can be attached to and detached from an imaging apparatus having an imaging element 102 . However, this embodiment is not limited to this, and can also be applied to an image pickup apparatus in which the image pickup apparatus main body and the image pickup optical system 101 are integrally configured.

The aperture value (F number) is determined by the aperture 101a or the light blocking member. The imaging device 102 photoelectrically converts a subject image (optical image) formed via the imaging optical system 101 and outputs image data. Image data (electrical signals) output from the image sensor 102 is output to the A/D converter 103 . The A/D converter 103 converts an electrical signal (analog signal) input from the image sensor 102 into a digital signal and outputs the digital signal (captured image) to the image processing unit 104 . The imaging device 102 and the A/D converter 103 constitute imaging means for photoelectrically converting an optical image (subject image) formed via the imaging optical system 101 and outputting a captured image.

The image processing unit 104 processes the digital signal output from the A/D converter 103 (the image generated based on the image data output from the image sensor 102) in the state detection unit 107 and the storage unit 108. Predetermined image processing is performed using the information. The image processing unit 104 has an image information acquisition unit (acquisition means) 104a, a normalization processing unit (normalization means) 104b, a classification processing unit (classification means) 104c, and an output unit (output means) 104d.

The image information acquisition unit 104a acquires an image (an image based on image data formed via the imaging optical system 101, that is, a captured image). The normalization processing unit 104b performs normalization processing on the image acquired by the image information acquisition unit 104a. The classification processing unit 104c classifies the pixels normalized by the normalization processing unit 104b into one group (color pattern) determined according to the color information of the pixels from among a plurality of groups (color patterns). The output unit 104d outputs the result of classification by the classification processing unit 104c.

The storage unit (storage means) 108 stores data related to the neural network NN used when the image processing unit 104 performs image processing (pixel classification processing). Data related to the neural network NN includes data related to the structure of the neural network NN (structural data) and data related to weighting factors (coefficient data). However, in this embodiment, the data related to the neural network NN may be stored in the internal memory of the image processing unit 104 instead of the storage unit 108 provided outside the image processing unit 104 . The storage unit 108 may store a plurality of data to be applied according to the imaging conditions as data relating to the neural network. For example, for an image acquired under a first imaging condition, first data including first structural data and first coefficient data is used, and for an image acquired under a second imaging condition, a second Pixel classification processing is performed using the second data including the structure data and the second coefficient data.

The state detection unit 107 acquires information about shooting conditions (shooting conditions such as shooting time, focal length, F-number, shooting distance, and image height). The system controller 110 adds information about shooting conditions to an image (shot image) and stores it in the storage unit 108 or the image recording medium 109 . The system controller 110 controls the image processing unit 104 , the display unit 105 , the imaging optical system control unit 106 , the state detection unit 107 and the image recording medium 109 . The imaging optical system control unit 106 controls the operation of the imaging optical system 101 . A state detection unit 107 detects the state of the imaging optical system 101 from the information obtained by the imaging optical system control unit 106 . When a desired image is selected from the image recording medium 109 after photographing, the image processing unit 104 can perform image processing (pixel classification processing) by performing normalization processing on the image.

The image processing unit 104 acquires data related to the neural network NN corresponding to the shooting conditions such as the shooting time acquired by the state detection unit 107 . Then, the image processing unit 104 reads the image data and the data regarding the neural network NN from the storage unit 108, and executes the normalization processing and pixel classification processing of any one of the first to third embodiments. A classification result output from the image processing unit 104 is displayed on the display unit 105 . Note that the image processing unit 104 can perform various settings such as performing normalization processing and pixel classification processing only on some pixels in the captured image.

(Other examples)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

According to each embodiment, there are provided an image processing apparatus, an imaging apparatus, an image processing method, an image processing program, and an image processing apparatus capable of performing color evaluation that is not easily affected by brightness such as sunlight without using large-scale equipment. and a storage medium can be provided.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

10 image processing device 11 image information acquisition unit (acquisition means)
12 normalization processing unit (normalization means)
13 Classification processing unit (classification means)

Claims (18)

Acquisition means for acquiring image information including image data related to plants ;
normalization means for normalizing the pixel value of each pixel in the image data for each color component;
and a classification means for classifying each pixel in the image data into a plurality of color patterns relating to the leaf color of the plant based on the pixel values normalized by the normalization means. 2. The image processing apparatus according to claim 1, wherein said normalization means normalizes said pixel values by obtaining a ratio of said color components of each pixel. 3. The image processing apparatus according to claim 1, wherein the image data includes the pixel values composed of the R, G, and B color components. The normalizing means normalizes the pixel value by dividing each of the R, G, and B color components by the sum of the R, G, and B color components for each pixel. 4. The image processing apparatus according to claim 3. The normalizing means normalizes the pixel value by setting the color component of G to 1 and dividing each of the color components of R and B by the color component of G for each pixel. 4. The image processing apparatus according to claim 3, characterized by: 3. The image processing apparatus according to claim 1, wherein the image data includes the pixel values composed of the color components of R, G, B, and IR. The normalizing means normalizes the pixel value by dividing, for each pixel, each of the color components R, G, B, IR by the sum of the color components R, G, B, IR. 7. The image processing apparatus according to claim 6, characterized by: 8. The image processing apparatus according to claim 1, wherein the image information is information including time. 9. The image processing apparatus according to any one of claims 1 to 8, wherein the image information is information including the direction of the sun. 10. The image processing apparatus according to any one of claims 1 to 9, wherein the image information includes weather information. 11. The image processing apparatus according to claim 1, wherein the image information includes information input by a user. The classification means classifies each pixel into one group determined according to the pixel value from among a plurality of groups based on the pixel value normalized by the normalization means. The image processing apparatus according to any one of claims 1 to 11. 13. The image processing apparatus according to claim 12, wherein said classifying means classifies said pixels using a neural network trained based on color samples corresponding to said plurality of groups. 13. The classifying means classifies the pixels into the one group out of the plurality of groups including color information corresponding to plant leaves and color information corresponding to non-plant leaves. 14. The image processing apparatus according to 13 . an imaging device;
An imaging apparatus comprising the image processing apparatus according to any one of claims 1 to 14. obtaining image information including image data about plants ;
normalizing the pixel value of each pixel in the image data for each color component;
and classifying each pixel in the image data into a plurality of color patterns relating to the leaf color of the plant based on the normalized pixel value. obtaining image information including image data about plants ;
normalizing the pixel value of each pixel in the image data for each color component;
and classifying each pixel in the image data into a plurality of color patterns relating to the leaf color of the plant based on the normalized pixel value. 8. A storage medium storing the image processing program according to claim 17 .

Images