JP6463153B2 - Image processing apparatus, imaging apparatus, image processing method, and program - Google Patents

Description

  The present technology relates to an image processing device, an imaging device, an image processing method, and a program for causing a computer to execute the method. Specifically, the present invention relates to an image processing apparatus, an imaging apparatus, an image processing method, and a program for causing a computer to execute the method.

  Conventionally, monitoring systems that detect moving objects such as objects and humans by performing image processing on images captured by a monitoring camera have been widely used in the field of crime prevention and measurement. An image processing apparatus using a background subtraction method has been proposed for detection of this moving object (see, for example, Patent Document 1). In this background subtraction method, in a difference image between an input image to be detected and a background image including only a stationary object, an area having a difference equal to or greater than a threshold is detected as a moving object.

JP 2000-324477 A

  However, in the above-described image processing device, when the brightness of the input image with respect to the background image changes due to lighting on or off, a stationary object region in which the difference exceeds a threshold due to the change may be erroneously detected as a moving object. There is. Therefore, there is a problem that it is difficult to detect a moving object with high accuracy.

  The present technology has been created in view of such a situation, and an object thereof is to improve the detection accuracy of a moving object without being influenced by a change in illumination.

  The present technology has been made to solve the above-described problems. The first aspect of the present technology is that each time an input image in which input pixels are arranged is input, the input image and reference pixels are arranged. A reference image and a difference degree acquisition unit for acquiring a difference value of each pixel value between the reference pixel and the input pixel corresponding to the reference pixel for each input pixel, and each of the reference image and the input image An image processing apparatus comprising: a determination unit that determines whether or not there is a change in illuminance exceeding a predetermined change amount at the time of imaging based on the distribution state of the difference for each of the input pixels; and an image processing method and the method A program for causing a computer to execute. This brings about the effect that it is determined that there is a change in illuminance based on the distribution state of the dissimilarity.

  Further, in the first aspect, a moving object detection unit that detects a moving object for the input image may be further included. This brings about the effect | action that a moving body is detected.

  In the first aspect, the determination unit may stop the detection of the moving object when the illuminance change occurs. This brings about the effect that the detection of the conductor stops when there is a change in illuminance.

  In the first aspect, the moving object detection unit may generate a difference image between the input image and the reference image and detect a region where the difference is larger than a predetermined difference threshold in the difference image as the moving object. Good. This brings about the effect | action that the area | region where a difference is larger than a predetermined difference threshold value is detected as a moving body.

  The first aspect may further include a mode value obtaining unit that obtains the mode value of the degree of difference in the distribution, and the determination unit may determine whether the mode value is within a predetermined range. The presence or absence of the illuminance change may be determined. This brings about the effect that the presence or absence of a change in illuminance is determined depending on whether or not the mode value of the degree of difference is within a predetermined range.

  The first aspect further includes an outlier removal unit that removes a set including an outlier from a set of pixel values of the reference pixel and the input pixel corresponding to the reference pixel. The difference degree acquisition unit may acquire the difference degree in the set from which the set including the outlier is removed. This brings about the effect that the dissimilarity is acquired in the set from which the set including outliers is removed.

  In the first aspect, the difference degree acquisition unit may acquire a difference between a logarithm of the pixel value of the reference pixel and a logarithm of the pixel value of the input pixel as the difference degree. Thereby, the difference between the logarithm of the pixel value of the reference pixel and the logarithm of the pixel value of the input pixel is obtained as the difference.

  In addition, according to a second aspect of the present technology, an imaging unit that captures an input image in which input pixels are arranged in time series order, and a reference image in which the input image and the reference pixel are arranged each time the input image is input The difference degree acquisition unit for acquiring the difference in pixel value between the reference pixel and the input pixel corresponding to the reference pixel for each input pixel, and each of the reference image and the input image A determination unit that determines whether or not there is an illuminance change exceeding a predetermined change amount based on a distribution state of the difference for each input pixel. This brings about the effect that it is determined that there is a change in illuminance based on the distribution state of the dissimilarity.

  In addition, in the second aspect, when there is no change in illuminance, an exposure control unit that controls the exposure amount of the imaging unit to a predetermined amount may be further provided. Thereby, when there is no illuminance change, the exposure amount of the imaging unit is controlled to a predetermined amount.

  According to the present technology, it is possible to achieve an excellent effect that the detection accuracy of a moving object can be improved. Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

1 is a block diagram illustrating a configuration example of an imaging apparatus according to a first embodiment. It is a block diagram which shows one structural example of the illumination intensity change determination part in 1st Embodiment. It is a figure for demonstrating the determination method of the presence or absence of the illumination intensity change in 1st Embodiment. It is a block diagram which shows one structural example of the image process part in 1st Embodiment. It is a figure which shows an example of the background image in 1st Embodiment, an input image, and a difference image. 3 is a flowchart illustrating an example of an operation of the imaging apparatus according to the first embodiment. It is a flowchart which shows the illumination intensity change determination process in 1st Embodiment. It is a flowchart which shows the exposure control process in 1st Embodiment. It is a figure for demonstrating operation | movement of the imaging device in 1st Embodiment. It is a figure which shows an example of the background image in a comparative example, an input image, and a difference image. It is a flowchart which shows an example of the illumination intensity change determination process in the 1st modification of 1st Embodiment. It is a figure for demonstrating the calculation method of the distance in the 1st modification of 1st Embodiment. It is an example of the distribution map in the 1st modification of 1st Embodiment. 10 is a flowchart illustrating an example of an operation of the imaging apparatus according to a second modification of the first embodiment. It is a block diagram which shows the example of 1 structure of the illumination intensity change determination part in 2nd Embodiment. It is a flowchart which shows the illumination intensity change determination process in 2nd Embodiment. It is a flowchart which shows the outlier removal process in 2nd Embodiment. It is an example of the distribution map before and behind the plot selection in 2nd Embodiment. It is an example of the distribution map which shows the plot counted by the outlier removal process in 2nd Embodiment. It is an example of the distribution map before and after the input image and outlier removal in the second embodiment. It is a block diagram which shows the example of 1 structure of the illumination intensity change determination part in 3rd Embodiment. It is an example of the distribution map and histogram in 3rd Embodiment. It is a flowchart which shows the illumination intensity change determination process in 3rd Embodiment. It is a figure for demonstrating operation | movement of the imaging device in 3rd Embodiment.

Hereinafter, modes for carrying out the present technology (hereinafter referred to as embodiments) will be described. The description will be made in the following order.
1. 1st Embodiment (example which determines the presence or absence of illumination intensity change from distribution of difference degree)
2. Second Embodiment (Example of determining presence or absence of illuminance change from distribution of difference obtained by removing outliers)
3. Third embodiment (example in which the mode value is obtained from the distribution of dissimilarities to determine the presence or absence of illuminance change)

<1. First Embodiment>
[Configuration example of imaging device]
FIG. 1 is a block diagram illustrating a configuration example of the imaging apparatus 100 according to the first embodiment. The imaging apparatus 100 includes an imaging unit 110, an image processing unit 120, an exposure control unit 130, a background image storage unit 140, a moving image recording unit 150, and an illuminance change determination unit 200.

  The imaging unit 110 captures a plurality of input images in chronological order according to the exposure parameter setting values supplied from the exposure control unit 130. Here, the exposure parameter is a parameter for controlling the exposure amount (exposure amount), and an aperture value, an exposure time (that is, shutter speed) and the like are assumed. In addition, the imaging unit 110 includes an imaging lens, an optical system such as a diaphragm, and an imaging element. For example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor is used as the imaging element. The imaging unit 110 supplies each of the captured input images to the image processing unit 120 and the illuminance change determination unit 200 via the signal line 119.

  The background image storage unit 140 stores a background image used as a reference in detecting a moving object. As this background image, for example, an image in which an imaging location where a moving object is to be detected is captured in a situation where no moving object exists is used. The background image is an example of a reference image in the claims.

  Here, each of the input image and the background image includes a plurality of pixels arranged in a two-dimensional lattice pattern. Hereinafter, the pixels in the input image are referred to as “input pixels”, and the pixels in the background image are referred to as “background pixels”. Further, it is assumed that the number of pixels and the aspect ratio of the input image and the background image are the same. The input image and the background image may have different pixel numbers and aspect ratios. For example, a configuration in which a background image larger than the input image is stored may be used. In this case, the imaging unit 110 captures a part of the imaging area where the background image is captured and generates an input image.

  The illuminance change determination unit 200 determines whether or not the illuminance change exceeds a predetermined change amount dL when the background image and the input image are captured. The illuminance change determination unit 200 supplies the result of determining the presence or absence of the illuminance change to the exposure control unit 130 and the image processing unit 120 via the signal line 209 as the illuminance change determination result.

  The image processing unit 120 performs predetermined image processing such as moving object detection processing, white balance processing, and demosaic processing on each of the input images. In this moving object detection process, a difference image between the background image and the input image is generated, and a region in which the difference value exceeds a predetermined difference threshold is detected as a moving object. However, the image processing unit 120 stops the moving object detection process when there is a change in illuminance. The image processing unit 120 supplies the input image after the image processing together with the moving object detection result to the moving image recording unit 150 via the signal line 129. The moving image recording unit 150 records a moving image including an input image and a moving object detection result.

  The exposure control unit 130 controls the exposure amount of the imaging unit 110. The exposure control unit 130 controls the exposure amount when there is a change in illuminance, and does not control the exposure amount when there is no change in illuminance. In controlling the exposure amount, the exposure control unit 130 obtains a photometric value from a photodetector (not shown) that measures the peripheral light amount, and whether or not the difference between the photometric value and a predetermined target value exceeds an allowable value. Determine whether. Then, when the difference exceeds a predetermined allowable value, the exposure control unit 130 sets an exposure parameter (aperture value or shutter speed) so that the photometric value becomes a predetermined target value, and the setting value is set to the imaging unit. 110.

  The exposure control unit 130 controls the aperture value and the shutter speed. However, the exposure control unit 130 may control the gain for the pixel value instead of the aperture value, or may control both the aperture value, the shutter speed, and the gain. May be.

  Moreover, the imaging device 100 may further include a display unit, and display an image or a moving image on the display unit. The imaging apparatus 100 may further include an interface, and an image or a moving image may be output to an external device via the interface.

  In addition, the imaging unit 110, the image processing unit 120, the exposure control unit 130, the background image storage unit 140, the moving image recording unit 150, and the illuminance change determination unit 200 are provided in the same device, but these are provided in separate devices. It is good also as a structure to provide. For example, the imaging unit 110 and the exposure control unit 130 may be provided in the imaging device 100, and the image processing unit 120 and the illuminance change determination unit 200 may be provided in the image processing device or the like.

[Example of configuration of illumination change determination unit]
FIG. 2 is a block diagram illustrating a configuration example of the illuminance change determination unit 200 according to the first embodiment. The illuminance change determination unit 200 includes a distance calculation unit 211, a distance comparison unit 212, a counting unit 213, and a count value comparison unit 214.

  The distance calculation unit 211 calculates, for each input pixel, the difference between pixel values (for example, luminance values) between the input pixel and the background pixel corresponding to the input pixel as a distance. Details of the method of calculating the difference will be described later. The distance calculation unit 211 supplies each calculated distance to the distance comparison unit 212. The distance calculation unit 211 is an example of a difference degree acquisition unit described in the claims.

  The distance comparison unit 212 determines whether or not the distance is larger than the distance threshold by comparing the calculated distance (that is, the degree of difference) with the distance and a predetermined distance threshold. The distance comparison unit 212 supplies the determination result for each distance to the counting unit 213.

  The counting unit 213 counts the number of distances larger than the distance threshold based on the determination result from the distance comparison unit 212. The counting unit 213 supplies the count value CNT to the count value comparison unit 214.

  The count value comparison unit 214 compares the count value CNT with a predetermined count value determination threshold value and determines whether or not the count value CNT is larger than the count value determination threshold value. The count value comparison unit 214 supplies the determination result to the exposure control unit 130 and the image processing unit 120 as the illuminance change determination result. The count value comparison unit 214 is an example of a determination unit described in the claims.

  FIG. 3 is a diagram for explaining a method for determining whether or not there is an illuminance change in the first embodiment. In the figure, a is an example of the background image 500, and b in the figure is an example of the input image 501. As illustrated in a and b in the figure, the input image 501 has an overall lower brightness than the background image 500. As a cause of the change in the overall brightness in this way, a change in the illuminance of the light source is assumed.

  C in FIG. 3 is an example of the luminance distribution of the background image 500, and d in the figure is an example of the luminance distribution of the input image 501. In the figure, the horizontal axes of c and d are the coordinates of the pixel, and the vertical axis is the luminance value of the pixel at that coordinate. As illustrated in c and d in the figure, the luminance of the input image is generally lowered with respect to the background image due to the illuminance change. For example, the luminance value I1 of the input pixel at the coordinate P1 is lower than the luminance value B1 of the background pixel at that coordinate. The luminance value I2 of the input pixel at the coordinate P2 is also lower than the luminance value B2 of the background pixel at that coordinate.

  E in FIG. 3 is a distribution diagram showing the relationship between the luminance values of the input pixel and the background pixel. In FIG. 5e, the vertical axis represents the luminance value of the input pixel, and the horizontal axis represents the luminance value of the background pixel. Further, in e of the figure, a white circle is a plot representing the luminance values of the background pixel and the input pixel at the coordinate P1, and a black circle is a plot representing the luminance value of the background pixel and the input pixel at the coordinate P2. is there. The solid line is a straight line passing through a plot corresponding to each of the coordinates P1 and P2. The alternate long and short dash line is a straight line having an inclination of “1” and an intercept of “0”. When plots corresponding to the coordinates other than the coordinates P1 and P2 are added, the plots are concentrated near the solid line. The slope of this solid line is smaller than “1”.

Here, if the gain with respect to the luminance is constant among the luminance, the illuminance, and the reflectance of the subject, the relationship shown in the following equation is established.
(Luminance) = (illuminance) x (reflectance)

From the above equation, if the luminance, illuminance, and reflectance on the input image side are input luminance, illuminance, and input reflectance, and the luminance, illuminance, and reflectance on the background image side are input luminance, illuminance, and input reflectance, The formula holds.
(Input luminance) = (Input illuminance) × (Input reflectivity) Equation 1
(Background luminance) = (Background illuminance) × (Background reflectivity) Equation 2

When the subjects of the input image and the background image are the same, the input reflectance and the background reflectance are the same. In this case, the following equation is obtained from equations 1 and 2.
(Input luminance) = (Input illuminance) / (Background illuminance) × (Background luminance) Equation 3

  From Equation 3, when there is no change in illuminance, the input luminance is the same value as the background luminance. When there is a change in illuminance, the ratio of the input luminance to the background luminance depends on the rate of change in illuminance. Value. Therefore, when there is no change in the illuminance of the light source, the plots corresponding to the respective coordinates are concentrated in the vicinity of the straight line having the inclination “1”. On the other hand, when there is a change in illuminance, the plot corresponding to each coordinate is concentrated near a straight line with a slope corresponding to the change rate of illuminance (input illuminance / background illuminance).

The imaging apparatus 100 obtains, for each coordinate, a distance D between a plot corresponding to the coordinate and a straight line having an inclination “1”. Here, on the xy plane, a distance D between a straight line of ax + by + c = 0 (a, b, and c are real numbers) and a point of coordinates (x ₀ , y ₀ ) is obtained by the following equation.
D = abs (ax ₀ + by ₀ + c) / (a ² + b ² ) ^1/2 Formula 4

In the above formula, abs () is a function that returns an absolute value. When the slope of the straight line is 1 and the intercept is 0, a = 1, b = −1, and c = 0. Substituting these into the above equation yields:
D = abs (x ₀ −y ₀ ) / 2 ^1/2 Formula 5

Therefore, if the x-axis is the background luminance and the y-axis is the input luminance, the distance of the background pixel is substituted for x ₀ in equation 5 and the luminance of the input pixel corresponding to the background pixel is substituted for y _0. D is calculated. This distance D indicates the degree of difference between the pixel values of the corresponding input pixel and background pixel. When the change in illuminance is small, there are many plots with a small distance D (difference), and when the change in illuminance is large, there are many plots with a large distance D (difference). For this reason, the imaging apparatus 100 can determine the presence or absence of an illuminance change exceeding the predetermined change amount dL from the number of plots of the distance D smaller than the distance threshold (that is, the count value CNT).

  Although the value obtained by Expression 5 is used as the degree of difference, the value obtained by another expression may be used as the degree of difference as long as the value indicates the degree of difference in pixel value. For example, the imaging apparatus 100 may obtain the ratio of the pixel values of the input pixel and the background pixel as the degree of difference.

  Further, c, d, and e in FIG. 3 are described for convenience of explanation, and these drawings are not actually generated in the imaging apparatus 100. Actually generated data is shown by f in FIG.

  In FIG. 3, f is a diagram showing the distribution of the distance D for each coordinate. The vertical axis of f in the figure is the distance D, and the horizontal axis is the coordinate. The imaging device 100 calculates the distance D for each coordinate as illustrated in f in the figure, and determines the presence or absence of illuminance change from the distribution state. For example, the imaging apparatus 100 counts the number of distances D smaller than the distance threshold, and determines that there is no illuminance change when the count value is greater than the count value threshold, and determines that there is illuminance change otherwise. As the distance threshold, a larger value is set as the change amount dL that is a threshold related to illuminance is larger. The count value determination threshold is set to a larger value as the number of pixels of the input image is larger.

[Configuration example of image processing unit]
FIG. 4 is a block diagram illustrating a configuration example of the image processing unit 120 according to the first embodiment. The image processing unit 120 includes a white balance processing unit 121, a demosaic processing unit 122, and a moving object detection unit 123.

  The white balance processing unit 121 performs processing for adjusting white balance on the input image. The white balance processing unit 121 supplies the processed input image to the demosaic processing unit 122.

  The demosaic processing unit 122 performs demosaic processing for interpolating missing color data for each pixel on the input image. The demosaic processing unit 122 supplies the processed input image to the moving image recording unit 150.

  The moving object detection unit 123 detects a moving object in the input image. However, when there is a change in illuminance, the moving object detection unit 123 stops the moving object detection process. When there is no illuminance change, the moving object detection unit 123 generates a difference image between the input image and the background image, and detects a region in the difference image where the difference value exceeds a predetermined difference threshold as a moving object. Then, the moving object detection unit 123 supplies the moving image recording unit 150 with a moving object detection result including a flag indicating the presence or absence of the moving object and information indicating the area of the moving object. In addition, although the moving body detection part 123 has detected the moving body about the input image before demosaicing, you may detect a moving body in the input image after demosaicing. In this case, the demosaiced image is used as the background image. Further, the illuminance change determination unit 200 also determines whether or not there is a change in illuminance from the demosaiced input image and the background image.

  Note that the image processing unit 120 may further perform image processing other than white balance processing and demosaic processing, such as gamma correction processing and noise removal processing. The image processing unit 120 may output the input image without performing at least one of the white balance process and the demosaic process.

  FIG. 5 is a diagram illustrating an example of a background image, an input image, and a difference image according to the first embodiment. In the figure, a is an example of the background image 505, and b in the figure is an example of the input image 506. As illustrated in a and b in the figure, there is no change in the illuminance of the background image 505 and the input image 506, and a human 507 not reflected in the background image 505 is reflected in the input image 506.

  C in FIG. 5 is an example of a difference image 508 between the background image 505 and the input image 506. In the difference image 508, a white area 509 indicates an area where the difference value exceeds the difference threshold value, and a black area indicates an area where the difference value is equal to or less than the difference threshold value. An area 509 corresponding to a person 507 in the input image 506 is white in the difference image 508 as illustrated in c in FIG. A region in the input image 506 corresponding to this region 509 is detected as a moving object. In this way, a method for detecting a moving object from a difference image between an input image and a background image is called a background difference method.

[Operation example of imaging device]
FIG. 6 is a flowchart illustrating an example of the operation of the imaging apparatus 100 according to the first embodiment. The process shown in this flowchart is executed every time an input image is captured. The imaging apparatus 100 executes an illuminance change determination process for determining the presence or absence of an illuminance change (step S910). Then, the imaging apparatus 100 determines whether or not it is determined that the illuminance change has occurred in the illuminance change determination process (step S901).

  When there is no change in illuminance (step S901: No), the imaging apparatus 100 detects a moving object in the input image using the background difference method (step S902). On the other hand, when there is a change in illuminance (step S901: Yes), the imaging apparatus 100 executes an exposure control process for controlling the exposure amount (step S950). After step S902 or S950, the imaging apparatus 100 ends the operation.

  FIG. 7 is a flowchart showing the illuminance change determination process in the first embodiment. The imaging apparatus 100 initializes the count value CNT to an initial value (for example, “0”) (step S911).

  Here, the horizontal coordinates of the pixels in the input image and the background image are 0 to W-1, and the vertical coordinates are 0 to H-1. The imaging apparatus 100 pays attention to the pixels from the pixel at the coordinates (0, 0) to the coordinates (W-1, H-1) in order, and executes the following steps S913 to S915 for the focused pixels (step S912). ).

  First, the imaging apparatus 100 calculates the distance D between the plot corresponding to the coordinate of the pixel of interest and the straight line with the inclination “1” using Equation 5 (step S913). The imaging apparatus 100 determines whether or not the calculated distance D is equal to or less than the distance threshold (step S914). When the distance D is less than or equal to the distance threshold (step S914: Yes), the imaging device 100 increments the count value CNT (step S915).

  After performing steps S913 to S915 for all coordinates, the imaging apparatus 100 determines whether or not the count value CNT is greater than the count value determination threshold (step S916). When the count value CNT is larger than the count value determination threshold (step S916: Yes), the imaging apparatus 100 determines that there is no illuminance change and sets “0” in the illuminance change determination flag (step S917). On the other hand, when the count value CNT is less than or equal to the count value determination threshold (step S916: No), the imaging apparatus 100 determines that there is a change in illuminance and sets “1” in the illuminance change determination flag (step S918). . After step S917 or S918, the imaging apparatus 100 ends the illuminance change determination process.

  FIG. 8 is a flowchart showing the exposure control process in the first embodiment. The imaging apparatus 100 acquires a photometric value (step S951), and determines whether or not the difference between the target value and the photometric value is within an allowable value (step S952). When the difference is not within the allowable value (step S952: No), the imaging apparatus 100 determines whether or not the photometric value is smaller than the target value (step S953).

  When the photometric value is smaller than the target value (step S953: Yes), the imaging apparatus 100 sets the exposure parameter to a value at which the image becomes bright (step S954). When the photometric value is equal to or greater than the target value (step S953: No), the imaging apparatus 100 sets the exposure parameter to a value at which the image becomes dark (step S955). After step S954 or S955, the imaging apparatus 100 controls the aperture and shutter speed according to the set value of the exposure parameter (step S956). When the difference between the target value and the photometric value is within the allowable value (step S952: Yes) or after step S956, the imaging apparatus 100 ends the exposure control process.

  Note that the imaging apparatus 100 may control the gain for the pixel value instead of the aperture value or the like, or may control both the aperture value, the shutter speed, and the gain.

  FIG. 9 is a diagram for explaining the operation of the imaging apparatus 100 according to the first embodiment. In the figure, a is an example of the background image 510, and b in the figure is an example of the input image 511. As illustrated in a and b in the figure, the input image 511 has almost no change in illuminance with respect to the background image 510, and no moving object is shown.

  FIG. 9C is a distribution diagram showing the relationship between the luminance values of the input pixels in the input image 511 and the background pixels in the background image 510. Since there is almost no change in illuminance, the luminance values of the input pixel and the background pixel are approximately the same at each coordinate, and each plot is concentrated in the vicinity of a straight line with a slope of “1”. For this reason, the distance D of each coordinate becomes a value close to “0”, and the number of distances D equal to or smaller than the distance threshold is larger than the count value determination threshold. In this case, the imaging apparatus 100 determines that there is no change in illuminance, and performs moving object detection.

  9 is a diagram illustrating an example of a difference image 512 between the input image 511 and the background image 510. Since there is no region in which the difference value exceeds the difference threshold in the difference image 512, it is determined that there is no moving object.

  E in FIG. 9 is an example of the input image 513. As illustrated in a and e in the figure, an illuminance change occurs in the input image 513 with respect to the background image 510, and no moving object is shown.

  In FIG. 9, f is a distribution diagram showing the relationship between the luminance values of the input pixels in the input image 513 and the background pixels in the background image 510. Since there is a change in illuminance, the luminance values of the input pixel and the background pixel are different values at each coordinate, and the plots corresponding to the coordinates are concentrated in the vicinity of a straight line with a slope corresponding to the change rate of the illuminance. For this reason, the distance D of each coordinate often becomes a value larger than “0”, and the number of distances D equal to or smaller than the distance threshold is smaller than the count value determination threshold. In this case, the imaging apparatus 100 determines that the illuminance has changed, stops moving object detection, and performs exposure control.

  9 is a diagram illustrating an example of the input image 514 after exposure control. Due to the exposure control, the brightness of the input image 514 is approximately the same as that of the background image 510.

  In FIG. 9, h is an example of the input image 515. As illustrated in a and h in the figure, the input image 515 has no change in illuminance with respect to the background image 510, and a person 516 is shown.

  I in FIG. 9 is a distribution diagram showing the relationship between the luminance values of the input pixels in the input image 515 and the background pixels in the background image 510. Since there is no change in illuminance, the luminance values of the input pixel and the background pixel are approximately the same at each coordinate, and each plot is concentrated in the vicinity of the straight line having the slope “1”. For this reason, the distance D of each coordinate becomes a value close to “0”, and the number of distances D equal to or smaller than the distance threshold is larger than the count value determination threshold. In this case, the imaging apparatus 100 determines that there is no change in illuminance, and performs moving object detection.

  9 is a diagram illustrating an example of a difference image 517 between the input image 515 and the background image 510. The difference image 517 includes an area 518 whose difference value exceeds the difference threshold, and an area corresponding to this area is detected as a moving object in the input image 515.

  As described above, the imaging apparatus 100 can accurately determine the presence or absence of the illuminance change based on whether or not the distance D (degree of difference) of each pixel is concentrated in the vicinity of “0”. Note that the imaging apparatus 100 can also determine the presence or absence of a change in illuminance by comparing the average value of the luminance of the input image and the average value of the luminance of the background image. There is a risk of misjudgment. For example, in the input image 515, although there is no change in illuminance, the average value of luminance is different from that in the input image 510 due to the presence of the moving object 516. For this reason, there is a possibility that it is erroneously determined that there has been a change in illuminance. On the other hand, the distribution state of whether or not the distance D is concentrated in the vicinity of “0” does not change much even in the input image 515 in which the moving object is shown. For this reason, the imaging apparatus 100 can accurately determine the presence or absence of an illuminance change from the count value CNT indicating the distribution state.

  Note that FIG. 9 illustrates processing for an input image (an input image 511, an input image 513, and an input image 515) that satisfies at least one of the fact that no change in illuminance has occurred and that no moving object is captured. However, it is also assumed that an input image with a change in illuminance and a moving object is captured. In that case, only the exposure control is performed similarly to the input image 513, and the moving object detection is not performed. With this exposure control, it is determined that there is no illuminance change in the next input image to be picked up, so that moving object detection is performed. Therefore, even in this case, the moving object is accurately detected.

  FIG. 10 is a diagram illustrating an example of the background image, the input image, and the difference image in the comparative example. In this comparative example, it is assumed that the imaging apparatus 100 performs moving object detection every time an input image is captured regardless of whether or not the illuminance changes. In the figure, a is an example of the background image 520, and b in the figure is an example of the input image 521. As illustrated in a and b in the figure, an illuminance change occurs in the input image 521 with respect to the background image 520, and no moving object is shown.

  C in FIG. 10 is an example of the difference image 522 between the background image 520 and the input image 521. In the difference image 522, regions 523 and 524 in which the luminance difference is greater than or equal to the threshold value are generated due to the illuminance change. In this comparative example, these areas are erroneously detected as moving objects. On the other hand, since the imaging apparatus 100 does not detect moving objects when there is a change in illuminance, such erroneous detection can be prevented.

  As described above, according to the first embodiment of the present technology, the imaging device 100 determines whether or not there is an illuminance change from the distribution state of the pixel value dissimilarity, and stops detecting a moving object if there is an illuminance change. Therefore, it is possible to prevent erroneous detection of a moving object when the illuminance changes. Thereby, the detection precision of a moving body can be improved without being influenced by the change of illumination.

[First Modification]
In the first embodiment described above, the difference in luminance value is detected as the distance D (difference) from Equation 5, but when the change rate of illuminance is constant, the input luminance is lower than Equation 4 and Equation 5. The closer the distance D is, the higher the risk that it will be below the distance threshold. For this reason, when the input image is relatively dark, there is a high possibility that it is erroneously determined that there is no illuminance change despite the illuminance change. If the logarithmic difference in luminance is obtained as the degree of difference, the distance D becomes sufficiently large even if the input luminance is low when there is a change in illuminance, and such erroneous determination can be prevented. The imaging apparatus 100 according to the first modified example is different from the first embodiment in that the difference between the logarithm of the pixel value of the input pixel and the background pixel is obtained as the degree of difference.

The imaging apparatus 100 according to the modified example obtains the distance D using the following equation instead of Equation 5.
D = abs {(log (x ₀ ) −log (y ₀ )} / 2 ^1/2 Formula 6

  FIG. 11 is a flowchart illustrating an example of illuminance change determination processing according to the first modification of the first embodiment. The illumination intensity change determination process of the modification is different from the first embodiment in that step S919 is executed instead of step S913.

  In step S <b> 919, the imaging apparatus 100 calculates a distance D between the plot corresponding to the coordinates (m, n) and the straight line with the inclination “1” on the log scale using Equation 6.

  FIG. 12 is a diagram for explaining a calculation method of the distance D in the first modification example of the first embodiment. A in the same figure is a figure which shows the relationship between the luminance value of each of an input pixel and a background pixel at the time of illuminance change. In the figure, the vertical axis of a is the luminance value of the input pixel, and the horizontal axis is the luminance value of the background pixel. When there is a change in illuminance, plots such as the plot 531 and the plot 532 are concentrated in the vicinity of a straight line with a slope corresponding to the change rate of the illuminance. The plot 532 has a relatively high input luminance, and therefore the distance D is relatively larger than in Equation 5. For this reason, the distance D is larger than the distance threshold. On the other hand, the plot 531 has a relatively low input luminance, and therefore the distance D is relatively small from Equation 5. For this reason, there exists a possibility that the distance D may become below a distance threshold value.

  Thus, since the distance D becomes smaller as the input luminance is lower in Equation 5, when the input image is relatively dark, there is a possibility that it is erroneously determined that there is no illuminance change despite the illuminance change. Get higher.

  On the other hand, b in FIG. 12 is a diagram illustrating a logarithmic relationship between the luminance values of the input pixel and the background pixel when the illuminance changes. In the figure, the vertical axis of b is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. When there is a change in illuminance, plots such as plot 533 and plot 534 are concentrated in the vicinity of a straight line with a slope corresponding to the change rate of illuminance. The plots 533 and 534 have different luminances, but in either case, the distance D is approximately the same as in Equation 6.

  As described above, in Expression 6 for calculating the logarithmic difference, the distance D does not change even when the input luminance is low. Therefore, even when the input image is relatively dark, it is possible to accurately determine whether or not the illuminance has changed.

  FIG. 13 is an example of a distribution diagram in the first modification of the first embodiment. A in the same figure is a figure which shows the relationship between each luminance value of an input pixel and a background pixel when the illumination intensity of a part in an image changes. In the figure, the vertical axis of a is the luminance value of the input pixel, and the horizontal axis is the luminance value of the background pixel. Moreover, the plot in the area | region enclosed with the dotted line is a plot corresponding to the area | region where illumination intensity changed. Since the distance D of these plots becomes smaller as the input luminance is lower, it becomes difficult to perform separation based on the distance threshold.

  B in FIG. 13 is a diagram illustrating a logarithmic relationship between the luminance values of the input pixels and the background pixels when the illuminance of a part of the image changes. In the figure, the vertical axis of b is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. Moreover, the plot in the area | region enclosed with the dotted line is a plot corresponding to the area | region where illumination intensity changed. Since the distance D of these plots does not change much even when the input luminance is low, it can be easily separated by the distance threshold.

  As described above, according to the first modification example of the first embodiment, the imaging apparatus 100 determines whether or not there is a change in illuminance from the distribution state of the logarithmic difference of pixel values. It is possible to suppress erroneous determination of illuminance change in the dark.

[Second Modification]
In the first embodiment described above, detection of a moving object is stopped if there is a change in illuminance, but detection of a moving object may be performed regardless of whether or not there is a change in illuminance. For example, when a slow illuminance change occurs, it is not necessary to stop the moving object detection. The imaging apparatus 100 according to the second modification of the first embodiment is different from the first embodiment in that a moving object is detected regardless of whether or not the illuminance changes.

  FIG. 14 is a flowchart illustrating an example of the operation of the imaging apparatus 100 according to the second modification of the first embodiment. When there is a change in illuminance (step S901: Yes), the imaging apparatus 100 according to the second modification performs an exposure control process (step S950). Further, when there is no change in illuminance (step S901: No), or after step S950, the moving object is detected (step S902). In the second modification, the exposure amount is controlled to an appropriate value based on the detection result of the presence / absence of illuminance change, so that the moving object detection accuracy is improved.

  As described above, according to the second modification example of the first embodiment, the imaging apparatus 100 detects a moving object regardless of whether or not the illuminance changes, so that even if the illuminance change occurs, the moving object is detected. Can be detected.

<2. Second Embodiment>
In the first embodiment described above, the presence / absence of a change in illuminance is determined using all the differences in luminance values. However, it is not necessary to use all the differences. When a moving object appears in the input image, the plot in the moving object often has a value greatly deviating from other values. If there are many such outliers, it is difficult to accurately determine whether or not the distance D is concentrated in the vicinity of “0”, and the imaging apparatus 100 may erroneously determine whether there is a change in illuminance. Therefore, it is desirable to obtain the distance D after removing the outlier. The imaging apparatus 100 according to the second embodiment is different from the first embodiment in that the distance D is obtained after removing the outlier.

  FIG. 15 is a block diagram illustrating a configuration example of the illuminance change determination unit 200 according to the second embodiment. The illuminance change determination unit 200 of the second embodiment is different from the first embodiment in that it further includes an outlier removal unit 221.

  The outlier removal unit 221 removes a plot including an outlier from a set of plots each composed of the pixel values of the corresponding input pixel and background pixel. In the removal of outliers, for example, a RANSAC (RANdom SAmple Consensus) algorithm is used. Details of the RANSAC algorithm will be described later. The outlier removal unit 221 supplies the remaining plot from which the outlier has been removed to the distance calculation unit 211.

  FIG. 16 is a flowchart illustrating an illuminance change determination process according to the second embodiment. The illuminance change determination process of the second embodiment is different from the first embodiment in that an outlier removal process (step S920) for removing an outlier is performed before the count value CNT is initialized. .

  Also in the second embodiment, as in the modification example, the imaging apparatus 100 may acquire a logarithmic difference of luminance values as a degree of difference.

  FIG. 17 is a flowchart illustrating an example of outlier removal processing according to the second embodiment. In this process, the RANSAC algorithm is used. The imaging apparatus 100 repeats steps S922 to S925 for a certain number of loops (for example, 30).

The imaging apparatus 100 selects two plots at random (step S922), and acquires a straight line _LN passing through these plots (step S923). Here, the subscript N is, for example, an integer from 0 to 29. The imaging apparatus 100 calculates the distance from the straight line _LN by Expression 4 for each plot (step S924). Then, the imaging apparatus 100 counts the number of plots _PN whose distance is equal to or less than a certain value (step S925).

Step S922 to certain number of times to S925, after repeating, the imaging device 100, _{P N} selects the largest linear _{L N} (step S926). Then, the imaging apparatus 100 obtains the distance from the selected _LN by Expression 4, and removes a plot in which the distance exceeds a certain value as an outlier plot (step S927).

Note that the imaging apparatus 100 calculates the distance from Expression 4 in steps S924 and S927, but instead of Expression 4, the distance may be calculated by the following expression.
D = abs (a × log ( x 0) + b × log (y o) + c) / (a 2 + b 2) 1/2 ··· Equation 7

  In addition, the imaging apparatus 100 uses the RANSAC algorithm for removing outliers, but an algorithm other than the RANSAC algorithm, such as an algorithm in the least square method or the M estimation method, may be used. When using the least square method or the M estimation method, the imaging apparatus 100 uses these to obtain a straight line ax + by = 0 that best fits the distribution of the plot, and uses a plot whose distance from the straight line exceeds a certain value as an outlier. Remove it.

FIG. 18 is an example of a distribution diagram before and after selecting a plot in the second embodiment. In the figure, a is an example of a distribution chart before plot selection, and b in the figure is an example of a distribution chart after plot selection. The vertical axis of a and b in the figure is the luminance value of the input pixel, and the horizontal axis is the luminance value of the background pixel. In addition, two black circles b in FIG. 6 are randomly selected plots. The imaging apparatus 100 acquires a straight line _LN that passes through these two plots.

FIG. 19 is an example of a distribution diagram showing plots counted in the outlier removal processing in the second embodiment. In the figure, the vertical axis represents the luminance value of the input pixel, and the horizontal axis represents the luminance value of the background pixel. A black circle is a plot that is selected and counted. Open circles are counted plots, and hatched circles are uncounted plots. As illustrated in the figure, the imaging apparatus 100 counts the number P _N of plots whose distance from the straight line L _N is within a certain value.

The procedure described with reference to FIGS. 18 and 19 is performed a predetermined number of times. The imaging device 100 selects the P _N is highest linear L _N, the distance from the selected L _N is removed as a plot outlier plots exceeds a certain value.

  FIG. 20 is an example of an input image and a distribution diagram before and after outlier removal in the second embodiment. In the figure, a is a diagram showing an example of the input image 551. This input image 551 does not show a moving object, and changes in illuminance occur. B in the figure is a distribution diagram showing the logarithmic relationship between the luminance values of the input pixels and the background pixels of the input image 551 before outlier removal. C in the figure is a distribution diagram showing the logarithmic relationship between the luminance values of the input pixels and background pixels of the input image 551 after outlier removal. The vertical axis of b and c in the figure is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. As illustrated in b and c in the figure, the outlier due to the influence of the illuminance change is sufficiently removed. For this reason, the imaging apparatus 100 can accurately determine whether or not the distance D of each plot is concentrated in the vicinity of “0”.

  In the figure, d is a diagram showing an example of the input image 552. This input image 552 shows a moving object, and there is no illuminance change. In the figure, e is a distribution diagram showing the logarithmic relationship between the luminance values of the input pixels and the background pixels of the input image 552 before outlier removal. F in the figure is a distribution diagram showing the logarithmic relationship between the luminance values of the input pixels and background pixels of the input image 552 after outlier removal. In the figure, the vertical axis of e and f is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. As illustrated in e and f in the figure, the outlier due to the influence of the moving object is sufficiently removed. For this reason, the imaging apparatus 100 can accurately determine whether or not the distance D of each plot is concentrated in the vicinity of “0”.

  Thus, according to the second embodiment of the present technology, since the imaging apparatus 100 calculates the distance D (dissimilarity) after removing the outlier plot, the distance D of each plot is “0”. It can be accurately determined whether or not it is concentrated in the vicinity. Thereby, a moving body can be detected accurately.

<3. Third Embodiment>
In the first embodiment described above, the imaging apparatus 100 counts the number of distances D that are equal to or less than the distance threshold value near “0”, and the distance D is based on the comparison result between the count value CNT and the count value determination threshold value. Has been determined to be concentrated around “0”. However, the imaging apparatus 100 can also obtain the mode value of the distance D and obtain the distribution state of the distance D from the comparison result between the mode value and a predetermined mode value determination threshold. In the configuration for obtaining the count value CNT, the count value CNT increases as the number of pixels of the image increases. Therefore, it is necessary to change the count value determination threshold according to the number of pixels. On the other hand, since the mode value of the distance D does not depend on the number of pixels, it is not necessary to change the mode value determination threshold according to the number of pixels in the method for obtaining the mode value. The imaging apparatus 100 according to the third embodiment is different from the first embodiment in that the mode value of the distance D is obtained and the distribution state of the distance D is obtained from the mode value.

  FIG. 21 is a block diagram illustrating a configuration example of the illuminance change determination unit 200 according to the third embodiment. The illuminance change determination unit 200 according to the third embodiment includes a histogram generation unit 231, a mode value acquisition unit 232, and a mode value comparison unit 233 instead of the distance comparison unit 212, the count unit 213, and the count value comparison unit 214. This is different from the first embodiment in that

The distance calculation unit 211 according to the third embodiment calculates a distance according to the following equation instead of the equation 5 and supplies the calculated distance to the histogram generation unit 231.
D = {log (x ₀ ) −log (y ₀ )} / 2 ^1/2 Formula 8

The distance calculation unit 211 may calculate the distance D by the following equation instead of the equation 8.
D = (x ₀ −y ₀ ) / 2 ^1/2 Formula 9

  For each distance D, the histogram generation unit 231 generates a histogram indicating the number of times the distance is calculated as a frequency.

  The mode value acquisition unit 232 acquires a distance having the largest frequency in the histogram as the mode value.

  The mode value comparison unit 233 compares the absolute value of the mode value with a predetermined mode value determination threshold value and determines whether or not the mode value is larger than the mode value determination threshold value. As the mode value determination threshold, a larger predetermined real number is set as the change amount dL is larger. This mode value comparison unit 233 outputs the comparison result as an illuminance change determination result.

  FIG. 22 is an example of a distribution diagram and a histogram in the third embodiment. In the figure, a is an example of a distribution diagram showing the relationship between the luminance values of the input pixel and the background pixel when there is a change in illuminance. In the figure, the vertical axis a represents the luminance value of the input pixel, and the horizontal axis represents the luminance value of the background pixel.

  Further, b in FIG. 22 is an example of a distribution diagram showing a logarithmic relationship between the luminance values of the input pixel and the background pixel when there is a change in illuminance. The vertical axis of b in the figure is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. As illustrated in a and b in the figure, when there is a change in illuminance, the plot concentrates in the vicinity of a straight line having a slope different from “1”. When these plots are projected on a straight line having a slope “−1” and an intercept “0”, a histogram indicating the frequency for each distance D is obtained.

  C in FIG. 22 is an example of a histogram indicating the frequency for each distance D when there is a change in illuminance. The vertical axis of c in the figure is the frequency, and the horizontal axis is the distance D. In this histogram, the imaging apparatus 100 obtains the distance D having the highest frequency as the mode value. When there is a change in illuminance, the distance D does not concentrate in the vicinity of “0”, so the mode value of the distance D is a value away from 0.

  FIG. 23 is a flowchart illustrating an illuminance change determination process according to the third embodiment. The imaging apparatus 100 calculates the distance D between the plot corresponding to the coordinates (m, n) and the straight line with the inclination “1” on the log scale with Expression 8 for all plots (step S919).

  The imaging apparatus 100 generates a histogram indicating the frequency for each distance D (step S930), and obtains the distance D having the highest frequency as the mode value in the histogram (step S931).

  The imaging apparatus 100 determines whether or not the mode value is equal to or less than the mode value determination threshold (step S932). When the mode value is equal to or less than the mode value determination threshold (step S932: Yes), the imaging apparatus 100 sets “0” in the illuminance change determination flag (step S917). On the other hand, when the mode value is larger than the mode value determination threshold (step S932: No), the imaging apparatus 100 sets “1” in the illuminance change determination flag (step S918). After step S917 or S918, the imaging apparatus 100 ends the illuminance change determination process.

  FIG. 24 is a diagram for explaining the operation of the imaging apparatus 100 according to the third embodiment. A in the figure is an example of an input image 561 in which there is no change in illuminance and no moving object is shown. B in the figure is an example of a distribution diagram showing a logarithmic relationship between the luminance values of the input pixels and the background pixels of the input image 561. The vertical axis of b in the figure is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. As illustrated in b in the figure, since there is no illuminance change, the distance D is concentrated in the vicinity of “0”.

  C in FIG. 24 is an example of a histogram generated from the input image 561 and the background image. In the figure, the vertical axis of c is the frequency, and the horizontal axis is the distance D. In c in the figure, the distance D (mode) with the highest frequency is “0”. Since the mode value is equal to or less than the mode value determination threshold, moving object detection is performed in the input image 561. In the figure, d is an example of the input image 561 and the background image difference image 562. In the difference image 562, there is no region where the difference is equal to or greater than the threshold value, and no moving object is detected.

  E in FIG. 24 is an example of an input image 563 that has a change in illuminance and does not show a moving object. F in the figure is an example of a distribution diagram showing the logarithmic relationship between the luminance values of the input pixels and the background pixels of the input image 563. In the figure, the vertical axis of f is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. As illustrated in f in the figure, since the illuminance has changed, the distance D does not concentrate in the vicinity of “0”.

  24 is an example of a histogram generated from the input image 563 and the background image. In the figure, the vertical axis of g is the frequency, and the horizontal axis is the distance D. In g in the figure, the distance D (mode) with the highest frequency is larger than “0” and exceeds the mode value determination threshold. Since the mode value is larger than the mode value determination threshold, the moving object is not detected in the input image 563, and exposure control is performed. In the figure, h is an example of the input image 564 after exposure control.

  In FIG. 24, i is an example of an input image 565 in which there is no change in illuminance and a person 566 is captured. In the figure, j is an example of a distribution diagram showing the logarithmic relationship between the luminance values of the input pixels and the background pixels of the input image 565. In the figure, the vertical axis of j is the logarithm of the luminance value of the input pixel, and the horizontal axis is the logarithm of the luminance value of the background pixel. As illustrated in j in the figure, since there is no illuminance change, the distance D is concentrated in the vicinity of “0”.

  24 is an example of a histogram generated from the input image 565 and the background image. In the figure, the vertical axis of k is the frequency, and the horizontal axis is the distance D. In k in the figure, the distance D (mode) with the highest frequency is “0”. Since the mode value is equal to or less than the mode value determination threshold, moving object detection is performed in the input image 565. In the figure, h is an example of the input image 565 and the background image difference image 567. In the difference image 567, an area 568 corresponding to the human 566 is detected as a moving object.

  As described above, according to the third embodiment of the present technology, the imaging apparatus 100 determines whether or not there is a change in illuminance depending on whether or not the mode of the distance D is equal to or less than a threshold value. The moving object can be accurately detected in the input image.

  The above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the invention-specific matters in the claims have a corresponding relationship. Similarly, the invention specific matter in the claims and the matter in the embodiment of the present technology having the same name as this have a corresponding relationship. However, the present technology is not limited to the embodiment, and can be embodied by making various modifications to the embodiment without departing from the gist thereof.

  Further, the processing procedure described in the above embodiment may be regarded as a method having a series of these procedures, and a program for causing a computer to execute these series of procedures or a recording medium storing the program. You may catch it. As this recording medium, for example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disc), a memory card, a Blu-ray disc (Blu-ray (registered trademark) Disc), or the like can be used.

  Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

In addition, this technique can also take the following structures.
(1) Each time an input image in which input pixels are arranged is input, each pixel value of the reference pixel and the input pixel corresponding to the reference pixel in the input image and the reference image in which the reference pixels are arranged A difference degree acquisition unit for acquiring the difference degree for each input pixel;
An image processing apparatus comprising: a determination unit that determines whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on a distribution state of the difference for each input pixel; .
(2) The image processing apparatus according to (1), further including a moving object detection unit that detects a moving object for the input image.
(3) The image processing apparatus according to (2), wherein the determination unit stops the detection of the moving object when the illuminance changes.
(4) The moving object detection unit generates a difference image between the input image and the reference image, and detects, as the moving object, a region having a difference larger than a predetermined difference threshold in the difference image. The image processing apparatus described.
(5) further comprising a counting unit that counts the number of the dissimilarities of values within a predetermined range in the distribution and outputs a count value;
The image processing apparatus according to any one of (1) to (4), wherein the determination unit determines the presence or absence of the illuminance change based on whether or not the count value exceeds a predetermined count value determination threshold.
(6) A mode value obtaining unit for obtaining a mode value of the dissimilarity in the distribution is further provided,
The image processing apparatus according to any one of (1) to (5), wherein the determination unit determines the presence or absence of the illuminance change based on whether or not the mode value is within a predetermined range.
(7) further comprising an outlier removal unit that removes a set including an outlier from a set of sets of pixel values of the reference pixel and the input pixel corresponding to the reference pixel;
The image processing apparatus according to any one of (1) to (6), wherein the dissimilarity acquisition unit acquires the dissimilarity in the set from which the set including the outlier is removed.
(8) The difference degree acquisition unit acquires the difference between the logarithm of the pixel value of the reference pixel and the logarithm of the pixel value of the input pixel as the difference degree, according to any one of (1) to (7). Image processing apparatus.
(9) an imaging unit that captures an input image in which input pixels are arranged in time-series order;
Each time the input image is input, the difference between the pixel values of the reference pixel and the input pixel corresponding to the reference pixel in the input image and the reference image in which the reference pixels are arranged is determined for each input pixel. A difference degree acquisition unit to be acquired
An imaging apparatus comprising: a determination unit that determines whether or not there is an illuminance change that exceeds a predetermined change amount when each of the reference image and the input image is captured based on a distribution state of the difference for each input pixel.
(10) The imaging apparatus according to (9), further including an exposure control unit that controls an exposure amount of the imaging unit to a predetermined amount when there is no change in the illuminance.
(11) Each time the input image in which the input pixels are arranged is input, the difference degree acquisition unit includes the reference pixel and the input pixel corresponding to the reference pixel in the reference image in which the reference pixel is arranged. A difference degree acquisition procedure for acquiring a difference degree of each pixel value for each of the input pixels;
A determination procedure for determining whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on the distribution state of the difference for each of the input pixels. Image processing method.
(12) Each time the input image in which the input pixels are arranged is input, the difference degree acquisition unit includes the reference pixels and the input pixels corresponding to the reference pixels in the reference image in which the reference pixels are arranged. A difference degree acquisition procedure for acquiring a difference degree of each pixel value for each of the input pixels;
A determination procedure in which the determination unit determines whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on the distribution state of the difference for each input pixel. A program to make it run.

DESCRIPTION OF SYMBOLS 100 Imaging device 110 Imaging part 120 Image processing part 121 White balance processing part 122 Demosaic processing part 123 Moving body detection part 130 Exposure control part 140 Background image memory | storage part 150 Movie recording part 200 Illuminance change determination part 211 Distance calculation part 212 Distance comparison part 213 Count unit 214 Count value comparison unit 221 Outlier removal unit 231 Histogram generation unit 232 Mode value acquisition unit 233 Mode value comparison unit

Claims (11)

Each time an input image in which input pixels are arranged is input, the difference in pixel value between the reference pixel and the input pixel corresponding to the reference pixel in the input image and the reference image in which the reference pixels are arranged For each input pixel,
A determination unit that determines whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on a state of distribution of the difference for each input pixel ;
The image processing apparatus acquires the difference between the logarithm of the pixel value of the reference pixel and the logarithm of the pixel value of the input pixel as the dissimilarity . The image processing apparatus according to claim 1, further comprising a moving object detection unit configured to detect a moving object for the input image. The image processing apparatus according to claim 2, wherein the determination unit stops the detection of the moving object when the illuminance change occurs.   The image processing apparatus according to claim 2, wherein the moving object detection unit generates a difference image between the input image and the reference image and detects an area in the difference image where the difference is greater than a predetermined difference threshold as the moving object. A counter that counts the number of the dissimilarities of values within a predetermined range in the distribution and outputs a count value;
The image processing apparatus according to claim 1, wherein the determination unit determines the presence or absence of the illuminance change based on whether or not the count value exceeds a predetermined count value determination threshold. A mode value obtaining unit for obtaining a mode value of the dissimilarity in the distribution;
The image processing apparatus according to claim 1, wherein the determination unit determines the presence or absence of the illuminance change based on whether or not the mode value is within a predetermined range. An outlier removal unit that removes a set including an outlier from a set of pixel values of the reference pixel and the input pixel corresponding to the reference pixel;
The image processing apparatus according to claim 1, wherein the dissimilarity acquisition unit acquires the dissimilarity in the set from which the set including the outlier is removed. An imaging unit for imaging an input image in which input pixels are arranged in time-series order;
Each time the input image is input, the difference between the pixel values of the reference pixel and the input pixel corresponding to the reference pixel in the input image and the reference image in which the reference pixels are arranged is determined for each input pixel. A difference degree acquisition unit to be acquired
A determination unit that determines whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on a state of distribution of the difference for each input pixel ;
The imaging device is an imaging device that acquires a difference between a logarithm of a pixel value of the reference pixel and a logarithm of a pixel value of the input pixel as the difference . The imaging apparatus according to claim 8, wherein comprising further the exposure control unit for controlling the exposure amount of the imaging unit in a predetermined amount when there is a change in illuminance. Each time the input image in which the input pixels are arranged is input, the difference degree acquisition unit includes the reference pixel and the input pixel corresponding to the reference pixel in the reference image in which the reference pixel is arranged. A degree-of-difference acquisition procedure for acquiring the degree of difference in pixel values for each of the input pixels;
A determination procedure for determining whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on the distribution state of the difference for each of the input pixels. And
The image processing method, wherein the difference acquisition unit acquires a difference between a logarithm of a pixel value of the reference pixel and a logarithm of a pixel value of the input pixel as the difference . Each time the input image in which the input pixels are arranged is input, the difference degree acquisition unit includes the reference pixel and the input pixel corresponding to the reference pixel in the reference image in which the reference pixel is arranged. A degree-of-difference acquisition procedure for acquiring the degree of difference in pixel values for each of the input pixels;
A determination procedure in which the determination unit determines whether or not there is an illuminance change exceeding a predetermined change amount when each of the reference image and the input image is captured based on the distribution state of the difference for each input pixel. a program to be executed by a,
The difference acquisition unit acquires a difference between a logarithm of a pixel value of the reference pixel and a logarithm of a pixel value of the input pixel as the difference.
program.

Images