JP3697082B2 - Image processing apparatus, image processing method, and computer-readable storage medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus, an image processing method, and a computer-readable storage medium, and more particularly to an imaging system, and more specifically, to an imaging system in which optical parameters at the time of imaging can be freely controlled by a computer. .
[0002]
[Prior art]
An imaging system that detects the presence of a moving object from a change in an image by an image processing technique called background difference has been put into practical use. Conventionally, in this type of moving object detection system, it has been a common method to update a background image using image information as a clue in order to cope with a change in brightness of a scene of interest.
[0003]
[Problems to be solved by the invention]
The response obtained from the image processing technique called background difference may be caused by the presence of a moving object or may be caused by a change in brightness of the scene being noticed. When it is caused by the presence of the moving object, it is necessary to notify it, and when it is caused by the brightness change, it is necessary to update the background image.
[0004]
However, in the prior art, since both are separated from only the image information, there is a problem that ambiguity remains when it is determined which is the response.
[0005]
The present invention eliminates such problems and clearly separates a response caused by the presence of a moving object and a response caused by a change in brightness of a scene of interest, and performs a more accurate moving object detection. The purpose is to be able to.
[0006]
[Means for Solving the Problems]
The image processing apparatus of the present invention includes an input unit that inputs image data obtained by the imaging unit, and exposure information and congruence distance information of the imaging unit in a plurality of areas when the image data is obtained, and the input unit. Storage means for storing input image data and exposure information and point distance information corresponding to a plurality of regions of the input image data; point distance information and exposure information corresponding to target image data; and storage in the storage means The comparison means for comparing the plurality of areas and the exposure information obtained with respect to the plurality of areas, and the comparison of the plurality of areas with the comparison means results in the difference between the distance information of the attention image data and the intersection distance information of the storage means. If the image data of the storage means is not updated, the moving object detection notification process is performed, and the congruence distance of the target image data Even if the information is the same as the exposure information in the storage means, if any exposure information in the plurality of regions of the target image data is different from the exposure information in the storage means, the image data stored in the storage means And control means for controlling to update exposure information and intersection distance information of the plurality of areas stored in the storage means.
[0007]
The image processing method of the present invention includes an input step of inputting image data obtained by the imaging unit, and exposure information and congruence distance information of the imaging unit in a plurality of areas when the image data is obtained, and the input step A storage step of storing input image data and exposure information and intersection distance information corresponding to a plurality of regions of the input image data in storage means; intersection distance information and exposure information corresponding to attention image data and the storage As a result of comparison between the plurality of regions with the distance information and exposure information stored in the means, and the comparison with respect to the plurality of regions in the comparison step, the point distance information of the attention image data is the point distance of the storage means. If the information is different from the information, the notification processing for moving object detection is performed without updating the image data in the storage means, and the attention image data Even if the point information is the same as the exposure information of the storage means, if any exposure information of the plurality of regions of the attention image data is different from the exposure information of the storage means, it is stored in the storage means. And a control step of performing control so as to update image data and also update exposure information and intersection distance information of the plurality of areas stored in the storage unit.
[0008]
The computer-readable storage medium of the present invention includes an input step of inputting image data obtained by an imaging unit, and exposure information and intersection distance information of the imaging unit in a plurality of areas when the image data is obtained; A storage step of storing in the storage means the image data input by the input step and the exposure information and the point distance information corresponding to the plurality of regions of the input image data; the point distance information and the exposure information corresponding to the target image data And the comparison distance information and exposure information stored in the storage means with respect to the plurality of areas, and the comparison distance information of the attention image data as a result of the comparison with respect to the plurality of areas in the comparison process. If it is different from the point distance information, the moving object detection notification process is performed without updating the image data in the storage means. If the exposure information of the plurality of regions of the attention image data is different from the exposure information of the storage means even if the congruent distance information of the attention image data is the same as the exposure information of the storage means, A program for causing a computer to execute a control process for controlling image data stored in a storage unit and updating the exposure information and congruent distance information of the plurality of areas stored in the storage unit. Remember.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
In this embodiment, an optical (imaging) parameter is optimized, and an imaging system capable of external reference and external control is used. In addition, this photographing system has a function of setting and displaying a target region (referred to as a measurement region) on an image when performing optical parameter optimization, external reference, and external control.
[0010]
Using such an imaging system, an optimal optical parameter is stored for each measurement region while a background image is captured. Then, when the optical parameters change due to changes in illumination conditions, etc., the background image and the optical parameters are calculated by calculating the difference between the stored optical parameters and the current optical parameters for each measurement region. Update automatically.
[0011]
Various optical parameters such as shutter speed, iris (aperture), gain (video signal amplification factor), white balance, focus, and zoom can be considered. In this embodiment, shutter speed, iris, gain, An example using four optical parameters of focus is shown.
[0012]
The focus here is information on the distance to the subject obtained from the position of the focus lens in the in-focus state. In the following description, the four optical parameters used in this embodiment are roughly divided into two, and the shutter speed, iris, and gain are represented as exposure information, and the focus is represented as distance information. The hardware configuration, principle, and processing procedure of this embodiment will be described below.
[0013]
FIG. 1 is a block diagram illustrating a hardware configuration of the image processing apparatus according to the present embodiment. The hardware is composed of a camera system 1-1 and a camera system control device 1-6.
[0014]
As shown in FIG. 1, the camera system 1-1 includes an imaging system 1-2 for inputting an image, an optical parameter control device 1-3 for controlling the optical parameters of the camera, and an optical parameter control device 1- 3 is a measurement area control device 1-4 that sets the area to be controlled.
[0015]
The optical parameter control device 1-3 is connected to the camera system control device 1-6 through a control signal line. By communicating a control signal, the optical parameter control device 1-3 sets optical parameters from the camera system control device 1-6. Controls such as acquisition and optimization are possible.
[0016]
Here, RS-232C, parallel IO, and the like are available as control signal lines, but the communication system standards are not limited. Note that the video output signal of the imaging system 1-2 has an NTSC format, YC separation method, etc., and the video is directly output to the external video monitor 1-15.
[0017]
The optical parameter control device 1-3 also has a function of automatically controlling optical parameters according to the scene to be photographed. This includes, for example, a function for automatically adjusting the brightness balance of the screen and a function for automatically obtaining a focused image.
[0018]
It is also possible to obtain the distance from the focus lens position in focus to the subject. These functions can be easily realized by using a camera module built in a home video camera.
[0019]
It can be instructed from the camera system control device 1-6 through the control signal line whether the optical parameter adjustment is performed from the camera system control device 1-6 or it is left to the automatic control function of the optical parameter control device. .
[0020]
The measurement area controller 1-4 can be controlled through a control signal from the camera system controller 1-6, and sets the size and position of the measurement area to be controlled by the optical parameter controller 1-3. Can do.
[0021]
The camera system controller 1-6 is externally connected to the CPU 1-7, RAM 1-8, ROM 1-9, secondary storage device 1-10, I / O 1-11 for communication with the camera system, and network 1-17. A communication device 1-12 that performs a camera system control command is provided.
[0022]
The CPU 1-7, RAM 1-8, ROM 1-9, secondary storage device 1-1O, I / O 1-11, and communication device 1-12 are connected to the bus 1-16. The camera system control device 1-6 can be realized by a general-purpose computer. A keyboard 1-13, a pointing device 1-14 such as a mouse, and a video monitor 1-15 are connected to the bus 1-16.
[0023]
The camera system control device 1-6 sends signals to the optical parameter control device 1-3 through the control signal line in accordance with the input from the keyboard and mouse or the control signal transmitted from a remote place through the network. To control.
[0024]
Next, the basic principle of the image processing apparatus according to the present embodiment will be described with reference to FIG.
Fig. 2- (a) shows how the camera system 1-1 captures an image of a scene, and at the same time, uses the optical parameter control device 1-3 and the measurement area control device 1-4. It shows how the optimum optical parameters in a given scene are acquired for each measurement region.
[0025]
In FIG. 2, 2-1 is the camera system 1-1 in FIG. 1, and 2-2 is the scene of interest. In addition, 2-3 is an enlarged view of the focused scene 2-2. The measurement area control device 1-4 is used to set a measurement area (such as 2-4) as shaded. ing.
[0026]
As described above, after the optical parameter control device 1-3 optimizes the shutter speed, iris, gain, and focus for each measurement region in the scene, the size of the measurement region is added to the optimum values. The position value is stored in the RAM 1-8 provided in the camera system control device 1-6.
[0027]
FIG. 2B is an example of the scene image captured in FIG. 2A and each optical parameter stored in the RAM 1-8. As shown in FIG. 2- (b), each optical parameter is stored for each measurement region. In FIG. 2- (b), 2-5 is the result of storing 2-2 of FIG. 2- (a) in the RAM 1-8.
[0028]
In the present embodiment, the background image and optical parameter update processing is performed as follows. That is, the optical parameter control device 1-3 provided in the camera system 1-1 uses the measurement region control device 1-4 and pays attention again to the measurement region unit as shown in FIG. An optimal value of an optical parameter composed of exposure information and distance information in a scene is obtained.
[0029]
As a result, if the value of distance information is the same as the value stored in all measurement areas, and if the exposure information is different from the stored value in all measurement areas, a change in brightness has occurred in the background image. The background image and the optical parameters are updated.
[0030]
On the other hand, if the distance information value is different from the stored value in at least one measurement area, or if the exposure information changes only in a part of the measurement area, the moving object exists. Notify that presence has been detected. A program for realizing this processing is in the ROM 1-9, the secondary storage device 1-10, or an appropriate place not shown, and is executed using the CPU 1-7.
[0031]
In this way, the optimal optical parameter value at a certain point in time is stored in the RAM 1-8, and the stored optical parameter value is compared with the current optical parameter value, thereby complicating. It is possible to detect a change in the brightness of the background image and to update the image without performing any image processing.
[0032]
In addition, by using a plurality of pieces of information such as exposure information and distance information, it is possible to determine with high accuracy whether it is timing to update the background image or detection of the presence of a moving object. is there.
[0033]
Next, the processing procedure of this embodiment will be described with reference to the flowchart of FIG. The image processing apparatus according to the present embodiment first determines the size of the measurement region as an initial setting in the first step S301 after starting the program.
[0034]
Next, the optical parameters are stored in the processes of steps S302 to S306. In this embodiment, since a plurality of measurement areas are set, the process is repeated until the optimum values of the optical parameters are referred to and stored in all the measurement areas.
[0035]
That is, first, in step S302, the measurement region is moved to the initial position, and optimum values of optical parameters such as shutter speed, iris, gain, and focus are referred to and stored in the RAM 1-8 (steps S304 and S305). ).
[0036]
Next, in step S306, it is determined whether or not the processing has been completed for all the areas. If not, the process proceeds to step S303, the measurement area is moved, and the optical parameters are referred to and stored again (step S303). Step S304, Step S305).
[0037]
On the other hand, as a result of the determination in step S306, if the processing has been completed in all areas, the scene at that time is similarly stored in the RAM 1-8 as a background image in step S307.
[0038]
Next, processing for determining whether or not to update the background image in steps S308 to S313 will be described. Again, in this embodiment, optical parameters are compared in all regions because a plurality of measurement regions are set.
[0039]
First, in step S308, the measurement area is moved to the initial position, and then the process proceeds to step S310 to refer to optical parameters of exposure information (shutter speed, iris, gain) and distance information (focus).
[0040]
Next, the process proceeds to step S311 and attention is paid to distance information among the referenced optical parameters. Now, the reference distance information is compared with the distance information of the same measurement area stored in the RAM 1-8, and when they are different, the process proceeds to step S314 to perform the process of notifying the detection of the moving object. In this case, the background image is not updated.
[0041]
On the other hand, if it is determined in step S311 that the distance information is equal, the process proceeds to step S312 and attention is paid to the exposure information. Compare the exposure information just referenced with each value of the exposure information of the same measurement area stored in the memory, and if any of them is different, move to step S313 because there is a possibility of background image update. The areas are similarly referenced and compared (step S308 to step S312).
[0042]
If the distance information is equal to the memory in all measurement areas and the exposure information is different from the memory in all measurement areas, the process proceeds from step S313 to step S302 to update the background image.
[0043]
In this series of processing, the distance information is the same as the memory in all the measurement areas, but if there is even one measurement area that is the same as the memory for the exposure information (step S312), the brightness of the scene of interest In this case, the process returns to the process of determining whether or not to update the background image starting from step S308 without updating the background image.
[0044]
As described above, in the present embodiment, the exposure information handled in steps S304, S305, S310, and S312 uses three types of parameters, shutter speed, iris, and gain. It is not limited to these parameters, and other exposure information such as white balance can be used or used together.
[0045]
Further, for example, for a camera system equipped with an optical parameter control device in which the shutter speed and iris are fixed values and only the gain is variable, the processes of steps S304, S305, S310, and S312 are performed only with the gain value. It can also be realized.
[0046]
In addition, when comparing optical parameters in steps S311 and S312, first, distance information is compared first, and then exposure information is compared, but in general this order is not limited. However, even if the order is changed, the same operation can be realized by appropriately changing the flow of steps.
[0047]
(Second Embodiment)
In steps S311 and S312 in the first embodiment described above, an example is described in which determination is made based on whether or not the stored value and the current measured value are the same value, but the present invention is not necessarily limited to this. That is, the absolute value of the difference between the stored value and the current measured value may be determined based on whether or not the value is equal to or greater than a predetermined value.
[0048]
Focusing on exposure information, for example, an absolute value ΔS of a difference in shutter speed value, an absolute value ΔI of an iris value difference, and a value ΔE determined by an absolute value ΔG of a gain value difference are appropriately determined in advance, for example. Using the coefficients αS, αI, αG,
ΔE = αs ΔS + αI ΔI + αG ΔG
For example, a determination may be made based on a combination of a plurality of exposure information changes, such as determining whether ΔE is greater than a predetermined value ΔETH.
[0049]
(Third embodiment)
In the first embodiment described above, changes in optical parameters are detected in a plurality of measurement regions, distance information does not change in all of these measurement regions, and exposure information is present in all of the measurement regions. Although the background image is updated when it has changed, the present invention is not necessarily limited to this. For example, when the distance information does not change in most of the plurality of measurement areas (for example, 80% or more, etc.) and the exposure information changes, the background image is considered to be changed and the background image is updated. You may comprise.
[0050]
(Other embodiments of the present invention)
The present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.) or an apparatus composed of a single device.
[0051]
In addition, software for realizing the functions of the above-described embodiments is provided to an apparatus or a computer in the system connected to the various devices so that the various devices are operated so as to realize the functions of the above-described embodiments. What is implemented by supplying a program code and operating the various devices according to a program stored in a computer (CPU or MPU) of the system or apparatus is also included in the scope of the present invention.
[0052]
In this case, the program code of the software itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, the program code is stored. The storage medium constitutes the present invention. As a storage medium for storing the program code, for example, a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.
[0053]
Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) or other application software in which the program code is running on the computer, etc. It goes without saying that the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized jointly.
[0054]
Further, after the supplied program code is stored in the memory provided in the function expansion board of the computer or the function expansion unit connected to the computer, the CPU provided in the function expansion board or function expansion unit based on the instruction of the program code Needless to say, the present invention also includes a case where the functions of the above-described embodiment are realized by performing part or all of the actual processing.
[0055]
【The invention's effect】
According to the present invention, it is possible to update a background image and imaging parameters used for moving object detection as well as to detect a moving object related to a plurality of regions, so that more accurate moving object detection can be performed.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention.
FIG. 2 is an explanatory diagram of a basic principle of a background image update process using an optical parameter reference function.
FIG. 3 is a flowchart for explaining a processing procedure for realizing the principle;
[Explanation of symbols]
1-1 Camera system
1-2 Imaging system
1-3 Optical parameter controller 1
1-4 Measurement area controller
1-6 Camera system controller
1-7 CPU
1-8 RAM
1-9 ROM
1-10 Secondary storage device
1-11 I / O
1-12 Communication device
1-13 Keyboard
1-14 Pointing device
1-15 Video monitor
1-16 Bus
1-17 Network

Claims (7)

Input means for inputting image data obtained by the imaging means, and exposure information and congruence distance information of the imaging means in a plurality of areas when obtaining the image data;
Storage means for storing image data input by the input means and exposure information and congruence distance information corresponding to a plurality of regions of the input image data;
Comparing means for comparing the distance information and exposure information corresponding to the target image data with the distance information and exposure information stored in the storage means for the plurality of areas ;
As a result of the comparison of the plurality of regions by the comparison unit, when the point distance information of the target image data is different from the point distance information of the storage unit, the image data of the storage unit is not updated and the moving object detection is performed. Notification processing is performed, and even if the congruent distance information of the target image data is the same as the exposure information of the storage unit, the exposure information of any of the plurality of regions of the target image data is different from the exposure information of the storage unit. A control unit that controls to update the image data stored in the storage unit and to update the exposure information and the intersection distance information of the plurality of areas stored in the storage unit;
An image processing apparatus comprising: The exposure information and the shutter speed, iris, an image processing apparatus according to claim 1, wherein the one of the information about the gain, or a combination thereof. It said input means, the image processing apparatus according to claim 1 or 2, characterized in that it comprises the imaging means. The image processing apparatus according to any one of claim 1 to 3, characterized in that it comprises a setting means for setting the plurality of regions. The image processing apparatus according to claim 1, further comprising a unit that notifies the result to the outside as the notification process . An input step for inputting image data obtained by the imaging means, and exposure information and congruence distance information of the imaging means in a plurality of areas when obtaining the image data;
A storage step of storing, in a storage unit, image data input by the input step and exposure information and congruence distance information corresponding to a plurality of regions of the input image data;
A comparison step of comparing the point distance information and exposure information corresponding to the target image data with the point distance information and exposure information stored in the storage unit with respect to the plurality of regions ;
As a result of the comparison regarding the plurality of regions in the comparison step, when the point distance information of the target image data is different from the point distance information of the storage unit, the image data of the storage unit is not updated and the moving object detection is performed. Notification processing is performed, and even if the congruent distance information of the target image data is the same as the exposure information of the storage unit, the exposure information of any of the plurality of regions of the target image data is different from the exposure information of the storage unit. A control step of controlling to update the image data stored in the storage unit and also update the exposure information and the intersection distance information of the plurality of areas stored in the storage unit;
And an image processing method. An input step for inputting image data obtained by the imaging means, and exposure information and congruence distance information of the imaging means in a plurality of areas when obtaining the image data;
A storage step of storing, in a storage unit, image data input by the input step and exposure information and congruence distance information corresponding to a plurality of regions of the input image data;
A comparison step of comparing the point distance information and exposure information corresponding to the target image data with the point distance information and exposure information stored in the storage unit with respect to the plurality of regions ;
As a result of the comparison regarding the plurality of regions in the comparison step, when the point distance information of the target image data is different from the point distance information of the storage unit, the image data of the storage unit is not updated and the moving object detection is performed. Notification processing is performed, and even if the congruent distance information of the target image data is the same as the exposure information of the storage unit, the exposure information of any of the plurality of regions of the target image data is different from the exposure information of the storage unit. A control step of controlling to update the image data stored in the storage unit and also update the exposure information and the intersection distance information of the plurality of areas stored in the storage unit ;
A computer-readable storage medium storing a program for causing a computer to execute.

Images