JP3086688B2 - Moving object extraction device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Purpose of the Invention] (Industrial application field) The present invention relates to an image monitoring apparatus for detecting an intruder or the like based on an image, for example, a movement for extracting a moving object from an input image. The present invention relates to an object extraction device.

(Prior Art) In recent years, for example, in a security system, an image monitoring device that detects an illegal intruder or the like by using an image captured by a television camera has been widely used. This is also because recent remarkable developments in semiconductor technology and the spread of digital image processing technology have made it possible to produce low-cost devices.

In such an image monitoring apparatus, for example, in order to detect a trespasser, it is necessary to extract change information caused by the trespasser moving in the image. Conventionally, as a method of extracting such a changed region, a moved portion (changed region) is extracted by simply performing a subtraction process with an image that is temporally separated.

In an image monitoring device using a television camera,
A region of change is extracted by subtracting a temporally distant image taken mainly from obliquely above, and then analyzing the extracted region of change to obtain a person (moving object)
Is determined. However, in this case, the actual size of the moving object in the image of the extracted change area is not considered, and is directly used as the determination object as the change area image.

(Problems to be Solved by the Invention) However, in the conventional method of extracting a changed portion, depending on the shooting environment of the input image, for example, particularly on a sunny day or a rainy day, the moving direction of the moving object and the input It is difficult to extract the original shape of the moving object as it is depending on the interval of the image capturing time and the like. In fact, in many cases, only a part of the moving object is actually extracted.

Conventionally, since the size of the extracted moving object is not taken into account, in the determination processing targeting the size of the shape of the change area, a moving object other than the moving object to be extracted, such as a small animal, is erroneously detected. Judgment is often made and an alarm is issued in the security system. Therefore, if such an alarm is issued frequently, humans become accustomed to the alarm, and in the worst case, the power of the security system linked to the image monitoring apparatus may be turned off.

Therefore, the present invention provides a method in which a change area obtained by moving an object to be extracted is a part of an original object due to an environmental change, such as a shooting environment of an input image or a direction of a moving object. It is an object of the present invention to provide a moving object extraction device that can extract the entire shape of a moving object using a part of the changed part.

Further, the present invention can correct the size of the shape with respect to the change area obtained by moving the target object to be extracted, and the same size that is apparently different depending on an arbitrary position in the perspective image. It is an object of the present invention to provide a moving object extraction device that can handle a target object as information of the same size regardless of the extraction position in the image.

[Structure of the Invention] (Means for Solving the Problems) A moving object extraction device according to a first invention is a moving object extraction device used for an image monitoring device, comprising: an image input unit for inputting an image; Means for receiving an image input by the means and subtracting the image from the image separated in time to extract an image indicating a changed area of at least a part of the moving object; and the image input means A peripheral rectangle determining unit that obtains, as rectangle information, a peripheral region including the changed region and the background, based on the image input in step S1 and the image that indicates the changed region extracted by the changed region extracting unit. Density projection creating means for creating density projection information for the image information input by the image input means corresponding to the position indicated by the rectangular information; A moving object that separates the moving object from the background based on the density projection information created in the step, and extracts an image of the moving object from the image information input by the image input unit corresponding to the position indicated by the rectangular information; Extraction means.

A moving object extraction device according to a second aspect of the present invention provides an image input unit for inputting an image, an image input by the image input unit, and an arithmetic operation between the image and a temporally separated image. Changing area extracting means for extracting an image indicating a changing area of an object; image display means for visually displaying an image input by the image input means; and a display screen of the image display means for displaying the image. A rectangle designating means for arbitrarily designating a rectangle, and correction information of the size of the moving object are obtained by using the rectangle information of a plurality of places specified by the rectangle specification means, and the change area is obtained by using the obtained correction information. A size correction unit configured to correct the size of the moving object at an arbitrary position in the image output from the extraction unit.

(Operation) The moving object extraction device according to the first invention uses a change area image of a moving object obtained as a result of arithmetic processing with an image temporally separated from the input image, and an input image corresponding thereto. With the surrounding area including the change area as a processing target, density projection information within the processing target in the input image is obtained, the moving object and the background are separated based on the obtained density projection information, and the position indicated by the rectangular information is obtained. Is to extract the image of the moving object from the image information corresponding to.

The moving object extraction device according to the second invention, when targeting a perspective image obtained by photographing from obliquely above, when a moving object originally having the same size is captured near and far from the screen, the apparent Size is different. Therefore, an object having a size to be extracted is found from near and far places in the image, and at least two rectangles including the respective objects are set in the image, so that the size of the object is reduced. The correction information is obtained, and the size of the object at an arbitrary position in the image is corrected using the correction information.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 shows a configuration of a moving object extracting apparatus in an image monitoring apparatus for detecting, for example, an illegal intruder according to a first embodiment of the present invention. That is, reference numeral 1 denotes an image input unit (image input means) mainly composed of, for example, a television camera, which inputs an image (hereinafter referred to as a time-series image) separated by a predetermined time as multivalued information. The multi-valued image input from the image input unit 1 is supplied to a change region extraction unit (change region extraction unit) 2 based on a difference image, and is subjected to a subtraction process for each pixel using a plurality of time-series images to move the image. A change area of the object is extracted. As the method, there are a method using two time-series images generally used and a method using three time-series images.
In the present embodiment, for example, a difference image obtained by a subtraction process using three time-series images is used.

The process of obtaining the difference image is ideal if the shape of the moving object can be completely extracted, but in practice, in most cases, the moving object can only be partially captured. It is a fact.

Now, the binarized image indicating the changed area extracted by the changed area extracting unit 2 is supplied to the changed area surrounding rectangle determining unit (surrounding rectangle determining unit) 3 together with the input image from the image input unit 1. The change area surrounding rectangle determination unit 3 includes a change area,
The surrounding area is obtained in the form of rectangle information, and the inside of the rectangle is set as a target of the target moving object extraction processing. For the determination of the rectangular information, information such as the size of the person to be extracted on the image can be used in advance.

The rectangle information determined by the change area surrounding rectangle determination unit 3 is:
Along with the input image from the image input unit 1, it is supplied to a density projection creation unit (density projection creation unit) 4 in line units. The image to be processed by the density projection creating unit 4 is a multivalued input image, and obtains the density projection information for each line with respect to the image information at the position of the multivalued input image corresponding to the position indicated by the rectangle.

The density projection information created by the density projection creating section 4 is supplied to the moving object extracting section (moving object extracting means) 5 together with the changed area image extracted by the changed area extracting section 2. The moving object extracting unit 5 extracts the moving object from the change area image based on the density projection information in line units. That is,
A moving object within the processing target (that is, within the rectangle) is extracted based on the density projection information in line units.

The image extracted by the moving object extraction unit 5 is supplied to the determination unit 6. The determination unit 6 determines the presence or absence of an illegal intruder based on the image from the moving object extraction unit 5, and outputs the determination result.

FIG. 2 shows an installation state of a television camera 1a used in the image input unit 1. In this embodiment, the television camera 1a is installed at a height h, and a situation in which a person 7 is moving toward the television camera 1a in a shooting state from obliquely above will be described. This situation setting is considered to be a typical case in which the moving object (person 7) in question in the present invention can only be partially captured.

FIG. 3 shows a process of extracting a changing region based on three time-series images in the changing region extracting unit 2 in the photographing environment shown in FIG. Note that the horizontal line drawn in the image is for making it easier to understand the moving state of the person. It is also assumed that the images at t1, t2, and t3 are images separated by a certain time interval, and that time elapses from t1 to t2 and from t2 to t3.

First, by subtracting the image of t1 from the image of t2, t1−
The difference image of t2 can be obtained. Here, the part of the head and the foot of the person is extracted as the change area. Then, t3
The difference image of t2−t3 can be obtained by subtracting the image of t2 from the image of. Here, as in the case of the difference image between t1 and t2, the part of the head and the foot of the person is extracted as the change area, and the only difference is the position on the image. Further, by performing a logical product process of the image of t2−t3 and the image of t1−t2, a difference image of R can be finally obtained. The portions a and b shown in the difference image R are change regions.

The difference image R is actually a difference binary image obtained by taking the absolute value of the difference multi-valued image and binarizing it with a predetermined threshold.

As described above, when the target object moves little by little and the target object overlaps between the time-series images, the change region is not usually extracted in the difference image R.
However, in practice, since the photographing situation is changing moment by moment, it can be extracted as a change area at a boundary portion of a moving object particularly between time-series images. The change region in the difference image R can reliably extract a part of the moving object. Using this important information, a moving object is extracted by finely analyzing the density of a peripheral region including the background of the changing region. As described above, even if the entire shape cannot always be extracted from the difference image, it is possible to extract the entire shape of the target object by performing the processing stepwise using the original multi-valued image.

FIG. 4A shows a multi-valued input image, and FIG. 4B shows the final difference image R shown in FIG. FIG. 9C shows the superimposed images of FIGS. 9A and 9B, and shows that the changed portion just overlaps the head, upper part, and foot of the person.

FIG. 4 (d) shows a case where the surrounding area including the background, including the changing area, is determined using the size information on the screen of the person to be extracted based on the positional information of the changing area. In this case, the rectangle K is displayed. As a result, it is possible to set the target of the area in which the processing until the extraction of the moving object is performed thereafter within the rectangle K.

FIG. 4 (e) is an enlarged view of the inside of the rectangle K, and the processing for obtaining the density projection for each line is performed using the line YS
From line YE. FIG. 4F shows the density projections of the line 1 and the line l + y in the rectangle K shown in FIG. 4E, which are used to separate the background and the moving object from the density projection information. The threshold is set to th (l), t for each line.
h (l + y) is obtained, and the area O and the background B of the moving object are separated. Here, the reason why the threshold value is set for each line is that, for example, in the case of a person, since the density is partially different depending on clothes, the density frequency in the local region is 2%.
This is because when the value is converted, the entire moving object is not always extracted.

The hatched portion shown in FIG. 4 (g) shows the moving object finally extracted by the above processing.

As described above, the first embodiment uses the change area image of the moving object obtained as a result of the subtraction processing with the image temporally distant from the input image and the input image corresponding thereto, and To obtain the density projection information for each line within the processing target in the input image with this peripheral area as the processing target, and to extract the moving target object for each line based on the obtained density projection information for each line The moving object is extracted from the change area image based on the obtained threshold value for each line.

Thereby, even if the change area obtained by moving the target object to be extracted is a part of the original target object due to the influence of environmental changes such as the shooting environment of the input image or the direction of the moving object, the change area The entire shape of the moving object can be extracted using the changed portion.

In other words, in the method of extracting the changed area using the difference image, the original movement is performed when the moving environment is parallel to the environment of the shooting by the television camera and the direction of the moving object, particularly the installation position of the television camera. It is practically very difficult to extract the entire shape of the object to be processed. Rather, in most cases, only a part of the moving object is generally extracted in most cases.

On the other hand, using the change area indicated by the part of the moving object as a clue, using a multi-valued input image, processing the periphery of the change area and processing density projection information for each line. By obtaining and extracting a moving object using this density projection information, even in the case of the shooting environment,
The entire shape of the moving object can be extracted.

The present invention is not limited to the above embodiment. For example, when the change area by the difference image is used as a clue for extracting the moving object,
In general, if the background around the moving object is simple, an appropriate threshold value is determined using the density frequency information around the change area, a binarized image is created, and the logical sum processing with the change area image is performed. It is also possible to extract the entire area of the moving object by performing the above operation.

In addition, when the change area occupies a large part of the image, that is, when there are a plurality of moving objects of various types, it is necessary to analyze and extract the target moving object from the entire change image in detail, due to noise components. There is a risk that the rate of false detection will increase. At this time, it is more effective to roughly divide and integrate the area roughly. Therefore, in such a case, it is also possible to apply the information of the change area as a clue and roughly divide and integrate the area with respect to the input image corresponding thereto.

FIG. 5 shows a configuration of a moving object extracting device in an image monitoring device for detecting, for example, an illegal intruder according to a second embodiment of the present invention. That is, reference numeral 11 denotes an image input unit (image input means) mainly composed of, for example, a television camera, and inputs an image (time-series image) separated by a certain time as multi-value information. The multi-valued image input from the image input unit 11 is supplied to a change area extraction unit (change area extraction unit) 12 using a difference image, and here, by performing a subtraction process for each pixel using a plurality of time-series images, A change area of the object is extracted. As the method, there are a method using two time-series images generally used and a method using three time-series images. In the present embodiment, for example, a difference image obtained by a subtraction process using three time-series images is used.

The multi-valued image input from the image input unit 11 is directly supplied to an image display unit (image display unit) 13 such as a CRT display device, and is always displayed on the screen as a visible image. Then, at the initial setting stage of the image monitoring device, the operator of the image monitoring device
By using a pointing device such as a mouse in a rectangle information designating unit (rectangle designating unit) 14 for an image capturing an extraction target displayed in the image, a rectangle surrounding the extraction target in the image is extracted. specify. This operation is performed for at least two or more locations in the image.

The rectangular information specified by the rectangular information specifying unit 14 is supplied to a size correction unit (size correction unit) 15 of the moving object.
The moving object size correction unit 15 first corrects the shape of the moving object to the shape indicated by the original size even if there is a moving object at any position in the entire image area from the specified two or more rectangular information. Find the linear coefficient for performing Next, a correction process is performed on the actually obtained change area image, that is, the output image of the change area extraction unit 12, in accordance with the above-described correction coefficient.

The corrected image output from the change area extract 12 is supplied to the determination unit 16. The determination unit 16 determines the presence or absence of an illegal intruder based on the image from the change area extraction unit 12, and outputs the determination result.

FIG. 6 (a) shows an example of a multi-value image displayed on the image display unit 13. In this image, it is assumed that a person exists as an extraction target at at least two places in the image. In this example, an example is shown in which a person exists in the area Mf and the area Mn. In the area Mf, a rectangle surrounded by (XSf, YSf) and (XEf, YEf) is designated and displayed. Similarly, in the area Mn, (XSn, YSn) and (XEn
(n, YEn) to specify and display the rectangle. FIG. 6A shows a case where it is generally unnecessary to perform horizontal correction when the angle in the horizontal direction is shallow. This is because the correction by the present method is originally simple, and the accuracy is not relatively a problem.

FIG. 6 (b) is different from the case of FIG. 6 (a), in a case where the horizontal monitoring range is wide, that is, when a wide-angle lens or the like is used, a designation method in a case where horizontal correction processing cannot be ignored. It is shown. That is, designation of two or more rectangles in the image is performed diagonally to each other such that the area Mf is at the upper right of the image and the area Mn is at the lower left of the image. By doing so, the linear correction coefficient in the vertical direction and the horizontal direction in the image is obtained, so that it is possible to correct even if any moving object in the image exists. Further, the accuracy of the correction process can be improved by handling the correction coefficient related to the lens.

The two persons indicated by rectangles are the same person. In other words, two persons have apparently different sizes due to the perspective images taken from obliquely above by the television camera.

As described above, in the second embodiment, when a perspective image obtained by photographing a television camera from obliquely above is targeted, when a moving object originally having the same size is captured near and far from the screen, The apparent size is different. Therefore, an object having a size to be extracted is found from near and far places in the image, and at least two rectangles including the respective objects are set in the image, so that the size of the object is reduced. A correction coefficient is obtained, and the size of the object at an arbitrary position in the image is corrected using the correction coefficient.

As a result, the size of the shape can be corrected with respect to the change area obtained by moving the target object to be extracted, and the target of the same size which is apparently different in size depending on an arbitrary position in the perspective image An object can be handled as information of the same size regardless of the extraction position in the image.

The present invention is not limited to the above embodiment. For example, although the extraction target in the photographed image is manually specified one by one, it is also possible to automatically obtain an image in which the extraction target is captured by image processing.

Further, for example, when it is desired to widen the monitoring range, it is possible to analyze the characteristics of the appearance of an extreme wide-angle lens and to make a table of the information to cope with the characteristic.

[Effects of the Invention] As described above in detail, according to the moving object extraction device of the present invention, the change area obtained by moving the object to be extracted is:
Due to the influence of environmental changes, such as the shooting environment of the input image or the direction of the moving object, the entire shape of the moving object can be extracted using the changed part of the original object even if it is a part of the original object. .

Further, according to the moving object extraction device of the present invention, the size of the shape can be corrected for the change area obtained by moving the target object to be extracted, and the size can be changed by an arbitrary position in the perspective image. However, apparently different objects having the same size can be handled as information having the same size regardless of the extraction position in the image.

[Brief description of the drawings]

1 to 4 are diagrams for explaining a first embodiment of the present invention. FIG. 1 is a block diagram schematically showing an overall configuration, and FIG. 2 is a diagram showing an installation state of a television camera. FIG. 3 is a diagram illustrating a process of obtaining a difference image using three time-series images, FIG. 4 is a diagram illustrating a process, and FIG.
FIG. 6 and FIG. 6 are for explaining the second embodiment of the present invention. FIG. 5 is a block diagram schematically showing the overall configuration, and FIG. FIG. 6B is a diagram for explaining the setting of rectangular information in a relatively narrow case, and FIG. 6B is a diagram for explaining the setting of rectangular information in a case where the horizontal monitoring range is relatively wide. 1,11 ... Image input unit (image input means), 2,12 ... Change area extraction unit (change area extraction means), 3 ... Change area peripheral rectangle determination unit (surround rectangle determination means), 4 ... Density Projection creation unit (density projection creation means), 5: Moving object extraction unit (moving object extraction means), 13: Image display unit (image display means), 14: Rectangular information designation unit (rectangle designation means) , 15 ……
A size correction unit (size correction unit) of the moving object.

Continuation of the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) G06T 1/00 G06T 7/00 G08B 13/196 H04N 7/18

Claims (2)

(57) [Claims] 1. A moving object extracting apparatus used in an image monitoring apparatus, comprising: an image input unit for inputting an image; an image input by the image input unit; By this, a change area extracting means for extracting an image indicating a change area for at least a part of the moving object, and an image input by the image input means and an image indicating the change area extracted by the change area extraction means Peripheral rectangle determining means for obtaining, as rectangular information, a peripheral area including the change area and the background; and image information input by the image input means corresponding to a position indicated by the rectangular information obtained by the peripheral rectangular determining means. Density projection creating means for creating density projection information; separating the moving object and the background based on the density projection information created by the density projection creating means; Moving object extraction apparatus characterized by comprising a moving object extracting means for extracting an image of a moving object from the image information input by the image input unit corresponding to the position indicated by the rectangle information. 2. An image input means for inputting an image, and an image indicating a change area of a moving object by receiving an image input by the image input means and performing an arithmetic process on the image separated in time. Area extracting means for extracting the image, image display means for visually displaying the image input by the image input means, and a rectangle for arbitrarily specifying a rectangle for the display image on the display screen of the image display means An image output from the change area extracting means using the specified correction information, and obtaining the correction information of the size of the moving object using the rectangular information at a plurality of locations specified by the specifying means; And a size correcting means for correcting the size of the moving object at an arbitrary place in the moving object extracting apparatus.