JP7101080B2 - Image processing equipment - Google Patents

Description

The present invention relates to an image processing device that detects an image area (human area) corresponding to a person in an image.

Conventionally, by comparing a captured image acquired by photographing a monitoring area with a camera and a reference image, a change area, which is an area where there is a change between the two images, is obtained, and the size or shape of the change area, etc. There is an image processing device that determines "humanity" based on the image features of the image and detects a detection target such as an intruder from the captured image.

In such an image processing device, a change region corresponding to one person (intruder or the like) may be separated into a plurality of areas and extracted. For example, such a phenomenon may occur when the color of the intruder's clothes is similar to the background and the difference is small.

In view of this, conventionally, a process of integrating a plurality of change areas corresponding to one person has been performed.

For example, in Patent Document 1, a range on an image corresponding to the size of a person (intruder) at a position of an extracted predetermined change area is obtained by using camera installation conditions and the like, and is included in the range. A technique for integrating a set of a plurality of change areas as a change area by one intruder is disclosed.

Japanese Unexamined Patent Publication No. 9-231363

For example, when a person is walking, the person may be photographed with his / her legs open. When a person in the captured image has his legs open, the width of the change area corresponding to the lower body of the person is detected to be larger than the range corresponding to the size of the person obtained at the position of the change area. As a result, there is a problem that the change area corresponding to the lower body and the change area corresponding to other parts of the person (for example, the upper body) are not integrated.

An object of the present invention is to appropriately integrate a plurality of change regions corresponding to a person with open legs detected in a captured image.

The present invention is an image processing device for obtaining a person area indicating an image area corresponding to the person from a photographed image of the person, and the stride ratio information indicating the ratio of the person's stride to the height of the person and a reference. A storage means for storing an image in advance, an extraction means for comparing the captured image with the reference image and extracting a change region changed from the reference image in the captured image, and a first change of the change region. The candidate area calculation means for obtaining the candidate area of the person consisting of the area and the second change area presumed to be lower in the real space than the first change area, and the height of the candidate area in the real space. The estimated height to be shown is obtained, the lower body width in the real space is estimated using the estimated height and the stride ratio information, and the candidate area is obtained by comparing the lower body width with the horizontal width in the real space of the second change region. It is an image processing apparatus characterized by comprising a person area determination means for determining the person area relating to one person.

Desirably, the storage means further stores the reference height in the real space of the person, and the human area determination means determines whether the candidate area is the person area by comparing the estimated height with the reference height. It is characterized in that it determines whether or not it is.

Desirably, the storage means further stores the reference upper body width in the real space of the person, and the human area determination means is based on a comparison between the horizontal width in the real space of the first change region and the reference upper body width. It is characterized in that it is determined whether or not the candidate area is the person area.

Desirably, the human region determining means has a first position indicating either the center of gravity position or the center position of the first change region, and the first position indicating either the center of gravity position or the center position of the second change region. It is characterized in that two positions are obtained, and whether or not the candidate region is the human region is determined based on the horizontal distance in the real space between the first position and the second position.

Desirably, the stride ratio information indicates the ratio that increases as the moving speed of the person increases, the extraction means extracts the change region from each of the sequentially acquired captured images, and the candidate region calculation means , The candidate area is obtained from each of the sequentially acquired images, and further includes a tracking means for tracking the candidate area in time, and the human area determination means is the candidate area based on the tracking result by the tracking means. The movement speed of at least one of the first change region and the second change region is obtained, and the lower body width is estimated using the movement speed and the stride ratio information.

According to the present invention, it is possible to appropriately integrate a plurality of change regions corresponding to a person with open legs detected in a captured image.

It is a block diagram of the security system which concerns on this embodiment. It is a block diagram of the image sensor which concerns on this embodiment. It is a figure which shows the photographed image (a), the reference image (b), and the change area image (c). It is a figure which shows the result of the initial integration processing. It is a figure which shows the height of a candidate area. It is a figure which shows the width lw of a reference label area. It is a figure which shows the integrated label area. It is a figure which shows the width w of a target label area. It is a figure which shows the state of finding the horizontal distance between the position of the center of gravity of a target label area, and the position of the center of gravity of a reference label area. It is a flowchart which shows the flow of a label integration process.

Hereinafter, embodiments of the present invention will be described.

FIG. 1 is a schematic configuration diagram of the security system 10 according to the present embodiment. The security system 10 is a security center connected to each security device 14 via a security device 14 installed in each monitored property 12 such as a store, an office, an apartment, a warehouse, and a house, and a communication network 16 such as a public telephone line. The device 18 and the user device 20 are included. Further, the security system 10 has an image sensor 22 as one or more image processing devices for detecting an abnormality in the monitored property 12 based on a monitored image taken of the monitored area of the monitored property 12, and an image sensor 22. It is configured to include a recording device 24 for recording a surveillance image taken by the user. The image sensor 22 and the recording device 24 are communicably connected to the security device 14.

When the security device 14 receives an alarm signal from the image sensor 22 connected to itself via the premises LAN or the like, the security device 14 detects the alarm signal, the identification signal of the security device 14 itself, the monitored property 12, or an abnormality. The identification signal of the image sensor 22 is transmitted to the security center device 18. Therefore, the security device 14 has a communication interface for communicating with the image sensor 22, a communication interface for communicating with the security center device 18 and the user device 20, and a control unit for controlling them.

The security center device 18 is composed of a so-called computer, and includes a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a notification unit composed of a buzzer, an LED, or the like. .. When the security center device 18 receives an alarm signal from the security device 14 via the communication network 16, the security center device 18 notifies the monitored property 12 in which the security device 14 that has transmitted the alarm signal is installed, and the content of the detected abnormality. Notify the observer through the display device.

The user device 20 is also composed of a so-called computer, and remotely operates the security device 14 such as a communication interface for communicating with the security device 14 via the communication network 16, a display device such as a liquid crystal display, and a keyboard and a mouse. It has a user interface for inputting operation commands for. When the user device 20 observes the pre-registered monitored property 12 via the user interface, the user device 20 currently photographs the security device 14 installed in the registered monitored property 12. Various image request signals requesting that the surveillance image inside or the surveillance image recorded in the recording device 24 be transmitted to the user device 20 are transmitted. Then, when the surveillance image is received from the security device 14, the user apparatus 20 displays the requested surveillance image on the display device.

The recording device 24 is a recording medium that can be attached to and detached from the recording device 24, such as a magnetic disk device such as an HDD, a magnetic tape such as a DAT, and an optical recording medium such as a DVD-RAM, and data is accessed by accessing the recording medium. It consists of a device that reads and writes. The recording device 24 receives the surveillance image taken by the image sensor 22 from the security device 14 and records it in association with the shooting time.

FIG. 2 is a schematic configuration diagram of the image sensor 22.

The communication unit 30 is an input / output interface for transmitting and receiving various setting signals and control signals between the image sensor 22 and the security device 14 via a communication network such as a local area network, and various types such as Ethernet (registered trademark). It consists of the communication interface circuit of the above and the driver software that drives them. Specifically, when an intruder is detected by the signal processing unit 42 described later, the communication unit 30 outputs an intrusion alarm signal indicating that the intruder has been detected to the security device 14.

The photographing unit 32 includes a two-dimensional detector composed of a photoelectric converter having sensitivity to visible light such as a CCD, and an imaging optical system that forms an image of a monitoring area on the two-dimensional detector. It is composed. The photographing unit 32 acquires a photographed image by photographing the monitoring area. In the present embodiment, the photographing unit 32 photographs at regular time intervals (for example, 1/5 second), but the photographing method of the photographing unit 32 is not limited to this. The acquired captured image is stored in the storage unit 34.

In the present embodiment, the photographing unit 32 is configured to include a fisheye lens, that is, the photographing unit 32 is an omnidirectional camera capable of setting an omnidirectional (360 degrees) as a photographing area (surveillance area). The omnidirectional camera is installed, for example, on the ceiling of the property 12 to be monitored, the monitoring area is downward, and the optical center is substantially vertically downward. However, the photographing unit 32 is not limited to the omnidirectional camera, and may be a unidirectional camera (for example, a camera having a viewing angle of 90 degrees).

The captured image captured by the photographing unit 32, which is an omnidirectional camera, is an omnidirectional image, which is a circular image as a whole, and the center of the circle is a position corresponding to the optical center. In the captured image, the vertically upward direction of the monitored region is the radial direction of the captured image and is the direction from the optical center to the outside, and the horizontal direction of the monitored region is the circumferential direction of the captured image.

When an intruder exists in the monitoring area, the captured image acquired by the photographing unit 32 includes a person area corresponding to the person (intruder).

The storage unit 34 is a storage unit composed of a semiconductor memory, a magnetic disk (HDD), an optical disk drive such as a CD-ROM or a DVD-RAM, and a recording medium thereof. The storage unit 34 stores an image processing program for operating each unit of the image sensor 22. Further, as shown in FIG. 2, the storage unit 34 stores the captured image 36, the reference image 38, and the reference information 40 acquired by the photographing unit 32.

The reference image 38 is created by the signal processing unit 42 based on the captured image 36 when no person is present in the monitoring area. The reference image 38 may be one, but a plurality of reference images 38 may be created. For example, the signal processing unit 42 obtains the frame-to-frame difference of the captured images 36 sequentially acquired by the photographing unit 32, and obtains the average value of the absolute values of the luminance differences between the corresponding pixels between the frames. Then, the signal processing unit 42 stores the captured image 36 whose average value is smaller than the predetermined reference in the storage unit 34 as the reference image 38. Further, it is preferable that the reference image 38 is updated at regular intervals (for example, every 10 minutes) in order to cope with fluctuations in the monitoring area such as fluctuations in lighting conditions and diurnal fluctuations in the sun.

The reference information 40 is information representing the characteristics of the person to be detected, and has a reference height (for example, 180 cm) indicating the height of the person in the real space, a reference upper body width (for example, 60 cm) indicating the width of the upper body of the person in the real space, and the like. It also includes stride ratio information indicating the ratio of the person's stride to the height of the person.

It is preferable that the stride ratio information indicates a ratio according to the movement speed of the person, particularly, the larger the movement speed of the person, the larger the ratio. For example, the stride when the person is walking slowly is height x 0.37, the stride when the person is walking at normal speed is height x 0.45, and the stride when the person is walking fast. The stride is like height x 0.50. That is, the stride ratio information is stored as information in which the threshold value defining the movement speed range of the person and the ratio of the stride to the height of the person are associated with each other.

The signal processing unit 42 has a built-in microprocessor unit, a memory such as a ROM and a RAM, and peripheral circuits thereof, and executes various signal processing of the image sensor 22. As shown in FIG. 2, the signal processing unit 42 exerts the functions of the extraction means 44, the label shaping means 46, the tracking means 48, and the determination means 50. When the signal processing unit 42 exerts these means, a human area, which is an image area corresponding to a person, is detected in the captured image 36. Hereinafter, each means included in the signal processing unit 42 will be described.

The extraction means 44 compares the captured image 36 with the reference image 38 stored in advance in the storage unit 34, extracts pixels that differ between the two images, and consists of extracted pixels and non-extracted pixels. Performs processing to obtain a value image (change area image). Here, the "pixels that differ between the two images" mean the pixels of the captured image 36 in which the difference in brightness value or color component between the captured image 36 and the reference image 38 is equal to or greater than a predetermined value, and the pixel is referred to as the pixel. It is called an extraction pixel.

FIG. 3 shows examples of the captured image 36 (a), the reference image 38 (b), and the change region image 39 (c). When the photographed image 36 includes the person image 60 as in the example of FIG. 3A, the pixels corresponding to the person image 60 are the extraction pixels.

Further, the extraction means 44 performs a labeling process for assigning an identifier (label) to each extraction pixel. As the label, for example, a unique numerical value in the change area image 39 can be used. In the labeling process, when a certain extraction pixel is focused on, a group of extraction pixels composed of other extraction pixels adjacent to the extraction pixel is regarded as one change region 64 (see FIG. 3C). The pixel adjacent to the extraction pixel may be a pixel adjacent to the extraction pixel in the vertical and horizontal directions, or may include pixels adjacent to the extraction pixel in the diagonal direction. Then, a different label is given to each change area 64, and a process of storing as a pixel value of the change area image 39 is performed. Hereinafter, the change area 64 to which the label is attached is referred to as a label area 66, and the image including the label area 66 is referred to as a label area image.

As a result, in the captured image 36, one or a plurality of changed regions 64 (label regions 66), which are regions changed from the reference image 38, are extracted. In the present embodiment, as shown in FIG. 3C, the label area 66 corresponding to one person is separated into a plurality of label areas 66a to 66c and extracted.

The label shaping means 46 executes a process of shaping the label area 66 in order to obtain a human area from each label area 66 extracted by the extraction means 44. In the present embodiment, the plurality of label areas 66 are integrated based on predetermined conditions such as whether or not the plurality of label areas 66 within a predetermined range become a human-like size or shape area when integrated. Execute label integration processing. According to the label integration process, for example, when the upper body and the lower body of the same person are extracted as different label areas 66, the two label areas 66 are integrated into one corresponding to the image of one person. It can be the label area 66. In particular, in the present embodiment, even when the photographed image 36 includes a person with his / her legs open, the label area 66 corresponding to the lower body of the person corresponds to another part (for example, the upper body) of the person. It can be integrated with the label area 66 to be used. The label shaping means 46 may perform a label separation process for separating one label area 66 into a plurality of label areas 66, or a label deletion process for deleting the label area 66.

For example, the label separation process is a process of cutting out a human area from the label area 66 and using it as another label area 66. Specifically, the label separation process includes a process of cutting out a human area from one label area 66 including a human area and a shadow area, and an optical area corresponding to a portion exposed to light such as a human area and a headlight. It includes a process of cutting out a human area from one label area 66. The label deletion process is a process of deleting a label area 66 that can be clearly determined not to be a person. For example, the label area 66 that is too large or too small for the size of the person is deleted.

As shown in FIG. 2, the label shaping means 46 includes the initial integration means 52, the candidate area calculation means 54, and the human area determination means 56, and the label integration process is appropriately executed by these means. Details of the initial integration means 52, the candidate area calculation means 54, and the human area determination means 56, and the details of the label integration process will be described later.

The tracking means 48 performs a process of temporally tracking the label area 66 extracted from each of the captured images 36 sequentially acquired by the photographing unit 32 and shaped by the label shaping means 46. Specifically, the tracking means 48 is derived from each label area 66 corresponding to the human area, which is extracted from the captured image 36 acquired this time and shaped by the label shaping means 46, and the captured image 36 acquired in the past. A process of identifying a label area 66 (hereinafter referred to as "tracking label area") that has been extracted and stored as tracking information in a storage unit 34 (a process of determining that the area is a person corresponding to the same person). I do. Here, the tracking information is information related to the tracking ID, the position on the captured image 36, and the tracking feature amount with respect to the tracking label area. The tracking feature amount is the size and shape of the tracking label area in real space, the luminance histogram, the texture information, and the like. Further, in the present embodiment, the position on the captured image 36 is the coordinate of the position at the lowermost center of the tracking label area.

The tracking means 48 compares the tracking feature amount and position of the label area 66 of the captured image 36 acquired this time with the tracking feature amount and position of the tracking label area. For example, identification is performed using features having similar tracking features and close positions. Then, the same tracking ID as the identified tracking label area is assigned to the label area 66 of the captured image 36 acquired this time, and the tracking of the label area 66 of the captured image 36 acquired this time is tracked in chronological order. After associating the ID with the position on the captured image 36 and the like, the ID is stored in the storage unit 34 as tracking information.

In the label area 66 of the captured image 36 acquired this time, those that could not be identified as the tracking label area are assumed to have appeared as a new person, and new tracking is performed on the tracking feature amount of the label area 66. Give an ID. In addition, if there is a tracking label area that has been tracked so far that cannot be identified as the label area 66 of the captured image 36 acquired this time, it is assumed that the person corresponding to the tracking label area has disappeared. , Delete the tracking information.

The determination means 50 determines whether or not each of the tracking label areas is an intruder. In the present embodiment, a tracking label area continuously tracked as a human area for a predetermined number of frames (for example, 5 frames) or more is determined to be an intruder. By obtaining the feature amount such as the estimated movement speed of the tracking label area and comparing the obtained feature amount with the pre-stored human-like feature amount (human-like speed, etc.), is the tracking label area human-like? May be further determined to determine whether or not it is an intruder.

The configuration outline of the image sensor 22 is as described above. Hereinafter, the details of the label integration process in the present embodiment will be described together with the details of the initial integration means 52, the candidate area calculation means 54, and the human area determination means 56 included in the label shaping means 46.

The initial integration means 52 performs an initial integration process for integrating a certain label area 66 extracted from the captured image 36 and another label area 66 around the label area 66. The initial integration process determines whether or not the integrated label area 66 is within the range of human-like size when a plurality of label areas 66 that are candidates for integration are temporarily integrated, and if it is within the range, the relevant label area 66 is concerned. This is a process of integrating a plurality of label areas 66 of integration candidates.

In the present embodiment, the initial integration means 52 sets an integration frame 70 (see FIG. 4) corresponding to the size of a person (size including the reference height of the reference information 40 and the reference upper body width) in the label area image. , A plurality of label areas 66 that fit inside the integration frame 70 are integrated. The initial integration means 52 performs the initial integration process while moving the integration frame 70 on the label area image. In the present embodiment, the captured image 36 is an omnidirectional image (circular image), and the vertically upward direction of the monitoring region is the radial direction of the captured image 36 and is a direction toward the outside from the optical center 72. Therefore, the width (length in the circumferential direction) of the integrated frame 70 becomes narrower as it approaches the optical center 72 side.

In the example of FIG. 4, since the label area 66a corresponding to the head of the person (see FIG. 3C) and the label area 66b corresponding to the upper body portion of the person are contained inside the integrated frame 70, the initial stage is shown. In the integration process, the label area 66a and the label area 66b are integrated into one label area 66d. On the other hand, since the width of the label area 66c corresponding to the lower body of the person is widened due to the person spreading his legs, a part of the label area 66c protrudes from the integrated frame 70, so that the initial stage In the integration process, the label area 66c is not integrated. The initial integration means 52 may determine whether or not the integrated label area 66 is within the range of human-like size without using the integration frame 70. For example, for a certain label area 66, the size (width, height) in the real space when the other label areas 66 around the label area 66 are integrated is obtained, and the obtained size is described in the reference information 40. When the height and the width of the upper body of the person are smaller than the height and the width of the upper body, the initial integration process may be executed by repeatedly executing the process of integrating these label areas 66 for all the label areas 66.

The candidate area calculation means 54 identifies the target label area (first change area), which is the label area 66 to be processed, in the label area image subjected to the initial integration processing, and then in the real space rather than the target label area. A reference label area (second change area), which is a label area 66 presumed to be in a low position, is specified. In the example of FIG. 4, when the label area 66d is specified as the target label area, the label area 66c located at a position lower in the real space than the label area 66d, that is, located radially inside the label area 66d is the reference label area. Specified as. The label area 66 located inside the label area 66d in the radial direction is, for example, a label area in which at least a part thereof exists within an angular range separated by a straight line connecting each of both ends in the circumferential direction of the label area 66d and the optical center 72. 66. The target label area is the label area 66 expected to correspond to the upper body of the person, and the reference label area is the label area 66 expected to correspond to the lower body of the person.

The candidate area calculation means 54 specifies an area including the target label area (label area 66d in this example) and the reference label area (label area 66c in this example) specified in this way as the candidate area 74. When a plurality of label areas 66 exist in the label area image, the candidate area calculation means 54 specifies each label area 66 as a target label area, and then sets a candidate area for each target label area as described above. Perform the process to specify. When the reference label area for the specified target label area does not exist (the label area 66 at a position lower in the real space than the target label area does not exist), the candidate area calculation means 54 uses the candidate area including the target label. Do not specify.

First, the human area determination means 56 obtains the height t (length in the vertical direction of the real space, see FIG. 5) of the candidate area 74 obtained by the candidate area calculation means 54. In the present embodiment, the candidate area calculation means 54 includes the pixel farthest from the optical center 72 in the label area 66d which is the target label area and the pixel closest to the optical center 72 in the label area 66c which is the reference label area. The radial distance between them can be the height t.

The human area determination means 56 calculates the height of the candidate area 74 in the real space based on the obtained height t. The height of the candidate area 74 in the real space is called the estimated height.

The conversion from the distance in the captured image 36 (label area image) to the distance in the real space includes the imaging condition information of the captured image 36 (for example, the installation height of the photographing unit 32, the zoom ratio (magnification), the focal length, etc.). ) Can be used. Specifically, based on the above-mentioned shooting condition information, it is possible to calculate how many centimeters a pixel corresponds to in real space according to the distance from the optical center 72. Thereby, the estimated height on the real space scale can be calculated based on the height t in the label area image.

Next, the human area determination means 56 calculates the lower body width of the person corresponding to the candidate area 74 in the real space by using the calculated estimated height and the stride ratio information included in the reference information 40.

Preferably, the human area determination means 56 calculates the movement speed of the person corresponding to the candidate area 74 based on the tracking result by the tracking means 48, and the stride ratio information indicating the ratio of the larger stride as the movement speed of the person is larger. And the estimated height of the person, the lower body width of the person corresponding to the candidate area 74 in the real space may be calculated. As a matter of course, in this case, the higher the moving speed of the person corresponding to the candidate area 74, the larger the estimated lower body width.

For example, when the moving speed of the person corresponding to the candidate area 74 is equal to or less than the first speed threshold value, it is determined that the person is "walking slowly", and the lower body width of the person is estimated height × 0. It is set to .37. Further, when the moving speed of the person corresponding to the candidate area 74 exceeds the first speed threshold and is equal to or less than the second speed threshold, it is determined that the person is "walking at a normal speed". The lower body width of the person is estimated height × 0.45. If the moving speed of the person corresponding to the candidate area 74 exceeds the second speed threshold value, it is determined that the person is "walking fast", and the lower body width of the person is estimated height × 0.50. And. In this embodiment, the stride ratio information of the reference information 40 is stored as information in which the movement speed range of the person and the ratio of the stride to the height of the person are associated with each other. However, the present invention is not limited to this, and the human area determination means 56 may dynamically obtain the ratio of the stride to the height of the person according to the moving speed of the person and store it as the stride ratio information. That is, the human area determination means 56 may calculate so that the ratio of the stride to the height of the person increases as the moving speed of the person increases, and may store it as the stride ratio information of the reference information 40.

The moving speed of the person corresponding to the candidate area 74 can be the moving speed of at least one of the target label area and the reference label area. The moving speed of the target label area is calculated based on the acquisition time of each of the captured images 36 sequentially acquired and the position of the target label in the candidate area 74 obtained by the candidate region calculation means 54 in each captured image 36. be able to. Similarly, the moving speed of the reference label area is based on the acquisition time of each of the sequentially acquired captured images 36 and the position of the reference label in the candidate area 74 obtained by the candidate region calculation means 54 in each captured image 36. Can be calculated. The movement speed of the person corresponding to the candidate area 74 may be calculated in consideration of the movement speeds of both the target label area and the reference label area. For example, the average value of the moving speeds of the target label area and the reference label area may be used as the moving speed of the person corresponding to the candidate area 74.

Further, the human area determination means 56 calculates the horizontal width of the reference label area in the real space based on the width lw of the reference label area (horizontal length of the real space, see FIG. 6), and estimates the lower body width. And the horizontal width of the calculated reference label area in the real space are compared with each other to integrate the target label area and the reference label area. That is, it is determined that the candidate area 74 is a person area related to one person.

The horizontal width in the real space can be calculated from the width l of the reference label area based on the above-mentioned shooting condition information. Since the captured image 36 is an omnidirectional image in the present embodiment, it is calculated how many cm in width one pixel corresponds to in the real space at the position of the distance rl from the optical center 72 based on the above imaging condition information. Can be obtained by. As a result, after finding the number of pixels the length (that is, width lw) of the arc consisting of the angle θl at the position of the distance rl from the optical center 72, the obtained number of pixels and the real space of 1 pixel are obtained. By multiplying by the size in, the width in the real space of the width lw of the label area 66c which is the reference label area can be calculated.

The human area determination means 56 is a candidate when the horizontal width of the calculated reference label area in the real space is equal to or less than the lower body width threshold determined based on the lower body width calculated from the estimated height of the person and the stride ratio information. The target label area (label area 66d in this example) included in the area 74 and the reference label area (label area 66c in this example) are integrated to form one label area 66e (see FIG. 7). As a result, it is determined that the candidate area 74 is a person area related to one person. On the other hand, when the calculated horizontal width of the reference label area in the real space exceeds the lower body width threshold value, the human area determination means 56 does not integrate the target label area and the reference label area included in the candidate area 74.

Preferably, the human area determination means 56 determines whether or not the candidate area 74 is a human area by comparing the calculated estimated height with the reference height included in the reference information 40, and determines that the candidate area 74 is not a human area. If so, the process of integrating the reference label area may not be performed for the candidate area 74. For example, when the calculated estimated height exceeds the height threshold value determined based on the reference height, it can be determined that the candidate area 74 does not correspond to the human area.

Further, preferably, the human area determination means 56 calculates and refers to the horizontal width of the target label area in the real space based on the width w of the target label area (length in the horizontal direction of the real space, see FIG. 8). By comparing the reference upper body width included in the information 40 with the calculated horizontal width of the target label area in the real space, it is determined whether or not the candidate area 74 is a human area, and if it is determined that the candidate area 74 is not a human area, it is determined. , The process of integrating the reference label area may not be performed for the candidate area 74. For example, when the calculated horizontal width exceeds the upper body width threshold value determined based on the reference upper body width, it can be determined that the candidate area 74 does not correspond to the human area.

When the target label area is the one in which a plurality of label areas 66 are integrated in the initial integration process, it is obvious that the horizontal width in the real space corresponding to the width of the target label area is equal to or less than the upper body width threshold value. Is. This is because, in the initial integration process, when the plurality of label areas 66 are within the range of human-like size, the plurality of label areas are integrated. However, if the target label area is not integrated by the initial integration process, the horizontal width in the real space corresponding to the width of the target label area may exceed the upper body width threshold, and therefore the target label as described above. It is preferable to compare the horizontal width of the area with the reference upper body width.

The horizontal width in the real space is calculated from the width w of the target label area in the same manner as the horizontal width in the real space is calculated from the width l of the reference label area based on the above-mentioned shooting condition information. Can be done. That is, based on the above-mentioned shooting condition information, it is possible to calculate how many centimeters one pixel corresponds to in the real space at the position of the distance ru from the optical center 72. As a result, after finding how many pixels the length (that is, width uwa) of the arc consisting of the angle θu corresponds to at the position of the distance ru from the optical center 72, the obtained number of pixels and the real space of 1 pixel are obtained. By multiplying the size in the above, the width of the width uw of the label area 66d, which is the target label area, can be calculated in the real space.

Further, preferably, in the human area determination means 56, the candidate area 74 corresponds to one person based on the amount of horizontal displacement in the real space between the target label area and the reference label area. It may be determined whether or not. This is because if the target label area corresponds to the upper body of the person and the reference label area corresponds to the lower body of the person, then there is a large horizontal misalignment between the target label area and the reference label area. Is not possible.

Specifically, as shown in FIG. 9, the human area determining means 56 first has a center of gravity position 80d as the first position of the label area 66d as the target label area and a label area 66c as the reference label area. The position of the center of gravity 80c as the second position is obtained. In the present embodiment, the center of gravity position 80d of the target label area is obtained as the first position, and the center of gravity position 80c of the reference label area is obtained as the second position. The center position of the label area may be obtained.

Next, the human area determination means 56 calculates the circumferential distance d between the center of gravity position 80d and the center of gravity position 80c, and then, based on the circumferential distance d, the actual distance between the center of gravity position 80d and the center of gravity position 80c. Calculate the horizontal distance in space. Here, too, the calculation of the horizontal distance in the real space from the circumferential distance d of the target label region can be calculated based on the above-mentioned shooting condition information.

Then, the human area determination means 56 determines whether or not the candidate area 74 including the target label area and the reference label area is a human area based on the calculated horizontal distance. Specifically, when the calculated horizontal distance exceeds a predetermined distance threshold value, the human area determination means 56 determines that the candidate area 74 is not a human area. A straight line connecting the optical center 72 and the center of gravity position 80d and a straight line connecting the optical center 72 and the center of gravity position 80c are obtained, and when the angle formed by the two straight lines is equal to or greater than a predetermined angle threshold, the horizontal distance determines the distance threshold. If it exceeds, the human area determination means 56 may determine that the candidate area 74 is not a human area.

As described above, according to the present embodiment, after specifying the candidate area 74 from the label area image, the estimated height based on the height t of the candidate area 74 is calculated, and the lower body width of the person is calculated from the estimated height. do. Then, the horizontal width of the reference label in the real space is calculated based on the width lw of the reference label considered to correspond to the lower body of the person, and based on the comparison between the calculated lower body width and the horizontal width of the reference label in the real space. The reference label is integrated with the other label area 66 corresponding to the other part. As a result, even when a person such as an intruder is photographed with his / her legs open, the plurality of label areas 66 corresponding to the person can be appropriately integrated in the photographed image 36.

Hereinafter, the flow of the label integration process will be described according to the flowchart shown in FIG.

In step S10, the initial integration means 52 executes the initial integration process on the label area image extracted by the extraction means 44. As a result, a plurality of label areas 66 that fit inside the integration frame 70 (see FIG. 4) are integrated.

Each step after step S12 is repeatedly executed by the number of label areas 66 remaining after the initial integration process in step S10.

In step S12, the candidate area calculation means 54 specifies one label area 66 as the target label area from the label area image. Further, the human area determination means 56 obtains the width uwa (see FIG. 8) of the specified target label area, and then calculates the horizontal width of the target label area in the real space based on the width uwa.

In step S14, the human area determination means 56 determines whether or not the horizontal width of the target label area calculated in step S12 is equal to or less than the upper body width threshold value determined based on the reference upper body width included in the reference information 40. .. If the horizontal width exceeds the upper body width threshold value, it can be determined that the target label area specified in step S12 does not correspond to the upper body of the person. Therefore, the processing for the target label area is terminated, and the other step S12 is performed again. The label area 66 of the above is specified as the target label area, and the process of step S12 or less is performed. If the horizontal width is equal to or less than the upper body width threshold value, the process proceeds to step S16.

In step S16, the candidate area calculation means 54 specifies a reference label area located at a position lower in the real space than the target label area, and specifies a candidate area 74 including the target label area and the reference label area. The human area determination means 56 obtains the height t (see FIG. 5) of the candidate area 74, and then calculates the estimated height, which is the height of the candidate area 74 in the real space, based on the height t.

In step S18, the human area determination means 56 determines whether or not the estimated height calculated in step S16 is equal to or less than the height threshold value determined based on the reference height included in the reference information 40. When the estimated height exceeds the height threshold value, it can be determined that the candidate area 74 specified in step S16 does not correspond to a person, so the processing of step S12 or less is performed for the other target label areas. If the estimated height is equal to or less than the height threshold value, the process proceeds to step S20.

In step S20, the human area determination means 56 obtains the circumferential distance d between the center of gravity position of the target label area and the center of gravity position of the reference label area, and based on the circumferential distance d (see FIG. 9). Calculate the horizontal distance in real space between the position of the center of gravity of the target label area and the position of the center of gravity of the reference label area.

In step S22, the human area determination means 56 determines whether or not the horizontal distance calculated in step S20 is equal to or less than a predetermined distance threshold value. When the horizontal distance exceeds the distance threshold value, it can be determined that the candidate area 74 specified in step S16 does not correspond to a person, so the processing of step S12 or less is performed for the other target label areas. If the horizontal distance is equal to or less than the distance threshold value, the process proceeds to step S24.

In step S24, the human area determination means 56 calculates the lower body width of the person corresponding to the candidate area 74 in the real space based on the estimated height calculated in step S16 and the stride ratio information included in the reference information 40. ..

In step S26, the human area determination means 56 is specified in step S16, obtains the width lw of the reference label area (see FIG. 6), and then calculates the horizontal width of the reference label area in the real space based on the width lw. .. Further, the human area determination means 56 determines whether or not the calculated horizontal width of the reference label area is equal to or less than the lower body width threshold value determined based on the lower body width calculated in step S24. When the horizontal width of the reference label area exceeds the lower body width threshold value, the human area determination means 56 performs the process of step S12 or less for the other target label areas without integrating the target label area and the reference label area. .. If the horizontal width of the reference label area is equal to or less than the lower body width threshold value, the process proceeds to step S28.

In step S28, the human area determination means 56 integrates the target label area and the reference label area.

Although the embodiments according to the present invention have been described above, the present invention is not limited to the above-described embodiments, and various modifications can be made without departing from the spirit of the present invention.

10 security system, 12 monitored property, 14 security device, 16 communication network, 18 security center device, 20 user device, 22 image sensor, 24 recording device, 30 communication unit, 32 imaging unit, 34 storage unit, 36 captured image , 38 reference image, 40 reference information, 42 signal processing unit, 44 extraction means, 46 label shaping means, 48 tracking means, 50 determination means, 52 initial integration means, 54 candidate area calculation means, 56 person area determination means.

Claims (5)

An image processing device that obtains a human area indicating an image area corresponding to the person from a photographed image of the person.
Stride ratio information indicating the ratio of the stride of the person to the height of the person, a storage means for storing a reference image in advance, and a storage means.
An extraction means for comparing the captured image with the reference image and extracting a changed region changed from the reference image in the captured image.
A candidate area calculation means for obtaining a candidate area for the person, which is composed of a first change area and a second change area presumed to be lower in the real space than the first change area.
The estimated height indicating the height of the candidate region in the real space is obtained, the lower body width in the real space is estimated using the estimated height and the stride ratio information, and the lower body width and the second change region in the real space. A human area determination means for determining the candidate area as the person area related to one person by comparison with the horizontal width, and
An image processing device characterized by comprising. The storage means further stores the reference height of the person in real space.
The human area determination means determines whether or not the candidate area is the human area by comparing the estimated height with the reference height.
The image processing apparatus according to claim 1. The storage means further stores the reference upper body width in the real space of the person.
The human area determination means determines whether or not the candidate area is the human area by comparing the horizontal width of the first change area in the real space with the reference upper body width.
The image processing apparatus according to claim 1 or 2. The human region determining means has a first position indicating either the center of gravity position or the center position of the first change region, and a second position indicating either the center of gravity position or the center position of the second change region. Is determined, and whether or not the candidate region is the human region is determined based on the horizontal distance in the real space between the first position and the second position.
The image processing apparatus according to any one of claims 1 to 3. The stride ratio information indicates the ratio as the moving speed of the person increases.
The extraction means extracts the change region from each of the captured images sequentially acquired.
The candidate area calculation means obtains the candidate area from each of the captured images sequentially acquired, and obtains the candidate area.
Further provided with a tracking means for tracking the candidate area in time,
The human area determination means obtains the movement speed of at least one of the candidate area, the first change area, and the second change area based on the tracking result by the tracking means, and obtains the movement speed and the stride ratio information. To estimate the lower body width using
The image processing apparatus according to any one of claims 1 to 4.

Images