JP4580189B2 - Sensing device - Google Patents

Description

  The present invention relates to a sensing device that detects an intruder based on image information and acoustic information, and a height measuring device that measures a human height based on acoustic information.

  In recent years, security systems that detect intruders or the like that have entered a monitoring space have been widely used. As a sensing device used in such a security system, there is one that detects an intruder or the like using an image acquired by an imaging device such as an image sensor. In a sensing device using such an image, a reference background image is recorded, and a method for detecting an intruder based on a difference image between a captured image and a background image captured every moment, There is a method for detecting an intruder based on a difference image between images.

  For example, in Japanese Patent Laid-Open No. 3-97080, a difference image of luminance between a captured image and a background image is generated, pixels whose difference values are equal to or greater than a predetermined threshold are extracted as variation pixels, and continuous variation pixels are extracted. A technique is disclosed in which, when the number is equal to or greater than a predetermined determination value, it is determined that a changing area constituted by changing pixels is an intruder and an alarm driving signal is output. Further, a method of calculating a moving distance and a moving speed of an object by associating a fluctuating region between a plurality of captured images and tracking (tracking) the moving state, and detecting an intruder based on these values Is also used.

  As a composite sensing device with an image sensor, as described in JP 2000-348265 A, a microwave is transmitted to a monitoring space, and a microwave reflected by an object existing in the monitoring space is received. Thus, a method of detecting an intruder based on a change in the waveform of the received wave is also known. As described above, the detection method using microwaves has a property that it is not affected by intangible changes in the monitoring space such as light and shadow.

Japanese Patent Laid-Open No. 3-97080 JP 2000-348265 A

  However, in the intruder detection using the conventional image sensor, it is necessary to set a very wide tracking search range in order to surely detect even when the intruder runs through the monitoring space. Therefore, there is a high probability that a fluctuating region mistracked due to the influence of noise or the like is erroneously detected as an intruder, or a detection omission occurs.

  Also, by assuming that the fluctuating region is in contact with the floor surface, the distance to the object and the size of the object in the real space are estimated based on the installation height of the imaging camera from the floor and the depression angle with respect to the monitoring space. be able to. However, there are cases where the object region cannot be accurately extracted, for example, the object cannot be distinguished from the background color.At that time, the exact size of the object is not known, so it is impossible to distinguish humans from other small animals. An alarm may be output.

  The present invention has been made in view of the above-described problems of the prior art, and an object of the present invention is to provide a sensing device with improved intruder detection accuracy. In addition, the height of a person can be measured by applying the technology related to the present invention.

  The present invention includes: an imaging unit that acquires a captured image obtained by imaging a monitoring target space; and a transmission / reception unit that transmits a sound wave to the monitoring target space and receives a sound wave from the monitoring target space as an acoustic signal, A sensing device that detects a moving body in the monitoring target space based on a captured image acquired by an imaging unit and an acoustic signal received by the transmission / reception unit, and a difference between the captured image and a reference background image The fluctuation region is extracted from the captured image based on the difference image, and the image processing unit for obtaining the movement direction of the fluctuation region is included in the movement waveform of the fluctuation region and the difference waveform that is the difference between the plurality of acoustic signals. An acoustic processing unit that determines an attitude of an object corresponding to the fluctuation region based on a number of peaks corresponding to the fluctuation region or a temporal change in an interval between peaks. .

Here, the sound processing unit, the moving direction of the change area, when it is substantially maintained direction a distance between the transmitting and receiving unit, spacing of rupees click to respond to the changing area is below a predetermined threshold value When it is large, the intruder corresponding to the fluctuation area is standing, and when the number of peaks corresponding to the fluctuation area is one, or when the peak interval is equal to or less than the predetermined threshold, it corresponds to the fluctuation area. It is determined that the intruder is in a prone position. Further, when the moving direction of the fluctuation region is a direction approaching or moving away from the transmission / reception unit, the acoustic processing unit is configured such that the time change of the interval between peaks corresponding to the fluctuation region is larger than a predetermined threshold value. The intruder corresponding to the fluctuation area is assumed to be in a standing position, and the intruder corresponding to the fluctuation area is in the prone position when the time change of the interval between the peaks corresponding to the fluctuation area is equal to or less than the predetermined threshold. Is determined.

  The image processing unit includes a tracking unit that tracks temporal movement of the fluctuation region, and the tracking unit changes a tracking search range based on the posture of the object determined by the acoustic processing unit. Is preferred.

  Furthermore, it is preferable that the acoustic processing unit further includes an object size calculation unit that calculates a size of an object corresponding to the variation region from an interval between peaks corresponding to the variation region.

  Further, the present invention is a height measuring apparatus comprising a transmitting / receiving unit that radiates sound waves isotropically from an obliquely upward position of a standing human and receives the sound waves reflected by the human as an acoustic signal, the acoustic signal The height of the human being is measured based on a peak interval corresponding to the human being included in. Here, the peak corresponding to the human is a peak corresponding to a reflected wave reflected by the human head and a reflected wave reflected by the human foot.

  The device of the present invention can be realized by a control program that causes a computer to function as a sensing device or a height measuring device that performs the above processing.

  According to the present invention, it is possible to reduce false detection of an intruder due to tracking errors. In addition, false detection of an intruder due to insufficient estimation accuracy of the size of the intruding object can be reduced. Furthermore, the height of the intruder (human) can be measured based on the peak of the acoustic signal.

  As shown in FIG. 1, the sensing device 100 according to the embodiment of the present invention includes an imaging unit 10, an image processing unit 12, a transmission / reception unit 14, an acoustic processing unit 16, a storage unit 18, and an information processing unit 20. The The sensing device 100 can be configured by a computer including an imaging device such as an image sensor and a camera, a sound wave transmission / reception device, and the like.

The imaging unit 10 includes an imaging device such as an image sensor or a camera. The imaging unit 10 converts an optical image of the monitoring space into an electrical image signal and outputs the electrical image signal to the image processing unit 12. It is also preferable that the imaging unit 10 has a function of delivering the image signal to the image processing unit 12 after performing pre-processing such as amplification, filter processing, and digitization processing on the image signal. In the present embodiment, the captured image is converted into a digitized image including discrete pixel groups, and each pixel is input to the image processing unit 12 and stored in the storage unit 18 as a captured image I expressed by a luminance value. And retained. Hereinafter, a captured image I ⁽ⁿ⁾ obtained by the n-th measurement among the captured images I that are repeatedly captured is shown.

The transmission / reception unit 14 includes an ultrasonic pulse transmitter and an ultrasonic sensor. The transmitter / receiver 14 isotropically transmits ultrasonic pulses to the monitoring space at predetermined time intervals using an ultrasonic pulse transmitter, and receives ultrasonic waves from the monitoring space as an acoustic signal. At this time, an ultrasonic pulse reflected by an object or wall existing in the monitoring space is received as an acoustic signal S (t) that varies with time. When the monitoring distance is sufficiently short, the time from transmission of ultrasonic waves to reception is negligible, and by matching the transmission timing of ultrasonic waves with the imaging timing of the imaging unit 10, the monitoring space of the same situation at substantially the same time The situation can be sensed. Hereinafter, the acoustic signal S (t) repeatedly measured and obtained in synchronization with the n-th imaging is indicated as a signal S ⁽ⁿ⁾ (t).

  The received acoustic signal S (t) is transmitted to the acoustic processing unit 16 and stored and held in the storage unit 18. Moreover, it is also preferable that the transmission / reception unit 14 has a function of passing the acoustic signal S (t) to the acoustic processing unit 16 after performing pre-stage processing such as amplification, filter processing, and digitization processing.

  In this embodiment, the case where ultrasonic waves are used will be described as an example. However, the present invention is not limited to this, and sound waves in an audible range may be used. However, ultrasonic waves have the advantage that they cannot be heard by humans. Moreover, it is preferable to change the observation time of the acoustic signal S (t) according to the distance to be monitored. For example, if the distance to be monitored is L (m) and the sound speed is C (m / s), the observation time t (second) of the acoustic signal S (t) necessary for monitoring can be calculated by t = 2L / C. it can.

  Hereinafter, a process of detecting an intruder from the monitoring space using the captured image I acquired by the imaging unit 10 and the ultrasonic acoustic signal S (t) acquired by the transmission / reception unit 14 will be described. The function of each part of the sensing device 100 can be executed by a processing device of a computer by making each step of the flowchart shown in FIG. 2 a control program that can be executed by the computer.

In step S10, an image of the monitoring space is acquired. The imaging unit 10 acquires a captured image I of one frame at a predetermined time interval. Here, it is assumed that the captured image I ⁽ⁿ⁾ is acquired at the n-th imaging timing. The captured image I ⁽ⁿ⁾ is transferred to the image processing unit 12.

In step S12, the image processing unit 12 determines whether or not the captured image I ⁽ⁿ⁾ includes a variable region. As shown in FIG. 1, the image processing unit 12 includes a variable region extraction unit 22, a tracking unit 24, and an image feature amount calculation unit 26. The image processing unit 12 receives the digitized captured image and performs background difference processing, binarization processing, tracking processing, and image feature amount calculation processing.

In the fluctuation region extraction unit 22, a difference image D ⁽ⁿ⁾ is generated by taking a difference value between pixels corresponding to each other in the captured image I ⁽ⁿ⁾ and the reference background image B. The background image B is an image captured in a state where no intruder or the like exists in the monitoring space, and is acquired in advance before starting the intruder detection process, and is stored and held in the storage unit 18. . Next, the difference image is binarized based on the magnitude relationship between the luminance (characteristic value) of each pixel included in the obtained difference image D ⁽ⁿ ) and a predetermined threshold value. That represents each image included in the captured image I ⁽ⁿ⁾ by ^{I (n) (i, j} ), represents each pixel contained in the background image B by B (i, j), the threshold is a T _hv Then, the value of each pixel D ⁽ⁿ⁾ (i, j) included in the difference image D ⁽ⁿ⁾ can be determined by Equation (1).

これによって、図３に示すように、特性値の変動が閾値Ｔ_hv以上の変動であった変動画素（１）と閾値Ｔ_hvより小さい変動であった非変動画素（０）とに２値化された差分画像Ｄ⁽ⁿ⁾として表現される。算出された差分画像Ｄ⁽ⁿ⁾は、次回の撮像画像に対するトラッキング処理の基準とするために記憶部１８に格納及び保持される。

As a result, as shown in FIG. 3, _binarization is performed on the fluctuation pixel (1) whose characteristic value fluctuates more than the threshold value _Thv and the non-fluctuation pixel (0) whose fluctuation is smaller than the threshold value _Thv. Is represented as a difference image D ⁽ⁿ⁾ . The calculated difference image D ⁽ⁿ⁾ is stored and held in the storage unit 18 to be used as a reference for tracking processing for the next captured image.

In the present embodiment, the background image B is an image obtained by capturing the surveillance space in advance in a situation where there is no intruder, but it is also preferable to update the background image B with an image captured every predetermined time. Further, by using the captured image acquired last time as the background image B, a difference image D ⁽ⁿ⁾ between consecutive frames may be obtained and used for processing.

Here, if the captured image I ⁽ⁿ⁾ includes a variable region, the process proceeds to step S14. On the other hand, ^if the fluctuation area is not included in the captured image I ⁽ⁿ⁾ , the process proceeds to step S18. Below, the case where the fluctuation area is included in the captured image I ⁽ⁿ⁾ will be described.

In step S14, the tracking process of the fluctuation region is performed by comparing the difference image D ⁽ⁿ⁾ acquired by the current imaging with the difference image acquired in the past. In the tracking unit 24, substantially continuous variable pixel groups included in the binarized difference image D ⁽ⁿ⁾ are grouped as one variable region, and each variable region is labeled with a unique label. Subsequently, the past difference image held in the storage unit 18 is read, and the current difference corresponding to the tracking search range set around each variable region included in the difference image D ^(n−1). If there are fluctuating areas in the area in the image, the fluctuating areas that are estimated to be areas in which the same subject is imaged are associated with each other based on the similarity of the feature quantities such as the size and shape of the areas. The tracking search range setting method will be described later. The difference image D ⁽ⁿ⁾ and the labeling information are sent to the image feature quantity calculation unit 26.

In step S16, various feature amounts relating to the image are calculated based on the difference image D ⁽ⁿ⁾ and the labeling information. The image feature amount calculation unit 26 includes, for example, an image distance calculation unit, a region size calculation unit, and a movement vector calculation unit.

In the image distance calculation unit, the distance from the imaging device included in the imaging unit 10 to the object imaged in each variable region is calculated based on the difference image D ⁽ⁿ⁾ . By assuming that the floor surface of the monitoring space is flat and the object imaged in the fluctuation region is in contact with the floor surface, the depression angle, the installation height, and the fluctuation region of the image pickup unit 10 in the image of the fluctuation region Based on the position, it is possible to estimate a linear distance from the imaging device to the object imaged in the fluctuation region.

For simplicity, a case will be described in which a variable region is extracted in front of the imaging unit 10 as shown in FIG. The vertical size of the difference image D is Y, the length to the upper end of the fluctuation area in the image is y _h , the length to the lower end of the fluctuation area in the image is y _f , and the distance between the pixels is p. As shown in FIG. 5, when the depression angle of the imaging device is θ, the installation height is H, and the focal length is F, an image in real space from the imaging device to the upper end of the object imaged in the fluctuation region using Equation (2) The distance R _h can be calculated, and the image distance R _f in the real space from the imaging device to the lower end of the object imaged in the fluctuation region can be calculated using Equation (3). When higher accuracy is required, correction may be performed in consideration of the characteristics (lens, CCD) of the imaging unit 10.

以下では、ｎ回目の測定の差分画像Ｄ⁽ⁿ⁾に含まれる各変動領域に対して求められた画像距離Ｒ_fを代表値として画像距離Ｒ_Vi ⁽ⁿ⁾と表す。ｉは、変動領域のラベリング番号である。また、実空間における撮像装置から各変動領域に対応する位置までの位置ベクトルＰ_Vi ⁽ⁿ⁾も算出する。

Hereinafter, the image distance R _f obtained for each variable region included in the difference image D ⁽ⁿ⁾ of the n-th measurement is represented as an image distance R _Vi ⁽ⁿ⁾ as a representative value. i is the labeling number of the variable region. Further, a position vector P _Vi ⁽ⁿ⁾ from the imaging device in the real space to the position corresponding to each fluctuation region is also calculated.

The area size calculation unit calculates the size of the image of each variable area included in the difference image D ⁽ⁿ⁾ . For example, a rectangular area circumscribing each variable area is obtained, and the size of the rectangular area is set as the size of each variable area.

In the movement vector calculation unit, the movement vector of the fluctuation region labeled as the same subject image by the tracking unit 24 is calculated. The movement vector is the speed and direction of movement of the object in real space, and the position vector P _Vi ⁽ estimated by the image distance calculation unit between a plurality of difference images obtained from captured images taken at different times. It can be calculated from the fluctuation of ⁿ⁾ .

  Note that the processing in the image distance calculation unit, region size calculation unit, and movement vector calculation unit is an example, and the method is not limited to the processing method of the present embodiment as long as the same feature amount can be calculated.

In step S18, the received wave fluctuation extraction unit 28 obtains a differential waveform from the acoustic signal S ⁽ⁿ⁾ (t). On the acoustic signal S ⁽ⁿ⁾ (t), as shown in FIG. 6A, a pulse signal reflected by an object existing in the monitoring space is superimposed. Therefore, the received wave fluctuation extraction unit 28 reads out the acoustic signal S ^(n-1) (t) (FIG. 6B) acquired at the previous measurement from the storage unit 18 and acquires the acoustic signal acquired at the previous measurement. Envelope signals of the signal S ^(n-1) (t) and the acoustic signal S ⁽ⁿ⁾ (t) acquired this time are respectively obtained. The envelope signal can be obtained by existing processing. The difference waveform d ⁽ⁿ⁾ (t) (FIG. 6 (c)) is obtained by calculating these differences using Equation (4). A peak appears in the differential waveform d ⁽ⁿ⁾ (t) when there is an object that has newly appeared in the monitoring space or when the object has moved.

ステップＳ２０では、差分波形ｄ⁽ⁿ⁾（ｔ）に含まれているピークが抽出され、ピークが含まれている場合にはピークに対応する位置までの距離が算出される。このステップＳ２０はサブルーチン化されており、受信波変動抽出部２８及び音響特徴量算出部３０において、図７に示すフローチャートに沿って実行される。

In step S20, the peak included in the differential waveform d ⁽ⁿ⁾ (t) is extracted, and when the peak is included, the distance to the position corresponding to the peak is calculated. This step S20 is made into a subroutine, and is executed by the received wave fluctuation extracting unit 28 and the acoustic feature quantity calculating unit 30 according to the flowchart shown in FIG.

In step S20-1, it extracts a region exceeding the threshold T _h of the difference waveform d ^{(n) (t)} as a peak. Here, it is also preferable to detect a peak using a threshold value T _h (t) expressed as a function of time t after reflection of the ultrasonic wave from the transmission / reception unit 14. The threshold value T _h (t) is preferably set to a smaller value as the reflected wave of an object at a farther distance is based on the attenuation of the sound wave distance, that is, as the time t increases. Furthermore, the threshold value T _h (t) can also be set based on the image feature amount (for example, movement speed) obtained in step S16.

For example, as shown in FIG. 8, when a differential waveform d ⁽ⁿ⁾ (t) and a threshold value T _h (t) are represented, a region t _S0 ⁽ⁿ⁾ <t <t _{E0 that is} ^{equal to} or greater than the threshold value _Th (t). ⁽ⁿ⁾ , t _S1 ⁽ⁿ⁾ <t <t _E1 ⁽ⁿ⁾ ,... are extracted as peak regions.

In step S20-2, the peak position which is the maximum value in each peak region is extracted. For ^{_{example, t S0 (n) <t}} <t E0 (n) t for _{^{0 (n), t S1 (}} n) <t <t E1 (n) t 1 (n) for, ... The peak position is determined from each peak area.

In step S20-3, the distance to the object corresponding to each peak position is calculated. Hereinafter, in order to distinguish from the image distance R _Vi ⁽ⁿ⁾ calculated based on the image signal, the distance corresponding to the peak position t _i ⁽ⁿ⁾ is referred to as an acoustic distance R _Ai ⁽ⁿ⁾ . The acoustic distance R _Ai ⁽ⁿ⁾ can be calculated using Equation (5). Here, C (m / s) is the speed of sound.

ステップＳ２２では、音響特徴量算出部３０の体勢検出部において、差分波形ｄ⁽ⁿ⁾（ｔ）から抽出されたピーク位置に基づいて、監視空間に侵入者が存在するとした場合の侵入者の姿勢（形状）を求める。監視空間の天井に送受信部１４を設置して超音波を天井から斜め下方の監視空間に等方的に放射し、監視空間に在る物体からの反射波を受信すると、監視空間に侵入者が存在した場合、侵入者の姿勢によって異なる態様の反射パルスが受信されることとなる。

In step S ^< b ^> 22, the posture detection unit of the acoustic feature quantity calculation unit 30 assumes the intruder's posture when the intruder exists in the monitoring space based on the peak position extracted from the differential waveform d ⁽ⁿ⁾ (t). Find (shape). When the transmitter / receiver 14 is installed on the ceiling of the monitoring space to radiate ultrasonic waves isotropically to the monitoring space obliquely below the ceiling and receive a reflected wave from an object in the monitoring space, an intruder enters the monitoring space. When it exists, the reflected pulse of a different aspect will be received with the attitude | position of an intruder.

FIGS. 9 and 10 show the transition of the peak arrival time when an intruder having a height of 1.7 m stands in the monitoring space and moves at a speed of 0.5 m / s. As shown in FIGS. 9 and 10, two peaks appear at different times t _wf and t _wh in the differential waveform. Since the intruder's head has a spherical shape, as shown in FIG. 11, the ultrasonic waves radiated isotropically to the monitoring space are reflected by a part of the head facing the transmission / reception unit 14 and are transmitted / received. Come back to 14. This reflected wave is detected as a peak corresponding to time _twh . On the other hand, the ultrasonic waves reflected from the back, chest, abdomen, and legs of the intruder are not detected as peaks in the differential waveform because they are directed in a different direction from the transmitter / receiver 14. Further, the ultrasonic wave reflected by the intruder's feet is directed toward the transmitting / receiving unit 14 by reflection on two surfaces forming 90 ° by the intruder's feet and the floor surface. This reflected wave is detected as a peak corresponding to the time t _wf .

  In addition, the change in the time interval of the peak detected from the differential waveform differs depending on whether the intruder moves away from or approaches the transmitter / receiver 14 and crosses so that the distance from the transmitter / receiver 14 is substantially constant. Become. That is, when an intruder moves away from or approaches the transmitter / receiver 14, the interval between two peaks detected from the differential waveform varies depending on the measurement time, as shown in FIG. When the intruder crosses so that the distance from the transmission / reception unit 14 is substantially constant, as shown in FIG. 10, the two peaks detected from the differential waveform depending on the measurement time are substantially constant regardless of the measurement time.

On the other hand, FIGS. 12 and 13 show the transition of the peak arrival time when an intruder lies on the floor of the monitoring space and moves at a speed of 0.5 m / s. As shown in FIG. 12 or 13, the differential waveform includes a pattern in which two peaks appear at different times t _cf and t _ch and a pattern in which only one peak appears at time t _cb . That is, as shown in FIG. 14, when the intruder lies in parallel with the direction of the ultrasonic wave emitted from the transmitting / receiving unit 14, a peak corresponding to the reflected wave reflected by the intruder's head appears. A peak corresponding to the reflected wave reflected at the intruder's foot at time t _ch appears at time t _cf. At this time, the ultrasonic waves reflected from the back, chest, abdomen, and legs of the intruder are not detected as peaks in the differential waveform because they are directed in a different direction from the transmitter / receiver 14. As shown in FIG. 15, when the intruder lies orthogonal to the direction of the ultrasonic wave from the transmission / reception unit 14, reflection by two surfaces forming 90 ° by the intruder and the floor surface. The direction goes to the transmitter / receiver 14.

  In addition, when an intruder is lying down and lying, the peak detected from the differential waveform depending on whether the distance from the transmitter / receiver 14 is closer or closer and that the distance from the transmitter / receiver 14 is almost constant The change in time interval will be different. That is, when an intruder moves away from or approaches the transmitter / receiver 14, the interval between the two peaks detected from the differential waveform is substantially constant regardless of the measurement time, as shown in FIG. When the intruder crosses so that the distance from the transmission / reception unit 14 is substantially constant, one peak is measured regardless of the measurement time as shown in FIG.

  Table 1 summarizes these patterns. As described above, the following four patterns are classified based on the temporal change in the number of peaks and the peak interval included in the differential waveform, and the movement vector of the fluctuation region on the image. (Pattern A) Moving back and forth in the standing position: The moving vector is in the front-rear direction, and the interval between the two peaks included in the differential waveform changes with time. (Pattern B) Moving horizontally in the standing position: The moving vector is in the parallel direction The interval between two peaks included in the difference waveform does not change in time, (Pattern C) 匍匐 moves back and forth: the movement vector is in the front-rear direction, and the interval between the two peaks included in the difference waveform is temporal (Pattern D) Horizontal movement: The movement vector is in the parallel direction, and there is one peak included in the differential waveform.

ステップＳ２２は、具体的にはサブルーチン化されており、図１６に示すフローチャートに沿って処理される。

Step S22 is specifically a subroutine, and is processed according to the flowchart shown in FIG.

In step S22-1, acoustic distances that exist within a predetermined distance range from each image distance are grouped for each variation region of the difference image D ⁽ⁿ⁾ . That is, in the g-th fluctuation region extracted from the difference image D ⁽ⁿ⁾ , all acoustic distances R _Ai ⁽ⁿ within the range of R _Vg ⁽ⁿ⁾ ± Δ with respect to the image distance R _Vg ⁽ⁿ⁾ . ^{) And} their acoustic distance R _Ai ⁽ⁿ⁾ is group g. Here, Δ is preferably an acoustic distance corresponding to general human height.

In step S22-2, the counter m is initialized to 1. The counter m is used as a label for specifying each fluctuation region, and in the following processing, the acoustic distance R _Ai ⁽ⁿ⁾ belonging to each group corresponds to a peak reflected from the intruder's head and feet. It is used to determine whether or not.

In step S22-3, the group of the m-th variable domain determines the acoustic distance R _Ai ⁽ⁿ⁾ farthest from each other among the acoustic distance R _Ai ⁽ⁿ⁾ belonging to the group, than the threshold T _hdl Determine whether it is larger. For example, if the acoustic distances R _Ai ⁽ⁿ⁾ , R _{Ai + 1} ⁽ⁿ⁾ , R _{Ai + 2} ⁽ⁿ⁾ ,... R _{Ai + j} ⁽ⁿ⁾ belong to the group m, the largest acoustics It is determined whether or not the distance R _{Ai + j} ⁽ⁿ⁾ −R _Ai ⁽ⁿ⁾ between the distance R _{Ai + j} ⁽ⁿ⁾ and the smallest acoustic distance R _Ai ⁽ⁿ⁾ is larger than the threshold T _hdl . If it is larger than the threshold value T _hdl , the respective acoustic distances R _{Ai + j} ⁽ⁿ⁾ and R _Ai ⁽ⁿ⁾ are determined to be two peaks caused by the reflected wave from the same object, and the process proceeds to step S22-4. If the threshold value T _hdl or less, it is determined that there is one reflected wave due to the same object, and the process proceeds to step S22-10.

In step S22-4, the magnitude and direction of movement of the mth fluctuation region included in the difference image D (n) are obtained. In order to appropriately determine the movement of the fluctuation area, the position vector P _Vm ^{(n) of} the m-th fluctuation area included in the difference image D ⁽ⁿ⁾ obtained by the n-th imaging and the past n−k times difference image D obtained by the imaging of the position vector P _Vm of m-th variable domain included in the ^{(nk) (nk)} difference image D absolute value of the difference vector is greater than the threshold value T _hd2 the ^(nk) Search for.

As a specific example, as shown in FIG. 17, a case will be described in which the difference image D ⁽ⁿ⁾ includes two fluctuation regions g1 and g2. when m is 1, the variation region g1, past the difference image ^{D (n-1), D} (n-2), successively goes back and ^{D (n-3) ··· D} (nk), a difference image D position vector P _Vg1 at ^{(n) (n)} and the previous difference image ^{D (n-1), D} (n-2), the position vector P _Vg1 at ^{D (n-3) ··· D} (nk) _{^{(n-1), P Vg1}} (n-2), P Vg1 (n-3) ··· P Vg1 difference vector P _Vg1 ⁽ⁿ⁾ -P _Vg1 with each of ^{(nk) (n-1)} , P _Vg1 ⁽ⁿ⁾ _-PVg1 ^(n-2) , _PVg1 ⁽ⁿ⁾ _-PVg1 ^(n-3) ... _PVg1 ⁽ⁿ⁾ _-PVg1 ^(nk) Search for k that is greater than _hd2 . In the example shown in FIG. 17, if the threshold value T _hd2 is 3, k = 2 for the fluctuation region g1. Further, the coefficient a _Vg1 when the difference vector P _Vg1 ⁽ⁿ⁾ −P _Vg1 ^(nk) at that time is ^expressed as P _Vg1 ⁽ⁿ⁾ −P _Vg1 ^(nk) = a _Vg1 ⁽ⁿ⁾ x + b _Vg1 ⁽ⁿ⁾ y ⁽ⁿ⁾ , b _Vg1 ⁽ⁿ⁾ is obtained. Where x is a unit vector corresponding to the horizontal direction of the image (the direction of movement while maintaining a distance from the imaging unit 10 and the transmission / reception unit 14), and y is the vertical direction of the image (approaching the imaging unit 10 and the transmission / reception unit 14) This is a unit vector corresponding to the direction of moving away.

Similarly, when m is 2, the past difference images D ^(n-1) , D ^(n-2) , D ^(n-3) ... D ^(nk) are sequentially traced with respect to the fluctuation region g2. , the position vector P _Vg2 ⁽ⁿ⁾ and the previous difference image D ^(n-1) for the difference image ^{D (n), D (n} -2), in the ^{D (n-3) ··· D} (nk) Position vectors P _Vg2 ^(n-1) , P _Vg2 ^(n-2) , P _Vg2 ^(n-3) ... P _Vg2 ^(nk) and the difference vector P _Vg2 ⁽ⁿ⁾ −P _Vg2 ^{(n− 1)} , P _Vg2 ⁽ⁿ⁾ -P _Vg2 ^(n-2) , P _Vg2 ⁽ⁿ⁾ -P _Vg2 ^(n-3) ... P _Vg2 ⁽ⁿ⁾ -P _Vg2 ^(nk) Search for k whose value is greater than the threshold T _hd2 . For the fluctuation region g2, k = 3. Further, the coefficient a _Vg2 when the difference vector P _Vg2 ⁽ⁿ⁾ −P _Vg2 ^(nk) at this time is ^expressed as P _Vg2 ⁽ⁿ⁾ −P _Vg2 ^(nk) = a _Vg2 ⁽ⁿ⁾ x + b _Vg2 ⁽ⁿ⁾ y ⁽ⁿ⁾ , b _Vg2 ⁽ⁿ⁾ is obtained.

In step S22-5, the moving direction of the mth fluctuation region included in the difference image D ⁽ⁿ⁾ is determined. If the coefficient a _Vgm ⁽ⁿ⁾ obtained in step S22-4 is larger than b _Vgm ^(n), it is assumed that the object corresponding to the fluctuation region is moving in the lateral direction, and the process proceeds to step S22-9. If the coefficient a _Vgm ⁽ⁿ⁾ is equal to or less than b _Vgm ^(n), it is assumed that the object corresponding to the fluctuation region is moving in the vertical direction, and the process proceeds to step S22-6.

In step S22-6, it is determined whether or not there is a change in the interval between the acoustic distances R _{Ai + j} ⁽ⁿ⁾ and R _Ai ⁽ⁿ⁾ corresponding to the two peaks corresponding to the mth fluctuation region. As shown in FIG. 18, | (R _{Ai + j} ⁽ⁿ⁾ −R _Ai ⁽ⁿ⁾ ) − (R _{Ai + j} ^(nk) −R _Ai ^{(nk )} is obtained using k obtained in step S22-4. )) | Is determined whether or not | is larger than a predetermined threshold value T _hd3 . For example, since k = 2 in the fluctuation region g1, | (R _A2 ⁽ⁿ⁾ −R _A1 ⁽ⁿ⁾ ) − (R _A2 ⁽ⁿ⁻²⁾ −R _A1 ⁽ⁿ⁻² )) | It is determined whether i = 1 and j = 1) is larger than the threshold T _hd3 . Threshold T _hd3 greater if it moves the process to step S22-7 as there is a change in the spacing of the peak than, the process proceeds to step S22-8 that there is no change in the distance between the peaks if the threshold value T _hd3 less Let

In step S22-7, when the object corresponding to the m-th fluctuation region included in the difference image D ⁽ⁿ⁾ is an intruder, the intruder stands in a standing posture with respect to the imaging device 10 and the transmission / reception unit 14. It is determined that the person is walking in a direction approaching or moving away. In step S22-8, when the object corresponding to the m-th fluctuation region included in the difference image D ⁽ⁿ⁾ is an intruder, the intruder lies on the imaging device 10 and the transmission / reception unit 14 in a lying posture. It is determined that the person is leaning in the direction of approaching or moving away. In step S22-9, when the object corresponding to the m-th fluctuation region included in the difference image D ⁽ⁿ⁾ is an intruder, the intruder stands from the imaging device 10 and the transmission / reception unit 14 in a standing posture. It is determined that the user is walking with the distance kept substantially constant. In step S22-10, when the object corresponding to the m-th fluctuation region included in the difference image D ⁽ⁿ⁾ is an intruder, the intruder lies down from the imaging device 10 and the transmission / reception unit 14. It is determined that the user is hesitating while keeping the distance substantially constant. These determination results are stored and held in the storage unit 18 in association with an identifier for specifying each variable region.

In step S22-11, it is determined whether or not there remains a variation area in which the posture and the moving direction are not yet determined in the difference image D ⁽ⁿ⁾ . If it is necessary to repeat the process, m is incremented by 1 in step S22-12, and the processes after step S22-3 are repeated. If the posture and the moving direction are determined for all the fluctuation regions, the process is returned to the main routine.

In step S24, the size of the object corresponding to each variable region included in the difference image D ⁽ⁿ⁾ is calculated. Step S24 is converted into a subroutine, and is processed by the size detection unit of the acoustic feature quantity calculation unit 30 according to the flowchart shown in FIG.

  In step S24-1, the counter m is initialized to 1. The counter m specifies a label of the fluctuation area, and is used to specify the fluctuation area for calculating the size of the corresponding object in the following processing.

  In step S24-2, assuming that the mth fluctuation image is an intruder, it is checked whether the intruder is determined to be in a standing posture or a lying posture. If it is determined in step S22 that the mth variation image is in the standing posture, the process proceeds to step S24-6, and if it is determined that the posture is lying, the process proceeds to step S24-3. Shift processing.

  In step S24-3, if the mth fluctuation image is an intruder, it is checked whether it is determined that the intruder has moved back and forth or has moved sideways. In step S22, when it is determined that the mth fluctuation image is moving in a direction approaching or moving away from the imaging device 10 and the transmission / reception unit 14, the process proceeds to step S24-5, and the imaging device 10 is moved. If it is determined that the distance from the transmission / reception unit 14 is approximately constant, the process proceeds to step S24-4.

In step S24-4, the acoustic distance R _Am to the object ^(n), the lateral pixel number P _W, and next to the m-th variable domain of on the difference image D ⁽ⁿ⁾ of the difference image D ⁽ⁿ⁾ The object size h (the height of the intruder) is calculated from Equation (6) using the number of pixels in the direction P _wm ⁽ⁿ⁾ and the angle of view δ in the horizontal direction of the camera. In step S24-5, mathematical expressions are used by using the arrival times t _ch and t _{cf of the} two peaks associated with the m-th fluctuation region, the height H from the floor to the transmission / reception unit 14, and the velocity C of the sound wave. From (7), the size h of the object (height of the intruder) is calculated. In step S24-6, the m-th two peaks associated with the variable domain of the arrival time t _wh and t _wf, using the velocity C of the height H and the sound wave from the floor to transceiver 14 formula From (7), the size h of the object (height of the intruder) is calculated.

ステップＳ２４−７では、差分画像Ｄ⁽ⁿ⁾に未だ大きさ算出していない変動領域が残っているか否かが判断される。さらに処理を繰り返す必要があればステップＳ２４−８でｍを１だけ増加させて、ステップＳ２４−２以降の処理を繰り返す。総ての変動領域について姿勢と移動方向が判定されていればメインルーチンに処理を戻す。

In step S24-7, it is determined whether or not there is a fluctuation region whose size has not yet been calculated in the difference image D ⁽ⁿ⁾ . If it is necessary to repeat the process, m is incremented by 1 in step S24-8, and the processes after step S24-2 are repeated. If the posture and the moving direction are determined for all the fluctuation regions, the process is returned to the main routine.

In step S26, a tracking range is set. If it is determined that the object corresponding to each fluctuation area included in the difference image D ⁽ⁿ⁾ is an intruder, and if it is determined that the object is lying, the search range of the area r is set around the fluctuation area. On the other hand, if it is determined that the object corresponding to each fluctuation area included in the difference image D ⁽ⁿ⁾ is an intruder, the search range of the area R is set around the fluctuation area when it is determined that the object is standing. . At this time, the region R is preferably set larger than the region r.

  In this way, when the object is assumed to be an intruder, the tracking search range is changed depending on the intruder's posture, so that the tracking process is performed in an appropriate search range corresponding to the movable speed of the intruder. Is possible.

In step S28, the instantaneous attribute degree of the fluctuation region is calculated. First, the matching processing unit 32 of the information processing unit 20 performs matching between the image distance and the acoustic distance. The association processing unit 32 determines whether there is an acoustic distance R _Aj ⁽ⁿ⁾ corresponding to the image distance R _Vi ⁽ⁿ⁾ of each variable region included in the difference image D ⁽ⁿ⁾ .

When the matching between the image distance R _Vi ⁽ⁿ⁾ and the acoustic distance R _Aj ⁽ⁿ⁾ is successful, the object obtained in step S24 with respect to the fluctuation region corresponding to the image distance R _Vi ^{(n) for} which the matching is successful. If the size is larger than the size of the person who is generally considered to be the smallest person, the instantaneous attribute level setting unit 34 increases the instantaneous human attribute level, assuming that the variation image corresponds to a human. The instantaneous human attribute is a parameter indicating the probability that a person is captured in a captured image for each measurement, and the higher the parameter value, the higher the probability that a person is captured in the captured image. It is shown. On the other hand, if the size of the fluctuation region is smaller than the size of the minimum person, it is determined that the imaged object is a small animal (dog, cat, mouse, etc.), and the instantaneous small animal attribute is increased. The instantaneous small animal attribute is a parameter indicating the probability that a small animal is captured in a captured image for each measurement, and the higher the parameter value, the higher the probability that a small animal is captured in the captured image. It is shown.

On the other hand, if matching between the image distance R _Vi ⁽ⁿ⁾ and the acoustic distance R _Aj ⁽ⁿ⁾ fails, does the acoustic distance R _Aj ⁽ⁿ⁾ corresponding to a distance closer to the image distance R _Vi ⁽ⁿ⁾ exist? It is determined whether or not. As shown in FIG. 20, when the acoustic distance R _Aj ⁽ⁿ⁾ is present at a distance shorter than the distance range of the image distance R _Vi ⁽ⁿ⁾ , as shown in FIG. There is a high possibility that the small animal such as is imaged and the object appears to be imaged at the image distance R _Vi ⁽ⁿ⁾ of the difference image D in appearance.

Therefore, based on the size of the fluctuation region, it is investigated whether or not a small animal such as a bird or an insect has been imaged. The size of the variation image that should appear in the difference image D ⁽ⁿ⁾ when a small animal of the maximum size that can be considered as a small animal exists at the acoustic distance R _Aj ⁽ⁿ⁾ is obtained, and the variation image actually included in the difference image D is obtained. It is determined whether the subject is a small animal or a human by comparing with the size. The size on the image of the largest small animal is compared with the size of the fluctuation region. If the size of the fluctuation region is equal to or smaller than the size of the largest small animal, the instantaneous attribute setting unit 34 increases the instantaneous small animal attribute. On the other hand, if the size of the fluctuation region is larger than the size of the largest small animal, it is determined that the imaged object is a human, and the instantaneous attribute level setting unit 34 increases the instantaneous attribute level.

If the acoustic distance R _Aj closer than the distance range of the image distance R _Vi ^{(n) (n)} does not exist, the luminance variation area corresponding to the image distance R _Vi ⁽ⁿ⁾ is examined. When the luminance (average luminance) of the variable region is equal to or greater than a predetermined threshold, the instantaneous light attribute level is increased in the instantaneous attribute level setting unit 34 as a variable region caused by intangible light. The instantaneous light attribute is a parameter indicating the probability that intangible light is captured in the captured image for each measurement, and the higher the parameter value, the more invisible light is captured in the captured image. This indicates that the probability is high. On the other hand, when the luminance (average luminance) of the fluctuation region is smaller than the predetermined threshold, the instantaneous shadow attribute level is increased in the instantaneous attribute level setting unit 34 as a fluctuation region caused by an intangible shadow. The instantaneous shadow attribute is a parameter indicating the probability that an intangible shadow is captured in a captured image for each measurement. The higher the parameter value, the more invisible shadow is captured in the captured image. This indicates that the probability is high.

  In step S30, the storage attribute level calculation unit 36 calculates the storage attribute level. The accumulated attribute level is a parameter calculated based on the instantaneous attribute level and the acoustic feature amount obtained by a plurality of measurements, and indicates whether the fluctuation region is a person, a small animal, light, or a shadow. is there. For example, the accumulated person attribute level can be calculated by adding the instantaneous person attribute level obtained in step S28 a predetermined number of times. In this way, it is possible to more accurately determine whether or not the fluctuation region is a person by using the accumulated person attribute degree obtained by integrating the instantaneous person attribute degree a plurality of times. Similarly, the accumulated attribute level corresponding to each instantaneous attribute level can be calculated by integrating the other instantaneous attribute levels for a predetermined number of times of measurement.

  In step S32, the comprehensive determination unit 38 determines whether there is an intruder in the monitoring space using at least one of the instantaneous attribute level and the accumulated attribute level. For example, it is determined that there is an intruder in the monitoring space when the accumulated person attribute level has a value equal to or greater than a predetermined threshold. In addition, it is also preferable to perform a combination determination with other accumulation attribute levels, image feature amounts, and acoustic feature amounts. If it is determined that there is an intruder, the process proceeds to step S34 to perform processing such as issuing an alarm, and if it is determined that there is no intruder, the process ends. Further, the processing may be repeated from step S10.

  As described above, according to the present embodiment, it is possible to reliably detect an object existing in the monitoring space and to specify whether or not the object is an intruder.

It is a block diagram which shows the sensing apparatus in embodiment of this invention. It is a figure which shows the flowchart of intruder detection in embodiment of this invention. It is a figure explaining the production | generation method of the difference image in embodiment of this invention. It is a figure explaining the method to calculate the image distance in embodiment of this invention. It is a figure explaining the method to calculate the image distance in embodiment of this invention. It is a figure explaining the production | generation method of the differential waveform of the acoustic signal in embodiment of this invention. It is a figure which shows the flowchart of the subroutine of step S20 in embodiment of this invention. It is a figure explaining the method of peak detection in an embodiment of the invention. It is a graph which shows the mode of the peak when an intruder stands up and moves in the front-rear direction. It is a graph which shows the aspect of the peak when an intruder moves to a standing position and a horizontal direction. It is a figure explaining the mode of reflection of a sound wave when an intruder is standing. It is a graph which shows the mode of a peak when an intruder moves in a prone position and the front-back direction. It is a graph which shows the aspect of a peak when an intruder moves in a prone position and a horizontal direction. It is a figure explaining the mode of reflection of the sound wave in case an intruder is lying down and moving in the front-back direction. It is a figure explaining the mode of reflection of the sound wave in case an intruder is lying down and moving in the horizontal direction. It is a figure which shows the flowchart of the subroutine of step S22 in embodiment of this invention. It is a figure explaining the method of calculating | requiring the moving amount and moving direction of the object corresponding to each fluctuation | variation area | region. It is a figure explaining the method of calculating the change of the space | interval of the peak corresponding to each fluctuation | variation area | region. It is a figure which shows the flowchart of the subroutine of step S24 in embodiment of this invention. It is a figure which shows the case where matching with image distance and acoustic distance was not able to be taken in embodiment of this invention. It is a figure which shows the condition where the image distance and the acoustic distance were not able to be taken in embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 Imaging part, 12 Image processing part, 14 Transmission / reception part, 16 Sound processing part, 18 Storage part, 20 Information processing part, 22 Fluctuation area extraction part, 24 Tracking part, 26 Image feature-value calculation part, 28 Received wave fluctuation extraction part , 30 acoustic feature amount calculation unit, 32 association processing unit, 34 instantaneous attribute level setting unit, 36 accumulated attribute level calculation unit, 38 comprehensive determination unit, 100 sensing device.

Claims (5)

An imaging unit that acquires a captured image obtained by imaging the monitoring target space;
A transmission / reception unit for transmitting sound waves to the monitoring target space and receiving sound waves as acoustic signals from the monitoring target space;
A sensing device that detects a moving body in the monitoring target space based on a captured image acquired by the imaging unit and an acoustic signal received by the transmission / reception unit,
An image processing unit that extracts a variation area from the captured image based on a difference image that is a difference between the captured image and a reference background image, and obtains a moving direction of the variation area;
An object corresponding to the variation region based on the moving direction of the variation region and the number of peaks corresponding to the variation region included in a differential waveform that is a difference between a plurality of acoustic signals or a temporal change in the interval between peaks. An acoustic processing unit for determining the posture of
A sensing device comprising: The sensing device according to claim 1,
The acoustic processing unit
When the main movement direction of the variable region is a direction that substantially maintains the distance from the transmission / reception unit,
When the peak interval corresponding to the fluctuation area is larger than a predetermined threshold, the intruder corresponding to the fluctuation area is standing,
The sensing device according to claim 1, wherein when the number of peaks corresponding to the fluctuation region is one or the peak interval is equal to or less than the predetermined threshold, it is determined that the intruder corresponding to the fluctuation region is in a prone position. The sensing device according to claim 1,
The acoustic processing unit
When the main moving direction of the fluctuation region is a direction approaching or moving away from the transmission / reception unit,
When the time change of the interval between the peaks corresponding to the fluctuation area is larger than a predetermined threshold, the intruder corresponding to the fluctuation area is standing,
The sensing device, wherein when the time change of the interval between the peaks corresponding to the fluctuation region is equal to or less than the predetermined threshold, it is determined that the intruder corresponding to the fluctuation region is in the prone position. In the sensing device according to any one of claims 1 to 3,
The image processing unit includes a tracking unit that tracks temporal movement of the variable region,
In the tracking unit, the tracking search range is changed based on the posture of the object determined in the acoustic processing unit. In the sensing device according to any one of claims 1 to 4,
The acoustic processing unit includes an object size calculation unit that calculates a size of an object corresponding to the variation region from an interval between peaks corresponding to the variation region.

Images