JP5691764B2 - Abandoned or removed detection system - Google Patents

Description

  The present invention relates to a leaving or removal detection system, and more particularly to a leaving or removal detection system having an output data automatic generation function.

Conventionally, for example, in an area where a large number of unspecified persons come and go, a surveillance camera that records the behavior of a person who has placed an object at the time of entry into the surveillance area from area shooting information at the time when the abandoned object was left A system is known (see Patent Document 1).
In this system, when the intruder is detected, the surveillance camera records the intrusion time, and further tracks the detected intruder. If there is an object left behind after the intruder has passed through the monitoring area and the object remains stationary for a certain period of time, the alarm information and the intrusion time are transmitted to the video recording apparatus. The video recording device overwrites and records video information transmitted from the surveillance camera for each fixed amount of information in the memory, and when receiving alarm information and a person's intrusion time, from the intrusion time to the time corresponding to the alarm information. Transfer video information from memory to hard disk and record to hard disk.

In addition, when an object is left or taken away, it is automatically detected, and a surveillance image memory that automatically identifies the person by photographing the face of the person who left or removed the object A system is also known (see Patent Document 2).
In this system, the time at which the object detection image was taken is specified as the object detection time, and the face image information taken during the past delay time from this object detection time is used to leave or take away It tries to identify a person.

  All of these systems respond to the need to see the situation and person at the moment when the object was left or removed in addition to the detected object that was left or removed, and installed in the monitoring area. Input the video (or image) of the surveillance camera, and after the detection of leaving or taking away, cut out and display or save the camera video before the detection time required to determine that it was set to leave or take away Yes.

  In these systems, when the timer value of the timer reaches a predetermined time (when the state of leaving or taking away continues for a predetermined time), it is determined that there has been leaving or taking away, and leaving has occurred based on that determination. A video before a predetermined time from the occurrence time is cut out and played back or saved.

  However, even if the timer reaches a predetermined time, there is no guarantee that leaving or removal has occurred at the time of occurrence. In other words, the occurrence time at which the leaving or taking-out actually occurs may be before or after the actual occurrence, and the accuracy of the leaving or taking-out occurrence time may be deteriorated.

  If the accuracy of the time of leaving or taking away is poor, for example, if a video at the time interval before leaving or taking out is cut out and an image (output image) at the time of leaving or taking out is obtained, the time interval is set. It is necessary to set a longer time and cut out a long-time image. In that case, since there is always unnecessary data in the recorder (or storage), not only does the amount of data to be saved increase, but also in order to identify the person who has actually left or taken away, When confirming by looking at a video (image), since there are many unrelated images in the image data, the work is labor intensive.

JP 2010-187124 A JP 2010-88072 A

  The present invention has been made in view of the above-described conventional problems in a leaving or taking away detection system (hereinafter simply referred to as a detection system), and the purpose thereof is to image a person who has left behind in a monitoring area. In this case, the capacity of the storage means for storing the confirmation image can be confirmed with a moving image or a still image (herein, collectively referred to as an image) captured within a short time. It is also necessary to reduce the labor of the person who visually confirms.

According to the first aspect of the present invention, an image capturing unit that captures an image of the monitoring area, a storage unit that stores the captured image, a leaving or removal determining unit, and a cutout range of the image stored in the storage unit are created. A leave or take-away detection system having a cutout range creation unit, an output image creation unit, and a display unit for displaying an output image, wherein the leave or take-off determination unit has determined that there has been a leave or take away The cutout range creation unit creates a cutout range based on a position within a predetermined range including the position of the foreign object and a predetermined period predicted to include the leaving or removal occurrence time, and creates the cutout range The unit also creates a cutout range based on the movement trajectory of the moving object created by the tracking unit that forms the movement trajectory of the person in the monitoring area. In the cutout range, only the part where the movement locus exists is cut out, and the cutout part is pasted to create a cutout range, and the output image creation unit is configured to capture an image captured in the cutout range. A left or taken away detection system characterized in that an output image is created by cutting out from an image stored in a storage means .
The invention of claim 2 is the desertion or taking away detection system according to claim 1, wherein the image is a desertion or taking away detection system characterized in that it is a moving image.
A third aspect of the present invention, the desertion or taking away detection system according to claim 1, wherein the image is a desertion or taking away detection system characterized in that it is a still image.
According to a fourth aspect of the present invention, in the leaving or taking away detection system according to the third aspect , the moving body specifying that identifies the moving body related to the leaving or removing based on the position of the foreign matter and the movement locus of the moving body. A leaving or taking away detection system characterized by comprising means.
The invention of claim 5 has a best shot selection unit for selecting a best shot in the leaving or removal detection system according to claim 4 , wherein the best shot selection unit is specified by the moving body specifying means. A left or taken away detection system characterized by selecting a best shot image of a moving object.

  According to the present invention, when a moving image is cut out, not only a predetermined time (Δt) is cut out but also cut out only when a person leaves and passes near the site, so that the search target image can be a short video. In addition to reducing the storage capacity of the video storage means, it is possible to reduce the burden on humans who check moving images.

It is a block diagram which shows this detection system roughly. It is a time chart showing the accumulated timer value of the detection timer T1 that changes with time, with the timer value of the detection timer T1 on the vertical axis and the time T on the horizontal axis. It is a figure showing the locus | trajectory of the mobile bodies A, B, and C in the monitoring area | region produced by the tracking part by taking the distance Y of the vertical direction on the vertical axis | shaft, and taking time T on the horizontal axis. Although it is the same figure as FIG. 3, the area | region S1 enclosed by the range of distance (DELTA) y and the range of time (DELTA) t on a time axis centering on the position y1 of a foreign material is set. It is a figure which shows three-dimensional locus | trajectory data cut out by distance (DELTA) x, distance (DELTA) y, and time (DELTA) t in three dimensions. This is an image created by superimposing the position of a foreign object, the trajectory of a moving object, and the monitoring region S2 in the X-axis and Y-axis directions on an actual captured image. FIG. 7A is a diagram showing the trajectories of the moving bodies A and B in the time zone when the leaving occurred, and FIG. 7B is a diagram showing the range of moving images to be stored in the moving bodies A and B. It is a flowchart which shows the process sequence in the case of cutting out a moving image. It is a flowchart which shows the process sequence in the case of cutting out a still image. It is a flowchart which shows the process sequence of the tracking with best shot evaluation.

FIG. 1 is a block diagram schematically showing the present detection system.
This detection system includes a control unit 100, a monitoring camera 20 that is a photographing device as an external device connected to the input unit 16 of the control unit 100, a keyboard 21 for changing settings, a mouse 22, and a control. Display device (monitor) 30 that displays a still image or a moving image as an external device connected to the output unit 18 of the unit 100, and a speaker for notifying by voice or the like when it is determined to leave or take away 31 and a recorder 32 having storage means such as a hard disk (HDD) for storing an image captured by the monitoring camera 20 captured via the input unit 16 of the control unit 100 together with the capturing time. .

  The control unit 100 includes a computer including a central processing unit (CPU) 10, a read only memory (ROM) 12, a random access memory (RAM) 14, an input unit 16 that is an input side interface (I / F), and an output. The output unit 18 is a side interface (I / F). The control unit 100 controls the entire detection system and functions as a function realization unit realized by a program. An initial setting unit 101, a foreign object detection unit 102, a left or taken away determination unit 103, a tracking unit 104, A cutout range creating unit 105, an output image creating unit 106, a moving object specifying unit 107, and a best shot selecting unit 108 for selecting the best shot of the moving object are provided.

The foreign object detection unit 102 converts a background image (image data) stored in the recorder (or storage) 32 in the monitoring area and an input image (image data) obtained by A / D converting the captured image of the monitoring camera 20, for example, Foreign matter is detected by obtaining a difference value of a luminance image formed by binarizing the luminance value of each pixel with a threshold value.
The leaving or leaving determination unit 103 monitors the foreign matter detected by the foreign matter detection unit 102, measures the time with a detection timer described later, and when the accumulated timer value of the detection timer becomes a value representing the threshold value, It is determined to be left behind or taken away (the timer value may be a count value of the counter or a value representing the time itself).

The tracking unit 104 obtains, for example, a temporal change in position coordinates of the moving body, tracks the movement of the moving body in the image captured by the monitoring camera 20, and creates a trajectory thereof (note that the tracking process is performed on the moving body). The creation of the movement locus itself is known as described in, for example, JP-A-2005-69977).
In the present embodiment, the tracking unit 104 starts tracking when a moving object enters the monitoring area, and creates a movement locus thereof.

The cut-out range creation unit 105 is stored in the recorder 32 based on setting values (for example, t, Δt, Δx, Δy, which will be described later) input by the user with the keyboard 21 or the mouse 22 and set in the initial setting unit 101, for example. A range of the image (image data) to be cut out from the obtained image (image data) is created.
The output image creation unit 106 follows the cutout range created by the cutout range creation unit 105, and the image (image data) in the region S2 surrounded by the vertical and horizontal lengths Δy and Δx in the image captured during the period Δt (FIG. 6). Is cut out from the image stored in the recorder 32 to create an output display image.
The moving object specifying unit 107 specifies a moving object (person) that has been left or removed based on the moving locus of the moving object.
The best shot selection unit 108 selects the best shot image of the moving object specified by the moving object specifying unit 107.

FIG. 2 is a time chart showing the accumulated timer value of the detection timer T1 that changes with time, with the timer value of the detection timer T1 on the vertical axis and the time T on the horizontal axis.
As shown in the figure, when the foreign object detection unit 102 detects a foreign object at time t0 (for example, detection of leaving), measurement of the timer value of the timer T1 is started from that time t0, and the measured accumulated timer value reaches the threshold value t. Sometimes, the leaving or taking away determination unit 103 determines that there has been a leaving. Note that the abandonment detection (or recognition) time t1 on the time axis T indicates the time (time) at which it was determined that the abandonment, and the abandonment detection time t2 represents the actual time until the timer value of the detection timer T1 reaches the threshold value t. Show. In other words, the detection of a foreign object related to leaving or taking away is calculated as t2 hours before the time when it is determined to be left behind or taken away. However, since there is no guarantee that a foreign object has been reliably detected before t2 hours, here, a period Δt that is considered to include the true leaving or taking-out occurrence time is set in advance. The Δt is set in the initial setting unit 101 by the user from the keyboard 21, for example, together with the threshold value t of the timer.

  In this detection system, when the monitoring camera 20 is started after setting the setting value in the initial setting unit 101, the foreign object detection unit 102 has already described the image captured by the monitoring camera 20 while being overwritten and stored in the recorder 32 by a certain amount. A luminance image of the input image and the stored image (background image) is created as described above, and the foreign object and its position are detected from the difference. When the foreign object detection unit 102 detects the occurrence of a foreign object, that is, left behind at a certain time t0, the detection timer T1 starts measurement. During measurement, for example, in the case of leaving, when a foreign object that is left behind cannot be detected in the monitoring area, the detection timer T1 (timer value thereof) is stopped from increasing or reversely controlled. When the cumulative timer value reaches the threshold value t as indicated by the line graph shown in FIG. 2, the leaving or taking away determination unit 103 determines that there has been leaving or removed.

FIG. 3 is a diagram showing the trajectories of the moving bodies A, B, and C in the monitoring area created by the tracking unit 104 with the vertical distance Y on the vertical axis and the time T on the horizontal axis.
Since the moving bodies A, B, and C move with the passage of time T in the X and Y planes that are the monitoring areas, the actual trajectory data is originally three-dimensional (X, Y, and T). Here, the two-dimensional display, ie, the X-axis (lateral movement) in the monitoring area is ignored, and the screen is configured simply by the Y-axis (position in the vertical direction of the image) and the time axis T (elapsed time). It is written.
In the figure, y1 indicates the position (shown as a point on the Y axis) of a foreign object (left or left) on the Y axis.

In FIG. 3, the moving object A first enters from the upper end of the screen and does not stop at all, and has moved out to the lower end as time passes.
Next, the moving body B enters from the lower end of the screen and temporarily stops at the position y1 (until time ta to tb), and then exits to the upper end, and the moving body C finally enters from the upper end of the screen and stops temporarily. It shows that it has returned without changing through y1 and has exited to the upper end of the screen before moving body B.

FIG. 4 is a view similar to FIG. In the figure, a rectangular area S1 surrounded by a distance range Δy on the Y-axis and a period Δt on the time axis with the position y1 as the center is a cut-out range created by the cut-out range creation unit 105.
That is, this area S1 includes the trajectory of the moving object and the position (information) of the foreign object related to leaving or removing, and is not only used for specifying the moving object (person) related to leaving or removing, which will be described later. This is a cut-out range for cutting out an image for identifying the moving object.
By creating this area S1, the moving body C that has not passed through the area S1 can be left behind and excluded from the candidates, and the candidates can be narrowed down to only the moving bodies A and B.

  Next, the moving body specifying unit 107 compares the candidates narrowed down in this way, and specifies the optimum candidate from among them. In other words, the moving object specifying unit 107 has always been in the region S1, in which the moving object B stays in the vicinity of the position y1 that is the leaving position (ta to tb) is longer than the moving object A (zero). It is determined that B is more likely to be a leaver than A because it is always within the distance range Δy that is in the vicinity of the foreign object during Δt.

  Incidentally, when the trajectory data is cut by Δx, Δy, Δt in three-dimensional coordinates instead of the two-dimensional coordinates, it is shown in FIGS. 5A and 5B. FIG. 6 shows an image obtained by superimposing the position of the foreign object, the locus of the moving object, and the monitoring area S2 in the X-axis and Y-axis directions on the actual captured image.

By the way, even if the moving body specifying unit 107 specifies the moving body A, it is necessary to display an image on the monitor 30 in order to identify the moving body (here, a person) A.
Therefore, first, with reference to FIG. 7, a description will be given of how to cut out a moving image when the moving object A is identified using the moving image when the image is a moving image.
FIG. 7 is a view similar to FIG. 4 and shows the trajectories of the moving bodies A and B. Here, when the moving body is outside the range of Δy, the relationship with the foreign matter that is left behind or removed is light, and it is not necessary to store these image data as output image data. Further, it can be seen from the movement trajectory that the moving object A was within the range of Δy outside the period Δt during which it was assumed that the object was left or taken away.
Therefore, the storage is required only when the mobile objects A and B exist within the distance Δy in the vicinity of the foreign object in the period (time zone) Δt in which the mobile objects A and B are considered to have been left or removed.

Therefore, the output image creation unit 106 creates an image for storage by cutting out an image in the range defined by the region S1, that is, a moving image having a distance range in the Y-axis direction of Δy and a shooting time interval of Δt.
At this time, in this embodiment, in order to reduce the data amount of the stored image, the cutout range creation unit 105 cuts and pastes each time region (hatched portion) when the moving bodies A and B exist in the region S1. The output image creation unit 106 creates an output image by editing a moving image (FIG. 7B) shot in the corresponding time zone based on the cutout range, and saves only this portion. It is said.

Next, in the detection system described above, a processing procedure for cutting out a necessary moving image from an image will be described with reference to the flowchart of FIG.
First, a process of A / D-converting video captured by the monitoring camera 20 and inputting it to the control unit 100 is performed (S101). Subsequently, the foreign object detection unit 102 reads the input image and the past image (background image) already stored in the recorder 32, creates each luminance image as described above, takes the difference between them, and detects the foreign object. Detection is performed (therefore, detection of leaving or removal has occurred) (S102). The tracking unit 104 starts tracking at the same time as detecting a foreign object (S103). Subsequently, when a predetermined detection time t1 has elapsed from the foreign object detection time (the time at which the removal or removal has occurred) t0, the removal or removal determination unit 104 determines that there has been removal or removal (S104), The cutout range creation unit 105 creates a cutout range (S1) (S105). The output image creation unit 106 cuts out a moving image shot during the period Δt around the foreign object detection time t0. In this case, as already described with reference to FIG. 7, for example, the output image creation unit 106 has the respective time regions (hatched lines) when the mobile objects A and B calculated based on the tracking data exist in the region S1. (Part) is cut and pasted, and the video in the area S2 of the moving image (FIG. 7B) shot in the cut and pasted time area is cut out and edited to create an output video (S106). The moving image is output to the monitor 30 (S107), and the person who has taken away or left the moving image from the moving image is identified.

Next, a case where a moving body is identified using a still image when the image is a still image will be described.
Even in the case of a still image, a cutout range is formed in the area S1 in FIG. 7 as in the case of a moving image, and a similar output image is created and stored. Thereby, the storage capacity of the output screen can be saved.
In this case, after identifying the moving object, the face of the person who is the moving object A is recognized, and the best shot image is selected when the tracking unit 104 tracks the moving object, and only the best shot image is selected. It can also be saved. In this case, the storage capacity can be greatly reduced.

  Here, in order to recognize a face from the identified moving body image, the control unit 100 performs image processing on the input image (image data) to recognize a human face included in the captured image. A face recognition unit may be provided. As a face recognition process for recognizing a human face included in a captured image, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2004-326281 can be used.

Further, the best shot image is an image that clearly displays a person who is left behind or left behind, for example, the following image (video). That is,
(I) The face is clearly visible, the image (video) is large,
(Ii) An image (still image or moving image) that does not overlap (not be hidden) with other people or objects. Note that the selection of the best shot image itself is already well known, and therefore, the description thereof is omitted by exemplifying them (for example, JP 2005-227957 A and JP 2006-92026 A). No., No. 2007-190076, No. 2009-55156, and No. 2009-89077).

  When the best shot image is cut out from the moving image, the output image creation unit 106 selects only the moving image portion where the moving body (person) exists near the left or removed position (cutout region S2 in FIG. 6). By creating and saving an output image and providing it, it is possible to reduce the burden on the person who checks the moving image later.

Next, a processing procedure for extracting a necessary still image from an image stored by the detection system will be described with reference to a flowchart of FIG.
As described with reference to FIG. 8, first, an image (image data) photographed by the monitoring camera 20 is A / D converted and input to the control unit 100 (S201). Subsequently, the foreign matter detection unit 102 reads the input image and the background image (past image) already stored in the recorder 32, creates each of the luminance images, and takes the difference between them to detect the foreign matter (and therefore leave behind). Alternatively, it is detected that removal has occurred (S202). When a foreign object is detected, the tracking unit 104 starts tracking, and at the same time, a moving object (person) related to the moving object specifying unit 107 leaving or taking away the still image taken within the period Δt. The best shot selection unit 108 performs evaluation and selection of the best shot of the specified person (here, these are collectively referred to as tracking with the best shot evaluation) (S203).

  Subsequently, when a predetermined detection time t1 has elapsed since the foreign object detection, the removal or leaving determination unit 103 determines that there has actually been removal or leaving (S204), and the moving body identification unit 107 creates the cutout range. From the trajectory of the moving body in the rectangular region S1 surrounded by the period Δt before and after the time when the leaving or removal occurred and the range Δy before and after the position y where the removal or removal occurred, cut out by the unit 105 The best shot image (still image) that identifies the moving body (person) that has been left or taken away according to the criteria already described, and that can identify the face of the moving body (person), is obtained. The still image is output to the monitor 30 (S205), and the supervisor identifies the moving body (person).

FIG. 10 is a flowchart for explaining the processing procedure of tracking with best shot evaluation in FIG.
That is, the tracking unit 104 determines whether or not there is a tracking candidate within a period Δt around the time t0 when the object is left or removed when a foreign object is detected (that is, when the object is left or removed). If not (S301), (S301, none), the process is terminated.
When there is a tracking candidate (Yes in S301), matching processing with the trajectory data of the candidate's past image (for example, the identity is quantified by a plurality of parameters and created as an index (identity index)). The identity is judged from the degree of match) (S302), and the trajectory data of the candidate is updated (S303). Subsequently, the output image creation unit 106 cuts out an image based on the region S1 created by the cutout range creation unit 105 (S304), and as described above, the best shot selection unit 108 evaluates the best shot of the captured image. Are compared (S305), and the best shot still image is updated (S306). The tracking process with the best shot evaluation may be performed every frame.

According to the present embodiment, when a moving image is cut out, not only a predetermined period (time zone) (Δt) is cut out but also cut out only when a person leaves and passes near the site (area S1). Thus, the video can be provided with a shorter length, and not only the storage capacity of the video storage means can be reduced, but also the burden on the person who confirms the video can be reduced.
In addition, since priority can be obtained from the trajectory that has passed through the cutout range (the range surrounded by Δx, Δy, and Δt), the priority is not cut out when people leave and pass near the site. It is also possible to cut out only a range where a person with a high height exists.
In addition, since it is possible to provide only a still image of a person who has passed through the cut-out range instead of a moving image, a more slim and speedy operation is possible.

  10 ... CPU, 12 ... ROM, 14 ... RAM, 16 ... input unit, 18 ... output unit, 20 ... monitoring camera, 21 ... keyboard, 22 ... mouse 30 ... monitor, 31 ... speaker, 32 ... recorder, 100 ... control unit, 101 ... initial setting unit, 102 ... foreign matter detection unit, 103 ... leave or take away Judgment unit, 104 ... tracking unit, 105 ... cutout range creation unit, 106 ... output image creation unit, 107 ... moving body identification unit, 108 ... best shot selection unit.

Claims (5)

An imaging unit that captures an image of the monitoring area, a storage unit that stores the captured image, a leaving or removal determination unit, a cutout range creation unit that creates a cutout range of the image stored in the storage unit, and an output A leave or take away detection system having an image creation unit and a display unit for displaying an output image,
When the leaving or taking-out determining unit determines that there has been leaving or taken away, the cutout range creating unit predicts that it includes a position within a predetermined range including the position of a foreign object and the time at which the leaving or removed has occurred. Create a clipping range based on a predetermined period,
The cutout range creating unit creates a cutout range based on the moving locus of the moving body created by the tracking unit that forms a movement locus of the person in the monitoring area, and the moving locus exists in the cutout range. Cut out only the part to be cut out and paste the cut out part to create a cutout range,
The left-out or left-out detection system, wherein the output image creation unit creates an output image by cutting out an image captured in the cutout range from an image stored in the storage unit . In the leaving or taking away detection system according to claim 1 ,
A left or removed detection system, wherein the image is a moving image. In the leaving or taking away detection system according to claim 1 ,
A left or taken away detection system, wherein the image is a still image. In the leaving or taking away detection system according to claim 3 ,
A leaving or removal detection system comprising moving body specifying means for specifying a moving body related to leaving or taking away based on the position of the foreign object and the movement locus of the moving body. In the leaving or taking away detection system according to claim 4 ,
It has a best shot selection part that selects the best shot,
The left shot or removal detection system, wherein the best shot selection unit selects a best shot image of a moving object specified by the moving object specifying means.

Images