JP6947508B2 - Moving object detection device, moving object detection system, and moving object detection method - Google Patents

Description

The present invention relates to a moving object detection device or the like that detects a predetermined moving object (whole object or part of the object) from a moving image.

Object detection plays an important role in many image processing tasks. The motivation behind object detection is to detect an object or part of an object in an image. Furthermore, in order to receive information on the size of an object and the distance to the object in reality, high accuracy is required in estimating the size of the object.

In object detection, a so-called sliding window is generally used. The idea behind this approach is to take an arbitrary area from the image and calculate the likelihood that this area represents a particular moving object. If this possibility exceeds a given threshold, the object will be detected. High accuracy is achieved according to this sliding window approach. However, this approach has the disadvantage of high computational cost when a priori information about possible object positions is not available. In this case, the windows need to be placed in many different positions and sizes in order to reach a given threshold of possibility. Often more than a few thousand areas must be tested to detect an object.

In addition, this sliding window approach often utilizes template matching to calculate the likelihood of a particular object in the area. These templates may be predefined and do not fit well as the appearance of the object changes over time. In this case, the template must be updated to refit with high accuracy.

For example, as a technique for detecting an object, the coordinates of facial feature points detected for each frame image are compared between frame images to verify the detection accuracy, and the coordinates of facial feature points that do not meet the predetermined detection accuracy. There is known a technique for suppressing a decrease in the accuracy of face authentication by correcting the detection result of the detected facial feature points (see Patent Document 1).

Japanese Unexamined Patent Publication No. 2013-65119

In the process of detecting an object from an image, it is required to detect the position of the object in the image and the size of the object with high accuracy. Further, it is required to reduce the calculation cost required for the process of detecting an object.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of quickly and appropriately detecting a predetermined moving object from a moving image.

In order to achieve the above object, the moving object detection device according to one viewpoint is a moving object detection device that detects a moving object to be detected from a moving image, and has a moving image input unit for inputting the moving image and a moving object in the moving image. A keypoint processing unit that identifies the corresponding keypoints between two frame images, identifies the motion vector of the keypoints, and generates a set of keypoints that are supposed to constitute the same moving object based on the motion vector. A first region estimation unit that estimates a first region containing key points of the same set in one frame image of two frame images, and one or more second regions that have undergone predetermined deformation for the first region. A second region determination unit, an object classification unit that evaluates the certainty that the image of the region is a moving object to be detected, and an object classification unit for the first region and the second region. A region having a high probability of being a moving object to be detected from the first region and the second region is provided as an existing region determining unit for determining the existing region in which the moving object exists.

According to the present invention, a predetermined moving object can be quickly and appropriately detected from a moving image.

FIG. 1 is an overall configuration diagram of a moving object detection device according to the first embodiment. FIG. 2 is a functional configuration diagram of the moving object detection device according to the first embodiment. FIG. 3 is a diagram illustrating key point processing by the key point processing unit according to the first embodiment. FIG. 4 is a diagram illustrating a candidate region according to the first embodiment. FIG. 5 is a flowchart of the moving object detection process according to the first embodiment. FIG. 6 is a diagram showing an example of range specifying information according to the second embodiment. FIG. 7 is a flowchart of the moving object detection process according to the second embodiment. FIG. 8 is an overall configuration diagram of the moving object detection system according to the third embodiment. FIG. 9 is a sequence diagram of a part of the moving object detection process according to the fourth embodiment.

Some embodiments will be described with reference to the drawings. It should be noted that the examples described below do not limit the invention according to the claims, and all of the elements and combinations thereof described in the examples are indispensable for the means for solving the invention. Is not always.

<First Example>
FIG. 1 is an overall configuration diagram of a moving object detection device according to the first embodiment.

The moving object detection device 10 is composed of, for example, a PC (Personal Computer), and inputs a CPU (Central Processing Unit) 11, a memory 12, an auxiliary storage device 13, an external device I / F14, and a display device 15. A device 16 and a bus 17 that connects each unit in a communicable manner are provided.

The external device I / F 14 is an interface for communicating with an external device (for example, a surveillance camera 20) via a cable or the like. The surveillance camera 20 is installed so as to observe the monitored portion, and outputs moving image data of the monitored portion to the moving object detection device 10.

The CPU 11 executes a program called from the auxiliary storage device 13 to the memory 12, and executes various processes using the data stored in the auxiliary storage device 13. The memory 12 is, for example, a RAM (RANDOM ACCESS MEMORY), and stores a program executed by the CPU 11 and necessary information. The auxiliary storage device 13 is, for example, a bird disk, a flash memory, or the like, and stores a program executed by the CPU 11 and data used by the CPU 11. The display device 15 is, for example, a liquid crystal display and displays various information. The input device 16 is, for example, a mouse, a keyboard, or the like, and accepts various inputs.

FIG. 2 is a functional configuration diagram of the moving object detection device according to the first embodiment.

The moving object detection device 10 includes an image input unit 101 as an example of a moving image input unit, a background subtraction processing unit 102, a key point processing unit 103, and an initial area estimation unit 104 as an example of a first area estimation unit. A candidate area determination unit 105 as an example of the second area determination unit, an object classification unit 106, an existence area determination unit 107, and an existence area display unit 108 are provided. The CPU 11 of the image input unit 101, background subtraction processing unit 102, key point processing unit 103, initial area estimation unit 104, candidate area determination unit 105, object classification unit 106, existence area determination unit 107, and existence area display unit 108 This is realized by executing the program read into the memory 12.

The image input unit 101 inputs a moving image (an image of each frame of the moving image (frame image)) taken by the surveillance camera 20.

The background subtraction processing unit 102 executes a process of acquiring background subtraction in the input frame image (background subtraction processing). In background subtraction processing, a background image that does not include moving objects at the monitoring location captured by the surveillance camera 20 is required. This background image is, for example, taken in advance by the surveillance camera 20 and stored in the auxiliary storage device 13. The background image determines parts of the frame image that are not important in detecting moving objects.

The background subtraction processing unit 102 executes a pixel-by-pixel comparison between a new frame image taken by the same surveillance camera 20 and a background image. Specifically, the background subtraction processing unit 102 compares the color information of each pixel of the frame image with the color information of the same pixel of the background image. When the color difference between the pixels exceeds a predetermined threshold value, the image difference processing unit 102 marks the pixels as different, that is, representing a moving object. Further, when the pixels have the same color information, the image difference processing unit 102 marks the pixels as the background. As a result, the pixel groups marked as different in the frame image indicate a moving object, and key points (feature points) are present in the pixel groups.

The key point processing unit 103 extracts each key point (feature point) of two different frame images (for example, two time-consecutive frame images), and matches the key points between the respective frame images. , The moving vector of the matching key points between the frame images is specified, and the key point processing for grouping (clustering) the key points indicating the same moving object based on the moving vector is executed.

FIG. 3 is a diagram illustrating key point processing by the key point processing unit according to the first embodiment.

In the keypoint processing, the keypoint processing unit 103 performs edge detection on the frame image subjected to background subtraction processing and calculates the gradient at the edge. A large gradient is an indicator of good key points. The key point processing unit 103 extracts key points based on the gradient. After determining the key point, the key point processing unit 103 calculates the feature vector of the key point by using the edge pattern.

The key point processing unit 103 calculates the feature vectors for all the key points extracted from the two different frame images, and then calculates the Euclidean distance for each. The key point processing unit 103 considers that the feature vector corresponding to each key point in one frame image is the key point that best matches the key point corresponding to the closest feature vector in the other frame image. 3A and 3B show a key point 31 in one frame image f1 and a key point 32 in the other frame image f2 corresponding to the key point 31. At this stage, an inaccurate match occurs. However, there is no problem because the inaccurate match is detected in the subsequent stages.

When performing key point clustering, the key point processing unit 103 calculates a motion vector between matched key points. The key point processing unit 103 acquires matching key points from two consecutive frame images and analyzes the position of each pixel in the two frame images. The key point processing unit 103 allocates a motion vector by calculating the Euclidean distance in the frame of the frame image. FIG. 3C shows the motion vector 33 thus obtained.

Next, the key point processing unit 103 performs clustering to make key points belonging to the same moving object into the same cluster (group). The key point processing unit 103 regards adjacent key points whose motion vectors indicate the same direction as belonging to the same moving object and makes them the same cluster. Further, in the key point processing unit 103, as in the key point 35 of FIG. 3D, the motion vectors are in the same direction, but the key points away from the majority of the key points belong to different moving objects. That is, they are not considered to be the same cluster. Further, as shown in the key point 34 of FIG. 3D, the key point processing unit 103 sets the key points in the cluster area to which the majority of the key points belong or in the vicinity of the cluster area to the same moving object. It is considered to belong. It is probable that such a key point was the one in which the detection of the matching key point was wrong in the previous stage.

As shown in FIG. 3D, the initial region estimation unit 104 covers an image region (an image region) that covers a target moving object corresponding to the cluster with respect to the key point group formed into the same cluster by the key point processing unit 103. Initial region IR: first region) is estimated. Here, the initial region IR may be the smallest rectangular region including the key points of the same cluster, or may be a rectangular region slightly larger than this so that the moving object is covered by this region.

The candidate region determination unit 105 determines a plurality of candidate region TRs (second regions) in which a moving object may exist based on the initial region IR.

FIG. 4 is a diagram illustrating a candidate region according to the first embodiment. It should be noted that any three sides of the initial region IR and the candidate regions (TR1 to TR8) overlap at exactly the same positions, but in FIG. 4, for convenience, in order to make it easier to grasp each region. It is shown in a slightly shifted state.

In this embodiment, as shown in FIG. 4, in the candidate region determination unit 105, only the right side RS of the initial region IR has a width (horizontal width) perpendicular to the right side RS of a predetermined ratio (for example, 10%). Candidate area TR1 moved to expand (move to the right) and candidate to move (move to the left) so that only the right side RS of the initial area IR is reduced by a predetermined ratio (for example, 10%). Only the area TR2 and the left side LS of the initial area IR are moved (moved to the left) so that the width expands by a predetermined ratio (for example, 10%), and only the width of the left side LS of the initial area IR. The width (vertical width) of only the candidate region TR4 moved (moved to the right) so as to be reduced by a predetermined ratio (for example, 10%) and the upper side US of the initial region IR (for example, the width in the vertical direction) is a predetermined ratio (for example). Only the candidate area TR5 moved (moved upward) so as to expand by 10%) and the upper side US of the initial area IR are moved (moved downward) so that the width is reduced by a predetermined ratio (for example, 10%). ) And the candidate area TR7 in which only the lower side DS of the initial area IR is moved (moved downward) so that the width (width in the vertical direction) is expanded by a predetermined ratio (for example, 10%). , The candidate region TR8 in which only the lower side DS of the initial region IR is moved (moved upward) so that the width is reduced by a predetermined ratio (for example, 10%) is determined as the candidate region. By determining the candidate region in this way, it becomes possible to appropriately detect the candidate of the region that matches the size of the moving object of interest. In particular, by preparing a candidate region that has been moved for each side of the initial region IR, it is possible to appropriately obtain a region in which the moving object exists regardless of which direction the moving object is displaced.

The object classification unit 106 evaluates whether or not the initial region IR and each of the nine regions of the candidate regions TR1 to TR8 are target moving objects. The moving object of the object may be the whole object or a part of the object, for example, a human being or a face which is a human part. In the present embodiment, since it is evaluated whether or not the target moving object exists only in these a small number of regions smaller than the frame image, the processing amount can be reduced and the processing time can be reduced. Can be done.

The object classification unit 106 determines the certainty indicating the certainty that the region is a predetermined moving object as an evaluation as to whether or not the object is a moving object. The object classification unit 106 may have a configuration in which, for example, a CNN (Convolutional Neural Network) outputs an accuracy indicating the certainty that the area is a moving object of an object by using an image of the area. The CNN needs to be trained in advance so that the accuracy of the target moving object can be appropriately determined. By using CNN as the configuration of the object classification unit 106, it is possible to realize extremely high accuracy and real-time performance (execution of processing in a short time) required for the monitoring system.

The accuracy determined by the object classification unit 106 can be used to determine a region that is more accurately adapted to the size of the moving object of interest. That is, it can be said that the region with higher determined accuracy is the region that is more accurately adapted to the size of the moving object of interest.

The existence area determination unit 107 identifies the area determined with the highest accuracy based on the accuracy for each area determined by the object classification unit 106, and uses this area as the area where the moving object exists (existence area). Is determined, and the position of the existing area in the frame image is notified to the existing area display unit 108. The image of the determined existing region may be used for further processing (for example, more detailed recognition processing for a moving object). Since this existing region is more accurately adapted to the size of the moving object, it is possible to improve the processing accuracy in the further processing of the moving object.

The existence area display unit 108 causes the display device 15 to display a frame image based on the position of the existence area notified from the existence area determination unit 107, and displays a range corresponding to the notified existence area in the frame image. Display in an identifiable manner. As a result, the existing area of the moving object in the frame image of the display device 15 can be appropriately displayed, and the user can appropriately grasp the moving object.

Next, the processing operation of the moving object detection device according to the first embodiment will be described.

FIG. 5 is a flowchart of the moving object detection process according to the first embodiment.

First, the image input unit 101 inputs each frame image of the moving image captured by the surveillance camera 20 (step S10).

Next, the background subtraction processing unit 102 executes a process of acquiring background subtraction in the input frame image (step S11).

Next, the key point processing unit 103 extracts each key point of the two different frame images (step S12), matches the key points between the respective frame images (step S13), and matches between the frame images. The movement vector of the key points is specified, and the key points indicating the same moving object are clustered based on the movement vector (step S14).

Next, the initial region estimation unit 104 estimates the initial region IR of the image covering the moving object corresponding to the cluster for the key point group formed into the same cluster by the key point processing unit 103 (step S15).

Next, the candidate region determination unit 105 determines a plurality of candidate region TRs in which a moving object may exist based on the initial region IR (step S16).

Next, the object classification unit 106 evaluates whether or not a target moving object exists in each of the initial region IR and the candidate region TR (step S17).

Next, the existence area determination unit 107 identifies the area determined with the highest accuracy based on the accuracy for each area determined by the object classification unit 106, and sets the area as the existence area where the moving object exists. The determination is made, and the position of the existing area in the frame image is notified to the existing area display unit 108 (step S18).

Next, the existence area display unit 108 causes the display device 15 to display the frame image based on the position of the existence area notified from the existence area determination unit 107, and sets the range corresponding to the notified existence area in the frame image. It is displayed so that it can be identified (step S19). After the completion of step S19, the moving object detection device 10 performs the same processing as the processing from step S10 with a new frame image of the two frame images used for the processing and a subsequent frame image. Is executed for.

According to the above embodiment, the region most suitable for the moving object can be easily and appropriately detected from the initial region IR and other candidate regions.

<Second Example>
Next, the moving object detection device according to the second embodiment will be described. The second embodiment will be described with reference to each configuration shown in FIG.

In the second embodiment, the existing region of the moving object in the next frame image is detected by using the range of the existing region with respect to the initial region IR of the moving object at the time of processing the immediately preceding frame image. It is designed to reduce the load of. This focuses on the fact that the range of the existing region with respect to the initial region IR is relatively high in the possibility that the same correspondence is maintained even if the frame images are different if the moving objects are the same.

The existence area determination unit 107 (an example of the range identification information registration unit) in the second embodiment stores information (range identification information) capable of specifying the range of the existence area with respect to the initial area in the memory 12 (an example of the storage unit). do.

FIG. 6 is a diagram showing an example of range specifying information according to the second embodiment.

As shown in FIG. 6, the range identification information includes, for example, a vector V from the reference position of the initial region IR (for example, the upper left point) to the reference position of the existing region OR (for example, the upper left point) and the existing region OR. The information may include the vector W from the reference point of the above to the opposite point (lower right point) of the existence region OR. According to this range specifying information, the range of the existing region OR can be appropriately expressed with reference to the reference position of the initial region IR.

Further, the existence area determination unit 107 (an example of the third area identification unit) determines whether or not the memory 12 has the range identification information in the immediately preceding process, and if there is the range identification information, the current initial area. The area corresponding to the range specifying information is determined as an existing area candidate (an example of the third area) based on the above, and the object classification unit 106 is notified. Further, the existence area determination unit 107 determines whether or not the accuracy notified from the object classification unit 106 is equal to or higher than a predetermined threshold value, and if the accuracy is equal to or higher than the predetermined threshold value, the existence area candidate is a candidate for existence. However, when the accuracy is smaller than a predetermined threshold value, the existence area candidate is not suitable for the existence candidate, so that the existence area candidate is determined as the existence area. The same process as that of the above, that is, the initial range IR and the process for determining the existing area from the plurality of candidate areas are performed.

The object classification unit 106 identifies the accuracy of the target moving object for the existence area candidate notified from the existence area determination unit 107, and notifies the existence area determination unit 107 of the specified accuracy.

Next, the processing operation of the moving object detection device according to the second embodiment will be described.

FIG. 7 is a flowchart of the moving object detection process according to the second embodiment. The steps similar to the moving object detection process shown in FIG. 5 are designated by the same reference numerals.

After the processing of steps S10 to S15 is completed, the existence area determination unit 107 determines whether or not the memory 12 has the range specifying information in the immediately preceding processing (step S21). As a result, when there is no range specifying information in the immediately preceding process (step S21: NO), the existence area determination unit 107 causes the processes of steps S16 to S18 to be executed.

On the other hand, when there is range specifying information in the immediately preceding process (step S21: YES), an area corresponding to the range specifying information is set as an existing area based on the current initial area (initial area estimated in the immediately preceding step S15). The candidate is determined, and the existing area candidate is notified to the object classification unit 106 (step S22).

Next, the object classification unit 106 identifies the certainty of the target moving object for the notified existence area candidate, and notifies the existence area determination unit 107 (step S23).

Next, the existence region determination unit 107 determines whether or not the accuracy notified from the object classification unit 106 is equal to or higher than a predetermined threshold value (step S24). As a result, when the accuracy is smaller than the predetermined threshold value (step S24: NO), the existence area determination unit 107 causes the processing of steps S16 to S18 to be executed because the existence area candidate is not suitable for the existence area. To.

On the other hand, when the accuracy is equal to or higher than a predetermined threshold value (step S24: YES), it means that the existing region candidate is suitable as the existing region, so that the existing region determination unit 107 exists as the existing region candidate. The region is determined, and the process proceeds to step S26 (step S25).

Next, after determining the existence area (after executing step S18 or step S25), the existence area determination unit 107 stores the range identification information indicating the range of the existence area with respect to the initial area in the memory 12 (after executing the step S18 or step S25). Step S26), the process proceeds to step S19.

According to the second embodiment, the existing area candidate is determined by using the range identification information in the immediately preceding process, and the accuracy indicating the certainty that the image of the existing area candidate is the target moving object is equal to or higher than a predetermined threshold value. In some cases, since the existing area candidate is determined as the existing area, it is not necessary to determine a plurality of candidate areas and evaluate each candidate area, which can reduce the processing load and shorten the processing time. be able to.

<Third Example>
Next, the moving object detection system according to the third embodiment will be described.

The moving object detection system according to the third embodiment is a system in which the processing by the moving object detection device 10 is realized by the client device 200 and the server device 300 connected via a network.

The moving object detection system includes a client device 200 and a server device 300. The client device 200 and the server device 300 are connected to each other via the network 400. The network 400 may be the Internet or an intranet. The server device 300 does not have to be located near the client device 200, and can be located, for example, in a remote location (for example, a foreign country).

The client device 200 is a bus 208 that communicably connects the CPU 201, the memory 202, the auxiliary storage device 203, the external device I / F 204, the communication I / F 205, the display device 206, the input device 207, and each unit. And. The configuration having the same name as the moving object detection device 10 shown in FIG. 1 has the same configuration. The communication I / F 205 is an interface for communicating with the server device 300 via the network 400.

The client device 200 includes the image input unit 101 shown in FIG. 2, and further has a transmission unit that transmits the frame image input by the image input unit 101 to the server device 300 via the network 400.

The server device 300 includes a CPU 301, a memory 302, an auxiliary storage device 303, a communication I / F 304, a display device 305, an input device 306, and a bus 307 that communicably connects each unit. The configuration having the same name as the moving object detection device 10 shown in FIG. 1 has the same configuration. The communication I / F 304 is an interface for communicating with the client device 200 via the network 400.

The server device 300 includes a background subtraction processing unit 102, a key point processing unit 103, an initial area estimation unit 104, a candidate area determination unit 105, an object classification unit 106, and an existence area determination unit 107, as shown in FIG. To be equipped. In this embodiment, the background subtraction processing unit 102 further has a function of receiving a frame image transmitted from the client device 200 via the network 400.

The existence area display unit 108 may be provided on either the client device 200 or the server device 300, and when the existing area display unit 108 is provided on the client device 200 side, the server device 300 is provided via the network 400. The existence area display unit 108 of the client device 200 may have a transmission unit for notifying the position of the existence area, and the existence area may be identifiablely displayed in the frame image based on the notification.

According to the moving object detection system according to the third embodiment, the processing load on the client device 200 can be reduced.

<Fourth Example>
Next, the moving object detection system according to the fourth embodiment will be described.

The hardware configuration of the moving object detection system according to the fourth embodiment is the same as that of the moving object detection system shown in FIG.

The moving object detection system according to the fourth embodiment reduces the size of the transmitted image data by transmitting a part of the frame image from the client device 200 to the server device 300 instead of the entire frame image. It is the one that was made.

The client device 200 includes an image input unit 101 shown in FIG. 2, a background subtraction processing unit 102, a key point processing unit 103, and an initial area estimation unit 104. The initial region estimation unit 104 further has a function of transmitting a partial region of the frame image including the estimated initial region. The initial area estimation unit 104 transmits to the server device 300 a partial area (partial image) of the frame image including the initial area and the area used as the existence area by the existence area determination unit 107 of the server device 300.

The server device 300 includes a candidate area determination unit 105 shown in FIG. 2, an object classification unit 106, and an existence area determination unit 107. The candidate area determination unit 105 determines a candidate area using a partial image transmitted from the client device 200 via the network 400. The existence area display unit 108 may be provided in either the client device 200 or the server device 300.

Next, the processing operation of the moving object detection system according to the fourth embodiment will be described.

FIG. 9 is a sequence diagram of a part of the moving object detection process according to the fourth embodiment.

First, the surveillance camera 20 acquires the frame image f1 (step S100) and transmits the acquired frame image f1 to the client device 200 (step S101). Next, the surveillance camera 20 acquires the next frame image f2 (step S102) and transmits the acquired frame image f2 to the client device 200 (step S103). After that, the surveillance camera 20 sequentially acquires the frame image and transmits the acquired frame image to the client device 200.

On the other hand, when the key point processing unit 103 of the client device 200 acquires the frame image f1 transmitted from the surveillance camera 20, the key point kp1 is specified from the frame image f1 (step S201). Next, when the key point processing unit 103 of the client device 200 acquires the frame image f2 transmitted from the surveillance camera 20, the key point kp2 is specified from the frame image f2 (step S202).

Next, the key point processing unit 103 performs matching between the key point kp1 of the frame image f1 and the key point kp2 of the frame image f2 (step S203), and identifies the motion vector between the matched key points (step). S204), clustering key points based on the motion vector (step S205).

Next, the initial region determination unit 104 estimates the initial region IR of the image covering the moving object corresponding to the cluster for the key point group formed into the same cluster by the key point processing unit 103 (step S206). A partial image including the estimated initial area IR and an area that may be a candidate area in the server device 300 is transmitted to the server device 300 (step S207).

On the other hand, the server device 300 executes the same process as the process after step S16 in FIG. 5 using the partial image transmitted from the client device 200.

In the moving object detection system according to the fourth embodiment, the client device 200 transmits a part of the frame image to the server device 300 instead of the entire frame image. Therefore, for example, the client device 200 and the server device Even when the network with the 300 has a narrow bandwidth such as long-distance wireless communication, there is no problem and the moving object detection process can be performed.

<Fifth Example>
Next, the moving object detection device according to the fifth embodiment will be described.

The moving object detection device 10 according to the fifth embodiment is a modification of the processing by the candidate area determination unit 105. The candidate area determination unit 105 receives the magnitude of the motion vector of the key point from the key point processing unit 103. , The ratio of the fluctuating width to the width of the initial region IR when determining the candidate region as shown in FIG. 4 is changed based on the magnitude of the motion vector. For example, the larger the magnitude of the motion vector, the larger the ratio of the fluctuating width may be. The magnitude of the motion vector used as the reference for the fluctuating width may be the average of the magnitudes of the motion vectors of all the key points in the initial region, and is the largest motion vector among the motion vectors of the key points in the initial region. It may be the size of.

According to the fifth embodiment, the size of the candidate range can be appropriately adjusted according to the movement of the moving object, and the possibility that the moving object exists in the candidate range can be increased.

The present invention is not limited to the above-described embodiment, and can be appropriately modified and implemented without departing from the spirit of the present invention.

For example, in the above embodiment, the moving images taken by the surveillance camera are sequentially input and processed, but the present invention is not limited to this, and for example, the moving images taken by the camera in advance are stored in the auxiliary storage device 13. The moving image may be processed, or the moving image transmitted from an external device may be processed.

The key point detection method and extraction method in the above embodiment are not limited to the above methods, but are not limited to the above methods, such as SIFT (Scale-Invariant Feature Transition), SURF (Speed Up Robot Features), HOG (Histogram of Orientation). Binary Robust Invariant Scalable Keypoints) may be used.

Further, in the above embodiment, the accuracy that the image of the initial region or the candidate region is the target moving object is determined by using CNN, but the present invention is not limited to this, and a method other than CNN, for example, , Image matching may be performed to determine the certainty that the image is a moving object of interest.

Further, in the above embodiment, in the moving object detection process, the process is performed on two consecutive frame images, but the present invention is not limited to this, and for example, two frames separated by a predetermined frame. The processing may be performed on the frame image. By doing so, it is possible to identify the existing region of the target moving object with relatively high accuracy while reducing the processing amount.

Further, in the above embodiment, the region having the highest accuracy is determined as the existing region from the initial region and the candidate region for the initial region, but the present invention is not limited to this, and for example, the region having the highest accuracy is determined. When the accuracy of the area is less than or equal to the predetermined value, the same processing is performed with this area as a new initial area, and the same processing is repeatedly executed until the obtained accuracy becomes more than or equal to the predetermined value. You may. In this way, a region with higher accuracy can be set as the existing region.

Further, in the above embodiment, an example is shown in which each functional unit of the moving object detection device and the moving object detection system is configured by executing a program by a processor, but the present invention is not limited to this, and each function is not limited to this. A part or all of the parts may be configured by hardware such as an integrated circuit. Further, in the above embodiment, the program constituting the functional unit may be provided by a recording medium on which the program code is recorded. In this case, the functional unit can be realized by reading and executing the program of the recording medium by the processor of the computer. Examples of the storage medium for supplying the program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, and non-volatile materials. A memory card, ROM, or the like may be used. Further, by distributing the programs constituting the functional units via the network, they are stored in a storage unit such as a hard disk or memory of a computer or a storage medium such as a CD-RW or CD-R, and stored in a processor provided in the computer. The program code stored in the unit or the storage medium may be read and executed.

10 ... Mobile detection device, 11 ... CPU, 12 ... Memory, 13 ... Auxiliary storage device, 14 ... External device I / F, 15 ... Display device, 16 ... Input device, 20 ... Surveillance camera, 101 ... Image input unit, 102 ... Background subtraction processing unit, 103 ... Key point processing unit, 104 ... Initial area estimation unit, 105 ... Candidate area determination unit, 106 ... Object classification unit, 107 ... Existence area determination unit, 108 ... Existence area display unit, 200 ... Client device, 300 ... Server device, 400 ... Network

Claims (15)

In a moving object detection device that detects a moving object to be detected from a moving image
A moving image input unit for inputting moving images and
The corresponding key points between the two frame images in the moving image are specified, the motion vector of the key points is specified, and a set of key points that are assumed to constitute the same moving object is generated based on the motion vector. Key point processing unit and
A first region estimation unit that estimates a first region containing key points of the same set in one frame image of the two frame images, and a first region estimation unit.
With respect to the first region, a second region determination unit that determines one or more second regions that have undergone a predetermined deformation, and a second region determination unit.
An object classification unit that evaluates the accuracy of the first region and the second region to indicate the certainty that the image of the region is a moving object to be detected.
An existence region determination unit that determines a region having a high accuracy of being a moving object to be detected from the first region and the second region as an existence region in which the moving object exists.
A moving object detector comprising. The moving object detection device according to claim 1, wherein the two frame images that specify the key points are frame images that are continuous in time. The first region is a rectangular region and is a rectangular region.
It said second region determination unit moving object detection apparatus according to claim 1 or claim 2 for determining the area obtained by the deformation moving either one side of the first region as the second region. The second region includes four enlarged regions obtained by moving one side of the first region to expand each side of the first region, and each side of the first region. The moving object detection device according to claim 3, further comprising four reduced regions in which one side of the first region is moved to be smaller than the first region. The third or fourth aspect, wherein the second region is a region which is moved in a direction perpendicular to any one side of the first region by a predetermined ratio with respect to the width of the first region in that direction. Moving object detector. The moving object detection device according to claim 5, wherein the second region determination unit determines the predetermined ratio based on the length of the motion vector of the key point included in the first region. The object classification unit is configured to output the probability that the area is a moving object to be detected by using the data of the area by the convolutional neural network, according to any one of claims 1 to 6. The moving object detector according to the description. A storage unit that stores the background image of the moving image,
It further has a background subtraction processing unit that removes the components of the background image from each of the frame images.
The moving object detection device according to any one of claims 1 to 7, wherein the key point processing unit executes processing using a frame image from which the components of the background image have been removed. A range-specific information registration unit for registering range-specific information that can specify the range of the existing area with respect to the first area,
Based on the position of the first region with respect to the new frame image estimated by the first region estimation unit, a third region identification unit for specifying the third region specified by the range identification information is further provided.
The object classification unit evaluates whether or not the third region is the predetermined moving object, and evaluates whether or not the third region is a predetermined moving object.
The existing area determination unit determines the third area as the existing area where the moving object exists when the evaluation result for the third area is equal to or higher than a predetermined evaluation, according to claims 1 to 8. The moving object detection device according to any one of the following items. The range specifying information is diagonal to the vector from the reference position of the first region to the reference position of the third region and the reference position of the third region to the reference position of the third region. The moving object detection device according to claim 9, which includes a vector to a position. When the evaluation result for the third region is less than a predetermined evaluation, the second region determination unit determines the second region, and the object classification unit determines the first region and the second region. On the other hand, it is evaluated whether or not it is the predetermined moving object, and the existing region determining unit determines the region having a high evaluation of being the moving object from the first region and the second region. The moving object detection device according to claim 9 or 10, wherein the moving object is determined as an existing area in which the moving object exists. A moving object detection system including a client device and a server device connected via a network.
The client device
A moving image input unit for inputting moving images and
The corresponding key points between the two frame images in the moving image are specified, the motion vector of the key points is specified, and a set of key points that are assumed to constitute the same moving object is generated based on the motion vector. Key point processing unit and
A first region determination unit that estimates a first region containing key points of the same set in a frame image,
A transmission unit that transmits an image including a first region of a frame image of the moving image to the server device is provided.
The server device
A second region determination unit that determines one or more second regions that have undergone a predetermined modification with respect to the first region based on an image transmitted from the client device.
An object classification unit that evaluates the certainty that the image of the first region and the second region indicates the certainty that the image of the region is the predetermined moving object.
An existence region determination unit that determines a region having a high accuracy of being a moving object from the first region and the second region as a region in which the moving object exists.
A moving object detection system. The moving object detection system according to claim 12, wherein the transmission unit transmits an image having a size smaller than the frame image and having a size including all the second regions. The server device
It further has a transmitter that transmits information indicating the position of the region where the moving object exists to the client device.
The client device
The moving object detection system according to claim 12 or 13, further comprising an existing area display unit for displaying the moving image and identifiablely displaying an area corresponding to the existing area in the moving image. A moving object detection method using a moving object detection device that detects a predetermined moving object from a moving image.
The moving object detection device is
Enter the video and
The corresponding key point between the two frame images in the moving image is specified, the motion vector of the key point is specified, and the motion vector of the key point is specified.
Based on the motion vector, a set of key points that are supposed to constitute the same moving object is generated.
Estimate the first region containing the key points of the same set in the frame image and
With respect to the first region, one or more second regions that have undergone a predetermined deformation are determined.
For the first region and the second region, the accuracy of indicating the certainty that the image of the region is the predetermined moving object is evaluated.
A moving object detection method for determining a region having a high accuracy of being a moving object from the first region and the second region as a region in which the moving object exists.

Images