JP6653719B2 - State determination device and state determination program - Google Patents

Description

  The present invention relates to a technique for determining a state of an object from image data obtained by a photographing device.

  Research has been conducted on a technology for determining whether an object is present or not based on image data obtained by an imaging device such as a monitoring camera. Patent Literature 1 describes a technique for comparing image data acquired at time intervals to specify that a train has entered or the like.

JP-A-2004-066904

It may be difficult to accurately determine the state of a target object only by comparing image data acquired at time intervals.
An object of the present invention is to determine the state of a target with high accuracy from image data obtained by a photographing device.

The state determination device according to the present invention includes:
Between target image data obtained by shooting a shooting region with a shooting device and background image data obtained by shooting the shooting region with the shooting device in a state where there is no object in the shooting region. A background determination unit that determines whether the background difference is greater than a background threshold,
It is determined whether or not a difference between frames between the target image data and a previous frame image data obtained by photographing a photographing region by the photographing device before the target image data is larger than a frame threshold. A frame determination unit;
Based on the result determined by the background determination unit and the result determined by the frame determination unit, the state in which the object is moving in the imaging region, and the object stops in the imaging region And a state determining unit that determines which state of the object is in the state where the object is not present in the shooting area.

  When the background determination unit determines that the background difference is not larger than the background threshold, the state determination unit determines that the object is not present in the imaging region.

  The state determination unit, when the background difference is determined by the background determination unit is greater than the background threshold, and, when the inter-frame difference is determined by the frame determination unit is greater than the frame threshold, the It is determined that the object is moving in the shooting area.

  The state determination unit, when the background difference is determined by the background determination unit is greater than the background threshold, and, when the frame determination unit determines that the inter-frame difference is not greater than the frame threshold, It is determined that the object is stopped in the shooting area.

  The frame threshold is a reference number of frames before the target image data, the number of pixels having a difference between the image data of the target frame and the image data of the frame before the target frame, It was determined based on the average value of the difference rate divided by the number of pixels of the image data of the target frame.

  The frame threshold is calculated by multiplying the difference ratio between the target image data and the image data of a frame preceding the target image data by a standard deviation of the past average value.

A state determination program according to the present invention includes:
Between target image data obtained by shooting a shooting region with a shooting device and background image data obtained by shooting the shooting region with the shooting device in a state where there is no object in the shooting region. Background determination processing for determining whether the background difference is greater than a background threshold,
It is determined whether or not a difference between frames between the target image data and a previous frame image data obtained by photographing a photographing region by the photographing device before the target image data is larger than a frame threshold. Frame determination processing;
Based on the result determined by the background determination process and the result determined by the frame determination process, a state in which the object is moving in the imaging region, and the object stops in the imaging region The computer causes the computer to execute a state determination process of determining which state is present or a state in which the object is not present in the imaging region.

  According to the present invention, the state of the target object is determined based on whether there is a background difference and whether there is an inter-frame difference. Thereby, it is possible to determine the state of the target object with high accuracy.

FIG. 1 is a configuration diagram of a state determination device 10 according to the first embodiment. 5 is a flowchart of an overall operation of the state determination device 10 according to the first embodiment. FIG. 3 is a diagram showing an installation state of the imaging device 41 according to the first embodiment. 5 is a flowchart of a background determination process according to the first embodiment. FIG. 4 is an explanatory diagram of a method for comparing target image data and background image data according to the first embodiment. 5 is a flowchart of a frame determination process according to the first embodiment. FIG. 5 is an explanatory diagram of changes between a background difference and an inter-frame difference according to the first embodiment. FIG. 4 is an explanatory diagram of a frame threshold calculation process according to the first embodiment. FIG. 4 is an explanatory diagram of a change in a frame threshold Th according to the first embodiment. The block diagram of the state determination apparatus 10 which concerns on the modification 2.

Embodiment 1 FIG.
*** Configuration description ***
With reference to FIG. 1, the configuration of the state determination device 10 according to the first embodiment will be described.
The state determination device 10 is a computer.
The state determination device 10 determines the state of an object. Specifically, the state determination device 10 determines which of a state in which the object is moving, a state in which the object is stopped, and a state in which the object is not present. In the first embodiment, the description will be given on the assumption that the target object is a train. However, the target object is not limited to a train, but may be another vehicle such as a four-wheeled vehicle, a two-wheeled vehicle, a ship, or another moving object such as a person.

  The state determination device 10 includes hardware of a processor 11, a memory 12, a storage 13, and a communication interface 14. The processor 11 is connected to other hardware via a signal line, and controls the other hardware.

  The processor 11 is an IC (Integrated Circuit) that performs processing. The processor 11 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit) as a specific example.

  The memory 12 is a storage device that temporarily stores data. The memory 12 is, specifically, an SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory).

  The storage 13 is a storage device that stores data. The storage 13 is, as a specific example, an HDD (Hard Disk Drive). The storage 13 includes an SD (Secure Digital) memory card, a CF (CompactFlash, registered trademark), a NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, a DVD (Digital Versatile Disk), and the like. It may be a portable recording medium.

  The communication interface 14 is an interface for communicating with an external device. The communication interface 14 is a port of Ethernet (registered trademark), USB (Universal Serial Bus), or HDMI (registered trademark, High-Definition Multimedia Interface) as a specific example.

  The state determination device 10 is connected via a communication interface 14 to an imaging device 41 such as a monitoring camera.

The state determination device 10 includes, as functional components, an image data acquisition unit 21, a background determination unit 22, a frame determination unit 23, and a state determination unit 24. The function of each functional component of the state determination device 10 is realized by software.
The storage 13 stores a program that implements the function of each functional component of the state determination device 10. This program is read into the memory 12 by the processor 11 and executed by the processor 11. Thereby, the function of each functional component of the state determination device 10 is realized.

  The storage 13 implements the function of the image storage unit 31. The image storage unit 31 stores background image data. The background image data is image data obtained by photographing the photographing area with the photographing device 41 in a state where there is no object in the photographing region of the photographing device 41.

  In FIG. 1, only one processor 11 is shown. However, a plurality of processors 11 may be provided, and a plurality of processors 11 may execute programs for realizing the respective functions in cooperation with each other.

*** Explanation of operation ***
The operation of the state determination device 10 according to the first embodiment will be described with reference to FIGS.
The operation of the state determination device 10 according to the first embodiment corresponds to the state determination method according to the first embodiment. The operation of the state determination device 10 according to the first embodiment corresponds to the processing of the state determination program according to the first embodiment.

With reference to FIG. 2, the overall operation of state determination device 10 according to Embodiment 1 will be described.
The process shown in FIG. 2 is repeatedly executed at regular intervals.

(Step S1: image data acquisition processing in FIG. 2)
The image data acquisition unit 21 acquires image data photographed by the photographing device 41 as target image data.
Specifically, the image data acquisition unit 21 receives, via the communication interface 14, target image data obtained by imaging the imaging range with the imaging device 41. The image data acquisition unit 21 writes the target image data into the memory 12.
The photographing device 41 is installed so as to photograph a fixed photographing area. In the first embodiment, as shown in FIG. 3, the photographing device 41 is installed on the platform of the station, and is installed such that when the train is stopped at the station, the range where the train is reflected falls within the photographing range. Have been. The image data acquisition object 21 cuts out a range in which the background and the object are photographed in the photographing area as target image data. In FIG. 3, since there is no train, the area where the wall surface on the opposite side of the track from the platform is photographed is the target image data. It is desirable that the target image data has a range in which another target object (for example, a person) does not move. In addition, if the shooting region is a region where the background and the target object are shot and the other target object does not move, the shooting region may be treated as the target image data.

(Step S2 in FIG. 2: background determination processing)
The background determination unit 22 includes target image data obtained by shooting the shooting region with the shooting device 41 and a background image obtained by shooting the shooting region with the shooting device 41 in a state where there is no target in the shooting region. It is determined whether there is a background difference larger than the background threshold value between the data and the data.

The background determination process according to the first embodiment will be described with reference to FIG.
(Step S21 in FIG. 4: image acquisition processing)
The background determination unit 22 reads out the target image data acquired in step S1 from the memory 12. Further, the background determination unit 22 reads the background image data from the image storage unit 31.

(Step S22 of FIG. 4: image comparison processing)
The background determining unit 22 compares the target image data and the background image data read in step S21 for each pixel, and specifies a pixel having a different pixel value.
Here, the target image data and the background image data are image data obtained by photographing the same photographing region. Therefore, the background determination unit 22 compares pixels at the same position in the target image data and the background image data to determine whether the pixel values are different. That is, as shown in FIG. 5, the background determination unit 22 compares the upper left pixel with the target image data and the background image data, compares the pixels on the right with the upper left pixel, and so on. As described above, the pixels are compared in order from the upper left to the lower right pixel to determine whether or not the pixel values of the respective pixels are different.

数１において、ｐは、背景差分である。ｆｒａｍｅＤｉｆｆｐｉｘｅｌは、ステップＳ２２で画素値が異なると判定された画素の数である。ｒｅｃｔＷは、対象画像データの横方向の画素数である。ｒｅｃｔＨは、対象画像データの縦方向の画素数である。ここでは、対象画像データは、矩形である。 (Step S23 in FIG. 4: background difference calculation processing)
The background determination unit 22 calculates, as a background difference, a value obtained by dividing the number of pixels determined to have different pixel values in step S22 by the number of pixels of the target image data.
For example, the background difference is calculated by Equation 1.

In Equation 1, p is a background difference. frameDiffpixel is the number of pixels determined to have different pixel values in step S22. RectW is the number of pixels in the horizontal direction of the target image data. RectH is the number of pixels in the vertical direction of the target image data. Here, the target image data is rectangular.

(Step S24 in FIG. 4: difference determination processing)
The background determination unit 22 determines whether the background difference calculated in step S23 is larger than a background threshold. The background threshold is sufficiently smaller than the background difference for the target image data when the target train is present, and is sufficiently smaller than the background difference for the target image data when the target train is not present. A large value is set.
If the background determination unit 22 determines that the background difference is larger than the background threshold, the process proceeds to step S3 in FIG. On the other hand, when the background determination unit 22 determines that the background difference is not larger than the background threshold, the process proceeds to step S6 in FIG.

(Step S3 in FIG. 2: frame determination processing)
The frame determination unit 23 determines whether there is an inter-frame difference larger than the frame threshold between the target image data and the previous frame image data obtained by shooting the shooting region by the shooting device before the target image data. Determine whether or not.

With reference to FIG. 6, a frame determination process according to the first embodiment will be described.
(Step S31 in FIG. 6: Image acquisition processing)
The frame determination unit 23 reads out the target image data acquired in step S1 from the memory 12. Further, the frame determination unit 23 reads out the target image data acquired in the previous step S1 from the memory 12 as the previous frame image data. That is, since the process illustrated in FIG. 2 is repeatedly performed, the frame determination unit 23 sets the target image data acquired in the immediately preceding step S1 as the previous frame image data.

(Step S32 in FIG. 6: Image comparison processing)
The frame determining unit 23 compares the target image data read in step S31 with the previous frame image data for each pixel, and specifies a pixel having a different pixel value.
Here, the target image data and the previous frame image data are image data obtained by shooting the same shooting area. Therefore, the frame determination unit 23 compares pixels at the same position in the target image data and the previous frame image data to determine whether or not the pixel values are different. That is, as shown in FIG. 5, it is determined whether or not the pixel values of the respective pixels are different by sequentially comparing the pixels from the upper left to the lower right pixel.

(Step S33 in FIG. 6: Inter-frame difference calculation processing)
The frame determination unit 23 calculates, as an inter-frame difference, a value obtained by dividing the number of pixels determined to have different pixel values in step S32 by the number of pixels of the target image data. The method of calculating the difference between frames is the same as the method of calculating the background difference. The frame determination unit 23 writes the difference between frames into the memory 12.

(Step S34 in FIG. 6: difference determination processing)
The frame determination unit 23 determines whether the inter-frame difference calculated in step S33 is larger than a frame threshold. The frame threshold is calculated dynamically. The method of calculating the frame threshold will be described later.
When it is determined that the inter-frame difference is larger than the frame threshold, the frame determination unit 23 proceeds to step S4 in FIG. On the other hand, when the background determination unit 22 determines that the inter-frame difference is not larger than the frame threshold, the process proceeds to step S5.

(Step S4 in FIG. 2: movement determination processing)
The state determination unit 24 determines that the target train is moving.
That is, when the background difference is determined to be greater than the background threshold and the inter-frame difference is determined to be greater than the frame threshold, the state determination unit 24 determines that the train in which the target area is the shooting area of the shooting device 41 It is determined that it is moving.

(Step S5 in FIG. 2: stop determination processing)
The state determination unit 24 determines that the target train is in a stopped state.
That is, the state determination unit 24 determines that the train that is the object in the shooting area stops when the background difference is determined to be greater than the background threshold and the inter-frame difference is not determined to be greater than the frame threshold. Is determined to be in a state.

(Step S6 of FIG. 2: absence determination processing)
The state determination unit 24 determines that there is no state of the target train.
That is, when it is determined that the background difference is not larger than the background threshold, the state determination unit 24 determines that there is no train as a target in the shooting region.

  Note that the processing from step S4 to step S6 in FIG. 2 is referred to as state determination processing.

As shown in FIG. 7, the background difference and the inter-frame difference are obtained while the train entering the station, stopping, and leaving the station starts when the train as the target object does not exist at the station as the shooting area. Changes.
When there is no train, the background difference and the inter-frame difference are almost zero. When the train enters the station, the value of the background difference and the value of the inter-frame difference gradually increase. When the train stops, the value of the inter-frame difference decreases while the value of the background difference remains large. Then, as the train leaves the station, the value of the background difference further increases, and the value of the inter-frame difference increases.

With reference to FIG. 8, a description will be given of a frame threshold calculation process according to the first embodiment.
The process illustrated in FIG. 8 is performed between the process in step S33 and the process in step S34 in FIG. Here, it is assumed that the processing in FIG. 2 is repeatedly executed by the amount necessary for the following calculation.

(Step S41 of FIG. 8: Inter-frame difference acquisition processing)
The frame determination unit 23 reads out the past inter-frame difference p _i (i = t− (2N−1),..., T) from the memory 12. Here, t indicates the present time. That is, the frame determination unit 23 reads out the inter-frame difference p _i for the past 2N times including the present time from the memory 12. N is an integer of 2 or more.

(Step S42 in FIG. 8: Average value calculation processing)
As shown in Expression 2, the frame determination unit 23 determines that j = t− (N−1),. . . , T, the average value a _j of the past N inter-frame differences p _i (i = j− (N−1),..., J) read in step S41 is calculated. N in step S42 is a reference number.

数３において、ａ_ａｖｅは、ｊ＝ｔ−（Ｎ−１），．．．，ｔの各整数ｊについての平均値ａ_ｊの平均値である。補正値αは、０より大きい値であり、例えば０．０１である。補正値αは、フレーム間差分がない状態が継続した場合に、フレーム閾値Ｔｈ_ｔが０になり、図６のステップＳ３４の判定が適切に行われなくなることを防止するために加算される。 (Step S43 in FIG. 8: threshold value calculation processing)
Frame determination unit 23, as shown in Equation 3, the difference _{p t} between the last frame, j = t- (N-1 ) calculated in step S42,. . . , Multiplied by the standard deviation of the mean a _j for each integer j of t, by adding the correction value alpha, calculates the frame threshold Th _t at the present time. Here, the standard deviation of the N average values a _j is used, but the number of the standard deviations of the average values a _j to be used may not be N as long as the number is one or more.

In _Equation 3, a _ave is j = t− (N−1),. . . Is the average value of the average value a _j for each integer j of t. The correction value α is a value larger than 0, for example, 0.01. The correction value alpha, when the absence of difference between frames is continued, the frame threshold Th _t becomes 0, is added to the determination in step S34 in FIG. 6 to prevent the no longer properly.

  As shown in FIG. 9, as the inter-frame difference p increases, the frame threshold Th also increases slightly. However, by considering the standard deviation of the average value a, even if the inter-frame difference p temporarily changes, the frame threshold value Th hardly changes.

In step S34 in FIG. 6, determination is made using the frame threshold Th _t at the present time calculated in step S43 in FIG. 8.

*** Effect of Embodiment 1 ***
As described above, the state determination device 10 according to the first embodiment determines the state of the target based on whether there is a background difference and whether there is an inter-frame difference. Thereby, it is possible to determine the state of the target object with high accuracy.

  Further, the state determination device 10 according to the first embodiment dynamically calculates the frame threshold. As a result, the frame threshold can always be kept at an appropriate value, and an appropriate determination can be made. Further, it is not necessary to perform the process of setting the frame threshold manually, and it is possible to reduce the number of steps involved in the operation of setting the frame threshold.

  When calculating the frame threshold, the state determination device 10 according to the first embodiment uses the standard deviation of the average value of the difference rates. This makes it possible to prevent a large change in the threshold value when the inter-frame difference temporarily changes significantly, thereby making it possible to make an appropriate determination.

*** Other configuration ***
<Modification 1>
In the first embodiment, the background threshold is sufficiently smaller than the background difference of the target image data in the case where the target train exists, and the target threshold is the target image data in the case where the target train does not exist. It is assumed that a value sufficiently larger than the background difference is set.
Image data obtained by the image capturing device 41 when there is no train as an object may change depending on factors such as time and weather. Therefore, the background threshold may be set for each factor. Specifically, for each factor, target image data in the case where there is no target train is acquired, and a value sufficiently larger than the background difference of the acquired target image data is set as the background value of the factor. Set as a threshold.

<Modification 2>
In the first embodiment, each functional component is realized by software. However, as a second modification, each functional component may be realized by hardware. Regarding this modified example 2, points different from the first embodiment will be described.

The configuration of the state determination device 10 according to the second modification will be described with reference to FIG.
When each functional component is realized by hardware, the state determination device 10 includes an electronic circuit 15 instead of the processor 11, the memory 12, and the storage 13. The electronic circuit 15 is a dedicated circuit for realizing the functions of the functional components, the memory 12, and the storage 13.

The electronic circuit 15 includes a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable Gate Array). is assumed.
Each functional component may be realized by one electronic circuit 15, or each functional component may be realized by being distributed to a plurality of electronic circuits 15.

<Modification 3>
As a third modification, some of the functional components may be implemented by hardware, and other functional components may be implemented by software.

  The processor 11, the memory 12, the storage 13, and the electronic circuit 15 are called a processing circuit. That is, the function of each functional component is realized by the processing circuit.

  DESCRIPTION OF SYMBOLS 10 State determination apparatus, 11 processor, 12 memory, 13 storage, 14 communication interface, 15 electronic circuit, 21 image data acquisition part, 22 background determination part, 23 frame determination part, 24 state determination part, 31 image storage part, 41 photography apparatus.

Claims (5)

With the running area of the train on the platform of the station as a shooting area, target image data obtained by shooting the shooting area with a shooting apparatus, and shooting the shooting area with the shooting apparatus in a state where there is no train in the shooting area. A background difference between the background image data and the background determination unit that determines whether the background difference is greater than a background threshold.
The difference between the number of pixels having a difference between the target image data and the image data of the frame preceding the target image data by the number of pixels of the target image data, and the difference ratio between the target image data and the target image data before the target image data. For the reference frame, the number of pixels having a difference between the image data of the target frame and the image data of the frame before the target frame is divided by the number of pixels of the image data of the target frame. The target image data, and an image capturing area captured by the image capturing device before the target image data, based on a frame threshold calculated by multiplying the average value of the difference rates by the standard deviation of the average value calculated in the past. A frame determination unit that determines whether the inter-frame difference between the previous frame image data obtained by performing the
Based on the result determined by the background determination unit and the result determined by the frame determination unit, a state in which the train is moving in the shooting area, and a state in which the train is stopped in the shooting area And a state determining unit that determines which state is the state in which the train does not exist in the shooting area. The state determination unit according to claim 1, wherein the state determination unit determines that the train does not exist in the shooting region when the background determination unit determines that the background difference is not greater than the background threshold. apparatus.   The state determination unit, when the background difference is determined by the background determination unit is greater than the background threshold, and, when the inter-frame difference is determined by the frame determination unit is greater than the frame threshold, the The state determination device according to claim 1, wherein the imaging region is determined to be in a state where the train is moving. The state determination unit, when the background difference is determined by the background determination unit is greater than the background threshold, and, when the frame determination unit determines that the inter-frame difference is not greater than the frame threshold, The state determination device according to any one of claims 1 to 3, wherein it is determined that the train is stopped in the shooting area. With the running area of the train on the platform of the station as a shooting area, target image data obtained by shooting the shooting area with a shooting apparatus, and shooting the shooting area with the shooting apparatus in a state where there is no train in the shooting area. Background determination processing for determining whether or not the background difference between the obtained background image data and the background image data is greater than a background threshold;
The difference between the number of pixels having a difference between the target image data and the image data of the frame preceding the target image data by the number of pixels of the target image data, and the difference ratio between the target image data and the target image data before the target image data. For the reference frame, the number of pixels having a difference between the image data of the target frame and the image data of the frame before the target frame is divided by the number of pixels of the image data of the target frame. The target image data, and an image capturing area captured by the image capturing device before the target image data, based on a frame threshold calculated by multiplying the average value of the difference rates by the standard deviation of the average value calculated in the past. A frame determination process for determining whether or not the inter-frame difference between the previous frame image data obtained by performing the
Based on the result determined by the background determination process and the result determined by the frame determination process, a state in which the train is moving in the shooting area, and a state in which the train is stopped in the shooting area And a state determination process for causing a computer to execute a state determination process of determining which state is the state where the train does not exist in the shooting area.

Images