JP7042736B2 - Foreground extraction device, foreground extraction method, and foreground extraction program - Google Patents

Description

The present invention relates to a foreground extraction device for extracting an object region that does not exist in a background image from a video frame, a foreground extraction method, and a foreground extraction program.

Conventionally, there has been known a foreground extraction technique for comparing a background image prepared in advance with a reference image and extracting an object region that does not exist in the background image from the reference image. As a technique for extracting the foreground, a method of extracting a difference for each pixel based on the brightness and extracting a portion having a large difference as a foreground, or a method of calculating the similarity of the brightness distribution for each local region of the image and having a low degree of similarity. There is a method to extract the part as the foreground.
Since the former takes the difference in brightness for each pixel, there is a problem that erroneous extraction often occurs in an image in which the sunshine condition of the reference image is significantly different from that of the background image and the brightness is significantly different.

On the other hand, in the latter method, erroneous extraction can be suppressed by using the texture feature representing the light and dark pattern of the image for the calculation of the degree of similarity. This is because the texture feature representing the light-dark pattern is unlikely to change even if the brightness changes due to the change in sunshine.

For example, Patent Document 1 discloses a technique of calculating the similarity by comparing the patterns of the background image and the input image for each local region of the image and extracting the foreground. This method uses texture features that are robust against brightness fluctuations due to changes in sunshine, and detects shadow candidate blocks that are likely to be shadows to accurately estimate the foreground region even with dark shadows. Is possible.

Japanese Unexamined Patent Publication No. 2018-26068

Kiyoshi Nagashima, Takafumi Aoki, Tatsuo Higuchi, Koji Kobayashi "Higher Precision Subpixel Image Matching Based on Phase-Limited Correlation Method" The 218th Workshop of the Tohoku Branch of the Society of Instrument and Control Engineers (2014.10.9)

However, in the technique disclosed in Patent Document 1, the similarity for each local region is calculated in order to extract the foreground region, and when the foreground of the entire image is extracted, the local region is displayed on the screen. Since it is necessary to scan the entire image, there is a problem that the amount of calculation is large and the processing load is high.

An object of the present invention is to reduce the amount of calculation and the processing load when extracting the foreground.

The foreground extraction device of the present invention
A foreground extraction unit that extracts an edge from a video frame and extracts a foreground from the video frame using the edge of the video frame and a background image shot in a state where there is no foreground object in the shooting area of the video frame. Equipped with.

Since the foreground extraction device of the present invention extracts an edge from a video frame and extracts the foreground using the edge, the amount of calculation can be reduced and the processing load can be reduced when extracting the foreground.

The block diagram of the foreground extraction apparatus 20 which concerns on Embodiment 1. FIG. The hardware block diagram of the foreground extraction apparatus 20 which concerns on Embodiment 1. FIG. The flowchart which shows the operation of the foreground extraction apparatus 20 which concerns on Embodiment 1. The block diagram of the foreground extraction part 202 which concerns on Embodiment 1. FIG. The flowchart which shows the operation of the foreground extraction part 202 which concerns on Embodiment 1. The processing schematic diagram which concerns on Embodiment 1. FIG. 6 is a configuration diagram of a modified example of the foreground extraction device 20 according to the first embodiment. The block diagram of the foreground extraction part 202 which concerns on Embodiment 2. The flowchart which shows the operation of the foreground extraction part 202 which concerns on Embodiment 2. The block diagram of the foreground extraction part 202 which concerns on Embodiment 3. The flowchart which shows the operation of the foreground extraction part 202 which concerns on Embodiment 3. The processing schematic diagram which concerns on Embodiment 3. The processing schematic diagram which concerns on Embodiment 3.

Embodiment 1.
Hereinafter, embodiments will be described in detail with reference to the drawings.

*** Explanation of configuration ***
FIG. 1 is a block diagram of the foreground extraction device 20 according to the first embodiment.
As shown in FIG. 1, the foreground extraction device 20 is connected to a camera 10 that shoots and distributes an image in a shooting area, and decodes a compressed video stream distributed from the camera 10 and converts it into a video frame. A unit 201, a foreground extraction unit 202 for extracting the foreground using a video frame, and a foreground image output unit 203 for outputting the extraction result to the outside of the apparatus are provided.
Further, the foreground extraction device 20 includes a parameter storage storage 204 for storing a background image and storing analysis parameters for analyzing an image of a video frame.

FIG. 2 is a hardware configuration diagram of the foreground extraction device 20 according to the first embodiment.
The foreground extraction device 20 is composed of the general computer 100 shown in FIG.
The video reception / decoding unit 201 and the foreground image output unit 203 are composed of a processor 1001, a memory 1002, and a network interface 1003. The foreground extraction unit 202 and the foreground image output unit 203 are composed of a processor 1001 and a memory 1002. The parameter storage storage 204 is composed of a storage device 1004.

The processor 1001 is connected to other hardware via a signal line and controls these other hardware.
The storage device 1004 stores a program that realizes the functions of each functional component of the foreground extraction device 20. This program is read into memory 1002 by processor 1001 and executed by processor 1001. As a result, the functions of each functional component of the foreground extraction device 20 are realized.

The processor 1001 is an IC (Integrated Circuit) that performs arithmetic processing. Specific examples of the processor 1001 are a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and a GPU (Graphics Processing Unit).

The memory 1002 is a storage device that temporarily stores data. As a specific example, the memory 1002 is a SRAM (Static Random Access Memory) or a DRAM (Dynamic Random Access Memory).

The storage device 1004 is a storage device for storing data. The storage device 1004 is, as a specific example, an HDD (Hard Disk Drive). Further, the storage device 1004 may be a portable recording medium such as a memory card, a NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD (Digital Versaille Disc).

The network interface 1003 is an interface for communicating with an external device. As a specific example, the network interface 1003 has a port of Ethernet (registered trademark), USB (Universal Serial Bus), or HDMI (registered trademark, High-Definition Multimedia Interface).

Although only one processor 1001 is shown in FIG. 2, the foreground extraction device 20 may include a plurality of processors that replace the processor 1001. These plurality of processors share the execution of the program that realizes the function of each functional component.

*** Explanation of operation ***
Hereinafter, the operation of the foreground extraction device according to the first embodiment will be described.
FIG. 3 is a flowchart showing the operation of the foreground extraction device 20.

(Step ST01: Parameter reading process for analysis)
The foreground extraction unit 202 reads the background image and the analysis parameters from the parameter storage storage 204. The read background image and parameters are used in the process of the foreground extraction unit 202 described later.
The background image is image data obtained by shooting a shooting area with the camera 10 in a state where there is no foreground object in the shooting area of the camera 10.

(Step ST02: Moving image reception / decoding processing)
The video reception / decoding unit 201 waits for receiving the compressed video stream from the camera 10 connected to the foreground extraction device 20, and when the compressed video stream is received, the compressed video stream is decoded by one frame and converted into a video frame. Convert.
The video frame refers to one still image, and includes a video signal composed of vertical X pixels and horizontal Y pixels.

The video stream to be received here is, for example, H. 262 / MPEG-2 video, H.M. 264 / AVC, H. MPEG-2 TS (Moving Picture) of video coded data compressed by a video compression coding method such as 265 / HEVC or JPEG.
Experts Group 2 Transport Stream), RTP / RTSP (Real-time Transport Protocol / Real Time Streaming Protocol), MMT (MPEG Media Transport) or DASH (Digital Protocol) Target what is done.

The video stream may be distributed in an encoding format or distribution format other than the above, or may be distributed in an uncompressed transmission standard format such as SDI (Serial Digital Interface) or HD-SDI (High Definition Serial Digital Interface). May be good.

(Step ST03: Foreground image generation process)
The foreground extraction unit 202 extracts an edge from the video frame input from the video reception / decoding unit 201, and uses the edge of the video frame and the background image taken in a state where there is no foreground object in the shooting area of the video frame. Then, the foreground is extracted from the video frame and output. The details of the operation of the foreground extraction unit 202 will be described later.

(Step ST04: Output processing of foreground extraction result)
Next, the foreground image output unit 203 outputs the foreground extraction result output from the foreground extraction unit 202 to the outside of the foreground extraction device 20.
Examples of the output format include display on a monitor, output to a log file, output to an externally connected device, transmission to a network, and the like. Alternatively, the output may be in other formats. Further, it may be output to the outside every time the foreground extraction result is output from the foreground extraction unit 202, or it may be output after a specific period or a specific number of foreground extraction results are aggregated or statistically processed. May be.
After step ST04, the process returns to step ST02 to process the next video frame.

*** Explanation of the configuration of the foreground extraction unit 202 ***
FIG. 4 is a configuration diagram of the foreground extraction unit 202 in the present embodiment.
The foreground extraction unit 202 includes an edge extraction unit 2021 that extracts an edge from an input video frame, a local area setting unit 2022 that specifies a local area centered on the extracted edge, and a video frame in the designated local area. A similarity calculation unit 2023 for calculating the similarity of the background image and a binarization processing unit 2024 for extracting the foreground from the calculated similarity distribution are provided.

*** Explanation of the operation of the foreground extraction unit 202 ***
Hereinafter, the detailed operation of the foreground extraction unit 202 will be described.
FIG. 5 is an operation flowchart of the foreground extraction unit 202 in the present embodiment.
FIG. 6 is a schematic processing diagram of the present embodiment.

(Step ST11: Edge extraction process)
First, the edge extraction unit 2021 performs edge extraction on the input video frame, and outputs the edge extraction result to the local area setting unit 2022.
An edge is a part of an image where the brightness or color changes suddenly.
Edge extraction refers to extracting a part of an image where the brightness or color suddenly changes.
The edge extraction of the edge extraction unit 2021 can be realized by a known technique.
In FIG. 6, the edge extraction unit 2021 extracts two diagonal lines and a humanoid as an image edge from the image frame.

(Step ST12: Local area / comparison area setting process)
Next, the local area setting unit 2022 sets a local area centered on the pixel from which the edge is extracted by using the edge extraction result input from the edge extraction unit 2021.
Further, the local area setting unit 2022 sets the area corresponding to the local area of the background image as a comparison area to be compared with the local area.
The local region is, for example, a rectangular image region of n pixels × n pixels (n is an integer of 3 or more) centered on any one pixel extracted as an edge.
The comparison area is a rectangular image area of n pixels × n pixels having the same position and size as the local area on the background image.
In FIG. 6, the local area setting unit 2022 sets a local area for the pixels constituting the two diagonal lines and the humanoid shape with respect to the video frame based on the video edge extraction result, and further, the local area of the background image is set. The comparison area is set at the position corresponding to the area.

(Step ST13: Similarity calculation process)
Next, the similarity calculation unit 2023 calculates the similarity between the background image and the video frame in the local area set by the local area setting unit 2022. The similarity calculation unit 2023 calculates the degree of similarity between the image of the local area set by the local area setting unit 2022 and the image of the comparison area of the background image.

To calculate the degree of similarity, a method that uses texture features that are robust to sunshine conditions such as sun and shade is used. As the texture feature to be used, for example, a phase-limited correlation method (see, for example, Non-Patent Document 1) in which the degree of similarity is calculated from the phase information of an image can be considered. Other types of texture features that can be used include edges, luminance gradients, local binary patterns, correlations, histograms, entropy, and the like.

The simplest texture feature is the edge, but when the edge is misaligned, the similarity is significantly reduced and erroneous extraction is likely to occur. On the other hand, phase information, correlation, histogram, and entropy use statistical data of region occurrence frequency or light / dark pattern, so even if misalignment occurs, the decrease in similarity is smaller than that of edges, which is erroneous. It is difficult to generate extraction.

Among them, the phase-limited correlation method using phase information is a method of calculating the similarity after considering the amount of misalignment, and even if the misalignment occurs, the degree of similarity does not decrease so much that the misalignment occurs. Robust. Therefore, by using the phase-limited correlation method to calculate the degree of similarity, it is possible to extract a foreground that is robust against misalignment.
Necessary parameters are set for the analysis parameters according to the similarity calculation method, and the similarity calculation unit 2023 uses the analysis parameters for the similarity calculation.

Steps ST12 and ST13 are repeated for all the pixels included in the edges extracted by the edge extraction unit 2021. That is, the local area setting unit 2022 sets a local area centered on all the pixels constituting the edge, and then sets a comparison area corresponding to the local area as the background image. The similarity calculation unit 2023 calculates the similarity between the local area of the video frame and the comparison area of the background image in all the local areas set by the local area setting unit 2022. Specifically, the process of setting the local area from which the pixels are extracted and the corresponding comparison area for which the similarity is not calculated in step ST12 and calculating the similarity of the video frame in step S13 is performed for all the pixels. Execute for the local area corresponding to.
In FIG. 6, the similarity between the local region set for the two diagonal lines and the comparison region is high, and the similarity between the local region set for the humanoid and the comparison region is low.

(Step ST14: Binarization process)
The binarization processing unit 2024 obtains the distribution of similarity from the similarity of all the pixels constituting the edge obtained by repeating steps ST12 and ST13. The binarization processing unit 2024 binarizes the distribution of similarity with a reference value, separates the foreground and the background, and extracts only the region with low similarity as the foreground.
In FIG. 6, in the binarization processing unit 2024, the similarity between the local region set for the two diagonal lines and the comparison region is higher than the reference value, so that the pixel in the center of the local region is the background. Judging and ignoring.
On the other hand, in the binarization processing unit 2024, the similarity between the local region set for the humanoid and the comparison region is lower than the reference value, so that the pixel in the center of the local region is the pixel constituting the foreground. Extract as.
In FIG. 6, a set of pixels in the center of a region having a low degree of similarity is shown as a foreground extraction result (binary value). As a result, FIG. 6 shows that the humanoid foreground was extracted.

*** Features of Embodiment 1 ***
The foreground extraction device of the first embodiment extracts a parameter storage storage 204 for holding analysis parameters and an edge from the video frame, and there is no foreground object in the edge of the video frame and the shooting area of the video frame. It is characterized by including a foreground extraction unit 202 that extracts a foreground from a video frame using a background image taken in the state, and a foreground image output unit 203 that outputs the foreground extracted by the foreground extraction unit 202.

The parameter storage storage 204 holds a background image.
The foreground extraction unit 202 sets an edge extraction unit 2021 that extracts an edge from a video frame, a local area centered on a pixel constituting the extracted edge, and a comparison area corresponding to a local area of a background image. It is provided with a setting unit 2022.
The foreground extraction unit 202 has a similarity calculation unit 2023 for calculating the similarity between the image of the local area set by the local area setting unit and the image of the comparison area, and the similarity calculation unit 2023 for calculating the similarity degree. It is provided with a binarization processing unit 0204 that performs binarization processing and extracts the foreground.

The foreground extraction device of the first embodiment realizes high speed by omitting the similarity calculation of the local region of an unnecessary portion when extracting the foreground. Specifically, the foreground extraction device of the first embodiment pays attention to the fact that the foreground often appears around the edge in the foreground extraction using the texture feature, and processes only the edge periphery to improve the accuracy. Achieve high speed while maintaining.

*** Explanation of the effect of Embodiment 1 ***
As described above, according to the first embodiment, in the foreground extraction technique capable of extracting different regions of the background image and the reference image, not only can the foreground be accurately extracted even under different sunshine conditions such as sunlight or shade. , High speed can be realized while maintaining the accuracy of foreground extraction.
In particular, the foreground extraction device 20 can extract the foreground at high speed with less erroneous extraction even between images having different sunshine conditions.
Further, by using the phase-limited correlation method in the similarity calculation unit 2023, it is possible to extract a foreground robustly even if the position shifts.

In particular, the foreground extraction device of the first embodiment is highly effective in an environment where the sunshine conditions change. For example, detection of leftover objects in an outdoor environment can be mentioned.

<Modification 1>
The foreground extraction device 20 of the first embodiment is configured to target the camera 10 as an input device, but is not limited to the RGB camera. For example, a thermal camera (Thermal Camera) or a depth camera (Depte Camera) may be used as an input device.

<Modification 2>
The local area setting unit 2022 does not have to set one local area for each pixel constituting the edge. The local area setting unit 2022 may set one local area for a plurality of pixels constituting the edge. For example, the local area setting unit 2022 may set one local area for an adjacent and continuous specific number or arbitrary number of pixels.
The local region may include at least one pixel constituting the extracted edge.

<Modification 3>
In the first embodiment, the case where each functional component is realized by software has been described. However, as a modification, each functional component may be realized by hardware.

The configuration of the foreground extraction device 20 according to the modified example will be described with reference to FIG. 7.
When each functional component is realized by hardware, the foreground extraction device 20 includes a processing circuit 1005 instead of the processor 1001. Alternatively, although not shown, the foreground extraction device 20 includes a processing circuit 1005 in place of the processor 1001, the memory 1002, and the storage device 1004. The processing circuit 1005 is a dedicated electronic circuit that realizes the functions of each functional component (and the memory 1002 and the storage device 1004).

The processing circuit 1005 is assumed to be a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), and an FPGA (Field-Programmable Gate Array). Will be done.
Each functional component may be realized by one processing circuit 1005, or each functional component may be distributed and realized by a plurality of processing circuits 1005.

Alternatively, some functional components may be implemented in hardware and other functional components may be implemented in software.

The processor 1001, the memory 1002, the storage device 1004, and the processing circuit 1005 described above are collectively referred to as "processing circuit Lee". That is, the function of each functional component is realized by the processing circuit.

Embodiment 2.
Hereinafter, the points different from the above-described embodiments will be described with reference to the drawings.

*** Explanation of configuration ***
FIG. 8 is a block diagram of the foreground extraction unit 202 in the present embodiment.
In the above-described embodiment, if the background image and the video frame are misaligned, the foreground may be erroneously extracted. In order to solve this problem, in the second embodiment, the misalignment amount calculation unit 2025 is added to the similarity calculation unit 2023 of the first embodiment to realize a robust foreground extraction against the misalignment.

*** Explanation of operation ***
FIG. 9 is a flowchart showing the operation of the foreground extraction unit 202 in the present embodiment.

(Step ST12: Local area / comparison area setting process)
In the local area setting unit 2022, the local area centered on the pixel from which the edge is extracted and the comparison area of the background image are set by using the edge extraction result input from the edge extraction unit 2021.
(Step ST15: Similarity / misalignment amount calculation process)
When the similarity calculation unit 2023 operates, the position shift amount calculation unit 2025 calculates the position shift amount between the local region and the comparison region by using the phase-limited correlation method.
If the misalignment amount is not 0, the local region setting unit 2022 sets the local region again so that the misalignment amount becomes 0 (step ST12) in consideration of the misalignment amount, and re-sets the misalignment amount. Calculate (step ST15). This recalculation may be performed once or may be repeated until no improvement in the amount of misalignment is observed.
When the amount of misalignment becomes 0, the similarity calculation unit 2023 calculates the similarity. The similarity calculation unit 2023 may calculate the similarity every time together with the calculation of the amount of misalignment.
In this way, the similarity calculation unit 2023 calculates the amount of misalignment between the image in the local region and the image in the comparison region, suppresses the amount of misalignment, and calculates the degree of similarity.

*** Explanation of the effect of Embodiment 2 ***
According to the second embodiment, since the local area setting unit 2022 sets the local area so that the position deviation calculated by the position deviation amount calculation unit 2025 disappears, the foreground extraction robust to the position deviation becomes possible.

Embodiment 3.
Hereinafter, the points different from the above-described embodiments will be described with reference to the drawings.

*** Explanation of configuration ***
FIG. 10 is a block diagram of the foreground extraction unit 202 according to the present embodiment.
The foreground extraction unit 202 includes a background edge extraction unit 2026 that extracts the edges of the background image, and an edge mask unit 2027 that masks the result of the edge extraction unit 2021 with the result of the background edge extraction unit 2026.

In the above-described embodiment, since the similarity is calculated for all the edge portions obtained from the video frame, the similarity is calculated up to the part where the calculation is unnecessary, such as the edge portion of the structure existing in the background. It will be.
In addition, when a change in sunshine occurs around the edge of the structure, reflection or the like occurs and the degree of similarity with the background image tends to decrease, so that erroneous extraction of the foreground may easily occur.
In the third embodiment, as a preliminary step in the first embodiment, the background edge extraction unit 2026 is added in parallel with the edge extraction unit 2021, and the edge mask unit 2027 is added between the edge extraction unit 2021 and the local area setting unit 2022. do.

*** Explanation of operation ***
FIG. 11 is a flowchart showing the operation of the foreground extraction unit 202 in the present embodiment.
12 and 13 are schematic processing diagrams of the present embodiment.

(Step ST16: Background edge extraction process)
The background edge extraction unit 2026 acquires a background image from the parameter storage storage 204. The background edge extraction unit 2026 extracts the edge of the background image as the background edge and outputs it to the edge mask unit 2027.
In FIG. 12, the background edge extraction unit 2026 extracts two diagonal lines as the background edge of the background image.

(Step ST17: Edge mask application process)
Next, in the edge mask unit 2027, the edge extraction result obtained by the edge extraction unit 2021 is masked by using the edge extraction result obtained from the background edge extraction unit 2026.
The edge mask unit 2027 determines that the edge of the video frame that overlaps the edge of the background image is an edge existing in the background, and removes the edge existing in the background from the edge of the video frame to remove the edge existing in the background. Extract as an edge.
In FIG. 12, the edge mask unit 2027 generates a mask edge image by taking the exclusive OR of each pixel of the video edge extraction result and the background edge extraction result.
In FIG. 12, a person who determines that the edge mask portion 2027 is an edge existing in the background with two diagonal lines where the background edges overlap in the video edge, and deletes the two diagonal lines from the edge of the video frame. Only the edges of the type are extracted as the foreground edges.

(Step ST12: Local area / comparison area setting process)
Next, in the local area setting unit 2022, the local area centered on the pixel from which the edge is extracted and the comparison area of the background image are set by using the human-shaped edge extraction result extracted by the edge mask unit 2027.
In FIG. 13, the local area setting unit 2022 sets a local area centered on the pixels constituting the edge of the humanoid foreground and a comparison area corresponding to the local area. The local area setting unit 2022 does not set the local area centered on the pixels constituting the edge existing in the background and the comparison area corresponding to the local area.

(Step ST13: Similarity calculation process)
Next, the similarity calculation unit 2023 calculates the similarity between the background image and the video frame in the local area set by the local area setting unit 2022. The similarity calculation unit 2023 calculates the similarity only around the unmasked edges.
In FIG. 13, the similarity calculation unit 2023 calculates the similarity of the local region centered on the pixels constituting the edge of the humanoid foreground. The similarity calculation unit 2023 does not calculate the similarity of the local region centered on the pixels constituting the edges existing in the background.

*** Explanation of the effect of Embodiment 3 ***
According to the third embodiment, since the edge existing in the background can be excluded from the similarity calculation, the foreground extraction can be performed at higher speed and with less erroneous extraction.

In particular, by performing the edge mask application process once, the local area setting process and the similarity calculation process can be excluded as many times as the number of pixels constituting the background edge, and high-speed processing can be realized.

*** Other embodiments ***
It is possible to freely combine the above-described embodiments, modify any component of each embodiment, or omit any component in each embodiment.

The foreground extraction device of each of the above-described embodiments can be used in an image comparison device, a difference extraction device, an image comparison system, and a difference extraction system that compare two images and detect the difference between the two images. ..
When the foreground extraction device is used for an image comparison device, a difference extraction device, an image comparison system, or a difference extraction system, the "foreground" is the "difference between two images", and the foreground extraction unit 202 extracts the difference. The foreground image output unit 203 is a difference image output unit.

The foreground extraction device of each of the above-described embodiments can be incorporated as a part of an image processing device, an image processing system, an animation processing device, or an animation processing system.

10 camera, 20 foreground extractor, 100 computer, 201 video reception / decoding unit, 202 foreground extraction unit, 203 foreground image output unit, 204 parameter storage storage, 1001 processor, 1002 memory, 1003 network interface, 1004 storage device, 1005 processing Circuit, 2021 edge extraction unit, 2022 local area setting unit, 2023 similarity calculation unit, 2024 binarization processing unit, 2025 misalignment amount calculation unit, 2026 background edge extraction unit, 2027 edge mask unit.

Claims (6)

A foreground extraction unit that extracts an edge from a video frame and extracts a foreground from the video frame using the edge of the video frame and a background image shot in a state where there is no foreground object in the shooting area of the video frame. Equipped with
The foreground extraction unit
An edge extraction unit that extracts edges from the video frame,
A local area setting unit that sets a local area including pixels constituting the extracted edge and a comparison area corresponding to the local area of the background image, and a local area setting unit.
A similarity calculation unit that calculates the similarity between the image of the local region and the image of the comparison region set by the local region setting unit, and the similarity calculation unit.
With the binarization processing unit that performs binarization processing on the similarity calculated by the similarity calculation unit and extracts the foreground.
Foreground extraction device equipped with . The local area setting unit sets the local area for the pixels constituting the edge extracted by the edge extraction unit, and then sets the comparison area in the background image.
The foreground according to claim 1 , wherein the similarity calculation unit calculates the similarity between the image of the local region and the image of the comparison region when the local region setting unit sets the local region and the comparison region. Extractor. The invention according to claim 1 or 2 , wherein the similarity calculation unit calculates the amount of misalignment between the image of the local region and the image of the comparison region, suppresses the amount of misalignment, and calculates the degree of similarity. Foreground extraction device. The foreground extractor also
A background edge extraction unit that extracts edges of the background image,
The foreground extraction device according to any one of claims 1 to 3 , further comprising an edge mask unit that masks the edge extracted by the edge extraction unit with the edge extracted by the background edge extraction unit. In the foreground extraction method of the foreground extraction device
The foreground extraction unit extracts an edge from the video frame, and uses the edge of the video frame and the background image shot with no object as the foreground in the shooting area of the video frame to extract the foreground from the video frame. It is a foreground extraction method to extract.
The foreground extraction unit
The edge extraction unit extracts an edge from the video frame,
The local area setting unit sets a local area including pixels constituting the extracted edge and a comparison area corresponding to the local area of the background image.
The similarity calculation unit calculates the similarity between the image of the local area set by the local area setting unit and the image of the comparison area.
A foreground extraction method in which the binarization processing unit performs binarization processing on the similarity calculated by the similarity calculation unit and extracts the foreground . On the computer
Foreground extraction processing that extracts an edge from a video frame and extracts a foreground from the video frame using the edge of the video frame and a background image shot in a state where there is no foreground object in the shooting area of the video frame. It is a foreground extraction program that executes
The foreground extraction process
Edge extraction processing that extracts edges from the video frame and
A local area setting process for setting a local area including pixels constituting the extracted edge and a comparison area corresponding to the local area of the background image, and
The similarity calculation process for calculating the similarity between the image of the local area and the image of the comparison area set by the local area setting process, and the similarity calculation process.
A binarization process for extracting the foreground by performing a binarization process on the similarity calculated by the similarity calculation process.
Foreground extraction program to run .

Images