JP6949656B2 - Object distribution estimator - Google Patents

Description

The present invention relates to an object distribution estimation device that estimates the distribution of an object from time-series photographed images in which a space in which a predetermined object such as a person can exist is sequentially photographed.

In spaces where congestion may occur, such as event venues, it is necessary to take measures such as allocating a large number of security guards in areas where congestion is occurring in order to prevent accidents. Therefore, by arranging surveillance cameras at various places in the venue, estimating the distribution of people from the captured image, and displaying the estimated distribution, it is possible to facilitate the grasp of the congestion situation by the observer.

One of the methods for estimating the distribution of people is to scan the captured image with a classifier that has learned the characteristics of the image at the time of congestion of people in advance. For example, in the crowd analysis device described in Patent Document 1 below, the person density is estimated using a discriminator machine-learned for each person density using a learning image in which a crowd having a person density exceeding the lower limit of density is captured in advance. It is stated that by doing so, the occurrence of the crowd is determined.

Japanese Unexamined Patent Publication No. 2017-068598

Event venues and the like are generally vast, and a large number of images taken by a large number of surveillance cameras are acquired every moment and become an estimation target. Therefore, it is strongly required to suppress the processing cost related to the images taken by each surveillance camera.

Here, in the area where people are crowded in the captured image, since the movement of people is slow, a sudden change in the congestion situation is unlikely to occur, and the change that occurs in a short time in the display of the distribution of people is minute.

However, in the prior art, since the person density is estimated for the entire captured image each time the captured image is input regardless of the presence or absence of a congested area, wasteful processing cost is incurred.

The present invention has been made in view of the above problems, and provides a distribution estimation device that estimates the distribution of an object with a low load from time-series photographed images in which a space in which a predetermined object can exist is sequentially photographed. With the goal.

(1) The object distribution estimation device according to the present invention includes an image acquisition means for sequentially acquiring captured images of a space in which a predetermined object can exist at predetermined time intervals, and a plurality of local regions set in the captured images. Congestion estimation means for estimating the degree of congestion of the object in each of the local regions as distribution information of the object in the space by performing analysis for each, and each of the local regions corresponds to the local region by the congestion estimation means. The update interval setting means for setting the update time interval of the distribution information to be set to a length corresponding to the degree of congestion of the local region is provided.

(2) In the object distribution estimation device according to (1) above, the update interval setting means is configured to set the update time interval longer as the local region has a higher degree of congestion estimated by the congestion estimation means. be able to.

(3) In the object distribution estimation device according to the above (1) and (2), the update interval setting means has the lowest degree of congestion in the block for each block composed of two or more of the local regions. The update time interval can be set according to the above.

(4) In the object distribution estimation device according to the above (1) to (3), the congestion estimation means captures the characteristics of each density image obtained by photographing the space in which the object exists at a predetermined density at the density. Using the learned density estimator, the density of the object representing the degree of congestion can be estimated for each local region.

According to the present invention, the higher the congestion degree of the object being photographed, the longer the congestion degree of the object is updated. Therefore, the space in which a predetermined object such as a person can exist is sequentially photographed in a time series. The distribution of objects can be estimated from the captured image with a low load.

It is a block diagram which shows the schematic structure of the object distribution estimation apparatus which concerns on embodiment of this invention. It is a functional block diagram of the object distribution estimation apparatus which concerns on embodiment of this invention. It is a schematic flow chart of the operation of the image processing unit mainly in the object distribution estimation apparatus which concerns on embodiment of this invention. It is a schematic flow chart of the density estimation process for each block. It is a schematic diagram which shows the processing example at a certain time t of the object distribution estimation apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the processing example at time t + 1 of the object distribution estimation apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the processing example at time t + 2 of the object distribution estimation apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the processing example at time t + 3 of the object distribution estimation apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the processing example at time t + 4 of the object distribution estimation apparatus which concerns on embodiment of this invention.

Hereinafter, the object distribution estimation device 1 according to the embodiment of the present invention (hereinafter referred to as the embodiment) will be described with reference to the drawings. The object distribution estimation device 1 estimates the distribution of people in the space (monitoring space) in which people can exist by sequentially analyzing the captured images taken at predetermined time intervals, and monitors the distribution image which is the estimation result. Display to members.

[Structure of object distribution estimation device 1]
FIG. 1 is a block diagram showing a schematic configuration of the object distribution estimation device 1. The object distribution estimation device 1 includes a photographing unit 2, a communication unit 3, a storage unit 4, an image processing unit 5, and a display unit 6.

The photographing unit 2 is a surveillance camera, is connected to the image processing unit 5 via the communication unit 3, captures the monitoring space at predetermined time intervals to generate a captured image, and sequentially captures the captured images in the image processing unit 5. It is a shooting means to input to. Hereinafter, the shooting timing of the shooting unit 2 will be referred to as a time.

For example, the photographing unit 2 is installed on a pole installed at the event venue with a field of view overlooking the monitoring space. The field of view may be fixed, or may be changed according to a schedule in advance or an instruction from the outside via the communication unit 3. For example, the photographing unit 2 photographs the monitoring space with a frame period of 1 second to generate a color image. Further, the photographing unit 2 may generate a monochrome image instead of the color image.

The communication unit 3 is a communication circuit, one end of which is connected to the image processing unit 5, and the other end of the communication unit 3 is connected to the photographing unit 2 and the display unit 6 via a communication network such as a coaxial cable, LAN (Local Area Network), or the Internet. Be connected. The communication unit 3 acquires a captured image from the photographing unit 2 and inputs it to the image processing unit 5, and outputs the estimation result input from the image processing unit 5 to the display unit 6.

The storage unit 4 is a memory device such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and stores various programs and various data. The storage unit 4 is connected to the image processing unit 5 and inputs / outputs these information to and from the image processing unit 5.

The image processing unit 5 is composed of arithmetic units such as a CPU (Central Processing Unit), a DSP (Digital Signal Processor), and an MCU (Micro Control Unit). The image processing unit 5 is connected to the storage unit 4 and operates as various processing means / control means by reading and executing a program from the storage unit 4, storing various data in the storage unit 4, and also from the storage unit 4. read out. Further, the image processing unit 5 is also connected to the photographing unit 2 and the display unit 6 via the communication unit 3, and analyzes the photographed image acquired from the photographing unit 2 via the communication unit 3 to obtain the distribution of people in the monitoring space. The estimation is performed, and the estimation result is displayed on the display unit 6 via the communication unit 3.

The display unit 6 is a display device such as a liquid crystal display or a CRT (Cathode Ray Tube) display, and is a display means connected to the image processing unit 5 via the communication unit 3 and displaying the estimation result by the image processing unit 5. .. The observer visually recognizes the displayed estimation result, judges the occurrence of congestion, etc., and takes measures such as changing the staffing as necessary.

In this embodiment, the object distribution estimation device 1 in which the number of the photographing unit 2 and the image processing unit 5 is 1: 1 is illustrated, but in another embodiment, the photographing unit 2 and the image processing unit 5 The number can be many-to-one or many-to-many.

[Function of object distribution estimation device 1]
FIG. 2 is a functional block diagram of the object distribution estimation device 1. The communication unit 3 functions as an image acquisition means 30, an object distribution output means 31, and the like, and a storage unit 4 functions as a density estimator storage means 40, an object distribution storage means 41, an analysis waiting time storage means 42, and the like. The image processing unit 5 functions as a density estimation means 50, an update interval setting means 51, and the like.

Hereinafter, each means will be described with reference to FIG.

The image acquisition means 30 sequentially acquires captured images from the photographing unit 2 which is the photographing means, and sequentially outputs the acquired captured images to the density estimation means 50.

The density estimator storage means 40 is an estimated density calculation function that learns the image features of each density image obtained by photographing a space in which an object (person) exists at the density at a predetermined density, and obtains the feature amount of the image. When input, the estimated value (estimated density) of the density of the object captured in the image is calculated, and the information of the estimator (density estimator) that outputs the calculated estimated density is stored in advance. That is, parameters such as the coefficient of the estimated density calculation function are stored in advance as information of the density estimator.

The density estimation means 50 analyzes each of a plurality of local regions set in the captured image, and estimates the density of the object photographed in the region as the degree of congestion (congestion degree) of the object in the region. It is an estimation means and estimates the degree of congestion of objects in each local area as distribution information of objects in the monitoring space. Specifically, the density estimation means 50 extracts the feature amount for density estimation from an arbitrary local region in the captured image input from the image acquisition means 30, and reads out the density estimator from the density estimator storage means 40. , The density is estimated by inputting each of the extracted features into the density estimator. By performing this estimation at a plurality of positions in the captured image, the estimated density distribution (object density distribution) in the captured image is obtained, and the density estimation means 50 calculates the estimated density distribution by the object distribution output means 31. , Is output to the update interval setting means 51 and the object distribution storage means 41.

The processing of density estimation and the density estimator will be specifically described.

In this embodiment, each pixel in the captured image is set as a local region. The density estimation means 50 sets a window for density estimation at the position of each pixel of the captured image, and extracts a feature amount from the captured image in each window. The feature quantity is a GLCM (Gray Level Co-occurrence Matrix) feature.

It is desirable that the areas in the surveillance space photographed by each window are the same size. That is, preferably, the density estimation means 50 reads out the camera parameters of the photographing unit 2 stored in advance from the camera parameter storage means (not shown), and is photographed on an arbitrary pixel of the captured image by homography conversion using the camera parameters. After transforming the captured image so that the area in the surveillance space is the same size, the window is set and the feature amount is extracted. Each window can be an area having the same shape and size as the density image described later.

The density estimator can be realized by a discriminator that discriminates a multi-class image, and can be a discriminant function learned by a multi-class SVM (Support Vector Machine) method.

Density, for example, there is no human "Background" class is 0 people / m higher than ² is two / m ² or less "low density" class, higher than two / m ² 4 persons / m ² or less It can be defined as 4 classes of "medium density" class and ^{"high density" class higher than 4 people / m 2.}

The estimated density is a value given in advance to each class and is a value output as a result of distribution estimation. In this embodiment, the values corresponding to each class are described as "background", "low density", "medium density", and "high density".

That is, the density estimator applies the multi-class SVM method to the features of a large number of images (density images) belonging to each of the "background" class, "low density" class, "medium density" class, and "high density" class. This is an identification function for distinguishing the density image of each class from other classes, which is obtained by learning. The parameters of the discriminant function derived by this learning are stored as a density estimator. The feature amount of the density image is the same as the feature amount extracted by the density estimation means 50, and is a GLCM feature.

The density estimation means 50 acquires the estimated density, which is the output value, by inputting each of the feature quantities extracted corresponding to each pixel into the density estimator. When the feature amount is extracted by deforming the captured image, the density estimation means 50 transforms the density distribution into the shape of the original captured image by homography transformation using camera parameters.

The set of estimated densities for each pixel of the captured image obtained in this way is the density distribution. Here, in the local region (highly congested region) where the estimated density is in the “high density” class, the movement of people is slowed down due to congestion, and a sudden change in the degree of congestion is unlikely to occur. Therefore, the highly congested area has little influence on the judgment of the observer even if it is not analyzed at short time intervals. On the other hand, in the local region (low congestion region) where the estimated density is the "background" class and the "low density" class, it is possible for a person to run and a sudden change in the degree of congestion can occur. Therefore, it is desirable to analyze the low-congestion area at short time intervals and present the state of change to the observer one by one. In addition, the speed of movement of people in a local region (medium-congested region) whose estimated density is in the "medium-density" class is considered to be an intermediate speed between a high-congested region and a low-congested region.

Therefore, the object distribution estimation device 1 suppresses the processing cost of density estimation while ensuring the reliability of the density estimation result by updating the estimated density at longer time intervals in the local region where the estimated density is higher.

Therefore, the update interval setting means 51 sets an update time interval (hereinafter, referred to as an update interval) for updating the distribution information corresponding to the local region by the density estimation means 50 for each local region. Then, at that time, the update interval setting means 51 sets a longer update interval as the local region with a higher degree of congestion estimated by the density estimation means 50. As a result, the update interval setting means 51 sets the update interval for each local region according to the estimated density estimated by the density estimation means 50, and the density estimation means 50 sets the density of the object for each local region in the local region. Estimate at the updated update interval.

For example, the update interval setting means 51 sets the update interval of the local region whose estimated density is the "high density" class to 4 hours, and the update interval of the local region whose estimated density is the "medium density" class to 2 hours. , The update interval of the local area whose estimated density was the "background" class and the local area whose estimated density was the "low density" class is set to 1 time.

Specifically, in order to control the update interval in real time, the update interval setting means 51 writes, for example, the update interval for each local area as the analysis waiting time in the analysis waiting time storage means 42, and newly uses the update interval setting means 51. The analysis waiting time is reduced each time a captured image is acquired. The length of time such as the update interval and the analysis waiting time is defined in units of the above-mentioned time, and corresponds to the number of frames of the captured image. The density estimation means 50 estimates the density of the local region where the analysis holding time becomes 0 and the update timing has arrived with reference to the analysis waiting time storage means 42.

Further, among the high-congested areas, the vicinity of the boundary with the medium-congested area or the low-congested area is more likely to have a change in the degree of congestion than the other areas. Similarly, in the medium-congested region near the boundary with the low-congested region, the degree of congestion is more likely to change than in the other regions. Therefore, the update interval setting means 51 sets an update interval according to the lowest density in the block for each of a plurality of blocks each consisting of two or more local regions, and the density estimation means 50 updates the update timing in block units. Judge the arrival of. The block is set in advance, and the update interval setting means 51 and the density estimation means 50 share the setting. The size of the block should be about one or two objects (one or two people). This size is the smallest unit that causes a change in the degree of congestion. By forming blocks according to the size of the object in this way, it is possible to prevent an update delay near the boundary and prevent unnecessary frequent updates other than near the boundary.

Further, in response to the fact that the density estimation by the density estimation means 50 is performed asynchronously for each local region or block, the density estimation means 50 stores the latest estimated density for each local region in the object distribution storage means 41. Update the density distribution with. By doing so, the object distribution storage means 41 holds the latest estimated density of all local regions. Each time the density distribution is updated, the density estimation means 50 reads out the estimated density of all the local regions stored in the object distribution storage means 41 and outputs the estimated density to the object distribution output means 31.

The density distribution may be output as the value of the estimated density, but it may also be a distribution image in which a predetermined color is set in the corresponding pixel according to the estimated density. For example, the density estimation means 50 estimates that the pixel value of the local region estimated to be the "high density" class is red, the pixel value of the local region estimated to be the "medium density" class is yellow, and the pixel value of the local region is yellow, and the "low density" class. A distribution image is generated and output in which the pixel value of the local area is set to green and the pixel value of the local area estimated to be in the "background" class is set to black.

Further, more preferably, in the present embodiment to be displayed, the density estimation means 50 generates and outputs a distribution image in which the distribution image is transparently synthesized with the captured image.

That is, the update interval setting means 51 sets the update interval of the estimated density of each of the plurality of local regions set in the captured image, and sets the update interval for each set local region as the analysis waiting time in the analysis waiting time storage means 42. Remember. At that time, the update interval setting means 51 sets a longer update interval as the density is higher in the local region with reference to the density (congestion degree) estimated by the density estimation means 50 (congestion estimation means). The density estimation means 50 is a density estimator stored in the density estimator storage means 40, and is a density estimation that learns the characteristics of each density image obtained by photographing a space in which an object exists at a predetermined density for each predetermined density. Using a device, the density of the object for each local area is estimated at the update interval set for the local area, and the estimated density for each local area is output to the update interval setting means 51, and for each estimated local area. Is stored in the object distribution storage means 41. Further, the density estimation means 50 reads out the density of each local region stored in the object distribution storage means 41 and outputs the density to the object distribution output means 31.

By doing so, it is possible to suppress the processing cost of density estimation while ensuring the reliability of the output density distribution.

Further, the update interval setting means 51 sets an update interval according to the lowest density in the block for each of a plurality of blocks each consisting of two or more local regions. By doing so, the reliability of the density distribution near the boundary of the region for each density can be made higher.

The object distribution output means 31 sequentially outputs the density distribution input from the density estimation means 50 to the display unit 6, and the display unit 6 displays the density distribution input from the object distribution output means 31. By visually recognizing the displayed density distribution, the observer grasps the point where congestion is occurring in the monitoring space, and takes measures such as dispatching or increasing the number of guards to the point.

[Operation of object distribution estimation device 1]
The operation of the object distribution estimation device 1 will be described with reference to the flow charts of FIGS. 3 and 4.

When the object distribution estimation device 1 starts operation, the photographing unit 2 installed at the event venue photographs the monitoring space at predetermined time intervals, and the photographed images are sequentially sent to the image analysis center where the image processing unit 5 is installed. Send. Further, when the field of view is changed in the photographing unit 2, the photographing unit 2 notifies the image analysis center that the field of view has been changed.

FIG. 3 is a schematic flow chart of the operation of the image processing unit 5 mainly in the image analysis center of the object distribution estimation device 1. The image processing unit 5 repeats the processes of steps S1 to S8 of FIG. 3 every time a photographed image is received from the photographing unit 2.

First, the image processing unit 5 operates as the update interval setting means 51, and confirms whether or not it is the first start-up and whether or not the field of view has been changed (step S1). That is, the update interval setting means 51 refers to the analysis waiting time storage means 42, and if the analysis waiting time information is not stored, determines that it is the first start-up. Further, the update interval setting means 51 confirms the notification from the photographing unit 2 via the communication unit 3, and if the notification of the field of view change is received, it is determined that the field of view has been changed.

When it is the first start-up, or when there is a change in the field of view (when "YES" in step S1), the update interval setting means 51 sets 1 [time] as the initial value of the update interval in all blocks and analyzes it. The update interval is stored in the analysis waiting time storage means 42 as the waiting time information (step S2). On the other hand, if it is not the first time to start and there is no change in the field of view (when "NO" in step S1), the initialization process of the update interval and the analysis waiting time in step S2 is skipped, and the analysis waiting time storage means has already been performed. The value stored in 42 is maintained.

Next, the communication unit 3 operates as the image acquisition means 30, and is in a state of waiting for reception of the captured image from the photographing unit 2. The image acquisition means 30 that has acquired the captured image outputs the captured image to the image processing unit 5 (step S3).

The image processing unit 5 in which the captured image is input operates as the update interval setting means 51, and causes the analysis waiting time of all the blocks stored in the analysis waiting time storage means 42 to be subtracted by 1 [time] (step S4). By this process, when the end of the update interval of the estimated density is reached, the analysis waiting time of the block becomes 0 [time].

Since the analysis waiting time is 0, the image processing unit 5 detects a block for which the estimated density update timing has arrived, and performs a process of updating the estimated density for the block. Specifically, the image processing unit 5 into which the captured image is input operates as the density estimation means 50, and performs loop processing in which each block set in the captured image is sequentially set as a block of interest (steps S5 to S7). ), The process related to density estimation (step S6) is performed.

FIG. 4 is a schematic flow chart of the process S6 relating to the density estimation for each block. The process S6 will be described with reference to FIG.

The density estimation means 50 confirms whether or not the analysis waiting time is 0 by referring to the analysis waiting time storage means 42 for the blocks of interest that are sequentially set in step S5 (step S80). If the analysis waiting time of the attention block is 0 (when “YES” in step S80), the density estimation means 50 estimates the density of each pixel in the attention block and stores it in the object distribution storage means 41. , The density of each pixel in the attention block is updated (step S81).

That is, the density estimation means 50 sets a window based on the position of each pixel in the captured image, extracts the feature amount in the window, and reads out the density estimator from the density estimator storage means 40, respectively. The feature amount of the pixel is input to the density estimator, and the estimated density of the pixel is obtained as the output value of the density estimator.

Further, the density estimation means 50 outputs the estimated density of each pixel in the block of interest to the update interval setting means 51. The update interval setting means 51 determines the update interval until the next update of the estimated density for the block of interest based on the input estimated density, and waits for a new analysis in the analysis waiting time storage means 42 for the update interval. Set as time.

Specifically, the update interval setting means 51 confirms whether or not the estimated density input for each pixel in the block of interest includes a value indicating a "background" class or a "low density" class (step). S82). Then, when the attention block includes the "background" class or the "low density" class (when "YES" in step S82), the update interval setting means 51 is stored in the analysis waiting time storage means 42. The analysis waiting time of the attention block is updated to 1 [time] (step S83).

On the other hand, when the attention block does not include the "background" class or the "low density" class (when "NO" in step S82), the update interval setting means 51 has the "medium density" class in the input estimated density. It is confirmed whether or not the value indicating the above is included (step S84). When the attention block includes the "medium density" class (when "YES" in step S84), the update interval setting means 51 sets the analysis waiting time of the attention block stored in the analysis waiting time storage means 42. 2 Update to [Time] (step S85).

Further, when the attention block does not include the "medium density" class (when "NO" in step S84), the update interval setting means 51 assumes that the attention block contains only the "high density" class and waits for analysis. The analysis waiting time of the block of interest stored in the storage means 42 is updated to 4 [time] (step S86).

If the analysis waiting time of the attention block is not 0 (when “NO” in step S80), the processing of steps S81 to S86 for the attention block is omitted.

When the density estimation process S6 for the block of interest is completed in this way, the process proceeds to step S7 of FIG. 3, and the density estimation means 50 confirms whether or not all the blocks have been processed. When there is an unprocessed block (when “NO” in step S7), the density estimation means 50 returns the process to step S5, sets the unprocessed block as the next block of interest, and continues the loop process.

On the other hand, when all the blocks have been processed (when "YES" in step S7), the density estimation means 50 reads the estimated densities of all the pixels from the object distribution storage means 41, converts the estimated densities into colors, and converts the estimated densities into colors. A distribution image is generated by performing transparent synthesis with the captured image, and the generated distribution image is output to the communication unit 3 as information on the object distribution (step S8).

The communication unit 3 to which the distribution image is input operates as the object distribution output means 31, and transmits the distribution image to the display unit 6.

[Processing example]
5 to 9 show how the analysis waiting time stored in the analysis waiting time storage means 42 is updated by the processing over 5 hours from the time t to the time (t + 4), and the density of the block whose update timing has arrived. Is a schematic diagram illustrating a state in which is estimated and a state in which the estimated density stored in the object distribution storage means 41 is updated.

The analysis waiting time information 100, 102, 110, 112, 120, 122, 130, 132, 140, 142 schematically describes the arrangement of blocks in the two-dimensional coordinates corresponding to the captured image and the analysis waiting time of each block. Shown as an image. In the image, each of the plurality of rectangles arranged two-dimensionally in a matrix represents a block, and the numerical value in the rectangle represents the analysis waiting time of the block represented by the rectangle.

Here, the information 100, 110, 120, 130, 140 shown as "analysis waiting time (at the time of image acquisition)" in FIGS. 5 to 9 is specifically subtracted by one time in step S4 shown in FIG. It shows the analysis waiting time after the operation. On the other hand, the information 102, 112, 122, 132, 142 shown as "analysis waiting time (after density estimation)" specifically waits for analysis after the loop processing related to density estimation in steps S5 to S7 shown in FIG. 3 is completed. It represents the time and reflects the update result in the processing (steps S83, S85, S86) in step S6.

The density estimation results 101, 111, 121, 131, 141 shown as the “density estimation result (updated portion)” in FIGS. 5 to 9 show the estimated density newly obtained at each time in two dimensions corresponding to the captured image. It is schematically shown as an image in coordinates. The estimated density is the density updated in the process (step S81) in step S6 shown in FIG. 3, and only the portion of the block group corresponding to the captured image whose density is updated is the density estimation in FIGS. 5 to 9. It is shown as the resulting image.

Specifically, the "density estimation result (updated part)" corresponds to the block in which the analysis waiting time is 0 in the information 100, 110, 120, 130, 140 shown as "analysis waiting time (at the time of image acquisition)". The density estimation results 101, 111, 121, 131, 141 shown as "" are generated.

In the images of the density estimation results in FIGS. 5 to 9, the white areas are pixels in the "background" class, the shaded areas are pixels in the "low density" class, and the horizontal lines are in the "medium density" class. Pixels and shaded areas represent pixels of the "high density" class.

As explained with reference to FIG. 4, a new update interval is set for the update portion of the estimated density for which the density estimation results 101, 111, 121, 131, 141 are generated, and the result is reflected in "waiting for analysis". Information 102, 112, 122, 132, 142 shown as "time (after density estimation)" is generated.

The density estimation results 103, 113, 123, 133, and 143 shown as "density estimation results (synthesis results)" in FIGS. 5 to 9 correspond to the estimated densities stored in the object distribution storage means 41 corresponding to the captured images, respectively. It is schematically shown in a dimensional image. Of the density estimation results 103, 113, 123, 133, and 143, those at arbitrary time T have the estimated density generated one time before the time and stored in the object distribution storage means 41, and the density estimation of the updated portion. The result 101, 111, 121, 131, 141 is generated by synthesizing the one generated at the time T, and is stored in the object distribution storage means 41. For example, the density estimation result 113 at time t + 1 is generated by overwriting the density estimation result 111 of the updated portion at time t + 1 with the density estimation result 103 at time t + 1.

Hereinafter, the processing at each time of time t to t + 4 will be described more specifically.

FIG. 5 relates to processing at time t. The captured image at time t is acquired immediately after the initial activation or the change of the field of view, and therefore, immediately before that, all the blocks are initialized to the analysis waiting time of 1 (step S2 in FIG. 3). Then, when the captured image at time t is acquired, the analysis waiting time of all the blocks is subtracted by 1 (step S4 in FIG. 3), and the analysis waiting time information 100 at the time of image acquisition is all 0. As a result, assuming that the update timing of the estimated density has arrived for all the blocks, the density is estimated for each pixel in all the blocks as shown in the image of the density estimation result 101.

Corresponding to the density estimation result 101, a new analysis waiting time is set according to the estimated density for all blocks, and the analysis waiting time information 102 is generated. Specifically, the analysis waiting time for blocks containing "background" class or "low density" class pixels is 1, and the analysis waiting time for blocks containing other "medium density" class pixels is 2. The blocks other than the above are considered to consist of pixels of only the "high density" class, and the analysis waiting time is updated to 4.

Further, the density estimation result 101 is generated for all blocks, and the estimated density of all blocks in the density estimation result 103 is updated with the information of the density estimation result 101.

FIG. 6 relates to processing at time t + 1. When the captured image is acquired at time t + 1, the analysis waiting time of all blocks is subtracted by 1, and the analysis waiting time information 102 is updated like the analysis waiting time information 110. As a result, the analysis waiting time of 63 blocks out of all 120 blocks becomes 0, and in the density estimation result 111, the density of 63 blocks is estimated assuming that the update timing has arrived.

Then, the analysis waiting time is updated for the 63 blocks whose density is estimated out of the analysis waiting time information 110, and the analysis waiting time information 112 is generated. Specifically, the analysis waiting time of 62 blocks including the pixels of the "background" class or the "low density" class out of the 63 blocks is 1, and the analysis waiting time of the other blocks including the pixels of the "medium density" class is 1. The analysis wait time has been updated to 2. Incidentally, i-th row from the top, to represent the j-th column block and B _ij from the left, one block analysis latency is set to 2 is B _27. In the density estimation result 111, _{a horizontal line area indicating pixels of the "medium density" class is obtained in addition to B 27} , but some blocks in the area are in the "low density" class shown by the shaded area. The analysis waiting time is set to 1 because it includes the pixels of.

Further, the 63 blocks of the density estimation result 103 are updated with the information of the density estimation result 111 to generate the density estimation result 113.

FIG. 7 relates to the processing at time t + 2. When the captured image is acquired at time t + 2, the analysis waiting time of all blocks is subtracted by 1, the analysis waiting time information 112 is updated like the analysis waiting time information 120, and the update timing is changed to 91 blocks out of all 120 blocks. Is coming. As a result, the density estimation result 121 consisting of the 91 blocks is generated as the updated portion of the estimated density, the analysis waiting time is updated for the 91 blocks based on the updated estimated density, and the analysis waiting time information 122 is generated. Will be done. Specifically, the analysis waiting time of 64 blocks including the pixels of the "background" class or the "low density" class among the 91 blocks is 1, and the analysis waiting time of the other 27 blocks including the pixels of the "medium density" class is 1. The analysis wait time has been updated to 2. Further, the 91 blocks of the density estimation result 113 are updated with the information of the density estimation result 121 to generate the density estimation result 123.

The processing at time t + 3 shown in FIG. 8 and the processing at time t + 4 shown in FIG. 9 are also performed in the same manner, and the analysis waiting time updated at the previous time and stored in the analysis waiting time storage means 42 waits for analysis at the time of image acquisition. The estimated density is updated for the block for which the time is generated and the update timing has arrived, and based on the result, the analysis waiting time is updated and the composite result of the estimated density for the entire captured image is generated.

As a result of such processing, the processing cost of density estimation for 144 blocks is reduced in 5 hours. On the other hand, in the density estimation results 103, 113, 123, 133, and 143, changes in the congestion situation are obtained with high accuracy.

[Modification example]
(1) In the above embodiment, an example in which the object to be detected is a human is shown, but the object to be detected is not limited to this, and the object to be detected may be a vehicle, an animal such as a cow or a sheep, or the like.

(2) In the above embodiment and its modifications, the density estimator learned by the multi-class SVM method is illustrated, but instead of the multi-class SVM method, a decision tree type random forest method and multi-class AdaBoost ( Various density estimators such as the density estimator learned by the AdaBoost) method or the multiclass logistic regression method can be used.

Alternatively, it can be a density estimator using an identification type CNN (Convolutional Neural Network).

(3) In the above-described embodiment and each modification thereof, the classes of densities other than the background estimated by the density estimator are set to 3 classes, but the classes may be further divided. In that case, the update interval is set at a finer stage corresponding to the classification.

(4) In the above-described embodiment and each modification thereof, a density estimator classified into multiple classes is illustrated, but instead of this, a regression-type density estimator that returns a density value (estimated density) from a feature amount. It can also be. That is, with a density estimator that has learned the parameters of the regression function for obtaining the estimated density from the features by the ridge regression method, the support vector regression method, the random forest method of the regression tree type, or the Gaussian process regression. can do.

Alternatively, it can be a density estimator using a regression type CNN.

In these cases, the range of the estimated density that is output as a continuous value instead of the value of the density class is set in association with the update interval.

(5) The update interval shown in the above embodiment and each modification thereof is an example, and depends on the shooting conditions such as the range setting of the estimated density, the speed of the detection target, the shooting cycle of the shooting unit 2, and the angle of view. It can be another value suitable for those conditions.

(6) In the above-described embodiment and each modification thereof, the GLCM feature is exemplified as the feature amount learned by the density estimator and the feature amount used for the density estimation, but these are local binary patterns (instead of the GLCM feature). It can be various features such as Local Binary Pattern (LBP) feature, Haar-like feature, HOG (Histograms of Oriented Gradients) feature, brightness pattern, or GLCM feature and among them. It is also possible to use a combination of a plurality of features.

(7) In the above-described embodiment and each modification thereof, an example in which the scanning interval of the density estimation means 50 on the captured image is set to each pixel is shown, but the scanning interval is set to every two or more pixels. It is also possible to open the space.

(8) In the above-described embodiment and each modification thereof, the density estimation means 50 exemplified as the congestion estimation means estimates the distribution of the object by estimating the density of the object as the degree of congestion. The estimation means can also estimate the distribution of the object by analyzing the complexity of the image. For example, the congestion estimation means divides the captured image into regions for each adjacent pixel whose colors are similar to each other, counts the divided regions having overlap with the block for each of the blocks described above, and the higher the count value, the higher the complexity. Ask for. Alternatively, the congestion estimation means performs frequency analysis for each block of captured images, and the higher the average frequency, the higher the complexity. Then, the relationship between the complexity and the degree of congestion is determined through an experiment in advance (the higher the complexity, the higher the degree of congestion), and the degree of congestion corresponding to the obtained complexity is used as an estimated value for each block.

1 Object distribution estimation device, 2 Imaging unit, 3 Communication unit, 4 Storage unit, 5 Image processing unit, 6 Display unit, 30 Image acquisition means, 31 Object distribution output means, 40 Density estimator storage means, 41 Object distribution storage means , 42 Analysis waiting time storage means, 50 Density estimation means, 51 Update interval setting means.

Claims (3)

An image acquisition means for sequentially acquiring captured images taken at predetermined time intervals in a space where a predetermined object can exist, and
A congestion estimation means that analyzes each of a plurality of local regions set in the captured image and estimates the degree of congestion of the object in each of the local regions as distribution information of the object in the space.
For each of the local regions, an update interval setting means for setting the update time interval of the distribution information corresponding to the local region by the congestion estimation means to a length corresponding to the degree of congestion of the local region is provided .
The update interval setting means is an object distribution estimation device that sets a longer update time interval as the local region has a higher degree of congestion estimated by the congestion estimation means. The update interval setting means, for each block, each consisting of two or more of the local region, by setting the update time interval corresponding to the lowest said congestion degree within the block, to claim 1, wherein The described object distribution estimation device. The congestion estimation means expresses the degree of congestion for each local region by using a density estimator that learns the characteristics of each density image obtained by photographing a space in which the object exists at the density at a predetermined density. The object distribution estimation device according to claim 1 or 2 , wherein the density of an object is estimated.

Images