JP7039171B2 - Image processing equipment, object detection method and object detection program - Google Patents

Description

The present invention relates to an image processing apparatus, an object detection method and an object detection program.

Conventionally, a technique for detecting an object from an input image is known (see, for example, Patent Document 1). In the technique described in Patent Document 1, as shown in FIG. 1A, a plurality of windows constituting a predetermined area are extracted from an input image, and the feature amount of a block containing a plurality of pixels in the extracted windows is calculated. Then, by normalizing the feature amount of the block, the object in the input image is detected.

Japanese Patent No. 5201184

However, in the technique described in Patent Document 1, a block feature amount calculation process and a normalization process are performed for each window extracted in the input image. That is, as shown in FIG. 1B, the block feature amount calculation processing and normalization processing performed in another window (solid line) are also performed in the window (broken line) having a portion overlapping with the window, so that the duplication occurs. There is a problem that the processing in the portion is performed again, which in turn increases the processing amount in the object detection processing.

An object of the present invention is to provide an image processing device, an object detection method, and an object detection program capable of reducing the amount of processing in the object detection process.

The image processing apparatus according to the present invention is
A reduced image generator that reduces an image at a rate of less than 1 to generate an input image that includes a person.
A cell feature amount generation unit that generates a cell feature amount of each cell based on a plurality of pixels constituting all the cells arranged in the input image.
A block feature amount generation unit that generates a block feature amount of each block based on the cell feature amount of a plurality of the cells constituting all the blocks arranged so as to be offset by the first predetermined pixel in the input image.
A storage unit that stores each block feature amount generated by the block feature amount generation unit, and a storage unit.
A window feature amount generation unit that generates a window feature amount of each window based on the block feature amount of a plurality of the blocks constituting all the windows arranged so as to be offset by a second predetermined pixel in the input image.
Using the window feature amount generated by the window feature amount generation unit, an object detection unit that detects whether or not the window corresponding to the window feature amount is a region representing a person, and an object detection unit.
Equipped with
The window feature amount generation unit selects a plurality of block feature amounts corresponding to the plurality of blocks included in the window from the storage unit , and generates the selected plurality of block feature amounts as the window feature amount.

The object detection method according to the present invention is
It is an object detection method of an image processing device equipped with a storage unit.
Reduce the image by less than 1 to generate an input image that includes people,
A cell feature amount of each cell is generated based on a plurality of pixels constituting all the cells arranged in the input image.
The block feature amount of each block is generated based on the cell feature amount of the plurality of cells constituting all the blocks arranged so as to be offset by the first predetermined pixel in the input image.
Each generated block feature amount is stored in the storage unit, and is stored.
The window feature amount of each window is generated based on the block feature amount of the plurality of blocks constituting all the windows arranged so as to be offset by the second predetermined pixel in the input image.
Using the generated window feature amount, it is detected whether or not the window corresponding to the window feature amount is an area representing a person.
A plurality of block feature amounts corresponding to the plurality of blocks included in the window are selected from the storage unit, and the selected plurality of block feature amounts are generated as the window feature amount.

The object detection program according to the present invention is
In an object detection program of an image processing device equipped with a storage unit
On the computer
A reduced image generation process that reduces an image at a rate of less than 1 to generate an input image that includes a person.
Cell feature amount processing that generates a cell feature amount of each cell based on a plurality of pixels constituting all the cells arranged in the input image, and
A block feature amount process that generates a block feature amount of each block based on the cell feature amount of a plurality of the cells constituting all the blocks arranged so as to be offset by the first predetermined pixel in the input image.
A storage process for storing each block feature amount generated by the block feature amount process, and a storage process for storing each block feature amount.
A window feature amount generation process that generates a window feature amount of each window based on the block feature amount of a plurality of the blocks constituting all the windows arranged so as to be offset by a second predetermined pixel in the input image.
Using the window feature amount generated by the window feature amount generation process, an object detection process for detecting whether or not the window corresponding to the window feature amount is an area representing a person, and an object detection process.
To execute,
In the window feature amount generation process, a plurality of block feature amounts corresponding to a plurality of blocks included in the window are selected from the storage unit, and the selected plurality of block feature amounts are generated as the window feature amount.

According to the present invention, the amount of processing in the object detection process can be reduced.

It is a figure for demonstrating the object detection operation in the conventional example. It is a figure which shows the image processing apparatus which concerns on this embodiment. It is a figure for demonstrating the cell feature quantity in an input image. It is a figure for demonstrating the block feature amount in an input image. It is a figure for demonstrating a storage part. It is a figure for demonstrating an object detection operation. It is a figure for demonstrating an object detection operation. It is a flowchart which shows an example of the operation example of the object detection control in an image processing apparatus. It is a figure for demonstrating the 3rd predetermined pixel. It is a figure which shows the relationship between the input image size and the speed-up magnification. It is a figure which shows the relationship between the 2nd predetermined pixel and the 3rd predetermined pixel, and the speed-up magnification.

Hereinafter, the present embodiment will be described in detail with reference to the drawings. FIG. 2 is a diagram showing an image processing device 1 according to the present embodiment.

As shown in FIG. 2, the image processing device 1 is a device for performing image processing for detecting an object in a region imaged by, for example, a security camera or the like. The image processing device 1 includes an image acquisition unit 2, a reduced image generation unit 3, a cell feature amount generation unit 4, a block feature amount generation unit 5, a storage unit 6, a window feature amount generation unit 7, and an object detection unit. It is provided with a part 8.

The image acquisition unit 2 acquires an image (for example, a grayscale image of 1280 pixels × 960 pixels) from an image pickup unit (not shown) such as a camera, and outputs the acquired image to the reduced image generation unit 3.

The reduced image generation unit 3 reduces the acquired image at a rate of less than 1 (for example, 0.9) to generate an input image, and outputs the input image to the cell feature amount generation unit 4. When the detection of the object by the object detection unit 8 described later is completed in the input image, the reduced image generation unit 3 reduces the input image again to a ratio of less than 1 to generate an input image, and in this input image. , Object detection is repeated.

Further, when the input image is reduced to a predetermined size (for example, 320 pixels × 240 pixels) by the reduced image generation unit 3, the object detection process by the image processing device 1 is completed.

The cell feature amount generation unit 4 generates a cell feature amount based on the pixels in the cell of the input image. As shown in FIG. 3, the cell C is a region including a plurality of pixels (for example, 3 horizontal pixels × 3 vertical pixels) in the input image I.

The cell feature amount D1 is generated for each cell C, and may be generated using, for example, a histogram of the edge angle as described in Patent Document 1, or the maximum value and the minimum value of RGB in the cell C. It may be generated using any of a value, an average value, and a median value.

As shown in FIG. 2, the block feature amount generation unit 5 generates a block feature amount based on the cell feature amount of the cells included in the block. As shown in FIG. 4, the block B is a region including a plurality of cells C, and is arranged so as to be offset by the first predetermined pixel in the input image I. The first predetermined pixel is set to, for example, three pixels having the greatest common divisor of the second predetermined pixel (for example, 3 pixels) and the third predetermined pixel (9 pixels corresponding to 3 cells in FIG. 4), which will be described later.

The block feature amount D2 is generated for each block B and is processed to normalize the cell feature amount D1. The block feature amount D2 may be normalized so that the sum of squares of the cell feature amount D1 in the block B is 1, for example.

Further, when the cell C has an n-dimensional feature amount, the block feature amount D2 is any one of n maximum values, minimum values, average values, and medians compared with the cells C in each block B. It may be generated by the process of calculating. Further, when the block feature amount D2 has an n-dimensional feature amount, the information of any of n maximum value, minimum value, and median cell C is calculated by comparing the cell C in each block B. It may be generated by the processing to be performed. Note that n is a natural number of 1 or more.

As shown in FIG. 2, the storage unit 6 stores each block feature amount generated by the block feature amount generation unit 5. As shown in FIG. 5, the storage unit 6 changes the location for storing the block feature amount according to the block feature amount.

Since the blocks are arranged so as to be offset by a predetermined pixel, the block feature quantities do not completely match each other. Therefore, by securing a storage location corresponding to each block feature amount, it is possible to identify where each feature amount corresponds. In the example shown in FIG. 5, the block feature amount in the block B in the left corner of the input image I is stored in the first storage area 61 in the storage unit 6, and the block feature amount in the block B adjacent to the first storage area 61 is stored in the storage unit 6. It is stored in the second storage area 62.

As shown in FIG. 2, the window feature amount generation unit 7 generates a window feature amount based on the block feature amount corresponding to the block included in the window. The window is an area including a plurality of blocks (for example, see FIG. 6), and is arranged so as to be offset by a second predetermined pixel (for example, 3 pixels) in the input image.

Further, the blocks in the window are arranged so as to be offset by a third predetermined pixel. In FIG. 6, since the blocks in the window do not overlap each other, the number of pixels corresponds to 3 cells (for example, 18 pixels).

The window feature amount generation unit 7 selects a block feature amount included in each window from the storage unit 6, and generates a window feature amount based on the block feature amount for each window. The window feature is the block feature in all the blocks in the window. The window feature amount generation unit 7 outputs the generated window feature amount to the object detection unit 8.

The object detection unit 8 detects whether or not the window corresponding to the window feature amount is an area representing an object by using the window feature amount generated by the window feature amount generation unit 7 (hereinafter, “object detection process”). ").

As described in Patent Document 1, for example, the object detection unit 8 is based on a learning model from a learning storage unit that stores an SVM (support vector machine) model as a learning model and a window feature amount. Performs object detection processing in the window. Further, the object detection unit 8 may perform the object detection process in the window based on the decision tree, the random forest, and the window feature amount.

After the object detection unit 8 performs the object detection process in the window, the object detection unit 8 performs the object detection process of a window different from the window in the input image I at any time. Specifically, as shown in FIG. 6, the object detection process of the window (broken line) deviated from the window (solid line) on which the object detection process is performed by the second predetermined pixel is performed, and the object detection process of the window is performed. Then, the object detection process of the window deviated from the window by the second predetermined pixel is performed again.

As shown in FIG. 7, when the window (broken line) reaches the right end of the input image, the object detection process of the window (two-dot chain line) deviated from the left end window (solid line) by the second predetermined pixel is performed. go. By repeating such work, the object detection process of the entire input image is performed.

Further, the second predetermined pixel has a width smaller than the short side of the window, and is, for example, 3 pixels in FIG. As a result, since the adjacent windows have overlapping portions, the object detection process of the entire input image can be performed evenly.

However, when the cell feature amount and the block feature amount are generated for each window, the cell feature amount and the block feature amount in the overlapping portions of a plurality of windows are performed a plurality of times, which in turn increases the processing amount in the object detection process. There was a problem that it would end up.

However, in the present embodiment, the window feature amount generation unit 7 selects the block feature amount included in each window from the storage unit 6, and generates a window feature amount based on the block feature amount for each window. By doing so, since the feature amount in the block is not generated again in the windows having overlapping portions, the processing amount for generating the block feature amount for each window can be significantly reduced. can.

Specifically, under the conditions shown below, the difference in the processing amount between the present embodiment and the comparative example is compared. As a condition, as shown in FIG. 6, the input image size is 640 pixels in width × 480 pixels in height, the cell size is 6 pixels in width × 6 pixels in height, the number of cell feature quantities in the cell is 9, and the number of cells in the block is. 9 cells (horizontal 3 cells x vertical 3 cells: vertical and horizontal 18 pixels), the number of blocks in the window is 15 blocks (horizontal 3 blocks x vertical 5 blocks: horizontal 54 pixels x vertical 90 pixels), and the second predetermined pixel is 3 pixels. be. The third predetermined pixel is 18 pixels corresponding to 3 cells.

As a comparative example, a configuration in which a cell feature amount is generated for each window and then a block feature amount is generated is illustrated.

First, the number of times the block feature amount is generated when the object detection process is performed in the comparative example is the total number of windows in the input image I × the number of blocks in the window because the block feature amount is generated for each window. The total number of windows is (int (640 pixels-54 pixels) / 3 + 1) × (int (480 pixels-90 pixels) / 3 + 1) = 196 × 131. Since the number of blocks in the window is 15, the total number of blocks in the comparative example is 15 × 196 × 131. In addition, "3" in the above formula means a second predetermined pixel.

On the other hand, the number of times the block feature amount is generated when the object detection process is performed in the present embodiment is the total number of blocks in the input image I. The total number of blocks is (int (640 pixels-18 pixels) / 3 + 1) × (int (480 pixels-18 pixels) / 3 + 1) = 208 × 155 blocks. In addition, "3" in the above formula means the 1st predetermined pixel which is the greatest common divisor of the 2nd predetermined pixel and the 3rd predetermined pixel.

Here, normalization processing is performed, but in the processing, the coefficient corresponding to the number of cells and the number of cell feature quantities is multiplied by the number of blocks in both addition and integration, so the magnitude of the processing amount is determined by the number of blocks. can do. Comparing the number of blocks between the present embodiment and the comparative example, the processing amount of the present embodiment is (208 × 155) / (15 × 196 × 131) = 0 with respect to the processing amount of the comparative example. It becomes .0837, which can be reduced to about 1/12.

An example of the operation of the object detection control in the image processing device 1 configured as described above will be described. FIG. 8 is a flowchart showing an example of an operation example of object detection control in the image processing device 1. The process shown in FIG. 8 is appropriately executed when the object detection control is performed in the image processing device 1.

As shown in FIG. 8, the image acquisition unit 2 acquires an image from an image pickup unit or the like (step S101). Next, the reduced image generation unit 3 generates a reduced image, that is, an input image based on the image (step S102). Next, the cell feature amount generation unit 4 generates a cell feature amount from the input image (step S103).

Next, the block feature amount generation unit 5 generates a block feature amount from the generated cell feature amount (step S104). Next, the storage unit 6 stores the generated block feature amount (step S105). Next, the window feature amount generation unit 7 extracts the window, selects the block feature amount corresponding to the extracted window from the storage unit 6, and generates the window feature amount (step S106).

Next, the object detection unit 8 performs an object detection process based on the window feature amount (step S107). Next, the image processing device 1 determines whether or not the object detection process in the input image is completed (step S108).

As a result of the determination, when the object detection process is not completed (step S108, NO), the window feature amount generation unit 7 extracts a window shifted laterally to the second predetermined pixel from the window subjected to the object detection process (step). S109). Here, when the window reaches the right end of the input image, for example, the left end window shifted downward by a second predetermined pixel from the window is extracted.

After step S109, the process returns to step S106. Returning to the determination in step S108, when the object detection process is completed (step S108, YES), the image processing device 1 determines whether or not the input image has a predetermined size (for example, 320 pixels × 240 pixels) (for example, 320 pixels × 240 pixels). Step S110).

As a result of the determination, when the input image is not a predetermined size (step S110, NO), the process returns to step S102. On the other hand, when the input image has a predetermined size (step S110, YES), this control ends.

According to the present embodiment configured as described above, since the feature amount in the block is not generated again in the windows having overlapping portions, the process for generating the block feature amount for each window is performed. The amount can be significantly reduced.

In the above embodiment, the window is set so that the blocks in the window do not have overlapping portions. That is, in the above embodiment, the third predetermined pixel, which is the amount of pixel shift of each block in the window, is 18 pixels, but the present invention is not limited to this, and the third predetermined pixel is each in the window. You may set the window to duplicate blocks. That is, the third predetermined pixel may be set to be smaller than the short side of the block.

For example, as shown in FIG. 9, assuming that the third predetermined pixel (pixel deviation amount) is 9 pixels, the number of blocks in the window is 78 blocks (6 horizontal blocks x 13 vertical blocks: 63 horizontal pixels x 126 vertical pixels). It becomes. By stacking blocks in this way, the number of blocks for which object detection processing is performed increases, so that object detection processing can be performed in more detail.

In this case, the total number of windows is (int (640 pixels-63 pixels) / 3 + 1) × (int (480 pixels-126 pixels) / 3 + 1) = 193 × 119. Since the number of blocks in the window is 78 blocks, the total number of blocks in the above-mentioned comparative example is 78 × 193 × 119. In addition, "3" in the above formula means a second predetermined pixel.

Therefore, in the present embodiment, it is the total number of blocks in the input image I, and when the first predetermined pixel does not change, it is 208 × 155, similarly to the number of blocks described above. Comparing the two, the processing amount of the present embodiment is (208 × 155) / (78 × 193 × 119) = 0.01799 with respect to the processing amount of the comparative example, which is reduced to about 1/56. be able to. As a result, the more detailed the object detection process is set, the more the processing amount can be significantly reduced.

When the speed-up magnification indicating the ratio of the reduction amount of the processing amount to the comparative example is calculated for each input image size, the data as shown in FIG. 10 can be obtained. In this data, the cell size is 6 horizontal pixels x 6 vertical pixels, the number of cell features in the cell is 9, the number of cells in the block is 9 cells (3 horizontal cells x 3 vertical cells: 18 pixels in total), and in the window. The number of blocks is 78 blocks (6 horizontal blocks x 13 vertical blocks: 63 horizontal pixels x 126 vertical pixels), and the second predetermined pixel is 3 pixels.

As the input image size increases, the detection range of the object increases, so that the processing amount tends to increase. However, in the present embodiment, it can be confirmed that the larger the input image, the higher the speed-up magnification. Therefore, although the input image size is 960 pixels × 720 pixels and the speed-up magnification is becoming saturated, it can be confirmed that the present invention is effective as the input image size increases.

Further, in the above embodiment, the first predetermined pixel is set to the greatest common divisor of the second predetermined pixel and the third predetermined pixel, but the present invention is not limited to this and is not set to the greatest common divisor. May be. However, from the viewpoint shown below, it is desirable that the first predetermined pixel is set to the greatest common divisor of the second predetermined pixel and the third predetermined pixel.

In order to calculate the block feature amount constituting the window shifted by the second predetermined pixel, the block feature amount arranged by the common divisor of the second predetermined pixel and the third predetermined pixel is required. Therefore, when the first predetermined pixel is set to the greatest common divisor of the second predetermined pixel and the third predetermined pixel, it is possible to calculate by paying attention to the required block feature amount, and the reusability is most improved. , The processing time in the entire device can be significantly reduced.

For example, in the example shown in FIG. 11, the first predetermined pixel is set to the greatest common divisor of the second predetermined pixel and the third predetermined pixel, and the relationship between the second predetermined pixel and the third predetermined pixel and the speed-up magnification is shown. It is a thing. In many cases in FIG. 11, it can be confirmed that the effect of speeding up is obtained.

Further, the higher the reusability of the block feature amount calculated for each first predetermined pixel, the better the effect of speeding up. For example, when the third predetermined pixel is nine pixels and the second predetermined pixel is one pixel and three pixels which are divisors of the third predetermined pixel, the speed-up magnification is about 56. That is, in the case where the first predetermined pixel and the second predetermined pixel match, the block feature amount calculated for each first predetermined pixel is highly reusable, so that the effect of speeding up is obtained. You can check it.

Further, in the above embodiment, the window feature amount is generated using the block feature amount stored by the storage unit 6, but since the block feature amount is large in size (in FIG. 4, nine cell feature amounts are used). ), The storage unit 6 may not be able to store all the data. Therefore, when the size of the block feature amount generated by the block feature amount generation unit 5 is larger than the predetermined size, the storage unit 6 stores the cell feature amount.

Then, the block feature amount generation unit 5 selects the cell feature amount stored in the storage unit 6 and generates the block feature amount based on the selected cell feature amount. When the storage unit 6 stores the cell feature amount, the window feature amount generation unit 7 generates the window feature amount by using the block feature amount generated by the block feature amount generation unit 5.

By doing so, even if the block feature amount exceeds the size of the storage unit 6, the processing amount is larger than that when the block feature amount is stored in the storage unit 6, but the object detection process is performed. Can be done.

Further, when the size of the block feature amount is not more than a predetermined size, the storage unit 6 performs the same processing as that of the above embodiment.

In addition, the above embodiments are merely examples of embodiment of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its gist or its main features.

1 Image processing device 2 Image acquisition unit 3 Reduced image generation unit 4 Cell feature amount generation unit 5 Block feature amount generation unit 6 Storage unit 7 Window feature amount generation unit 8 Object detection unit

Claims (8)

A reduced image generator that reduces an image at a rate of less than 1 to generate an input image that includes a person.
A cell feature amount generation unit that generates a cell feature amount of each cell based on a plurality of pixels constituting all the cells arranged in the input image.
A block feature amount generation unit that generates a block feature amount of each block based on the cell feature amount of a plurality of the cells constituting all the blocks arranged so as to be offset by the first predetermined pixel in the input image.
A storage unit that stores each block feature amount generated by the block feature amount generation unit, and a storage unit.
A window feature amount generation unit that generates a window feature amount of each window based on the block feature amount of a plurality of the blocks constituting all the windows arranged so as to be offset by a second predetermined pixel in the input image.
Using the window feature amount generated by the window feature amount generation unit, an object detection unit that detects whether or not the window corresponding to the window feature amount is a region representing a person, and an object detection unit.
Equipped with
The window feature amount generation unit selects a plurality of block feature amounts corresponding to the plurality of blocks included in the window from the storage unit, and generates the selected plurality of block feature amounts as the window feature amount.
Image processing device. The blocks in the window are arranged so as to be offset by a third predetermined pixel.
The first predetermined pixel is set to the greatest common divisor of the second predetermined pixel and the third predetermined pixel.
The image processing apparatus according to claim 1. When the size of the block feature amount generated by the block feature amount generation unit is larger than a predetermined size, the storage unit stores the cell feature amount.
The block feature amount generation unit selects a cell feature amount corresponding to a cell included in the block from the storage unit, and generates the block feature amount based on the selected cell feature amount.
The window feature amount generation unit generates the window feature amount using the block feature amount generated by the block feature amount generation unit.
The image processing apparatus according to claim 1 or 2. The block feature amount generation unit generates the block feature amount by performing a process of normalizing the cell feature amount generated by the cell feature amount generation unit.
The image processing apparatus according to any one of claims 1 to 3. The second predetermined pixel is set smaller than the short side of the window.
The image processing apparatus according to any one of claims 1 to 4. The blocks in the window are arranged so as to be offset by a third predetermined pixel.
The third predetermined pixel is set smaller than the short side of the block.
The image processing apparatus according to any one of claims 1 to 5. It is an object detection method of an image processing device equipped with a storage unit.
Reduce the image by less than 1 to generate an input image that includes people,
A cell feature amount of each cell is generated based on a plurality of pixels constituting all the cells arranged in the input image.
The block feature amount of each block is generated based on the cell feature amount of the plurality of cells constituting all the blocks arranged so as to be offset by the first predetermined pixel in the input image.
Each generated block feature amount is stored in the storage unit, and is stored.
The window feature amount of each window is generated based on the block feature amount of the plurality of blocks constituting all the windows arranged so as to be offset by the second predetermined pixel in the input image.
Using the generated window feature amount, it is detected whether or not the window corresponding to the window feature amount is an area representing a person.
An object detection method in which a plurality of block feature amounts corresponding to a plurality of blocks included in the window are selected from the storage unit, and the selected plurality of block feature amounts are generated as the window feature amount. In an object detection program of an image processing device equipped with a storage unit
On the computer
A reduced image generation process that reduces an image at a rate of less than 1 to generate an input image that includes a person.
Cell feature amount processing that generates a cell feature amount of each cell based on a plurality of pixels constituting all the cells arranged in the input image, and
A block feature amount process that generates a block feature amount of each block based on the cell feature amount of a plurality of the cells constituting all the blocks arranged so as to be offset by the first predetermined pixel in the input image.
A storage process for storing each block feature amount generated by the block feature amount process, and a storage process for storing each block feature amount.
A window feature amount generation process that generates a window feature amount of each window based on the block feature amount of a plurality of the blocks constituting all the windows arranged so as to be offset by a second predetermined pixel in the input image.
Using the window feature amount generated by the window feature amount generation process, an object detection process for detecting whether or not the window corresponding to the window feature amount is an area representing a person, and an object detection process.
To execute,
In the window feature amount generation process, a plurality of block feature amounts corresponding to a plurality of blocks included in the window are selected from the storage unit, and the selected plurality of block feature amounts are generated as the window feature amount.
Object detection program.

Images