JP7069840B2 - Individual counting device, individual counting method, individual counting program, and individual counting system - Google Patents

Description

The present invention relates to an individual counting device for counting moving individuals, an individual counting method, an individual counting program, and an individual counting system.

In the fishing or aquaculture industry, it is difficult to accurately grasp the number of individuals of a large amount of fishery resources by the conventional visual counting of the number of individuals of fishery resources. Therefore, a proposal has been made to accurately grasp the quantity of individuals of a large amount of fishery resources. See, for example, Patent Document 1 and Patent Document 2.

The system for counting the number of individuals of moving objects moving in water or on the surface of water disclosed in Patent Document 1 first converts continuously captured images into binary images (images in which pixels are represented by 1 or 0). In a binary image, a group of one pixel whose distance from each other is within a certain distance is extracted as a concatenated area. Here, the articulated region is a region corresponding to a moving body included in the image. Subsequently, the system disclosed in Patent Document 1 compares the concatenated region group of the latest extracted image with the concatenated region group of the image captured before that, and uses the comparison result to determine the moving body included in the image. Extract. Further, in the system disclosed in Patent Document 1, when moving bodies overlap, the number of individual moving bodies is counted based on the overlap and separation of the connected region group included in each of the continuously captured images. It is stated that it should be done. As described above, according to the system disclosed in Patent Document 1, the number of individuals of fish and other moving bodies running up the river can be automatically counted.

The fish counting device disclosed in Patent Document 2 first selects the m-1st image from the continuously captured images, and then captures the image based on the position of the fish detected from the m-1st image. Estimate the position of the fish in the mth image. Subsequently, in the fish counting device disclosed in Patent Document 2, when the estimated position and the position of the fish detected from the mth image match, the fish detected in the m-1st sheet and the mth sheet is detected. Judge that they are the same. As described above, according to the fish counting device disclosed in Patent Document 2, by tracking the position of the fish in the image, it is possible to prevent the same fish from being counted twice.

Japanese Unexamined Patent Publication No. 2005-309485. Japanese Unexamined Patent Publication No. 2016-165238

However, in the system disclosed in Patent Document 1, the overlap and separation of the connected region group included in each of the continuously captured images is collated with a predetermined rule indicating the overlap and separation set in advance, and based on the collation result. I'm counting. Therefore, if it deviates from the predetermined rule, the number of moving individuals cannot be accurately counted. For example, if the connected region groups continue to overlap, the number of moving objects cannot be accurately counted because they deviate from the predetermined rule.

The fish counting device disclosed in Patent Document 2 cannot accurately count the number of moving fish because the position of the fish cannot be tracked from the image when the fish overlap in the captured image.

An example of an object of the present invention is an individual counting device, an individual counting method, an individual counting program, and an individual counting system that can accurately count the number of individuals even when moving individuals are imaged in an overlapping manner by solving the above problems. To provide.

In order to achieve the above object, the individual counting device in one aspect of the present invention is
An estimation unit that estimates an estimated individual region in which the individual is estimated to move in a second image captured after capturing the first image, based on an individual region indicating an individual detected from the first image. When,
When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the determination unit that determines that the individual area of the first image and the second image is the same. ,
When the individual region includes a preset characteristic region indicating the characteristics of the individual, a counting unit that counts the quantity of the characteristic region as the quantity of the individual corresponding to the individual region.
The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted as the time. With the integration unit, which integrates each time
It is characterized by having.

Further, in order to achieve the above object, the individual counting method in one aspect of the present invention is:
(A) Based on the individual region indicating the individual detected from the first image, the estimated individual region in which the individual is estimated to move is estimated in the second image captured after the first image is captured. , Steps and
(B) When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the individual areas of the first image and the second image are determined to be the same. Steps and
(C) When a preset characteristic region indicating the characteristics of the individual is included in the individual region, the quantity of the characteristic region is counted as the quantity of the individual corresponding to the individual region, and (d) the above. The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted as the time. Steps and steps that are accumulated each time
It is characterized by having.

Further, in order to achieve the above object, the individual counting program in one aspect of the present invention is provided.
On the computer
(A) Based on the individual region indicating the individual detected from the first image, the estimated individual region in which the individual is estimated to move is estimated in the second image captured after the first image is captured. , Steps and
(B) When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the individual areas of the first image and the second image are determined to be the same. Steps and
(C) When a preset characteristic region indicating the characteristics of the individual is included in the individual region, the quantity of the characteristic region is counted as the quantity of the individual corresponding to the individual region, and (d) the above. The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted as the time. Steps and steps that are accumulated each time
It is characterized by having.

Further, in order to achieve the above object, the individual counting system in one aspect of the present invention is:
An imaging unit that captures a path for moving the individual, which is at least set in width according to the size of the individual.
Estimating the estimated individual area in which the individual is estimated to move in the second image taken after the first image is taken, based on the individual area indicating the individual detected from the first image. Department and
When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the determination unit that determines that the individual area of the first image and the second image is the same. ,
When the individual region includes a preset characteristic region indicating the characteristics of the individual, a counting unit that counts the quantity of the characteristic region as the quantity of the individual corresponding to the individual region.
The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted as the time. With the integration unit, which integrates each time
It is characterized by having.

As described above, according to the present invention, the number of individuals can be accurately counted even when moving individuals are imaged in an overlapping manner.

FIG. 1 is a diagram showing an example of an individual counting device. FIG. 2 is a diagram showing an example of a system having an individual counting device. FIG. 3 is a diagram showing an example of an image of an individual moving along a path. FIG. 4 is a diagram showing an example of an image of an individual moving along a path. FIG. 5 is a diagram for explaining the same determination of the estimated individual area. FIG. 6 is a diagram showing an example of the operation of the individual counting device. FIG. 7 is a diagram showing an example of detailed operation of the individual counting device. FIG. 8 is a diagram for explaining the operation of the individual counting device. FIG. 9 is a diagram showing an example of a computer that realizes an individual counting device.

(Embodiment)
Hereinafter, the individual counting device, the individual counting method, the individual counting program, and the individual counting system according to the embodiment of the present invention will be described with reference to FIGS. 1 to 9.

[Device configuration]
First, the configuration of the individual counting device in the present embodiment will be described with reference to FIG. 1. FIG. 1 is a diagram showing an example of an individual counting device. The individual counting device 1 in the present embodiment shown in FIG. 1 is a device for accurately counting the number of one or more individuals moving along the route from the moving source to the moving destination. The individual counting device 1 has an estimation unit 2, a determination unit 3, a counting unit 4, and an integration unit 5.

The estimation unit 2 estimates an estimated individual region in which an individual is estimated to move in a second image captured after capturing the first image, based on an individual region indicating an individual detected from the first image. .. When the overlap between the estimated individual area and the individual area detected from the second image is equal to or greater than the predetermined area value, the determination unit 3 determines that the individual areas of the first image and the second image are the same. When the individual region includes a characteristic region showing the characteristics of a preset individual, the counting unit 4 counts the quantity of the characteristic region as the quantity of the individual corresponding to the individual region. The integration unit 5 counts the number of individuals corresponding to the individual area for each time from the time when the individual area is detected to the time when the individual area is no longer detected, and the time is the number of individuals corresponding to the counted individual area. Accumulate each time it elapses.

As described above, in the present embodiment, the individual counting device 1 tracks the individual area using the captured image, and the individual is based on the characteristics of the individual at each time when the individual area is continuously present in the image. Count the number of individuals corresponding to the area. Then, the individual counting device 1 integrates the number of individuals corresponding to the individual area counted for each time as time passes. By doing so, the number of individuals can be accurately counted even when the individuals moving along the route overlap.

[System configuration]
Subsequently, with reference to FIG. 2, a system having the individual counting device 1 and the individual counting device 1 will be specifically described. FIG. 2 is a diagram showing an example of a system having an individual counting device. The individual counting system 20 includes an individual counting device 1 and an imaging unit 21. In the present embodiment, the imaging unit 21 captures an image of a path 25 for moving the individual 22, whose width is set at least according to the size of the individual 22. Further, the image pickup unit 21 is, for example, a high-speed shutter camera or the like. This will be described in detail. The imaging unit 21 continuously images the entire path 25 or a specific location for the individual 22 to move from the source 23 to the destination 24. Further, for example, when the image pickup unit 21 uses an individual 22 as a fish, a moving source 23 as a moving source cage, a moving destination 24 as a moving destination cage, and a path 25 as a transparent pipe connecting the cages. The path 25 and the fish are continuously imaged. The shutter speed of the imaging unit 21 is set based on at least one or both of the flow velocity of the liquid flowing through the path 25 and the speed at which the individual 22 moves along the path 25.

The case where the individual 22 moves along the path 25 will be described as an example of the individual counting device 1. 3 and 4 are diagrams showing an example of an image of an individual moving along the route 25. Specifically, in the images shown in FIGS. 3 and 4 (a plurality of images including images 1 to 6), the individual region 31 which is an image corresponding to the individual 22 is an image corresponding to the route 25, and the route image 32. It shows how to move. The plurality of images shown in FIG. 3 are images taken before the plurality of images shown in FIG. 4.

Image 1 of FIG. 3 shows a state in which the individual region 31 (31a, 31b) does not exist in the route image 32 at the time point Twait. Image 2 of FIG. 3 shows a state in which the entire two individual regions 31a and 31b are first imaged by the path image 32 at the time point Tstart. Images 3 to 4 of FIG. 3 show a state in which two individual regions 31a and 31b have moved from the time point Tm after the time point Tstart to the time point Tm + a. Image 5 of FIG. 3 shows a state in which the entire two individual regions 31a and 31b were last present in the route image 32 by the time point Tm + b after the time point Tm + a. Image 6 of FIG. 3 shows a state in which the entire two individual regions 31a and 31b are not captured by the path image 32 at the time point Tend after the time point Tm + b. That is, it shows a state of shifting to the time point Twait shown in image 1 of FIG.

Image 1 of FIG. 4 shows a state in which the individual region 31 does not exist in the route image 32 at the time point Twait. In image 2 of FIG. 4, at the time point Tstart, the entire two individual regions 31c and 31d (individual region 31c corresponding to one individual and individual region 31d in which two individuals overlap) are first imaged in the path image 32. Shows the state. Images 3 to 4 of FIG. 4 show a state in which two individual regions 31c and 31d have moved from the time point Tn after the time point Tstart to the time point Tn + a. Image 5 of FIG. 4 shows a state in which the entire two individual regions 31c and 31d were last present in the path image 32 by the time point Tn + b after the time point Tn + a. Image 6 of FIG. 4 shows a state in which the entire two individual regions 31c and 31d are not captured by the path image 32 at the time point Tend after the time point Tn + b.

The estimation unit 2 will be described later, in which the individual region 31 is estimated to move in the second image captured after the first image is captured, based on the individual region 31 showing the individual 22 detected from the first image. Estimate the estimated individual area. This will be described in detail. The estimation unit 2 binarizes the first image in which the individual 22 and the path 25 shown in FIGS. 3 and 4 are captured, executes a labeling process on the binarized image, and performs the individual region 31 and the individual region 31. And detect the quantity of. The first image is, for example, one of a plurality of images captured from the time point Tstart shown in FIGS. 3 and 4 to the time point before the time point Tend.

Subsequently, the estimation unit 2 estimates that the individual region 31 captured by the first image moves in the second image in which the individual 22 and the path 25 are captured, which is captured after the first image is captured. The position of the solid region 31 to be formed is estimated. The second image is, for example, an image captured immediately after the first image is captured, or an image captured after a predetermined time has elapsed from the time when the first image was captured. The predetermined time is, for example, a time set based on the shutter speed of the image pickup unit 21.

The estimation of the estimated individual area is based on, for example, the flow velocity of the liquid flowing through the path 25 and the speed at which the individual 22 moves along the path 25. Estimate the estimated individual area that is estimated to move before being imaged. For the estimation of the estimated individual region, for example, an algorithm such as a Kalman filter, a particle filter, a KLT (Kanade Lucas Tomasi), or a tracker may be used.

When the overlap between the estimated individual area and the individual area 31 detected from the second image is equal to or greater than the predetermined area value, the determination unit 3 determines that the individual area 31 of the first image and the second image are the same. This will be described in detail. The determination unit 3 binarizes the second image in which the individual 22 and the path 25 shown in FIGS. 3 and 4 are captured, executes a labeling process on the binarized image, and performs the individual area 31 and the individual area 31. And detect the quantity of. The estimation unit 2 may detect the individual region 31 and the quantity of the individual region 31 in the second image.

FIG. 5 is a diagram for explaining the same determination of the estimated individual area. The first image and the second image shown in FIG. 5 are enlarged views of the individual region 31 and its vicinity. In the determination unit 3, the estimated individual area 51 corresponding to the individual area 31 of the first image estimated by the estimation unit 2 and the individual area 31 in the second image to which the individual area 31 of the first image has moved overlap. The area of the portion 52 is obtained. The area may be represented by, for example, the number of pixels.

Subsequently, when the area of the portion 52 where the estimated individual area 51 and the individual area 31 in the second image overlap is equal to or larger than the predetermined area (predetermined area value), the determination unit 3 determines the first image and the second image. It is determined that the individual area 31 with the image is the same. That is, the individual region 31 of the second image shown in FIG. 5 is determined to have moved the individual region 31 of the first image. The predetermined region value is determined by, for example, an experiment or a simulation.

As shown in the images 3 and 4 of FIG. 3 and the images 2 to 5 of FIG. 4, when a plurality of individuals 22 are superimposed and imaged, the plurality of individuals 22 are superimposed and imaged on the first image. The individual region 31 is regarded as one region, and the estimated individual region in which the individual region 31 is estimated to move is estimated in the second image.

As described above, in the present embodiment, by using the estimation unit 2 and the determination unit 3, as shown in FIGS. 3 and 4, the individual 22 moving along the route 25 from the time point Tstart to the time point Tend is tracked. can do.

The counting unit 4 counts the individual area 31 on the image by combining one, all, or two of the counting processes shown in (1), (2), and (3). Further, the counting process of (4) may be used instead of (3).
(1) Counting process using the characteristic area showing the characteristics of the individual area 22 (2) Counting process using the ratio of the value indicating the size of the individual area 31 to the reference value (3) the first image and the second Counting process using the ratio of values indicating the size of the individual area 31 of the image (4) The counting result of the individual area 31 in the first image and the size of different individual areas 31 existing in the second image are shown. Counting process using ratio of values

The counting process of (1) will be described. When the individual region 31 includes a characteristic region showing the characteristics of the individual 22 set in advance, the counting unit 4 counts the quantity of the characteristic region as the quantity of the individual 22 corresponding to the individual region 31. This will be described in detail. For example, a case where the individual 22 is a fish and the tail fin or the head of the fish is set as a characteristic region will be described. The counting unit 4 collates the individual region 31 corresponding to the fish with the characteristic region corresponding to the tail fin, and counts the quantity of the characteristic region included in the individual region 31. The collation between the individual region 31 and the characteristic region is, for example, a pattern matching process executed for the individual region 31 using the characteristic region showing the average shape of the tail fin of the target fish as a reference image.

In image 2 of FIG. 3, each of the individual regions 31a and 31b has a region corresponding to the tail fin, and these regions coincide with the characteristic region. Therefore, in image 2, there are two characteristic regions, so that the number of fish is two. It becomes. The same can be said for image 5. Further, in the image 3 of FIG. 3, although the individual regions 31a and 31b overlap each other, there are regions corresponding to the tail fins, and these regions coincide with the characteristic regions, so that the image 3 also includes two characteristic regions. Therefore, the number of fish is two. The same can be said for image 4.

Further, in the image 2 of FIG. 4, there are regions corresponding to the tail fins in each of the individual regions 31c and 31d, and these regions coincide with the characteristic regions. Therefore, in the image 2, there are three characteristic regions, so that the number of fish is large. There will be three. The same can be said for image 5. Further, in the image 3 of FIG. 4, although the individual regions 31c and 31d overlap each other, there are regions corresponding to the tail fins, and these regions coincide with the characteristic regions, so that the image 3 also includes three characteristic regions. Therefore, the number of fish is three. The same can be said for image 4.

In the above-mentioned example, the characteristic region is the tail fin, but the characteristic region may be the head of a fish. That is, it is desirable that the characteristic region be a characteristic portion of the individual 22. Further, the shape of the entire individual 22 may be used as a characteristic region. Further, the shape of the individual region 31 when a plurality of individuals 22 are overlapped may be used as the characteristic region. Furthermore, images with and without a plurality of feature regions acquired in advance are used as sample data, feature quantities in the sample data are evaluated, and counting processing is performed using a learning model generated by machine learning. May be good.

The counting process of (2) will be described. The counting unit 4 counts the number of individuals 22 corresponding to the individual area 31 based on the ratio of the value indicating the size of the individual area 31 to the preset value indicating the size of the individual 22. This will be described in detail. The counting unit 4 calculates the area of each individual region 31 on the image. The area may be, for example, the number of pixels forming the individual region 31.

Subsequently, the counting unit 4 calculates the area ratio between the calculated area of the individual area 31 and the area (reference area) of the individual 22 which is a preset reference value. The reference area is, for example, the area of an average individual 22. Then, the counting unit 4 determines the quantity of the individual 22 corresponding to the individual area 31 according to the calculated area ratio and the preset reference area ratio.

For example, when the area ratio is considered to be the same as the reference area ratio (for example, when the area ratio ≈ the reference area ratio ± a predetermined value), the number of individuals 22 corresponding to the individual area 31 on the image is one. In addition, when the area ratio is larger than the reference area ratio and can be regarded as double (for example, when the reference area ratio <area ratio ≤ 2 / reference area ratio ± predetermined value), it corresponds to the individual area 31 on the image. The number of individuals 22 is two. Furthermore, when the area ratio is considered to be larger than twice and three times the reference area ratio (for example, when the reference area ratio <area ratio ≤ 3 / reference area ratio ± predetermined value), the individual on the image. The number of individuals 22 corresponding to the area 31 is three. The reference area ratio and the predetermined value may be determined by, for example, an experiment or a simulation.

The case where the individual 22 is a fish will be described. In the image 2 of FIG. 3, the counting unit 4 calculates the areas Si1 and Si2 of the individual regions 31a and 31b, respectively, and calculates the area ratios Ar1 and Ar2 with respect to the individual regions 31a and 31b using the reference area St. The area ratio Ar1 is St / Si1 (or Si1 / St), and the area ratio Ar2 is St / Si2 (or Si2 / St). When the area ratios Ar1 and Ar2 are considered to be the same as the reference area ratio Arth (or 1 / Arth), the counting unit 4 sets the number of individuals 22 corresponding to the individual areas 31a and 31b to one. For the individual regions 31a and 31b of the image 5 of FIG. 3, the individual regions 31c of the image 2 and the image 5 of FIG. 4, the individual regions 31 may be counted using the area ratio.

Further, in the image 2 of FIG. 4, the counting unit 4 calculates the areas Si1 and Si2 of the individual area 31c and the individual area 31d (individual area imaged by overlapping the individual 22), respectively, and uses the reference area St. The area ratios Ar1 and Ar2 with respect to the individual regions 31c and 31d are calculated. The area ratio Ar1 is St / Si1 (or Si1 / St), and the area ratio Ar2 is St / Si2 (or Si2 / St). When the area ratio Ar1 is considered to be the same as the reference area ratio Arth (or 1 / Arth), the counting unit 4 sets the number of individuals 22 corresponding to the individual area 31c to one. When the area ratio Ar2 is larger than the reference area ratio Art and can be regarded as twice, the number of individuals 22 corresponding to the individual region 31c is set to two.

The counting process of (3) will be described. The counting unit 4 is an individual 22 corresponding to the individual region 31 based on the ratio of the value indicating the size of the individual region 31 in the first image and the value indicating the size of the individual region 31 in the second image. Count the quantity of. This will be described in detail. The counting unit 4 calculates the area ratio between the area of the individual area 31 in the first image and the area of the individual area 31 in the second image. Then, the counting unit 4 determines the quantity of the individual region 31 according to the quantity of the individual 22 corresponding to the individual region 31 in the first image and the calculated area ratio.

The image 4 of FIG. 3 will be referred to as a first image, and the image 5 of FIG. 3 will be described as a second image. The counting unit 4 calculates the area Si3 of the individual region 31 (formed from 31a and 31b) imaged by overlapping the two individuals 22 in the image 4. Further, the counting unit 4 calculates the area Si4 of the individual region 31a and the area Si5 of the individual region 31b in which the two individuals 22 are separately imaged in the image 5. Subsequently, the counting unit 4 sets the area ratio of the area Si3 of the individual area 31 calculated in the image 4 to the area Si4 of the individual area 31a calculated in the image 5 and the area Si3 of the individual area 31 calculated in the image 4. , The area ratio of the individual region 31b calculated in the image 5 to the area Si5 is calculated.

Subsequently, the counting unit 4 determines the quantity of the individual region 31 according to the quantity of the individual 22 corresponding to the individual region 31 in the image 4 of FIG. 3 and the calculated area ratio. For example, if the number of individuals 22 corresponding to the individual region 31 of the image 4 is two and the area ratio Ar3 (Si3 / Si4) between the areas Si3 and the area Si4 is considered to be double, the individual region 31a in the image 5 The number of individuals 22 corresponding to is one. Similarly, if the number of individuals 22 corresponding to the individual region 31 of the image 4 is two and the area ratio Ar4 (Si3 / Si5) between the areas Si3 and the area Si5 is considered to be double, the individual region in the image 5 The quantity of the individual 22 corresponding to 31a is one. The range that can be regarded as double is double ± predetermined value. The predetermined value is determined by, for example, an experiment or a simulation. When the area ratio Ar3 between the area Si3 and the area Si4 is Si4 / Si3, or when the area ratio Ar4 between the area Si3 and the area Si4 is Si5 / Si3, it can be regarded as 1/2 times. The number of individuals 22 corresponding to the individual areas 31a and 31b in the image 5 is one. The range that can be regarded as 1/2 times is 1/2 times ± predetermined value.

Further, the image 4 of FIG. 4 will be referred to as a first image, and the image 5 of FIG. 4 will be referred to as a second image.
The counting unit 4 calculates the area Si3 of the individual region 31 (formed from 31c and 31d) imaged by overlapping the three individuals 22 in the image 4. Further, the counting unit 4 calculates the area Si4 of the individual region 31c and the area Si5 of the individual region 31d, which are captured by separating the three individuals 22 in the image 5. Subsequently, the counting unit 4 sets the area ratio of the area Si3 of the individual area 31 calculated in the image 4 to the area Si4 of the individual area 31c calculated in the image 5 and the area Si3 of the individual area 31 calculated in the image 4. , The area ratio with the area Si5 calculated in the image 5 is calculated.

Subsequently, the counting unit 4 determines the quantity of the individual region 31 according to the quantity of the individual 22 corresponding to the individual region 31 in the image 4 of FIG. 4 and the calculated area ratio. For example, if the number of individuals 22 corresponding to the individual region 31 of the image 4 is three and the area ratio Ar3 (Si3 / Si4) of the area Si3 to the area Si4 is considered to be three times, the individual region in the image 5 The number of individuals 22 corresponding to 31c is one. Similarly, if the area ratio Ar4 (Si3 / Si5) of the area Si3 and the area Si5 is considered to be 1.5 times, the number of individuals 22 corresponding to the individual region 31d in the image 5 is two. When the area ratio Ar3 of the area Si3 and the area Si4 is Si4 / Si3, if it is 1/3 times or more, the quantity of the solid region 31c in the image 5 is one. Further, when the area ratio Ar4 of the area Si3 and the area Si5 is Si5 / Si3, if it is 2/3 times or more, the number of individuals 22 corresponding to the individual region 31d in the image 5 is two.

The counting process of (4) will be described. The counting unit 4 in the second image is based on the ratio of the count of the individual area 31 already calculated in the first image to the value indicating the size of the different individual area 31 existing in the second image. The quantity of the individual 22 corresponding to the individual area 31 is counted. This will be described in detail. The counting unit 4 acquires the quantity of the individual 22 corresponding to the individual region 31 in the first image, which has already been calculated by, for example, the counting processing of (1) to (4). Subsequently, the counting unit 4 calculates the area ratio of the different individual regions 31 existing in the second image. Then, the counting unit 4 determines the quantity of the individual region 31 in the second image according to the quantity of the individual 22 corresponding to the individual region 31 in the first image and the calculated area ratio.

The image 4 of FIG. 4 will be referred to as a first image, and the image 5 of FIG. 4 will be described as a second image. When the quantity of the individual regions 31c and 31d has already been calculated in the image 4, the counting unit 4 acquires 3 (three animals) as the quantity of the individual regions 31c and 31d. Subsequently, the counting unit 4 calculates the area ratio of the individual regions 31c and 31d in the image 5. That is, the counting unit 4 calculates the area ratio Ar6 (Si4: Si5 = 1: 2) of the area Si4 of the solid region 31c and the area Si5 of the solid region 31d. In the image 5, the area ratio Ar6 is assumed to be 1: 2.

Subsequently, the counting unit 4 calculates the quantity for each of the individual regions 31c and 31d. The quantity of the individual area 31c is calculated as 3 (3 animals) × 1/3 = 1 (1 animal), and the quantity of the individual area 31d is 3 (3 animals) × 2/3 = 2 (2 animals). Calculate as follows.

Furthermore, by using the counting process of (1) or (1) (2), or (1) (3), or (1) (2) (3), the accuracy of counting the quantity of the individual region 31 is high. improves. If the result of the counting process of (1) and the result of the counting process of (2) or (3) are different, the result of the counting process of (1), (2) and (3) is used and the individual is decided by majority vote. The quantity of the individual 22 corresponding to the region 31 is determined. Further, when the results of the counting processes of (1), (2) and (3) are all different, the result of the counting process of (1) is preferentially used. Further, the counting process of (4) may be used instead of (3).

The integrating unit 5 counts the quantity of the individual region 31 for each time from the time Tstart when the individual region 31 is detected to the time Tend when the individual region 31 is no longer detected, and the counted quantity of the individual region 31 elapses. Accumulate for each. This will be described in detail. As shown in FIG. 3, the integrating unit 5 takes the time from the time point Tstart when the individual areas 31a and 31b are detected in the image 2 to the time point Tend when the individual areas 31a and 31b are not detected in the image 6 in the individual area 31a. , 31b count the quantity of the individual 22. In the example of FIG. 3, the quantity of the individual 22 corresponding to the individual regions 31a and 31b in the time from the time point Tstart to the time point Tend is 2.

Subsequently, as shown in FIG. 4, the integrating unit 5 takes a time from Tstart when the individual regions 31c and 31d are detected in the image 2 to Tend when the individual regions 31c and 31d are no longer detected in the image 6. The quantity of the individual 22 corresponding to the individual regions 31c and 31d is counted. In the example of FIG. 4, the quantity of the individual 22 corresponding to the individual regions 31c and 31d in the time from the time point Tstart to the time point Tend is 3.

Subsequently, the integration unit 5 integrates the quantity of the individual 22 corresponding to the individual region 31 counted for each time from the time point Tstart to the time point Tend for each time point Tend after the lapse of time, from the movement source 23. The number of individuals 22 moving to the destination 24 through the route 25 is counted. In the examples of FIGS. 3 and 4, the number of individuals 22 at the time point Tend in image 6 of FIG. 3 is 2, and the number of individuals 22 at the time point Tend in image 6 of FIG. 4 is 3, so the integrated value is 5. Become.

[Device operation]
The individual counting method in this embodiment will be described. The individual counting method is carried out by operating the individual counting device 1 in the present embodiment shown in FIGS. 1 and 2. Therefore, in the description of the individual counting method in the present embodiment, the operation of the individual counting device 1 will be described with reference to FIGS. 1 to 5 as appropriate.

First, the overall operation of the individual counting device 1 will be described with reference to FIG. FIG. 6 is a diagram showing an example of the operation of the individual counting device 1.

As shown in FIG. 6, the individual counting device 1 estimates the estimated individual area 51 based on the individual area 31 in the first image (step A1). Subsequently, the individual counting device 1 sets the individual regions of the first image and the second image to be the same when the overlap between the estimated individual region 51 and the individual region 31 in the second image is equal to or greater than the predetermined region value. Determination (step A2). Subsequently, when the individual region 31 includes the preset characteristic region, the individual counting device 1 counts the quantity of the characteristic region as the quantity of the individual 22 corresponding to the individual region 31 (step A3). Subsequently, the individual counting device 1 integrates the quantity of the individual 22 corresponding to the individual area 31 counted from the time when the individual area 31 is detected to the time when the individual area 31 is not detected (step A4). ).

Subsequently, the processes (steps A1 to A4) in the estimation unit 2, the determination unit 3, the counting unit 4, and the integration unit 5 shown in FIGS. 1 and 2 will be described in detail with reference to FIGS. 7 and 8. FIG. 7 is a diagram showing an example of detailed operation of the individual counting device. FIG. 8 is a diagram for explaining the operation of the individual counting device. Note that Table 81 in FIG. 8 corresponds to FIG. 3, and Table 82 in FIG. 8 corresponds to FIG.

The individual counting device 1 acquires the first image from the image pickup unit 21 (step A11). Subsequently, the individual counting device 1 executes a process of detecting the individual region 31 from the first image (step A12). After that, when the individual region 31 is present in the first image (step A13: Yes), in the individual counting device 1, the individual region 31 detected in the first image is the image of the first image in the second image. An estimated individual area 51 in which the individual area 31 is estimated to move is estimated (step A14). Subsequently, the individual counting device 1 acquires the first image again when the individual region 31 is not detected from the first image (step A13: No).

In step A11, the estimation unit 2 of the individual counting device 1 acquires the first image transmitted from the image pickup unit 21 when the image pickup unit 21 starts shooting. Subsequently, in step A12, the estimation unit 2 of the individual counting device 1 executes binarization and labeling processing on the acquired first image, and then the individual region existing in the acquired first image. The process of detecting 31 is executed. That is, the estimation unit 2 detects the quantity of the individual region 31 and the individual region 31 in the first image. In step A13, when the individual area 31 is present in the first image, the estimation unit 2 of the individual counting device 1 shifts to step A14 and estimates the estimated individual area 51 in the second image. Further, when the individual region 31 does not exist in the first image, the estimation unit 2 acquires the first image again.

A specific description will be given with reference to FIGS. 3 and 4. First, the estimation unit 2 acquires the image 1 shown in FIG. 3 or FIG. 4 as the first image. However, since the estimation unit 2 cannot detect the individual region 31 from the image 1, the waiting state Twait is continued until the individual region 31 is detected (step A11 → A12 → A13: No → A11). Next, the estimation unit 2 acquires the image 2 shown in FIG. 3 or FIG. 4 as the first image. In this case, in FIG. 3, since the individual regions 31a and 31b could be detected, the estimated individual region for the individual regions 31a and 31b is estimated (step A11 → A12 → A13: Yes → A14). Further, since the estimation unit 2 first detects the individual regions 31a and 31b in the image 2, it is set to Tstart when the tracking is started.

Further, when the estimation unit 2 acquires the image 3, the image 4, and the image 5 shown in FIG. 3 or 4 as the first image after the time point Tstart, the individual regions 31a and 31b can be detected, so that the individual regions can be detected. The estimated individual area 51 for 31a and 31b is estimated (step A19 → A11 → A12 → A13: Yes → A14).

Subsequently, the individual counting device 1 acquires the second image when the individual region 31 is detected in the first image (step A15). However, the second image may be acquired immediately after step A11. Subsequently, the individual counting device 1 executes a process of detecting the individual region 31 from the second image (step A16). After that, the individual counting device 1 determines the overlap between the individual region 31 existing in the second image and the estimated individual region estimated using the first image (step A17). As a result of the determination, when the overlap between the estimated individual area and the individual area 31 detected from the second image is equal to or more than the predetermined area value (step A17: Yes), the individual counting device 1 takes over the individual area 31.

In step A15, the determination unit 3 of the individual counting device 1 acquires the second image transmitted from the image pickup unit 21. Subsequently, in step A16, the determination unit 3 of the individual counting device 1 executes binarization and labeling processing on the acquired second image, and then the individual region existing in the acquired second image. The process of detecting 31 is executed. That is, the determination unit 3 detects the quantity of the individual region 31 and the individual region 31 in the second image. The estimation unit 2 may detect the individual region 31 and the quantity of the individual region 31 in the second image in step A11.

In step 17, when the overlap between the estimated individual area 51 and the individual area 31 detected from the second image is equal to or greater than the predetermined area value, the determination unit 3 of the individual counting device 1 determines the first image and the second image. It is determined that the individual regions 31 of the above are the same. Subsequently, when it is determined that the individual region 31 of the first image and the second image are the same, in step A18, the determination unit 3 of the individual counting device 1 determines the individual region 31 of the first image and the second image. Is associated with the individual area 31 of. For example, the identification number is associated with the overlapping part of the individual area. By doing so, the individual area 31 is tracked. If the individual regions 31 of the first image and the second image are not determined to be the same, the determination unit 3 of the individual counting device 1 shifts to step A20.

A specific description will be given with reference to FIGS. 3 and 4. For example, when the image 2, image 3, image 4, etc. shown in FIG. 3 or FIG. 4 are used as the first image, the determination unit 3 uses the image captured after capturing the image 2, image 3, and image 4 as the first image. It is acquired as a second image (step A15), and the individual regions 31a and 31b in the second image are detected (step A16). Subsequently, as shown in FIG. 5, the determination unit 3 determines that the overlapping portion 52 of the estimated individual region 51 and the individual region 31 detected from the second image is equal to or greater than the predetermined region value (step A17: Yes). In step A18, the individual area 31 of the first image and the individual area 31 of the second image are associated with each other. Further, the determination unit 3 sets the step A20 when the overlapping portion 52 between the estimated individual region 51 and the individual region 31 detected from the second image is smaller than the predetermined region value or when there is no overlap (step A17: No). Transition. Further, in such a case, since the determination unit 3 cannot track the individual area 31, the Tend is set as the time when the tracking is completed.

In step A19, the counting unit 4 of the individual counting device 1 has one, all, or two of the counting processes shown in (1), (2), and (3) described above for each individual area 31 being tracked. Are combined to count the individual region 31 on the image. Further, the counting process of (4) may be used instead of (3).

A specific description will be given with reference to FIGS. 3, 4, and 8. For example, when the individual regions 31a and 31b are tracked in FIG. 3, the counting unit 4 associates the number of individuals 22 corresponding to the individual regions 31a and 31b with each of the individual regions 31a and 31b for each image. See Table 81 in FIG. Further, when the individual regions 31c and 31d are traced in FIG. 4, the quantity of the individual 22 corresponding to the individual regions 31c and 31d is associated with each of the individual regions 31c and 31d for each image. See Table 82 in FIG.

In step A20, the integration unit 5 of the individual counting device 1 counts the quantity of the individual region 31 for each time from Tstart when the individual region 31 is detected to Tend when the individual region 31 is no longer detected, and the counted individual. The quantity of the area 31 is integrated every time the time elapses.

A specific description will be given with reference to FIGS. 3, 4, and 8. For example, when the quantity of the individual 22 is 0 before the time point Tstart in FIG. 3, and the quantity of the individual 22 corresponding to the traced individual regions 31a and 31b is 2 in the time from the time point Tstart to the time point Tend in FIG. , The integration unit 5 sets the integrated value to 2. See Table 81 in FIG. Further, when the quantity of the individual 22 corresponding to the traced individual regions 31c and 31d is 3 in the time from the time point Tstart to the time point Tend in FIG. 4 after the time point Tend in FIG. Is 5 (= 2 + 3). See Table 82 in FIG.

[Effect of this embodiment]
In such an individual counting device 1, in the present embodiment, the overlap between the estimated individual area estimated based on the individual area 31 in the first image and the individual area 31 in the second image is equal to or more than the predetermined area value. In some cases, the individual regions 31 of the first image and the second image are determined to be the same. By doing so, the individual region 31 corresponding to the individual 22 imaged in the time from the time point Tstart to the time point Tend can be tracked.

Further, in the present embodiment, the individual counting device 1 counts the quantity of the individual 22 corresponding to the individual region 31 by using the counting processing of (1), (2), (3) and (4) described above. Even when the moving individuals 22 are overlapped and imaged, the number of individuals 22 corresponding to the overlapping imaged individual regions 31 can be accurately counted.

Further, in the present embodiment, the individual counting device 1 counts individual regions by using the counting processing of (1), (2), (3), and (4) described above for each time from the time point Tstart to the time point Tend. 31
The quantity of the individual 22 corresponding to is integrated every time the time point Tend elapses. By doing so, the quantity of the individual 22 can be counted accurately.

[Computer]
The individual counting program according to the embodiment of the present invention may be any program that causes a computer to execute the steps shown in FIGS. 6 and 7. By installing and executing this program on a computer, the individual counting device 1 and the individual counting method according to the present embodiment can be realized. In this case, the processor of the computer functions as at least an estimation unit 2, a determination unit 3, a counting unit 4, and an integration unit 5 to perform processing.

Further, the program in the present embodiment may be executed by a computer system constructed by a plurality of computers. In this case, for example, each computer may function as at least one of an estimation unit 2, a determination unit 3, a counting unit 4, and an integration unit 5.
[Physical configuration]
Here, a computer that realizes the individual counting device 1 by executing the program in the embodiment will be described with reference to FIG. FIG. 9 is a block diagram showing an example of a computer that realizes the individual counting device 1.

As shown in FIG. 9, the computer 110 includes a CPU (Central Processing Unit) 111, a main memory 112, a storage device 113, an input interface 114, a display controller 115, a data reader / writer 116, and a communication interface 117. And. Each of these parts is connected to each other via a bus 121 so as to be capable of data communication. The computer 110 may have a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), or the like in addition to the CPU 111 or in place of the CPU 111.

The CPU 111 expands the programs (codes) of the present embodiment stored in the storage device 113 into the main memory 112 and executes them in a predetermined order to perform various operations. The main memory 112 is typically a volatile storage device such as a DRAM (Dynamic Random Access Memory). Further, the program in the present embodiment is provided in a state of being stored in a computer-readable recording medium 120. The program in the present embodiment may be distributed on the Internet connected via the communication interface 117.

Further, specific examples of the storage device 113 include a semiconductor storage device such as a flash memory in addition to a hard disk drive. The input interface 114 mediates data transmission between the CPU 111 and an input device 118 such as a keyboard and mouse. The display controller 115 is connected to the display device 119 and controls the display on the display device 119.

The data reader / writer 116 mediates the data transmission between the CPU 111 and the recording medium 120, reads the program from the recording medium 120, and writes the processing result in the computer 110 to the recording medium 120. The communication interface 117 mediates data transmission between the CPU 111 and another computer.

Specific examples of the recording medium 120 include a general-purpose semiconductor storage device such as CF (Compact Flash (registered trademark)) and SD (Secure Digital), a magnetic recording medium such as a flexible disk, or a CD-. Examples include optical recording media such as ROM (Compact Disk Read Only Memory).

The individual counting device 1 in the present embodiment can also be realized by using the hardware corresponding to each part instead of the computer in which the program is installed. Further, the individual counting device 1 may be partially realized by a program and the rest may be realized by hardware.

As described above, according to the present invention, the number of individuals can be accurately counted even when moving individuals are imaged in an overlapping manner. The present invention is useful in fields where individual counting is required.

1 Individual counting device 2 Estimating unit 3 Judging unit 4 Counting unit 5 Integration unit 21 Imaging unit 22 Individual 23 Movement source 24 Destination 25 Route 31 Individual area 31a, 31b, 31c, 31d Individual area 32 Route image 51 Estimated individual area 52 Overlap Part 110 computer 111 CPU
112 Main memory 113 Storage device 114 Input interface 115 Display controller 116 Data reader / writer 117 Communication interface 118 Input device 119 Display device 120 Recording medium 121 Bus

Claims (12)

An estimation unit that estimates an estimated individual region in which the individual is estimated to move in a second image captured after capturing the first image, based on an individual region indicating an individual detected from the first image. When,
When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the determination unit that determines that the individual area of the first image and the second image is the same. ,
When the individual region includes a preset characteristic region indicating the characteristics of the individual, a counting unit that counts the quantity of the characteristic region as the quantity of the individual corresponding to the individual region.
The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted as the time. With the integration unit, which integrates each time
An individual counting device characterized by having. The individual counting device according to claim 1.
The counting unit further counts the number of the individual corresponding to the individual area based on the ratio of the value indicating the size of the individual area to the preset value indicating the size of the individual. An individual counting device characterized by. The individual counting device according to claim 1 or 2.
The counting unit further determines the individual region based on the ratio of the value indicating the size of the individual region in the first image to the value indicating the size of the individual region in the second image. An individual counting device characterized by counting the quantity of the corresponding individual. (A) Based on the individual region indicating the individual detected from the first image, the estimated individual region in which the individual is estimated to move is estimated in the second image captured after the first image is captured. , Steps and
(B) When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the individual areas of the first image and the second image are determined to be the same. Steps and
(C) When the individual region includes a preset characteristic region indicating the characteristics of the individual, the step of counting the quantity of the characteristic region as the quantity of the individual corresponding to the individual region, and the step.
(D) The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted. With each step that accumulates each time the time elapses,
An individual counting method characterized by having. The individual counting method according to claim 4.
In the step (c), the number of the individual corresponding to the individual region is further determined based on the ratio of the value indicating the size of the individual region to the preset value indicating the size of the individual. An individual counting method characterized by counting. The individual counting method according to claim 4 or 5.
In the step (c), further, based on the ratio of the value indicating the size of the individual region in the first image and the value indicating the size of the individual region in the second image, the said. An individual counting method characterized by counting the quantity of the individual corresponding to the individual area. On the computer
(A) Based on the individual region indicating the individual detected from the first image, the estimated individual region in which the individual is estimated to move is estimated in the second image captured after the first image is captured. , Steps and
(B) When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the individual areas of the first image and the second image are determined to be the same. Steps and
(C) When the individual region includes a preset characteristic region indicating the characteristics of the individual, the step of counting the quantity of the characteristic region as the quantity of the individual corresponding to the individual region, and the step.
(D) The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted. With each step that accumulates each time the time elapses,
An individual counting program for executing . The individual counting program according to claim 7.
To the computer
In the step (c), the number of the individual corresponding to the individual region is further determined based on the ratio of the value indicating the size of the individual region to the preset value indicating the size of the individual. Let me count
An individual counting program characterized by that. The individual counting program according to claim 7 or 8.
To the computer
In the step (c), further, based on the ratio of the value indicating the size of the individual region in the first image and the value indicating the size of the individual region in the second image, the said. Count the quantity of the individual corresponding to the individual area
An individual counting program characterized by that. An image pickup unit that captures an image of a path for moving the individual, which has a width set at least according to the size of the individual.
Based on the individual area indicating the individual detected from the first image, the estimated individual area in which the individual is estimated to move is estimated in the second image captured after the first image is captured. Department and
When the overlap between the estimated individual area and the individual area detected from the second image is equal to or more than the predetermined area value, the determination unit that determines that the individual area of the first image and the second image is the same. ,
When the individual region includes a preset characteristic region indicating the characteristics of the individual, a counting unit that counts the quantity of the characteristic region as the quantity of the individual corresponding to the individual region.
The quantity of the individual corresponding to the individual region is counted for each time from the time when the individual region is detected to the time when the individual region is no longer detected, and the quantity of the individual corresponding to the counted individual region is counted as the time. With the integration unit, which integrates each time
An individual counting system characterized by having. The individual counting system according to claim 10.
The counting unit further counts the number of the individual corresponding to the individual area based on the ratio of the value indicating the size of the individual area to the preset value indicating the size of the individual. An individual counting system characterized by. The individual counting system according to claim 10 or 11.
The counting unit further determines the individual region based on the ratio of the value indicating the size of the individual region in the first image to the value indicating the size of the individual region in the second image. An individual counting system characterized by counting the quantity of the corresponding individual.

Images