JP7704646B2 - Person counting system and gate device using said system - Google Patents

Description

The present invention relates to a people counting system that, when it detects an object approaching passersby, determines whether the object is a person or an object and counts the number of people, and to a gate device that uses this system.

Patent Document 1 describes an entrance/exit management device that controls whether to unlock an electric lock based on whether the number of people entering, as read by a card reader, matches the number of people in a monitored area determined by image processing. In other words, the technology unlocks (= allows entry) by confirming that the number of people read from the card matches the number of people in a monitored area by a camera, and is capable of detecting tailgating when multiple people overlap and enter the detection line of an optical sensor.

Patent document 2 describes a gate monitoring system that can effectively prevent tailgating by combining a people counting means that uses a stereo image processing method to accurately grasp the number of people passing through the gate with an authentication means that authenticates the authenticated person.

JP 2004-169441 A JP 2008-59179 A

However, the entrance/exit management device of Patent Document 1 and the gate monitoring system of Patent Document 2 are unable to distinguish whether an object is a person or an object when it is detected approaching a passerby. For example, in Patent Document 1, the means for detecting the number of people within the monitored area recognizes people using differential images between multiple images taken using a time difference, so if a passerby is holding an infant or child (referred to as a passerby carrying a baby), it is highly likely that the number of people will be determined to be "one person" rather than "two people."

In addition, in Patent Document 2, tailgating detection is performed by comparing the number of people allowed to pass through with the number of people entering the gate, using tracking detection with an infrared sensor, but tailgating detection is not possible for people passing through while carrying children.

As such, when conventional technology detects an object approaching a passerby, it is unable to distinguish whether the object is a person or an object, and is unable to detect infants or children being held by an adult (cuddle position) and accurately count the number of people.

The present invention was made in consideration of the above circumstances, and its purpose is to provide a people counting system that can determine whether an object is a person or an object when it is detected approaching passersby, and can accurately count the number of people, and a gate device using this system.

A people counting system according to one embodiment of the present invention comprises a point cloud data acquisition unit that is provided above a specific position on a passageway and acquires point cloud data of passersby from different directions, an infrared data acquisition unit that is provided above the specific position and acquires heat source data of the passersby, and an integration processing unit that performs image processing on the point cloud data from the point cloud data acquisition unit, and when an object is detected in the vicinity of the passersby, superimposes the heat source data on the image processing results to determine whether the object is a person or an object.

A gate device according to one embodiment of the present invention includes a person counting system, an information reading unit that reads pass information from a pass medium, a gate that allows the pedestrian to pass through the specific location when open and prevents the pedestrian from passing through the specific location when closed, and a control unit that opens and closes the gate, and the control unit is configured to open the gate based on the pass information and close the gate when the number of people allowed to pass included in the pass information does not match the counting result of the person counting system.

The present invention provides a people counting system that can determine whether an object is a person or an object when it detects an object approaching a passerby and can accurately count the number of people, and a gate device that uses this system.

1 is an external perspective view showing a schematic configuration of a gate device in which a people counting system according to an embodiment of the present invention is used; 2A to 2C are a plan view, a side view, and a front view of a person counting unit in the gate device of FIG. 1 . FIG. 2 is a side view of a main body in the gate device of FIG. 1 . 1 is a block diagram showing a configuration example of a gate device according to an embodiment of the present invention; 5 is a block diagram showing an example of the configuration of a person counting unit in FIG. 4 . 2 is a schematic diagram for explaining the operations of a main control unit, a passage control unit, and a person counting unit in the gate device according to the embodiment of the present invention. FIG. FIG. 13 is an image captured by a ToF camera. 6 is a flowchart showing an outline of a holding determination process executed by the integration processing unit shown in FIG. 5 . FIG. 13 shows a point cloud image of the first ToF camera, in which areas considered to be "possibly holding" have been extracted. FIG. 13 shows a point cloud image of a second ToF camera. FIG. 13 is a diagram showing heat source data obtained by photographing with an IR camera. FIG. 13 is a diagram showing a point cloud image of a ToF camera rotated to a side view. This is a diagram showing areas where the point cloud image of the ToF camera is rotated to a side view and where it is thought that there is a possibility of hugging. FIG. 2 is a diagram showing a two-dimensional image taken by an IR camera from the front.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
1 shows a schematic configuration of a gate device in which a people counting system according to an embodiment of the present invention is used. The gate device 10 includes a pair of main bodies 11-1 and 11-2 arranged opposite each other along a passage (specific position) 11a indicated by an arrow AA, and a people counting unit 12 arranged above the main bodies 11-1 and 11-2.

The person counting unit 12 is attached to the main body 11-1 by an L-shaped arm 15, and the point cloud data acquisition unit and infrared data acquisition unit are placed above a specific position in the passage. Then, based on the three-dimensional point cloud data of the pedestrian acquired by the point cloud data acquisition unit and the heat source data acquired by the infrared data acquisition unit, it is possible to determine whether an object approaching a pedestrian is a person or an object, and to detect the presence of a person carrying a child.

An information reading unit 13 is provided at the entrance side of main body 11-1, and traffic control gates (gates) 14-1 and 14-2 are provided at the exit sides of main bodies 11-1 and 11-2. Information reading unit 13 reads QR codes (registered trademark) printed on travel pass media (admission tickets and boarding passes) and travel pass information recorded on IC cards, etc. When open, gates 14-1 and 14-2 allow pedestrians to pass through passage 11a, but when closed, they prevent pedestrians from passing through passage 11a.

Although not shown, a transmission sensor and a reflection sensor are arranged facing each other on the surfaces of the main bodies 11-1 and 11-2 facing the passage 11a. The main body 11-1 also has a built-in main control unit and a passage control unit. The main control unit determines the number of passersby based on the passage ticket information read by the information reading unit 13 and the number of people counted by the person counting unit 12.

Figure 2 shows the person counting unit 12 in the gate device 10 shown in Figure 1, with (a) being a plan view, (b) being a side view, and (c) being a front view. In this example, the point cloud data acquisition unit is made up of first and second ToF (Time of Flight) cameras 16-1 and 16-2, which acquire point cloud data of passersby from different directions. The infrared data acquisition unit is made up of an IR (Infrared) camera 17, which photographs passersby to acquire heat source data. The IR camera 17 is placed between the ToF cameras 16-1 and 16-2, and is aligned with the coordinate axes of the ToF cameras 16-1 and 16-2 so that it can photograph the same range.

In addition, an integration processing unit is provided that performs image processing on the point cloud data from the ToF cameras 16-1 and 16-2, and when an object approaching a passerby is detected, it overlays the heat source data captured by the IR camera 17 on the image processing result to determine whether the object is a person or an object. The integration processing unit may be built into the person counting unit 12 or into the main body 11-1. Alternatively, similar processing can be achieved by a program using some functions of the main control unit and the passage control unit without providing a dedicated processing unit.

Figure 3 is a side view of the main body 11-1 of the gate device shown in Figure 1. Although not shown, the main body 11-2 is basically constructed in the same way, and 16 pairs of transmission type sensors 20-1 to 20-16 and 4 pairs of reflection type sensors 21-1 to 21-4 are arranged facing each other on the side faces of the passage 11a of the main bodies 11-1 and 11-2. The transmission type sensors 20-1 to 20-16 are paired with a light emitter and a light receiver, and the light emitted from the light emitter is received by the light receiver. These transmission type sensors 20-1 to 20-16 function as human detection sensors, and detect people when a pedestrian passing through the passage 11a blocks the light.

Reflective sensors 21-1 to 21-4 have a pair of a light emitter and a light receiver mounted on the same side, and the light emitted from the emitter is reflected by a person and received by the receiver. In this way, reflective sensors 21-1 to 21-4 detect people by detecting light reflected by passersby. Transmissive sensors 20-1 to 20-16 detect the movement of passersby in passage 11a, and the reflective sensor can determine whether the passerby is an adult or a child.

Figure 4 is a block diagram showing an example of the configuration of a gate device according to an embodiment of the present invention. The gate device 10 comprises a main control unit 22, a passage control unit 23, and a person counting unit 12. The main control unit 22 receives QR code (registered trademark) information printed on an admission ticket or boarding pass read by a QR reader 24 provided in the information reading unit 13, or passage ticket information recorded on an IC card or the like read by an IC reader/writer (ICRW) 25. The information read from the QR code (registered trademark) or IC card includes the expected number of people passing through, whether they are adults or children, etc. This read information is input to the main control unit 22.

The passage control unit 23 receives detection information of passersby from the transmission sensors 20 (20-1 to 20-16) and the reflection sensors 21 (21-1 to 21-4), such as the passing position, number of people, whether they are adults or children, etc. The person counting unit 12 also receives three-dimensional point cloud data of passersby acquired by the ToF cameras 16-1 and 16-2 and heat source data of passersby acquired by the IR camera 17, and the results of the determination by the person counting unit 12 (the number of people and whether or not there are any passersby carrying babies) are input to the passage control unit 23. Then, information from the passage control unit 23 is input to the main control unit 22, which controls the opening and closing of the gate 14 (or the door unit) based on information read by the QR reader 24 and IC reader/writer 25, detection information of passersby detected by the transmission sensor 20 and reflective sensor 21, and information obtained by the ToF cameras 16-1, 16-2 and IR camera 17 to determine whether an object approaching a passerby is a person or an object.

Figure 5 shows an example of the configuration of the person counting unit 12 in Figure 4. In this example, an integration processing unit 12a is provided within the person counting unit 12. Images captured by the ToF cameras 16-1 and 16-2 are input to the integration processing unit 12a. The ToF camera 16-1 is composed of a sensor 16a and an image processing unit 16b that processes the output image of the sensor 16a, and the ToF camera 16-2 is composed of a sensor 16c and an image processing unit 16d that processes the output image of the sensor 16c. The processing results of the image processing units 16b and 16d are then input to the integration processing unit 12a. Images captured by the IR camera 17 are also input to the integration processing unit 12a.

When the point cloud data and heat source data are superimposed, the integration processing unit 12a determines that the object is a person if the heat source of the object exceeds a temperature threshold or if the number of point cloud data points with a heat source is equal to or exceeds a certain value.

Figure 6 is a schematic diagram for explaining the operation of the main control unit 22, passage control unit 23, and person count unit 12 in a gate device according to an embodiment of the present invention. When a passerby holds a ticket (passage ticket medium) over the information reading unit 13 on the entrance side of the main body 11-1, a QR code (registered trademark) is read by, for example, the QR reader 24, and a command to open the door (gate) is input from the main control unit 22 to the passage control unit 23, and the gate 14 (14-1, 14-2) opens. The main control unit 22 recognizes that there is one person passing through and executes "credit + 1." Here, "credit" refers to the right to pass that is generated by reading the ticket, and if there is not enough credit, an error is generated.

When a person enters the passage 11a between the main bodies 11-1 and 11-2 and the transmission type sensor 20 (20-1 to 20-16) detects the passage of the person, the passage control unit 23 notifies the main control unit 22 that a person has been detected. If the passage control unit 23 detects a person's likeness based on the use of the ticket and the movement conditions detected by the transmission type sensor 20 and the reflection type sensor 21, it assigns a sorting number to the person detected.

On the other hand, when a person enters the area monitored by the ToF cameras 16-1, 16-2 and the IR camera 17 (reaches the camera shooting point), a shooting notification is sent to the person counting unit 12, and the person is photographed by the ToF cameras 16-1, 16-2 and the IR camera 17. FIG. 7 shows an image captured by the ToF cameras 16-1, 16-2, and the shooting range of the ToF cameras 16-1, 16-2 becomes the monitoring area. The monitoring area is near the entrance of the main body 11-1, 11-2 of the gate device 10, and is detected before the ticket is held up to the information reading unit 13. When the person counting unit 12 notifies the passage control unit 23 that a photo has been taken, the passage control unit 23 notifies the person counting unit 12 of the assigned sorting number as a shooting response.

When the photographing response is input to the person counting unit 12, a holding determination process is executed. The determination result is input to the passage control unit 23 as a number of people notification, a camera detection notification is input to the main control unit 22, and a number of people response is returned to the person counting unit 12.

When a pedestrian arrives at the center of the gate device 10 (the center of the passage 11 a ), a human detection notification is executed to the passage control unit 23 , and the passage control unit 23 notifies the main control unit 22 .
Similarly, when a passerby arrives at the exit of the gate device 10 , a human detection notification is executed to the passage control unit 23 , and the passage control unit 23 notifies the main control unit 22 .

When a pedestrian leaves the exit of the gate device 10, the main control unit 22 executes "credit -1." This causes the main control unit 22 to input a command to close the door (gate 14) to the passage control unit 23, and the gate 14 is closed. In this state, light is received by all of the transmission type sensors 20 (20-1 to 20-16), and the passage control unit 23 checks with the main control unit 22 to confirm that no one is present.

Figure 8 is a flow chart showing an overview of the holding determination process executed by the integrated processing unit 12a shown in Figure 5. Figure 9 shows a point cloud image (point cloud data) acquired by the ToF sensor 16a, and Figure 10 shows a point cloud image (point cloud data) acquired by the ToF sensor 16c. Also, Figure 11 shows heat source data acquired by photographing with the IR camera 17.

Since the mounting positions of the ToF sensors 16a and 16c are shifted, the measurement data is slightly different, and an object close to a pedestrian is captured in the area surrounded by a two-dot chain line DL1 in Fig. 9. By capturing images from two different directions in this way, even if an object cannot be detected from a particular direction, it may be possible to detect it from another direction, thereby improving the accuracy of detection.
When the integration process is started, shape determination results are received from the image processors 16b and 16d of the ToF cameras 16-1 and 16-2 (step S1).

Next, the point cloud data acquired by the ToF sensors 16a and 16c are subjected to image processing by the image processors 16b and 16d, respectively. FIG. 12 shows a point cloud image of a photographed person viewed from the side, based on the results of the image processing performed by the image processors 16b and 16d. For convenience, the point cloud image is shown here rotated to a side view, but in reality it is an image taken from diagonally above and in front.

From this point cloud image, it is determined whether there is an object close to the passerby, i.e., whether there is a possibility of a child being held (step S2). If it is determined that there is a possibility, the area where "possible holding" is considered is extracted, as shown by the area surrounded by the two-dot chain line DL2 in FIG. 13.

If it is determined in step S2 that there is a possibility of the object being held, the integrated processing unit 12a overlays the heat source data (heat source information) from the IR camera 17 shown in FIG. 14 onto the point cloud image in FIG. 13 (step S3), and determines whether or not the detected object has a heat source (step S4).
FIG. 14 shows a two-dimensional image taken by an IR camera from the front, and the area surrounded by a two-dot chain line DL3 is a location that has been determined from the point cloud image to be a location where there is a possibility of a person being held.

If it is determined that there is a heat source within the area surrounded by the two-dot chain line DL3, it is determined that the person is carrying the child (step S5). For example, when the point cloud data and the heat source data are superimposed, if the heat source of the object exceeds a temperature threshold value, or if the number of point cloud data with a heat source is equal to or greater than a certain value, it is determined that the object is a person.
If it is determined that there is no heat source, and if it is determined in step S2 that there is no possibility of the baby being held, it is determined that the baby is traveling alone (step S6).

In the next step S7, the determination result is stored in the main control unit 22. Then, it is determined whether or not the number of people notification position has been reached, and if it is determined that the number of people notification position has been reached, a number of people notification is transmitted (step S8) and the integration process ends. On the other hand, if it is determined that the number of people notification position has not been reached, the process returns to step S1 and the shape determination result is received from the image processing units 16b and 16d of the ToF cameras 16-1 and 16-2.

With this type of people counting system and the gate device using this system, when an object is detected approaching a passerby, the heat source data is used to determine whether the object is a person or an object, so that, for example, a person holding a stuffed toy will not be mistakenly counted as two people, and the number of people can be counted accurately.

The configurations and control procedures described in the above embodiments are merely schematic illustrations to enable the present invention to be understood and implemented. Therefore, the present invention is not limited to the described embodiments, and can be modified in various forms without departing from the scope of the technical ideas set forth in the claims.

For example, the ToF cameras (point cloud data acquisition units) 16-1, 16-2 and the IR camera (infrared data acquisition unit) may each acquire point cloud data and heat source data multiple times at different positions as passersby move, and the integrated processing unit 12a may perform image processing on each of the multiple acquired point cloud data, overlay the heat source data corresponding to each image processing result, and determine whether an object approaching the passerby is a person or an object based on the multiple image processing results.

The system then makes a composite judgment based on factors such as the number of times the person was determined to be "passing with a baby" or the number of times the person was determined to be passing alone, or the order in which these combinations are arranged. In this way, by determining whether a nearby object is a person or an object from multiple judgments, it is possible to make a more accurate judgment.

In addition, while the example described above uses a people counting system applied to a gate device, it can of course also be applied to other devices and systems.

10...Gate device, 11-1, 11-2...Main body, 11a...Passage (specific position), 12...Person counting unit, 12a...Integrated processing unit, 13...Information reading unit, 14, 14-1, 14-2...Passage control gate (gate), 15...L-shaped arm, 16-1, 16-2...ToF camera (point cloud data acquisition unit), 16a, 16c...Sensor, 16b, 16d...Image processing unit, 17...IR camera (infrared data acquisition unit), 20-1 to 20-16...Transmission type sensor, 21-1 to 21-4...Reflection type sensor, 22...Main control unit, 23...Passage control unit, 24...QR reader, 25...IC reader/writer (ICRW)

Claims (5)

a point cloud data acquisition unit that is provided above a specific position of the passage and acquires point cloud data of passersby from different directions;
an infrared data acquisition unit provided above the specific position and configured to acquire heat source data of the pedestrian;
A person counting system comprising: an integrated processing unit that performs image processing on the point cloud data from the point cloud data acquisition unit, and when an object close to the passerby is detected, overlays the heat source data on the image processing results to determine whether the object is a person or an object. The person counting system of claim 1, wherein the integration processing unit determines that the object is a person when the heat source of the object exceeds a temperature threshold when the point cloud data and the heat source data are superimposed, or when the number of point cloud data having a heat source is equal to or greater than a certain value. the point cloud data acquisition unit and the infrared data acquisition unit acquire point cloud data and heat source data multiple times at different positions in accordance with the movement of the pedestrian,
The person counting system of claim 1 or 2, wherein the integration processing unit performs image processing on each of the acquired point cloud data multiple times, overlays the heat source data corresponding to each image processing result, and determines whether an object approaching the passerby is a person or an object based on the multiple image processing results. the point cloud data acquisition unit includes first and second Time of Flight (ToF) cameras that acquire three-dimensional point cloud data of the pedestrian from two different directions above the specific position;
The people counting system according to claim 1 , wherein the infrared data acquisition unit includes an IR (Infrared) camera whose coordinate axes are aligned with those of the first and second ToF cameras. The people counting system according to any one of claims 1 to 4,
an information reading unit that reads pass information from the pass medium;
a gate that allows the pedestrian to pass through the specific position when opened and prevents the pedestrian from passing through the specific position when closed;
A control unit that opens and closes the gate;
Equipped with
The control unit is configured to open the gate based on the pass information and to close the gate when the number of people allowed to pass included in the pass information does not match the counting result of the person counting system, said gate device.