JP7693591B2 - Ticket gate system, ticket gate method, and ticket gate program - Google Patents

Description

Embodiments of the present invention relate to a ticket gate system, a ticket gate method, and a ticket gate program.

When entering or exiting the ticket gate, the ticket gate or a server connected to the ticket gate uses facial recognition to determine whether or not entry or exit is permitted as a measure to prevent the transfer of discounts that can only be used by specific individuals.

When performing facial recognition, there is a possibility that a face may be mistakenly recognized even when the user is not visible in the image at all. Furthermore, there are cases where the face of a user who has no intention of entering or exiting is detected. For example, there is a technology that uses the fact that a passerby has crossed a certain boundary line as a trigger to photograph the user's face and perform facial recognition.

JP 2013-191172 A

When performing facial recognition, there are cases where the second user is in the shadow of the first user and the camera is unable to capture the second user's face. Or, the user's face may not be captured by the camera due to luggage or other factors. Meanwhile, the user has no way of knowing that the camera has not captured the user's face. This makes it difficult to maintain an appropriate distance between users for facial recognition.

This invention was made in light of the above-mentioned circumstances, and its purpose is to provide technology that can ensure the flow of users by reducing the possibility of stopping a user at a ticket gate if the user's face cannot be photographed.

The ticket gate system according to the embodiment includes an acquisition unit that acquires an image captured by a camera that captures the passageway of the ticket gate, a face authentication unit that authenticates the face of a first user based on the captured image, and a control device that tracks the first user based on the captured image and determines whether a second user has been detected when it is determined that the first user has been tracked.

FIG. 1 is a conceptual diagram showing an example of the configuration of a ticket gate system according to an embodiment. FIG. 2 is a block diagram showing an example of the configuration of the ticket gate according to the first embodiment. FIG. 3 is a block diagram showing an example of the configuration of the central server according to the first embodiment. FIG. 4 is a flowchart showing an example of a processing procedure in the ticket gate system shown in FIG. FIG. 5 is a flowchart showing an example of a processing procedure in the ticket gate system shown in FIG.

The ticket gate system, ticket gate method, and ticket gate program are described in detail below with reference to the drawings. Note that in the following embodiments, parts with the same numbers perform similar operations and will not be described repeatedly. For example, when there are multiple identical or similar elements, a common symbol may be used to describe each element without distinguishing between them, and a subnumber may be used in addition to the common symbol to describe each element with distinction between them.

FIG. 1 is a conceptual diagram showing an example of the configuration of a ticket gate system 10 according to an embodiment.
The ticket gate system 10 is, for example, a system for controlling entry and exit at a ticket gate in a transportation facility such as a train or bus. Of course, the ticket gate system 10 can also be applied to entry and exit at a live music venue or the like.

As shown in FIG. 1, the ticket gate system 10 includes a ticket gate 20, a camera 21, a medium detection unit 22, a door 23, a central server 30, etc.

The ticket gates 20 control whether or not to allow passage to users (passengers) who are attempting to pass through (enter or exit) a passageway. One or more ticket gates 20 may be installed at ticket gates in stations, etc. Each of these ticket gates 20 forms a passageway through which users can pass. The ticket gates 20 can also be connected to a central server 30 via a network or by wire, and transmit and receive various information to and from the central server 30.

The camera 21 is installed corresponding to the ticket gate 20. For example, the camera 21 is installed so as to ensure a shooting range that allows the face of a user passing through the passage to be photographed. Therefore, the camera 21 may be installed on the ticket gate 20, or may be installed in a location separate from the ticket gate 20. Here, in the example shown in FIG. 1, the camera 21 is installed in a location separate from the ticket gate 20.

The medium detection unit 22 is installed, for example, on the ticket gate 20 that performs entry and exit processing using a medium held by the user (for example, an IC card ticket, a two-dimensional code, a barcode, etc.). The medium detection unit 22 then detects the user's medium and reads the information stored on the medium. For example, when a user holds an IC card ticket over the medium detection unit 22, the medium detection unit 22 detects the IC card ticket and can send and receive various information to and from the IC card ticket.

The door 23 is installed at the ticket gate 20 and physically blocks the user's passage if the user is not permitted to pass.

The central server 30 is connected to the ticket gates 20 by wire or wirelessly. The central server 30 is installed in the station premises, the station control room, etc. The central server 30 can also perform various authentications such as facial recognition based on various information received from the ticket gates 20.

[First embodiment]
(composition)
Next, the configuration of the ticket gate 20 in the ticket gate system 10 according to the first embodiment will be described.

FIG. 2 is a block diagram showing an example of the configuration of the ticket gate 20 according to the first embodiment.
As shown in FIG. 2 , the ticket gate 20 includes a control device 201 , a memory unit 202 , a communication unit 203 , a medium authentication unit 204 , a face authentication unit 205 , an interface 206 , a passage control unit 207 , and a notification unit 208 .

The control device 201 controls various operations in the ticket gate 20. The control device 201 has a processor that executes programs. For example, the control device 201 includes a processor, memory, an interface, and the like. The control device 201 performs various processes by having the processor execute programs stored in the memory unit 202 or the memory within the control device 201.

For example, the control device 201 operates as an acquisition unit. The control device 201 acquires captured images from the camera 21 through the interface 206 described below. Furthermore, the control device 201 receives various information from the medium detection unit 22 through the interface 206. Then, the control device 201 stores the acquired various information in the memory unit 202. Furthermore, the control device 201 may transmit the acquired various information to the central server 30 through the communication unit 203 described below.

For example, the control device 201 operates as a detection unit. The control device 201 detects the face of the user from the captured image from the camera 21. Here, the control device 201 is not limited to only detecting the face of the user from the captured image, but is also capable of detecting the user's body and luggage.

The control device 201 also operates as a passage control unit. The control device 201 controls the passage control unit 207 to control the passage of users. If at least one of the authentication of the medium or the authentication of the user's face is successful, the control device 201 allows the user to pass (enter or exit). On the other hand, if either the authentication of the medium or the authentication of the face fails, the control device 201 prohibits the user from passing and controls the passage control unit 207 to prevent the user from passing.

The storage unit 202 is composed of a rewritable non-volatile memory such as a HDD (hard disk drive) or SSD (solid state drive). The storage unit 202 stores a face database. The face database is a database of face data registered in advance by users who use the ticket gate 20. For example, the control device 201 operating as an acquisition unit acquires face data based on a face image captured by a camera 21 installed in the ticket machine, and stores the face data in the face database 2021 of the storage unit 202. Alternatively, the control device 201 operating as an acquisition unit may receive a face image of the user's own face captured by a mobile terminal or the like held by the user, acquire face data based on the acquired face image, and store the face data in the face database 2021.

The communication unit 203 is a communication interface including a communication module for transmitting and receiving various data to and from the central server 30 via wired or wireless communication. The communication unit 203 transmits and receives various information to and from the central server 30. The communication module may be a general communication module as long as it is capable of communicating with the central server 30.

The media authentication unit 204 receives from the control device 201 the information that the media detection unit 22 has acquired from the medium. The media authentication unit 204 then determines whether the medium can be authenticated based on the received information. In other words, the media authentication unit 204 determines whether the user should be allowed to pass.

The face authentication unit 205 receives face detection information from the control device 201 indicating that the user's face has been detected from the image captured by the camera 21. The face authentication unit 205 performs face authentication based on the face detection information and the face database. The face authentication is performed, for example, depending on whether data corresponding to the data of the user's face detected by the control device 201 is present in the face database 2021. The face authentication unit 205 may output the authentication result to the control device 201.

The interface 206 is an interface that enables various information to be transmitted and received between the camera 21 and the medium detection unit 22 shown in FIG. 1. Here, the interface may include a communication interface. For example, it may be possible to wirelessly transmit and receive various information from the camera 21.

The camera 21 starts capturing images when the ticket gate 20 is activated, i.e., when it is ready to determine whether to allow the user to pass. The control device 201 acquires the captured images and detects the user's face. The control device 201 may also store the captured images in the memory unit 202.

The medium detection unit 22 starts up at the same time that the ticket gate 20 starts up. If the medium detection unit 22 can detect a medium, it may light up in a color such as green or blue to notify the user that authentication is possible. On the other hand, if authentication of the medium is to be canceled, it may turn off or light up in red to notify the user that authentication is not possible.

The passage control unit 207 controls the passage of users when passing through the passage (entering or exiting) next to the ticket gate 20. The passage control unit 207 is driven to control the passage of users in accordance with instructions from the control unit 201. For example, the passage control unit 207 has a door 23 and a drive mechanism that controls the opening and closing of the door 23. The passage control unit 207 operates to close the door 23 when preventing the user from passing through (entering or exiting), and to open the door 23 when allowing the user to pass through. The passage control unit 207 may also be controlled by the control unit 201.

The notification unit 208 includes a display, a speaker, a lamp, etc. The display displays various information to the user. Therefore, the display is provided in a position where it can be seen by a user who is going to pass through the ticket gate 20. For example, under the control of the control device 201, the display displays information indicating whether or not the user is allowed to pass based on the face authentication results. The speaker outputs sound at a volume that the user can recognize. Under the control of the control device 201, the speaker outputs a sound such as an alert to a user who is determined not to be allowed to pass. The lamp may or may not light up when the user is allowed to pass. On the other hand, when it is determined that the user is not allowed to pass, the lamp may light up in blue to inform the user that they are not allowed to pass.

FIG. 3 is a block diagram showing an example of the configuration of the central server 30 according to the first embodiment.
As shown in FIG. 3, the central server 30 includes a control device 301 , a storage unit 302 , a communication unit 303 , a medium authentication unit 304 , a face authentication unit 305 , and a display unit 306 .

The control device 301 controls various operations in the central server 30. The control device 301 has a processor that executes programs. For example, the control device 301 includes a processor, a memory, an interface, and the like. The control device 301 executes various processes by having the processor execute programs stored in the storage unit 302 or the memory in the control device 301.

For example, the control device 301 can execute the same operations as those described for the control device 201 of the ticket gate 20. For example, the control device 301 controls the passage of a user by transmitting information for controlling the passage control unit 207 to the ticket gate 20 via the communication unit 303 described below. If either the authentication of the medium or the authentication of the user's face is successful, the control device 301 transmits information to the ticket gate 20 permitting the user to pass (enter or exit). On the other hand, if either the authentication of the medium or the authentication of the face fails, the control device 301 transmits information to the ticket gate 20 that denies the user passage. The control device 201 of the ticket gate 20 that receives the information controls the passage control unit 207 to prevent the user from passing.

The memory unit 302 is configured with a rewritable non-volatile memory such as an HDD or SSD. The memory unit 202 stores a face database 3021. The face database 3021 is a database of face data registered in advance by users who use the ticket gate 20. Here, the face database 3021 may store the same data as the face database 2021 stored in the memory unit 202 of the ticket gate 20. For example, the face data is acquired based on a face image captured by a camera installed in the ticket machine. Alternatively, the face data is acquired based on a face image captured of the user's own face by a mobile terminal or the like carried by the user.

The communication unit 303 is a communication interface including a communication module for transmitting and receiving various data to and from the ticket gate 20 via a wired or wireless connection. The communication unit 303 transmits and receives various information to and from the ticket gate 20. For example, the communication unit 303 receives photographed data captured by the camera 21 arranged on the ticket gate 20. The communication unit 303 may then store the received photographed data in the memory unit 302. Furthermore, the communication unit 303 receives information acquired by the medium detection unit 22 from the ticket gate 20. The communication unit 303 may then output the information to the medium authentication unit 304 and store it in the memory unit 302. The communication module may be a general communication module as long as it is capable of communicating with the ticket gate 20.

Under the control of the control device 301, the media authentication unit 304 acquires information acquired by the media detection unit 22 of the ticket gate 20 from the ticket gate 20. The media authentication unit 304 then determines whether or not the user should be allowed to pass based on the acquired information. The media authentication unit 304 then transmits the result of this determination to the ticket gate 20 via the communication unit 303.

The face authentication unit 305 receives face detection information from the control device 301 indicating that a user's face has been detected from the captured image acquired from the communication unit 303. The face authentication unit 205 performs face authentication based on the face detection information and the face database 3031. Face authentication is performed, for example, depending on whether data corresponding to the data of the user's face detected by the control device 201 is present in the face database 2021. The face authentication unit 305 may output the authentication result to the control device 301. The control device 301 may then transmit the result to the ticket gate 20 via the communication unit 303.

The display unit 306 is a display device that uses, for example, liquid crystal, organic electroluminescence, or the like, and displays voice, text, images, and the like under the control of the control device 301. In other words, the display unit 306 is capable of displaying various types of information to the administrator who manages the central server 30.

(operation)
FIG. 4 is a flowchart showing an example of a processing procedure in the ticket gate system 10 shown in FIG.
The operation of this flowchart is realized by the control device 201, medium authentication unit 204, and face authentication unit 205 of the ticket gate 20 reading and executing a program stored in the memory unit 202. Note that a part of the operation of this flowchart may be realized by the control device 301 and face authentication unit 305 of the central server 30 reading and executing a program stored in the memory unit 302.

It is assumed that each user has registered their facial data linked to the ID of the medium in advance using an application or the web, or that the user's facial data has been photographed by the ticket vending machine and linked to the ID of the medium issued by the ticket vending machine. The facial data is then transmitted to the ticket gate 20 and the central server, and is registered in at least one of the facial database 2021 in the memory unit 202 of the ticket gate 20 or the facial database 3031 in the memory unit 302 of the central server 30.

In addition, the memory unit 202 of the ticket gate 20 or the memory unit 302 of the central server 30 may store the ID of a medium that can pass through the ticket gate 20.

First, this flowchart starts when the control device 201 detects the face of a user (first user) from an image captured by a camera. The control device 201 outputs face detection information indicating that the control device 201 has detected the user's face to the face authentication unit 205. Here, the control device 201 may detect not only the user, but also the user's body and luggage. Note that this flowchart may start when the medium detection unit 22 detects a medium and the control device 201 receives the detection information.

The face authentication unit 205 authenticates the first user (step ST101). The face authentication unit 205 acquires face data of the user based on the face detection information. Then, the face authentication unit 205 compares the acquired face data with the data stored in the face database 2021 of the storage unit 202. If data corresponding to the face data is in the face database 2021, the face authentication unit 205 authenticates the first user. The face authentication unit 205 may output face authentication information indicating that the first user has been authenticated to the control device 201. In the following description, on the other hand, if data corresponding to the face data is not in the face database 2021, the face authentication unit 205 may determine that the authentication of the first user has failed and output face non-authentication information to the control device 201. In this case, the control device 201 may output an instruction to the passage control unit 207 to close the door 23 so as not to allow the first user to pass.

When the medium detection unit 22 detects a medium, the control device 201 receives the medium information acquired by the medium detection unit 22 from the medium. The control device 201 then outputs the various pieces of information to the medium authentication unit 204. The medium authentication unit 204 authenticates the first user based on the medium information. In this case, the control device 201 may also acquire an image captured by the camera 21, and detect the face, body, luggage, etc. of the first user from the acquired captured image.

For simplicity, the following explanation will be given assuming that the face authentication unit 205 successfully authenticates the first user.

The control device 201 tracks the first user (step ST102). The control device 201 regards the first user obtainable from the photographed data captured by the camera 21 as an authenticated user, and tracks the face of the first user from the photographed image captured by the camera 21. In addition, at least one of the body or luggage of the first user may be included as a tracking target.

The control device 201 determines whether the first user can be tracked (step ST103). The control device 201 determines whether the first user can be tracked from the captured data. If it is determined that the face of the first user can no longer be tracked, the process proceeds to step ST104. On the other hand, if it is determined that the face of the first user, which is the target of tracking, can be tracked, the process proceeds to step ST106.

The control device 201 determines whether the face of the second user has been detected from the captured image (step ST104). The control device 201 determines whether the face of the second user, who is the next user, has been detected based on the captured image stored in the memory unit 202 or received from the camera 21. If the face of the second user has been detected, the control device 201 outputs face detection information indicating the detection to the face authentication unit 205. On the other hand, if the face of the second user has not been detected, the process returns to step ST104 again. In other words, the control device 201 continues the determination described in step ST104 until it can detect the face of the second user.

The face authentication unit 205 starts authenticating the second user (step ST105). The face authentication unit 205 acquires face data of the second user based on the face authentication information. Then, the face authentication unit 205 compares the face data with data stored in the face database 2021, as in step ST101. If it is determined as a result of the comparison that there is data corresponding to the face data, the face authentication unit 205 authenticates the second user. Although FIG. 4 shows an example in which the process ends, the process may return to step ST102 with the second user as the first user.

The control device 201 determines whether a face other than that of the first user has been detected (step ST106). The control device 201 determines whether a face other than that of the first user has been detected from the captured data. For example, if the face of a second user is detected behind the first user, the control device 201 outputs second detection information indicating that the face of the second user has been detected to the face authentication unit 205. Then, the process proceeds to step ST105. On the other hand, if a face other than that of the first user cannot be detected, the face of the second user may be in a blind spot caused by the first user (face, body, or luggage) being captured in the captured image. In this case, the process proceeds to step ST107.

The control device 201 stops the authentication process (step ST107). When stopping the authentication process for the second user, the control device 201 may output a signal to close the door 23 to the passage control unit 207 to close the door 23. Furthermore, the control device 201 may output information to stop the authentication process to the notification unit 208. The notification unit 208 may display information to indicate that the user's face has not been detected on the display of the notification unit 208. The notification unit 208 may turn on the lamp of the notification unit 208 in red or turn off the lamp. Then, the process returns to step ST103. That is, the control device 201 determines whether or not the first user has been tracked.

When the process returns to step ST103 and the face of a person other than the first user is detected in step ST106, the control device 201 will resume the authentication process. That is, the control device 201 may output a signal to open the door to the passage control unit 207, and may output information indicating that the authentication process has been resumed to the notification unit 208. The notification unit 208 may display on the display that the user's face has been detected, and may turn on a lamp in blue or green, for example.

In addition, while the example of the flowchart shown in FIG. 4 shows an example in which the ticket gate 20 performs the steps, the central server 30 may also perform each step. In this case, the ticket gate 20 transmits information acquired by the camera 21 and the medium detection unit 22 to the central server 30. Then, each unit of the central server 30 executes the processing shown in FIG. 4 and transmits the processing results to the ticket gate 20.

(Effects of the First Embodiment)
According to the embodiment described above, while the first user is being tracked, unless the face of the second user is recognized, the authentication process is stopped and the second user is notified that the authentication process has been stopped. This allows the second user to recognize that his or her face is not captured by the camera 21.

Following this notification, the second user will adjust the distance between them and the first user so that their face is captured by the camera 21. This will ensure that the distance between the users is appropriate, allowing the second user's face to be detected and authenticated.

Second Embodiment
(composition)
The configuration according to the second embodiment is the same as that of the first embodiment described with reference to FIGS. 1 to 3, and therefore a description thereof will be omitted here.

(operation)
FIG. 5 is a flowchart showing an example of a processing procedure in the ticket gate system 10 shown in FIG.
The operation of this flowchart is realized by the control device 201, medium authentication unit 204, and face authentication unit 205 of the ticket gate 20 reading and executing a program stored in the memory unit 202. Note that a part of the operation of this flowchart may be realized by the control device 301 and face authentication unit 305 of the central server 30 reading and executing a program stored in the memory unit 302.

The start timing and steps ST201 to ST205 in Figure 5 are the same as the start timing and steps ST101 to ST105 described with reference to Figure 4, so their description will be omitted.

The control device 201 determines whether a body or luggage other than that of the first user has been detected (step ST206). The control device 201 determines whether a body or luggage other than that of the first user has been detected from the captured data. For example, if the face of a second user cannot be detected behind the first user but a body or luggage is detected, the control device 201 determines that a second user is present. In this case, the process proceeds to step ST207. On the other hand, if the body or luggage of the second user cannot be detected, the process returns to step ST203.

Step ST207 is the same as step ST207 described with reference to FIG. 4, so the description will be omitted.

The control device 201 determines whether the face of the second user has been detected (step ST208). The control device 201 determines whether the face of the second user has been detected from the captured data. For example, if the second user moves to a location where the face can be captured by the camera 21, the control device 201 outputs second detection information indicating that the face of the second user has been detected to the face authentication unit 205. Then, the process proceeds to step ST205. On the other hand, if the face of the second user cannot be detected, the process returns to step ST203. In this case, the control device 201 determines whether the first user has been tracked while suspending the authentication process.

(Effects of the Second Embodiment)
According to the embodiment described above, if the body or baggage of the second user is detected but the face cannot be detected, there is a high possibility that the second user is in a blind spot behind the first user that cannot be captured by the camera 21. Therefore, the authentication process is stopped and the second user is notified that the authentication process has been stopped. This allows the second user to recognize that the face of the second user is not captured by the camera 21.

Following this notification, the second user will adjust the distance between them and the first user so that their face is captured by the camera 21. This will ensure that the distance between the users is appropriate, allowing the second user's face to be detected and authenticated.

In addition, while the example of the flowchart shown in FIG. 5 shows an example in which the ticket gate 20 performs the steps, the central server 30 may also perform each step. In this case, the ticket gate 20 transmits information acquired by the camera 21 and the medium detection unit 22 to the central server 30. Then, each unit of the central server 30 executes the processing shown in FIG. 5 and transmits the processing results to the ticket gate 20.

[Other embodiments]
It should be noted that the present invention is not limited to the above embodiment. For example, the first embodiment and the second embodiment may be combined. For example, when the face of the first person cannot be detected, the authentication process may not be immediately stopped, but the determination described in step ST206 may be performed. For example, when the body or baggage other than that of the first user cannot be detected, the process may return to step ST103.

In short, this invention is not limited to the above-mentioned embodiment, and various modifications can be made in the implementation stage without departing from the gist of the invention. Furthermore, the embodiments may be implemented in combination as appropriate as possible, in which case the combined effects can be obtained. Furthermore, the above-mentioned embodiment includes inventions at various stages, and various inventions can be extracted by appropriate combinations of the multiple constituent elements disclosed.

10... Ticket gate system 20... Ticket gate 201... Control device 202... Memory unit 2021... Face database 203... Communication unit 204... Medium authentication unit 205... Face authentication unit 206... Interface 207... Passage control unit 208... Notification unit 21... Camera 22... Medium detection unit 23... Door 30... Central server 301... Control device 302... Memory unit 3021... Face database 303... Communication unit 304... Medium authentication unit 305... Face authentication unit 306... Display unit

Claims (10)

an acquisition unit that acquires an image captured by a camera that captures an image of a ticket gate passage;
a face authentication unit that authenticates a face of a first user based on the captured image;
a control device that tracks the first user based on the captured image, and when it is determined that the first user has been tracked, determines whether or not a second user has been detected;
A ticket gate system comprising: The ticket gate system according to claim 1, wherein detecting the second user includes detecting a face of the second user. The ticket gate system according to claim 2, wherein the control device stops the authentication process for the second user if it is determined that the face of the second user cannot be detected. The ticket gate system according to claim 2, wherein when it is determined that the face of the second user has been detected, the face authentication unit performs face authentication of the second user. The ticket gate system according to any one of claims 1 to 4, wherein detecting the second user includes detecting at least one of the body or baggage of the second user. The ticket gate system according to claim 5, wherein the control device stops the authentication process for the second user when at least one of the user's body or baggage is detected. The ticket gate system according to any one of claims 1 to 6, wherein the tracking includes detecting the face, body, and baggage of the first user. The ticket gate system according to any one of claims 1 to 7, wherein if it is determined that the tracking is not possible, the control device determines whether the face of the second user is detected. Acquiring an image captured by a camera photographing a passageway of a ticket gate;
authenticating a face of a first user based on the captured image;
Tracking the first user based on the captured image;
determining whether a second user has been detected when it is determined that the first user has been tracked;
A ticket gate method comprising: A ticket gate program that causes a processor to function as each unit of the ticket gate system according to any one of claims 1 to 8.