JP7792572B2 - Door phone base unit and registration method - Google Patents

Description

This disclosure relates to a door phone base unit and registration method.

Conventionally, doorphone systems (or intercom systems) have been known that have the function of authenticating visitors by comparing video data received from a camera-equipped doorphone slave with pre-stored (registered) facial data (see, for example, Patent Documents 1 and 2).

Japanese Patent Application Laid-Open No. 2000-295603 Japanese Patent Application Laid-Open No. 2004-120521

In the above-mentioned conventional technology, pre-stored facial data is obtained by capturing a photograph (image) of the visitor's face using the camera in a camera-equipped intercom sub-unit, for example, when the visitor visits for the first time. Such camera-equipped intercom sub-units that capture images of the visitor's face are generally installed outside the building (near the front door, nameplate, or mailbox, etc.) or at the entrance to an apartment building.

As a result, in order to acquire and register facial data of a person (person to be authenticated), not limited to visitors (including residents, for example), that person must go out of their way to go outside the building or to the entrance of the apartment complex and wait for a certain amount of time. Depending on the season, that person may have to wait outside the building in the cold, or they may have to wait while other people are coming and going. As such, the person to be authenticated may be disadvantaged in terms of convenience when registering authentication data such as facial data.

Non-limiting examples of the present disclosure contribute to providing an indoor phone base unit and registration method that can mitigate the inconvenience that an authenticatee experiences when registering authentication data.

A door phone master unit according to one embodiment of the present disclosure is capable of unlocking a front door and includes a receiving unit that receives from a wireless mobile terminal an image of a person captured by a camera included in the wireless mobile terminal, an extracting unit that extracts from the image biometric data of the person for authentication used to unlock the front door, and a registering unit that registers the biometric data for authentication.

A registration method according to one embodiment of the present disclosure is executed by a door phone master unit capable of unlocking a front door, receives an image of a person captured by a camera installed in a wireless mobile terminal from the wireless mobile terminal, extracts from the image biometric data of the person for authentication used to unlock the front door, and registers the biometric data for authentication.

Note that these comprehensive or specific aspects may be realized as a system, device, method, integrated circuit, computer program, or recording medium, or as any combination of a system, device, method, integrated circuit, computer program, and recording medium.

According to one embodiment of the present disclosure, authentication data is registered based on an image of a person captured using a camera in a wireless mobile terminal, rather than using a camera in a door phone handset, thereby minimizing the inconvenience that the person being authenticated may experience when registering authentication data.

Further advantages and benefits of one embodiment of the present disclosure will become apparent from the specification and drawings. While such advantages and/or benefits may be provided by some of the embodiments and features described in the specification and drawings, not all of them necessarily need to be provided to obtain one or more identical features.

FIG. 1 is a system configuration diagram illustrating an example of a configuration of a door phone system according to an embodiment of the present disclosure. A simplified example of the door phone system. FIG. 1 is a block diagram illustrating an example of a configuration of a door phone master unit according to an embodiment of the present disclosure. FIG. 1 is a block diagram illustrating an example of a configuration of a front door slave unit according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a wireless terminal according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a door phone master unit according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a wireless terminal according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a door phone master unit according to an embodiment of the present disclosure. 1 is a sequence diagram illustrating an example of an operation of a door phone system according to an embodiment of the present disclosure. 1 is a sequence diagram illustrating an example of an operation of a door phone system according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a display screen of a wireless terminal according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a display screen of a wireless terminal according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a display screen of a wireless terminal according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a display screen of a wireless terminal according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a front door slave unit according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a front door slave unit according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a door phone master unit according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a door phone master unit according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example of an operation of a door phone master unit according to an embodiment of the present disclosure. FIG. 1 is a diagram illustrating an example of the operation of a wireless terminal according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a display screen of a wireless terminal according to an embodiment of the present disclosure.

Embodiments of the present disclosure will be described in detail below, with appropriate reference to the drawings. However, more detailed explanations than necessary may be omitted. For example, detailed explanations of matters that are already well-known or duplicate explanations of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following explanation and to facilitate understanding by those skilled in the art.

The accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

(Embodiment)
<Doorphone system>
First, a door phone system according to an embodiment of the present disclosure will be described.

Figure 1 is a system configuration diagram showing an example of the configuration of a door phone system 10 according to this embodiment. Figure 1 also shows the surrounding environment of the system configuration.

As shown in FIG. 1, the door phone system 10 includes, for example, a door phone master unit 100 placed inside a building 30, which is the home of a user 20; a front door slave unit (which may also be referred to as a door phone slave unit) 200 placed near the front door 31 (or nameplate or mailbox) of the building 30; and an electric lock 300 provided on the front door 31.

The intercom master unit 100 is equipped with, for example, a speaker, display, monitor button, call button, microphone, and lock/unlock button. The intercom master unit 100 is initially configured to pre-register the front door sub-unit 200 and electric lock 300. In addition to the function of answering calls (starting calls), the intercom master unit 100 also has the function of displaying images of the area around the front door sent from the front door sub-unit 200, accepting operations to lock/unlock the electric lock 300 on the front door 31, registering faces for authentication, and authenticating visitors by face.

For example, when the intercom master unit 100 and the front door sub-unit 200 are in a call state, the speaker outputs audio from the front door sub-unit 200 to the outside of the intercom master unit 100. The display, for example, displays video captured by the camera of the front door sub-unit 200. For example, when the monitor button is pressed, the camera of the front door sub-unit 200 is activated and the video captured by the camera of the front door sub-unit 200 begins to be displayed on the display. For example, when the call button is pressed, the intercom master unit 100 and the front door sub-unit 200 are in a call state. The microphone, for example, collects sounds around the intercom master unit 100. For example, when the lock/unlock button is pressed, the electric lock 300 of the front door 31 is locked/unlocked.

The front door slave unit 200 is equipped with an infrared sensor (human presence sensor), a call button, a speaker, a microphone, and a camera. A two-wire cable 201, for example, consisting of a pair of copper wires, is laid between the front door slave unit 200 and the main unit 100 of the indoor phone. In addition to the calling and conversation functions, the front door slave unit 200 has the function of activating a camera when a person is detected by the infrared sensor, capturing video of the area around the front door with the camera and transmitting the video to the main unit 100 of the indoor phone. The front door slave unit 200 and the main unit 100 may communicate wirelessly according to, for example, the DECT (Digital Enhanced Cordless Telecommunications) protocol.

The infrared sensor, for example, detects a person in front of the front door sub-unit. When the call button is pressed, for example, it sends a visitor notification to the intercom master unit 100. The speaker, for example, when the front door sub-unit 200 and the intercom master unit 100 are in a call state, outputs audio from the intercom master unit 100 to the outside of the front door sub-unit 200. The microphone, for example, collects sounds around the front door sub-unit 200. The camera, for example, captures video of the area around the entrance.

The electric lock 300 is a key device that unlocks and locks the front door 31 in response to an unlock request (instruction) or lock request (instruction), which is an electrical signal, in addition to the normal key operation (operation using a physical key) into the keyhole. An electric lock code 301, which is, for example, a JEM-A standard code, is installed between the electric lock 300 and the indoor phone base unit 100. The electric lock 300 unlocks and locks under the control of the indoor phone base unit 100.

The door phone system 10 also includes a communication router 400 located inside the building 30, a server 500 connected to a public network 510 such as the Internet, and a wireless terminal (which may also be referred to as a wireless mobile terminal) 600 such as a smartphone or tablet that can be taken outside the building 30. The wireless terminal 600 includes a wireless terminal carried by the user 20 and a wireless terminal carried by a person other than the user 20 (for example, a relative of the user 20). The following describes an example in which the wireless terminal 600 is primarily a smartphone.

The communication router 400 connects to the intercom master unit 100, the wireless terminal 600, and the public network 510 via wireless communication lines 401, 402, and 403, respectively, and relays communications between the intercom master unit 100 and the wireless terminal 600 and the public network 510, and relays communications between the intercom master unit 100 and the wireless terminal 600. The wireless communication lines 401 and 402 may be, for example, Wi-Fi (registered trademark; hereafter omitted) lines, and the wireless communication line 403 may be, for example, a cellular (mobile) line such as an LTE (Long Term Evolution) line or an NR (New Radio) line. Alternatively, the indoor apparatus 100 and the wireless terminal 600 may be directly connected via a wireless communication line (for example, using Wi-Fi Direct or Bluetooth (registered trademark)) without going through the communication router 400, and the communication router 400 may be connected to the public network 510 via a cable line, optical fiber line, or the like instead of the wireless communication line 403.

When the intercom master unit 100 is powered on, the server 500 connects to the intercom master unit 100 by connecting to the communication router 400 via the public network 510 or the like, and registers the intercom master unit 100. The server 500 then maintains a constant connection to the intercom master unit 100 via the public network 510 or the like, and exchanges various information with the intercom master unit 100. An encrypted communication path may be established between the server 500 and the intercom master unit 100, and the server 500 may perform encrypted communication with the intercom master unit 100. The server 500 also receives a request from the wireless terminal 600 of the user 20 to register the user's face on a remote wireless terminal 600 (not shown), and pairs the intercom master unit 100 with the remote wireless terminal 600. The server 500 may also be referred to as an authentication server.

When the wireless terminal 600 is located inside the building 30, for example, it connects to the communication router 400 via the wireless communication line 402 to access the intercom master unit 100. When the wireless terminal 600 is located outside the building 30, for example, it connects to the public network 510 via the wireless communication line 601 and accesses the intercom master unit 100 via the server 500. The wireless communication line 601 may be, for example, a cellular (mobile) line such as an LTE line or an NR line. The wireless terminal 600 may perform encrypted communication with the intercom master unit 100 from inside and outside the building 30.

The wireless terminal 600 also has a speaker, a touch-panel display, a microphone, a (terminal) monitor button, a (terminal) call button, a (terminal) hang-up button, a (terminal) unlock button, and a (terminal) lock button. The (terminal) monitor button, (terminal) call button, (terminal) hang-up button, (terminal) unlock button, and (terminal) lock button may be, for example, image buttons (icons) displayed on a touch-panel display that can be used on an application for using the functions of the door phone system 10, as described below.

The speaker outputs audio from the intercom master unit 100 to the outside of the wireless terminal 600, for example, when the wireless terminal 600 and the intercom master unit 100 are in a call state. The touch-panel display displays, for example, video captured by the camera of the front door sub-unit 200, a (terminal) monitor button, a (terminal) call button, a (terminal) disconnect button, a (terminal) unlock button, a (terminal) lock button, etc. When the (terminal) monitor button is pressed (tapped), for example, it starts outputting video of the area around the entrance sent from the front door sub-unit 200 via the intercom master unit 100. When the (terminal) call button is pressed (tapped), for example, it starts a call between the wireless terminal 600 and the front door sub-unit 200 via the intercom master unit 100. When the (terminal) disconnect button is pressed (tapped), for example, it disconnects the connection (communication, call) between the wireless terminal 600 and the intercom master unit 100. For example, when the (terminal) unlock button is pressed (tapped), it unlocks the electric lock 300 on the front door 31 via the intercom master unit 100. For example, when the (terminal) lock button is pressed (tapped), it locks the electric lock 300 on the front door 31 via the intercom master unit 100.

The intercom master unit 100 registers a wireless terminal 600 located within the building 30 in response to a request from the wireless terminal 600, for example, connected via the communication router 400 or directly. The intercom master unit 100 connects to the wireless terminal 600 as appropriate and exchanges various information with the wireless terminal 600.

As shown in Figure 2, which shows a simplified example of the door phone system 10, the door phone master unit 100 transfers video, audio, and various control information between the front door slave unit 200 and the wireless terminal 600. Furthermore, the door phone master unit 100 receives unlock/lock instructions (operation of the (terminal) unlock button/lock button) from the wireless terminal 600 and unlocks/locks the electric lock 300. Therefore, the door phone master unit 100 has the function of responding to calls (starting a call), displaying video of the area around the front door, and enabling the wireless terminal 600 to unlock/lock the electric lock 300. Note that while Figure 2 shows multiple front door slave units 200, the number of front door slave units 200 can be any number greater than or equal to one.

With this type of doorphone system 10, the user 20 can be notified when there is a visitor at their home, not only when they are inside the building 30, but also when they are outside the building 30. Furthermore, with this type of doorphone system 10, the user 20 can talk to the visitor while checking the doorphone video and remotely unlock the front door 31.

As mentioned above, door phone systems with the ability to authenticate visitors using facial data are known, but there is room for improvement in terms of convenience when registering facial data. Therefore, for example, when user 20 is inside building 30, door phone master unit 100 registers user 20's face by receiving a photograph of user 20's face taken by user 20's wireless terminal 600 from user 20's wireless terminal 600. Furthermore, when a person, such as a relative of user 20, is remotely located, door phone master unit 100 registers the person's face by receiving a photograph of that person's face taken by that person's wireless terminal 600 from that person's wireless terminal 600 via public network 510, server 500, etc. In this way, by registering the faces of user 20 and other people without using a camera installed in door phone slave unit 200, it is possible to minimize inconvenience to user 20 and other people.

<Configuration of each device>
Next, the configuration of the main devices of the intercom system 10 will be described. Each device constituting the intercom system 10 includes, for example, a central processing unit (CPU), a storage medium such as a read-only memory (ROM) storing a control program, a working memory such as a random access memory (RAM), and a communication circuit (not shown). In this case, the functions of each functional unit (block) described below are realized by the CPU executing the control program. Alternatively, each functional unit may be realized by a dedicated device or circuit rather than a general-purpose processing device (e.g., a CPU), or by a combination of a general-purpose processing device and a dedicated device or circuit.

<Configuration of the door phone master unit>
FIG. 3 is a block diagram illustrating an example of the configuration of indoor apparatus 100 according to the embodiment of the present disclosure.

The indoor phone base unit 100 includes a video processing unit 110, a display output unit 120, an audio processing unit 130, an audio output unit 140, an audio input unit 150, a call control unit 160, a network control unit 170, a memory unit 180, an equipment control unit 190, and a one-time password generation unit 195. As described above, the indoor phone base unit 100 may be a computer including, for example, a CPU, ROM, RAM, etc. The CPU, for example, reads a program corresponding to the processing content from the ROM and loads it into the RAM, and works in conjunction with the loaded program to control the operation of the indoor phone base unit 100.

The video processing unit 110 processes the input video data and generates an image to be displayed by the display output unit 120. The video processing unit 110 includes a video data input unit 112, a video data decoding unit 114, a face recognition storage unit 116, and a face recognition processing unit 118.

The video data input unit 112 inputs streaming video data from the front door slave unit 200. The video data may be digital or analog. The video data input unit 112 outputs the streaming video data input from the front door slave unit 200 to the video data decoding unit 114.

The video data decoding unit 114 converts the streaming video data input from the video data input unit 112 into image data. If the image data is used for facial authentication, the video data decoding unit 114 outputs the image data to the facial authentication processing unit 118. If the image data is not used for facial authentication, the video data decoding unit 114 outputs the image data to the display output unit 120 via the call control unit 160. The images are, for example, still images. In one example, the video data is converted into still image data for 15 still images per second.

The face authentication storage unit 116 stores a registration database in which the face authentication processing unit 118 registers facial images or facial features (also referred to as face data or simply faces) of the person (person to be authenticated) to be authenticated, in association with the person's name and other identifying information. The facial features are any quantities used by the face authentication processing unit 118 for face authentication, such as the cascade classifier used by the face authentication processing unit 118 for face detection, neural network parameters used for face matching, facial expression data, LBPH (Local Binary Patterns Histograms), SURF (Speeded Up Robust Features) features, etc. Registering at least one of a face image and/or facial features is simply referred to as registering a face. The registration database may be stored in the storage unit 180.

The facial authentication processing unit 118 detects and extracts the face of a person included in the image data input from the video data decoding unit 114, and performs facial authentication (identification) by comparing the detected and extracted face with the facial data stored in the registration database of the facial authentication storage unit 116. The person to be authenticated may be, for example, a visitor. If facial authentication is successful, the facial authentication processing unit 118 outputs a signal (information) indicating successful facial authentication to the electric lock control unit 192 via the network control unit 170. Although not shown in Figure 3, if facial authentication fails, the facial authentication processing unit 118 transmits a signal (information) indicating failed facial authentication via the network control unit 170 to a specific wireless terminal 600 among the wireless terminals 600 registered in the indoor phone master unit 100 (for example, the wireless terminal 600 of user 20, a resident of building 30).

Furthermore, as described below, in response to a request from the wireless terminal 600 of the user 20, for example, the facial authentication processing unit 118 extracts the face of the user 20 or another person (biometric data for authentication) to be used later for facial authentication from a video or image received from the same or a different wireless terminal 600 via the network control unit 170. The facial authentication processing unit 118 then registers (stores) the extracted face in a registration database in the facial authentication storage unit 116, together with identification information such as the name of the user 20 or another person entered by the user 20.

The face authentication processing unit 118 is an example of an extraction unit according to the present disclosure, an example of a registration unit according to the present disclosure, and an example of an authentication unit according to the present disclosure.

The display output unit 120 displays the image input from the video processing unit 110. The display output unit is, for example, a display such as an LCD display device or an OLED display device.

The voice processing unit 130 processes a linear electrical signal representing voice input from the voice input unit 150 via the call control unit 160, and outputs voice data. Furthermore, the voice processing unit 130 processes voice data input from the front door slave unit 200, and outputs a linear electrical signal representing voice. The voice processing unit 130 includes a voice data input unit 132, a voice data decoding unit 134, a voice data encoding unit 136, and a voice data output unit 138.

The audio data input unit 132 inputs streaming audio data from the front door slave unit 200. The audio data represents, for example, the voice of a visitor speaking into the front door slave unit 200. The audio data input unit 132 outputs the streaming audio data input from the front door slave unit 200 to the audio data decoding unit 134.

The audio data decoding unit 134 decodes the streaming audio data input from the audio data decoding unit 134 and generates a linear electrical signal representing the audio. In this embodiment, the decoding is digital-to-analog conversion. The decoding method corresponds to the encoding method used by the audio data encoding unit 236 (see Figure 4) of the front door slave unit 200. The audio data decoding unit 134 outputs the generated linear electrical signal representing the audio to the audio output unit 140 via the call control unit 160.

The audio data encoding unit 136 encodes the linear electrical signal representing the audio input from the audio input unit 150 via the call control unit 160 to generate streaming audio data. In this embodiment, the encoding is analog-to-digital conversion. The encoding method corresponds to the decoding method used by the audio data decoding unit 234 (see Figure 4) of the front door slave unit 200. The audio data encoding unit 136 outputs the generated streaming audio data to the audio data output unit 138.

The audio data output unit 138 outputs the streaming audio data input from the audio data encoding unit 136 to the front door unit 200.

The audio output unit 140 converts the linear electrical signal representing the audio input from the audio data decoding unit 134 via the call control unit 160 into audio and outputs the audio to the outside of the indoor apparatus 100. In this embodiment, the audio output unit 140 is a speaker.

The audio input unit 150 converts audio from around the indoor phone master unit 100 into a linear electrical signal. In this embodiment, the audio input unit 150 is a microphone.

The call control unit 160 controls calls from the front door unit 200. The call control unit 160 includes an incoming call processing unit 162, a timer processing unit 164, and a call processing unit 166.

The incoming call processing unit 162 processes incoming calls from the front door slave unit 200. In this embodiment, the incoming call processing unit 162 notifies the timer processing unit 164 of the incoming call in response to a notification of the incoming call from the front door slave unit 200.

The timer processing unit 164 performs timer processing from the reception of an incoming call from the front door unit 200 to the start of a call between the main unit 100 and the front door unit 200. In this embodiment, in response to notification of an incoming call from the incoming call processing unit 162, the timer processing unit 164 starts outputting a linear electrical signal representing a ring tone to the audio output unit 140 and starts communication of video data between the main unit 100 and the front door unit 200. Also in this embodiment, in response to detecting the pressing of an answer button (not shown) provided on the main unit 100, the timer processing unit 164 stops outputting the linear electrical signal representing the ring tone to the audio output unit 140 and notifies the call processing unit 166 of the answer. For example, the time from the start to the end of output of the linear electrical signal representing the ring tone is three seconds.

The call processing unit 166 processes calls between the indoor phone base unit 100 and the front door slave unit 200. In this embodiment, the call processing unit 166 starts communication of audio data between the indoor phone base unit 100 and the front door slave unit 200 in response to a response notification from the timer processing unit 164. Also, in this embodiment, the call processing unit 166 disconnects video and audio communication between the indoor phone base unit 100 and the front door slave unit 200 in response to detecting the end of the call.

The network control unit 170 controls communication in accordance with a communication protocol that corresponds to the type of network over which the indoor phone master unit 100 communicates. The network control unit 170 includes a communication processing unit 172.

The communication processing unit 172 performs communication (transmission and reception) between the indoor phone base unit 100 and the communication partner of the indoor phone base unit 100 in accordance with a communication protocol followed by the indoor phone base unit 100 and the communication partner of the indoor phone base unit 100. The communication processing unit 172 is an example of a receiving unit according to the present disclosure.

Although not shown in FIG. 3, when the indoor phone base unit 100 and a device such as the wireless terminal 600 communicate via the server 500, video, audio, and various control information are communicated via the network control unit 170 (communication processing unit 172). When the indoor phone base unit 100 is the transmitting side, the video, audio, and various control information are transmitted via the call control unit 160 and the network control unit 170 (communication processing unit 172) as appropriate. When the indoor phone base unit 100 is the receiving side, the video, audio, and various control information are received via the network control unit 170 (communication processing unit 172) and stored in the memory unit 180, output to the display output unit 120, or output to the audio output unit 140 as appropriate via the call control unit 160.

The memory unit 180 stores data necessary for the operation of the indoor phone base unit 100, programs executed by the CPU, data received via the network control unit 170 from devices such as the user 20's wireless terminal 600, data generated by the functional units of the indoor phone base unit 100, etc. The memory unit 180 may also store the registration database described above.

The equipment control unit 190 communicates signals with the electric lock 300 on the front door 31 and controls the locking and unlocking of the electric lock 300. The equipment control unit 190 includes an electric lock control unit 192 and an electric lock communication unit 194.

The electric lock control unit 192 outputs a signal (unlock instruction) to the electric lock communication unit 194 instructing the electric lock 300 to be unlocked in response to a signal (information) input via the network control unit 170 from the face authentication processing unit 118 or the wireless terminal 600. The electric lock control unit 192 also inputs a signal (information) from the electric lock communication unit 194 indicating that the electric lock 300 has been locked.

The electric lock communication unit 194 transmits the unlock request input from the electric lock control unit 192 to the electric lock 300. The electric lock communication unit 194 also inputs a signal (information) from the electric lock 300 indicating that the electric lock 300 has been locked. The electric lock communication unit 194 is an example of a transmission unit according to the present disclosure.

The one-time password generation unit 195 generates a one-time password when, for example, a request to register the face of the user of the remote wireless terminal 600 is received from the wireless terminal 600 of the user 20 via the network control unit 170. The one-time password generation unit 195 transmits the generated one-time password via the network control unit 170 to the wireless terminal 600 and server 500 that sent the request to register the face.

Note that a control unit of the master unit 100 (not shown) may control at least some of the functional units of the master unit 100 described above, and may also perform functions other than those of these functional units.

<Configuration of the entrance handset>
FIG. 4 is a block diagram showing an example of the configuration of the front door slave unit 200 according to the embodiment of the present disclosure.

The front door sub-unit 200 includes an infrared sensor 210, an image generation unit 220, an audio processing unit 230, an audio output unit 240, an audio input unit 250, and a control unit 260. As described above, the front door sub-unit 200 may be a computer including, for example, a CPU, ROM, RAM, etc. The CPU, for example, reads a program corresponding to the processing content from the ROM and loads it into the RAM, and works in conjunction with the loaded program to control the operation of the front door sub-unit 200.

The infrared sensor 210 detects a visitor in front of the entrance unit 200. The infrared sensor 210 is, for example, a PIR (Passive Infrared Ray) sensor, which is a human presence sensor. When the infrared sensor 210 detects a visitor, it activates the imaging unit 222 of the image generation unit 220, which will be described later.

The video generation unit 220 generates video data of a subject. The subject is, for example, a visitor (person to be authenticated) standing in front of the entrance unit. When the infrared sensor 210 activates the imaging unit 222, the video generation unit 220 first generates video data of the subject for authentication, and then continues to generate video data of the subject, for example, speaking into the entrance unit 200. The video generation unit 220 includes the imaging unit 222, a video data encoding unit 224, and a video data output unit 226.

The imaging unit 222 captures an image of the subject. The imaging unit 222 is, for example, a CMOS camera or a CCD camera. The imaging unit 222 outputs the video data of the captured subject to the video data encoding unit 224.

The video data encoding unit 224 encodes the video data of the subject input from the imaging unit 222 into a streaming format. In this embodiment, the encoding method may be a digital format such as MPEG-2 format, H.264 format, H.265 format, or H.266 format, or an analog format. The video data encoding unit 224 outputs the encoded streaming format video data to the video data output unit 226.

The video data output unit 226 outputs the streaming video data input from the video data encoding unit 224 to the indoor phone master unit 100 (see Figure 3).

The audio processing unit 230 processes a linear electrical signal representing the audio input from the audio input unit 250 and outputs audio data. Furthermore, the audio processing unit 230 processes the audio data input from the indoor phone master unit 100 and outputs a linear electrical signal representing the audio. The audio processing unit 230 includes an audio data input unit 232, an audio data decoding unit 234, an audio data encoding unit 236, and an audio data output unit 238. The audio data input unit 232, the audio data decoding unit 234, the audio data encoding unit 236, and the audio data output unit 238 have the same functions as the audio data input unit 132, the audio data decoding unit 134, the audio data encoding unit 136, and the audio data output unit 138 of the indoor phone master unit 100 (see FIG. 3), respectively, and therefore will not be described here.

The audio output unit 240 is a speaker that converts the linear electrical signal representing the audio input from the audio data decoding unit 234 into audio and outputs it to the outside of the front door unit 200.

The audio input unit 250 is a microphone that converts audio from the area around the entrance unit 200 into a linear electrical signal.

The control unit 260 controls the operation of the video generation unit 220 and the audio processing unit 230. For example, the control unit 260 causes the video generation unit 220 to start outputting video data in response to the infrared sensor 210 detecting a visitor and the activation of the imaging unit 222. Furthermore, for example, the control unit 260 notifies the master unit 100 of an incoming call in response to the depression of a call button (not shown) provided on the front door sub-unit 200, and causes the video generation unit 220 to start outputting video data and the audio processing unit 230 to start outputting audio data. Furthermore, for example, the control unit 260 causes the video generation unit 220 to stop outputting video data and the audio processing unit 230 to stop outputting audio data in response to the detection of the end of the call.

<Configuration of electric lock>
A known electric lock configuration can be adopted as the configuration of the electric lock 300. For example, the electric lock 300 includes a JEM-A adapter equipped with a JEM-A standard HA terminal-A to which one end of the electric lock code 301 is connected, and is configured to switch between unlocked and locked states in response to an input signal to the terminal.

With this configuration, the electric lock 300 can lock and unlock the front door 31 under the control of the door phone master unit 100.

<Configuration of communication router>
The communication router 400 may have the configuration of a known communication router that relays communications.

<Server configuration>
The configuration of the server 500 can be a known server configuration.

The server 500 includes, for example, an authentication processing unit (not shown) that performs authentication processing on a wireless terminal (e.g., wireless terminal 600) that has requested access to the indoor phone base unit 100. The server 500 further includes a communication unit (not shown) that, on the condition that authentication of the wireless terminal 600 is successful, establishes a connection with the wireless terminal 600 and relays communication between the wireless terminal 600 and the indoor phone base unit 100 (via the communication router 400, etc.).

On the other hand, when the indoor phone base unit 100 requests access to a wireless terminal (e.g., wireless terminal 600), the server 500 establishes a connection with the wireless terminal 600 via the communication unit without performing authentication processing, and relays communication between the wireless terminal 600 and the indoor phone base unit 100 (via the communication router 400, etc.).

<Configuration of wireless terminal>
The wireless terminal 600 may have the configuration of a known wireless terminal.

<Operation of each device>
Next, examples of the operation of the main devices constituting the door phone system 10 in several phases will be described.

[Registering a wireless terminal to a door phone master unit]
5 and 6 are flowcharts showing an example of the operation of registering wireless terminal (smartphone) 600 with indoor phone master unit 100 according to an embodiment of the present disclosure. Fig. 5 is a flowchart showing an example of the operation of smartphone 600, and Fig. 6 is a flowchart showing an example of the operation of indoor phone master unit 100. In the following description, it is assumed that smartphone 600 to be registered is a smartphone owned by user 20 who is a resident of building 30.

First, an example of the operation of the smartphone 600 will be described with reference to Figure 5.

In step S501, the smartphone 600 connects to the communication router 400 (connects to the Wi-Fi network).

In step S502, the smartphone 600 downloads an application for utilizing the functions of the door phone system 10, for example, in response to a user operation (such as pressing (tapping) an icon to download the application).

In step S503, the smartphone 600 installs the downloaded application, for example, in response to a user operation (such as pressing (tapping) an icon to install the application).

Note that steps S502 and S503 may be performed before the processing shown in FIG. 5 is executed, in which case steps S502 and S503 are omitted.

In step S504, the smartphone 600 launches the application installed in step S503 or previously installed in response to, for example, a user operation (such as pressing (tapping) an icon to launch the application).

In step S505, in response to a user operation (such as pressing (tapping) an icon to enter registration mode), the smartphone 600 causes the application to enter (transition to) registration mode for registering the smartphone 600 with the indoor phone master unit 100. This causes the smartphone 600 to send a message to the indoor phone master unit 100 via the application indicating that the smartphone 600 is requesting registration from the indoor phone master unit 100.

In step S506, the smartphone 600 starts a timer set to a predetermined time. The predetermined time is, for example, 60 seconds.

In step S507, the smartphone 600 waits for a message from the indoor phone master unit 100 indicating that the smartphone 600 has been registered with the indoor phone master unit 100.

In step S508, the smartphone 600 determines whether a message has been received from the indoor phone master unit 100.

If the smartphone 600 receives the message (step S508: YES), the smartphone 600 successfully pairs with the indoor phone master unit 100 and, for example, displays information indicating that the smartphone 600 has been registered to the indoor phone master unit 100 via an application, and the flow ends. For example, the smartphone 600 successfully pairs by storing (registering) the identification information of the indoor phone master unit 100 in a storage area of the smartphone 600's storage unit associated with the application.

On the other hand, if the smartphone 600 has not received a message (step S508: NO), in step S509, the smartphone 600 determines whether the timer has expired.

If the timer has not expired (step S509: NO), the flow returns to step S507 and the above-described process is repeated.

On the other hand, if the timer expires (step S509: YES), the smartphone 600 displays, for example, via an application, information indicating that registration of the smartphone 600 with the indoor phone master unit 100 failed due to a timeout, and the flow ends.

Next, an example of the operation of the indoor phone master unit 100 will be described with reference to Figure 6.

In step S601, the indoor phone master unit 100 (control unit, communication processing unit 172) connects to the communication router 400 (connects to the Wi-Fi network).

In step S602, the indoor phone master unit 100 (control unit) enters (transitions from) a normal mode in which no operation is being performed to a registration mode for registering the smartphone 600, for example, in response to a user operation (such as a menu selection to enter registration mode).

In step S603, the master unit 100 (control unit) starts a timer set to a predetermined time. The predetermined time is, for example, 60 seconds.

In step S604, the indoor phone master unit 100 (control unit, communication processing unit 172) waits for a message from the smartphone 600 indicating that the smartphone 600 is requesting registration from the indoor phone master unit 100 (the message sent in step S505 in Figure 5).

In step S605, the indoor phone master unit 100 (control unit, communication processing unit 172) determines whether a message has been received from the smartphone 600.

If the intercom master unit 100 receives the message (step S605: YES), in step S607, the intercom master unit 100 (control unit) registers the smartphone 600 (pairing is successful). At this time, the intercom master unit 100 (display output unit 120) may display information to the intercom master unit 100 indicating that the smartphone 600 has been registered. For example, the intercom master unit 100 (control unit) stores (registers) the identification information of the smartphone 600 in a memory area of the memory unit 180 that stores the pairing partner, thereby achieving successful pairing.

In step S608, the indoor phone master unit 100 (control unit, communication processing unit 172) sends a message indicating that the smartphone 600 has been registered (the message that the smartphone 600 waits for in step S507 in FIG. 5) to the smartphone 600. The indoor phone master unit 100 (control unit) then enters (transitions to) normal mode, and the flow ends.

On the other hand, if the indoor phone base unit 100 has not received a message (step S605: NO), in step S606, the indoor phone base unit 100 (control unit) determines whether the timer has expired.

If the timer has not expired (step S606: NO), the flow returns to step S604 and the above-described process is repeated.

On the other hand, if the timer expires (step S606: YES), the indoor phone base unit 100 (display output unit 120) displays information indicating that registration of the smartphone 600 to the indoor phone base unit 100 failed due to a timeout, and the indoor phone base unit 100 (control unit) enters (transitions to) normal mode, and the flow ends.

[Registering the face of the person to be authenticated to the door phone base unit]
7 and 8 are flowcharts illustrating an example of a face registration operation for the indoor phone base unit 100 according to an embodiment of the present disclosure. FIG. 7 is a flowchart illustrating an example of the operation of the smartphone 600, and FIG. 8 is a flowchart illustrating an example of the operation of the indoor phone base unit 100. The face registration operation illustrated in FIGS. 7 and 8 is an example performed by a user 20, who is a resident of a building 30, using the smartphone 600 carried by the user 20 within the building 30 (in front of the indoor phone base unit 100). However, the person to be registered and the smartphone 600 are not limited to the user 20 and the smartphone carried by the user 20, respectively. It is assumed in FIGS. 7 and 8 that the smartphone 600 and the indoor phone base unit 100 are already connected. It is also assumed in FIGS. 7 and 8 that no communication error occurs between the smartphone 600 and the indoor phone base unit 100 due to a timeout or the like.

First, an example of the operation of the smartphone 600 will be described with reference to Figure 7.

In step S701, the smartphone 600 launches the application installed in or before S503 in FIG. 5 in response to, for example, a user operation (such as pressing (tapping) an icon to launch the application).

In step S702, in response to a user operation (such as pressing (tapping) an icon to enter face authentication registration mode), the smartphone 600 causes the application to enter (transition to) face authentication registration mode, which allows the face of the person to be authenticated to be registered in the indoor phone base unit 100. As a result, the smartphone 600 sends a message to the indoor phone base unit 100 via the application indicating that the smartphone 600 is requesting face registration from the indoor phone base unit 100.

In step S703, the smartphone 600 waits for a message from the indoor phone master unit 100 indicating that the indoor phone master unit 100 has entered (transitioned to) face authentication registration mode (a message sent in step S803 in Figure 8, which will be described later).

In step S704, the smartphone 600 determines whether a message has been received from the indoor phone master unit 100.

If the smartphone 600 has not received a message (step S704: NO), the flow returns to step S703 and the above-described processing is repeated.

On the other hand, if the smartphone 600 receives a message (step S704: YES), in step S705, the smartphone 600 activates the front camera via the application and causes the front camera to capture video.

In step S706, the smartphone 600 transmits the video data to the indoor phone master unit 100 via the application.

In step S707, the smartphone 600 presents guidance for facial authentication registration through the application. For example, the smartphone 600 plays audio guidance registered in the application requesting that images of the user 20's face be taken from various angles.

In step S708, the smartphone 600 performs face authentication registration (capturing video data including the face and transmitting it to the indoor phone master unit 100) via the application in accordance with the guidance. The flow then ends.

Next, an example of the operation of the indoor phone master unit 100 will be described with reference to Figure 8.

In step S801, the indoor phone base unit 100 (control unit, communication processing unit 172) waits for a message from the smartphone 600 indicating that the smartphone 600 is requesting face registration from the indoor phone base unit 100 (the message sent in step S702 in Figure 7).

In step S802, the indoor phone master unit 100 (control unit, communication processing unit 172) determines whether a message has been received from the smartphone 600.

If the indoor phone master unit 100 has not received a message (step S802: NO), the flow returns to step S801 and the above-described processing is repeated.

On the other hand, if the intercom master unit 100 receives the message (step S802: YES), in step S803, the intercom master unit 100 (control unit) enters (transitions) from steady-state mode to face authentication registration mode for registering the face of the person to be authenticated. As a result, the intercom master unit 100 (control unit, communication processing unit 172) sends a message to the smartphone 600 indicating that the intercom master unit 100 has entered (transitioned) to face authentication registration mode.

In step S804, the indoor apparatus 100 (control unit, communication processing unit 172) waits for video data from the smartphone 600 (video data transmitted in step S706 in FIG. 7).

In step S805, the indoor phone master unit 100 (control unit, communication processing unit 172) determines whether video data has been received from the smartphone 600.

If the indoor phone master unit 100 has not received video data (step S805: NO), the flow returns to step S804 and the above-described processing is repeated.

On the other hand, if the intercom master unit 100 receives video data (step S805: YES), in step S806, the intercom master unit 100 (display output unit 120) displays video based on the received video data.

In step S807, the intercom master unit 100 (display output unit 120, audio output unit 140) presents guidance for facial authentication registration. For example, the intercom master unit 100 plays audio guidance registered in the intercom master unit 100 requesting that the face of the user 20 be photographed from various directions.

In step S808, the indoor phone base unit 100 (communication processing unit 172, face authentication processing unit 118) performs face authentication registration (receives video including a face, and detects, extracts, and registers the face) in accordance with the guidance. The indoor phone base unit 100 (control unit) then enters (transitions to) normal mode, and the flow ends.

[Remote registration of wireless terminal to door phone base unit]
9A and 9B are sequence diagrams illustrating an example of the operation of remotely registering a wireless terminal (smartphone) 600′ to indoor phone master 100 according to an embodiment of the present disclosure. In one example, smartphone 600′ to be registered (hereinafter referred to as remote smartphone 600′) is a smartphone owned by the mother of user 20, who is a resident of building 30, but is not limited thereto. Note that the resident smartphone 600 illustrated in FIGS. 9A and 9B will be described as a smartphone owned by user 20.

In step S901, the power to the indoor phone master unit 100 is turned on.

In step S902, the indoor phone master unit 100 (control unit, communication processing unit 172) is constantly connected to the server 500 via the communication router 400, the public network 510, etc.

In step S903, the resident smartphone 600 launches the application installed in or before S503 of Figure 5.

In step S904, the resident smartphone 600 selects application sharing to share the application, for example, in response to a user operation (such as pressing (tapping) an icon to share the application). As a result, the resident smartphone 600 sends a message to the intercom master 100 via the application indicating the presence of a remote wireless terminal 600 (smartphone 600') that is requesting registration with the intercom master 100.

When the intercom master unit 100 (control unit, communication processing unit 172) receives a message from the resident smartphone 600, in step S905, the intercom master unit 100 (control unit) enters (transitions) from steady-state mode to remote registration mode for registering the remote smartphone 600'.

In step S906, the indoor phone master unit 100 (control unit) obtains its own device ID from the internal information stored in the memory unit 180.

In step S907, the indoor phone master unit 100 (one-time password generation unit 195) generates a one-time password.

In step S908, the indoor phone master unit 100 (control unit, communication processing unit 172) transmits the device ID and one-time password to the resident smartphone 600 and the server 500.

When the resident smartphone 600 receives the device ID and one-time password from the indoor phone master unit 100 via the application, in step S909, the resident smartphone 600 launches an arbitrary messaging application, for example, in response to a user operation (such as pressing (tapping) an icon to launch a messaging application).

In step S910, the resident smartphone 600 transmits the application download URL, device ID, and one-time password entered by the user 20, for example, to the remote smartphone 600' via a messaging application in response to, for example, a user operation (such as pressing (tapping) an icon to send a message).

When the remote smartphone 600' receives the application download URL, device ID, and one-time password from the resident smartphone 600 via the messaging application, in step S911, the remote smartphone 600' downloads the application from the download URL, for example, in response to a user operation (such as pressing (tapping) the download URL link).

In step S912, the remote smartphone 600' installs the downloaded application, for example, in response to a user operation (such as pressing (tapping) an icon to install the application).

In step S913, the remote smartphone 600' launches the installed application in response to, for example, a user operation (such as pressing (tapping) an icon to launch the application).

In step S914, the remote smartphone 600' causes the application to enter (transition to) remote registration mode, for example, in response to a user operation (such as pressing (tapping) an icon to enter remote registration mode), in order to register the remote smartphone 600' with the indoor phone master unit 100.

In step S915, the remote smartphone 600' transmits the device ID and one-time password entered by the user of the remote smartphone 600' to the server 500 via the application.

In step S916, the server 500 checks whether the device ID and one-time password received from the remote smartphone 600' match the device ID and one-time password sent and received from the indoor phone master unit 100 in step S908, respectively.

If there is a match, in step S917, the server 500 associates and registers the indoor phone master unit 100 and the remote smartphone 600'. (If there is no match, the server 500 sends an error message to the remote smartphone 600'.)

In step S918, the server 500 sends a registration success message to the intercom master unit 100 and the remote smartphone 600' indicating that the intercom master unit 100 and the remote smartphone 600' have been associated and registered.

When the indoor phone base unit 100 (control unit, communication processing unit 172) receives a registration success message from the server 500, in step S919, the indoor phone base unit 100 (control unit) successfully pairs with the remote smartphone 600'. Similarly, when the remote smartphone 600' receives a registration success message from the server 500 via the application, in step S920, the remote smartphone 600' successfully pairs with the indoor phone base unit 100.

[Remote face registration to the door phone base unit]
9B , after the pairing between indoor apparatus 100 and remote smartphone 600′ is successful, remote smartphone 600′ can connect to (access) indoor apparatus 100 via server 500. After the pairing and connection, indoor apparatus 100 can register the face of a person based on the image of the person captured by the front camera of remote smartphone 600′ via server 500, as described with reference to FIGS.

Figures 10A to 10D are diagrams showing examples of display screens of wireless terminals (smartphones) 600, 600' according to an embodiment of the present disclosure.

The display screen shown in Figure 10A is displayed when application sharing is selected in step S904 of Figure 9A. The user 20 of the resident smartphone 600 can place the indoor phone master unit 100 into remote registration mode by pressing (tapping) the icon 1001 for sharing the application on the application.

The display screen shown in Figure 10B is displayed when the download URL, etc. is sent in step S910 of Figure 9A. The user 20 of the resident smartphone 600 can notify the user of the remote smartphone 600' of the download URL, device ID, and one-time password by entering and sending them on the messaging application.

The display screen shown in FIG. 10C is displayed when the application is placed into remote registration mode in step S914 of FIG. 9B. The user of the remote smartphone 600' can place the application into remote registration mode by pressing (tapping) the icon 1002 for entering remote registration mode on the application.

The display screen shown in FIG. 10D is displayed when the device ID and one-time password are sent in step S915 of FIG. 9B. The user of the remote smartphone 600' can send the device ID and one-time password to the server 500 by entering the device ID and one-time password in the device ID input field 1003 and one-time password input field 1004, respectively, and pressing (tapping) icon 1005 on the application.

As a result, the user of the remote smartphone 600' can unlock the electric lock 300 without using a physical key.

[Actions involving facial recognition when visiting]
11A, 11B, 12A to 12C, and 13 are flowcharts illustrating an example of an operation involving facial authentication when a visitor arrives, according to an embodiment of the present disclosure. FIGS. 11A and 11B are flowcharts illustrating an example of an operation of front door slave unit 200, FIGS. 12A to 12C are flowcharts illustrating an example of an operation of indoor phone master unit 100, and FIG. 13 is a flowchart illustrating an example of an operation of smartphone 600. The operations illustrated in FIGS. 11A, 11B, 12A to 12C, and 13 are examples performed using smartphone 600 carried by user 20, who is a resident of building 30, when user 20 is outside (going out) of building 30.

First, an example of the operation of the front door slave unit 200 will be described with reference to Figures 11A and 11B.

In step S1101, the front door unit 200 (infrared sensor 210) waits for a person (visitor) to stand in front of the front door unit 200.

In step S1102, the front door sub-unit 200 (infrared sensor 210) determines whether it has detected a person standing in front of the front door sub-unit 200.

If the front door unit 200 (infrared sensor 210) does not detect a person (step S1102: NO), the flow returns to step S1101 and the above-described processing is repeated.

On the other hand, if the front door sub-unit 200 (infrared sensor 210) detects a person (step S1102: YES), in step S1103, the front door sub-unit 200 (infrared sensor 210) activates the camera (imaging unit) 222.

In step S1104, the front door sub-unit 200 (control unit 260) instructs the main unit 100 to enter face authentication mode (sends an instruction to the main unit 100 to enter face authentication mode).

In step S1105, the front door slave unit 200 (control unit 260) starts a timer set to a predetermined time. The predetermined time is, for example, 60 seconds.

In step S1106, the front door slave unit 200 (camera 222) captures video of the surrounding area and transmits it to the indoor phone master unit 100.

In step S1107, the front door unit 200 (control unit 260) determines whether the call button has been pressed, for example, within a predetermined period of time (e.g., 15 seconds) after step S1104.

If the call button has been pressed (step S1107: YES), in step S1108, the front door sub-unit 200 (control unit 260) sends a signal (information) (visitor notification) indicating that the call button has been pressed to the indoor phone master unit 100. The flow then proceeds to step S1111.

On the other hand, if the call button has not been pressed (step S1107: NO), in step S1109, the front door sub-unit 200 (control unit 260) receives an instruction from the indoor phone master unit 100 to sound an error message (an instruction sent in step S1210 in Figure 12B, described below).

In step S1110, the front door unit 200 (audio output unit 240) sounds an error message indicating that the resident is being called. For example, the error message may be a message such as "Authentication failed, so a call is being made."

In step S1111, the front door sub-unit 200 (control unit 260) waits for a message from the indoor phone master unit 100 indicating that the front door sub-unit 200 and the smartphone 600 of the user 20 registered with the indoor phone master unit 100 will be placed into a call state (a message sent in step S1216 in Figure 12C, described below).

In step S1112, the front door slave unit 200 (control unit 260) determines whether a message has been received from the indoor phone master unit 100.

If the front door slave unit 200 (control unit 260) receives the message (step S1112: YES), in step S1113, the front door slave unit 200 (control unit 260, etc.) makes a call (communicates) with the smartphone 600 via the indoor phone master unit 100, exchanging video and audio with the smartphone 600 (however, video is not received). Then, when the connection between the indoor phone master unit 100 and the smartphone 600 is disconnected, the flow ends.

On the other hand, if the front door sub-unit 200 (control unit 260) has not received a message (step S1112: NO), in step S1114, the front door sub-unit 200 (control unit 260) determines whether the timer has expired.

If the timer has not expired (step S1114: NO), the flow returns to step S1111 and the above-described process is repeated.

On the other hand, if the timer expires (step S1114: YES), the flow ends.

Next, an example of the operation of the indoor phone master unit 100 will be described with reference to Figures 12A to 12C.

In step S1201, the indoor apparatus 100 (control unit, communication processing unit 172) waits for an instruction from the front door sub-unit 200 to enter face authentication mode (the instruction sent in step S1104 in Figure 11A).

In step S1202, the indoor phone master unit 100 (control unit, communication processing unit 172) determines whether or not an instruction has been received from the front door slave unit 200.

If the indoor phone master unit 100 has not received the instruction (step S1202: NO), the flow returns to step S1201 and the above-described processing is repeated.

On the other hand, if the intercom master unit 100 receives an instruction message (step S1202: YES), in step S1203, the intercom master unit 100 (control unit) enters (transitions from) normal mode to face authentication mode, which performs face authentication using a face captured by the camera 222 of the front door sub-unit 200.

In step S1204, the indoor phone master unit 100 (control unit, video data input unit 112) receives the video captured by the camera 222 from the front door slave unit 200 (the video transmitted in step S1106 in Figure 11A).

In step S1205, the indoor phone master unit 100 (face recognition processing unit 118) performs face recognition processing by comparing the face contained in the received video with the faces registered in the registration database of the face recognition memory unit 116.

In step S1206, the indoor phone master unit 100 (face authentication processing unit 118) determines whether authentication was successful as a result of the face authentication process.

If authentication is successful (step S1206: YES), in step S1207, the intercom master unit 100 (face authentication processing unit 118, communication processing unit 172, electric lock control unit 192, electric lock communication unit 194) instructs the electric lock 300 to unlock (sends an unlock instruction to the electric lock 300). The electric lock 300 then unlocks, the intercom master unit 100 (control unit) enters (transitions to) normal mode, and the flow ends. Note that after instructing the electric lock 300 to unlock or after the electric lock 300 is unlocked, the intercom master unit 100 (control unit, communication processing unit 172) may notify a specific smartphone 600 registered with the intercom master unit 100 that the electric lock 300 has been unlocked. At that time, the intercom master unit 100 (control unit, communication processing unit 172) may send an image including the person's face, captured by the camera 222 of the front door sub-unit 200, to the smartphone 600.

On the other hand, if authentication is not successful (step S1206: NO), in step S1208, the indoor apparatus 100 (control unit) determines whether the call button on the front door unit 200 has been pressed. The indoor apparatus 100 (control unit) makes this determination by determining whether the visitor notification sent by the front door unit 200 in step S1108 of FIG. 11B has been received within a predetermined period of time (e.g., 15 seconds) after step S1203.

If the call button on the front door unit 200 is pressed (step S1208: YES), in step S1209, the master unit 100 (control unit, communication processing unit 172) notifies a specific smartphone 600 registered with the master unit 100 that the call button has been pressed (a signal indicating that the call button has been pressed is sent to the smartphone 600).

On the other hand, if the call button on the front door sub-unit 200 has not been pressed (step S1208: NO), in step S1210, the master unit 100 (control unit) instructs the front door sub-unit 200 to sound an error message (sends an instruction to sound an error message to the front door sub-unit 200).

In step S1211, the indoor phone base unit 100 (control unit, communication processing unit 172) notifies a specific smartphone 600 registered with the indoor phone base unit 100 that facial authentication has failed (a signal indicating that facial authentication has failed is sent to the smartphone 600).

In step S1212, the indoor apparatus 100 (control unit) starts a timer set to a predetermined time. The predetermined time is, for example, 10 seconds.

In step S1213, the indoor phone master unit 100 (control unit, communication processing unit 172) waits for a message from the smartphone 600 indicating that a call (communication) with the front door slave unit 200 will be made (the message sent in step S1303 in Figure 13, which will be described later).

In step S1214, the indoor phone master unit 100 (control unit, communication processing unit 172) determines whether a message has been received from the smartphone 600.

If the indoor phone master unit 100 receives a message (step S1214: YES), the flow proceeds to step S1216.

On the other hand, if the indoor phone master unit 100 has not received a message (step S1214: NO), in step S1215, the indoor phone master unit 100 (control unit) determines whether the timer has expired.

If the timer has not expired (step S1215: NO), the flow returns to step S1213 and the above-described process is repeated.

On the other hand, if the timer expires (step S1215: YES), the indoor phone master unit 100 (control unit) enters (transitions to) steady-state mode, and the flow ends.

In step S1216, the indoor phone master unit 100 (control unit) sends a message to the front door sub-unit 200 indicating that the front door sub-unit 200 and the smartphone 600 are to be placed in a call state, thereby placing the front door sub-unit 200 and the smartphone 600 in a call state.

In step S1217, the indoor phone base unit 100 (control unit, communication processing unit 172) receives from the smartphone 600 a signal indicating a user operation by the smartphone 600 user on the smartphone 600. The user operation may be a selection (instruction) to unlock the electric lock 300, a selection (instruction) to lock the electric lock 300, or a selection (instruction) to disconnect the call with the indoor phone base unit 100 (and therefore the front door sub-unit 200).

In step S1218, the indoor phone master unit 100 (control unit) determines the user operation.

If the user operation selects unlocking the electric lock 300, in step S1219, the indoor phone master unit 100 (control unit, communication processing unit 172, electric lock control unit 192, electric lock communication unit 194) instructs the electric lock 300 to unlock (sends an unlock instruction to the electric lock 300). The electric lock 300 then unlocks, and the flow returns to step S1217.

If the user operation is to select locking of the electric lock 300, in step S1220, the indoor phone master unit 100 (control unit, communication processing unit 172, electric lock control unit 192, electric lock communication unit 194) instructs the electric lock 300 to lock (sends a lock instruction to the electric lock 300). The electric lock 300 then locks, and the flow returns to step S1217.

If the user operation is to select disconnection of the call with the indoor phone base unit 100, in step S1221, the indoor phone base unit 100 (control unit, communication processing unit 172) disconnects the call with the smartphone 600. The indoor phone base unit 100 (control unit) then enters (transitions to) steady mode, and the flow ends.

Next, an example of the operation of the smartphone 600 will be described with reference to Figure 13.

In step S1301, the smartphone 600 receives a notification from the indoor apparatus 100 through an application installed in or before S503 of FIG. 5. This notification may be the notification in step S1209 of FIG. 12B or the notification in step S1211 of FIG. 12B.

In step S1302, the smartphone 600 determines whether the user of the smartphone 600 will respond to the notification. For example, the smartphone 600 determines that the user will respond if the user chooses to respond by pressing (tapping) an icon for responding to the notification, which is valid for a predetermined time (e.g., 5 seconds) after receiving the notification.

If the user does not respond (step S1302: NO), the flow ends.

On the other hand, if the user responds (step S1302: YES), in step S1303, the smartphone 600 sends a message to the indoor phone master unit 100 via the application indicating that it will make a call (communicate) with the front door slave unit 200.

When a call with the intercom master unit 100 (and therefore the front door slave unit 200) becomes possible, in step S1304, the smartphone 600 initiates a call with the front door slave unit 200 via the intercom master unit 100 through an application. The smartphone 600 can then exchange video and audio with the front door slave unit 200 (however, video does not have to be transmitted).

In step S1305, the smartphone 600 displays, in addition to the image, an icon for instructing the user to unlock the electric lock 300, an icon for instructing the user to lock the electric lock 300, and an icon for instructing the user to disconnect the call with the indoor phone master unit 100 on the application.

In step S1306, the smartphone 600 determines the user operation performed by the user of the smartphone 600 on the smartphone 600.

If the user operation is the selection of an icon to instruct unlocking the electric lock 300, in step S1307, the smartphone 600 transmits a signal (information) to the indoor phone master unit 100 indicating that unlocking of the electric lock 300 has been selected (instructed). The flow then returns to step S1305.

If the user operation is the selection of an icon to instruct locking of the electric lock 300, in step S1308, the smartphone 600 transmits a signal (information) to the indoor phone master unit 100 indicating that locking of the electric lock 300 has been selected (instructed). The flow then returns to step S1305.

If the user operation is the selection of an icon to instruct the disconnection of the call with the indoor phone base unit 100, in step S1309, the smartphone 600 sends a signal to the indoor phone base unit 100 indicating that the disconnection of the call with the indoor phone base unit 100 has been selected (instructed).

In step S1310, the smartphone 600 disconnects the call with the indoor phone master unit 100. The flow then ends.

Figure 14 is a diagram showing an example of a display screen of a wireless terminal (smartphone) 600 according to an embodiment of the present disclosure. This display screen is displayed, for example, in step S1305 of Figure 13.

As shown in FIG. 14, this display screen includes an image 1201 captured by the camera 222 of the front door sub-unit 200, an icon 1202 for instructing the user to unlock the electric lock 300, an icon 1203 for instructing the user to lock the electric lock 300, and an icon 1204 for instructing the user to disconnect the call with the main door phone unit 100.

Using the microphone of the smartphone 600, the user 20 can converse with a person in the video 1201 who is in front of the front door sub-unit 200 via the intercom master unit 100. Such people include people whose faces are not registered in the intercom master unit 100, such as delivery people or the mother of the smartphone 600 user 20, as well as people whose faces are registered in the intercom master unit 100.

Furthermore, user 20 of smartphone 600 can unlock electric lock 300 by pressing (tapping) icon 1202 and selecting to unlock electric lock 300 (steps S1306 → S1307 in FIG. 13, steps S1218 → S1219 in FIG. 12C). For example, if the person in image 1201 standing in front of entrance sub-unit 200 is the mother of user 20 of smartphone 600, the user can unlock electric lock 300 even if user 20 is out and the mother does not have the physical key. Furthermore, even if the person's face is registered in door phone master unit 100, authentication may fail due to reasons such as wearing a mask. In such cases, user 20 can still unlock electric lock 300.

Furthermore, the user 20 of the smartphone 600 can lock the electric lock 300 by pressing (tapping) the icon 1203 and selecting to lock the electric lock 300 (steps S1306 → S1308 in Figure 13, steps S1218 → S1220 in Figure 12C).

Furthermore, the user 20 of the smartphone 600 can disconnect the call with the indoor phone base unit 100 by pressing (tapping) the icon 1204 and selecting to disconnect the call with the indoor phone base unit 100 (steps S1306 → S1309 → S1310 in Figure 13, steps S1218 → S1221 in Figure 12C).

<Modification of the embodiment>
At least some of the functions of indoor phone master unit 100 (functional units shown in FIG. 3 ) may be provided in server 500 or another server. In this case, indoor phone master unit 100 does not need to perform processing corresponding to the function, but simply transfers various data to server 500 or another server, and server 500 or another server performs the processing corresponding to the function. This reduces the cost of indoor phone master unit 100 and the processing load on indoor phone master unit 100.

The above describes an example of authentication using a person's face, but other biometric authentication methods may also be applied. For example, a contactless authentication method such as authentication using the iris, voice, or gait may be applied, or a non-contactless biometric authentication method such as authentication using veins or fingerprints may be applied. Therefore, instead of a face as authentication (biometric) data, the extraction unit according to the present disclosure may extract the iris, etc. from a video or image transmitted from the wireless terminal 600, and the extraction unit according to the present disclosure may register the iris, etc.

The above-mentioned functional units may be divided into further functional units, or two or more functional units may be integrated into a single functional unit. For example, the face authentication processing unit 118 may be divided into a registration unit that registers authentication data and a functional unit that performs the actual authentication process based on the video captured by the camera of the entrance door unit 200 and the registered authentication data, and this functional unit may be further divided into sub-functional units depending on the function to be executed.

The intercom master unit 100 may be realized as a single device having all of the above-mentioned functional units, or as multiple devices in which the above-mentioned functional units are distributed. If the intercom master unit 100 is realized as multiple devices, the functions of the intercom master unit 100 can be realized by the communication units of the multiple devices cooperating over a network.

<Effects of the embodiment>
As described above, according to the present embodiment, indoor phone master unit 100, which can unlock front door 31, receives, via communication processing unit 172 (receiving unit), an image of a person captured by the in-camera of wireless terminal 600 from wireless terminal 600. In addition, indoor phone master unit 100 extracts, via face recognition processing unit 118 (extraction unit), the person's face, iris, etc. (biometric data for authentication) to be used to unlock front door 31 from the captured image of the person, and then registers, via face recognition processing unit 118 (registration unit), the extracted face, iris, etc. in the registration database of face recognition storage unit 116.

Accordingly, according to this embodiment, the authentication data of a person is registered using the in-camera of the wireless terminal 600, without using the camera provided in the entrance slave unit 200, thereby minimizing any inconvenience to that person.

<Summary of the embodiment>
A door phone master unit (door phone master unit 100) according to one embodiment of the present disclosure is capable of unlocking a front door (front door 31), and includes a receiving unit (communication processing unit 172) that receives from a wireless mobile terminal (wireless terminal 600, 600') an image of a person captured by a camera possessed by the wireless mobile terminal, an extraction unit (face recognition processing unit 118) that extracts from the image biometric data (face, iris, etc.) of the person for authentication used to unlock the front door, and a registration unit (face recognition processing unit 118) that registers the biometric data for authentication.

With the above configuration, a person's biometric data for authentication is registered using the camera on the wireless mobile terminal, without using the camera on the intercom handset, thereby minimizing any inconvenience to that person.

In this intercom master unit, when the receiving unit receives the image of the person, the intercom master unit and the wireless mobile terminal belong to the same wireless network via a communication router (communication router 400).

With the above configuration, a person photographed by the camera of a wireless mobile terminal can register their biometric data for authentication while inside the building where the intercom master unit is located, thereby minimizing any inconvenience to that person.

In this intercom base unit, when the receiving unit receives the image of the person, the intercom base unit and the wireless mobile terminal are directly connected.

With the above configuration, a person photographed by the camera of a wireless mobile terminal can register their biometric data for authentication while inside the building where the intercom master unit is located, thereby minimizing any inconvenience to that person.

In this intercom master unit, when the receiving unit receives the image of the person, the intercom master unit and the wireless mobile terminal are connected via a server (server 500) connected to a public network (public network 510).

With the above configuration, a person photographed by the camera on a wireless mobile terminal can register their biometric data for authentication even if they are away from the building where the intercom master unit is located, thereby minimizing any inconvenience to that person.

This intercom master unit further includes an authentication unit (face authentication processing unit 118) that authenticates the person by comparing biometric data (face, iris, etc.) based on an image of the person taken by the camera (camera 222) of the intercom slave unit (front door slave unit 200) with the authentication biometric data, and a transmission unit (electric lock communication unit 194) that, if authentication is successful, sends a signal to the electric lock (300) on the front door instructing it to unlock the electric lock.

With the above configuration, if authentication is successful, the electric lock (front door) will be unlocked, ensuring security. Furthermore, since the electric lock (front door) can be unlocked even without a physical key, convenience for the person can be improved.

A registration method according to one embodiment of the present disclosure is executed by an intercom master unit (intercom master unit 100) capable of unlocking a front door (front door 31), receives an image of a person captured by a camera installed in a wireless mobile terminal (wireless terminal 600, 600') from the wireless mobile terminal, extracts from the image biometric data (face, iris, etc.) of the person for authentication used to unlock the front door, and registers the biometric data for authentication.

With the above configuration, a person's biometric data for authentication is registered using the camera on the wireless mobile terminal, without using the camera on the intercom handset, thereby minimizing any inconvenience to that person.

One embodiment of the present disclosure is useful as a door phone master unit and registration method used in a door phone system.

10 Doorphone system 20 User 30 Building 31 Entrance door 100 Doorphone master unit 110 Video processing unit 112 Video data input unit 114 Video data decoding unit 116 Face authentication storage unit 118 Face authentication processing unit 120 Display output unit 130 Audio processing unit 132 Audio data input unit 134 Audio data decoding unit 136 Audio data encoding unit 138 Audio data output unit 140 Audio output unit 150 Audio input unit 160 Call control unit 162 Incoming call processing unit 164 Timer processing unit 166 Call processing unit 170 Network control unit 172 Communication processing unit 180 Storage unit 190 Facility equipment control unit 192 Electric lock control unit 194 Electric lock communication unit 200 Entrance slave unit 210 Infrared sensor 220 Video generation unit 222 Imaging unit 224 Video data encoding unit 226 Video data output unit 230 Audio processing unit 232 Audio data input unit 234 Audio data decoding unit 236 Audio data encoding unit 238 Audio data output unit 240 Audio output unit 250 Audio input unit 260 Control unit 300 Electric lock 301 Electric lock code 400 Communication router 500 Server 510 Public network 401 Wireless communication line 402 Wireless communication line 403 Wireless communication line 600 Wireless terminal 600' Wireless terminal 601 Wireless communication line

Claims (6)

A door phone master unit capable of unlocking a front door,
a receiving unit that receives an image of a person taken by a camera included in the wireless mobile terminal from the wireless mobile terminal;
an extraction unit that extracts, from the image, biometric data for authentication of the person that is used to unlock the front door;
a registration unit that registers the authentication biometric data;
a network processing unit that controls communication with the wireless mobile terminal;
Equipped with
When the network processing unit receives a message from the wireless mobile terminal indicating that the authentication biometric data has been registered, the indoor phone master unit transitions from a normal mode to a face authentication registration mode,
the network processing unit transmits a message indicating that the indoor apparatus has transitioned to the face authentication registration mode to the wireless mobile terminal;
the receiving unit receives the image from a camera that is activated after the wireless mobile terminal receives the message;
the extraction unit extracts the authentication biometric data from the image.
Door phone main unit. When the receiving unit receives the image of the person, the indoor phone master unit and the wireless mobile terminal belong to the same wireless network via a communication router.
The door phone master unit according to claim 1. When the receiving unit receives the image of the person, the indoor phone master unit and the wireless mobile terminal are directly connected.
The door phone master unit according to claim 1. When the receiving unit receives the image of the person, the indoor phone master unit and the wireless mobile terminal are connected via a server connected to a public network.
The door phone master unit according to claim 1. an authentication unit that authenticates the person by comparing biometric data based on an image of the person taken by a camera included in the intercom slave unit with the authentication biometric data;
a transmitter that transmits a signal to the electric lock of the entrance door instructing the electric lock to unlock the electric lock if the authentication is successful;
The indoor phone master unit according to claim 1 , further comprising: A registration method executed by a door phone master unit capable of unlocking a front door,
receiving, from the wireless mobile terminal, an image of a person photographed by a camera included in the wireless mobile terminal;
extracting biometric data from the image for authentication of the person to be used to unlock the front door;
registering the biometric data for authentication;
Controlling communication with the wireless mobile terminal;
When a message indicating that the authentication biometric data has been registered is received from the wireless mobile terminal, the mode transitions from the normal mode to the face authentication registration mode,
transmitting a message indicating that the mode has been transitioned to the face authentication registration mode to the wireless mobile terminal;
receiving the image from a camera activated after the wireless mobile terminal receives the message;
extracting the authentication biometric data from the image;
How to register.