JP6931757B2 - Signage devices, display methods, and programs - Google Patents

Description

The present technology relates to a signage device, a display method, and a program, and more particularly to a signage device, a display method, and a program in which a user area for displaying information suitable for each user is provided on a display.

Digital signage, which can transmit information by displaying information such as maps, facility guide maps, and advertisements on a large display, is becoming widespread in facilities such as commercial buildings, tourist spots, stations, and airports.

In particular, in recent years, due to the combination of larger displays, improved information processing, faster communication networks, and the spread of wireless LANs, there are various new information transmission mechanisms using digital signage (hereinafter referred to as signage devices). Proposals have been made.

For example, in Patent Document 1, a user's face is registered in advance, and when a person approaches the signage device, the person's face is authenticated and it can be confirmed that the user is a registered user in advance. Inventions that display information suitable for the user have been proposed.

Japanese Unexamined Patent Publication No. 2011-118782

This technology was made in view of such a situation, and as a new usage method of the signage device, a user area dedicated to each user is provided on the display so that information suitable for each user can be displayed. be.

The signage device, which is the first aspect of the present technology, has a display for displaying information, a detection unit for detecting a user existing in front of the display based on an image captured in front of the display, and an image. A detection unit that detects at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the image, and a contour of the user in which at least one of the ID information or the user attribute is detected. based on the user area setting unit for setting a user area on the screen of the display, the background image generation section that generates a background image for displaying a map image on the display, based on the user attribute of the user, the It comprises a user area image generating unit that generates a map image that is written in the user's language as a user region image to be displayed on the user area, and a superimposing unit that superimposes the user area image into the background image.
The signage device, which is the second aspect of the present technology, has a display for displaying information, a detection unit for detecting a user existing in front of the display based on an image captured in front of the display, and an image. A detection unit that detects at least one of the detected ID information or user attributes indicated by the user's belongings based on the image, a communication unit that connects to a server that supplies the user's behavior data, and the ID. Based on the information or the outline of the user in which at least one of the user attributes is detected, a user area setting unit that sets the user area on the screen of the display and a background image for displaying the map image on the display are generated. A user area image generated as a user area image in which a background image generation unit and a map image to which information indicating the user's destination or the direction of the destination is added based on the user's behavior data are displayed in the user area. It includes a generation unit and a superimposing unit that superimposes the user area image on the background image.

The signage device, which is one aspect of the present technology, may further include a detection unit that detects at least one of the detected ID information or user attribute notified from the user side.

The signage device, which is one aspect of the present technology, can further include a user authentication processing unit that performs authentication processing of the detected user based on the detected pre-ID information.
When the authentication process is successful, the user area setting unit can set the user area on the screen of the display based on the contour of the user who has succeeded in the authentication process.

The detection unit can detect at least one of the ID information or the user attribute notified from the user side by the blinking pattern of the light emission of the possessed item based on the captured image.

The detection unit can detect at least one of the ID information or the user attribute based on the image information of the possessed item based on the captured image.

Before SL user area setting unit, to correspond to a plurality of users at least one of which is detected in the ID information or the user attribute setting each said user area, the user area image generating unit, a plurality of said user Each of the user area images to be displayed in each area is generated, and the superimposing unit can superimpose a plurality of the user area images on the background image.

When the plurality of users satisfy a predetermined condition, the user area setting unit can consider the plurality of user areas to correspond to one of the plurality of users.

The display method, which is the first aspect of the present technology, is a display method of a signage device provided with a display for displaying information, in front of the display based on an image obtained by the signage device in front of the display. A detection step for detecting an existing user, a detection step for detecting at least one of the detected ID information or user attribute indicated by the user's belongings based on the captured image, and the ID information or the said A user area setting step for setting a user area on the screen of the display based on the contour of the user in which at least one of the user attributes is detected, and a background image generation for generating a map image as a background image to be displayed on the display. A user area image generation step of generating a map image expressed in the language used by the user as a user area image to be displayed in the user area based on the user attribute of the user, and the user on the background image. Includes a superimposition step of superimposing region images.

The program, which is the first aspect of the present technology, has a detection unit that detects a user existing in front of the display based on an image of the computer in front of the display displaying information, and the captured image. Based on the detection unit that detects at least one of the ID information or the user attribute indicated by the detected personal belongings of the user, and the contour of the user in which at least one of the ID information or the user attribute is detected. Te, a user area setting unit for setting a user area on the screen of the display, the background image generation section that generates a background image for displaying a map image on the display, based on the user attribute of the user, the user It functions as a user area image generation unit that generates a map image expressed in the language used as a user area image to be displayed in the user area, and a superimposition unit that superimposes the user area image on the background image.

In the first aspect of the present technology, a user existing in front of the display is detected based on an image captured in front of the display displaying information, and the detected image is based on the captured image. A user area is set on the screen of the display based on the contour of the user in which at least one of the user's ID information or the user attribute is detected and at least one of the ID information or the user attribute is detected, and a map image is displayed. There is generated as a background image to be displayed on the display, based on the user attribute of the user, the map image being written in the language used in the user is generated as a user region image to be displayed on the user area, the user area A user area image to be displayed is generated, and the user area image is superimposed on the background image.
The display method, which is the second aspect of the present technology, is a display method of a signage device provided with a display for displaying information, in front of the display based on an image obtained by the signage device in front of the display. Detection steps to detect existing users and
A detection step of detecting at least one of the detected ID information or user attributes indicated by the user's belongings based on the captured image, and a communication step of connecting to a server that supplies the user's behavior data. A user area setting step for setting a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected, and a background image for displaying a map image on the display. Based on the background image generation step generated as and the behavior data of the user, a map image to which information indicating the destination of the user or the direction of the destination is added is generated as a user area image to be displayed in the user area. The user area image generation step and the superimposition step of superimposing the user area image on the background image are included.
The program, which is the second aspect of the present technology, has a detection unit that detects a user existing in front of the display based on an image of the computer in front of the display displaying information, and the captured image. Based on, a detection unit that detects at least one of the detected ID information or user attributes indicated by the user's belongings, a communication unit that connects to a server that supplies the user's behavior data, and the ID information or A user area setting unit that sets a user area on the screen of the display based on the contour of the user in which at least one of the user attributes is detected, and a background image that generates a map image as a background image to be displayed on the display. A user area image generation unit that generates a user area image in which a map image to which information indicating the user's destination or the direction of the destination is added is displayed in the user area based on the user's behavior data and the generation unit. And, it functions as a superimposing part which superimposes the user area image on the background image.
In the second aspect of the present technology, the user existing in front of the display is detected based on the image captured in front of the display, and the possession of the detected user is detected based on the captured image. Based on the contour of the user, at least one of the ID information or the user attribute indicated by the product is detected, connected to a server that supplies the behavior data of the user, and at least one of the ID information or the user attribute is detected. , A user area is set on the screen of the display, a map image is generated as a background image to be displayed on the display, and information indicating the destination of the user or the direction of the destination based on the behavior data of the user. Is generated as a user area image to be displayed in the user area, and the user area image is superimposed on the background image.

According to one aspect of the present technology, it is possible to provide an area dedicated to each user on the display and display information suitable for each user.

It is a figure for demonstrating the outline of the signage apparatus to which this technique is applied. It is a figure for demonstrating the user authentication process. It is a block diagram which shows the configuration example of the system including the signage apparatus. It is a block diagram which shows the configuration example of a signage apparatus. It is a block diagram which shows the configuration example of a user terminal apparatus. It is a block diagram which shows the configuration example of a service server. It is a figure which shows the type A to D which supply sources of display data are different. It is a flowchart explaining the processing of the signage apparatus corresponding to type A. It is a figure for demonstrating the processing of the signage apparatus corresponding to type A. It is a flowchart explaining the processing of the signage apparatus corresponding to type B. It is a figure which shows the display example of the signage apparatus corresponding to type B. It is a flowchart explaining the processing of the signage apparatus corresponding to type C. It is a figure which shows the display example of the signage apparatus corresponding to type C. It is a flowchart explaining the processing of the signage apparatus corresponding to type D. It is a block diagram which shows the configuration example of a general-purpose computer.

Hereinafter, the best mode for carrying out the present technology (hereinafter referred to as the embodiment) will be described in detail with reference to the drawings.

<Overview of signage equipment>
First, an outline of the signage device according to the embodiment of the present technology will be described with reference to FIG.

The signage device 10 is installed in a public place used by many people, such as a commercial building, a tourist spot, a street, a park, a train station, or an airport. However, for example, it may be used in a closed space (room) such as a house or a karaoke box.

The signage device 10 has a large display 11 and camera units 12 installed on the left and right sides of the large display 11, and based on an image from the camera unit 12 that constantly captures images, the front surface of the display 11 (a predetermined distance range). It determines whether or not there is a person, and if there is no person, it functions as a mirror by inverting the image from the camera unit 12 and displaying it on the display 11. Hereinafter, the state of functioning as a mirror is also referred to as a mirror display. A map, an advertisement, or the like may be displayed instead of the mirror display on the display 11.

When there is a person in front of the display 11, the mirror display of the display 11 is changed to display predetermined information (for example, a map or the like). Then, a user authentication process for determining whether or not the person is a user who has already registered for use is performed, and if the person is a user, the user is superimposed on the display of predetermined information on the display 11 to form the outline of the user. A dedicated display area for the user (hereinafter referred to as a user area) having the same shape as the contour determined based on the user is set, and information suitable for the user is displayed in the user area.

The user area does not have to have exactly the same shape as the user's contour, and may be a combination of a rectangle, a circle, an ellipse, another polygon, or the like based on the user's contour. Even in that case, the user area is expanded when the user approaches the display 11 and contracted when the user moves away from the display 11, and when the user moves left and right in front of the display 11, the user area moves left and right accordingly. And.

For example, as shown in FIG. 1, when users 81A and 81B who have been registered for use approach the front of the signage device 10, the mirror display of the display 11 displays a map in the local language (for example, Japanese). It is changed to the display. Then, when the user authentication process is successful, the user area for the user 81A and the user area for the user 81B are set on the display 11. Then, for the user 81A, a map in both Japanese and English (may be written only in English) is displayed in the user area according to the language used by the user 81A (set at the time of registration of use, for example, English). Similarly, for the user 81B, a map in both Japanese and Chinese (may be written only in Chinese) is displayed in the user area according to the language used by the user 81B (set at the time of registration for use, for example, Chinese). Will be done.

At this time, the areas other than the user areas of the users 81A and 81B of the display 11 remain unchanged in Japanese (local language) notation. Note that the signage device 10 can output audio, for example, English audio is output from the speaker closer to user 81A, and Chinese audio is output from the speaker closer to user 81B (both not shown). The speaker may be output according to the language used by each user so as to output.

<About user authentication processing>
FIG. 2 shows a method of transmitting a user ID or the like from the user terminal device 50 side to the signage device 10 for the user authentication process.

In FIG. A, a user terminal device 50 (assuming a smartphone, for example) owned by the user is used, and the flash LED provided on the back surface of the user terminal device 50 is blinked according to a pattern corresponding to the user ID, or is displayed on the front surface. This is an example in which the high-brightness screen displayed on the provided display is blinked according to the pattern corresponding to the user ID.

In FIG. B, the ID notification device 82 built into the smart watch or headphones blinks light according to the pattern corresponding to the user ID according to the control from the user terminal device 50 (for example, assuming a smartphone) possessed by the user. This is an example of making it.

FIG. C is an example of attaching a badge 83 on which image information such as a QR code representing a user ID is printed to the user's clothes. Instead of using image information such as a QR code (trademark), the badge 83 may be made to blink light or discolor light in a pattern corresponding to the user ID. When the badge 83 is used, it is possible to realize low cost and low power consumption as compared with the user terminal device 50 and the ID notification device 82. Further, the signage device 10 can be used even if the user does not have the user terminal device 50.

Although not shown, the user's face may be registered in advance and the user authentication process may be performed by face recognition. In this case, it is possible to save the user the trouble of possessing the ID notification device 82 and attaching the badge 83 to the clothes.

In the signage device 10 to which the user ID is notified by the method described above, the user authentication process is performed based on the user ID.

Instead of notifying the user ID, the user attribute may be notified. In that case, the user registration and user authentication processing can be simplified (omitted).

In the above example, the user side notifies the signage device 10 of the user ID and the like, but conversely, the signage device 10 may notify the user terminal device 50 of the ID of the signage device 10. Specifically, a QR code representing the device ID and device attributes of the signage device 10 may be displayed on the screen of the signage device 10, or a sticker on which the QR code is printed may be attached to the signage device 10. Then, the user terminal device 50 may image and read it.

In this case, when a plurality of signage devices 10 are present nearby, the user can explicitly select which signage device 10 to use. Further, it is possible to prevent the user ID from being detected (sniffed) from the signage device 10 which is not supposed to be used and the data from being read from the user terminal device 50.

<System configuration example including the signage device 10>
FIG. 3 shows a configuration example of a system including the signage device 10. The system includes a signage device 10, a user terminal device 50 owned by a user, an ID notification device 82 that notifies the signage device 10 of a user ID according to control from the user terminal device 50, and the signage device 10 and the user terminal device. 50 is composed of a service server 70 connected via the Internet 1.

The method of connecting the signage device 10 to the Internet 1 may be either wired or wireless. The method of connecting the user terminal device 50 to the Internet 1 may be via a mobile communication network or a nearby wireless LAN. Further, the signage device 10 may be provided with a function of operating as a wireless LAN access point, and the user terminal device 50 may be connected to the Internet 1 via the signage device 10 as an access point.

A predetermined wireless communication standard (Bluetooth (trademark), Wi-Fi, infrared communication, etc.) can be used for the connection between the signage device 10 and the user terminal device 50, and data is wirelessly communicated after mutual authentication processing. Can be done.

The connection between the user terminal device 50 and the ID notification device 82 can wirelessly communicate data according to a predetermined wireless communication standard (for example, Bluetooth).

<Configuration example of signage device 10>
FIG. 4 shows a configuration example of the signage device 10.

The signage device 10 includes a display 11, a camera unit 12, a background image generation unit 13, an image superimposition unit 14, a user area determination unit 15, a user attribute determination unit 16, a data provision unit 17, an information extraction unit 18, and an area image generation unit. It has 19 and a communication unit 20.

The display 11 performs a mirror display when there is no person in front of the display 11, and displays predetermined information (for example, a map in the local language (in this case, Japanese)) when there is a person in front of the display 11. Then, when the person is a registered user, a user area dedicated to the user is provided and information for the user (for example, a map in which the user's language is written) is displayed.

The camera unit 12 is composed of two cameras installed on the left and right sides of the display 11, constantly images the front of the display 11, and outputs a video signal obtained as a result to the subsequent stage. That is, the camera unit 12 outputs the stereo video signals obtained from the two cameras to the user area determination unit 15, and outputs one of the stereo video signals to the background image generation unit 13 and the user attribute determination unit 16.

The background image generation unit 13 determines whether or not there is a person in front of the display 11 (a predetermined distance range) based on the video signal from the camera unit 12. Then, when there is no person in front of the display 11, the background image generation unit 13 generates a mirror image for mirror display by inverting the video signal from the camera unit 12, and outputs it to the image superimposition unit 14 as a background image. do. On the contrary, when there is a person in front of the display 11, the background image generation unit 13 acquires predetermined display data (for example, a map in the local language notation) from the data providing unit 17, and the image superimposing unit 14 as the background image. Output to.

The image superimposition unit 14 superimposes the user area image input from the area image generation unit 19 on the background image input from the background image generation unit 13 and outputs it to the display 11.

The user area determination unit 15 determines a user area having the same shape as the contour of the user existing in front of the display 11 based on the stereo video signal from the camera unit 12, and generates a region image of the position information of the user area on the display 11. Notify department 19. When the number of people passing in front of the display 11 is large and it is difficult to extract the contour of the user, the distance is measured to the user based on the stereo video signal, and only the person located at a predetermined distance from the display 11 is focused on. And try to extract the outline.

The user attribute determination unit 16 identifies the user ID by reading the blinking pattern of the user terminal device 50, the ID notification device 82, etc. possessed by the user or the QR code of the badge 83 from the video signal input from the camera unit 12. User authentication processing is performed by referring to the data providing unit 17. When the user authentication process is successful, the user attribute determination unit 16 notifies the information extraction unit 18 of the user ID and its user attributes. If the user is notified of the user attribute instead of the user ID, the user authentication process may be omitted and the user attribute may be notified to the information extraction unit 18.

The data providing unit 17 holds the registration information (user ID, user attribute, SNS account, etc.) of each user, and provides the registration information to the user attribute determination unit 16 that performs the user authentication process. Further, the data providing unit 17 holds various display data (for example, a map written in each language and a map written in multiple languages, a facility guide map, an advertisement, etc.), and is derived from the background image generation unit 13 or the information extraction unit 18. Display data according to the request is provided to the information extraction unit 18.

The information extraction unit 18 acquires display data (for example, a map written in both Japanese and English) corresponding to the user attribute notified from the user attribute determination unit 16 from the data providing unit 17 and outputs the display data to the area image generation unit 19. Further, the information extraction unit 18 accesses the service server 70 via the communication unit 20 and the Internet 1, and the action data corresponding to the user ID notified from the user attribute determination unit 16 (for example, the user's reservation information and the destination). Information, schedule information, etc.) are acquired and output to the area image generation unit 19. Further, the information extraction unit 18 connects to the user terminal device 50 via the communication unit 20, acquires display data (for example, photographic data) from the user terminal device 50, and outputs the display data (for example, photographic data) to the area image generation unit 19.

The area image generation unit 19 is based on the position information of the user area on the display 11 notified from the user area determination unit 15, and the user is based on the display data (for example, a map written in both Japanese and English) input from the information extraction unit 18. A user area image is generated by extracting the area, and the brightness of the user area image is changed (for example, the brightness is slightly reduced) and output to the image superimposing unit 14. Further, the area image generation unit 19 can edit the user area image (for example, add an arrow indicating the destination and its direction) based on the action data input from the information extraction unit 18.

As a result, when an image in which the user area image is superimposed on the background image is displayed on the display 11, the user area image looks like a shadow of the user. Further, when another person standing behind the user looks at the display 11, the user area image becomes difficult to see because the user himself / herself is obstructed. Therefore, protect the user area image from the line of sight of the other person standing behind the user. (However, it does not completely hide it from others).

When a plurality of users are present in front of the display 11, a user area is set for each user, a user area image is generated, and the user area image is superimposed on the background image and displayed on the display 11. For example, as shown in FIG. 1, when two users 81A and 81B are present in front of the display 11, two user areas are set in the background image of the display 11, and the users 81A and 81B are respectively set. The image corresponding to the corresponding display data will be displayed.

The communication unit 20 wirelessly communicates predetermined data with the user terminal device 50 in accordance with a predetermined wireless communication standard. Further, the communication unit 20 connects to the service server 70 via the Internet 1 and acquires predetermined data. Further, the communication unit 20 has a built-in group token generation unit 21. The group token generation unit 21 issues group tokens to a plurality of user terminal devices 50 that are connected at the same time.

<Configuration example of user terminal device 50>
FIG. 5 shows a configuration example of the user terminal device 50. It is assumed that the user terminal device 50 is mounted on a smartphone, for example.

The user terminal device 50 includes an ID notification control unit 51, a camera unit 52, a signage device detection unit 53, an ID notification instruction button 54, a position information detection unit 55, a data provision unit 56, an ID notification unit 57, and an ID notification device control unit 58. , Communication unit 59, remote display control unit 60, application execution unit 61, application data storage unit 62, and display 63.

The ID notification control unit 51 controls the ID notification unit 57 or the ID notification device 82 in response to the trigger signal notified from the ID notification instruction button 54 or the like, and causes the signage device 10 to execute a process of notifying the user ID. ..

For example, the camera unit 52 captures the QR code displayed on the screen of the signage device 10, and outputs the image obtained as a result to the signage device detection unit 53. The signage device detection unit 53 detects a QR code from an image input from the camera unit 52, and notifies the ID notification control unit 51 of the device ID and device attributes represented by the QR code.

The ID notification instruction button 54 is a button for receiving an operation from a user who instructs the ID notification, and outputs an ID notification trigger signal to the ID notification control unit 51 in response to the operation from the user.

The position information detection unit 55 detects the position information of the user terminal device 50 by receiving and analyzing, for example, a GPS signal or the like. Further, the position information detection unit 55 refers to the position information of the signage device 10 held in the data providing unit 56 (or the signage device position information storage unit 72 (FIG. 6)), and the signage device 10 exists in the vicinity. If so, the trigger signal is output to the ID notification control unit 51.

The ID notification unit 57 corresponds to, for example, a flash LED or a display 63 provided on the back surface of a smartphone, and the signage is generated by blinking light emission according to a pattern corresponding to the user ID notified from the ID notification control unit 51. Notify the device 10 of the user ID.

The ID notification device control unit 58 accesses the ID notification device 82 mounted on the smart watch, headphones, or the like shown in FIG. 2B according to the control from the ID notification control unit 51, and follows the pattern corresponding to the user ID. By blinking the light emission, the signage device 10 is controlled to notify the user ID.

The communication unit 59 wirelessly communicates predetermined data with the signage device 10 in accordance with a predetermined wireless communication standard (for example, Wi-Fi). Further, the communication unit 59 communicates predetermined data with the ID notification device 82 in accordance with a predetermined wireless communication standard (for example, Bluetooth). Further, the communication unit 59 connects to the Internet 1 via a mobile communication network, a nearby wireless LAN, or a signage device 10 operating as an access point, and communicates data with the service server 70.

The remote display control unit 60 supplies the same video signal as the screen display on the display 63 by the application program executed by the application execution unit 61 to the signage device 10 via the communication unit 59.

The application execution unit 61 executes an installed application program (for example, a photo application for viewing imaged and stored photo data). In addition, the application execution unit 61 accesses the application server 90, uploads application data (for example, photo data) processed by the application program being executed, and shares the data with other users.

The application data storage unit 62 holds application data (for example, photo data) processed by the application program. The display 63 displays a screen corresponding to a video signal supplied from the application program being executed. As described above with reference to FIG. 2A, the display 63 may operate as the ID notification unit 57.

<Configuration example of service server 70>
FIG. 6 shows a configuration example of the service server 70.

The service server 70 includes a communication unit 71, a signage device position information storage unit 72, a matching determination unit 73, a data provision unit 74, and a user information extraction unit 75.

The communication unit 71 connects to the signage device 10 and the user terminal device 50 via the Internet 1.

The position information of the signage device 10 is stored in the signage device position information storage unit 72. When the matching determination unit 73 is notified of the position information of the user terminal device 50 from the user terminal device 50, the signage device 10 is placed in the vicinity of the user terminal device 50 by referring to the signage device position information storage unit 72. It is determined whether or not it exists, and the determination result is notified to the user terminal device 50.

User behavior data (for example, reservation information for transportation means such as airplanes, reservation information for conference rooms, restaurants, concerts, etc., destination information, schedule information, etc.) is registered in advance in the data providing unit 74. When the user ID is notified from the signage device 10, the user information extraction unit 75 extracts the action data corresponding to the user ID and provides it to the signage device 10 by referring to the data providing unit 74.

<About types A to D from different sources of display data>
Next, FIG. 7 shows a supply source of display data or the like displayed in the user area provided on the screen of the signage device 10.

In FIG. A, a user ID is notified from the user terminal device 50 to the signage device 10 for user authentication processing, and display data (for example, a map in which the user's language is written) is displayed from the data providing unit 17 of the signage device 10. ) Is provided. Hereinafter, this example will be referred to as type A.

FIG. B shows an example in which display data (for example, photographic data) is provided from the user terminal device 50 to the signage device 10 after the user authentication process is successful. Hereinafter, this example will be referred to as type B.

In FIG. C, after the user authentication process is successful, the signage device 10 notifies the service server 70 of the user ID notified from the user terminal device 50, and the service server 70 notifies the user's action data (reservation information, An example is shown in which the behavior data is reflected in the display data (for example, a map in which the user's language is written) provided by the data providing unit 17 of the signage device 10 in response to the provision of destination information, schedule information, etc. There is. Hereinafter, this example will be referred to as type C.

In FIG. D, after passing through the type B described above, the signage device 10 issues a group token to the plurality of user terminal devices 50 connected to the signage device 10, and the group tokens are issued to the plurality of user terminals. An example of sharing application data (for example, photo data) between the devices 50 via the application server 90 is shown. Hereinafter, this example will be referred to as type D.

Hereinafter, the processing of the signage device 10 corresponding to the above-mentioned types A to D will be described in detail.

Regarding which of the types A to D the signage device 10 is, any of the types A to D may be specified in advance for each signage device 10 so that the user can select the signage device 10 at a predetermined timing. You may do it.

<Processing of signage device 10 corresponding to type A>
FIG. 8 is a flowchart illustrating the processing of the signage device 10 corresponding to the type A. FIG. 9 is a diagram for explaining the processing of the signage device 10 corresponding to the type A.

In step S1, as shown in FIG. 9A, predetermined display data (in the case of the same figure, a map in Japanese notation, etc.) is performed on the display 11. That is, the background image generation unit 13 acquires predetermined display data from the data providing unit 17, outputs it to the image superimposition unit 14 as a background image, and the image superimposition unit 14 outputs the background image to the display 11 as it is. In addition, instead of the map written in Japanese, a mirror display or an advertisement display may be performed.

In step S2, the display of the map in Japanese is continued until a person appears in front of the display 11. That is, the background image generation unit 13 determines whether or not there is a person in front of the display 11 based on the video signal from the camera unit 12, and if no person appears in front of the display 11, the process returns to step S1. Then, the processes of steps S1 and S2 are repeated. After that, when a person appears in front of the display 11 as shown in FIG. 9B, the process proceeds to step S3.

In step S3, a user authentication process is performed on a person who appears in front of the display 11. That is, the user attribute determination unit 16 reads the blinking pattern of the ID notification device 82 or the like from the video signal input from the camera unit 12, identifies the corresponding user ID, and refers to the information extraction unit 18. Perform authentication processing.

In step S4, the success or failure of the user authentication process is determined. That is, when it is confirmed that the user ID specified by the user attribute determination unit 16 has been registered, it is determined that the user authentication process has succeeded, and the user attribute determination unit 16 determines the user ID and its user attributes (for example, After notifying the information extraction unit 18 (indicating that the language used is English), the process proceeds to step S5. On the contrary, when it is confirmed that the user ID specified by the user attribute determination unit 16 is not registered, it is determined that the user authentication process has failed, the process is returned to step S3, and the processes of steps S3 and S4 are performed. Is repeated. When the processes of steps S3 and S4 are repeated a predetermined number of times, that is, when the user authentication process fails a predetermined number of times, the process is executed again from step S1.

In step S5, the user area is determined. That is, the user area determination unit 15 determines the position of the user area having the same shape as the contour of the user existing in front of the display 11 based on the stereo image signal from the camera unit 12, and the position information of the user area on the display 11. Is notified to the area image generation unit 19.

In step S6, a user area image is generated. That is, the information extraction unit 18 acquires the display data (for example, a map written in both Japanese and English) corresponding to the user attribute notified from the user attribute determination unit 16 from the data providing unit 17 and outputs it to the area image generation unit 19. do. The area image generation unit 19 extracts the user area from the display data input from the information extraction unit 18 based on the position information of the user area on the display 11 notified from the user area determination unit 15, thereby extracting the user area image. It is generated, and further, the brightness of the user area image is changed and output to the image superimposing unit 14.

In step S7, the user area image is displayed in the user area of the display 11. That is, the image superimposition unit 14 inputs the user area image (in this case, the map in Japanese notation) input from the background image generation unit 13 to the user area image (in this case, the map in Japanese notation) input from the area image generation unit 19. A map written in both Japanese and English) is superimposed and output to the display 11.

When a plurality of people are present in front of the display 11, the processes of steps S3 to S7 described above are performed for each detected person. For example, as shown in FIG. 9B, when there are two people in front of the display 11, the language used by the user 81A is English, and the language used by the user 81B is Chinese, as shown in FIG. 9C. A map written in Japanese is displayed on the entire display 11 (background), a map written in Japanese and English is displayed on the shadow of the user 81A (user area 85A), and the shadow of the user 81B (user area). In 85B), a map written in both Japanese and Chinese will be displayed.

This is the end of the description of the processing of the signage device 10 corresponding to the type A. According to this process, if each user stands in front of the area he / she wants to see on the map displayed on the screen, the area he / she wants to see is included in his / her user area. Even if you do not do it, you can get information on the display of your own language.

<Processing of signage device 10 corresponding to type B>
FIG. 10 is a flowchart illustrating the processing of the signage device 10 corresponding to the type B. FIG. 11 is a diagram showing a display example of the signage device 10 corresponding to the type B.

In step S11, a mirror display is performed on the display 11. That is, the background image generation unit 13 generates a mirror image for mirror display by inverting the video signal from the camera unit 12, outputs the mirror image as a background image to the image superimposition unit 14, and the image superimposition unit 14 keeps the background image as it is. Output to display 11. Instead of the mirror display, for example, a map or an advertisement written in the local language may be displayed.

In step S12, the mirror display is continued until a person appears in front of the display 11. That is, the background image generation unit 13 determines whether or not there is a person in front of the display 11 based on the video signal from the camera unit 12, and if no person appears in front of the display 11, the process returns to step S1. Then, the processes of steps S11 and S12 are repeated. After that, when a person appears in front of the display 11, the process proceeds to step S13.

In step S13, predetermined display data (for example, a map written in Japanese) is displayed on the display 11. That is, the background image generation unit 13 acquires predetermined display data from the data providing unit 17, outputs it to the image superimposition unit 14 as a background image, and the image superimposition unit 14 outputs the background image to the display 11 as it is.

In step S14, a user authentication process is performed on a person who appears in front of the display 11. That is, the user attribute determination unit 16 reads the blinking pattern of the ID notification device 82 or the like from the video signal input from the camera unit 12, identifies the corresponding user ID, and refers to the information extraction unit 18. Perform authentication processing.

In step S15, the success or failure of the user authentication process is determined. That is, when it is confirmed that the user ID specified by the user attribute determination unit 16 has been registered, it is determined that the user authentication process has succeeded, and the user attribute determination unit 16 notifies the information extraction unit 18 of the user ID. After that, the process proceeds to step S16. On the contrary, when it is confirmed that the user ID specified by the user attribute determination unit 16 is not registered, it is determined that the user authentication process has failed, the process is returned to step S13, and the process of steps S13 to S15. Is repeated. If the processes of steps S13 to S15 are repeated a predetermined number of times, that is, if the user authentication process fails a predetermined number of times, the process is executed again from step S11.

In step S16, the user area is determined. That is, the user area determination unit 15 determines the position of the user area having the same shape as the contour of the user existing in front of the display 11 based on the stereo image signal from the camera unit 12, and the position information of the user area on the display 11. Is notified to the area image generation unit 19. Further, the user area determination unit 15 notifies the information extraction unit 18 of the size of the user area via the user attribute determination unit 16.

In step S17, a wireless connection is established with the user terminal device 50 in which the user authentication process is successful. That is, the information extraction unit 18 notifies the communication unit 20 of the user ID, and the communication unit 20 establishes a wireless connection with the user terminal device 50 corresponding to the user ID.

In step S18, the communication unit 20 notifies the user terminal device 50 of the size of the user area. In the user terminal device 50 that receives this notification, the application program (for example, a photo application) executed by the application execution unit 61 transmits the video signal of the screen to be displayed on the display 63 to the remote display control unit 60 and the display 63. The output is output, and the remote display control unit 60 transmits the video signal to the signage device 10 via the communication unit 59. In the signage device 10, the video signal is received by the communication unit 20 and supplied to the information extraction unit 18.

In step S19, a user area image is generated. That is, the information extraction unit 18 outputs the video signal from the user terminal device 50 supplied from the communication unit 20 to the area image generation unit 19. The area image generation unit 19 generates a user area image by rendering a video signal from the user terminal device 50 on the position information of the user area on the display 11 notified from the user area determination unit 15, and further, the user area. The brightness of the image is changed and output to the image superimposing unit 14.

In step S20, the user area image is displayed in the user area of the display 11. That is, the image superimposition unit 14 inputs the user area image (in this case, the map in Japanese notation) input from the background image generation unit 13 to the user area image (in this case, the map in Japanese notation) input from the area image generation unit 19. The image of the photo application executed by the user terminal device 50) is superimposed and output to the display 11.

When a plurality of people are present in front of the display 11, the processes of steps S14 to S20 described above are performed for each detected person. For example, when two people are present in front of the display 11, as shown in FIG. 11, the same image as the screen of the user terminal device 50 possessed by the user 81A is displayed in the shadow of the user 81A (user area 85A). Then, in the shadow of the user 81B (user area 85B), the same image as the screen of the user terminal device 50 possessed by the user 81B is displayed.

This is the end of the description of the processing of the signage device 10 corresponding to the type B. According to the process, each user can use his / her own user area provided on the screen as a remote display of the user terminal device 50 possessed by the user. Therefore, for example, the user 81A may refer to the adjacent user 81B. , The photograph or the like stored in the possessed user terminal device 50 can be displayed in the user area of the screen of the signage device 10 and shown. At this time, if the user approaches the display 11, the user area of the user expands, and if the user moves away from the display 11, the user area of the user narrows. Therefore, the user intuitively expands the user area of the user without performing any special operation. Can be adjusted.

When another person standing behind the user 81A looks at the display 11, the user area of the user 81A becomes difficult to see because the user 81A itself obstructs the user area. Therefore, the user area is seen from the line of sight of the other person standing behind the user. The image can be protected to some extent.

<Processing of signage device 10 corresponding to type C>
FIG. 12 is a flowchart illustrating the processing of the signage device 10 corresponding to the type C. FIG. 13 is a diagram showing a display example of the signage device 10 corresponding to the type C.

In step S31, predetermined display data (for example, a map written in Japanese) is displayed on the display 11. That is, the background image generation unit 13 acquires predetermined display data from the data providing unit 17, outputs it to the image superimposition unit 14 as a background image, and the image superimposition unit 14 outputs the background image to the display 11 as it is. In addition, instead of the map written in Japanese, a mirror display or an advertisement display may be performed.

In step S32, the display of the map in Japanese is continued until a person appears in front of the display 11. That is, the background image generation unit 13 determines whether or not there is a person in front of the display 11 based on the video signal from the camera unit 12, and if no person appears in front of the display 11, the process returns to step S31. Then, the processes of steps S31 and S32 are repeated. After that, when a person appears in front of the display 11 as shown in FIG. 9B, the process proceeds to step S33.

In step S33, a user authentication process is performed on a person who appears in front of the display 11. That is, the user attribute determination unit 16 reads the blinking pattern of the ID notification device 82 or the like from the video signal input from the camera unit 12, identifies the corresponding user ID, and refers to the information extraction unit 18. Perform authentication processing.

In step S34, the success or failure of the user authentication process is determined. That is, when it is confirmed that the user ID specified by the user attribute determination unit 16 has been registered, it is determined that the user authentication process has succeeded, and the user attribute determination unit 16 notifies the information extraction unit 18 of the user ID. After that, the process proceeds to step S35. On the contrary, when it is confirmed that the user ID specified by the user attribute determination unit 16 is not registered, it is determined that the user authentication process has failed, the process is returned to step S33, and the processes of steps S33 and S34 are performed. Is repeated. When the processes of steps S33 and S34 are repeated a predetermined number of times, that is, when the user authentication process fails a predetermined number of times, the process is executed again from step S31.

In step S35, the user area is determined. That is, the user area determination unit 15 determines the position of the user area having the same shape as the contour of the user existing in front of the display 11 based on the stereo image signal from the camera unit 12, and the position information of the user area on the display 11. Is notified to the area image generation unit 19.

In step S36, the user's behavior data is acquired from the service server 70. That is, the information extraction unit 18 connects to the service server 70 via the communication unit 20 and the Internet 1 to notify the user ID. In response to this, in the service server 70, the user information extraction unit 75 extracts the action data corresponding to the user ID from the data providing unit 74 and outputs it to the communication unit 71, and the communication unit 71 outputs the action data to the Internet 1. Reply to the signage device 10 via.

When the signage device 10 acquires the user's behavior data from the service server 70, the mutual authentication process may be executed in order to prevent data leakage due to spoofing or the like.

In the signage device 10, the information extraction unit 18 acquires the action data via the communication unit 20, receives it, and supplies it to the information extraction unit 18. The information extraction unit 18 extracts information related to the current time and position (for example, destination information) from the action data and outputs it to the area image generation unit 19.

In step S37, a user area image is generated. That is, the information extraction unit 18 acquires the display data (for example, a map written in both Japanese and English) corresponding to the user attribute notified from the user attribute determination unit 16 from the data providing unit 17 and outputs it to the area image generation unit 19. do. The area image generation unit 19 extracts the user area from the display data input from the information extraction unit 18 based on the position information of the user area on the display 11 notified from the user area determination unit 15, thereby extracting the user area image. Generate. Further, the area image generation unit 19 adds an arrow indicating the user's destination or its direction to the user area image based on the destination information input from the information extraction unit 18, and further changes the brightness of the user area image. And output to the image superimposing unit 14.

In step S38, the user area image is displayed in the user area of the display 11. That is, the image superimposition unit 14 inputs the user area image (in this case, the map in Japanese notation) input from the background image generation unit 13 to the user area image (in this case, the map in Japanese notation) input from the area image generation unit 19. A map written in both Japanese and English with arrows pointing to the destination and its direction) is superimposed and output to the display 11.

When a plurality of people are present in front of the display 11, the processes of steps S33 to S38 described above are performed for each detected person. For example, when there are two people in front of the display 11, the language used by the user 81A is English, and the language used by the user 81B is Chinese, the entire display 11 (background) is shown in FIG. Is displayed in Japanese, and in the shadow of user 81A (user area 85A), the destination of user 81A (outside the map displayed in user area 85A) is displayed on the map written in both Japanese and English. An arrow is displayed that roughly points in the direction of. Similarly, in the shadow of the user 81B (user area 85B), an arrow pointing to the destination of the user 81B (in the map displayed in the user area 85B) is displayed on the map written in both Japanese and Chinese.

This is the end of the description of the processing of the signage device 10 corresponding to the type C. According to the process, when the user's destination is not in the user's area, the user can move or reach out without any special operation by an intuitive action. The user area can be moved or enlarged to put one's own destination in the user area, and the exact position of the destination can be displayed.

Further, for example, a map or a schedule for an airplane boarding gate at an airport, a reserved event schedule, a venue, etc. is displayed on the entire display 11, and information for each user is superimposed and displayed on the display 11. As a result, even when a user who only needs to see the outline of the information (for example, a map) and a user who wants to see the details coexist, the information can be efficiently presented to each user in a short time. can.

<Processing of signage device 10 corresponding to type D>
FIG. 14 is a flowchart illustrating the processing of the signage device 10 corresponding to the type D.

The process corresponding to the type D is executed, for example, after the process of the signage device 10 corresponding to the type B described above.

In step S51, it is confirmed whether or not a plurality of user terminal devices 50 are connected at the same time. That is, the communication unit 20 determines whether or not a wireless connection has been established with a plurality of user terminal devices 50 at the same time. If a wireless connection is established with the plurality of user terminal devices 50 at the same time, the process proceeds to step S52. On the contrary, when the wireless connection is established only with one or less user terminal devices 50, step S52 is skipped.

In step S52, the group token generation unit 21 of the communication unit 20 issues and notifies the group tokens to the plurality of user terminal devices 50 for which the wireless connection is established.

After that, on the side of the plurality of user terminal devices 50 to which the group token is notified, the application data (for example, photo data) processed by the running application program (for example, a photo application) is shared via the application server 90. It will be in a possible state.

This completes the description of the processing of the signage device 10 corresponding to the type D. According to this process, each user can share application data with other users.

<First modification>
In the above description, when a plurality of users exist in front of the display 11, a user area is individually provided for each user, but the distance between the plurality of users is a predetermined threshold value (number). In the case of cm) or less, a plurality of user areas for each user may be regarded as one user area.

Specifically, for example, there are two users holding hands in front of the display 11, and after each user authentication process is successful and two user areas are provided, one user obtains a user ID. When the blinking pattern for notifying is continued and the blinking pattern for notifying the user ID is erased by the other user, the two user areas are regarded as corresponding to one user, and the display area of information suitable for one user is considered. May be expanded.

In this way, two or more users can see information suitable for one of the users and information provided by the user (for example, a photograph) in a wider display area.

<Second modification>
When there is no possibility that the display 11 of the signage device 10 can be seen by anyone other than the user, that is, when there is only a user who has succeeded in the user authentication process in front of the display 11, or when the signage device 10 is a closed space (in a house). When it is detected that the display is set to (room, etc.), the entire surface of the display 11 may be regarded as a user area, and information suitable for the user may be displayed. Further, in that case, when there are a plurality of users, the contour of each user to be detected is converted into a rectangular area and arranged on the display 11 according to the left and right positions and distances of each user with respect to the display 11. It is possible to reduce the area not used in and efficiently display information that is often premised on a rectangle.

<Third modification example>
The display 11 of the signage device 10 may be a table type so that the user can look down and use it.

<Fourth modification>
A touch sensor may be built in the display 11 of the signage device 10. When the touch sensor detects, for example, an action in which a user uses both hands to cover a part of the display 11, the area is dedicated to display highly confidential information to the user. It may be an area.

<Application example>
The screen of an SNS such as Twitter (trademark) may be displayed in the user area provided on the display 11 of the signage device 10. Specifically, the SNS account is registered in association with the user ID at the time of user registration, and the user wears a badge 83 or the like for notifying the user ID. After the user authentication process, the signage device 10 may access the SNS server using the SNS account corresponding to the user ID, acquire the information for the user, and display it in the user area of the display 11.

In this way, the user can confirm the message or the like from the SNS by using the signage device 10 even if he / she does not have the user terminal device 50 (smartphone or the like).

<Other embodiments>
The series of processes described above can be executed by hardware or software. When a series of processes are executed by software, the programs that make up the software are installed on the computer. Here, the computer includes a computer embedded in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.

FIG. 15 is a block diagram showing a configuration example of hardware of a computer that executes the above-mentioned series of processes programmatically.

In the computer 100, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, and a RAM (Random Access Memory) 103 are connected to each other by a bus 104.

An input / output interface 105 is further connected to the bus 104. An input unit 106, an output unit 107, a storage unit 108, a communication unit 109, and a drive 110 are connected to the input / output interface 105.

The input unit 106 includes a keyboard, a mouse, a microphone, and the like. The output unit 107 includes a display, a speaker, and the like. The storage unit 108 includes a hard disk, a non-volatile memory, and the like. The communication unit 109 includes a network interface and the like. The drive 110 drives a removable medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

In the computer 100 configured as described above, the CPU 101 loads the program stored in the storage unit 108 into the RAM 103 via the input / output interface 105 and the bus 104 and executes the program, as described above. A series of processing is performed.

The program executed by the computer 100 may be a program in which processing is performed in chronological order in the order described in this specification, or at a required timing such as in parallel or when a call is made. It may be a program that is processed by.

The embodiment of the present technology is not limited to the above-described embodiment, and various changes can be made without departing from the gist of the present technology.

1 Internet, 10 Signage device, 11 Display, 12 Camera unit, 13 Background image generation unit, 14 Image superimposition unit, 15 User area determination unit, 16 User attribute judgment unit, 17 Data provision unit, 18 Information extraction unit, 19 Area image Generation unit, 20 communication unit, 21 group token generation unit, 50 user terminal device, 51 ID notification control unit, 52 camera unit, 53 signage device detection unit, 54 ID notification instruction button, 55 position information detection unit, 56 data provision unit , 57 ID notification unit, 58 ID notification device control unit, 59 communication unit, 60 remote display control unit, 61 application execution unit, 62 application data storage unit, 63 display, 70 service server, 71 communication unit, 72 signage device location information Storage unit, 73 Matching judgment unit, 74 Data provision unit, 75 User information extraction unit, 81 users, 82 ID notification device, 83 badges, 85 user areas, 90 app servers, 100 computers, 101 CPUs

Claims (12)

A display that displays information and
A detection unit that detects a user existing in front of the display based on an image captured in front of the display.
A detection unit that detects at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the captured image.
A user area setting unit that sets a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected.
The background image generation section that generates a background image for displaying a map image on said display,
A user area image generating unit for generating a user area image to the based on the user attribute of the user, displays a map image that is written in the language used in the user to the user area,
A signage device including a superimposing portion for superimposing the user area image on the background image. A display that displays information and
A detection unit that detects a user existing in front of the display based on an image captured in front of the display.
A detection unit that detects at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the captured image.
A communication unit connected to the server that supplies the user's behavior data,
A user area setting unit that sets a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected.
A background image generator that generates a map image as a background image to be displayed on the display,
A user area image generation unit that generates a map image to which information indicating the user's destination or the direction of the destination is added based on the user's behavior data as a user area image to be displayed in the user area.
With a superimposing portion that superimposes the user area image on the background image
A signage device equipped with. The signage device according to any one of claims 1 to 2, wherein the user area setting unit sets the user area having the same shape as the detected contour of the user on the screen of the display. The signage device according to any one of claims 1 to 3 , further comprising a user authentication processing unit that performs authentication processing of the detected user based on the detected ID information. The signage device according to claim 4 , wherein when the authentication process is successful, the user area setting unit sets the user area on the screen of the display based on the contour of the user who has succeeded in the authentication process. The method according to any one of claims 1 to 5 , wherein the detection unit detects at least one of the ID information or the user attribute notified by the blinking pattern of the light emission of the possessed item based on the captured image. Signage device. The user area setting unit sets the user area corresponding to each of a plurality of users in which at least one of the ID information or the user attribute is detected.
The user area image generation unit generates the user area image to be displayed in each of the plurality of user areas, respectively.
The signage device according to any one of claims 1 to 6 , wherein the superimposing unit superimposes a plurality of the user area images on the background image. The signage device according to claim 7 , wherein the user area setting unit considers the plurality of user areas to correspond to one of the plurality of users when the plurality of users satisfy a predetermined condition. In the display method of a signage device equipped with a display for displaying information,
By the signage device
A detection step that detects a user existing in front of the display based on an image captured in front of the display, and
A detection step of detecting at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the captured image.
A user area setting step of setting a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected.
A background image generating step of generating a background image for displaying a map image on said display,
A user area image generation step of generating a map image expressed in the language used by the user as a user area image to be displayed in the user area based on the user attributes of the user.
A display method including a superimposition step of superimposing the user area image on the background image. Computer,
A detection unit that detects a user existing in front of the display based on an image captured in front of the display displaying information, and a detection unit.
A detection unit that detects at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the captured image.
A user area setting unit that sets a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected.
The background image generation section that generates a background image for displaying a map image on said display,
A user area image generating unit for generating a user area image to the based on the user attribute of the user, displays a map image that is written in the language used in the user to the user area,
A program that functions as a superimposing unit that superimposes the user area image on the background image. In the display method of a signage device equipped with a display for displaying information,
By the signage device
A detection step that detects a user existing in front of the display based on an image captured in front of the display, and
A detection step of detecting at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the captured image.
A communication step that connects to the server that supplies the user's behavior data,
A user area setting step of setting a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected.
A background image generation step of generating a map image as a background image to be displayed on the display, and
A user area image generation step of generating a map image to which information indicating the user's destination or the direction of the destination is added based on the user's behavior data is displayed as a user area image to be displayed in the user area.
With the superimposition step of superimposing the user area image on the background image
Display method including. Computer,
A detection unit that detects a user existing in front of the display based on an image captured in front of the display displaying information, and a detection unit.
A detection unit that detects at least one of the ID information or the user attribute indicated by the detected personal belongings of the user based on the captured image.
A communication unit connected to the server that supplies the user's behavior data,
A user area setting unit that sets a user area on the screen of the display based on the contour of the user in which at least one of the ID information or the user attribute is detected.
A background image generator that generates a map image as a background image to be displayed on the display,
A user area image generation unit that generates a map image to which information indicating the user's destination or the direction of the destination is added based on the user's behavior data as a user area image to be displayed in the user area.
With a superimposing portion that superimposes the user area image on the background image
A program that works.

Images