JP6828974B2 - Display control device, display control method and display control program - Google Patents

Description

The present invention relates to a display control device, a display control method, and a display control program that control the display of image information.

By wirelessly connecting a mobile device, for example, a smartphone to an external display such as a projector or TV, the screen of the smartphone can be mirrored to a larger external display or displayed in real time in the form of a multi-display. Wi-Fi Direct (registered trademark) or wireless LAN is used for wireless connection.

For example, a wireless connection device called a cast device is inserted into the HDMI (High-Definition Multimedia Interface, registered trademark) terminal of an external display such as a TV. Then, the screen of the smartphone is displayed by searching for the cast device and performing the connection operation on the smartphone. This makes it possible to share photos, videos, browsing, etc. on a large screen. For example, a smartphone outputs a screen to an external display by mirroring, displays an operation screen on the smartphone by a multi-display when playing a part of a moving image, and displays contents on the external display. While the smartphone is connected to the external display, the icon connecting to the external display is displayed small on the screen.

After connecting the smartphone to the external display via a cast device or the like, it is necessary to perform the following operations in order to disconnect the connection.
1. 1. Perform a disconnection operation on your smartphone.
2. 2. Turn off the external display (cast device) or smartphone. For example, it also includes when the smartphone moves out of the Wi-Fi Direct or the connectable range of the wireless LAN.

Conventionally, for example, there is a technique of connecting a mobile terminal device to an external projector and projecting a display content on a screen (see, for example, Patent Document 1 below). Further, there is a technique of superimposing a help display including a comment on a page screen when the line of sight of a user captured by an in-camera of a mobile terminal device gazes at an object on the screen (see, for example, Patent Document 2 below). Further, there is a technique in which the in-camera of the information terminal device determines whether the user's line of sight is gazing at the display, and if the user's line of sight is not gazing, the input operation to the touch panel is not accepted (see, for example, Patent Document 3 below). .. Further, there is a technique for returning the display driver from a stop to an operable state by detecting a touch operation after the sleep state of the touch panel is released (see, for example, Patent Document 4 below).

Japanese Unexamined Patent Publication No. 2014-026602 Japanese Unexamined Patent Publication No. 2015-153073 International Publication No. 2013/141161 Japanese Unexamined Patent Publication No. 2014-106866

As described above, after the smartphone is connected to the external display by wireless connection, the screen of the smartphone continues to be displayed on the external display unless the connection is disconnected. At this time, the wireless connection is maintained even when the smartphone is in the sleep state or the lock screen state, and is displayed on the external display again when returning from the sleep state. In the present invention, the sleep state refers to, for example, the transition of the smartphone to the power saving state.

Therefore, if the user who connects the smartphone to the external display by wireless connection forgets to disconnect and does not disconnect, the screen of the smartphone continues to be displayed on the external display against the intention of the user.

In recent years, with the development of wireless LAN technology, the wireless connectable range has been extended to more than a dozen meters. In addition, although radio attenuation occurs even on a certain wall, radio waves can pass through. Here, it is assumed that the user may not notice the icon connected to the external display or may not understand the meaning of the icon. Then, if you continue to operate the smartphone in a room other than the room where the external display is installed, the mail displayed on the smartphone, the operation of the browser, and the screen for browsing the phone book will be displayed on the external display, contrary to the user's intention. It is possible that it will be viewed by a third party.

It should be noted that the user cannot know that the display content of the smartphone is displayed on the external display only by making a judgment of the display gaze as in the prior art, and the user does not intend to use the external display. The display cannot be suppressed.

In one aspect, it is an object of the present invention to be able to prevent information leakage unintended by the user.

In one proposal, the display control device is wirelessly connected to the external display, and in the display control device that displays the information of the own device on the external display, the display control device is wirelessly connected to the external display before the own device goes to sleep. If this is the case, it is required that the user has a control unit that determines whether or not the own device is unintentionally connected to the external display when returning from the sleep state.

According to one embodiment, there is an effect that information leakage unintentional by the user can be prevented.

FIG. 1 is a block diagram showing an internal configuration of a display control device according to an embodiment. FIG. 2 is a diagram showing an example of a display control system configuration including a terminal according to the embodiment. FIG. 3 is a flowchart showing a display control example of the display control device according to the embodiment. FIG. 4 is a flowchart illustrating the transition of the accumulated data by the display control device according to the embodiment. FIG. 5 is a chart showing an example of transition of accumulated data by the display control device according to the embodiment. FIG. 6 is a diagram showing a pop-up display example by the display control device according to the embodiment. FIG. 7 is a flowchart illustrating a state of disconnection for each application based on the pop-up display by the display control device according to the embodiment. FIG. 8 is a chart showing a transition example of the monitoring time at the time of disconnection for each application by the display control device according to the embodiment.

(Embodiment)
FIG. 1 is a block diagram showing an internal configuration of a display control device according to an embodiment. The display control device is, for example, a mobile device carried by a user, and is a terminal such as a smartphone or a tablet.

As described above, the terminal 100 performs wireless communication with a cast device mounted on an external display such as a TV via the communication unit 105, and transfers the image information displayed on the display unit of the terminal 100 to the external display. Display it.

Then, the terminal 100 of the embodiment notifies the user that it is in a state of being wirelessly connected to the external display at an appropriate time, for example, at the time of operation after the terminal 100 is restored. This prevents the image information on the terminal 100 from being displayed on the external display against the intention of the user. As described above, it is assumed that after the terminal 100 is connected to the external display, the terminal 100 goes to sleep after a predetermined time has elapsed. Due to such a lapse of time, the user may forget that the terminal 100 maintains a wireless connection to the external display, and prevents leakage of user information via the external display in such a case.

FIG. 1 mainly describes a configuration related to a display control function of a terminal 100 such as a smartphone. The terminal 100 includes a control unit 101, a photographing unit 102, a storage unit 103, a display unit 104, and a communication unit 105. In addition, the terminal 100 includes an operation unit and the like, and for example, the operation unit is a transparent touch panel provided on the display unit 104.

The photographing unit 102 is, for example, a camera possessed by the terminal 100, and based on the control of the control unit 101, captures image information captured by the camera and outputs the captured image information to the control unit 101. The display unit 104 is, for example, a display such as an LCD owned by the terminal 100, and displays the contents of various applications on the screen based on the control of the control unit 101. The photographing unit 102 is, for example, an in-camera that photographs the face (line of sight) of the user who operates while looking at the display unit 104 of the terminal 100.

The control unit 101 supervises the control of each part of the terminal 100 and controls the display of the embodiment. The control unit 101 includes an image acquisition unit 111, a monitoring time control unit 112, and a determination unit 113.

The image acquisition unit 111 acquires the image information captured by the imaging unit 102 at a predetermined timing for a predetermined monitoring time, and outputs the image information to the determination unit 113. The monitoring time control unit 112 includes a monitoring time reading unit 121 and a monitoring time writing unit 122. The monitoring time reading unit 121 reads the monitoring time stored in the storage unit 103 (ROM131). The monitoring time writing unit 122 writes a new monitoring time in the storage unit (ROM131).

The determination unit 113 analyzes the image information of the user's face acquired by the image acquisition unit 111, and determines whether or not the user is gazing at the screen of the terminal 100. Then, the determination unit 113 outputs the determination result to the monitoring time control unit 112 and the display unit 104. For example, when the determination result of the determination unit 113 is that the user is "looking at" the screen, the monitoring time writing unit 122 updates the monitoring time, and the display unit 104 shows the state in which the terminal 100 is connected to the external display. A display is displayed to notify the user of the fact.

The storage unit 103 includes, for example, a readable / writable ROM 131 and a RAM 132, and stores information related to display control of the embodiment. The ROM 131 and the RAM 132 each execute reading and writing of information based on the control of the control unit 101.

The ROM 131 holds the monitoring time for each application that can be operated by the user as the accumulated data D. The monitoring time is a time preset for each application. The application is installed in the terminal 100 by a user operation, for example, and is stored and held in the storage unit 103 (ROM131).

The ROM 131 stores and holds the monitoring time read and written by the monitoring time control unit 112 as accumulated data D. The stored data D also includes information on the application running on the terminal 100, and the monitoring time for each application is set. The monitoring time for each application is updated and stored so as to be shorter or longer for each determination process by the control unit 101 (determination unit 113). The RAM 132 stores work data at the time of control execution of the control unit 101.

The communication unit 105 is composed of a wireless transmitter / receiver, an antenna, and the like that perform wireless communication with a cast device mounted on an external display under the control of the control unit 101. The communication unit 105 transmits the image information displayed on the display unit 104 to the cast device and displays it on the screen of the external display. The communication unit 105 is, for example, a wireless interface (IF) and wirelessly connects to and from the cast device. In addition, it has a function of wirelessly connecting to a base station via a mobile phone network.

The control unit 101 includes, for example, a CPU, and controls the operation of each unit of the terminal 100 by executing a control program stored in the ROM 131 of the storage unit 103. In addition, the control unit 101 programmatically executes various applications installed in the terminal 100 and stored in the ROM 131 or the like. When executing the program, the CPU uses the RAM 132 as a storage area for work data.

FIG. 2 is a diagram showing an example of a display control system configuration including a terminal according to the embodiment. In the display control system 200 shown in FIG. 2, the terminal 100 wirelessly connects to the cast device (communication device) 201, and the image information to be displayed on the terminal 100 is displayed on the external display 210 such as a TV via the cast device 201. This is a configuration example.

The terminal 100 has each function shown in FIG. The cast device 201 includes a control unit 202, a communication unit 203, and an output unit 204. The control unit 202 controls the cast device 201. The communication unit 203 includes a wireless transmitter / receiver, an antenna, and the like that perform wireless communication with the communication unit 105 of the terminal 100. The communication unit 203 receives the image information transmitted from the terminal 100 and outputs it to the control unit 202. The control unit 202 outputs the image information to the external display 210 via the output unit 204. The output unit 204 outputs image information through an interface such as HDMI.

The external display 210 includes a control unit 211, an input unit 212, and a display unit 213. The control unit 211 controls the external display. The input unit 212 outputs the image information input from the cast device 201 to the control unit 211. Image information is input to the input unit 212 by an interface such as HDMI. The display unit 213 is a display such as an LCD that displays the image information output by the control unit 211 on the screen. The terminal 100 and the external display 210 display image information by mirroring or multi-display as described above.

Further, the external display 210 may include a communication unit 214. When the external display 210 has the communication unit 214, the communication unit 214 can directly receive the image information transmitted by the terminal 100. That is, when the communication unit 214 is provided, it has the function of the cast device 201, and the cast device 201 can be eliminated. The communication unit 214 receives the image information transmitted from the terminal 100 and displays it on the display unit 213 via the control unit 211.

Regarding the cast device 201 and the external display 210 shown in FIG. 2, the control units 202 and 211, respectively, can be configured by using hardware such as a CPU. By executing a control program stored in a storage unit (for example, ROM) (not shown) by these CPUs, the functions of the cast device 201 and the external display 210 shown in FIG. 2 can be realized. At this time, the CPU uses a storage unit (for example, RAM) (not shown) as a storage area for work data.

(Terminal display control example)
Next, a display control example performed by the control unit 101 of the terminal 100 will be described. If the terminal 100 goes to sleep while the terminal 100 is connected to the external display 210, the terminal 100 maintains the connection to the external display 210, and the external display 210 displays a black screen, a lock screen, or the like. When the terminal 100 wakes up from the sleep state by the user operation and the unlock operation is performed, the screen of the smartphone is displayed on the external display 210 again. After that, the connection is maintained unless the user disconnects.

The control unit 101 of the terminal 100 wakes up the terminal 100 connected to the external display 210 from the sleep state, and after the unlocking operation, detects the operation of the terminal 100 by the user. When the control unit 101 detects these operations, the control unit 101 acquires the information of the foreground application running at that time from the accumulated data D of the storage unit 103 (ROM131) and the monitoring time corresponding to this application.

Then, the control unit 101 activates the photographing unit 102, starts the user's face detection with the in-camera, performs the image processing of the line-of-sight detection by the determination unit 113, and during the monitoring time, the user displays the display unit 104 of the terminal 100. Determine if you are looking at the screen (gaze). When the control unit 101 (determination unit 113) determines that the user is looking at the display unit 104, the control unit 101 notifies the display unit 104 of the disconnection of the terminal 100 to the external display 210 on a pop-up display screen or the like.

As a result, the user of the terminal 100 can grasp that the terminal 100 is connected to the external display 210 by the notification of the disconnection, and the user can determine whether or not the terminal 100 is disconnected from the external display 210. It can be executed as needed.

For example, if the user forgets that the terminal 100 is connected to the external display 210, the user can be made aware of this. Then, it is possible to prevent the image information being displayed by the terminal 100 from being continuously displayed on the display unit 213 of the external display 210 against the intention of the user.

Further, when a third party other than the user of the terminal 100 performs a connection operation between the user's terminal 100 and the external display 210, the user does not know that the connection state is established, so that the image of the terminal 100 is unintentionally displayed. The information will continue to be displayed on the external display. Even in such a case, the terminal 100 can notify that the terminal 100 is connected to the external display 210 at the time of the user's operation, and can perform the release operation.

As described above, the wireless connectable range extends to a dozen meters, and even when the terminal 100 is moved to a room where the external display 210 is installed and a separate room, radio waves pass and the wireless connection is maintained with each other. There is. At this time, the user may not notice the connected icon displayed on the external display 210 by the terminal 100. In addition, you may not understand the meaning of the icon, or you may have forgotten that you are connected. In such a state, when the user continues to operate the terminal 100 in a room different from the room in which the external display 210 is installed, the mail displayed on the terminal 100, the operation of the browser, and the screen for browsing the telephone directory are displayed in the separate room. It is conceivable that the display is displayed on the external display 210 of the above and is viewed by a third party against the intention of the user.

Further, even if the user is in the room where the external display 210 is installed, for example, it is assumed that the user turns his / her back to the external display 210. Similarly, in this case as well, the image information being displayed by the terminal 100 is displayed on the display unit 213 of the external display 210 against the intention of the user because the user has forgotten that the connection is being made, and the third It is conceivable that it will be viewed by a person.

Even if the user forgets the connection state between the terminal 100 and the external display 210 and moves to another room as in the above example, the terminal 100 is connected to the external display 210 when the user operates the terminal 100. Notify the user that it is in a state. Then, the user can appropriately release the connection state of the terminal 100 to the external display 210.

FIG. 3 is a flowchart showing a display control example of the display control device according to the embodiment. The control contents executed by the control unit 101 of the terminal 100 will be described.

First, the terminal 100 determines whether the image information displayed by the terminal 100 is being output to the external display 210 (step S301). If the image information is being output to the external display (step S301: Yes), then the terminal 100 determines whether the own terminal 100 has returned from the sleep state (step S302). On the other hand, when the image information is not being output to the external display (step S301: No) and when the own terminal 100 is not in the state of returning from the sleep state (step S302: No), the process ends.

In step S302, if the own terminal 100 has returned from the sleep state (step S302: Yes), the terminal 100 accepts the unlock operation by the user (step S303). For example, by a predetermined unlock operation (for example, password input) by the user, the lock state which is the lock screen or the black screen due to the sleep state is released, and the terminal 100 can be operated.

After the unlock operation, the terminal 100 then detects the operation content of the terminal 100 by the user (step S304). For example, it detects a specific flick operation or swipe operation as a user operation. When the user's operation is detected (step S304: Yes), the terminal 100 acquires the information of the foreground application running at that time (step S305). The storage unit 103 (ROM131) stores application information and monitoring time for each application as previously stored data D. If no user operation is detected (step S304: No), the above processing ends.

The terminal 100 (control unit 101) refers to the storage unit 103, reads out the monitoring time corresponding to the foreground application, and sets it as the current monitoring time (step S306). After that, the terminal 100 activates the in-camera of the photographing unit 102 (step S307).

Next, the terminal 100 (control unit 101) starts photographing the user's face by the photographing unit 102 (in-camera) (step S308), performs image processing by the determination unit 113, and performs the user's face detection determination process. (Step S309). This face detection is continuously performed for the monitoring time, and after the monitoring time elapses, the face detection determination process by the determination unit 113 ends (step S310), and the photographing unit 102 (in-camera) is stopped (step S311).

Then, the terminal 100 (determination unit 113) determines whether or not the user has continuously viewed the terminal 100 (display unit 104) during the monitoring time as the face detection determination process (step S312). For example, suppose that it is determined that the user has been continuously watching the terminal 100 (display unit 104) during the monitoring time (step S312: Yes). In this case, the control unit 101 notifies the display unit 104 of disconnection from the wireless connection with the external display 210 by a pop-up display or the like (step S313). On the other hand, when it is determined that the user is not looking at the terminal 100 (display unit 104) during the monitoring time (step S312: No), the terminal 100 ends the above processing.

After that, the terminal 100 disconnects the wireless connection if the user selects to disconnect the external display 210 from the terminal 100, and maintains the wireless connection without disconnecting if the disconnection is not selected. Then, the terminal 100 records the information of the application running on the terminal 100 at that time and the status of the wireless connection with the external display 210 when the user selects the disconnection based on the pop-up display. (Step S314). At this time, the monitoring time corresponding to the started application is recorded in the storage unit 103 (ROM131) as the accumulated data D based on a predetermined determination. If the application has already been recorded as accumulated data D, the monitoring time is updated and the above processing is completed.

As will be described later, the terminal 100 sets the monitoring time short for each of the above processes, and when disconnection is not selected, the foreground application used at that time is set in the accumulated data D, and the monitoring time is set to be higher than this time. Set a long monitoring time.

Further, a fixed time (1 minute or the like) is set as an initial setting for the monitoring time for determining whether the user is viewing the screen of the display unit 104 of the terminal 100. Then, when the user determines that the output is unintended to the external display 210 and performs the connection disconnection operation, the terminal 100 newly records the application running at that time. In the next and subsequent processes, if the terminal 100 outputs to the external display 210 while the same application is running, the terminal 100 notifies the user to disconnect from the connection in a shorter monitoring time than in the previous process.

On the other hand, if the user is urged to disconnect, but the disconnection operation is not performed, the terminal 100 determines that the image information is not output to the unintended external display 210, and is activated at that time. Record the application. If the same application is output to the external display 210 while the same application is running after the next process, the monitoring time is set longer than the previous time. Further, the monitoring time is set when the flick or swipe operation in step S304 is detected for the first time after the unlock operation (step S303) by the user described above. After the shooting unit 102 (in-camera) is started (step S307), the monitoring time is not changed even if the foreground application that is started is changed by the user's operation.

In the above description, the operations performed by the user while looking at the screen of the display unit 104 of the terminal 100 are fine touch operations (unlocking, application selection, browser page transition, photo and video selection). After these fine touch operations, a flick operation, a swipe operation, etc. are performed while looking at the external display 210. Based on this tendency, in step S304, a flick or swipe operation is taken as an example of a user operation performed while looking at the external display 210. As a result, the monitoring time used for the gaze determination process can be set to an appropriate time corresponding to the time required for the user's operation.

Further, in the embodiment, the terminal 100 is used in a state where the user is away from the external display 210 or the screen of the external display 210 cannot be seen while the terminal 100 is wirelessly connected to the external display 210. Is assumed. Correspondingly, if the user continues to look at the screen of the terminal 100 even after detecting the flick operation or the swipe operation of the user, the terminal 100 outputs the image information to the external display 210 unintentionally. I will judge.

The above-mentioned accumulated data D is (1) read from the storage unit 103 (ROM131) for setting the monitoring time, and (2) updated and written in the storage unit 103 (ROM131) based on the face detection determination result.

(1) Reading of stored data D FIG. 4 is a flowchart for explaining the transition of the stored data by the display control device according to the embodiment, and FIG. 5 is an example of the transition of the stored data by the display control device according to the embodiment. It is a chart which shows. A transition example of the stored data D stored in the storage unit 103 (ROM131) will be described with reference to FIGS. 4 and 5. The processing shown in FIG. 4 corresponds to the processing contents of steps S305 and S306 of FIG.

First, as shown in FIG. 4, the terminal 100 (control unit 101) acquires information on the application running on the terminal 100 (step S401). For example, when an application called "application B" is running, the terminal 100 confirms whether the information of the application corresponding to the stored data D of the storage unit 103 (ROM131) exists (step S402).

Here, if the "app B" is a new application, as shown in FIG. 5A, the information of the "app B" is not stored in the stored data D (step S402: No). The initial monitoring time is newly set in the accumulated data D (step S403). For example, as shown in FIG. 5B, “60 seconds” is set as the initial value as the monitoring time of the “app B (com.application.appB)” and recorded as new accumulated data in the storage unit 103. After that, the terminal 100 sets the monitoring time used in this processing (face detection determination processing) to "60 seconds" (step S404) based on the initially set accumulated data D, and proceeds to step S307 of FIG. ..

In step S402, when the information of the running application exists in the stored data D (step S402: Yes), the stored stored data D is read out and the monitoring time is set in step S404.

(2) Display of Pop-up and Writing of Stored Data D FIG. 6 is a diagram showing an example of pop-up display by the display control device according to the embodiment. As shown in FIG. 6, the terminal 100 performs a pop-up display 600 on the display unit 104 in the process of step S313 of FIG. For example, as shown in the figure, the user is notified that "Currently connecting to an external display. Do you want to disconnect?" On the pop-up display 600, a "Yes / No" selection button 601 for selecting the disconnection of the wireless connection with the external display 210 of the terminal 100 is displayed, and the disconnection can be selected by the user operation.

The pop-up display 600 shown in FIG. 6 is displayed on the display unit 104 when the terminal 100 returns from the sleep state, that is, before the terminal 100 enters the sleep state, for example, as shown in the processing example of FIG. Displayed in a pop-up before the displayed image is displayed. As a result, the user can disconnect the wireless connection from the external display 210 by the pop-up display 600. For example, when returning from the sleep state, the terminal 100 controls not to output the display image of the terminal 100 to the external display 210 once if the terminal 100 is wirelessly connected to the external display 210, and the user selects the selection button 601. Based on the above, the configuration may be such that the presence / absence of output of the display image to the external display 210 is switched.

FIG. 7 is a flowchart illustrating a state of disconnection for each application based on the pop-up display by the display control device according to the embodiment. FIG. 8 is a chart showing a transition example of the monitoring time at the time of disconnection for each application by the display control device according to the embodiment.

An example of transition of the accumulated data D when the user selects “Yes” with the selection button 601 of the pop-up display 600 and cancels the wireless connection with the external display 210 of the terminal 100 by the pop-up display will be described. The process shown in FIG. 7 corresponds to the process contents of steps S313 and S314 of FIG.

First, the terminal 100 performs a pop-up display 600 (step S701) and notifies the user of the contents of FIG. Then, when the user presses the "Yes" selection button 601, the terminal 100 starts and acquires the information of the application used by the user, and performs a process of disconnecting from the external display 210 (step S702). ..

Here, it is assumed that the application used by the user at the time of the pop-up display 600 (the application used when reading the accumulated data D described with reference to FIGS. 4 and 5) is "application B" (step). S703). In the case of this "app B", the terminal 100 reduces the monitoring time of the "app B" by 10 seconds from the recorded value. That is, in FIG. 5 (b), the monitoring time of "App B" was "60 seconds", but as shown in FIG. 8 (a), the terminal 100 reduced the read monitoring time to "50 seconds". The update process of changing and writing to the storage unit 103 (ROM131) is performed (step S704).

Further, it is assumed that the application used when the pop-up is displayed is "App C" (step S705). This "application C" is already stored as accumulated data D, and the monitoring time of this "application C" is also reduced by 10 seconds from the recorded value. That is, in FIG. 5 (b), the monitoring time of "App C" was "40 seconds", but as shown in FIG. 8 (b), the terminal 100 changed the monitoring time to "30 seconds". The update process for writing to the storage unit 103 (ROM131) is performed (step S704).

Further, it is assumed that the application used when the pop-up is displayed is "App D" (step S706). This "application D" is not stored as accumulated data D. Therefore, the terminal 100 determines that the "application D" is a new application. Here, even if it is a new application, the monitoring time is reduced by 10 seconds from the initial value after the pop-up is displayed. That is, the initial value monitoring time is "60 seconds", but as shown in FIG. 8C, the terminal 100 changes the monitoring time to "50 seconds" and provides information on the new application "App D". At the same time, an update process for writing to the storage unit 103 (ROM131) is performed (step S704).

When the "No" selection button 601 is pressed in the pop-up display 600 in step S701, the terminal 100 increases the monitoring time by "10 seconds" for each application in the monitoring time update process (step S704). Perform processing.

According to the above, the terminal 100 updates the monitoring time shorter every time the user disconnects after processing the pop-up display 600, as the application is frequently used by the user. On the contrary, after the processing of the pop-up display 600, the monitoring time of the application is updated for a long time every time the user does not disconnect. As a result, the continuous monitoring time as the face detection judgment process can be appropriately set according to the usage state of each application of the user, and the user's face detection (screen gaze judgment) can be performed by the monitoring time suitable for the user's usage state. You will be able to do it accurately.

According to the embodiment described above, the terminal determines the connection of the terminal to the external display in a state not intended by the user, and the image information being displayed by the terminal is displayed on the external display. Notify the user of this. As a result, the user can easily grasp the state in which the image information is displayed on the external display unintentionally, and can easily and appropriately disconnect the terminal from the external display when necessary.

In addition, by notifying the user when the user operates the terminal, it becomes possible to notify the user of the disconnection at an appropriate time. Then, if it is an operation while looking at the external display based on a peculiar operation performed by the user when the terminal is connected to the external display, for example, a flick or swipe operation, monitoring for determining the line-of-sight gaze Set the time. As a result, the monitoring time used for the user's line-of-sight gaze using the monitoring time can be set to an appropriate time.

Further, the time corresponding to the user operating the terminal is set as the monitoring time, and it becomes possible to accurately determine whether or not the user is gazing at the terminal continuously during the monitoring time. Furthermore, the foreground application used at the time of gaze judgment and the result of disconnection and connection continuation by user operation are recorded as accumulated data, and the monitoring time is updated for each next process. As a result, the learning effect makes it possible to improve the gaze determination accuracy using the monitoring time. For example, the application newly used by the user has an initial monitoring time, and the application frequently used by the user shortens the monitoring time for each judgment, so that the judgment time can be shortened while maintaining the judgment accuracy. The above-mentioned notification to the user can be quickly performed.

Further, in the embodiment, information leakage for a state in which some characters or images are displayed from the terminal to the external display has been described as an example. Not limited to this, information leakage of the terminal when some characters or images are not displayed on the external display from the terminal in the current state, but some characters or images are displayed by continuing the operation of the terminal. Can also be prevented. For example, suppose that when the terminal is wirelessly connected to the external display, no characters or images are displayed on the external display, and the user does not notice that the terminal is wirelessly connected to the external display. Then, when the user continues to operate the smartphone in a room different from the room in which the external display is installed, it is possible to prevent the screen displayed on the smartphone from being displayed on the external display, and the user's intention. It is possible to prevent information leakage to a third party or the like contrary to the above.

The display control method described in the present embodiment can be realized by executing a control program prepared in advance on a computer (CPU of a display control device or the like) of a target device. This control program is executed by being recorded on a computer-readable recording medium such as a magnetic disk, an optical disk, or a flash memory, and being read from the recording medium by the computer. Further, the control program may be distributed via a network such as the Internet.

The following additional notes are further disclosed with respect to the above-described embodiment.

(Appendix 1) In a display control device that wirelessly connects to an external display and displays information on its own device on the external display.
If the own device is wirelessly connected to the external display before it goes to sleep, is the user unintentionally connected to the external display when returning from the sleep state? A display control device including a control unit for determining.

(Appendix 2) It has a photographing unit that photographs the face of the user who operates the own device.
The control unit
The photographing unit is activated at the time of user operation after returning from the sleep state, and continuously for a predetermined monitoring time, it is determined whether or not the user who has photographed the photographing unit is gazing at the own device, and the determination result is obtained. The display control device according to Appendix 1, wherein the information of the own device is notified that the information of the own device is displayed on the external display.

(Appendix 3) It has a display unit that displays various information of its own device.
The description in Appendix 2, wherein the control unit displays on the display unit a notification that the information of the own device is displayed on the external display when it is determined that the user is gazing at the own device. Display control device.

(Appendix 4) A storage unit for storing the monitoring time is provided.
The control unit
When the own device detects a user's peculiar operation when wirelessly connected to the external display, the monitoring time is set for each type of application used by the user on the own device when the operation is performed. The display control device according to Appendix 2 or 3, wherein is set in the storage unit.

(Appendix 5) The control unit
An application peculiar to the user when the own device is wirelessly connected to the external display is detected, and the application used by the user on the own device when the operation is performed is not set in the storage unit. In this case, the display control device according to Appendix 4, wherein the application is determined to be a new application, and a predetermined initial value is set as the monitoring time.

(Appendix 6) The control unit
The display control according to Appendix 4 or 5, characterized in that a flick or swipe operation performed by the user while looking at the external display is detected as a user-specific operation when the own device is wirelessly connected to the external display. apparatus.

(Appendix 7) The control unit
At the time of the operation of canceling the wireless connection with the external display by the notification to the user, any one of the appendices 2 to 6 characterized in that the monitoring time of the application used by the user is shortened and the update setting is made. The display control device described in.

(Appendix 8) The control unit is
When the wireless connection with the external display is not canceled by the notification to the user, any of Appendix 2 to 7, characterized in that the monitoring time of the application used by the user is lengthened and updated. The display control device described in one.

(Appendix 9) The control unit
The display control device according to any one of Supplementary note 1 to 8, wherein the image information of the own device is displayed on the external display via a communication device mounted on the external display.

(Appendix 10) In a display control method of a display control device that wirelessly connects to an external display and displays information of the own device on the external display.
If the own device is wirelessly connected to the external display before it goes to sleep, is the user unintentionally connected to the external display when returning from the sleep state? A display control method characterized by determining.

(Appendix 11) In a display control program executed by a computer of a display control device that wirelessly connects to an external display and displays information of its own device on the external display.
If the own device is wirelessly connected to the external display before it goes to sleep, is the user unintentionally connected to the external display when returning from the sleep state? To judge,
A display control program characterized by causing the computer to execute processing.

100 Display control device (terminal)
101, 202, 211 Control unit (CPU)
102 Imaging unit 103 Storage unit 104, 213 Display unit 105, 203, 214 Communication unit 111 Image acquisition unit 112 Monitoring time control unit 113 Judgment unit 121 Monitoring time reading unit 122 Monitoring time writing unit 131 ROM
132 RAM
200 System 201 Cast device 204 Output unit 210 External display 212 Input unit 600 Pop-up display 601 Select button D Stored data

Claims (9)

In a display control device that wirelessly connects to an external display and displays information about its own device on the external display.
A shooting unit that shoots the face of the user who operates the own device,
If the device is wirelessly connected to the external display before the device goes to sleep, the photographing unit is activated at the time of user operation after returning from the sleep state, and continues for a predetermined monitoring time. It is characterized by having a control unit that determines whether the user who has taken a picture is gazing at the own device and notifies that the information of the own device is displayed on the external display based on the determination result. Display control device. It has a display unit that displays various information of its own device.
Wherein, at the time of determination that the user is gazing at the own device, according to claim 1, wherein the displaying the notification is a state where the information of the device is displayed on the external display on the display unit Display control device. It has a storage unit that stores the monitoring time.
The control unit
When the own device detects a user's peculiar operation when wirelessly connected to the external display, the monitoring time for each type of application used by the user on the own device when the operation is performed. The display control device according to claim 1 or 2 , wherein is set in the storage unit. The control unit
An application peculiar to the user when the own device is wirelessly connected to the external display is detected, and the application used by the user on the own device when the operation is performed is not set in the storage unit. The display control device according to claim 3 , wherein the display control device is determined to be a new application, and a predetermined initial value is set as the monitoring time. The control unit
The display according to claim 3 or 4 , wherein a flick or swipe operation performed by the user while looking at the external display is detected as a user-specific operation when the own device is wirelessly connected to the external display. Control device. The control unit
Any one of claims 1 to 5 , characterized in that, at the time of the operation of canceling the wireless connection with the external display by the notification to the user, the monitoring time of the application used by the user is shortened and the update setting is made. The display control device described in 1. The control unit
Claims 1 to 6 , wherein when the wireless connection with the external display is not canceled by the notification to the user, the monitoring time of the application used by the user is lengthened and updated. The display control device according to any one. In a display control method of a display control device including a photographing unit that wirelessly connects to an external display, displays information of the own device on the external display, and photographs the face of a user who operates the own device .
If the device is wirelessly connected to the external display before the device goes to sleep, the photographing unit is activated at the time of user operation after returning from the sleep state, and continues for a predetermined monitoring time. A display control method characterized in that the photographing unit determines whether or not the user who has taken a picture is gazing at the own device, and based on the determination result, notifies that the information of the own device is displayed on the external display. .. In a display control program executed by a computer of a display control device including a photographing unit that wirelessly connects to an external display, displays information of the own device on the external display, and photographs the face of a user who operates the own device .
If the device is wirelessly connected to the external display before the device goes to sleep, the photographing unit is activated at the time of user operation after returning from the sleep state, and continues for a predetermined monitoring time. The photographing unit determines whether or not the user who has taken a picture is gazing at the own device, and based on the determination result, notifies that the information of the own device is displayed on the external display .
A display control program characterized by causing the computer to execute processing.

Images