JP7704022B2 - Display device, system and program - Google Patents

Description

The present invention relates to a display device, a system, and a program.

Conventionally, technology has been developed that connects a projector and a terminal device via a network and displays characters on the projector based on a character input operation on the terminal device. For example, the system described in Patent Document 1 below converts handwritten characters input to a terminal device equipped with a touch panel into a character code and transmits the character code to the projector. Alternatively, the system displays a software keyboard on the terminal device and transmits the character code corresponding to the touched character to the projector. As a result, a character string based on the character code is displayed in the projection image of the projector.

JP 2016-148858 A

As projectors become more sophisticated, there may be a need to input character strings more quickly, in addition to the conventional simple operations using a cursor or buttons. In the above-mentioned Patent Document 1, although it is possible to input characters using a terminal device connected to a projector, the use of the input characters is limited to displaying the characters in a projected image. In Patent Document 1, when multiple applications can be executed on the projector side, it is not assumed that the terminal device will be used as a common interface for inputting characters into these multiple applications.

A display device according to one aspect of the present invention includes a processing device that executes a first communication device that communicates with a first terminal device, an optical device, a first service application that provides a service, a keyboard system application that is called by the first service application and provides a character input function, a first communication device driver that controls the first communication device, and an operating system that manages the first service application, the keyboard system application, and the first communication device driver, and the keyboard system application uses the first communication device driver to obtain first character information indicating characters input on the first terminal device, and the first service application uses the optical device to perform display based on the first character information.

The system according to one aspect of the present invention is a system including a display device and a terminal device, the display device includes a display device side communication device that communicates with the terminal device, an optical device, a service application that provides a service, a keyboard system application that is called by the service application and provides a character input function, a display device side communication device driver that controls the display device side communication device, and a display device side processing device that executes an operating system that manages the service application, the keyboard system application, and the display device side communication device driver, the terminal device includes a terminal device side processing device that executes a terminal device side communication device that communicates with the display device, a terminal device side communication device driver that controls the terminal device side communication device, and a character input application that accepts character input by a user, the character input application transmits character information corresponding to the character to the display device using the terminal device side communication device driver, the keyboard system application acquires the character information using the display device side communication device driver, and the service application uses the optical device to perform display based on the character information.

A program according to one aspect of the present invention causes a processing device to obtain character information indicating characters input on a terminal device using a communication device driver that controls a communication device that communicates with the terminal device, and to provide the character information to a service application that provides a service.

1 is a diagram showing a configuration of a display system 1 according to an embodiment. FIG. 2 is a block diagram showing a hardware configuration of the projector 10. FIG. 2 is a block diagram showing a hardware configuration of the smartphone 20. 3 is a block diagram showing a program executed by a processing device 110 of the projector 10. FIG. 2 is a block diagram showing a program executed by a processing device 210 of the smartphone 20. FIG. 1 is a schematic diagram of an image PC2 showing a home screen of a pay movie distribution service application AP2. 1 is a schematic diagram of an image PC3 showing a home screen of a smartphone 20. FIG. FIG. 2 is a schematic diagram showing a user interface of a keyboard service application AP3. FIG. 2 is a schematic diagram showing a user interface of a keyboard service application AP3. FIG. 2 is a schematic diagram showing a user interface of a keyboard service application AP3. FIG. 13 is a schematic diagram showing another user interface of the keyboard service application AP3. 1 is a schematic diagram of an image PC6 showing a home screen of a pay movie distribution service application AP2. 5 is a flowchart showing the operation of the processing device 110 of the projector 10. 4 is a flowchart showing an operation of a processing device 210 of the smartphone 20. FIG. 13 is a diagram showing a configuration of a display system 2 according to a second embodiment. FIG. 2 is a block diagram showing the hardware configuration of a remote controller 30. FIG. 11 is a block diagram showing a hardware configuration of a projector 10 in a second embodiment. FIG. 11 is a block diagram showing a program executed by a processing device 110 of a projector 10 in a second embodiment. FIG. 11 is a schematic diagram of an image PC8 showing a screen of a pay movie distribution service application AP2 in the second embodiment.

Below, preferred embodiments of the present invention will be described with reference to the attached drawings. Note that the dimensions or scale of each part in the drawings may differ from the actual dimensions, and some parts are shown diagrammatically to facilitate understanding. Furthermore, the scope of the present invention is not limited to these forms unless otherwise specified in the following description to the effect that the present invention is limited thereto.

1. First embodiment [System configuration]
1 is a diagram showing a configuration of a display system 1 according to an embodiment. The display system 1 is an example of a system. The display system 1 includes a projector 10 and a smartphone 20.

Projector 10 is an example of a display device. Projector 10 projects an image PC onto a display surface F such as a wall surface or a screen. Hereinafter, when it is necessary to identify the image PC projected by projector 10, it will be referred to as image PC1, PC2, etc. Projector 10 is connected to the Internet using a wireless LAN (Local Area Network) communication device 107 described later, and can project information on the Internet onto display surface F without going through a device such as a computer or smartphone 20.

In this embodiment, the projector 10 has an operating system installed similar to that of the smartphone 20 described below. Therefore, the projector 10 can execute various applications that are executed by the smartphone 20. For example, the image PC1 projected by the projector 10 in FIG. 1 is the home screen of the projector 10, and a list of service applications that can be executed by the projector 10 is displayed. In this embodiment, icons IC1 to IC3 are displayed as information that identifies each service application.

Icons IC1 to IC3 are displayed on the image PC1. When the user selects one of the icons IC1 to IC3 with the moving cursor CM displayed on the image PC1, an application corresponding to the selected icon is started. For example, icon IC1 corresponds to an application for a video sharing service. Also, icon IC2 corresponds to an application for a paid movie distribution service. Also, icon IC3 corresponds to an application for a Web (World Wide Web) browser. The movement of the moving cursor CM and the selection of icons IC1 to IC3 may be performed, for example, using an operation button 103 provided on the housing of the projector 10 or a remote controller (not shown). Note that the user interface used to select icons IC1 to IC3 is not limited to the moving cursor CM, and may be in a form in which the selected icon IC1 to IC3 is highlighted in response to a user operation, for example.

The smartphone 20 is an example of a terminal device. The smartphone 20 is, for example, a smartphone 20 owned by a user who is staying in the space in which the projector 10 is placed. In FIG. 1, two smartphones 20 are illustrated: a first smartphone 20A and a second smartphone 20B. The first smartphone 20A is an example of a first terminal device, and the second smartphone 20B is an example of a second terminal device. The number of smartphones 20 connected to the projector 10 may be one, or may be three or more.

In this embodiment, the smartphone 20 is used to input characters in a service application executed by the projector 10. As will be described in detail later, a keyboard service application AP3 for inputting characters in the projector 10 is installed on the smartphone 20. When it is necessary to input characters in a service application AP-P executed by the projector 10, the user starts the keyboard service application AP3 on the smartphone 20 and inputs characters on the keyboard service application AP3.

[Configuration of projector 10]
2 is a block diagram showing the hardware configuration of the projector 10. The projector 10 includes a projection device 101, an operation button 103, a short-range wireless communication device 105, a wireless LAN communication device 107, a storage device 108, and a processing device 110.

The projection device 101 projects an image PC onto a display surface F. The projection device 101 is an example of an optical device. The projection device 101 includes an image processing circuit, a frame memory, a liquid crystal light valve, a light valve driving circuit, a light source, a projection optical system, and the like. When the image processing circuit receives image data from an image supply device such as the processing device 110 or an image server (not shown), the image processing circuit expands the image data into a frame memory and performs necessary image processing. This image processing is, for example, a process of converting the resolution of the image data to the resolution of the liquid crystal light valve, a geometric correction process to eliminate keystone distortion, and the like. The image data after the image processing is converted into an image signal, and the image data is projected as an image by driving the liquid crystal light valve based on the image signal. In addition to the liquid crystal type using the above-mentioned liquid crystal light valve, the projection device 101 may also be, for example, a DLP (Digital Lighting Processing, registered trademark) type.

Operation buttons 103 are provided on the housing of projector 10 and accept various operations. Operation buttons 103 include, for example, a main power button, direction buttons, and a selection button.

The short-range wireless communication device 105 is used for communication between the projector 10 and the smartphone 20. That is, the short-range wireless communication device 105 communicates with the first smartphone 20A. The short-range wireless communication device 105 also communicates with the second smartphone 20B. The short-range wireless communication device 105 is an example of a first communication device and a display device side communication device. The short-range wireless communication device 105 communicates with the smartphone 20 by a short-range wireless communication method such as Bluetooth. Bluetooth is a registered trademark. In this embodiment, the short-range wireless communication device 105 is capable of communicating with multiple devices simultaneously by pairing with each of the multiple devices. For example, the short-range wireless communication device 105 is capable of communicating with the first smartphone 20A and the second smartphone 20B simultaneously.

The wireless LAN communication device 107 is used for communication with a wireless LAN router (not shown). The wireless LAN communication device 107 communicates with the wireless LAN router using a communication method such as Wi-Fi. Wi-Fi is a registered trademark. The wireless LAN router is a wireless LAN device that provides a wireless LAN access point and an intermediary function with the Internet. At least a part of the service application AP-P executed by the processing device 110 provides a service using information on the Internet. For example, a paid movie distribution service corresponding to the icon IC2 obtains video data of a movie specified by a user from a server connected to the Internet and plays the video data. The projector 10 may be connected to the Internet via a wired LAN.

The storage device 108 is a recording medium that can be read by the processing device 110. The storage device 108 includes at least one of, for example, a RAM (Random Access Memory), a ROM (Read Only Memory), an eMMC (embedded Multimedia Card), a DDR DRAM (Double Data Rate Dynamic Random Access Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). The storage device 108 stores various information and a program PG1 that is processed by the processing device 110. This program PG1 includes the programs of various applications shown in FIG. 4. Program PG1 does not refer to a specific program, but rather to the entire program stored in storage device 108.

The processing device 110 is, for example, composed of one or more processors. As an example, the processing device 110 is composed of one or more CPUs (Central Processing Units). The CPU may be built into a SoC (System on a Chip). Some or all of the functions of the processing device 110 may be composed of circuits such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 110 executes various processes in parallel or sequentially. The processing device 110 is an example of a display device side processing device.

[Configuration of smartphone 20]
Fig. 3 is a block diagram showing the hardware configuration of the smartphone 20. The smartphone 20 includes a touch panel 201, a short-range wireless communication device 202, a storage device 205, and a processing device 210. In addition to the configuration shown in Fig. 3, the smartphone 20 may also include a wireless LAN communication device similar to that of the projector 10, a mobile phone communication device for connecting to a mobile phone communication network provided by a mobile phone carrier, and the like.

The touch panel 201 is a device that integrates a display that displays images and an input device that receives various inputs. The touch panel 201 includes a light source, a display panel, and a sensor that detects contact with the surface of the display panel. The touch panel 201 displays various images as a display. The touch panel 201 also detects a touch position, which is the position where an object comes into contact with the surface of the touch panel 201. The content of the input is identified based on the content of the image displayed on the display and the touch position on the display.

The short-range wireless communication device 202 is a communication device that uses the same communication method as the short-range wireless communication device 105, and is used for communication between the projector 10 and the smartphone 20. The short-range wireless communication device 202 is an example of a terminal device-side communication device that communicates with the projector 10.

The storage device 205 is a recording medium that can be read by the processing device 210. The storage device 205 includes, for example, at least one of a RAM, a ROM, an eMMC, a DDR DRAM, and an EEPROM. The storage device 205 stores various information and a program PG2 that is processed by the processing device 210. This program PG2 includes the programs of various applications shown in FIG. 5. The program PG2 does not refer to a specific program, but rather to all of the programs stored in the storage device 205.

The processing device 210 is, for example, configured by one or more processors. As an example, the processing device 110 is configured by one or more CPUs. The CPU may be built into the SoC. Some or all of the functions of the processing device 110 may be configured by circuits such as a DSP, ASIC, PLD, FPGA, etc. The processing device 210 executes various processes in parallel or sequentially. The processing device 210 is an example of a terminal device side processing device.

[Program Configuration of Projector 10]
4 is a block diagram showing programs executed by the processing device 110 of the projector 10. The processing device 110 executes a service application AP-P, a system application SY-P, a hardware driver HD-P, and an operating system OS-P by performing processing according to the program PG1.

The service application AP-P provides a service. For example, the service application AP-P is an application corresponding to each of the icons IC1 to IC3 displayed on the image PC1 shown in FIG. 1. That is, the service application AP-P includes a video-sharing service application AP1, a paid movie distribution service application AP2, and the like. Note that, due to space limitations in the drawings, the word "application" is sometimes written as "app" in the drawings. The video-sharing service application AP1 is an example of a first service application, and the paid movie distribution service application AP2 is an example of a second service application.

The system applications SY-P are called by the service applications AP-P and provide various functions. The system applications SY-P include, for example, a first keyboard system application SY1 and a storage access SY2. The storage access SY2 is a system application that manages whether an album application, which is one of the service applications AP-P, may access the recorded data of the system.

The first keyboard system application SY1 provides a text input function for the service application AP-P. The first keyboard system application SY1 is an example of a keyboard system application. For example, when inputting text using a video hosting service application AP1 or a paid movie streaming service application AP2, the first keyboard system application SY1 is called to enable text input. For example, the first keyboard system application SY1 is called by the video hosting service application AP1 to provide a text input function. The first keyboard system application SY1 is also called by the paid movie streaming service application AP2 to provide a text input function.

Here, a typical keyboard system application, such as the second keyboard system application SY3 described below, has a function of displaying a software keyboard on the display surface F or a display, etc., and accepting the specification of characters on the software keyboard. In contrast, the first keyboard system application SY1 has a function of receiving characters input on another information processing device, such as the smartphone 20.

In this embodiment, the first keyboard system application SY1 has a system access function SA1. The first keyboard system application SY1 can access a short-range wireless communication device driver HD1, which will be described later, by the system access function SA1. When characters are input from the smartphone 20, the system access function SA1 accesses the short-range wireless communication device driver HD1 and acquires character information indicating the characters input on the smartphone 20. That is, the first keyboard system application SY1 acquires character information indicating the characters input on the first smartphone 20A using the short-range wireless communication device driver HD1. The character information indicating the characters input on the first smartphone 20A is the first character information. The first keyboard system application SY1 also acquires character information indicating the characters input on the second smartphone 20B using the short-range wireless communication device driver HD1. The character information indicating the characters input on the second smartphone 20B is the second character information. The character information is, for example, character code information obtained by converting the characters input by the user into character codes.

The service application AP-P that called the first keyboard system application SY1 displays characters on the application screen based on the acquired character information. For example, the video-sharing service application AP1 uses the projection device 101 to perform display based on at least either the first character information or the second character information.

The hardware driver HD-P controls the hardware HW-P that the projector 10 has. The hardware driver HD-P is prepared corresponding to the hardware HW-P that the projector 10 has. In FIG. 4, the short-range wireless communication device 105 and the wireless LAN communication device 107 are illustrated as examples of the hardware HW-P that the projector 10 has. As the hardware drivers HD-P corresponding to these, the short-range wireless communication device driver HD1 and the wireless LAN communication device driver HD2 are executed. The short-range wireless communication device driver HD1 is an example of a first communication device driver and a display device side communication device driver. The short-range wireless communication device driver HD1 controls the short-range wireless communication device 105. In addition, the wireless LAN communication device driver HD2 controls the wireless LAN communication device 107.

The operation system OS-P manages the entire internal system of the projector 10. For example, the operation system OS-P manages a service application AP-P including a video sharing service application AP1, a system application SY-P including a first keyboard system application SY1, and a hardware driver HD-P including a short-range wireless communication device driver HD1. In this embodiment, the operating system OS-P of the projector 10 is the same type of software as the operating system OS-S of the smartphone 20. Specifically, the operating system OS-P may be, for example, Android OS or iOS. Android OS and iOS are registered trademarks.

[Program configuration of smartphone 20]
5 is a block diagram showing a program executed by the processing device 210 of the smartphone 20. The processing device 210 executes a service application AP-S, a system application SY-S, a hardware driver HD-S, and an operating system OS-S by performing processing according to the program PG2. The basic functions of the service application AP-S are the same as those of the service application AP-P. Similarly, the basic functions of the system application SY-S are the same as those of the system application SY-P, the basic functions of the hardware driver HD-S are the same as those of the hardware driver HD-P, and the basic functions of the operating system OS-P are the same as those of the operating system OS-S.

The service application AP-S includes, for example, a keyboard service application AP3 and a video-sharing service application AP4. The video-sharing service application AP4 is an application that enables a user to upload videos that he or she has taken to a service provider's server so that the videos can be viewed by others, and enables a user to view videos that others have uploaded to the service provider's server.

The keyboard service application AP3 is an application for inputting characters from the smartphone 20 to the service application AP-P executed by the projector 10. The keyboard service application AP3 is an example of a character input application.

In this embodiment, the keyboard service application AP3 has a GUI (Graphical User Interface) function SA2 and a system access function SA3. The keyboard service application AP3 can access a short-range wireless communication device driver HD3, which will be described later, by using the system access function SA3. The keyboard service application AP3 also uses the GUI function SA2 to display a character input interface screen, for example, as shown in FIG. 8 to FIG. 10, on the touch panel 201. On the character input interface screen, a second keyboard system application SY3, which will be described later, is called, and keyboard input becomes possible. The user of the smartphone 20 inputs characters to the character input interface screen using the touch panel 201. That is, the keyboard service application AP3 accepts character input by the user. The user is generally the user of the smartphone 20. The keyboard service application AP3 converts characters input by the user into character information. The keyboard service application AP3 also transmits character information corresponding to the characters input by the user to the projector 10 by using the short-range wireless communication device driver HD3.

The keyboard service application AP3 may be provided with an encryption processing function, and the character information may be encrypted by the encryption processing function before being sent to the projector 10. In this case, the first keyboard system application SY1 of the projector 10 acquires the character information encrypted by the smartphone 20. The first keyboard system application SY1 provides the decrypted character information obtained by decrypting the encrypted character information to the service application AP-P. For example, the first keyboard system application SY1 receives first character information encrypted by the first smartphone 20A. The first keyboard system application SY1 provides the decrypted first character information obtained by decrypting the encrypted first character information to the video sharing service application AP1.

The system applications SY-S include, for example, a second keyboard system application SY3 and storage access SY4. Storage access SY4 is a system application that manages whether an album application, which is one of the service applications AP-S, may access the recorded data of the system.

The second keyboard system application SY3 is called by the service application AP-S and provides a text input function. For example, when inputting text using the keyboard service application AP3 or the video-sharing service application AP4, the second keyboard system application SY3 is called to enable text input. Therefore, when the user of the smartphone 20 uses the keyboard service application AP3 to input text to the projector 10, the user can input text using a keyboard of the same form as that used for other service applications such as the video-sharing service application AP4.

The second keyboard system application SY3 has a GUI function SA4 and a system access function SA5. The second keyboard system application SY3 can access the touch panel driver HD4 (described later) by using the system access function SA5. The second keyboard system application SY3 also displays a software keyboard on the touch panel 201 by using the GUI function SA4. The second keyboard system application SY3 identifies the type of characters input by the user by comparing the touch position by the user with the key arrangement on the software keyboard, and accepts them as input characters.

The system application SY-S may include multiple keyboard system applications. Each keyboard system application has its own characteristics, such as different key layouts, input methods for special characters such as emojis, and dictionaries to be used. The user of the smartphone 20 can set which keyboard system application to use when inputting characters in the service application AP-S. This allows the user to input characters more comfortably.

The hardware driver HD-S is prepared corresponding to the hardware HW-S equipped in the smartphone 20. In FIG. 4, a short-range wireless communication device 202 and a touch panel 201 are illustrated as an example of the hardware HW-P equipped in the projector 10. A short-range wireless communication device driver HD3 and a touch panel driver HD4 are executed as the hardware drivers HD-S corresponding to these. The short-range wireless communication device driver HD3 is an example of a terminal device side communication device driver. The short-range wireless communication device driver HD3 controls the short-range wireless communication device 202. Furthermore, the touch panel driver HD4 controls the touch panel 201.

[Details of character input in display system 1]
Next, a detailed description will be given of character input in the display system 1. The following describes an example in which character input is performed in a pay movie distribution service application AP2, which is an example of a service application AP-P.

When icon IC2 is selected on image PC1 shown in FIG. 1, paid movie distribution service application AP2 is launched. FIG. 6 is a schematic diagram of image PC2 showing the home screen of paid movie distribution service application AP2. In addition to thumbnails TH of viewable movies, image PC2 displays a search window WS for searching for viewable movies by keyword. When the user positions the movement cursor CM on the search window WS and presses the enter button, paid movie distribution service application AP2 calls up a first keyboard system application SY1. The first keyboard system application SY1 waits until it receives text information from the short-range wireless communication device driver HD1. Paid movie distribution service application AP2 displays a text input cursor CI in the search window WS to indicate to the user that it is waiting for text input.

When the smartphone 20 is in a standby state for character input, the user launches a keyboard service application AP3 on the smartphone 20. FIG. 7 is a schematic diagram of an image PC3 showing a home screen of the smartphone 20. The image PC3 is displayed on the touch panel 201 of the smartphone 20. Icons IC4 to IC6 are displayed on the image PC3. For example, icon IC4 corresponds to a web browser. Also, icon IC5 corresponds to a video sharing service application AP4. Also, icon IC6 corresponds to a keyboard service application AP3. When the user touches any of icons IC4 to IC6, the application corresponding to that icon IC4 to IC6 is launched.

When icon IC6 is touched on image PC3, keyboard service application AP3 is started. When keyboard service application AP3 is started, pairing between smartphone 20 and projector 10 is performed by short-range wireless communication. In addition, when keyboard service application AP3 is started, the display on touch panel 201 switches to image PC4.

Figures 8 to 10 are schematic diagrams showing the user interface of the keyboard service application AP3. The image PC4 shown in Figure 8 displays an input window WE in which input characters are displayed, and a message MS explaining how to operate. The image PC4 is displayed by the GUI function SA2 of the keyboard service application AP3. When the user taps the input window WE, the keyboard service application AP3 calls up the second keyboard system application SY3. In addition, the display on the touch panel 201 switches to the image PC5 shown in Figure 9.

In the image PC5 shown in FIG. 9, in addition to the input window WE similar to that in FIG. 8, a software keyboard SK1 is displayed. The software keyboard SK1 is displayed by the GUI function SA4 of the second keyboard system application SY3. The software keyboard SK1 in FIG. 9 has a group of keys K1 arranged thereon corresponding to characters whose vowel is "a", allowing flick input. The software keyboard SK1 may be displayed on the screen immediately after the keyboard service application AP3 is started up. In other words, a display like the image PC5 may be displayed immediately after the keyboard service application AP3 is started up.

When the character type switching key K2 is pressed, a software keyboard SK2 with different character types that can be input is displayed, as shown in FIG. 10, for example. The software keyboard SK2 shown in FIG. 10 has a key group K3 corresponding to numbers arranged thereon. In addition, a software keyboard that can input alphabets or a software keyboard that can input pictorial characters may be displayed.

The software keyboards SK1 and SK2 are also displayed when entering text into another service application AP-S, such as a video sharing service application AP4. This allows the user to enter text into the projector 10 using a keyboard that the user is familiar with.

Each time the user inputs a character, the second keyboard system application SY3 passes character information indicating the input character to the keyboard service application AP3. The keyboard service application AP3 uses the GUI function SA2 to display a character image corresponding to the input character in the input window WE. In the example of Figures 9 and 10, the character "hidamari" (sunny) has been input into the input window WE.

When character input is finished, the user presses the completion key K4. When the completion key K4 is pressed, the keyboard service application AP3 accesses the short-range wireless communication device driver HD3 and transmits character information corresponding to the character string entered in the input window WE to the projector 10 using short-range wireless communication. In other words, the keyboard service application AP3 transmits character information corresponding to the characters entered by the user to the projector 10 using the short-range wireless communication device driver HD3.

The timing of transmitting the character information is not limited to when the completion key K4 is pressed. For example, each time the user inputs a character, character information corresponding to that character may be transmitted to the projector 10.

The first keyboard system application SY1 receives the character information transmitted by the keyboard service application AP3 via short-range wireless communication. That is, the first keyboard system application SY1 obtains character information indicating characters input on the smartphone 20 using the short-range wireless communication device driver HD1. The first keyboard system application SY1 also passes the received character information to the paid movie distribution service application AP2.

The paid movie distribution service application AP2 displays an image of characters corresponding to the delivered character information in the search window SW, for example, as shown in image PC6 in FIG. 12. In the example of FIG. 12, the characters "Hidamari" are displayed in the search window SW. That is, the paid movie distribution service application AP2 uses the projection device 101 to perform display based on the character information acquired by the first keyboard system application SY1.

Then, the paid movie distribution service application AP2 searches the movie database using the input character string "Hidamari" and displays a list of movies related to the character string "Hidamari" on the display surface F. The user selects a desired movie from the displayed list of movies and watches that movie.

[Operation of the processing device 110 of the projector 10]
Fig. 13 is a flowchart showing the operation of the processing device 110 of the projector 10. The processing device 110 controls the projection device 101 by using a hardware driver HD-P of the projection device 101 (not shown) to project a home screen such as the image PC1 exemplified in Fig. 1 onto the display surface F (step S100). The processing device 110 waits until one of the service applications AP-P is selected by selecting one of the icons IC1 to IC3 displayed on the home screen (step S102: NO).

When any of the service applications AP-P is selected (step S102: YES), the processing device 110 starts the selected service application AP-P and controls the projection device 101 to display the screen of the service application AP-P (step S104). For example, when the icon IC2 shown in FIG. 1 is selected, the projection device 101 starts the paid movie distribution service application AP2 and displays a home screen such as the image PC2 in FIG. 6 on the display surface F. The screen of the service application AP-P can be switched as appropriate by the user's operation, etc.

Until a character input window is specified on the screen of the service application (step S106: NO), the processing device 110 returns the process to step S104. The character input window of the service application AP-P is, for example, the search window WS in the image PC2 of FIG. 6. When a character input window is specified on the screen of the service application (step S106: YES), the processing device 110 starts the first keyboard system application SY1 (step S108) and waits until character information is received via short-range wireless communication (step S110: NO).

When the text information is received (step S110: YES), the processing device 110 displays an image of the text corresponding to the text information in the input window on the screen of the service application AP-P (step S112). For example, the search window WS of the image PC6 in FIG. 12 displays the character string "Sunny" based on the text information received from the smartphone 20. The processing device 110 returns the process to step S104 until the user closes the service application AP-P (step S114: NO). When the user closes the service application AP-P (step S114: YES), the processing device 110 ends the process according to this flowchart.

[Operation of the processing device 210 of the smartphone 20]
14 is a flowchart showing the operation of the processing device 210 of the smartphone 20. The processing device 210 waits until an instruction to start the keyboard service application AP3 is received (step S200: NO). The instruction to start the keyboard service application AP3 is received by touching an icon IC6 on a home screen such as the image PC3 illustrated in FIG. 7.

When an instruction to start the keyboard service application AP3 is given (step S200: YES), the processing device 210 starts the keyboard service application AP3 and displays the screen of the keyboard service application AP3 on the touch panel 201 by the GUI function SA2 of the keyboard service application AP3 (step S202). The image PC4 in FIG. 8 is an example of the screen of the keyboard service application AP3.

If the input window WE of the keyboard service application AP3 is not touched (step S204: NO), the processing device 210 transitions the process to step S218. If the input window WE of the keyboard service application AP3 is touched (step S204: YES), the processing device 210 launches the second keyboard system application SY3 (step S206).

The processing device 210 uses the GUI function of the second keyboard system application SY3 to display a software keyboard on the touch panel 201 (step S208). For example, the image PC5 in FIG. 9 displays a software keyboard SK1 that allows input of hiragana. The processing device 210 waits until a character key in the software keyboard is touched (step S210: NO), and when a character key is touched (step S210: YES), the processing device 210 displays an image showing a character corresponding to the character key in the input window WE of the keyboard service application AP3 (step S212). In the example of FIG. 9, an image showing the characters "hidamari" is displayed in the input window.

The processing device 210 returns the process to step S210 until the user finishes character input (step S214: NO). End of character input corresponds to, for example, pressing of the completion key K4 shown in FIG. 9, etc. When the user finishes character input (step S214: YES), the processing device 210 transmits character information of the input characters to the projector 10 using short-range wireless communication (step S216). At this time, the processing device 210 accesses the short-range wireless communication device driver HD3 by the system access function SA3 of the keyboard service application AP3, and transmits the character information. For example, in the example of FIG. 9, character information corresponding to the character string "sunny" is transmitted to the projector 10.

The processing device 210 returns the process to step S204 until the user terminates the keyboard service application AP3 (step S218: NO). When the user terminates the keyboard service application AP3 (step S218: YES), the processing device 210 terminates the process according to this flowchart.

[Character input from multiple smartphones 20]
1 , a plurality of smartphones 20 can be connected to the projector 10. For example, when a plurality of users view an image displayed on the display surface F, the smartphones 20 owned by each user can be connected to the projector 10. In this embodiment, when text input is made to the projector 10 from a plurality of smartphones 20, all of the text inputs from the plurality of smartphones 20 are accepted.

That is, the short-range wireless communication device driver HD1 communicates with a first smartphone 20A and a second smartphone 20B that is different from the first smartphone 20A. The first keyboard system application SY1 uses the short-range wireless communication device driver HD1 to obtain first character information indicating characters input on the first smartphone 20A. The first keyboard system application SY1 also uses the short-range wireless communication device driver HD1 to obtain second character information indicating characters input on the second smartphone 20B. The paid movie distribution service application AP2 uses the projection device 101 to perform display based on the first character information and the second character information.

For example, as shown in FIG. 12, if the character string "Hidamari" is input from the first smartphone 20A, and then the character string "2010" is input from the second smartphone 20B before the search button BS is pressed, "Hidamari 2010" is displayed in the search window WS. By allowing one input window to accept character input from multiple smartphones 20, it is possible to improve convenience when multiple users use the projector 10 at the same time.

In addition, the first keyboard system application SY1 may be configured to accept character input only from the smartphone 20 that first inputs characters, for example, the first smartphone 20A or the second smartphone 20B. Also, the first keyboard system application SY1 may be configured to accept character input only from a pre-specified smartphone 20, for example, the first smartphone 20A or the second smartphone 20B.

[GUI Modification of Keyboard Service Application AP3]
Fig. 11 is a schematic diagram showing another user interface of the keyboard service application AP3. In the image PC7 shown in Fig. 11, in addition to the input window WE, the software keyboard SK1, the character type switching key K2, and the completion key K4 similar to those in Fig. 9 and the like, a directional input key K5 and an OK key K6 are displayed.

The directional input key K5 is used, for example, when moving the moving cursor CM on the image PC1 shown in FIG. 1. The directional input key K5 is arranged 360 degrees around the OK key K6. The user touches a position in the directional input key K5 area that corresponds to the direction in which the object to be moved is to be moved when the OK key K6 is the current position of the object to be moved. For example, to move the moving cursor CM in FIG. 1 to the right, the user touches the right side of the OK key K6 in the directional input key K5 area.

The OK key K6 is pressed when determining the position of the object to be moved. For example, if the user wishes to launch a paid movie distribution service application AP2, while the image PC1 in FIG. 1 is being displayed, the user uses the directional input key K5 to move the moving cursor CM to a position where it overlaps with the icon IC2. When the OK key K6 is pressed with the moving cursor CM overlapping with the icon IC2, the processing device 110 recognizes that the icon IC2 has been selected, and launches the paid movie distribution service application AP2.

The shape of the directional input key K5 and the OK key K6 is not limited to that shown in FIG. 11. For example, the directional input key K5 may be a cross-shaped key with keys arranged on the top, bottom, left and right. The OK key K6 may also be arranged at a position separate from the directional input key K5.

In this way, by providing keys that can input directions to the interface of the keyboard service application AP3, it becomes possible to perform operations other than character input using the keyboard service application AP3, thereby improving the operability of the projector 10.

As described above, in the display system 1 according to the first embodiment, the first keyboard system application SY1 of the projector 10 acquires character information indicating characters input on the smartphone 20 using the short-range wireless communication device driver HD1. Furthermore, the service application AP-P of the projector 10 performs display based on the character information using the projection device 101. This allows the user to input characters to the projector 10 using the character input interface of the smartphone 20, which is expected to be used more frequently than the projector 10. This improves the efficiency of character input. Furthermore, since the smartphone 20 is equipped with a touch panel 201, character input can be performed more efficiently than, for example, inputting characters from a software keyboard using a remote controller.

The first keyboard system application SY1 is also called by multiple service applications AP-P, such as a paid movie distribution service application AP2 and a video sharing service application AP1, to provide a text input function. This allows the user to input text using the smartphone 20 in all service applications AP-P installed in the projector 10. This improves the convenience of text input.

In addition, if the character information indicating the characters entered on the smartphone 20 is encrypted before being transmitted and received via short-range wireless communication, it is possible to prevent a third party from illegally using the character information.

The first keyboard system application SY1 also acquires character information from multiple smartphones 20 and passes the acquired character information sequentially to the service application AP-P. This improves the convenience when multiple users operate the projector 10.

2. Second embodiment Hereinafter, a second embodiment of the present invention will be described. In the following exemplary embodiment, for elements whose actions and functions are similar to those of the first embodiment, the reference numerals used in the description of the first embodiment will be used and detailed descriptions of each will be omitted as appropriate.

[System configuration]
15 is a diagram showing the configuration of a display system 2 according to the second embodiment. The display system 2 includes a remote controller 30 in addition to a projector 10 and a smartphone 20.

The remote controller 30 is an operation interface for operating the projector 10 from a remote location. The remote controller 30 has a plurality of operation buttons 320 provided on its housing. When a user presses one of the operation buttons 320, a signal corresponding to that operation button 320 is transmitted from the remote controller 30 to the projector 10. The projector 10 accepts the received signal as an input from the remote controller 30.

[Configuration of remote controller 30]
16 is a block diagram showing the hardware configuration of the remote controller 30. The remote controller 30 includes an infrared light emitting device 301, a storage device 302, and a processing device 310, in addition to the above-mentioned operation buttons 320.

The operation buttons 320 include, for example, a power button for turning the projector 10 on and off, a cross key for specifying a direction, a decision key for confirming a selection, and number keys for inputting numbers.

The infrared light emitting device 301 is, for example, an LED (Light Emitting Diode) light emitting element with a peak wavelength of about 950 nm. The infrared light emitting device 301 is turned on and off under the control of the processing device 310.

The storage device 302 is a recording medium that can be read by the processing device 310. The storage device 302 includes, for example, at least one of a RAM, a ROM, an eMMC, a DDR DRAM, and an EEPROM. The storage device 302 stores various information and a program PG3 that are processed by the processing device 210.

The processing device 310 is, for example, configured with one or more processors. As an example, the processing device 310 is configured with one or more CPUs. The CPU may be built into the SoC. Some or all of the functions of the processing device 310 may be configured with circuits such as a DSP, ASIC, PLD, FPGA, etc. The processing device 310 executes various processes in parallel or sequentially. A microcontroller in which the memory device 302 and the processing device 310 are integrated may be used.

[Configuration of projector 10]
17 is a block diagram showing the hardware configuration of a projector 10 in the second embodiment. In the second embodiment, the projector 10 includes an infrared light receiving device 106 in addition to the configuration shown in the first embodiment.

The infrared light receiving device 106 receives infrared light emitted by the infrared light emitting device 301 of the remote controller 30. The infrared light receiving device 106 is used for communication between the projector 10 and the remote controller 30. That is, the infrared light receiving device 106 communicates with the remote controller 30. The infrared light receiving device 106 is an example of a second communication device. In this embodiment, infrared light is used for communication between the projector 10 and the remote controller 30, but this is not limiting, and for example, high frequency (RF: Radio Frequency) may also be used.

FIG. 18 is a block diagram showing a program executed by the processing device 110 of the projector 10 in the second embodiment. In addition to the program shown in FIG. 4, the processing device 110 executes an infrared light receiving device driver HD5 that controls the infrared light receiving device 106 as one of the hardware drivers HD-P. The infrared light receiving device driver HD5 is an example of a second communication device driver.

When an operation button 320 of the remote controller 30 is pressed, the processing device 310 turns the infrared light emitting device 301 on and off at intervals corresponding to the pressed operation button 320. The infrared light emitted from the infrared light emitting device 301 is received by the infrared light receiving device 106 of the projector 10. The processing device 110 of the projector 10 identifies the operation button 320 that corresponds to the timing of turning on and off the infrared light, and performs processing corresponding to the identified operation button 320.

[Operation of the first keyboard system application SY1]
In the second embodiment, the first keyboard system application SY1 accepts character input using the remote controller 30, in addition to character input using the keyboard service application AP3. For example, assume that an image PC2 shown in Fig. 6 is displayed on the display surface F. When the remote controller 30 is operated with a character input cursor CI displayed in a search window WS of the image PC2, the first keyboard system application SY1 displays an image PC8 shown in Fig. 19 on the display surface F.

Image PC8 includes a search window WS and a software keyboard SK3. On the software keyboard SK3, keys for each of the 50 Hiragana characters, keys for voiced and semi-voiced sounds, etc. are displayed. On the software keyboard SK3 in FIG. 19, the color of the "a" key is different from the other keys, which indicates that the "a" key is in a selected state.

The user uses the cross key of the remote controller 30 to move the selected key, and when the key of the desired character is selected, presses the enter key. The remote controller 30 turns the infrared light emitting device 301 on and off at intervals corresponding to the character selected by the user. The projector 10 receives infrared light with the infrared light receiving device 106. The first keyboard system application SY1 accesses the infrared light receiving device driver HD5 with the system access function SA1 and acquires the infrared light receiving interval. The first keyboard system application SY1 also identifies the character specified by the user based on the infrared light receiving interval, and passes the character code indicating the identified character to the paid movie distribution service application AP2. The paid movie distribution service application AP2 displays an image of the character corresponding to the passed character code in the search window SW.

In this way, when the remote controller 30 is operated with the character input cursor CI displayed in the search window, the first keyboard system application SY1 operates in the same way as a general keyboard system application.

That is, in the second embodiment, the first keyboard system application SY1 detects that the remote controller 30 has been operated using the infrared light receiving device driver HD5. When the remote controller 30 is operated, the first keyboard system application SY1 displays the software keyboard SK3 using the projection device 101. The first keyboard system application SY1 accepts character input by accepting the designation of characters on the software keyboard SK3 based on the operation of the operation button 320 using the infrared light receiving device driver HD5.

In this way, in the second embodiment, the first keyboard system application SY1 accepts character input using the remote controller 30 in addition to character input using the keyboard service application AP3. This allows, for example, a user who does not have a smartphone 20 to operate the projector 10, improving convenience.

3. Modifications Each of the above-mentioned embodiments may be modified in various ways. Specific modifications that may be applied to each of the above-mentioned embodiments are illustrated below. Two or more embodiments selected from the following examples may be combined as appropriate to the extent that they are not mutually contradictory.

[Modification 1]
The input accepted by the keyboard service application AP3 of the smartphone 20 is not limited to character input. The keyboard service application AP3 may display an image of, for example, a piano keyboard, a whack-a-mole game, an audio channel mixer, or the like as an interface and accept touching the image as input. In this case, the keyboard service application AP3 may assign character codes to, for example, individual piano keys. The keyboard service application AP3 then transmits the character code assigned to the input content corresponding to the touch position to the projector 10.

[Modification 2]
Input to the smartphone 20 is not limited to input using the touch panel 201. For example, the projector 10 may be operated using various sensors mounted on the smartphone 20. Examples of sensors mounted on the smartphone 20 and parameters detected by the sensors are listed below in the form of "sensor/parameter." Gyro sensor/motion, acceleration sensor/posture, geomagnetic sensor/direction, air pressure sensor/height and vertical movement, proximity sensor/proximity, ambient light sensor/brightness, GNSS (Global Navigation Satellite System)/latitude, longitude, pressure detection display/contact.

[Modification 3]
By accepting handwritten input in the keyboard service application AP3, it is possible to input not only characters but also continuous shapes such as lines and figures using the touch panel 201 of the smartphone 20. Furthermore, by providing the smartphone 20 with a microphone and a system application SY-S for voice input, it becomes possible to input characters using voice.

[Modification 4]
In this embodiment, the display system 1 projects an image PC onto a display surface F using the projector 10, but the display system 1 is not limited to this and may be, for example, a system that displays an image on a direct-view display. In this case, the display serves as the display device. When the display serves as the display device, a light source and a display panel, etc. are provided instead of the projection device 101 shown in FIG. 2. The light source and the display panel are examples of optical devices.

[Modification 5]
In addition, in the present embodiment, the terminal device is a smartphone 20, but is not limited to this and may be, for example, a tablet terminal, a personal computer, or the like.

1, 2...Display system, 10...Projector, 20...Smartphone, 20A...First smartphone, 20B...Second smartphone, 30...Remote controller, 101...Projection device, 103...Operation button, 105...Near-field wireless communication device, 106...Infrared light receiving device, 107...Wireless LAN communication device, 108...Storage device, 110...Processing device, 201...Touch panel, 202...Near-field wireless communication device, 205...Storage device, 210...Processing device, 301...Infrared light emitting device, 302...Storage device, 310...Processing device, 320...Operation button, AP -P, AP-S...service application, AP2...paid movie distribution service application, AP3...keyboard service application, F...display surface, HD-P, HD-S...hardware driver, HD1, HD3...near-field wireless communication device driver, HD4...touch panel driver, HD5...infrared light receiving device driver, HW-P, HW-S...hardware, SY-P, SY-S...system application, SY1...first keyboard system application, SY3...second keyboard system application.

Claims (7)

a first communication device for communicating with a first terminal device;
An optical device;
a processing device that executes a first service application that provides a service, a keyboard system application that is called by the first service application and provides a character input function, a first communication device driver that controls the first communication device, and an operating system that manages the first service application, the keyboard system application, and the first communication device driver;
the keyboard system application obtains, using the first communication device driver, first character information indicating a character input on the first terminal device;
The first service application performs a display based on the first character information by using the optical device.
Display device. The processing device further executes a second service application different from the first service application;
The keyboard system application is called by the second service application to provide a function of character input.
The display device according to claim 1. The keyboard system application
Obtaining the first character information encrypted by the first terminal device;
providing the decrypted first character information obtained by decrypting the encrypted first character information to the first service application;
3. The display device according to claim 1 or 2. The first communication device further communicates with a second terminal device different from the first terminal device;
the keyboard system application obtains, using the first communication device driver, second character information indicating a character input on the second terminal device;
the first service application performs display based on the first character information and the second character information using the optical device;
The display device according to any one of claims 1 to 3. Further comprising a second communication device that communicates with a remote controller having an operation button;
the processing unit further executes a second communication device driver to control the second communication device;
The keyboard system application
Detecting that the remote controller is operated using the second communication device driver;
When it is detected that the remote controller is operated, the optical device is used to display a software keyboard;
accepting character input by accepting designation of a character on the software keyboard based on an operation on the operation button using the second communication device driver;
The display device according to any one of claims 1 to 4. A system including a display device and a terminal device,
The display device includes:
a display device side communication device that communicates with the terminal device;
An optical device;
a display device-side processing device that executes a service application that provides a service, a keyboard system application that is called by the service application and provides a character input function, a display device-side communication device driver that controls the display device-side communication device, and an operating system that manages the service application, the keyboard system application, and the display device-side communication device driver;
The terminal device
A terminal device side communication device that communicates with the display device;
a terminal device-side processing device that executes a terminal device-side communication device driver that controls the terminal device-side communication device and a character input application that accepts character input by a user;
the character input application transmits character information corresponding to the character to the display device using the terminal device side communication device driver;
the keyboard system application acquires the character information using the display device side communication device driver;
the service application uses the optical device to perform a display based on the character information;
system. The processing device includes:
obtaining character information indicating characters inputted into a terminal device using a communication device driver that controls a communication device that communicates with the terminal device;
providing the character information to a service application that provides a service;
program.

Images