JP5263597B2 - Image display device, image display method, and image display program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To switch a mode of the full-screen display of a content on a display panel in a plurality of display panels and a mode of the full-screen display of a content in the plurality of display panels by using the display panels as one screen. <P>SOLUTION: The image display device 100 is equipped with: first and second display panels 140 and 240; a receiving means 112 receiving a first command during a first mode; a deciding means 113 deciding, when the first command is received, whether or not the first image is displayed in a prescribed area or not; and a switching means 114 switched into a third mode when the first image is displayed in a prescribed area and switching into a second mode when the first image is not displayed in the prescribed area. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to an image display device, an image display method, and an image display program that change a display mode of an image displayed on a display panel according to a command from a user, and in particular, has a plurality of display panels, The present invention relates to an image display device, an image display method, and an image display program that can display an image on at least one of the plurality of display panels.

  An image display device capable of displaying content such as a still image and a moving image is known. Some of such image display apparatuses have a plurality of display panels, and one content can be displayed on the entire screen on the plurality of display panels.

For example, Japanese Patent Application Laid-Open No. 2006-314610 (Patent Document 1) discloses a video game device. According to Japanese Patent Laying-Open No. 2006-314610 (Patent Document 1), a video game apparatus has a display unit, and a touch operation unit is provided on a screen of the display unit. Two different game images are respectively displayed on the display unit, and the game image on the display unit is an enlarged image than the game image on the display unit. When the player performs a predetermined touch operation with the touch operation unit, the game image on the display unit and the game image on the display unit are interchanged, and the touch operation can be performed on the game image displayed on the display unit. When the player performs a touch operation on a specific object in the game image displayed on the display unit, a special game effect corresponding to the touch operation is generated.
JP 2006-314610 A

  However, in a conventional image display apparatus having a plurality of display panels, one content can be displayed on a full screen on one display panel, but one content is displayed on a full screen across a plurality of display panels. I couldn't. In other words, an image display device having a plurality of display panels has a mode in which a plurality of display panels are displayed as a single screen and content is displayed on the plurality of display panels, and one display panel among the plurality of display panels. As a single screen, it was not possible to easily switch between the mode for displaying the content on the single display panel in full screen.

  The present invention has been made to solve such a problem, and an object of the present invention is to have a plurality of display panels and at least part of at least one of the plurality of display panels according to a user's request. From the first mode in which the content is displayed, the second mode in which the content is displayed on the entire screen on one of the plurality of display panels and the plurality of display panels as one screen are displayed on the plurality of display panels. To provide an image display device, an image display method, and an image display program capable of easily switching to a third mode in which one content is displayed on a full screen.

According to one aspect of the present invention, an image display device is provided. The image display device displays the first image on the first and second display panels and a part of the first display panel in the first mode, and corresponds to the first image in the second mode. Display control means for displaying two images on the first display panel on a full screen and displaying a third image corresponding to the first image on the first and second display panels in the third mode; A receiving unit that receives a first command during the first mode, a determination unit that determines whether or not a first image is displayed in a predetermined area when the first command is received, and a determination unit; When it is determined that the first image is displayed in the predetermined area, the mode is switched to the third mode. When the determination unit determines that the first image is not displayed in the predetermined area, the mode is switched to the second mode. Switching means. The determining means determines whether or not the display control means is displaying the first image on the first display panel and the second display panel, and the display control means is the first display panel and the second display. When the first image is displayed on the panel, it is determined that the first image is displayed in the predetermined area.

  Preferably, the accepting means accepts the second command during the third mode. The switching means switches to the first mode when receiving the second command.

  Preferably, the accepting unit accepts the second command during the third mode, and the position on any of the first and second display panels via any of the first and second display panels. The first designation command for designating is accepted. The display control means displays the first image at the designated position on either the first or second display panel based on the first designation command.

  Preferably, the accepting unit accepts a third command during the second mode. The switching means switches to the first mode when receiving the third command.

  Preferably, the accepting unit accepts a third command during the second mode and accepts a second designation command for designating a position on the first display panel via the first display panel. The display control means displays the first image at the designated position on the first display panel based on the second designation command.

  Preferably, each of the first and second display panels includes a plurality of photo sensor circuits that generate an input signal according to incident light and a plurality of pixel circuits that emit light according to an output signal. The accepting means accepts an instruction based on input signals from the first and second display panels. The display control means causes the first and second display panels to display images by outputting output signals to the first and second display panels.

When another situation of this invention is followed, the image display method in an image display apparatus provided with a 1st and 2nd display panel and a calculation process part is provided. In the image display method, the arithmetic processing unit displays the first image on a part of the first display panel in the first mode, and the arithmetic processing unit corresponds to the first image in the second mode. The second image to be displayed on the first display panel on the full screen, and the arithmetic processing unit displays the third image corresponding to the first image on the first and second display panels in the third mode. A step of displaying the entire screen; a step in which the arithmetic processing unit accepts the first command during the first mode; and a step in which the first processing unit accepts the first command when the arithmetic processing unit accepts the first command. A step of determining whether the image is displayed; a step of switching to the third mode when the arithmetic processing unit determines that the first image is displayed in the predetermined area; and the arithmetic processing unit Image is displayed in the specified area If it is determined that there is no and a step of switching to the second mode. In the determining step, it is determined whether or not the first image is displayed on the first display panel and the second display panel, and the first image is displayed on the first display panel and the second display panel. Is displayed, it is determined that the first image is displayed in the predetermined area.

If another situation of this invention is followed, the image display program for displaying an image on the image display apparatus provided with the 1st and 2nd display panel and the arithmetic processing part will be provided. The image display program causes the arithmetic processing unit to display a first image on a part of the first display panel in the first mode, and a second image corresponding to the first image in the second mode. Displaying the first image on the first display panel, displaying the third image corresponding to the first image in the third mode on the first and second display panels, A step of receiving a first command during the mode, a step of determining whether or not the first image is displayed in a predetermined area when the first command is received, and the first image in the predetermined area A step of switching to the third mode when it is determined that it is displayed and a step of switching to the second mode when it is determined that the first image is not displayed in the predetermined area are executed. In the determining step, it is determined whether or not the first image is displayed on the first display panel and the second display panel, and the first image is displayed on the first display panel and the second display panel. Is displayed, it is determined that the first image is displayed in the predetermined area.

  As described above, according to the present invention, a plurality of display panels are provided, and a plurality of display panels are displayed from the first mode in which content is displayed on a part of at least one of the plurality of display panels according to a user's request. A second mode in which content is displayed in full screen on one of the display panels, and a third mode in which a plurality of display panels are used as one screen, and one content is displayed on the plurality of display panels in full screen. An image display device, an image display method, and an image display program that can be easily switched between modes are provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[Embodiment 1]
<Overall Configuration of Electronic Device 100>
First, an overall configuration of electronic device 100 that is an example of an image display device according to the present embodiment will be described. In the present embodiment, a window showing content such as a still image, a moving image, and text will be described as an example of an image displayed on the display panel. However, an image is a concept including a still image itself, a moving image itself, and text itself. The full screen display not only displays the photographic image itself on the entire surface of the display panel but also displays a window (so-called viewer window) for displaying the photographic image on the entire surface of the display panel (so-called maximum). It is a concept that also includes a computerized display. On the contrary, the maximization command is a concept including not only a command for displaying a window for displaying a photographic image on the entire surface of the display panel but also a command for displaying the photographic image itself on the entire surface of the display panel.

  FIG. 1 is a schematic diagram showing an external appearance of an electronic device 100 according to the present embodiment. More specifically, FIG. 1A is an image diagram showing the electronic device 100 in a state where a window X (first image) for displaying a photographic image is displayed on a part of the first display panel 140. FIG. 1B is an image diagram showing the electronic device 100 in a state where a photographic image (second image) is displayed on the full screen on the first display panel 140, and FIG. 2 is an image diagram showing the electronic device 100 in a state in which a window X for displaying a photographic image is displayed across a part of the display panel 140 and a part of the second display panel 240. FIG. ) And FIG. (E) are image diagrams showing the electronic device 100 showing a state in which a photographic image (third image) is displayed on the full screen on the first display panel 140 and the second display panel 240. 1 (F) is the second display panel Is a schematic diagram showing an electronic device 100 in a state where the window X is displayed for displaying a photographic image on a part of the 40.

  Referring to FIGS. 1A to 1F, electronic device 100 includes a first housing 100A and a second housing 100B. The first casing 100A and the second casing 100B are foldably connected by a hinge 100C. The first housing 100A includes a first photosensor built-in liquid crystal panel 140 (hereinafter also referred to as a first display panel 140). The second housing 100B includes a second photosensor built-in liquid crystal panel 240 (hereinafter also referred to as a second display panel 240). Electronic device 100 according to the present embodiment displays an image on at least one of first and second display panels 140 and 240.

  Electronic device 100 according to the present embodiment includes two liquid crystal panels with a built-in optical sensor, but may include two touch panels having a so-called tablet function. The electronic device 100 is configured as a portable device having a display function such as a PDA (Personal Digital Assistant), a notebook personal computer, a portable telephone, and an electronic dictionary.

<Outline of Operation of Electronic Device 100>
Next, with reference to FIG. 1A to FIG. 1F, an outline of operation of electronic device 100 according to the present embodiment will be described.

  Referring to FIG. 1A, electronic device 100 causes window X (first image) including a photographic image to be displayed on a part of first display panel 140. In this state, when the user inputs a command (maximization command) for displaying the window X in full screen on the electronic device 100, the electronic device 100 displays the first display panel as shown in FIG. A photographic image (second image) is displayed on the full screen 140.

  That is, when electronic device 100 according to the present embodiment receives a maximization command in a state where the display area of window X is within first display panel 140, first display panel 140 is set as one screen. A photographic image corresponding to the window X is displayed in full screen. Here, the maximization command is, for example, a contact operation on a predetermined portion of the window X displayed as the maximization button by the user, or a predetermined function key (operation key 177 described later) by the user. It may be a press.

  However, when the electronic device 100 receives a so-called maximization command in a state where the display area of the window X is within the first display panel 140, the electronic device 100 sets the window X to the so-called maximum using the first display panel 140 as one screen. It may be displayed in a digitized form. When the electronic device 100 accepts a so-called full-screen command in a state where the display area of the window X is within the first display panel 140, the electronic device 100 uses the first display panel 140 as one screen to display a photographic image. It may be displayed on the screen.

  Referring to FIG. 1C, the user drags window X to the boundary between first display panel 140 and second display panel 240 using stylus pen 99. That is, the electronic device 100 displays a window X (first image) including a photographic image across the lower portion of the first display panel 140 and the upper portion of the second display panel 240. That is, electronic device 100 moves window X to a predetermined area in accordance with a movement command from the user. In this state, when the user inputs a command (maximization command) for displaying the window X in full screen on the electronic device 100, the electronic device 100 displays the first display panel as shown in FIG. 140 and the second display panel 240 display a photographic image (third image) on a full screen.

  That is, electronic device 100 according to the present embodiment has a state in which the display area of window X extends between first display panel 140 and second display panel 240 (window X is positioned in a predetermined area set in advance). When the maximization command is received, the first display panel 140 and the second display panel 240 are combined and the photographic image displayed in the window X as a single screen is displayed on the full screen.

  More specifically, for example, when electronic device 100 receives a “maximization command” in a state where the display area of window X extends over first display panel 140 and second display panel 240, The “window X” may be so-called “maximized display” with the display panel 140 and the second display panel 240 as one screen. When the electronic device 100 accepts the “full screen display command” in a state where the display area of the window X extends over the first display panel 140 and the second display panel 240, the electronic device 100 The “photo image itself” may be so-called “full screen display” with the second display panel 240 as one screen.

  Referring to FIG. 1 (E), electronic device 100 displays a photographic image in full screen as a single screen by combining first display panel 140 and second display panel 240. In this state, the user inputs a position designation for designating the position of the first display panel 140 or the second display panel 240 to the electronic device 100 using the stylus pen 99 or the like, and opens the window X on the electronic device 100. When a command for normal display (normalization command) is input, the electronic device 100 displays a window X at the designated position as shown in FIG.

  Electronic device 100 accepts a press of a predetermined function key (an operation key 177 described later) by the user as a normalization command. Note that the size of the window X at this time is the size (normal size) of the window X before full screen display.

  However, when the electronic device 100 receives a position designation during full screen display, the electronic device 100 may display a normal size window X at the designated position.

  Alternatively, when a plurality of contact operations are input within a predetermined range within a predetermined time, electronic device 100 may accept the plurality of contact operations as a normalization command. In this case, the electronic device 100 may accept the second contact position as the designated position.

  As described above, electronic device 100 according to the present embodiment has a plurality of display panels 140 and 240. When electronic device 100 according to the present embodiment receives a window X (content) maximization command, electronic device 100 is displayed on first display panel 140 as window X according to the display position of window X. The displayed content is displayed on the full screen, or the content is displayed on the full screen on the first display panel 140 and the second display panel 240.

  That is, when electronic device 100 according to the present embodiment receives a window X maximization command, electronic device 100 assigns the entire surface of first display panel 140 to content according to the display position of window X, or first After determining whether the entire display panel 140 and the entire second display panel 240 are assigned to the content, the content is displayed on the full screen on at least one of the first display panel 140 and the second display panel 240. To do.

  As a result, electronic apparatus 100 according to the present embodiment has a plurality of display panels 140 and 240, and may be a part of at least one display panel 140 of the plurality of display panels 140 and 240 according to a user's request. From the first mode in which content is displayed, the second mode in which content is displayed in full screen on one display panel 140 of the plurality of display panels 140 and 240, and the plurality of display panels 140 and 240 in one It is possible to easily switch to the third mode in which one content is displayed as a full screen on the plurality of display panels 140 and 240 as a screen.

  Hereinafter, a configuration for realizing such a function will be described in detail. In this embodiment, the case where the electronic device 100 includes the optical sensor built-in liquid crystal panels 140 and 240 will be described. However, the electronic device 100 as an image display device may be a normal touch panel display panel. Good.

<Hardware Configuration of Electronic Device 100>
Next, an aspect of a specific configuration of the electronic device 100 will be described. FIG. 2 is a block diagram illustrating a hardware configuration of electronic device 100. Referring to FIG. 2, electronic device 100 includes a main device 101, a first display device 102, and a second display device 103 as main components.

  The main unit 101 includes a CPU (Central Processing Unit) 110, a RAM (Random Access Memory) 171, a ROM (Read-Only Memory) 172, a memory card reader / writer 173, a communication unit 174, a microphone 175, and a speaker. 176 and operation keys 177 are included. Each component is connected to each other by a data bus DB1. A memory card 1731 is attached to the memory card reader / writer 173.

  CPU 110 executes a program. The operation key 177 receives an instruction input from the user of the electronic device 100. The RAM 171 stores data generated by the execution of the program by the CPU 110 or data input via the operation keys 177 in a volatile manner. The ROM 172 stores data in a nonvolatile manner. The ROM 172 is a ROM capable of writing and erasing data such as an EPROM (Erasable Programmable Read-Only Memory) and a flash memory. The communication unit 174 performs wireless communication with other electronic devices (not shown). Although not illustrated in FIG. 2, the electronic device 100 may include an interface (IF) for connecting to another electronic device by wire.

  The display device 102 includes a driver 130, a first optical sensor built-in liquid crystal panel 140, an internal IF 178, a backlight 179, and an image processing engine 180.

  The driver 130 is a drive circuit for driving the liquid crystal panel 140 and the backlight 179. Various drive circuits included in the driver 130 will be described later.

  The liquid crystal panel 140 is a device having a liquid crystal display function and a photosensor function. That is, the liquid crystal panel 140 can perform image display using liquid crystal and sensing using an optical sensor. Details of the liquid crystal panel 140 will be described later.

  An internal IF (Interface) 178 mediates exchange of data between the main device 101 and the display device 102.

  The backlight 179 is a light source disposed on the back surface of the liquid crystal panel 140. The backlight 179 irradiates the back surface with uniform light.

  The image processing engine 180 controls the operation of the liquid crystal panel 140 via the driver 130. Here, the control is performed based on various data sent from the main apparatus 101 via the internal IF 178. Note that the various data includes commands to be described later. Further, the image processing engine 180 processes data output from the liquid crystal panel 140 and sends the processed data to the main apparatus 101 via the internal IF 178. Further, the image processing engine 180 includes a driver control unit 181, a timer 182, and a signal processing unit 183.

  The driver control unit 181 controls the operation of the driver 130 by sending a control signal to the driver 130. In addition, the driver control unit 181 analyzes a command transmitted from the main device 101. Then, the driver control unit 181 sends a control signal based on the analysis result to the driver 130. Details of the operation of the driver 130 will be described later.

The timer 182 generates time information and sends the time information to the signal processing unit 183.
The signal processing unit 183 receives data output from the optical sensor. Here, since the data output from the optical sensor is analog data, the signal processing unit 183 first converts the analog data into digital data. Further, the signal processing unit 183 performs data processing on the digital data in accordance with the content of the command sent from the main device 101. Then, the signal processing unit 183 sends data (hereinafter referred to as response data) including data after the above data processing and time information acquired from the timer 182 to the main unit 101. The signal processing unit 183 includes a RAM (not shown) that can store a plurality of scan data, which will be described later, continuously.

  The command includes a sensing command for instructing sensing by the optical sensor. Details of the sensing command and the response data will be described later (FIGS. 7 to 9).

  Note that the timer 182 is not necessarily provided in the image processing engine 180. For example, the timer 182 may be provided outside the image processing engine 180 in the display device 102. Alternatively, the timer 182 may be provided in the main body device 101. Further, the microphone 175 and the speaker 176 are not always provided in the electronic device 100, and may be configured so as not to include either or both of the microphone 175 and the speaker 176 depending on the embodiment of the electronic device 100.

  Here, the display device 102 includes a system liquid crystal. The system liquid crystal is a device obtained by integrally forming peripheral devices of the liquid crystal panel 140 on the glass substrate of the liquid crystal panel 140. In the present embodiment, the driver 130 (excluding the circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are integrally formed on the glass substrate of the liquid crystal panel 140. Note that the display device 102 is not necessarily configured using the system liquid crystal, and the driver 130 (excluding a circuit that drives the backlight 179), the internal IF 178, and the image processing engine 180 are included in the glass substrate. Other substrates may be configured.

  The display device 103 includes a driver 230, a second optical sensor built-in liquid crystal panel 240, an internal IF 278, a backlight 279, and an image processing engine 280. The image processing engine 280 includes a driver control unit 281, a timer 282, and a signal processing unit 283.

  The display device 103 has the same configuration as the display device 102. That is, the driver 230, the liquid crystal panel 240, the internal IF 278, the backlight 279, and the image processing engine 280 have the same configuration as the driver 130, the liquid crystal panel 140, the internal IF 178, the backlight 179, and the image processing engine 180 in the display device 102. Have each. The driver control unit 281, the timer 282, and the signal processing unit 283 have the same configurations as the driver control unit 181, the timer 182, and the signal processing unit 183 in the display device 102, respectively. Therefore, description of each functional block included in display device 103 will not be repeated.

  By the way, the processing in the electronic device 100 is realized by each hardware and software executed by the CPU 110. Such software may be stored in the ROM 172 in advance. The software may be stored in a memory card 1731 or other storage medium and distributed as a program product. Alternatively, the software may be provided as a program product that can be downloaded by an information provider connected to the so-called Internet. Such software is read from the storage medium by the memory card reader / writer 173 or other reading device, or downloaded via the communication unit 174 or communication IF (not shown), and then temporarily stored in the ROM 172. The The software is read from the ROM 172 by the CPU 110 and stored in the RAM 171 in the form of an executable program. CPU 110 executes the program.

  Each component constituting the main device 101 of the electronic device 100 shown in FIG. 2 is a general one. Therefore, it can be said that the essential part of the present invention is the software stored in the RAM 171, the ROM 172, the memory card 1731 and other storage media, or the software downloadable via the network. Since the hardware operation of main device 101 of electronic device 100 is well known, detailed description will not be repeated.

  The storage medium is not limited to a memory card, but is a CD-ROM, FD (Flexible Disk), hard disk, magnetic tape, cassette tape, optical disk (MO (Magnetic Optical Disc) / MD (Mini Disc) / DVD (Digital Versatile). Disc)), IC (Integrated Circuit) cards (excluding memory cards), optical cards, mask ROM, EPROM, EEPROM (Electronically Erasable Programmable Read-Only Memory), flash ROM, and other semiconductor memories, etc. It may be a medium to be used.

  The program here includes not only a program directly executable by the CPU but also a program in a source program format, a compressed program, an encrypted program, and the like.

<Configuration and Drive of Photosensor Built-in Liquid Crystal Panel 140>
Next, the configuration of the liquid crystal panel 140 and the configuration of peripheral circuits of the liquid crystal panel 140 will be described. FIG. 3 is a diagram showing a configuration of the liquid crystal panel 140 and peripheral circuits of the liquid crystal panel 140.

  Referring to FIG. 3, the liquid crystal panel 140 includes a pixel circuit 141, an optical sensor circuit 144, a scanning signal line Gi, a data signal line SRj, a data signal line SGj, a data signal line SBj, and a sensor signal line. SSj, sensor signal line SDj, read signal line RWi, and reset signal line RSi are included. Note that i is a natural number satisfying 1 ≦ i ≦ m, and j is a natural number satisfying 1 ≦ j ≦ n.

  2 includes a scanning signal line driving circuit 131, a data signal line driving circuit 132, an optical sensor driving circuit 133, a switch 134, as peripheral circuits of the liquid crystal panel 140. And an amplifier 135.

  The scanning signal line drive circuit 131 receives the control signal TC1 from the driver control unit 181 shown in FIG. The scanning signal line drive circuit 131 applies a predetermined voltage in order from the scanning signal line G1 to each scanning signal line (G1 to Gm) based on the control signal TC1. More specifically, the scanning signal line driving circuit 131 sequentially selects one scanning signal line from the scanning signal lines (G1 to Gm) per unit time, and a TFT (to be described later) with respect to the selected scanning signal line. A voltage capable of turning on the gate of the thin film transistor 142 is applied (hereinafter referred to as a high level voltage). Note that a low level voltage is applied to a scanning signal line that is not selected without applying a high level voltage.

  The data signal line driving circuit 132 receives image data (DR, DG, DB) from the driver control unit 181 shown in FIG. The data signal line driving circuit 132 applies a voltage corresponding to one row of image data to the 3n data signal lines (SR1 to SRn, SG1 to SGn, SB1 to SBn) for each unit time. Apply sequentially.

  Note that although a driving method called a so-called line-sequential method has been described here, the driving method is not limited to this.

  The pixel circuit 141 is a circuit for setting the luminance (transmittance) of one pixel. Further, m × n pixel circuits 141 are arranged in a matrix. More specifically, m pixel circuits 141 are arranged in the vertical direction in FIG. 3 and n pixel circuits in the horizontal direction.

  The pixel circuit 141 includes an R subpixel circuit 141r, a G subpixel circuit 141g, and a B subpixel circuit 141b. Each of these three circuits (141r, 141g, 141b) includes a TFT 142, a pair of electrode pairs 143 including a pixel electrode and a counter electrode, and a capacitor (not shown).

  In addition, CMOS (Complementary Metal Oxide Semiconductor) capable of forming n-type transistors and p-type transistors can be realized, and the movement speed of carriers (electrons or holes) is several hundreds compared to amorphous silicon thin film transistors (a-Si TFTs). For example, a polycrystalline silicon thin film transistor (p-Si TFT) is used as the TFT 142 in the display device 102 because of its double speed. Note that the TFT 142 will be described as an n-channel field effect transistor. However, the TFT 142 may be a p-type channel field effect transistor.

  The source of the TFT 142 in the R subpixel circuit 141r is connected to the data signal line SRj. The gate of the TFT 142 is connected to the scanning signal line Gi. Further, the drain of the TFT 142 is connected to the pixel electrode of the electrode pair 143. A liquid crystal is disposed between the pixel electrode and the counter electrode. The G sub-pixel circuit 141g and the B sub-pixel circuit 141b have the same configuration as the R sub-pixel circuit 141r except that the data signal line to which the source of each TFT 142 is connected is different. Therefore, description of these two circuits (141g, 141b) will not be repeated.

  Here, setting of luminance in the pixel circuit 141 will be described. First, the high level voltage is applied to the scanning signal line Gi. By the application of the high level voltage, the gate of the TFT 142 is turned on. In this manner, with the gate of the TFT 142 turned on, a specified voltage (voltage corresponding to image data for one pixel) is applied to each data signal line (SRj, SGj, SBj). Thereby, a voltage based on the designated voltage is applied to the pixel electrode. As a result, a potential difference is generated between the pixel electrode and the counter electrode. Based on this potential difference, the liquid crystal responds and the luminance of the pixel is set to a predetermined luminance. Note that the potential difference is held by the capacitor (auxiliary capacitor) (not shown) until the scanning signal line Gi is selected in the next frame period.

  The optical sensor drive circuit 133 receives the control signal TC2 from the driver control unit 181 shown in FIG.

  Then, the optical sensor drive circuit 133 sequentially selects one signal line from the reset signal lines (RS1 to RSm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, the voltage VDDR that is higher than usual is applied. Note that a voltage VSSR lower than the voltage applied to the selected reset signal line is kept applied to the unselected reset signal line. For example, the voltage VDDR may be set to 0V and the voltage VSSR may be set to −5V.

  Further, the optical sensor driving circuit 133 sequentially selects one signal line from the readout signal lines (RW1 to RWm) for each unit time based on the control signal TC2, and determines in advance for the selected signal line. At a given timing, a voltage VDD higher than usual is applied. Note that the voltage VSSR is applied to the read signal line that is not selected. For example, the value of VDD may be set to 8V.

  The timing for applying the voltage VDDR and the timing for applying the voltage VDD will be described later.

  The optical sensor circuit 144 includes a photodiode 145, a capacitor 146, and a TFT 147. In the following description, it is assumed that the TFT 147 is an n-type channel field effect transistor. However, the TFT 147 may be a p-type channel field effect transistor.

  The anode of the photodiode 145 is connected to the reset signal line RSi. On the other hand, the cathode of the photodiode 145 is connected to one electrode of the capacitor 146. The other electrode of the capacitor 146 is connected to the read signal line RWi. Hereinafter, a connection point between the photodiode 145 and the capacitor 146 is referred to as a node N.

  The gate of the TFT 147 is connected to the node N. The drain of the TFT 147 is connected to the sensor signal line SDj. Further, the source of the TFT 147 is connected to the sensor signal line SSj. Details of sensing using the optical sensor circuit 144 will be described later.

  The switch 134 is a switch provided to switch whether or not to apply a predetermined voltage to the sensor signal lines (SD1 to SDn). The switching operation of the switch 134 is performed by the optical sensor driving circuit 133. Note that the voltage applied to the sensor signal lines (SD1 to SDn) when the switch 134 is turned on will be described later.

  The amplifier 135 amplifies the voltage output from each sensor signal line (SS1 to SSn). The amplified voltage is sent to the signal processing unit 183 shown in FIG.

  The image processing engine 180 controls the timing for displaying an image on the liquid crystal panel 140 using the pixel circuit 141 and the timing for sensing using the optical sensor circuit 144.

  FIG. 4 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179. Referring to FIG. 4, liquid crystal panel 140 includes an active matrix substrate 151A, a counter substrate 151B, and a liquid crystal layer 152. The counter substrate 151B is disposed to face the active matrix substrate 151A. The liquid crystal layer 152 is sandwiched between the active matrix substrate 151A and the counter substrate 151B. The backlight 179 is disposed on the opposite side of the liquid crystal layer 152 with respect to the active matrix substrate 151A.

  The active matrix substrate 151 </ b> A includes a polarizing filter 161, a glass substrate 162, a pixel electrode 143 a that constitutes the electrode pair 143, a photodiode 145, a data signal line 157, and an alignment film 164. Further, although not shown in FIG. 4, the active matrix substrate 151A includes the capacitor 146, the TFT 147, the TFT 142, and the scanning signal line Gi shown in FIG.

  In the active matrix substrate 151A, the polarizing filter 161, the glass substrate 162, the pixel electrode 143a, and the alignment film 164 are arranged in this order from the backlight 179 side. The photodiode 145 and the data signal line 157 are formed on the liquid crystal layer 152 side of the glass substrate 162.

  The counter substrate 151B includes a polarizing filter 161, a glass substrate 162, a light shielding film 163, color filters (153r, 153g, 153b), a counter electrode 143b constituting the electrode pair 143, and an alignment film 164.

  In the counter substrate 151B, the alignment film 164, the counter electrode 143b, the color filters (153r, 153g, 153b), the glass substrate 162, and the polarizing filter 161 are arranged in this order from the liquid crystal layer 152 side. The light shielding film 163 is formed in the same layer as the color filters (153r, 153g, 153b).

  The color filter 153r is a filter that transmits light having a red wavelength. The color filter 153g is a filter that transmits light having a green wavelength. The color filter 153b is a filter that transmits light having a blue wavelength. Here, the photodiode 145 is arranged at a position facing the color filter 153b.

  The liquid crystal panel 140 displays an image by blocking external light or light emitted from a light source such as a backlight 179 or transmitting the light. Specifically, the liquid crystal panel 140 changes the direction of liquid crystal molecules in the liquid crystal layer 152 by applying a voltage between the pixel electrode 143a and the counter electrode 143b, thereby blocking or transmitting the light. However, since the light cannot be completely blocked by the liquid crystal alone, a polarizing filter 161 that transmits only light having a specific polarization direction is provided.

  Note that the position of the photodiode 145 is not limited to the above position, and may be provided at a position facing the color filter 153r or a position facing the color filter 153g.

  Here, the operation of the optical sensor circuit 144 will be described. FIG. 5 is a diagram showing a timing chart when the optical sensor circuit 144 is operated. In FIG. 5, a voltage VINT indicates a potential at the node N in the photosensor circuit 144. A voltage VPIX is an output voltage from the sensor signal line SSj shown in FIG. 3 and is a voltage before being amplified by the amplifier 135.

  The following description will be divided into a reset period for resetting the optical sensor circuit 144, a sensing period for sensing light using the optical sensor circuit 144, and a readout period for reading the sensed result.

  First, the reset period will be described. In the reset period, the voltage applied to the reset signal line RSi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDDR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR). As described above, by applying the high-level voltage to the reset signal line RSi, a current starts to flow in the forward direction (from the anode side to the cathode side) of the photodiode 145. As a result, the voltage VINT which is the potential of the node N has a value represented by the following expression (1). In Equation (1), the forward voltage drop amount in the photodiode 145 is Vf.

VINT = VSSR + | VDDR−VSSR | −Vf (1)
Therefore, the potential of the node N is a value smaller by Vf than the voltage VDDR as shown in FIG.

  Here, since the voltage VINT is not more than the threshold value for turning on the gate of the TFT 147, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change. Further, a difference corresponding to the voltage VINT occurs between the electrodes of the capacitor 146. For this reason, the capacitor 146 accumulates charges corresponding to the difference.

  Next, the sensing period will be described. In the sensing period following the reset period, the voltage applied to the reset signal line RSi instantaneously switches from the high level (voltage VDDR) to the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is kept at the low level (voltage VSSR).

  Thus, by changing the voltage applied to the reset signal line RSi to the low level, the potential of the node N becomes higher than the voltage of the reset signal line RSi and the voltage of the read signal line RWi. For this reason, in the photodiode 145, the voltage on the cathode side becomes higher than the voltage on the anode side. That is, the photodiode 145 is in a reverse bias state. In such a reverse bias state, when the photodiode 145 receives light from the light source, current starts to flow from the cathode side to the anode side of the photodiode 145. As a result, as shown in FIG. 5, the potential of the node N (that is, the voltage VINT) becomes lower with the passage of time.

  Since the voltage VINT continues to decrease in this way, the gate of the TFT 147 does not turn on. Therefore, there is no output from the sensor signal line SSj. For this reason, the voltage VPIX does not change.

  Next, the reading period will be described. In the readout period following the sensing period, the voltage applied to the reset signal line RSi is kept at the low level (voltage VSSR). On the other hand, the voltage applied to the read signal line RWi is instantaneously switched from the low level (voltage VSSR) to the high level (voltage VDD). Here, the voltage VDD is higher than the voltage VDDR.

  Thus, by applying a high level voltage instantaneously to the read signal line RWi, the potential of the node N is raised through the capacitor 146 as shown in FIG. Note that the increase width of the potential of the node N is a value corresponding to the voltage applied to the read signal line RWi. Here, since the potential of the node N (that is, the voltage VINT) is raised to a threshold value that turns on the gate of the TFT 147, the gate of the TFT 147 is turned on.

  At this time, if a constant voltage is applied in advance to the sensor signal line SDj (see FIG. 3) connected to the drain side of the TFT 147, the sensor signal line SSj connected to the source side of the TFT 147 will cause the VPIX in FIG. As shown in the graph, a voltage corresponding to the potential of the node N is output.

  Here, when the amount of light received by the photodiode 145 (hereinafter referred to as the amount of received light) is small, the slope of the straight line shown in the VINT graph of FIG. 5 becomes gentle. As a result, the voltage VPIX is higher than when the amount of received light is large. As described above, the optical sensor circuit 144 changes the value of the voltage output to the sensor signal line SSj in accordance with the amount of light received by the photodiode 145.

  By the way, in the above, the operation | movement was demonstrated paying attention to one optical sensor circuit 144 among the m * n optical sensor circuits which exist. Below, operation | movement of each photosensor circuit in the liquid crystal panel 140 is demonstrated.

  First, the optical sensor drive circuit 133 applies a predetermined voltage to all n sensor signal lines (SD1 to SDn). Next, the photosensor drive circuit 133 applies a voltage VDDR that is higher than normal to the reset signal line RS1. The other reset signal lines (RS2 to RSm) and read signal lines (RW1 to RWm) are kept in a state where a low level voltage is applied. As a result, the n photosensor circuits in the first row in FIG. 3 enter the reset period described above. Thereafter, the n photosensor circuits in the first row enter a sensing period. Further, thereafter, the n photosensor circuits in the first row enter a reading period.

  Note that the timing at which a predetermined voltage is applied to all n sensor signal lines (SD1 to SDn) is not limited to the above timing, and may be any timing that is applied at least before the readout period. .

  When the readout period of the n photosensor circuits in the first row is completed, the photosensor drive circuit 133 applies a voltage VDDR that is higher than usual to the reset signal line RS2. That is, the reset period of the n photosensor circuits in the second row starts. When the reset period ends, the n photosensor circuits in the second row enter a sensing period, and thereafter enter a reading period.

  Thereafter, the processing described above is sequentially performed on the n photosensor circuits in the third row, the n photosensor circuits in the fourth row,..., The n photosensor circuits in the m row. As a result, the sensing result of the first row, the sensing result of the second row,..., The sensing result of the m-th row are output in this order from the sensor signal lines (SS1 to SSn).

  In the display device 102, sensing is performed for each row as described above, and a sensing result is output from the liquid crystal panel 140 for each row. For this reason, hereinafter, the data after the signal processing unit 183 performs the above-described data processing on the data regarding the voltage for m rows from the first row to the m-th row output from the liquid crystal panel 140, This is called “scan data”. That is, scan data refers to image data obtained by scanning a scan target (for example, a user's finger). An image displayed based on the scan data is referred to as a “scanned image”. Further, in the following, sensing is referred to as “scan”.

  Moreover, in the above, although the structure which scans using all the m * n photosensor circuits was mentioned as an example, it is not limited to this. Scanning may be performed on a partial region of the surface of the liquid crystal panel 140 using a photosensor circuit selected in advance.

  In the following, it is assumed that the electronic device 100 can take either of the two configurations. Further, switching between the components is assumed to be performed by a command sent from the main device 101 based on an input via the operation key 177 or the like. Note that when scanning is performed on a partial area on the surface of the liquid crystal panel 140, the image processing engine 180 sets a scan target area. The setting of the area may be configured to be specified by the user via the operation key 177.

  As described above, when scanning is performed on a partial region of the surface of the liquid crystal panel 140, there are the following modes of use for displaying an image. The first is a mode in which an image is displayed in a surface area other than the partial area (hereinafter referred to as a scan area). The second is a mode in which no image is displayed in the surface area other than the scan area. Which mode is used is based on a command sent from the main apparatus 101 to the image processing engine 180.

  FIG. 6 is a cross-sectional view of the liquid crystal panel 140 and the backlight 179, showing a configuration in which the photodiode 145 receives light from the backlight 179 during scanning.

  Referring to FIG. 6, when the user's finger 900 is in contact with the surface of the liquid crystal panel 140, a part of the light emitted from the backlight 179 is part of the user's finger 900 (abbreviated in the contacted area). Reflected on the plane). The photodiode 145 receives the reflected light.

  Even in a region where the finger 900 is not in contact, part of the light emitted from the backlight 179 is reflected by the user's finger 900. Even in this case, the photodiode 145 receives the reflected light. However, since the finger 900 is not in contact with the surface of the liquid crystal panel 140 in the region, the amount of light received by the photodiode 145 is smaller than the region in which the finger 900 is in contact. Note that the photodiode 145 cannot receive most of the light emitted from the backlight 179 that does not reach the user's finger 900.

  Here, by turning on the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the amount of light reflected by the user's finger 900 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 with respect to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter, also referred to as a reflected image) obtained by reflecting light with the finger 900.

Note that examples of the scan object other than the finger 900 include a stylus pen.
By the way, in the present embodiment, a liquid crystal panel is described as an example of the display device of electronic device 100, but another panel such as an organic EL (Electro-Luminescence) panel is used instead of the liquid crystal panel. Also good.

<About data>
Next, the sensing command will be described. In the display device 102, the image processing engine 180 analyzes the content of the sensing command and sends back data (that is, response data) according to the analysis result to the main body device 101.

  FIG. 7 is a diagram showing a schematic configuration of a sensing command. Referring to FIG. 7, the sensing command includes a header data area DA01, a timing data area DA02, a data type data area DA03, a reading method data area DA04, and image gradation data. An area DA05, a data area DA06 indicating resolution, and a spare data area DA07 are included.

  FIG. 8 is a diagram showing data values in each area of the sensing command and the meaning contents indicated by the values.

  Referring to FIG. 8, the sensing command in which “00” is set in the data area indicating the timing requests image processing engine 180 to transmit scan data at that time. That is, the sensing command requests transmission of scan data obtained by scanning using the optical sensor circuit 144 after the image processing engine 180 receives the sensing command. A sensing command in which “01” is set in the data area indicating the timing requests transmission of scan data when the scan result has changed. Furthermore, a sensing command in which “10” is set in the data area indicating the timing requests transmission of scan data at regular intervals.

  A sensing command in which “001” is set in the data area indicating the data type requests transmission of the coordinate value of the center coordinate in the partial image. In addition, a sensing command in which “010” is set in the data area indicating the data type requests transmission of only a partial image whose scan result has changed. Note that the change in the scan result indicates that the previous scan result is different from the current scan result. Further, a sensing command in which “100” is set in the data area indicating the data type requests transmission of the entire image.

  Here, the “entire image” is an image generated by the image processing engine 180 based on the voltage output from each photosensor circuit when scanning using m × n photosensor circuits. . The “partial image” is a part of the entire image. The reason why the partial image is requested to transmit only the partial image whose scan result has changed will be described later.

  The coordinate value and the partial image or the entire image may be requested at the same time. Further, in the case of a configuration in which scanning is performed on a partial area on the surface of the liquid crystal panel 140, the entire image is an image corresponding to the area to be scanned.

  A sensing command in which “00” is set in the data area indicating the reading method requests to scan with the backlight 179 on. A sensing command in which “01” is set in the data area indicating the reading method requests scanning with the backlight 179 off. A configuration for scanning with the backlight 179 off is described later (FIG. 11). Furthermore, a sensing command in which “10” is set in the data area indicating the reading method requests scanning using both reflection and transmission. Note that the combined use of reflection and transmission refers to scanning a scan object by switching between a method in which the backlight 179 is turned on for scanning and a method in which the backlight is turned off for scanning.

  A sensing command in which “00” is set in the data area indicating the image gradation requests monochrome image data of black and white. A sensing command in which “01” is set in the data area indicating the image gradation requests multi-gradation image data. Furthermore, a sensing command in which “10” is set in the data area indicating the image gradation requests RGB color image data.

  A sensing command in which “0” is set in the data area indicating the resolution requests image data with a high resolution. A sensing command in which “1” is set in the data area indicating the resolution requests image data with a low resolution.

  In addition to the data shown in FIG. 7 and FIG. 8, the sensing command includes designation of an area to be scanned (area of pixels that drive the optical sensor circuit 144), timing to perform scanning, and timing to turn on the backlight 179. Etc. are described.

  FIG. 9 is a diagram showing a schematic configuration of response data. The response data is data corresponding to the content of the sensing command, and is data that the image processing engine 180 of the display device 102 transmits to the main body device 101.

  Referring to FIG. 9, the response data includes a header data area DA11, a coordinate data area DA12, a time data area DA13, and an image data area DA14. Here, the value of the center coordinates of the partial image is written in the data area DA12 indicating the coordinates. In addition, time information acquired from the timer 182 of the image processing engine 180 is written in the data area indicating the time. Further, image data (that is, scan data) after being processed by the image processing engine 180 is written in the data area indicating the image.

  FIG. 10 is a diagram illustrating an image obtained by scanning the finger 900 (that is, a scanned image). Referring to FIG. 10, an image of a region W1 surrounded by a thick solid line is an entire image, and an image of a region P1 surrounded by a broken line is a partial image. The center point C1 of the cross indicated by a thick line is the center coordinate.

  In the present embodiment, a pixel (that is, a predetermined gradation or a pixel having a photosensor circuit which is a rectangular region and whose output voltage from the sensor signal line SSj is equal to or higher than a predetermined value). An area including all of the pixels having a luminance equal to or higher than a predetermined luminance is set as a partial image area.

  The center coordinates are coordinates determined in consideration of the gradation of each pixel in the partial image area. Specifically, for each pixel in the partial image, the center coordinate is determined by performing a weighting process based on the gradation of the pixel and the distance between the pixel and the rectangular center point (that is, the centroid). Is done. That is, the center coordinates do not necessarily match the centroid of the partial image.

  However, the position of the center coordinates is not necessarily limited to the above, and the center coordinates may be the coordinates of the centroid or the coordinates near the centroid.

  When “001” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate. In this case, the image processing engine 180 does not write image data in the data area DA14 indicating an image. After writing the value of the center coordinate, the image processing engine 180 sends response data including the value of the center coordinate to the main body device 101. As described above, when “001” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate without requesting the output of the image data.

  Further, when “010” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the image data of the partial image whose scan result has changed in the data area DA14 indicating the image. . In this case, the image processing engine 180 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. The image processing engine 180 writes the image data of the partial image whose scan result has changed, and then sends response data including the image data of the partial image to the main body device 101. As described above, when “010” is set in the data area indicating the data type, the sensing command does not request the output of the value of the center coordinate, and the image data of the partial image whose scan result has changed. Request output.

  As described above, the reason for requesting transmission of only the partial image whose scan result has changed is that the scan data of the partial image area of the scan data is more important than the scan data of the other area. This is because the data is high, and the scan data in the region corresponding to the region of the partial image in the scan data is likely to change due to the contact state with the scan object such as the finger 900.

  When “011” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also displays the data area indicating the image. The image data of the partial image whose scan result has changed is written in DA14. Thereafter, the image processing engine 180 sends response data including the value of the center coordinate and the image data of the partial image to the main body device 101. As described above, when “011” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate and the output of the image data of the partial image whose scan result has changed. To do.

  If “100” is set in the data area indicating the data type of the sensing command, the image processing engine 180 displays the image of the entire image in the data area DA14 indicating the image of the response data shown in FIG. Write data. In this case, the image processing engine 180 does not write the value of the center coordinate in the data area DA12 indicating the coordinate. After writing the image data of the entire image, the image processing engine 180 sends response data including the image data of the entire image to the main body device 101. As described above, when “100” is set in the data area indicating the data type, the sensing command requests the output of the image data of the entire image without requesting the output of the value of the center coordinate.

  When “101” is set in the data area indicating the data type of the sensing command, the image processing engine 180 writes the value of the central coordinate in the data area DA12 indicating the coordinate and also displays the data area indicating the image. Write the image data of the entire image to DA14. Thereafter, the image processing engine 180 sends response data including the value of the center coordinate and the image data of the entire image to the main body device 101. As described above, when “101” is set in the data area indicating the data type, the sensing command requests the output of the value of the center coordinate and the output of the image data of the entire image.

<Variation of scanning method>
Next, a scanning method different from the scanning method described with reference to FIG. 6 (that is, the method of scanning the reflected image) will be described with reference to FIG.

  FIG. 11 is a cross-sectional view showing a configuration in which a photodiode receives external light during scanning. Referring to FIG. 11, part of the outside light is blocked by finger 900. Therefore, the photodiode disposed under the surface region of the liquid crystal panel 140 that is in contact with the finger 900 can hardly receive external light. In addition, although the photodiodes disposed under the surface area where the shadow of the finger 900 is formed can receive a certain amount of external light, the amount of external light received is small compared to the surface area where no shadow is formed.

  Here, by turning off the backlight 179 at least during the sensing period, the optical sensor circuit 144 can output a voltage corresponding to the position of the finger 900 with respect to the surface of the liquid crystal panel 140 from the sensor signal line SSj. it can. In this manner, by controlling the backlight 179 to be turned on and off, the liquid crystal panel 140 has a contact position of the finger 900, a range in which the finger 900 is in contact (determined by the pressing force of the finger 900), and the liquid crystal panel 140. The voltage output from the sensor signal lines (SS1 to SSn) varies depending on the direction of the finger 900 relative to the surface of the sensor.

  As described above, the display device 102 can scan an image (hereinafter also referred to as a shadow image) obtained by blocking external light with the finger 900.

  Further, the display device 102 may be configured to perform scanning by turning on the backlight 179 and then performing scanning again by turning off the backlight 179. Alternatively, the display device 102 may be configured to perform scanning by turning off the backlight 179 and then performing scanning again by turning on the backlight 179.

  In this case, since two scanning methods are used together, two scan data can be obtained. Therefore, it is possible to obtain a highly accurate result as compared with the case of scanning using only one scanning method.

<About display devices>
The operation of the display device 103 is controlled in accordance with a command (for example, a sensing command) from the main body device 101 as in the operation of the display device 102. The display device 103 has the same configuration as the display device 102. Therefore, when the display device 103 receives the same command as the display device 102 from the main body device 101, the display device 103 performs the same operation as the display device 102. For this reason, description of the configuration and operation of the display device 103 will not be repeated.

  Note that the main device 101 can send commands having different commands to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform different operations. Further, the main device 101 may send a command to either the display device 102 or the display device 103. In this case, only one display device performs an operation according to the command. Further, the main device 101 may send a command having the same command to the display device 102 and the display device 103. In this case, the display device 102 and the display device 103 perform the same operation.

  Note that the size of the liquid crystal panel 140 of the display device 102 and the size of the liquid crystal panel 240 of the display device 103 may be the same or different. Further, the resolution of the liquid crystal panel 140 and the resolution of the liquid crystal panel 240 may be the same or different.

  And the liquid crystal panels 140 and 240 of the electronic device 100 according to the present embodiment may not be of the photosensor built-in type. That is, the liquid crystal panels 140 and 240 of the electronic device 100 according to the present embodiment may both be normal touch panels.

<Functional configuration of electronic device 100 according to the present embodiment>
Hereinafter, a functional configuration of electronic device 100 according to the present embodiment will be described with reference to FIGS. 1, 2, and 12. FIG. 12 is a block diagram showing a functional configuration of electronic device 100 according to the present embodiment.

  Electronic device 100 according to the present embodiment includes a display control unit 111, a reception unit 112, a determination unit 113, and a switching unit 114. In addition, as illustrated in FIG. 2, the electronic device 100 includes a first display panel 140 including a plurality of photosensor circuits 144 and a plurality of pixel circuits 141, a plurality of photosensor circuits 244, and a plurality of pixel circuits. 241 including a second display panel 240 and operation keys 177. The first display panel may be a concept showing the first display device 102 including the first photosensor built-in liquid crystal panel 140, the image processing engine 180, the internal IF 178, and the like. The second display panel may be a concept showing the second display device 103 including the second optical sensor built-in liquid crystal panel 240, the image processing engine 280, the internal IF 278, and the like.

  First, each of the plurality of photosensor circuits 144 of the first display panel 140 receives incident light and generates an electrical signal corresponding to the incident light. The plurality of photosensor circuits 144 as a whole input an electrical signal (image data) corresponding to incident light to the receiving unit 112 via the image processing engine 180 (FIG. 2) or the like. Note that the plurality of optical sensor circuits 144 may read the position of the finger or the stylus pen with the backlight 179 turned off as shown in FIG.

  As described above, the plurality of optical sensor circuits 144 and the image processing engine 180 according to the present embodiment realize the first operation unit as a whole. Then, the operation unit accepts a movement command for moving an image such as a window X displayed on the first display panel 140 via the first display panel 140, or sets the position on the first display panel 140. Accepts a position specification command to be specified.

  Each of the plurality of pixel circuits 141 of the first display panel 140 emits visible light to the outside based on image data and text data from the display control unit 111, that is, based on an output signal from the CPU 110. More specifically, the plurality of pixel circuits 141 use light from the backlight 179 as a whole based on image data or text data from the display control unit 111 via the image processing engine 180 (FIG. 2) or the like. While displaying content such as images and text.

  Thus, the plurality of pixel circuits 141 and the image processing engine 180 according to the present embodiment as a whole realize the first display unit. That is, the first display unit displays an image on the first display panel 140.

  Although the 1st operation part and 1st display part which concern on this Embodiment are implement | achieved by the 1st display panel 140 (1st display apparatus 102), the 1st operation part and 1st display part May be realized by a normal touch panel type display panel. Note that the configuration of the second display panel 240 (second display device 103) is substantially the same as the configuration of the first display panel 140 (first display device 102), and thus description thereof will be repeated here. Absent.

  The display control unit 111, the reception unit 112, the determination unit 113, and the switching unit 114 are functions realized by the CPU 110 and the like. More specifically, each function of the CPU 110 is a function realized by the CPU 110 executing a control program stored in the RAM 171 or the like to control each hardware shown in FIG.

  First, in the first mode, the display control unit 111 causes the first display panel 140 to display a window X on a part thereof. In the second mode, the display control unit 111 causes the first display panel 140 or the second display panel 240 to display the content included in the window X in full screen. In the third mode, the display control unit 111 causes the first display panel 140 and the second display panel 240 to display the content included in the window X in full screen.

  That is, the display control unit 111 allocates a part of the first display panel 140 as the display area of the window X in the first mode. In the second mode, the display control unit 111 assigns all of the first display panel 140 or all of the second display panel 240 to the display area of the window X or the content displayed in the window X. In the third mode, the display control unit 111 assigns all of the first display panel 140 and all of the second display panel 240 to the display area of the content displayed in the window X or the window X.

  The reception unit 112 is a position for designating a position within the first display panel 140 and a movement command (such as a so-called drag and drop operation) of the window X input to the first and second display panels 140 and 240. A designation command, a position designation command for designating the position of the second display panel 240, and the like are received. More specifically, the accepting unit 112 receives the first and first signals from the plurality of photosensor circuits 144 and 244 via the finger 900 and the stylus pen based on the electrical signals input via the image processing engines 180 and 280. The movement command and the position designation command input to the second display panels 140 and 240 are received.

  More specifically, the reception unit 112 acquires image data output from the image processing engine 180 of the first display panel 140 at every sensing time, and issues a movement command, a position designation command, or the like based on the image data. Accept. The accepting unit 112 acquires image data output from the image processing engine 280 of the second display panel 240 at every sensing time, and accepts a movement command, a position designation command, and the like based on the image data.

  During the first mode, the reception unit 112 receives the window X or the window X based on the electric signal input from the plurality of photosensor circuits 144 via the image processing engine 180 or the electric signal from the operation key 177. A maximization command for maximizing and displaying the content displayed in window X (full screen display) is received.

  During the third mode, the reception unit 112 receives the first signal based on the electrical signal input from the plurality of photosensor circuits 144 via the image processing engine 180 or based on the electrical signal from the operation key 177. The display panel 140 and the second display panel 240 receive normalization instructions for content displayed on the full screen. The accepting unit 112 accepts the normalization command during the third mode, and the first display panel 140 or the second display via either the first display panel 140 or the second display panel 240. A first position designation command for designating a position on any of the panels 240 is received.

  The display control unit 111 displays the window X at the designated position on the first display panel 140 or the second display panel 240 based on the normalization command and the first position designation command.

  During the second mode, the receiving unit 112 receives the first signal based on the electrical signal input from the plurality of photosensor circuits 144 via the image processing engine 180 or based on the electrical signal from the operation key 177. A normalization command for a window X or content displayed in full screen on either the display panel 140 or the second display panel 240 is accepted. The accepting unit 112 accepts a normalization command during the second mode, and the first display panel 140 or the second display via either the first display panel 140 or the second display panel 240. A second position designation command for designating a position on any of the panels 240 is received.

  The display control unit 111 displays the window X at the designated position on the first display panel 140 or the second display panel 240 based on the second position designation command. When the content is displayed on the full screen on one of the first display panel 140 or the second display panel 240, the display control unit 111 receives the second position designation command for the one from the reception unit 112. In response, the window X is displayed at the one designated position. However, when the content is displayed on the full screen on one of the first display panel 140 or the second display panel 240, the display control unit 111 receives the first display panel 140 or the second display panel from the reception unit 112. In response to the second position designation command for the other display panel 240, the window X may be displayed at the other designated position.

  Here, the receiving unit 112 may store the first image data output from the image processing engine 180 in the RAM 171. That is, the reception unit 112 may constantly update the first image data in the RAM 171 to the latest image data. Similarly, the reception unit 112 may store the second image data output from the image processing engine 280 in the RAM 171. That is, the reception unit 112 may constantly update the second image data in the RAM 171 to the latest image data.

  The reception unit 112 may have a function realized by the CPU 110, the plurality of photosensor circuits 144 of the first display panel 140, and the plurality of photosensor circuits 244 of the second display panel 240. That is, the accepting unit 112 may be a concept showing a functional block including a part of the function of the CPU 110, the light receiving function of the first display panel 140, and the light receiving function of the second display panel 240.

  When the reception unit 112 receives the maximization command, the determination unit 113 displays whether the window X is displayed in a predetermined area set on the first display panel 140 or the second display panel 240 that is set in advance. Judge whether or not. For example, the predetermined area may include any one of the upper end portion, the right end portion, the lower end portion, the left end portion, and the center portion of each of the first display panel 140 or the second display panel 240. Multiple portions thereof may be included.

  That is, the determination unit 113 determines whether the window X is displayed in the predetermined area according to whether the upper left coordinates of the window X and the center coordinates of the window X are located in the predetermined area. . Alternatively, the determination unit 113 determines whether or not the window X is displayed in the predetermined area according to whether or not the display area of the window X is within the predetermined area.

  As described below, the determination unit 113 according to the present embodiment determines whether or not the display control unit 111 displays the window X across the first display panel 140 and the second display panel 240. Determine whether.

  More specifically, the determination unit 113 displays the window X in a predetermined area when the display control unit 111 displays the window X across the first display panel 140 and the second display panel 240. Judge that Conversely, the determination unit 113 displays the window X in the second display panel 240 when the display control unit 111 displays the window X in the first display panel 140 and the display control unit 111 displays the window X in the second display panel 240. In this case, it is determined that the window X is not displayed in the predetermined area.

  In other words, the determination unit 113 acquires window information indicating the display area assigned to the window X from the display control unit 111. For example, the window information may be the coordinates of the four corners of the window X for specifying the display area assigned to the window X. Alternatively, the window information may include the coordinates of the upper left corner of the window X, the horizontal size (number of dots) of the window X, and the vertical size (number of dots) of the window X.

  Then, the determination unit 113 acquires a first coordinate range corresponding to the first display panel 140 from the RAM 171. The determination unit 113 may acquire the second coordinate range corresponding to the second display panel 240 from the RAM 171.

  The determination unit 113 determines whether the display area of the window X extends between the first display panel 140 and the second display panel 240 based on the window information, the first coordinate range, and the second coordinate range. Judging. That is, the determination unit 113 determines that the window X is displayed in the predetermined area when the display area of the window X extends over the first display panel 140 and the second display panel 240.

  The determination unit 113 may determine whether or not the display area of the window X protrudes from the first display panel 140 based on the window information and the first coordinate range. That is, when the display area of the window X protrudes from the first display panel 140, it may be determined that the window X is displayed in the predetermined area.

  The determination unit 113 may determine whether or not the display area of the window X protrudes from the second display panel 240 based on the window information and the second coordinate range. That is, when the display area of the window X protrudes from the second display panel 240, it may be determined that the window X is displayed in the predetermined area.

  Alternatively, the determination unit 113 determines whether or not the window X is positioned at the lower end portion of the predetermined first display panel 140 based on the window information, and the second display in which the window X is determined in advance. It may be determined whether or not it is located at the upper end portion of the panel 240. Then, the determination unit 113 displays the window X in a predetermined area when the window X is located at the lower end portion of the first display panel 140 and the window X is located at the upper end portion of the second display panel 240. It may be determined that

  The switching unit 114 switches to the third mode when the determination unit 113 determines that the window X is displayed in the predetermined area when the reception unit 112 receives the maximization command. Is switched to the second mode when it is determined that is not displayed in the predetermined area. The switching unit 114 switches to the first mode when the receiving unit 112 receives the normalization command during the third mode and the second mode.

<Image Display Processing According to this Embodiment>
Next, image display processing in electronic device 100 according to the present embodiment will be described with reference to FIGS. FIG. 13 is an image diagram showing a processing procedure of image display processing in electronic device 100 according to the present embodiment. Hereinafter, the contents displayed in the window X displayed on the first display panel 140 or the second display panel 240 are all transferred to at least one of the first display panel 140 and the second display panel 240. A case where the screen is displayed will be described.

  First, the CPU 110 functioning as the reception unit 112 determines whether or not it is in the first mode (step S102). In the first mode (when YES at step S102), CPU 110 converts the electrical signals from the plurality of photosensor circuits 144 and 244 of first and second display panels 140 and 240 into image processing engine 180. , 280 to receive as image data. In other words, the CPU 110 always acquires image data from the image processing engines 180 and 280 (display devices 102 and 103).

  CPU 110 determines whether or not an instruction to move window X has been received based on the image data from image processing engines 180 and 280 (step S104). CPU 110 functioning as display control unit 111 moves window X on first display panel 140 based on the move command when accepting unit 112 accepts the move command for window X (if YES in step S104). (Step S106). CPU 110 determines whether an image maximization command (image full-screen display command) has been received (step S108).

  CPU110 repeats the process from step S104, when an image maximization command is not received (when it is NO in step S108). When CPU 110 functioning as determination unit 113 receives an image maximization command (YES in step S108), window X is displayed across first display panel 140 and second display panel 240. It is judged whether it is (step S110).

  When window X is displayed across first display panel 140 and second display panel 240 (YES in step S110), CPU 110 functioning as switching unit 114 switches to the third mode. (Step S114). Conversely, when window X is not displayed across first display panel 140 and second display panel 240 (NO in step S110), that is, window X is in first display panel 140. If the window X is displayed in the second display panel 240, the CPU 110 switches to the second mode (step S112).

  On the other hand, when it is not the first mode (NO in step S102), CPU 110 determines whether or not a normalization command has been received (step S122). When CPU 110 receives a normalization command (YES in step S122), CPU 110 determines whether or not a position designation command is received within a predetermined time (step S124).

  When the CPU 110 receives a position designation command within a predetermined time (YES in step S124), the CPU 110 switches to the first mode (step S126), and then the first display panel 140 or the second display The window X is displayed at the designated position on the display panel 240 (step S128). However, the CPU 110 may accept the normalization command while receiving the position designation command.

  When CPU 110 does not accept a position designation command within a predetermined time (NO in step S124), CPU 110 switches to the first mode (step S130), and then displays first display panel 140 or second display. The window X is displayed at the designated position where the window X was originally displayed on the panel 240 (step S132).

<Other embodiments>
It goes without saying that the present invention can also be applied to a case where it is achieved by supplying a program to a system or apparatus. Then, a storage medium storing a program represented by software for achieving the present invention is supplied to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the program code stored in the storage medium It is possible to enjoy the effects of the present invention also by reading and executing.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card (IC memory card), ROM (mask ROM, flash) EEPROM, etc.) can be used.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also an OS (operating system) running on the computer based on the instruction of the program code However, it is needless to say that a case where the function of the above-described embodiment is realized by performing part or all of the actual processing and the processing is included.

  Further, after the program code read from the storage medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 is a schematic diagram illustrating an external appearance of an electronic device according to a first embodiment. It is a block diagram showing the hardware constitutions of an electronic device. It is the figure which showed the structure of the liquid crystal panel, and the peripheral circuit of the said liquid crystal panel. It is sectional drawing of a liquid crystal panel and a backlight. It is the figure which showed the timing chart at the time of operating an optical sensor circuit. It is sectional drawing which showed the structure which a photodiode receives the light from a backlight in the case of a scan. It is the figure which showed schematic structure of the sensing command. It is the figure which showed the value of the data in each area | region of a sensing command, and the meaning content which the said value shows. It is the figure which showed schematic structure of the response data. It is the figure which showed the image obtained by scanning a finger | toe. It is sectional drawing which showed the structure which a photodiode receives external light in the case of a scan. 1 is a block diagram illustrating a functional configuration of an electronic device according to a first embodiment. 6 is an image diagram illustrating a processing procedure of image display processing in the electronic apparatus according to Embodiment 1. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 Electronic device, 101 Main body apparatus, 102,103 Display apparatus, 110 CPU, 111 Display control part, 112 Reception part, 113 Judgment part, 114 Switching part, 130 Driver, 131 Scan signal line drive circuit, 132 Data signal line drive circuit 133 Photosensor driving circuit, 134 switch, 135 amplifier, 140 First optical sensor built-in liquid crystal panel (first display panel), 141, 241 pixel circuit, 141b, 141g, 141r subpixel circuit, 143 electrode pair, 143a Pixel electrode, 143b Counter electrode, 144, 244 Photo sensor circuit, 145, 145b, 145g, 145r Photodiode, 146 capacitor, 151A Active matrix substrate, 151B Counter substrate, 152 Liquid crystal layer, 153b, 153g, 153r Color Filter, 157 data signal line, 161 polarizing filter, 162 glass substrate, 163 light shielding film, 164 alignment film, 171 RAM, 173 memory card reader / writer, 174 communication device, 175 microphone, 176 speaker, 177 operation key, 179 backlight, 180 image processing engine, 181 driver control unit, 182 timer, 183 signal processing unit, 240 second optical sensor built-in liquid crystal panel (second display panel), 1731 memory card, DB1, DB2 data bus.

Claims (8)

First and second display panels;
A first image is displayed on a part of the first display panel in the first mode, and a second image corresponding to the first image is displayed on the first display panel in the full screen in the second mode. is displayed, and display control means for the entire screen displayed in the third third of the first and second display panels images that corresponds to the first image in the mode,
Receiving means for receiving a first command during the first mode;
Determining means for determining whether or not the first image is displayed in a predetermined area when the first instruction is received;
When the determination unit determines that the first image is displayed in the predetermined area, the mode is switched to the third mode, and the determination unit determines that the first image is not displayed in the predetermined area. and a switching means for switching when the second mode,
The determination means determines whether the display control means is displaying the first image on the first display panel and the second display panel, and the display control means is the first display panel. An image display device that determines that the first image is displayed in the predetermined area when the first image is displayed on a display panel and the second display panel . The accepting means accepts a second command during the third mode;
The image display device according to claim 1, wherein the switching unit switches to the first mode when receiving the second command. The accepting means accepts the second command during the third mode, and either one of the first and second display panels via one of the first and second display panels. Accepts a first designation command to designate a position at
3. The image display according to claim 2 , wherein the display control unit displays the first image at the designated position in one of the first and second display panels based on the first designation command. 4. apparatus. The accepting means accepts a third command during the second mode,
Said switching means, said third when the accepted instruction to switch to the first mode, the image display apparatus according to claim 3. The accepting unit accepts the third command during the second mode, and accepts a second designation command for designating a position on the first display panel via the first display panel. ,
The image display device according to claim 4 , wherein the display control unit displays the first image at the designated position on the first display panel based on the second designation command. Each of the first and second display panels includes:
A plurality of photosensor circuits that generate input signals in response to incident light;
A plurality of pixel circuits that emit light according to an output signal,
The accepting means accepts the command based on the input signals from the first and second display panels;
Wherein the display control unit by outputting the output signal to said first and second display panel to display the image on the first and second display panels, one of claims 1 to 5 1 The image display device according to item. An image display method in an image display device including first and second display panels and an arithmetic processing unit,
The arithmetic processing unit displaying a first image on a part of the first display panel in a first mode;
The arithmetic processing unit displaying a second image corresponding to the first image in the second mode on the first display panel in a full screen;
A step wherein the processing unit is to be full screen third image that corresponds to the first image in the third mode to the first and second display panels,
The arithmetic processing unit accepting a first command during the first mode;
A step of determining whether or not the first image is displayed in a predetermined area when the arithmetic processing unit receives the first instruction;
A step of switching to the third mode when the arithmetic processing unit determines that the first image is displayed in the predetermined area;
A step of switching to the second mode when the arithmetic processing unit determines that the first image is not displayed in the predetermined area;
The determining step determines whether or not the first image is displayed on the first display panel and the second display panel, and the first display panel and the second display panel. And determining that the first image is displayed in the predetermined area when the first image is displayed. An image display program for displaying an image on an image display device including first and second display panels and an arithmetic processing unit,
In the arithmetic processing unit,
Displaying a first image on a portion of the first display panel in a first mode;
Displaying a second image corresponding to the first image on the first display panel in a second mode in a second mode;
A step of full screen of a third image that corresponds to the first image to the first and second display panels in the third mode,
Receiving a first command during the first mode;
Determining whether or not the first image is displayed in a predetermined area when the first command is received;
Switching to the third mode when it is determined that the first image is displayed in the predetermined area;
Executing the step of switching to the second mode when it is determined that the first image is not displayed in the predetermined area ;
The determining step determines whether or not the first image is displayed on the first display panel and the second display panel, and the first display panel and the second display panel. When the first image is displayed, the image display program includes determining that the first image is displayed in the predetermined area .

Images