JP6330179B2 - Multi-window display device, relay device, control method for multi-window display device, control method for relay device, and control method for information processing device - Google Patents

Description

  The present invention relates to a multi-window display device, a relay device, a control method for a multi-window display device, a control method for a relay device, and a control method for an information processing device.

  Conventionally, one information processing apparatus having a plurality of display screens (or displaying a plurality of windows) is operated by a user on one of the plurality of display screens (hereinafter referred to as “screen operation”). ) In general. The same applies to an information processing apparatus that displays a plurality of windows, and is generally configured to accept an operation by a user on any one display screen of the plurality of windows.

  In this case, each of the display screen or the window (hereinafter referred to as “display screen or the like”) cannot accept the active state in which the operation corresponding to the application corresponding to the display screen or the like can be accepted, and the above operation. It is configured to be capable of taking two states, the inactive state. The plurality of display screens are managed so that one of them is in an active state and the display screens other than the active state are in an inactive state.

  When a screen operation is performed on an active display screen or the like, the screen operation is provided to an application corresponding to the display screen. On the other hand, when a screen operation is performed on a display screen in an inactive state, the display screen is changed to an active state, and display screens other than the display screen that has been changed to an active state are changed to an inactive state. . As described above, the display screen that was in the active state becomes inactive. At the same time, the display screen that was in the inactive state becomes active so that the active state moves between the display screens. Is also called focus switching.

  Patent Document 1 discloses a technique for performing an operation on each of applications executed on two display screens by a flick operation on a button. According to the technique disclosed in Patent Document 1, an operation can be performed without arranging a button on each of the two display screens or switching the focus of the display screen.

JP 2012-54880 A

  However, when a plurality of users perform screen operations on the plurality of display screens of the information processing apparatus in parallel, focus switching is performed when the user performs one screen operation on the display screen in an inactive state. The above screen operation is not accepted as an operation for the application. In order for the user to perform an operation on the application displayed on the display screen in the inactive state, two screen operations are required, and the operation by the user is not intuitively performed.

  Even in consideration of applying the technique described in Patent Document 1 in such a case, the technique described in Patent Document 1 is used when the physical positional relationship between the two display screens is not defined. Cannot be applied due to restrictions such as inability to apply.

  The present invention has been made to solve the above problem, and does not depend on a physical positional relationship between a plurality of display screens, and multi-window display that appropriately accepts a user's screen operation on the plurality of display screens. An object is to provide a device or the like.

  In order to solve the above problems, a multi-window display device according to an aspect of the present invention is a multi-window display device for displaying a plurality of windows on one or more display screens, and any one of the plurality of windows. A reception unit that receives an operation by the user and generates operation data indicating the operation, a replication unit that generates a copy of the operation data generated by the reception unit, and a plurality of functional units corresponding to each of the plurality of windows And (a) first processing for transitioning the window to an active state based on operation data indicating an operation on a window in an inactive state, and (b) operation data indicating an operation on a window in an active state, A second process of providing the operation data to a function unit corresponding to the window among the plurality of function units. When the operation data is generated by the window management unit for performing the operations, and the reception unit and the duplication unit, the target window that is the target of the operation indicated by the operation data is (i) in the active state In the case, based on the operation data generated by one of the reception unit and the duplication unit, the window management unit performs the second process, and (ii) when in an inactive state, A processing control unit configured to cause the window management unit to perform the second process after performing the first process based on the operation data generated by each of the reception unit and the duplication unit.

  According to this, the multi-window display device accepts one screen operation by the user as one operation for an active window and two operations for an inactive window. Process. By doing in this way, one operation data can be appropriately provided to the application corresponding to each window of an active state and an inactive state by one screen operation by a user. At this time, it is possible to prevent the occurrence of an unintended result of the user such as two pieces of operation data being provided to the application or none of the operation data being provided. Therefore, the multi-window display device can appropriately accept screen operations performed by the user on a plurality of display screens.

  Further, every time the operation data generated by the reception unit or the duplication unit is received, the window management unit notifies the processing control unit of generation of the operation data, and as a response to the notification, When a response indicating that the operation data is valid is received, the first process or the second process is performed according to the state of the target window, and the process control unit is configured so that each of the reception unit and the duplication unit The generated notification of the generation of the two operation data is acquired, and when the target window is (i) in the active state, either one of the two notifications of the generation of the operation data A response to be validated may be made, and (ii) in the case of the inactive state, a response to be valid for both of the two notifications of operation data generation may be made.

  According to this, the processing control unit receives a notification that the operation data has been generated, and responds to the notification whether the operation data is valid or invalid, specifically, the window Processing by the management unit can be controlled. In this way, the multi-window display device is configured more specifically.

  In addition, when the target window is in an active state, the processing control unit causes the window management unit to transfer one of the operation data generated by the other of the receiving unit and the duplicating unit to the other. It may be discarded.

  According to this, the process control unit explicitly discards the operation data not to be processed by the window management unit. By discarding data that is not used for processing, there are effects of reducing the processing load of the multi-window display device and reducing the resources used. In this way, the multi-window display device is configured more specifically.

  The operation is an object included in any of the plurality of windows, and is associated with a predetermined process by a function unit corresponding to the window including the object among the plurality of function units. Each of the plurality of functional units is associated with an object that is a target of the operation indicated by the provided operation data when the operation data is provided from the window management unit. The predetermined process may be executed.

  According to this, the multi-window display device can provide a specific information processing result to the user by the functional unit performing processing based on the screen operation performed on the window by the user.

  The multi-window display device includes a plurality of display devices, and each of the plurality of display devices has one display screen of the one or more display screens, and the plurality of display devices are included in the one display screen. One of the windows may be displayed in a maximized state.

  According to this, the multi-window display device can accept an operation from a different user for each of the plurality of display devices.

  The multi-window display device further includes a relay unit having the replication unit, the relay unit transmitting the operation data generated by the reception unit and the replication unit via a network, and the window management The unit may acquire the operation data via the network.

  The multi-window display device further includes a relay unit that transmits the operation data generated by the reception unit via a network. The window management unit further includes the duplication unit, and the window management unit The unit may acquire the operation data via the network and acquire a copy of the operation data generated by the replication unit.

  According to this, since the multi-window display device can be configured so that the display devices are arranged in a plurality of locations via the network, the degree of freedom of arrangement of the multi-window display device is improved. In addition, the multi-window display device can be configured inexpensively and efficiently using a relay unit having a simple function of copying the contents of operation data.

  Further, a control method for a multi-window display device according to an aspect of the present invention is a control method for a multi-window display device for displaying a plurality of windows on one or more display screens, and the multi-window display device includes: A plurality of functional units corresponding to each of the plurality of windows, wherein the control method receives an operation performed by a user on any of the plurality of windows, and generates an operation data indicating the operation; A replication step for generating a copy of the operation data generated in the step; (a) a first process for transitioning the window to the active state based on operation data indicating an operation on the inactive window; and (b) active The plurality of functional units based on operation data indicating an operation on the window in the state. The operation data when the operation data is generated by the window management step for performing the second process of providing the operation data to the function unit corresponding to the window, the reception step, and the duplication step. If the target window that is the target of the operation indicated by (i) is in the active state, the window management step uses the operation data generated by one of the reception step and the duplication step in the window management step. And (ii) when in the inactive state, perform the first process in the window management step based on the operation data generated in each of the reception step and the duplication step. And a process control step for causing the second process to be performed.

  According to this, the same effect as that of the multi-window display device can be obtained.

  The relay device according to an aspect of the present invention is a relay device in a multi-window display device for displaying a plurality of windows on a display screen, and operation data indicating an operation by a user on any of the plurality of windows. And a duplicating unit that creates a duplicate of the obtained operation data, and the duplicating unit further displays the generated operation data in an active state or an inactive state of each of the plurality of windows. The window management unit that manages whether or not there is a transmission is provided so as to be provided to a plurality of function units corresponding to each of the plurality of windows according to the state.

  According to this, the multi-window display device configured using the relay device has the same effect as the multi-window display device.

  A relay device control method according to an aspect of the present invention is a relay device control method in a multi-window display device for displaying a plurality of windows on a display screen, and the user controls any of the plurality of windows. An operation step of acquiring operation data indicating an operation by the operation, and a replication step of generating a copy of the operation data acquired in the acquisition step. In the replication step, the generated operation data is further transmitted to the plurality of windows. The window management step for managing whether each state is an active state or an inactive state is transmitted to be provided to a plurality of functional units corresponding to each of the plurality of windows according to the state. To do.

  According to this, the multi-window display device configured using the relay device has the same effect as the multi-window display device.

  An information processing apparatus control method according to an aspect of the present invention is a method for controlling an information processing apparatus in a multi-window display apparatus for displaying a plurality of windows on one or more display screens. The apparatus includes a display device, a relay device, and the information processing device, and the display device includes a reception unit that generates operation data indicating an operation performed by a user on any of the plurality of windows. Includes a replication unit that generates a copy of the operation data generated by the reception unit, and the information processing apparatus includes: (a) a plurality of functional units corresponding to each of the plurality of windows; A first process for transitioning the window to an active state based on operation data indicating an operation; and (b) a window in the active state. A window management unit for performing a second process of providing the operation data to a function unit corresponding to the window among the plurality of function units based on operation data indicating an operation on the window. In the method, when the operation data is generated by the reception unit and the duplication unit, if the target window that is the operation target indicated by the operation data is (i) active, the reception unit And the window management unit performs the second process based on the operation data generated by any one of the replication unit, and (ii) when in the inactive state, the reception unit and the replication unit A process control step for causing the window management unit to perform the second process after performing the first process based on the operation data generated by each Including.

  According to this, the same effect as that of the multi-window display device can be obtained.

  A program according to one embodiment of the present invention is a program for causing a computer to execute the above control method.

  According to this, the same effect as that of the multi-window display device can be obtained.

  Note that the present invention can be realized not only as an apparatus but also as a method using steps as processing units constituting the apparatus, as a program for causing a computer to execute the steps, or as a computer read recording the program. It can also be realized as a possible recording medium such as a CD-ROM, or as information, data or a signal indicating the program. These programs, information, data, and signals may be distributed via a communication network such as the Internet.

  The multi-window display device according to the present invention can appropriately accept screen operations by a user on a plurality of display screens.

FIG. 1 is a configuration diagram of a multi-window display device according to an embodiment. FIG. 2 is a block diagram illustrating a hardware configuration of the information processing apparatus according to the embodiment. FIG. 3 is a block diagram illustrating a hardware configuration of the relay device according to the embodiment. FIG. 4 is a block diagram showing a functional configuration of the multi-window display device according to the embodiment. FIG. 5 is an explanatory diagram of generation and duplication of operation data according to the embodiment. FIG. 6 is a flowchart illustrating processing of operation data by the window management unit according to the embodiment. FIG. 7 is a flowchart showing processing by the processing control unit according to the embodiment. FIG. 8 is a first sequence diagram showing a flow of processing in the multi-window display device according to the embodiment. FIG. 9 is a second sequence diagram illustrating a processing flow in the multi-window display device according to the embodiment. FIG. 10 is an explanatory diagram illustrating a user's screen operation on the multi-window display device according to the embodiment. FIG. 11 is an explanatory diagram illustrating another example of the display image according to the embodiment. FIG. 12 is a block diagram illustrating a functional configuration of a multi-window display device according to a modification of the embodiment.

  Hereinafter, embodiments will be specifically described with reference to the drawings.

  Each of the embodiments described below shows a preferred specific example of the present invention. The numerical values, shapes, materials, constituent elements, arrangement positions and connecting forms of the constituent elements, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present invention. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept of the present invention are described as optional constituent elements that constitute a more preferable embodiment.

  In addition, the same code | symbol is attached | subjected to the same component and description may be abbreviate | omitted.

(Embodiment)
In the present embodiment, a multi-window display device or the like that appropriately receives screen operations on a plurality of display screens will be described.

  FIG. 1 is a configuration diagram of a multi-window display device 1 according to the present embodiment.

  As shown in FIG. 1, the multi-window display device 1 according to the present embodiment includes an information processing device 10, relay devices R1 and R2, display devices D1 and D2, a network 40, and communication lines C1 and C2. With.

  The information processing apparatus 10 is an information processing apparatus that performs main information processing in the multi-window display apparatus 1. In the information processing apparatus 10, an OS (Operating System) capable of displaying one or more windows on one or more display screens and an application operating on the OS are running. Examples of the OS include Windows (registered trademark), Mac OS, and Linux (registered trademark).

  The information processing apparatus 10 is connected to the relay apparatus R1 via the network 40. The information processing device 10 receives input of information from the users U1 and U2 via the display devices D1 and D2, respectively, performs information processing according to the input information, and performs the information processing according to the input information, and the user via the display devices D1 and D2. Output information to U1 and U2. The input and output of information as described above are performed in parallel. The information processing apparatus 10 is realized by, for example, a PC (Personal Computer) or a server apparatus. In addition, as long as it has a function equivalent to the above, a mobile phone terminal, a smart phone (high-function mobile phone terminal), a tablet, or the like may be used.

  The relay device R1 is a relay device that relays various communications performed between the information processing device 10 and the display device D1. The relay device R1 converts the communication standard as necessary when relaying. Specifically, the communication signal according to the communication standard of the network 40 and the communication signal according to the communication standard of the communication line C1 are mutually converted.

  The display device D1 is a display device that presents information to the user U1 through a display screen and accepts a screen operation by the user U1. Specifically, the display device D1 receives an image signal from the information processing device 10 from the communication line C1, and displays an image on the display screen based on the received image signal. In addition, the display device D1 receives a screen operation from the user U1, and transmits operation data indicating the received screen operation to the communication line C1. The display device D1 is realized by, for example, a touch panel display. Hereinafter, an example in which the display device D1 is realized by a touch panel display will be described. However, other devices (for example, a display and a mouse) may be used as long as they have a similar function.

  The network 40 is a communication network that connects the information processing apparatus 10 and the relay apparatus R1 so that they can communicate with each other. Any communication standard may be used for the network 40. The network 40 may be, for example, a wired LAN interface that conforms to the IEEE 802.3 standard or the like, or may be a wireless LAN interface that conforms to the IEEE 802.11a, b, g, or n standard.

  The communication line C1 is a communication line that connects the relay device R1 and the display device D1 so that they can communicate with each other. The signal communication standard from the relay device R1 to the display device D1 may be any communication standard that can transmit an image signal. As this communication standard, for example, there is HDMI (registered trademark) (High-Definition Multimedia Interface), DVI (Digital Visual Interface), or the like.

  The communication standard of the signal from the display device D1 to the relay device R1 may be any communication standard that can transmit operation data indicating a screen operation. As this communication standard, for example, there is a USB (Universal Serial Bus) standard. Hereinafter, a case where the USB standard is used will be described as an example.

  Note that the relay device R2, the display device D2, and the communication line C2 have the same functions as the relay device R1, the display device D1, and the communication line C1, respectively. The user U2 receives information from the display device D2 and performs screen operations. The multi-window display device 1 may further include a relay device, a display device, and a communication line that have the same functions as the relay device R1, the display device D1, and the communication line C1, respectively. Since the operations of the relay device, the display device, and the communication line can be easily inferred from the description in the present embodiment, the detailed description is omitted.

  Hereinafter, hardware configurations of the information processing apparatus 10, the relay apparatus R1, and the display apparatus D1 will be described.

  FIG. 2 is a block diagram showing a hardware configuration of the information processing apparatus 10 according to the present embodiment.

  As illustrated in FIG. 2, the information processing apparatus 10 includes a CPU 101 (Central Processing Unit), a main memory 102, a storage 103, and a network interface 104 (Network Interface).

  The CPU 101 is a processor that executes a control program stored in a storage or the like.

  The main memory 102 is a volatile storage area used as a work area used when the CPU 101 executes a control program.

  The storage 103 is a non-volatile storage area that holds a control program, content, and the like.

  The NIF 104 is a network interface that transmits data to other devices via the network 40 or receives data from other devices.

  FIG. 3 is a block diagram showing a hardware configuration of relay apparatus R1 according to the present embodiment.

  As illustrated in FIG. 3, the relay device R1 includes a CPU 201, a main memory 202, a storage 203, an NIF 204, an operation input IF 205, and an image output IF 206.

  The CPU 201, the main memory 202, the storage 203, and the NIF 204 have the same functions as blocks with the same name in the information processing apparatus 10.

  The operation input IF 205 is an interface for acquiring operation data from the communication line C1.

  The image output IF 206 is an interface for outputting image data to the communication line C1.

  The functions and operations of the multi-window display device 1 realized by using the information processing device 10 and the relay device R1 having such a hardware configuration will be described below.

  FIG. 4 is a block diagram showing a functional configuration of the multi-window display device 1 according to the present embodiment.

  As shown in FIG. 4, the multi-window display device 1 includes a receiving unit 111, a duplicating unit 112, a converting unit 113, a window managing unit 114, functional units A1 and A2, and processing control units P1 and P2. The display control unit 115 and the display screen 116 are provided.

  The accepting unit 111 is an interface processing unit that accepts a screen operation by the user for any of a plurality of windows displayed on the display screen 116 and generates operation data indicating the accepted screen operation. Specifically, the reception unit 111 detects a touch on the touch panel by the user as a screen operation, and generates operation data including position information indicating a position on the touched touch panel. The accepting unit 111 generates a USB communication frame including the generated operation data, and transmits the generated USB communication frame onto the communication line C1. The accepting unit 111 is realized by a touch panel portion in the touch panel display. Note that the screen operation is also simply referred to as an operation.

  The duplication unit 112 is a processing unit that creates a duplication of the operation data generated by the reception unit 111. Specifically, the duplication unit 112 acquires the USB communication frame generated by the reception unit 111 and generates the same USB communication frame as the acquired USB communication frame, thereby generating the copy. In addition, the duplication unit 112 passes the USB communication frame generated by the reception unit 111 and the USB communication frame generated by the duplication unit 112 to the conversion unit 113. The duplication unit 112 is realized by the CPU 201 or the like.

  The conversion unit 113 acquires the USB communication frame generated by the reception unit 111 and the duplication unit 112, and transmits the acquired USB communication frame to the information processing apparatus 10 to the network 40. When transmitting, the conversion unit 113 converts the acquired USB communication frame into a format communicable on the network 40. Specifically, the conversion unit 113 encapsulates the USB communication frame into a communication frame that can be communicated with the network 40 and transmits it to the network 40. Further, the conversion unit 113 acquires display information generated by the display control unit 115 and converted into a format communicable on the network 40 by the conversion unit 113A of the information processing apparatus 10 via the network 40, and the display device The data is converted into a format that can be acquired on the display screen 116 of D1, and transmitted through the communication line C1. The conversion unit 113 is realized by the CPU 201 or the like.

  The conversion unit 113A receives the encapsulated USB communication frame transmitted from the conversion unit 113 and converts it into a format that can be processed by the window management unit 114. Further, the data displayed on the display screen 116 generated by the display control unit 115 is converted (encapsulated) into a format communicable on the network 40 and transmitted to the relay apparatus R1 through the network 40. Note that the conversion unit 113A is realized by the CPU 101 or the like.

  The window management unit 114 manages the state (active state or inactive state) of a plurality of windows displayed on the display screen 116, and operates the operation data acquired from the reception unit 111 or the duplication unit 112 with an appropriate function unit A1 or the like. Is a processing unit to be provided. Note that the window management unit 114 is generally realized as a part of an OS function such as Windows (registered trademark) and the hardware is realized by the CPU 101 or the like.

  Specifically, the window management unit 114 identifies a target window that is a target window on which the screen operation indicated by the operation data is performed based on the content of the operation data acquired from the reception unit 111 or the duplication unit 112. . This specification is performed, for example, by specifying a window including coordinates on the display screen on which the screen operation is performed.

  In addition, the window management unit 114 acquires operation data from the reception unit 111 or the duplication unit 112 using a USB communication frame, and when the acquired operation data indicates a screen operation on a window in an inactive state, the operation data Based on, a process for transitioning the window to the active state (also referred to as a transition process or a first process) is performed.

  In addition, when the operation data acquired from the reception unit 111 and the duplication unit 112 indicates a screen operation for an active window, the window management unit 114, based on the operation data, the functional unit A1 corresponding to the window. Etc., a process of providing the operation data to the process (also referred to as a providing process or a second process).

  In addition, when the reception unit 111 and the duplication unit 112 generate operation data, the window management unit 114 notifies the functional unit A1 corresponding to the target window that is the target of the screen operation that the operation data has been generated. To do. Then, the transition process or the provision process is performed based on the operation data (that is, treated as valid), or the transition process or the provision process based on the operation data is not performed (that is, treated as invalid). Is accepted as a response to the notification. In accordance with this response, the window management unit 114 determines whether to perform transition processing or provision processing.

  Further, when the window management unit 114 receives an inquiry about the window state from the processing control unit P1, the window management unit 114 answers the state of the window that has received the inquiry.

  In the transition process described above, when one inactive window is transitioned to the active state, the other windows are transitioned to the inactive state. In this way, the window management unit 114 performs a transition process so that the number of active windows is not two or more. The transition processing is performed because the information processing apparatus 10 is designed for use by a single user. That is, the user performs screen operations on one window and the functional unit A1 that the user is paying attention to. This is because it is assumed that no screen operation is performed on the window and the function unit A1 that the user does not pay attention to.

  The functional unit A1 is a processing unit that performs main information processing in the information processing apparatus 10. The functional unit A1 performs information processing based on operation data acquired from the receiving unit 111 or the duplicating unit 112 via the window management unit 114 or the like. In addition, the function unit A1 generates drawing information indicating an image to be drawn in a window corresponding to the function unit A1 in order to receive a new screen operation from the user. The generated information is transferred to the display control unit 115 via the window management unit 114 or the like, and a window is displayed on the display screen 116 of the display device D1 based on this information. The image drawn in the window includes an object that is a target of screen operation by the user U1. This object is associated with a predetermined process by the functional unit A1. When the user U1 performs a screen operation on the object, the functional unit A1 is provided with operation data indicating the screen operation from the window management unit 114, and is associated with a predetermined object associated with the object that is the target of the screen operation. Execute the process.

  The functional unit A1 can display one or more windows on the display screen 116 of the display device D1 or D2. Hereinafter, a case where the functional unit A1 displays one window on the display screen 116 of the display device D1 so as to be displayed on the entire display screen 116 (in a so-called maximized state) will be described as an example. To do. Note that the window displayed by the function unit A1 may be expressed as a window corresponding to the function unit A1. The function unit A1 is realized as application software that operates on the OS based on the program described so as to operate as described above, and the hardware is realized by the CPU 101 or the like.

  The function of the function unit A2 is the same as that of the function unit A1. However, the function realized as application software may be the same as or different from the function unit A1. The functional unit A2 operates independently of the functional unit A1, and displays an image to be presented to the user U2 on the display screen 116 of the display device D2.

  When the operation data is generated by the receiving unit 111 and the duplicating unit 112, the processing control unit P1 causes the window management unit 114 to perform processing according to the state of the target window that is the target of the screen operation indicated by the operation data. It is a processing unit. One processing control unit P1 is provided for one functional unit A1 as one function of the functional unit A1 or independently of the functional unit A1. The processing control unit P1 is realized as application software that operates on the OS in the same manner as the functional unit A1.

  Specifically, when the target window is in an active state, the process control unit P1 provides the window management unit 114 with processing provided based on operation data generated by either the reception unit 111 or the duplication unit 112. To do. In addition, when the target window is in an inactive state, the process control unit P1 performs a transition process to the window management unit 114 based on a total of two pieces of operation data generated by the reception unit 111 and the duplication unit 112, respectively. And provide processing.

  Specifically, the processing control unit P1 receives a notification from the window management unit 114 that operation data has been generated when a total of two pieces of operation data are generated by the reception unit 111 and the duplication unit 112, respectively. . If the target window is in an active state, a response is made to the effect that one of the two operation data is valid. On the other hand, when the target window is in an inactive state, it responds to the notification that both the two operation data are valid.

  The process control unit P2 is provided corresponding to the function unit A2. The function of the process control unit P2 is the same as that of the process control unit P1.

  The display control unit 115 is a processing unit that generates information to be displayed in a window displayed on the display screen 116 such as the display device D1. Specifically, the display control unit 115 generates display information indicating information to be displayed in the window based on the drawing information acquired from the functional unit A1 via the window management unit 114. Then, the display control unit 115 transmits the generated display information to the display device D1 via the relay device R1. The display control unit 115 is realized as software that operates on the OS, and as hardware, it is realized by the CPU 101 or the like. The display control unit 115 corresponds to a device driver of a display device (display).

  The display screen 116 is a display screen that presents the image to the user by displaying the image. The display screen 116 is realized by, for example, a display portion of a touch panel display that is configured integrally with a touch panel as the reception unit 111.

  Processing of the multi-window display device 1 configured as described above will be described in detail below.

  FIG. 5 is an explanatory diagram of generation and duplication of the operation data 501 according to the present embodiment.

  The operation data 501 is generated by the receiving unit 111 when the receiving unit 111 receives a screen operation on the display screen 116 by the user U1. The reception unit 111 transmits the generated operation data 501 to the window management unit 114.

  The duplication unit 112 acquires the operation data 501 transmitted by the reception unit 111, generates operation data 501 </ b> A that is a copy of the acquired operation data 501, and transmits the generated operation data 501 </ b> A to the window management unit 114.

  In this way, when one screen operation is performed by the user U1, the window management unit 114 acquires two operation data having the same contents.

  FIG. 6 is a flowchart showing processing of operation data by the window management unit 114 according to the present embodiment. The flowchart of FIG. 6 is a flowchart showing processing when the window management unit 114 acquires one piece of operation data. Note that the window management unit 114 treats the operation data in the same manner regardless of whether the operation data is generated by the reception unit 111 or the duplication unit 112.

  In step S101, the window management unit 114 acquires operation data. This operation data is generated by the reception unit 111 or the duplication unit 112, and may be the operation data 501 in FIG. 5 or the operation data 501A.

  In step S102, the window management unit 114 identifies a target window that is a target of the screen operation indicated by the operation data, based on the content of the operation data.

  In step S103, the window management unit 114 notifies the processing control unit P1 corresponding to the target window identified in step S102 that operation data has been generated. This notification is acquired by the processing control unit P1 in step S201 or S205 (described later) in FIG.

  In step S104, the window management unit 114 answers the inquiry from the processing control unit P1 as to whether the target window is in an active state. The window management unit 114 answers the inquiry by notifying whether the target window is in an active state or an inactive state.

  In step S105, the window management unit 114 acquires whether the operation data is valid or invalid as a response to the notification in step S103 from the processing control unit P1. This response is made by the processing control unit P1 in step S204, S206, S211 or S213 (described later) in FIG.

  In step S106, the window management unit 114 determines whether the response acquired in step S105 is valid or invalid for the operation data. If it is valid (“valid” in step S106), the process proceeds to step S107. On the other hand, if it is invalid (“invalid” in step S106), the process proceeds to step S110.

  In step S107, the window management unit 114 determines whether the target window is in an active state or an inactive state. When it is in an active state (Yes in step S107), the process proceeds to step S108. On the other hand, when it is in an inactive state (No in step S107), the process proceeds to step S109.

  In step S108, the window management unit 114 performs a providing process of providing the operation data to the function unit A1 corresponding to the target window.

  In step S109, the window management unit 114 performs a transition process for transitioning the target window to the active state.

  In step S110, the window management unit 114 discards the operation data.

  In step S108, S109 or S110, the series of processes shown in FIG. Thereafter, the window management unit 114 waits to receive operation data from the reception unit 111 or the duplication unit 112. When new operation data is generated, step S101 is executed again.

  FIG. 7 is a flowchart showing processing by the processing control unit P1 according to the present embodiment. The flowchart of FIG. 7 shows processing by the processing control unit P1 for the notification of generation of a total of two pieces of operation data generated by the receiving unit 111 and the duplicating unit 112. In the following, the two pieces of operation data are expressed as the first or second, but may be the operation data generated by the reception unit 111 or the operation generated by the duplication unit 112, respectively. It may be data.

  In step S201, the process control unit P1 determines whether or not a notification of operation data generation has been received from the window management unit 114. If not, the process control unit P1 executes step S201 again. If operation data has been acquired, the process proceeds to step S202. That is, the process control unit P1 waits until receiving notification of operation data generation from the window management unit 114. The notification received in step S201 is a notification of generation of the first operation data received by the window management unit 114.

  The fact that the processing control unit P1 has received the notification from the window management unit 114 in step S201 means that the target window that is the target of the screen operation corresponds to the processing control unit P1.

  In step S202, the process control unit P1 inquires of the window management unit 114 whether or not the target window (that is, the window corresponding to the function unit A1) is in an active state.

  In step S203, the process control unit P1 determines whether the target window is in an active state based on the result of the inquiry in step S202. If it is in the active state (Yes in step S203), the process proceeds to step S204. On the other hand, when it is in an inactive state (No in step S203), the process proceeds to step S211.

  In step S204, the processing control unit P1 responds to the window management unit 114 when the first operation data notified in step S201 is validated.

  In step S <b> 205, the processing control unit P <b> 1 waits until receiving notification of operation data generation from the window management unit 114. The notification received in step S205 is a generation notification for the second operation data received by the window management unit 114.

  In step S206, the process control unit P1 responds to the window management unit 114 when the second operation data notified in step S205 is invalidated. With this response, the processing control unit P1 causes the window management unit 114 to discard the second operation data.

  In step S211, the processing control unit P1 responds to the window management unit 114 when the first operation data notified in step S201 is validated.

  In step S <b> 212, the processing control unit P <b> 1 waits until receiving notification of operation data generation from the window management unit 114. The notification received in step S212 is a notification of generation of the second operation data received by the window management unit 114.

  In step S213, the processing control unit P1 responds to the window management unit 114 when the second operation data notified in step S201 is validated.

  As described above, when the target window is in the active state, the process control unit P1 validates one of the total two pieces of operation data generated by the reception unit 111 and the duplication unit 112 and invalidates the other. To respond (processing of steps S204 to S206).

  On the other hand, when the target window is in an inactive state, the process control unit P1 responds to validate both of the total two pieces of operation data generated by the receiving unit 111 and the duplicating unit 112 (step S1). Processing in S211 to S213). In the above description, the case where the first one of the notifications of the generation of two pieces of operation data is valid and the second one is invalid is described. The same result can be obtained by invalidating the one received and validating the second one.

  FIG. 8 is a first sequence diagram showing a processing flow in the multi-window display device 1 according to the present embodiment. The sequence diagram shown in FIG. 8 shows a series of processing performed by the multi-window display device 1 when the user U1 performs a screen operation on an active window. In addition, about the content already demonstrated so far, the same code | symbol may be attached | subjected and detailed description may be abbreviate | omitted.

  When the user U1 performs a screen operation on the display screen 116, the reception unit 111 receives the screen operation (step S301), and generates operation data 501 indicating the screen operation (step S302). The reception unit 111 transmits the generated operation data 501 to the duplication unit 112.

  The duplication unit 112 generates operation data 501A that is a copy of the operation data 501 generated by the reception unit 111 (step S303), and transmits the operation data 501A to the window management unit 114.

  The window management unit 114 acquires the operation data 501 generated by the reception unit 111 and the operation data 501A generated by the duplication unit 112. Then, the processing described in detail in FIGS. 6 and 7 is performed by the window management unit 114 and the processing control unit P1. As a result, the operation data 501 is provided to the function unit A1, and the operation data 501A is discarded.

  If the operation data 501A is not discarded (step S110) by the window management unit 114 and the operation data 501A is provided to the function unit A1, the function unit A1 performs two operations based on a single screen operation by the user. Since the operation data of the operation is provided, the result is not intended by the user. Specifically, the user U1 operates the button only once for the button displayed on the display screen 116, and operates the button displayed thereafter. Processing of the functional unit A1 is performed.

  According to the present embodiment, the window management unit 114 discards the operation data 501A, so that the unintended result can be prevented in advance.

  FIG. 9 is a second sequence diagram showing the flow of processing in the multi-window display device 1 according to the present embodiment. The sequence diagram shown in FIG. 9 shows a series of processes performed by the multi-window display device 1 when the user U1 performs a screen operation on a window in an inactive state. In addition, about the content already demonstrated so far, the same code | symbol may be attached | subjected and detailed description may be abbreviate | omitted.

  The operation data 501 and the operation data 501A are transmitted to the window management unit 114 when the user U1 performs a screen operation on the display screen 116, as in the case of FIG.

  The window management unit 114 acquires the operation data 501 generated by the reception unit 111 and the operation data 501A generated by the duplication unit 112. Then, the processing described in detail in FIGS. 6 and 7 is performed by the window management unit 114 and the processing control unit P1. As a result, the target window is activated based on the operation data 501 and the operation data 501A is provided to the functional unit A1.

  Assuming that the operation data 501 is not replicated (generation of the operation data 501A) by the duplication unit 112, the window management unit 114 only performs a transition process for transitioning the target window to the active state based on the operation data 501. Yes, the operation data is not provided to the functional unit A1, and the result is not intended by the user.

  According to the present embodiment, the duplication unit 112 duplicates the operation data 501 so that the above unintended result can be prevented in advance.

  FIG. 10 is an explanatory diagram showing a user's screen operation on the multi-window display device according to the present embodiment. 10A to 10C show images displayed on the display screens 116 of the display devices D1 and D2, respectively. In each image, four buttons (A to D) are displayed, and a case where the users U1 and U2 perform a screen operation on these buttons is shown. Each button may be a character, a character string, a figure or a symbol, or a combination thereof. These are targets for screen operations by the user and correspond to objects. Note that the frame of the display screen 116 drawn with a double line indicates that the window of the display screen 116 is in an active state.

  First, the process of the multi-window display device based on the screen operation by the users U1 and U2 in the technology that is a comparative example of the technology in the present embodiment will be described. In the technology as a comparative example, the process proceeds in the order of (a), (b), and (c) in FIG. 10 based on the screen operations of the users U1 and U2.

  FIG. 10A shows a state immediately after the user U1 performs a screen operation on the button D in the window of the display screen 116 of the display device D1. In this state, the window of the display device D1 is managed as being in an active state.

  Next, when the user U2 operates the button D in the window of the display screen 116 of the display device D2, the window of the display device D2 changes to an active state, and the window of the display device D1 changes to an inactive state. At this time, the operation data generated based on the screen operation by the user is used for the transition process by the window management unit, and the operation data is not provided to the function unit A1 corresponding to the window. At this time, the user U2 recognizes that although the user has operated the button D on the screen, the process corresponding to the button D is not performed ((b) of FIG. 10).

  Next, when the user U2 operates the button D in the window of the display screen 116 of the display device D1 again, since this screen operation is for the active window, the operation data generated based on the screen operation is generated. Is provided to the functional unit A1. Accordingly, the user U2 recognizes that the process corresponding to the button D has been performed ((c) in FIG. 10).

  As described above, in the technique as the comparative example, the screen operation by the user U2 is required twice for the inactive window.

  Next, processing by the multi-window display device 1 in the present embodiment will be described using FIG. According to the multi-window display device 1, processing proceeds in the order of (a) and (c) in FIG. 10 based on the screen operations of the users U1 and U2.

  According to the multi-window display device 1 of the present embodiment, when a screen operation is performed once for a window by a user, the window management unit 114 and the processing control unit P1 perform the screen operation on the active window. One of the two pieces of operation data copied is invalidated and handled as one piece of operation data. On the other hand, if it is a screen operation for an inactive window, it is handled as two pieces of operation data by the processing by the window management unit 114 and the processing control unit P1, and the first is used for window transition. Is the original operation data of the operation of the button D.

  Therefore, when the user U2 operates the button D of the display device D2 in the state shown in FIG. 10A, the screen operation is provided as a single operation to the function unit A1 (FIG. 10C). ). In this way, it is possible to avoid a result unintended by the user as described in the comparative example. As a result, the multi-window display device 1 can appropriately accept screen operations on a plurality of display screens.

  In addition, in FIG. 10, although the example of four buttons used as the screen operation by the user was shown as the contents of the window displayed on the display device D1 or the like, what kind of contents are the window? Also good. Other examples will be described below.

  FIG. 11 is an explanatory diagram showing another example of the contents of the window according to the present embodiment. FIG. 11 shows one display screen for explanation. There are a plurality of display devices that display such screens, and a situation in which a plurality of users are using them in parallel is assumed.

  The window shown in FIG. 11 includes a character input space 1101 and a software keyboard 1102. The software keyboard is a keyboard that is displayed as an image. When the user operates the screen where a character is written, the character is input into the character input space 1101. In this case, each key of the software keyboard 1102 corresponds to one button in FIG. When the user operates (eg, clicks) one of the software keyboards 1102 in the inactive window, two operation data are transmitted to the information processing apparatus, and one of the two operation data is the window. It is used for switching, and the other is used for character input.

  Also, when the user operates any one of the software keyboards 1102 in the active window, two pieces of operation data are transmitted to the information processing apparatus. One of the two pieces of operation data is a character input. And the other is invalidated. These processes are the same as those shown in FIG.

  In this way, even when a plurality of users input characters in parallel on a plurality of display screens, information is input to the information processing device without being aware of the window state, that is, the active state or the inactive state. It can be performed.

  As described above, in the multi-window display device according to the present embodiment, one screen operation by the user is performed once for an active window, and twice for an inactive window. Accept and process as an operation. By doing in this way, one operation data can be appropriately provided to the application corresponding to each window of an active state and an inactive state by one screen operation by a user. At this time, it is possible to prevent the occurrence of an unintended result of the user such as two pieces of operation data being provided to the application or none of the operation data being provided. Therefore, the multi-window display device can appropriately accept screen operations performed by the user on a plurality of display screens.

  In addition, the processing control unit receives a notification that the operation data has been generated, and responds to the notification whether the operation data is valid or invalid. The process can be controlled. In this way, the multi-window display device is configured more specifically.

  Further, the process control unit explicitly discards operation data that is not processed by the window management unit. By discarding data that is not used for processing, there are effects of reducing the processing load of the multi-window display device and reducing the resources used. In this way, the multi-window display device is configured more specifically.

  In addition, the multi-window display device can provide a specific information processing result to the user when the functional unit performs processing based on the screen operation performed on the window by the user.

  In addition, the multi-window display device can accept an operation from a different user depending on each of the plurality of display devices.

In addition, since the multi-window display device can be configured so that the display devices are arranged in a plurality of locations via a network, the degree of freedom of arrangement of the multi-window display device is improved. In addition, the multi-window display device can be configured inexpensively and efficiently using a relay unit having a simple function of copying the contents of operation data.
(Modification of the embodiment)
In this modification, another configuration such as a multi-window display device that appropriately accepts screen operations on a plurality of display screens will be described. In particular, in this modification, a multi-window display device having an effect of suppressing the amount of data flowing through the network will be described.

  FIG. 12 is a block diagram showing a functional configuration of a multi-window display device 1A according to a modification of the embodiment. As shown in FIG. 12, the multi-window display device 1A has the same configuration as the multi-window display device 1 in the embodiment, but the position of the duplicating unit is different.

  Specifically, in the multi-window display device 1, the relay devices R1 and R2 include the duplication unit 112, but in the multi-window display device 1A, the information processing device 10A includes the duplication unit 112A. Further, the relay apparatuses R1A and R2A do not include a duplication unit.

  The duplication unit 112A acquires a communication frame for the network 40 in which operation data is encapsulated from the relay device R1A and the like, and generates a duplication of the communication frame. Then, the window management unit 114 acquires the communication frame acquired by the duplication unit 112A from the relay device R1A and the like, and the duplication communication frame generated by the duplication unit 112A. The subsequent processing is the same as that in the embodiment.

  With this configuration, the amount of data flowing through the network 40 can be suppressed. Specifically, in the multi-window display device 1 according to the embodiment, both the operation data generated by the reception unit 111 and the operation data generated by the duplication unit 112 by duplication flow through the network 40. In contrast, in the multi-window display device 1 </ b> A according to the present modification, only the operation data generated by the receiving unit 111 flows through the network 40.

  In this way, the multi-window display device 1A can appropriately accept screen operations by a user on a plurality of display screens as in the multi-window display device 1 while suppressing the amount of data flowing through the network 40 in half.

  In each of the above embodiments, each component may be configured by dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  As described above, the multi-window display device and the like of the present invention have been described based on the embodiment. However, the present invention is not limited to this embodiment. Unless it deviates from the meaning of this invention, the form which carried out the various deformation | transformation which those skilled in the art can think to this embodiment, and the structure constructed | assembled combining the component in different embodiment is also contained in the scope of the present invention. .

  INDUSTRIAL APPLICABILITY The present invention can be used for a multi-window display device that appropriately receives screen operations by a user on a plurality of display screens. Specifically, the present invention can be used for a PC having a plurality of display screens, a mobile phone terminal, a smartphone, a tablet, and the like.

1, 1A multi-window display device 10, 10A information processing device 40 network 101, 201 CPU
102, 202 Main memory 103, 203 Storage 104, 204 NIF
205 Operation input IF
206 Image output IF
111 reception unit 112, 112A replication unit 113, 113A conversion unit 114 window management unit 115 display control unit 116 display screen 1101 character input space 1102 software keyboard A1, A2 function unit C1, C2 communication line D1, D2 display device P1, P2 processing Control unit R1, R1A, R2, R2A Relay device U1, U2 User

Claims (12)

A multi-window display device for displaying a plurality of windows on one or more display screens,
An accepting unit that accepts an operation by a user for any of the plurality of windows and generates operation data indicating the operation;
A replication unit that generates a copy of the operation data generated by the reception unit;
A plurality of functional units corresponding to each of the plurality of windows;
(A) a first process for transitioning the window to an active state based on operation data indicating an operation on the inactive window;
(B) a window management unit for performing a second process of providing the operation data to a function unit corresponding to the window among the plurality of function units based on operation data indicating an operation on the window in the active state When,
When the operation data is generated by the receiving unit and the duplicating unit, a target window that is an operation target indicated by the operation data is
(I) In the active state, based on the operation data generated by one of the reception unit and the duplication unit, the window management unit performs the second process,
(Ii) In the inactive state, based on the operation data generated by each of the receiving unit and the duplicating unit, the window management unit performs the first process and then performs the second process. A multi-window display device comprising a processing control unit. The window manager is
Every time the operation data generated by the reception unit or the duplication unit is received, the processing control unit is notified of the generation of the operation data,
As a response to the notification, when receiving a response that the operation data is valid, the first process or the second process is performed according to the state of the target window,
The processing control unit
The notification of generation of the two operation data generated by each of the reception unit and the duplication unit is acquired, and the target window is
(I) If it is in the active state, a response is made valid for one of the two notifications of operation data generation,
The multi-window display device according to claim 1, wherein, in the inactive state, a response that is valid for both notifications of the generation of the two operation data is made. The processing control unit
The window management unit causes the window management unit to discard one of the operation data generated by the other of the reception unit and the duplication unit when the target window is in an active state. The multi-window display device described. The operation is an object included in any of the plurality of windows, and is associated with a predetermined process by a function unit corresponding to the window including the object among the plurality of function units. Is an operation on
Each of the plurality of functional units is
4. When the operation data is provided from the window management unit, a predetermined process associated with an object that is an operation target indicated by the provided operation data is executed. 2. A multi-window display device according to item 1. The multi-window display device includes a plurality of display devices,
Each of the plurality of display devices has one display screen of the one or more display screens, and displays one of the plurality of windows on the one display screen in a maximized state. The multi-window display device according to claim 1. The multi-window display device further includes:
A relay unit having the duplicating unit, the relay unit transmitting the operation data generated by the receiving unit and the duplicating unit via a network;
The window manager is
The multi-window display device according to any one of claims 1 to 5, wherein the operation data is acquired via the network. The multi-window display device further includes:
A relay unit that transmits the operation data generated by the reception unit via a network;
The window management unit further includes the duplication unit,
The multi-window display device according to claim 1, wherein the window management unit acquires the operation data via the network and acquires a copy of the operation data generated by the replication unit. . A control method of a multi-window display device for displaying a plurality of windows on one or more display screens,
The multi-window display device
A plurality of functional units corresponding to each of the plurality of windows;
The control method is:
An accepting step of accepting an operation by a user on any of the plurality of windows and generating operation data indicating the operation;
A replication step for generating a copy of the operation data generated in the reception step;
(A) a first process for transitioning the window to an active state based on operation data indicating an operation on the inactive window;
(B) a window management step for performing a second process of providing the operation data to a function unit corresponding to the window among the plurality of function units based on operation data indicating an operation on the window in the active state When,
When the operation data is generated by the reception step and the duplication step, a target window that is a target of the operation indicated by the operation data is
(I) When in the active state, based on the operation data generated by one of the reception step and the duplication step, the second process is performed in the window management step,
(Ii) In the inactive state, the second process is performed after the first process is performed in the window management step based on the operation data generated in each of the reception step and the duplication step. A control method including a process control step. A multi-window display device for displaying a plurality of windows on a display screen , the relay device in a multi-window display device comprising a display device, a relay device, and an information processing device ,
The information processing apparatus is communicably connected to the relay apparatus via a network, and an OS (Operating System) that operates independently of the relay apparatus is running.
A duplication unit that obtains operation data indicating a user's operation on any of the plurality of windows and generates a copy of the acquired operation data,
The duplication unit further includes:
The generated operation data corresponds to each of the plurality of windows according to the state by a window management unit that manages whether the state of each of the plurality of windows is an active state or an inactive state. A relay device that transmits to the information processing device via the network so as to be provided to a plurality of functional units. A multi-window display device for displaying a plurality of windows on a display screen, the method for controlling the relay device in a multi-window display device comprising a display device, a relay device, and an information processing device ,
The information processing apparatus is communicably connected to the relay apparatus via a network, and an OS (Operating System) that operates independently of the relay apparatus is running.
An acquisition step of acquiring operation data indicating an operation by a user on any of the plurality of windows;
A replication step for generating a copy of the operation data acquired in the acquisition step,
In the duplication step,
The generated operation data corresponds to each of the plurality of windows according to the state by a window management step for managing whether the state of each of the plurality of windows is an active state or an inactive state. A control method of transmitting to the information processing apparatus via the network so as to be provided to a plurality of functional units. A control method for an information processing device in a multi-window display device for displaying a plurality of windows on one or more display screens,
The multi-window display device includes a display device, a relay device, and the information processing device,
The display device
A reception unit that generates operation data indicating an operation by a user on any of the plurality of windows;
The relay device is
A replication unit that generates a copy of the operation data generated by the reception unit;
The information processing apparatus includes:
A plurality of functional units corresponding to each of the plurality of windows;
(A) a first process for transitioning the window to an active state based on operation data indicating an operation on the inactive window;
(B) a window management unit for performing a second process of providing the operation data to a function unit corresponding to the window among the plurality of function units based on operation data indicating an operation on the window in the active state And
The control method is:
When the operation data is generated by the receiving unit and the duplicating unit, a target window that is an operation target indicated by the operation data is
(I) In the active state, based on the operation data generated by one of the reception unit and the duplication unit, the window management unit performs the second process,
(Ii) In the inactive state, based on the operation data generated by each of the receiving unit and the duplicating unit, the window management unit performs the first process and then performs the second process. A control method including a process control step. A program for causing a computer to execute the control method according to claim 11.

Images