AU2013389985B2

AU2013389985B2 - User interface elements for multiple displays

Info

Publication number: AU2013389985B2
Application number: AU2013389985A
Authority: AU
Inventors: Zoran Dimov; Steve Frederickson; Keith Kelly; Henri-Charles MACHALANI; Christopher Swan; Marina Dukhon TAYLOR; Graham WARDLE
Original assignee: Microsoft Technology Licensing LLC
Current assignee: Microsoft Technology Licensing LLC
Priority date: 2013-05-23
Filing date: 2013-09-19
Publication date: 2019-05-02
Anticipated expiration: 2033-09-19
Also published as: WO2014189536A1; RU2646376C2; CN105378644A; RU2015150029A; BR112015028468A2; MX2015016067A; CA2911489A1; JP6273354B2; EP3000031A1; CN105378644B; US20140351722A1; AU2013389985A1; JP2016519383A; KR20160013162A

Abstract

Described herein are techniques for a computing device executing a windowing system that automatically maintains a tiled arrangement of application windows on a first display and on a second display. A user interface element has indicia of applications that can be used to open the applications. Responsive to a first user input the user interface element is displayed on the first display, and while the user interface element is displayed on the first display, the windowing system maintains two or more of the application windows in a tiled arrangement on the second display. The user interface element may be part of a system user interface and may be implemented in a variety of ways. For example, it may be a full-screen set of application representations, possibly user selected, or a list of recently used applications, or a list of open applications, etc.

Description

USER INTERFACE ELEMENTS FOR MULTIPLE DISPLAYS

2013389985 19 Mar 2019

BACKGROUND [001] In the field of computing, windowing environments have been used to provide applications with windows. Applications or programs executing on a computing device may have corresponding application windows through which a user interacts with the applications. In addition, it has been known how to concurrently display windows on multiple displays locally connected to one computing device. Typically, an operating system of a multi-display computing device handles details for managing multiple displays and may provide different display modes such as display mirroring or display concatenation.

[002] Usually, windowing systems or environments include systemic user interface elements that a user can interact with to control and manage windows. For example, OS X (tm) has an app launcher tool, the Android (tm) operating system provides a default

Launcher that is used to start applications and access system settings, and various versions of Microsoft Windows (tm) have provided a Start element, fast-switch lists, and other elements. In addition, there have been many third-party applications that have provided similar functionality.

[003] To date, such user interface elements for controlling applications have been unable to work efficiently and intuitively in the presence of multiple displays connected to a same device. Add-on user interface programs for application management often lack logic, perhaps at the kernel level, that might be helpful for smooth and consistent use across multiple displays. System-provided application managers such as those mentioned above have not been designed for a multi-display user experience and therefore fall short of providing the behavior a user might expect when using multiple displays.

[004] Techniques related to providing application management user interface elements for computing devices with multiple displays are discussed below.

[004a] It is desired to address or ameliorate one or more disadvantages or limitations associated with the prior art, or to at least provide a useful alternative.

SUMMARY [004b] According to some embodiments, the present invention provides a method of managing windows for a computing device, the method comprising:

2013389985 19 Mar 2019 executing a windowing system that automatically maintains windows for applications executing on the computing device, the windowing system enabling display of the application windows on a first display connected with the computing device and on a second display connected with the computing device, wherein the windowing system tracks which of the displays currently has input focus;

executing an application launcher on the computing device, the application launcher comprising indications of respective applications on the computing device, wherein the application launcher is concurrently activatable responsive to first user inputs not associated with the first display or the second display, wherein second user inputs directed to the indications are responded to by launching correspondingly selected applications, wherein the application launcher is displayed on only one display at a time, and wherein when the application launcher is displayed on either of the displays, user input directed to whichever display is not displaying the launcher does not cause the launcher to be undisplayed;

responding to each first user input that activates the launcher when the application launcher is not displayed on the first display and only the first display currently has the input focus by: (i) determining if the application launcher is displayed on the second display and if so determined undisplaying the application launcher from the second display, and (ii) displaying the application launcher on the first display;

responding to each first user input that activates the application launcher when the application launcher is not displayed on the first display and when only the second display currently has the input focus by: (i) determining if the application launcher is displayed on the first display and if so determined undisplaying the application launcher from the first display, and (ii) displaying the application launcher on the second display;

responding to each second user input that is associated only with whichever of the displays the application launcher is currently displayed on by: displaying a correspondingly selected application window on whichever of the displays the application launcher is currently displayed on, and undisplaying the application launcher; and responding to each second user input that is associated only with whichever of the displays the application launcher is not currently displayed on by: displaying a correspondingly selected application window on whichever of the displays the application launcher is not currently displayed on, and after displaying the correspondingly selected application window, maintaining display of the application launcher on whichever of the displays the application launcher is currently displayed on.

2013389985 19 Mar 2019 [004c] According to some embodiments, the present invention provides a computing device comprising a processor, storage, and a first display, the storage storing a windowing system that when executed by the processor displays application windows of applications executing by the processor, the computing device further comprising:

the windowing system, when executing, being capable of displaying application windows on the first display and on a second display when a second display is locally connected with the computing device, the windowing system further comprising a user interface element, wherein the windowing system displays application windows on either display;

the user interface element, wherein the user interface element is simultaneously activatable on the first display and activatable on the second display, and while so simultaneously activatable the user interface element can only be displayed on a single display at a time, wherein each activation of the user interface element displays the user interface element on, and according to, whichever of the displays is currently exclusively designated as having user input focus, wherein whenever the user interface element is displayed on the first display while a user input not associated with either display directing an activation of the user interface element to the second display is detected, the user input is handled by displaying the user interface element on the second display and by undisplaying the user interface element from the first the display, wherein whenever the user interface element is displayed on the second display while a user input not associated with either display directing an activation of the user interface element to the first display is detected, the user input is handled by displaying the user interface element on the second display and by undisplaying the user interface element from the first the display, and;

wherein whenever the user interface element is displayed on either of the displays, the user interface element remains displayed on the either of the displays during and after multitasking user input is directed to the other display that is not displaying the user interface element;

the user interface element is configured to respond to first user interactions with graphic representations of applications in the user interface element by displaying corresponding application windows on, and according to, whichever display is currently displaying the user interface element, and by undisplaying the user interface element; and

2A

2013389985 19 Mar 2019 the user interface element is further configured to respond to second user interactions with representations of applications in the user interface element by displaying corresponding application windows on, and according to, whichever display is not currently displaying the user interface element, and by continuing to display the user interface element after the corresponding application windows are displayed.

[004d] According to some embodiments, the present invention provides one or more computer storage devices storing information to enable a computing device to perform a process, the process comprising:

executing applications having respective application windows displayable by a windowing system executing on the computing device;

displaying a first application window on a first display connected with the computing device while displaying a second application window on a second display connected with the computing device, each application window substantially fully occupying the display on which it is displayed;

enabling a user to direct input to the first display to interact with the first application window while concurrently enabling the user to direct input to the second display to interact with the second application window;

providing an application-list comprising a set of application representations that when displayed can be interacted with by the user to open application windows of respectively represented applications, the application-list able to be displayed, at any given time, responsive to user input not associated with either display, on either display according to which display currently has user input focus associated therewith, wherein display of the application-list responsive to a user input comprises determining if the application-list is being displayed on the display that does not have the user input focus, and when so determined undisplaying the application-list from the display that does not have the user input focus;

responsive to a user input not associated with either display and received while the user focus is associated with the first display, displaying the application-list over the first application window and undisplaying the application-list from the second display, and responsive to a user input not associated with either display and received while the user focus is associated with the second display, displaying the application-list over the second application window and undisplaying the application-list from the first display; and while the application-list is displayed on one of the displays, responsive to a user input selecting one of the application representations for display on the one of the

2B

2013389985 19 Mar 2019 displays: undisplaying the application-list and displaying a third application window corresponding to the selected application representation on the one of the displays while maintaining the displaying of the first or second application window on the display other than the one of the displays; and while the application-list is displayed on the one of the displays, responsive to a user input selecting one of the application representations for display on the other of the displays: retaining display of the application-list on the one of the displays after displaying an application window corresponding and responsive to the selected one of the application representations.

[005] (Deleted) [006] (Deleted) [007] (Deleted)

BRIEF DESCRIPTION OF THE DRAWINGS [008] Some embodiments of the present invention are hereinafter described, by way of example only, with reference to the accompanying drawings, in which:

[009] Figure 1 shows a computing device having a first display and a second display. [010] Figure 2 shows example window layouts provided by a tiled windowing system. [011] Figure 3 shows window operations that may be performed either manually or as a result of system events.

[012] Figure 4 shows a graphical user interface provided by or implemented on top of the tiled windowing system.

[013] Figure 5 shows an example of a full-screen application launcher.

[014] Figure 6 shows an example of a user interface element that is activatable on the first display and the second display.

[015] Figure 7 shows a multi-display embodiment where a graphic representation may be moved across displays by the user.

[016] Figure 8 shows an embodiment implementing the application launcher on multiple displays.

[017] Figure 9 shows the application launcher displayed on the second display. [018] Figure 10 shows details of a computing device.

DETAILED DESCRIPTION [0018a] Described herein are techniques for a computing device executing a windowing system that automatically maintains a tiled arrangement of application windows on a first display and on a second display. A user interface element has indicia of applications that

2C

2013389985 19 Mar 2019 can be used to open the applications. Responsive to a first user input the user interface element is displayed on the first display, and while the user interface element is displayed on the first display, the windowing system maintains two or more of the application windows in a tiled arrangement on the second display. The user interface element may be 5 part of a system user interface and may be implemented in a variety of ways. For example, it may be a full-screen set of application representations, possibly user-selected, or a list of recently used applications, or a list of open applications, etc.

[019] Embodiments described below relate to providing application management user interface elements for computing devices with multiple displays. Discussion will begin

2D

WO 2014/189536

PCT/US2013/060497 with an overview of windowing systems for multiple displays. Tiled or non-occluding windowing systems will be discussed next. User interface elements for application window management and manipulation will be discussed next, followed by explanation of how such user interface elements can be integrated into a multi-display setting.

[020] Figure 1 shows a computing device 100 having an operating system 102 managing application processes 104 and displaying corresponding windows 106 on a first display 108A and a second display 108B, which may be connected to a display adapter or graphics ports of the computing device 100 (as used herein, first display and second display refer to any arbitrary displays and do not imply an order). For discussion, reference numeral 108 will indicate a display that may be either of the displays. In addition, all discussion herein of two displays is readily extendable to three or more displays; any description involving two displays implicitly applies to possible additional displays.

[021] The computing device 100 may be any of a variety of types which are described later with reference to Figure 10. The computing device 100 may be assumed to have at least storage and a processor for, respectively, storing and executing the operating system 102 and the application processes 104. Although there may be windowing systems with individual windows that represent multiple processes, and also processes without windows, for ease of discussion application processes 104 are assumed herein to each have at least a corresponding window 106 (also to be referred to as application windows). Processes 104 are well known operating system objects that are discretely managed units of execution, typically having a process identifier, memory at least partially managed by the operating system (heap and/or stack memory), and may be managed using a process table of process identifiers that is maintained by the operating system 102.

[022] A windowing system 110 may be partly integrated with or closely coupled with the operating system 102. For purposes herein, distinctions between these components are not significant; an operating system itself may be considered to be a windowing system. The windowing system 110 may have functionality that is known in the computing arts, such as handling of input events (e.g., touches/strokes, clicks, keypresses, mouse drags, etc.) inputted by a user with various types of input devices. The windowing system 110 also manages the application windows 106 by handling related events or user interactions such as closing windows, moving windows, resizing windows, directing input to windows, rendering multiple concurrent windows, and others. The windowing system 110 may also provide a background and system controls (user interface elements) not specific to an application, which will be addressed further below.

WO 2014/189536

PCT/US2013/060497 [023] Figure 2 shows example window layouts provided by a tiled windowing system. A tiled windowing system may manage shared display space (including multiple displays) for application windows such that the application windows substantially maximize occupancy of display space, in some cases without allowing application windows to occlude each other. In some cases, occlusion might be temporarily employed, for example to preview manual window rearrangements, to show windows being dragged or other transient conditions, to preview an expected operation, to display a dialog, to display systemic user interface elements, etc. In addition, substantially maximizing occupancy of a display refers to sizing application windows to cover display space up to or near the edges of a display, perhaps with some space allowed for system user interface elements such as resizing dividers, a clock, aesthetic margins, small gaps, information banners, and so forth.

[024] Some tiled windowing systems may omit from windows traditional window elements such as borders, title bars, menu bars, and other elements. That is to say, application windows may have immersive qualities, for instance they may have minimal or no window adornments and may have an appearance typically associated with the full screen mode of many software applications. Application windows may have such appearance even in cases where multiple application windows are displayed on a same display.

[025] Returning to Figure 2, the first example layout 120 shows a first application window 122 substantially occupying the display 108 or a managed display region 124 managed by a tiled windowing system.

[026] The second example layout 126 reflects the addition of a second application window 128, whether automatically or interactively inserted; the tiled windowing system automatically manages the window layout to cause the currently displayed application windows 122, 128 to somewhat maximize occupancy of the display 108. The tiled windowing system (window manager) may insert between windows a divider 130 that can be moved by a user to resize the application windows adjacent to the divider 130 while maintaining a tiled arrangement.

[027] The third example layout 132 shows a third application 134 having been inserted. With tiled layout management, a user may only needs to designate the third application window 134 to be inserted and/or possibly designate a slot or location for inserting the third application window 134; the tiled window manager may automatically resize the

WO 2014/189536

PCT/US2013/060497 displayed application windows or take other measures to accommodate the new application window.

[028] The fourth and fifth example layouts 136, 138 show other divisions of screen real estate that may be used. For ease of discussion, examples discussed below show tiling using only a single horizontal row of application windows, however, any arbitrary rectilinear arrangement may be used, possibly with asymmetries.

[029] Figure 3 shows window operations that may be performed either manually or as a result of system events. Given an initial window arrangement on the display 108, a tiled windowing system 140 may close application A resulting in application B being automatically resized to occupy the display 108. An application might be closed manually or might be closed due a system event such as an error. When application C is inserted into the initial layout applications A and B are automatically resized by the tiled windowing system 140 to accommodate the newly displayed application window of application C. A resize operation may be performed, for example by user manipulation of a divider 130, to change the apportionment of display space for the application windows. If a currently displayed application or another application enters a fullscreen mode or is resized to substantially fully (exclusively) occupy the display 108, then the other application windows are evicted from the display 108. The user may select an application (e.g., application C) to replace any other applications displayed.

[030] Figure 4 shows a graphical user interface 180 provided by or implemented on top of the tiled windowing system 140. The elements and features of the graphical user interface 180 are only examples and other types of features are contemplated and equally applicable to application windows and application groups (for example, a search feature might return both applications and application group objects). In addition, at times some or all of the user interface elements of the graphical user interface 180 may not be displayed or are only displayed when requested per input from a user.

[031] The tiled windowing system 140 may have various features or facilities that allow a user to manage applications on the computing device. Such features, which are sometimes referred to herein as user interface elements, or system elements, might include a recent applications module 182, an active applications module 184, and/or a favorites module 186.

[032] These modules cooperate with the windowing system (or are a part thereof) to track the semantically relevant information. When applications are opened or used the windowing system might populate a recent-applications list 188 with indicia of recently

WO 2014/189536

PCT/US2013/060497 used applications. The windowing system might similarly populate or provide an activeapplications list 190, which might include applications currently displayed on any connected monitors and/or applications that are executing or suspended but are not currently displayed. Similarly, a user might maintain a favorite-applications list 192. [033] These lists are used by the graphical user interface 180 to display corresponding user interface elements 194, 196, 198 that can be invoked and used by the user to activate applications or application groups, as the case may be. In some embodiments, the user interface elements 194, 196, 198 may be persistently displayed, and in other embodiments they are displayed only when activated by a user input such as after pressing and releasing a hardware or software button, or while a hot key is depressed, or after inputting a touch gesture, etc. Some of the user interface elements 194, 196, 198 may be undisplayed when they are used to open an application, or when display-sustaining input ends, or when a user invokes a close command.

[034] The system user interface elements 194, 196, 198 in Figure 4 are only examples and other bases for providing lists of applications may be used. For example, an all applications user interface element may be provided to show user-activatable graphic representations of all applications formally installed on the computing device. Any of the above-discussed types of user interface elements may also be implemented as full-screen elements and may, for example, be managed by the windowing system as special types of application windows.

[035] Figure 5 shows an example of a full-screen application launcher 220 (another type of system user interface element, sometimes referred to as a launcher, switcher, graphical user shell, etc.). The full-screen application launcher 220 is another means for allowing a user to interactively manage applications. The application launcher 220 may be user-configurable, for instance having user-selected or user-arranged application representations 222 representing corresponding applications. As used herein, the term launch refers to both initially opening an application (e.g., starting a new process) as well as well as opening (displaying) the already-running (possibly dormant) application window for an application.

[036] In one embodiment, an application representation 222 may display dynamically updated content received, for example, from other applications, system services, or from network-based resources. Such live updating may occur even when an application representation's application is not open. The app launcher may be implemented as a scrollable surface, and may also include dashboard-like features such as a clock, a logout

WO 2014/189536

PCT/US2013/060497 mechanism, network status information, system or application notifications, and so forth. At times, as discussed below, the application launcher 220 is not displayed until requested by a user.

[037] Application representations 222 may be interactively rearranged, removed, added, perhaps resized, configured with settings, updated by applications with dynamic content, etc. Application representations 222 may also be activated or selected by a user to open a corresponding application window. In some embodiments, the application launcher is undisplayed when a graphic application representation 222 is actuated to open an application window; the opened application window may supplant the application launcher on the display where the application launcher was used.

[038] Figure 6 shows an example of the user interface element 194 that is activatable on the first display 108A and on the second display 108B. For brevity, only multi-display behavior of the user interface element 194 (corresponding to the recent-applications list 188) will be described. However, this multi-display behavior can be readily implemented in any other type of system user interface element.

[039] The user interface element 194 includes graphic application representations 222 of corresponding applications. An application representation may be displayed as a thumbnail image of the corresponding application (live or previously captured) or a graphic symbol representing the application. The application representation 222 may be interactively selected, for instance by a click or touch, or dragged from the user interface element 194 by the user. When the application representation 222 is activated or released from a drag the corresponding application window is opened. Various visual effects may be used. For instance, a rendering of the prior or emerging layout may be animated as enlarging to occupy the display before switching to live activation of the windows. In some embodiments, the application representation 222 may represent a group of applications and can be used to open those applications all at once. Note that a pointer 224 may or may not be displayed, and in this description the pointer 224 may also depict not a graphic pointer but rather an input point moved or inputted by a user.

[040] Regarding the multi-display behavior of the user interface element 194, the user interface element may, in one embodiment, be opened by the user on either the first display 108A or on the second display 108B, and optionally may not be able to be displayed simultaneously on both displays, as shown in the four sequential stages of Figure 6. In addition, when the user interface element 194 is displayed, the application windows on the other display (the display not displaying the user interface element 194)

WO 2014/189536

PCT/US2013/060497 are maintained; they are not affected. Thus, one display may have a system user interface element while the other display has one or more application windows and interaction with one need not affect the other.

[041] Sequentially, the user interface element 194 is initially not displayed but is available to be activated by the user on either the first display 108A or the second display 108B. As shown in the first quarter of Figure 6, the user interface element 194 is displayed per user input and shows application representations 222 corresponding to recently used applications. Note that at this time the same application representations will be displayed regardless of which display the user interface element 194 is displayed on. However, when the application representation 222 for application app6 is activated by the user, the user interface element 194 is undisplayed and the tiled windowing system opens and displays the application window 106 for application app6 on the same display where the user interface element 194 was displayed; on the first display 108A. Because application app6 is now active, it is removed from the recent-applications list 188 (or potentially the source monitor, in the case where it was visible on the other display) and application appl3, which was replaced on the first display 108A by application app6, is added to the recent-applications list 188 (at this time, if the user interface element 194 were opened on the other display it would include appl3 but not app6). Note that the opening of the application window 106 for application app6 does not affect the application windows displayed on the first display 108B, which continue to be displayed and available for interactive multitasking.

[042] When the user activates the user interface element 194 on the second display 108B, as shown in the second quarter of Figure 6, the user selects the application representation for application app 13, which in turn alters the content of the second display 108B without affecting the content of the first display 108 A, and which also updates the recentapplications list 188 and displays the application window of application appl3 on the second display 108B. This process may be repeated indefinitely for arbitrary applications. The multi-display behavior may be implemented without requiring any one of the displays to be designated as a master or controlling display; the displays are treated as equal peers with respect to the user interface element 194 (or potentially any other user interface element) and the effects of its use.

[043] Figure 7 shows a multi-display embodiment where a graphic representation 222 may be moved across displays by the user. In this case, the graphic representation is dragged out of the user interface element 194 and can be deposited on either the initial

WO 2014/189536

PCT/US2013/060497 display (e.g., second display 108B) or the other display. The target application (app 13) may either replace another application or may be inserted among other applications. If the application representation 222 is dropped on the other display (after being moved across displays) then the unified recent-applications list 188 is updated accordingly to include the application replaced on the other display; e.g., application app6 is added.

[044] As can be seen from the discussion above, the content of the recent-applications list 188 and the appearance of the user interface element 194 is consistent across multiple displays, regardless of which display it is displayed on or which display an application windows is opened to. In addition, activity limited to one display does not affect what is displayed any other display, although if dormant the user interface element 194 will reflect such activity if later displayed on the other display.

[045] Figure 8 shows an embodiment implementing the application launcher 220 on multiple displays. Initially, as shown at the top of Figure 8, both the first display 108A and the second display 108B display one or more tiled (or immersive) application windows which are all available for multitasking user interactivity. There may be predefined inputs, e.g., gestures, key combinations, hardware or software buttons, etc., that can be inputted by the user to activate (display) the application launcher 220. While these activation inputs are not specific to any particular display, which display they will affect may depend on which display is currently being used by the user (e.g., which display has a pointer or which display has a window receiving user input, etc.). Or, which display an invocation input for the application launcher 220 will be directed to may be determined directly if the display also serves as or is joined with an input layer.

[046] When a user input is received and directed to a particular target display, which at one time could be either display, the windowing system displays the application launcher 220 on that target display, as shown in the middle portion of Figure 8. At the same time, the one or more application windows on the other display(s) (the non-target display(s)) are not directly affected by the display of the application launcher 220; they continue to be available for multitasking user input. Similarly, when the application launcher 220 is dismissed or undisplayed (explicitly or implicitly), the one or more application windows on the other display are not affected.

[047] When the application launcher 220 is displayed on one arbitrary display, user input directed to the other display, for instance interacting with an application window, does not cause the application launcher 220 to be undisplayed; the application launcher 220 remains available to be used by the user, possibly displaying live information in dashboard

WO 2014/189536

PCT/US2013/060497 fashion or providing other system functionality. If the application launcher 220 is displayed on a first display and launcher-invoking input is directed to a second display, then the application launcher 220 is removed from the first display and is displayed on the second display.

[048] Figure 9 shows the application launcher 220 displayed on the second display 108B. When the user is multitasking on the first display 108A and activates the application launcher 220, the windowing system shifts the application launcher 220 from the second display 108B to the first display 108A; any subsequent multitasking on the second display 108B need not affect the application launcher 220 on the first display 108A (the converse is also true).

[049] As shown in the middle portion of Figure 8, the displayed application launcher 220 can be interacted with to open an application window or perform other applicationmanagement functions. When the application representation for application app4 is selected by the user, the corresponding application window is displayed on the same display that was displaying the application launcher 220 (this might also update an application list used by one of the user interface elements 194, 196, 198). In one embodiment, the selected application window replaces the application launcher 220 and is displayed to substantially fully occupy the display, while at the same time any application windows on the other display are not affected. In one embodiment, the user can provide input to explicitly hide the application launcher 220, which may result in re-display of the application window(s) that were displayed before the application launcher 220 was opened.

[050] At times one or more of multiple displays may be disconnected or become inoperable. An application capture feature may be implemented to respond to the loss of a display by capturing indicia of the application windows that were displayed on that display. If the display or displays is/are reconnected within a predetermined period of time (e.g., five minutes), then the application windows are automatically displayed on the display or displays to reproduce the appearance of the display or displays before it/they was/were disconnected. Note that if all displays are disconnected, this timer might not apply. That is to say, there may be scenarios where there are no available displays, such as when display drivers are updating, when connecting to a remote machine, or occurrence of some system failures. In such cases, all screens can be restored.

[051] Further regarding how an operating system and/or a windowing system handle multiple monitors, the windowing system may also allow applications to be interactively

WO 2014/189536

PCT/US2013/060497 moved across displays. For example, a user might be allowed to drag a window on a first display over to a second display (or, the user might input a switch displays command). That is to say, the windowing system is able to maintain a tiled arrangement and provide user interface elements seamlessly within a concatenated display model.

[052] As can be seen from the embodiments described above, when a computing device has only one connected display, a number of system user interface elements may be available to open applications or perform other application management functions. When a second display is connected, those system user interface elements, even if dormant or not currently displayed, become equally available to be activated on both displays. In addition, their display or use on one display need not affect the contents of the other display. When the user interface elements are deactivated or undisplayed from one display, the contents (e.g., windows) of the other display may continue to be displayed thereon.

[053] Figure 10 shows an example of the computing device 100 on which embodiments described above may be implemented. The computing device 100 may have one or more displays 266, as well as storage devices 262 and a processor 264. These elements may cooperate in ways well understood in the art of computing. In addition, the input devices 168 may be integrated with or in communication with the computing device 100. The displays 266 may be any variety of devices used to display a signal outputted by computing devices, including, for example, solid-surface displays (e.g., plasma, liquid crystal), projectors, touch-sensitive surfaces, and others. The computing device 100 may have any form factor or be used in any type of encompassing device. For example, touchsensitive control panels are often used to control appliances, robots, and other machines. The computing device 100 may be in the form of a handheld device such as a smartphone, a tablet computer, a gaming device, a headless server, or others.

[054] Embodiments and features discussed above can be realized in the form of information stored in volatile or non-volatile computer-readable or device-readable devices. This is deemed to include at least devices such as optical storage (e.g., compactdisk read-only memory (CD-ROM)), magnetic media, flash read-only memory (ROM), or any other devices for storing digital information in physical matter. The stored information can be in the form of machine executable instructions (e.g., compiled executable binary code), source code, bytecode, or any other information that can be used to enable or configure computing devices to perform the various embodiments discussed above. This is also deemed to include at least volatile memory such as random-access

2013389985 19 Mar 2019 memory (RAM) and/or virtual memory storing information such as central processing unit (CPU) instructions during execution of a program carrying out an embodiment, as well as non-volatile media storing information that allows a program or executable to be loaded and executed. The embodiments and features can be performed on any type of computing 5 device, including portable devices, workstations, servers, mobile wireless devices, and so on.

[055] Throughout this specification and the claims which follow, unless the context requires otherwise, the word comprise, and variations such as comprises and comprising, will be understood to imply the inclusion of a stated integer or step or group 10 of integers or steps but not the exclusion of any other integer or step or group of integers or steps.

[056] The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that that prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:

2013389985 19 Mar 2019

1. A method of managing windows for a computing device, the method comprising:

executing a windowing system that automatically maintains windows for applications executing on the computing device, the windowing system enabling display of the application windows on a first display connected with the computing device and on a second display connected with the computing device, wherein the windowing system tracks which of the displays currently has input focus;

executing an application launcher on the computing device, the application launcher comprising indications of respective applications on the computing device, wherein the application launcher is concurrently activatable responsive to first user inputs not associated with the first display or the second display, wherein second user inputs directed to the indications are responded to by launching correspondingly selected applications, wherein the application launcher is displayed on only one display at a time, and wherein when the application launcher is displayed on either of the displays, user input directed to whichever display is not displaying the launcher does not cause the launcher to be undisplayed;

responding to each first user input that activates the launcher when the application launcher is not displayed on the first display and only the first display currently has the input focus by: (i) determining if the application launcher is displayed on the second display and if so determined undisplaying the application launcher from the second display, and (ii) displaying the application launcher on the first display;

responding to each first user input that activates the application launcher when the application launcher is not displayed on the first display and when only the second display currently has the input focus by: (i) determining if the application launcher is displayed on the first display and if so determined undisplaying the application launcher from the first display, and (ii) displaying the application launcher on the second display;

responding to each second user input that is associated only with whichever of the displays the application launcher is currently displayed on by: displaying a correspondingly selected application window on whichever of the displays the application launcher is currently displayed on, and undisplaying the application launcher; and responding to each second user input that is associated only with whichever of the displays the application launcher is not currently displayed on by: displaying a

2013389985 19 Mar 2019 correspondingly selected application window on whichever of the displays the application launcher is not currently displayed on, and after displaying the correspondingly selected application window, maintaining display of the application launcher on whichever of the displays the application launcher is currently displayed on.
2. A method according to claim 1, wherein each displaying of the application launcher responsive to a first user input comprises displaying the application launcher to fully occupy whichever of the displays the application launcher is being displayed on.
3. A method according to claim 2, wherein for each displaying of an application responsive to a second user input that is associated only with whichever of the displays the application launcher is currently displayed on, displaying the application to fully occupy whichever display was displaying the application launcher when the second user input was received.
4. A method according to claim 3, wherein, whenever an application window, that fully occupies a display as a result of having been displayed by a second user input, is undisplayed, in response, the application launcher is displayed on the display from which the application window was undisplayed.
5. A method according to any one of claims 1-4, wherein the application launcher comprises a list of recently used applications, wherein the list corresponds to applications recently used on both the first display and the second display.
6. A method according to any one of claims 1-5, wherein neither display is managed as a main display with respect to the application launcher, and wherein the input focus corresponds to whichever one of the displays is currently receiving or configured to receive user input.
7. A computing device comprising a processor, storage, and a first display, the storage storing a windowing system that when executed by the processor displays application windows of applications executing by the processor, the computing device further comprising:

2013389985 19 Mar 2019 the windowing system, when executing, being capable of displaying application windows on the first display and on a second display when a second display is locally connected with the computing device, the windowing system further comprising a user interface element, wherein the windowing system displays application windows on either display;

the user interface element, wherein the user interface element is simultaneously activatable on the first display and activatable on the second display, and while so simultaneously activatable the user interface element can only be displayed on a single display at a time, wherein each activation of the user interface element displays the user interface element on, and according to, whichever of the displays is currently exclusively designated as having user input focus, wherein whenever the user interface element is displayed on the first display while a user input not associated with either display directing an activation of the user interface element to the second display is detected, the user input is handled by displaying the user interface element on the second display and by undisplaying the user interface element from the first the display, wherein whenever the user interface element is displayed on the second display while a user input not associated with either display directing an activation of the user interface element to the first display is detected, the user input is handled by displaying the user interface element on the second display and by undisplaying the user interface element from the first the display, and;

wherein whenever the user interface element is displayed on either of the displays, the user interface element remains displayed on the either of the displays during and after multitasking user input is directed to the other display that is not displaying the user interface element;

the user interface element is configured to respond to first user interactions with graphic representations of applications in the user interface element by displaying corresponding application windows on, and according to, whichever display is currently displaying the user interface element, and by undisplaying the user interface element; and the user interface element is further configured to respond to second user interactions with representations of applications in the user interface element by displaying corresponding application windows on, and according to, whichever display is not currently displaying the user interface element, and by continuing to display the user interface element after the corresponding application windows are displayed.

2013389985 19 Mar 2019
8. A computing device according to claim 7, wherein the user input not associated with either display that directs an activation of the user interface element to either the first or second display comprises an input does not specify, designate, or select any display.
9. A computing device according to claim 7 or 8, wherein whenever the user interface element is being displayed on either one of the displays: the user is able to interact with any application windows displayed on the other display, the user is able to interact with the user interface element without disrupting display of any application windows displayed on the other display, and the user is able to interact with the user interface element to trigger the display of application windows on whichever of the displays is displaying the user interface element.
10. A computing device according to any one of claims 7-9, wherein when an application window is displayed responsive to a user interaction with the user interface element, the user interface element is undisplayed and the application window is displayed on the display where the user interface element was interacted with, the application window being displayed such that the application window substantially fully occupies the display where the user interface element was interacted with.
11. A computing device according to any one of claims 7-10, wherein the user interface element comprises application icons for launching corresponding applications.
12. A computing device according to claim 11, wherein the user interface element, when being displayed, always comprises a full-screen window that fully occupies whichever display is displaying the user interface element.
13. One or more computer storage devices storing information to enable a computing device to perform a process, the process comprising:

executing applications having respective application windows displayable by a windowing system executing on the computing device;

displaying a first application window on a first display connected with the computing device while displaying a second application window on a second display

2013389985 19 Mar 2019 connected with the computing device, each application window substantially fully occupying the display on which it is displayed;

enabling a user to direct input to the first display to interact with the first application window while concurrently enabling the user to direct input to the second display to interact with the second application window;

providing an application-list comprising a set of application representations that when displayed can be interacted with by the user to open application windows of respectively represented applications, the application-list able to be displayed, at any given time, responsive to user input not associated with either display, on either display according to which display currently has user input focus associated therewith, wherein display of the application-list responsive to a user input comprises determining if the application-list is being displayed on the display that does not have the user input focus, and when so determined undisplaying the application-list from the display that does not have the user input focus;

responsive to a user input not associated with either display and received while the user focus is associated with the first display, displaying the application-list over the first application window and undisplaying the application-list from the second display, and responsive to a user input not associated with either display and received while the user focus is associated with the second display, displaying the application-list over the second application window and undisplaying the application-list from the first display; and while the application-list is displayed on one of the displays, responsive to a user input selecting one of the application representations for display on the one of the displays: undisplaying the application-list and displaying a third application window corresponding to the selected application representation on the one of the displays while maintaining the displaying of the first or second application window on the display other than the one of the displays; and while the application-list is displayed on the one of the displays, responsive to a user input selecting one of the application representations for display on the other of the displays: retaining display of the application-list on the one of the displays after displaying an application window corresponding and responsive to the selected one of the application representations.
14. A storage device according to claim 13, wherein which application representations are included in the application-list is dynamically determined according to

2013389985 19 Mar 2019 interactive openings or closings of application windows on the first display and on the second display.
15. A storage device according to claim 14, wherein the application representations represent applications recently used by the user but not currently displayed.
16. A storage device according to any one of claims 13 - 15, wherein the user input selecting one of the applications comprises a drag input that drags the selected application representation from the application-list and the process further comprises enabling the user to interactively drag the application representation to the first display while also enabling the user to interactively drag the application representation to the second display, wherein which display the application representation is dragged to interactively select which display the selected application will be displayed on and also determines where on the selected display the third application window will be displayed.
17. A storage device according to claim 16, the process further comprising enabling, by the windowing system, interactive dragging of application windows between the first display and the second display.
18. A computing device according to any one of claims 7 - 12, wherein the user interface element is undisplayed from one of the displays responsive to a hide command inputted by the user or responsive to display of an application window on the one of the displays.
19. A computing device according to any one of claims 7 - 12, wherein the user input directing activation of the user interface element comprises a hot key of a keyboard, wherein the windowing system keeps track of which of the first and second displays are currently active, and when the hot key is pressed the user interface element is displayed according to the currently active display.