JP6887034B2 - Devices and methods for accessing common device features - Google Patents

Description

The present application generally relates to electronic devices having a touch-sensitive surface, including, but not limited to, electronic devices having a touch-sensitive surface including a plurality of user interfaces for accessing general device functions.

The use of touch-sensitive surfaces as input devices for computers and other electronic computing devices has increased significantly in recent years. Exemplary touch sensing surfaces include touchpads and touch screen displays. Such surfaces are widely used to manipulate user interface objects on the display.

Illustrative operations include accessing controls for controlling device functions and functions of remote devices. However, the method of performing these operations is cumbersome and inefficient. As the field of use of remote devices expands, access to specific controls and functions is becoming more time consuming. For example, entering a series of inputs to access a control via a drill-down menu can be time consuming for the user, especially if the desired control position is not known in advance by the user. Navigation between applications to access device features also takes time.

Therefore, there is a need for electronic devices that have faster and more efficient methods and interfaces to access common device functions. Such methods and interfaces optionally complement or replace conventional methods for accessing common device functions. Such methods and interfaces reduce the cognitive burden on the user and provide a more efficient human-machine interface. For battery-powered devices, such methods and interfaces save power and extend the time between battery charges.

The above-mentioned defects and other problems associated with the user interface for electronic devices having a touch-sensitive surface are reduced or eliminated by the devices of the present disclosure. In some embodiments, the device is a desktop computer. In some embodiments, the device is portable (eg, a notebook computer, tablet computer, or handheld device). In some embodiments, the device has a touchpad. In some embodiments, the device has a touch-sensitive display (also known as a "touch screen" or "touch screen display"). In some embodiments, the device is a graphical user interface (GUI), one or more processors, memory, and one or more modules, programs, or instruction sets stored in memory for performing multiple functions. Has. In some embodiments, the user interacts with the GUI primarily through a stylus on the touch-sensitive surface and / or finger contact and gestures. In some embodiments, the features include image editing, drawing, presenting, word processing, website creation, disc authoring, spreadsheet creation, gameplay, making phone calls, video conferencing, emailing, instant messaging. , Training support, digital photography, digital video recording, web browsing, digital music playback, and / or digital video playback are optionally included. Executable instructions that perform those functions are optionally included in non-temporary computer-readable storage media or other computer program products configured to be executed by one or more processors.

According to some embodiments, the method is performed in an electronic device having a display and a touch sensitive surface. The method is to detect the first input while the device is in the display off state, activate the device's display in response to detecting the first input, and on the display. Detects swipe gestures on the touch sensitive surface while displaying the first user interface corresponding to the device display on state and displaying the first user interface corresponding to the device display on state. In response to detecting a swipe gesture on the touch-sensitive surface, the device is in lock mode with display on, and the swipe gesture is in the first direction, according to the determination of the first user interface. The display is a second user interface, the second user interface replacing the display of the second user interface displaying the first content, and the device is in display-on unlock mode. According to the determination that the swipe gesture is in the first direction, the display of the first user interface is the second user interface, and the second user interface is the lock mode in which the display of the first content and the device is on. Includes replacing the display of the second user interface with displaying the first additional content that is not displayed when is in.

According to some embodiments, the method is performed in an electronic device having a display and a touch sensitive surface. Detecting the first input to activate the device's display and activating the device's display in response to detecting the first input while the device is in the display off state. And, while displaying the first user interface corresponding to the display on state of the device on the display and displaying the first user interface corresponding to the display on state of the device, on the touch sensing surface. Replaces the display of the first user interface with the display of the camera application user interface according to the detection of the swipe gesture and the determination that the swipe gesture is in the first direction in response to the detection of the swipe gesture. And, according to the determination that the swipe gesture is in a second direction different from the first direction, the display of the first user interface is displayed in the mini-application object user interface configured to contain multiple mini-application objects. Each mini-application object of multiple mini-application objects has a corresponding application stored in the device, the display and replacement of the mini-application object user interface, and the swipe gesture in the first direction and Displaying the first page of the multi-page control panel user interface and swiping gestures in the first, second, and third directions according to the determination that they are in a third direction that is different from the second direction. Includes displaying a notification user interface configured to display a plurality of notifications according to a determination that they are in a fourth direction that is different from the direction.

According to some embodiments, the method is performed in an electronic device having a display and a touch sensitive surface. The method is to display a first user interface that includes a plurality of application icons corresponding to different applications among a plurality of applications installed on the device, and a first application icon of the plurality of application icons. , The first application icon corresponds to the first application of a plurality of applications, detects the first input at the position on the touch sensing surface corresponding to the first application icon, and detects the first input. In response to the above, a preview of the first mini-application object or the first mini-application object in the overlay area, the first mini-application object corresponding to the first application of a plurality of applications. The overlay area displays a preview of the first mini-application object or the first mini-application object, including an accommodation for adding the first mini-application icon to the second user interface that displays multiple mini-application objects. And the detection of the second input at the position on the touch-sensitive surface corresponding to the affordance for adding the first mini-application icon to the second user interface, and the detection of the second input. In response to, the addition of the first mini-application object to the second user interface that displays a plurality of mini-application objects.

According to some embodiments, the method is performed in an electronic device having a display and a touch sensitive surface. The method is to detect the first gesture on the touch-sensitive surface while displaying the initial user interface on the display, and in response to detecting the first gesture, on the display. The first page of the multi-page control panel, which is the first page of the multi-page control panel, is the first page of the multi-page control panel, and the first page of the multi-page control panel is the first page of the multi-page control panel. While displaying the first page, in response to detecting the second gesture on the touch sensing surface and detecting the second gesture, it is the second page of the multi-page control panel. , The second page of the multi-page control panel contains multiple content playback control affordances, the second page of the multi-page control panel replaces the first page of the multi-page control panel on the display, multi-page control. Includes displaying the second page of the panel.

According to some embodiments, the electronic device comprises a display unit configured to display a user interface, a touch sensing surface unit configured to detect contact, and a display unit and a touch sensing surface unit. Includes combined processing units. The processing unit is configured to detect the first input while the device is in the display off state. In response to detecting the first input, the processing unit activates the display unit of the device and enables the display of the first user interface corresponding to the display on state of the device on the display unit. It is configured in. The processing unit is configured to detect a swipe gesture on the touch sensitive surface unit while displaying the first user interface corresponding to the display on state of the device. In response to detecting a swipe gesture on the touch-sensitive surface unit, the processing unit determines that the device is in display-on lock mode and the swipe gesture is in the first direction, and the first user interface. Is the display of the second user interface, the second user interface replaces the display of the second user interface displaying the first content, the device is in display-on unlock mode, and swipe. According to the determination that the gesture is in the first direction, the display of the first user interface is the second user interface, and the second user interface is in the lock mode in which the first content and device are displayed on. It is configured to replace a second user interface that displays a first additional content that is not displayed at one time.

According to some embodiments, the electronic device comprises a display unit configured to display a user interface, a touch sensing surface unit configured to detect contact, and a display unit and a touch sensing surface unit. Includes combined processing units. The processing unit is configured to detect a first input for activating the display unit of the device while the device is in the display off state. In response to detecting the first input, the processing unit activates the display of the device and enables the display of the first user interface on the display unit, which corresponds to the display on state of the device. It is configured. The processing unit is configured to detect a swipe gesture on the touch sensitive surface unit while displaying the first user interface corresponding to the display on state of the device. In response to detecting the swipe gesture, the processing unit replaces the display of the first user interface with the display of the camera application user interface according to the determination that the swipe gesture is in the first direction, and the swipe gesture is the first. A mini-application object user interface configured to include a plurality of mini-application objects for display of the first user interface according to a determination that the user interface is in a second direction different from the first direction. Each mini-application object of the object replaces the display of the mini-application object user interface, which has a corresponding application stored on the device, and the display swipe gesture is different from the first and second directions. According to the determination that it is in the 3rd direction, the display of the 1st page of the multi-page control panel user interface is enabled, and the swipe gesture is in the 1st direction, the 2nd direction, and the 4th direction different from the 3rd direction. It is configured to enable the display of the notification user interface, which is configured to display multiple notifications according to the determination in.

According to some embodiments, the electronic device is a display unit configured to display a first user interface that includes a plurality of application icons corresponding to different applications among the plurality of applications installed on the device. A touch sensing surface unit configured to detect contact, and a display unit and a processing unit coupled to the sensing surface unit. The processing unit is the first application icon of the plurality of application icons, and the first application icon corresponds to the first application of the plurality of applications, and is on the touch sensing surface unit corresponding to the first application icon. It is configured to detect the first input at the position. In response to detecting the first input, the processing unit is a preview of the first mini-application object or the first mini-application object in the overlay area, and the first mini-application object is a plurality of. Corresponding to the first application of the application, the overlay area is a first mini-application object or a first mini-application object that includes an accommodation for adding the first mini-application object to the second user interface that displays multiple mini-application objects. It is configured to enable the display of a preview of the first mini-application object. The processing unit is further configured to detect a second input at a position on the touch sensitive surface unit corresponding to the affordance for adding the first mini-application object to the second user interface. The processing unit is configured to add a first mini-application object to a second user interface that displays a plurality of mini-application objects in response to detecting a second input.

According to some embodiments, the electronic device is coupled to a display unit configured to display a user interface, a touch sensitive surface unit configured to contact, and a display unit and a touch sensitive surface unit. Includes processing units and processing units. The processing unit is configured to detect a first gesture on the touch sensitive surface unit while displaying the initial user interface on the display unit. In response to detecting the first gesture, the processing unit is on the display unit the first page of the multi-page control panel, the first page of the control panel containing multiple device control affordances. , Is configured to enable the display of the first page of the multi-page control panel. While displaying the first page of the multi-page control panel, the processing unit is configured to detect a second gesture on the touch sensitive surface unit. The processing unit is the second page of the multi-page control panel in response to detecting the second gesture, the second page of the multi-page control panel containing a plurality of content playback control affordances. The second page of the multi-page control panel is configured to enable the display of the second page of the multi-page control panel, which replaces the first page of the multi-page control panel on the display unit.

According to some embodiments, the electronic device comprises a display, a touch-sensitive surface, and optionally one or more sensors that detect the strength of contact with the touch-sensitive surface, one or more processors, and a memory. And one or more programs, one or more programs are stored in memory and configured to be executed by one or more processors, and one or more programs are described herein. Includes instructions to perform or cause any of the described methods to be performed. According to some embodiments, the computer-readable storage medium internally stores an instruction, which detects the intensity of contact between the display, the touch-sensitive surface, and optionally the touch-sensitive surface. When performed by an electronic device having one or more sensors, it causes the device to perform or perform any of the methods described herein. According to some embodiments, a display, a touch-sensitive surface, one or more sensors that optionally detect the strength of contact with the touch-sensitive surface, a memory, and one or more stored in the memory. A graphical user interface on an electronic device with one or more processors running a program of includes one or more of the elements displayed in any of the methods described above, which elements are described herein. Updated in response to input, as described in any of the methods described in the document. According to some embodiments, the electronic device comprises a display, a touch-sensitive surface, and optionally one or more sensors that detect the intensity of contact with the touch-sensitive surface, and any of the sensors described herein. Includes means for performing or causing the operation of the method to be performed. According to some embodiments, the electronic device is for use in an electronic device having a display, a touch-sensitive surface, and optionally one or more sensors that detect the strength of contact with the touch-sensitive surface. The information processing device includes means for performing or causing the operation of any of the methods described herein to be performed.

Therefore, one of the options for detecting the intensity of contact with the display, touch-sensitive surface, and touch-sensitive surface, having a faster and more efficient method and interface for accessing common device functions. Electronic devices equipped with one or more sensors are provided, thereby improving the effectiveness, efficiency, and user satisfaction of such devices. Such methods and interfaces can complement or replace conventional methods for accessing common device functions.

In order to better understand the various embodiments described, the following "forms for carrying out the invention" should be referred to along with the following drawings. Here, similar reference numbers refer to the corresponding parts throughout those figures.

FIG. 6 is a block diagram showing a portable multifunction device having a touch-sensitive display according to some embodiments. It is a block diagram which shows the exemplary component for event processing which concerns on some embodiments. A portable multifunctional device having a touch screen according to some embodiments is shown. FIG. 6 is a block diagram of an exemplary multifunctional device having a display and a touch sensitive surface, according to some embodiments. An exemplary user interface for menus of applications on portable multifunction devices, according to some embodiments. An exemplary user interface for a multifunctional device having a touch sensitive surface separated from a display, according to some embodiments. It is a figure which shows the example of the dynamic intensity threshold value which concerns on some embodiments. It is a figure which shows the example of the dynamic intensity threshold value which concerns on some embodiments. It is a figure which shows the example of the dynamic intensity threshold value which concerns on some embodiments. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. An exemplary user interface for accessing common device functions, according to some embodiments, is shown. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of accessing the control from the display-on-user interface which concerns on some embodiments. It is a flow diagram which shows the method of adding the mini-application object to the mini-application user interface which concerns on some embodiments. It is a flow diagram which shows the method of adding the mini-application object to the mini-application user interface which concerns on some embodiments. It is a flow diagram which shows the method of adding the mini-application object to the mini-application user interface which concerns on some embodiments. It is a flow diagram which shows the method of navigating control using a multi-page control panel which concerns on some embodiments. It is a flow diagram which shows the method of navigating control using a multi-page control panel which concerns on some embodiments. It is a flow diagram which shows the method of navigating control using a multi-page control panel which concerns on some embodiments. It is a flow diagram which shows the method of navigating control using a multi-page control panel which concerns on some embodiments. It is a flow diagram which shows the method of navigating control using a multi-page control panel which concerns on some embodiments. It is a functional block diagram of an electronic device which concerns on some embodiments. It is a functional block diagram of an electronic device which concerns on some embodiments. It is a functional block diagram of an electronic device which concerns on some embodiments. It is a functional block diagram of an electronic device which concerns on some embodiments.

Many electronic devices provide various interfaces for accessing application functions and changing device settings. Such an interface can require the user to activate the device, provide credentials, and / or drill down through multiple menus to access the desired application features or device settings. .. The methods described herein provide an interface for accessing common device features such as application features and device settings. Such an interface reduces power usage and battery of the device by reducing the number of inputs required to access such features and allowing users to use the device faster and more efficiently. Improve lifespan.

In the following, FIGS. 1A, 1B, 2 and 3 provide an exemplary device description. 4A, 4B and 5A-5BJ show exemplary user interfaces for providing haptic feedback. 6A-6E show a flow diagram of a method of accessing control from a display-on-user interface, according to some embodiments. 7A-7E show a flow diagram of a method of accessing control from a display-on-user interface, according to some embodiments. 8A-8C show a flow diagram of a method of adding a mini-application object to the mini-application user interface according to some embodiments. 9A-9E show a flow diagram of a method of navigating control using a multi-page control panel, according to some embodiments. The user interface of FIGS. 5A-5BJ is used to illustrate the processes of FIGS. 6A-6E, 7A-7E, 8A-8C, and 9A-9E.
Illustrative device

Here, a detailed reference is made to the embodiments shown in the accompanying drawings. The following detailed description provides a number of specific details to provide a complete understanding of the various embodiments described. However, it will be apparent to those skilled in the art that the various embodiments described may be practiced without these specific details. In other examples, well-known methods, procedures, components, circuits, and networks are not described in detail so as not to unnecessarily obscure aspects of the embodiment.

In the present specification, terms such as first and second are used to describe various elements in some embodiments, but these elements should not be limited by those terms. It will also be understood. These terms are used only to distinguish one element from another. For example, without departing from the scope of the various embodiments described, the first contact can be referred to as the second contact, and similarly, the second contact can be referred to as the first contact. .. The first and second contacts are both contacts, but they are not the same contact unless the context clearly indicates that they are not.

The terms used in the description of the various embodiments described herein are for the purpose of describing a particular embodiment only and are not intended to be limiting. As used in the claims of the various described embodiments and in the appended claims, the singular forms "a", "an", and "the" unless the context explicitly indicates otherwise. , Also intended to include the plural. As used herein, the term "and / or (and / or)" refers to and includes all possible combinations of any one or more of the associated enumerated items. Please understand that. The terms "includes", "includes", "comprises", and / or "comprising", as used herein, are the features, integers, steps, described. Specifying the existence of actions, elements, and / or components, but not excluding the existence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof. Will be further understood.

As used herein, the term "if" is optional and context sensitive to "when", "upon", "in response to". It is interpreted to mean "determining" or "in response to detecting". Similarly, the phrase "if it is determined" or "if (a statused condition or event) is detected" is optional. , "Uppon determining", "in response to determining", "upon detecting (the statused condition or event)" (conditions stated Or when an event) is detected) ", or" in response to detecting (the statused condition or event) (in response to the detection of the stated condition or event) ".

An electronic device, a user interface for such a device, and an embodiment of an associated process for using such a device are described. In some embodiments, the device is a portable communication device, such as a mobile phone, that also includes other functions such as a PDA function and / or a music player function. Exemplary embodiments of portable multifunction devices include Cupertino, Apple Inc., Calif. Includes, but is not limited to, iPhone®, iPod Touch®, and iPad® devices from. Other portable electronic devices such as laptops or tablet computers with touch sensitive surfaces (eg, touch screen displays and / or touchpads) are optionally used. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer having a touch sensitive surface (eg, a touch screen display and / or a touch pad).

In the following description, an electronic device including a display and a touch sensing surface will be described. However, it should be understood that electronic devices optionally include one or more other physical user interface devices such as physical keyboards, mice, and / or joysticks.

Devices are typically note-taking applications, drawing applications, presentation applications, word processing applications, website creation applications, disk authoring applications, spreadsheet applications, gaming applications, phone applications, video conferencing applications, email applications, and instant messaging. Supports a variety of applications such as applications, training support applications, photo management applications, digital camera applications, digital video camera applications, web browsing applications, digital music player applications and / or one or more of digital video player applications.

Various applications running on the device optionally use at least one common physical user interface device, such as a touch sensitive surface. One or more functions of the touch sensitive surface, as well as the corresponding information displayed on the device, may be adjusted and / or modified on a per-application basis and / or optionally within each application. Thus, the device's common physical architecture (such as touch-sensitive surfaces) optionally supports a variety of applications with user-intuitive and transparent user interfaces.

Here, attention is paid to an embodiment of a portable device including a touch-sensitive display. FIG. 1A is a block diagram showing a portable multifunction device 100 having a touch-sensitive display system 112 according to some embodiments. The touch-sensitive display system 112 may be referred to as a "touch screen" for convenience, and may simply be referred to as a touch-sensitive display. The device 100 includes a memory 102 (optionally including one or more computer-readable storage media), a memory controller 122, one or more processing units (CPU) 120, a peripheral device interface 118, an RF circuit 108, and an audio circuit 110. , Speaker 111, microphone 113, input / output (I / O) subsystem 106, other input or control device 116, and external port 124. The device 100 optionally includes one or more photosensors 164. The device 100 optionally includes one or more intensity sensors 165 for detecting the intensity of contact on the device 100 (eg, a touch-sensitive surface such as the touch-sensitive display system 112 of the device 100). The device 100 generates a tactile output on the device 100 (eg, a tactile output on a touch-sensitive surface, such as the touch-sensitive display system 112 of the device 100 or the touchpad 355 of the device 300). One or more tactile output generators 167 are optionally included. These components optionally communicate via one or more communication buses or signal lines 103.

As used herein and in the claims, the term "tactile output" is the physical displacement of the device with respect to the previous position of the device, the configuration of the device, which will be detected by the user by the user's tactile sensation. Refers to the physical displacement of an element (eg, a touch-sensitive surface) with respect to another component of the device (eg, housing), or the displacement of a component with respect to the center of mass of the device. For example, the tactile sensation generated by physical displacement in a situation where the device or a component of the device is in contact with a touch-sensitive user's surface (eg, the finger, palm, or other part of the user's hand). The output is interpreted by the user as a tactile sensation that corresponds to a perceived change in the physical properties of the device or its components. For example, the movement of a touch-sensitive surface (eg, a touch-sensitive display or trackpad) is optionally interpreted by the user as a "down-click" or "up-click" of a physical actuator button. In some cases, the user "down-clicks" or "up-clicks" even when there is no movement of the physical actuator button associated with the touch-sensitive surface physically pressed (eg, displaced) by the user's movement. You will feel the tactile sensation. As another embodiment, the movement of the touch-sensitive surface is optionally interpreted or perceived by the user as the "roughness" of the touch-sensitive surface, even if the smoothness of the touch-sensitive surface does not change. To. Although the interpretation of touch by such a user depends on the user's personal sensory perception, there are many sensory perceptions of touch that are common to the majority of users. Therefore, if the tactile output is described as corresponding to a particular sensory perception of the user (eg, "up-click", "down-click", "roughness"), the generated tactile sensation, unless otherwise stated. Intellectual output corresponds to the physical displacement of a device, or component of a device, that produces a typical (or average) user's described sensory cognition.

The device 100 is merely an embodiment of a portable multifunction device, the device 100 optionally having more or fewer components than those shown, and optionally two or more components. It should be understood that the components may be combined or optionally arranged to have different components or arrangements. The various components shown in FIG. 1A are implemented in hardware, software, firmware, or a combination thereof, including one or more signal processing circuits and / or application-specific integrated circuits.

The memory 102 optionally includes high speed random access memory and optionally also includes non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state memory devices. Access to the memory 102 by other components of the device 100, such as the CPU (s) 120 and the peripheral interface 118, is optionally controlled by the memory controller 122.

Peripheral interface 118 is used to combine the input and output peripherals of the device with the CPU (s) 120 and memory 102. The one or more processors 120 perform various functions of the device 100 and operate or execute various software programs and / or instruction sets stored in the memory 102 to process data.

In some embodiments, the peripheral interface 118, CPU (s) 120, and memory controller 122 are optionally mounted on a single chip, such as chip 104. In some other embodiments, they are optionally mounted on individual chips.

The RF (radio frequency) circuit 108 transmits and receives RF signals, also called electromagnetic signals. The RF circuit 108 converts an electric signal into an electromagnetic signal or converts the electromagnetic signal into an electric signal and communicates with a communication network and other communication devices via the electromagnetic signal. The RF circuit 108 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, and a memory. Includes, but is not limited to, well-known circuits for performing these functions, including, but not limited to, optionally. The RF circuit 108 is optionally the Internet (also called World Wide Web (WWW)), intranet and / or wireless network (cellular telephone network, wireless local area network (LAN) and / Alternatively, it communicates wirelessly with a network such as a metropolitan area network (MAN) and other devices. Wireless communication uses any one of a plurality of communication standards, communication protocols, and communication technologies arbitrarily, and the communication standards, communication protocols, and communication technologies are global systems for mobile communication (Global). System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), High-speed downlink packet access (HSDPA), High-speed uplink packet connection (high-speed uplink) packet access) (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA +, double cell HSPA (Dual-Cell HSPA) (DC-HSPDA), long term evolution (LTE), Near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (CDMA) time division multiple access) (TDMA), Bluetooth®, Willless Fidelity (Wi-Fi)® (eg, IEEE 802.11a, IEEE 802.11ac, IEEE 802.11ax, IEEE 802.11b, LTE 802.11g and / or IEEE 802.11n), voice over Internet Protocol (VoIP), Wi-MAX, protocols for e-mail (eg, Internet message access protocol) ( IMAP and / or post office protocol (POP), instant messaging (eg, extensible messaging and) Presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS), And / or Short Message Service (SMS), or any other suitable communication protocol, including, but not limited to, communication protocols that have not yet been developed as of the filing date of this document. Not done.

The voice circuit 110, the speaker 111, and the microphone 113 provide a voice interface between the user and the device 100. The voice circuit 110 receives voice data from the peripheral device interface 118, converts the voice data into an electric signal, and transmits the electric signal to the speaker 111. The speaker 111 converts an electrical signal into a human audible sound wave. The audio circuit 110 also receives an electrical signal converted from sound waves by the microphone 113. The voice circuit 110 converts the electrical signal into voice data and transmits the voice data to the peripheral device interface 118 for processing. The audio data is optionally acquired from the memory 102 and / or the RF circuit 108 by the peripheral interface 118 and / or transmitted to the memory 102 and / or the RF circuit 108. In some embodiments, the audio circuit 110 also includes a headset jack (eg, 212, FIG. 2). The headset jack is a removable audio input / output peripheral such as an audio circuit 110 and a headset that has both output-only headphones or outputs (eg, headphones for one or both ears) and inputs (eg, microphones). Provides an interface between.

The I / O subsystem 106 couples input / output peripherals on the device 100, such as the touch-sensitive display system 112 and other input or control devices 116, with the peripheral interface 118. The I / O subsystem 106 optionally includes a display controller 156, an optical sensor controller 158, an intensity sensor controller 159, a haptic feedback controller 161 and one or more input controllers 160 for other input or control devices. .. One or more input controllers 160 receive electrical signals from other inputs or control devices 116 and transmit electrical signals to them. Other input or control devices 116 are optional and include physical buttons (eg, push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and the like. In some alternative embodiments, the input controller (s) 160 is optionally coupled (or) with any of the pointer devices such as keyboard, infrared port, USB port, stylus, and / or mouse. , Not combined with either). One or more buttons (eg, 208, FIG. 2) optionally include up / down buttons for volume control of the speaker 111 and / or microphone 113. One or more buttons optionally include push buttons (eg 206, FIG. 2).

The touch-sensitive display system 112 provides an input and output interfaces between the device and the user. The display controller 156 receives an electrical signal from the touch-sensitive display system 112 and / or transmits the electrical signal to the touch-sensitive display system 112. The touch-sensitive display system 112 displays a visual output to the user. The visual output optionally includes graphics, text, icons, videos, and any combination thereof (collectively referred to as "graphics"). In some embodiments, some or all of the visual output corresponds to a user interface object. As used herein, the term "affordance" is a graphical user interface configured to respond to input directed towards a user-interactive graphical user interface object (eg, a graphical user interface object). Object). Examples of user-interactive graphical user interface objects include, but are not limited to, buttons, sliders, icons, selectable menu items, switches, hyperlinks, or other user interface controls.

The touch-sensitive display system 112 includes a touch-sensitive surface, a sensor, or a set of sensors that accept input from the user based on tactile and / or tactile contact. The touch-sensitive display system 112 and display controller 156 (along with any associated module and / or instruction set in memory 102) detect contact (and any movement or interruption of contact) on the touch-sensitive display system 112. Transforms the detected contact into an interaction with a user interface object (eg, one or more softkeys, icons, web pages, or images) displayed on the touch-sensitive display system 112. In one exemplary embodiment, the point of contact between the touch-sensitive display system 112 and the user corresponds to the user's finger or stylus.

The touch-sensitive display system 112 optionally uses LCD (liquid crystal display) (liquid crystal display) technology, LPD (light emtiting polymer display) (light emitting polymer display) technology, or LED (light emitting device) (light emitting diode) technology. However, in other embodiments, other display technologies are used. The touch-sensitive display system 112 and the display controller 156 are optional for determining capacitance technology, resistance technology, infrared technology, and surface acoustic wave technology, as well as one or more points of contact with the touch-sensitive display system 112. Use any of a number of currently known or later developed touch sensing techniques, including but not limited to other proximity sensor arrays or other elements, to make contact and any movement or interruption thereof. To detect. In an exemplary embodiment, Apple Inc. Projection-type mutual capacitance sensing techniques are used, such as those found in iPhone®, iPod Touch®, iPad®, etc. from (Cupertino, California).

The touch-sensitive display system 112 has an optional video resolution of over 100 dpi. In some embodiments, the video resolution of the touch screen exceeds 400 dpi (eg, 500 dpi, 800 dpi, or higher). The user optionally contacts the touch-sensitive display system 112 using any suitable object or accessory such as a stylus or finger. In some embodiments, the user interface is designed to work with finger-based contacts and gestures, but these are styluses due to the larger area of finger contact on the touch screen. It may be less accurate than the input based on. In some embodiments, the device translates a rough finger-based input into a precise pointer / cursor position or a command to perform the action desired by the user.

In some embodiments, in addition to the touch screen, the device 100 optionally includes a touchpad (not shown) for activating or deactivating a particular function. In some embodiments, the touchpad, unlike the touch screen, is a touch-sensitive area of the device that does not display visual output. The touchpad is optionally an extension of the touch-sensitive surface separated from the touch-sensitive display system 112 or the touch-sensitive surface formed by the touch screen.

The device 100 also includes a power system 162 for supplying power to various components. The power system 162 is an optional power management system, one or more power sources (eg, battery, alternating current (AC)), recharging system, power failure detection circuit, power converter or inverter, power status indicator (eg, light emitting diode (eg, light emitting diode). LED)), as well as any other components associated with power generation, management, and distribution within portable devices.

The device 100 also optionally includes one or more optical sensors 164. FIG. 1A shows an optical sensor coupled with an optical sensor controller 158 in the I / O subsystem 106. The optical sensor (s) 164 optionally includes a charge binding device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The light sensor (s) 164 receives light from the environment projected through one or more lenses and converts the light into data that represents an image. In cooperation with the imaging module 143 (also called the camera module), the optical sensor (s) 164 optionally captures still images and / or video. In some embodiments, the optical sensor is the opposite of the touch-sensitive display system 112 on the front of the device so that the touch screen can be used as a viewfinder for still and / or video image acquisition. , Arranged on the back of the device 100. In some embodiments, the user's image is captured (for example, for a selfie, for a video conference while the user is watching other video conference participants on the touch screen, etc.). Optical sensor is placed on the front of the device.

The device 100 also optionally includes one or more contact strength sensors 165. FIG. 1A shows a contact strength sensor coupled with a strength sensor controller 159 in the I / O subsystem 106. The contact strength sensor (s) 165 may be one or more piezo resistance strain gauges, capacitance type force sensors, electrical force sensors, pressure power sensors, optical force sensors, capacitive touch sensing surfaces, or other strength sensors (s). For example, a sensor used to measure the force (or pressure) of contact on the touch sensing surface) is optionally included. The contact strength sensor (s) 165 receives contact strength information (eg, pressure information or proxy for pressure information) from the environment. In some embodiments, at least one contact strength sensor is located on or in close proximity to a touch-sensitive surface (eg, touch-sensitive display system 112). In some embodiments, the at least one contact strength sensor is located on the back surface of the device 100, which is the opposite of the touch screen display system 112 located on the front surface of the device 100.

The device 100 also optionally includes one or more proximity sensors 166. FIG. 1A shows a proximity sensor 166 coupled with a peripheral interface 118. Instead, the proximity sensor 166 is coupled with the input controller 160 in the I / O subsystem 106. In some embodiments, the proximity sensor turns off and disables the touch-sensitive display system 112 when the multifunction device is placed close to the user's ear (eg, when the user is making a phone call). ..

The device 100 also optionally includes one or more tactile output generators 167. FIG. 1A shows a tactile output generator coupled with a tactile feedback controller 161 in the I / O subsystem 106. The tactile output generator (s) 167 is one or more electroacoustic devices such as speakers or other audio components and / or motors, solenoids, electroactive polymers, piezoelectric actuators, electrostatic actuators, or the like. An electromechanical device that converts energy into linear motion, such as a tactile output generation component (eg, a component that converts an electrical signal into a tactile output on a device), is optionally included. The tactile output generator (s) 167 receives a tactile feedback generation command from the tactile feedback module 133 and generates a tactile output perceptible to the user of the device 100 on the device 100. In some embodiments, at least one tactile output generator is located on or in close proximity to a touch-sensitive surface (eg, touch-sensitive display system 112) and optionally directs the touch-sensitive surface vertically. A tactile output is generated by moving (eg, inside / outside the surface of the device 100) or laterally (eg, back and forth in the same plane as the surface of the device 100). In some embodiments, the at least one tactile output generator sensor is located on the back surface of the device 100, which is the opposite of the touch sensitive display system 112 located on the front surface of the device 100.

The device 100 also optionally includes one or more accelerometers 168. FIG. 1A shows an accelerometer 168 coupled to the peripheral interface 118. Instead, the accelerometer 168 is optionally coupled with the input controller 160 in the I / O subsystem 106. In some embodiments, the information is displayed on the touch screen display in a vertical or horizontal view based on the analysis of data received from one or more accelerometers. The device 100, in addition to the accelerometer (s) 168, is a magnetometer (not shown) and GPS (or GLONASS or the like) for acquiring information about the position and orientation (eg, portrait or landscape) of the device 100. Global Navigation System) Includes receiver (not shown) as an option.

In some embodiments, the software components stored in the memory 102 are the operating system 126, the communication module (or instruction set) 128, the contact / motion module (or instruction set) 130, the graphic module (or instruction set). Tactile feedback module (or instruction set) 133, text input module (or instruction set) 134, Global Positioning System (GPS) module (or instruction set) 135, and application (set) 132, tactile feedback module (or instruction set) 133, and application (or instruction set) 135. Alternatively, the instruction set) 136 is included. Further, in some embodiments, the memory 102 stores the device / global internal state 157, as shown in FIGS. 1A and 3. Device / global internal state 157 indicates, if any, which application currently active active application state, which application, view, or other information occupies various areas of the touch-sensitive display system 112. Includes one or more of display states, sensor states including information obtained from various sensors and other inputs or control devices 116 of the device, and position and / or position information relating to the position and / or orientation of the device. ..

The operating system 126 (eg, embedded operating system such as iOS, Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS, or VxWorks) is an overall system task (eg, memory management, storage device control, power management, etc.). Includes various software components and / or drivers for controlling and managing the operating system, facilitating communication between various hardware and software components.

The communication module 128 facilitates communication with other devices via one or more external ports 124 and includes various software components for processing the data received by the RF circuit 108 and / or the external port 124. .. External ports 124 (eg, Universal Serial Bus, USB, FIREWIRE, etc.) may be coupled to other devices either directly or indirectly via a network (eg, Internet, wireless LAN, etc.). Is adapted to. In some embodiments, the external port is an Apple Inc. of Cupertino, Calif. Same or similar and / or compatible multi-pin (eg, eg) 30-pin connectors used within some iPhone®, iPod Touch®, and iPad® devices from. 30-pin) connector. In some embodiments, the external port is an Apple Inc. of Cupertino, Calif. The same or similar and / or compatible Lightning connectors used within some iPhone®, iPod Touch®, and iPad® devices from.

The contact / motion module 130 optionally detects contact with the touch-sensitive display system 112 (in cooperation with the display controller 156) and contact with other touch-sensitive devices (eg, touchpad or physical click wheel). .. The contact / movement module 130 determines whether a contact has occurred (eg, detects a finger down event), determines the strength of the contact (eg, to a contact force or pressure, or to a contact force or pressure. Alternatives), determining if there is contact movement and tracking movement across the touch-sensitive surface (eg, detecting one or more finger drag events), and whether contact has stopped. Includes software components for performing various actions related to contact detection (eg, by a finger or stylus), such as determining (eg, detecting a finger-up event or interruption of contact). The contact / motion module 130 receives contact data from the touch sensing surface. Determining the movement of a contact point represented by a series of contact data is the speed (magnitude), velocity (magnitude and direction), and / or acceleration (change in magnitude and / or direction) of the contact point. Is included as an option. These actions optionally apply to a single contact (eg, one finger contact or stylus contact) or multiple simultaneous contacts (eg, "multi-touch" / multiple finger contacts). .. In some embodiments, the contact / motion module 130 and the display controller 156 detect contact on the touchpad.

The contact / motion module 130 optionally detects gesture input by the user. Different gestures on the touch-sensitive surface have different contact patterns (eg, different movements, timings, and / or intensities of detected contacts). Therefore, the gesture is optionally detected by detecting a specific contact pattern. For example, detecting a finger tap gesture detects a finger down event and then the finger at the same position (or substantially the same position) as the finger down event (eg, at the position of the icon). Includes detecting up (lift off) events. As another embodiment, finger swipe gesture detection on the touch sensitive surface detects a finger down event, followed by one or more finger drag events, followed by a finger up (lift off) event. Including doing. Similarly, taps, swipes, drags, and other gestures are optionally detected by detecting a particular contact pattern for the stylus with respect to the stylus.

In some embodiments, detecting a finger tap gesture depends on the length of time between detecting a finger down event and detecting a finger up event, but detecting a finger down event. It is not related to the strength of finger contact between doing and detecting a finger-up event. In some embodiments, the tap gesture allows the strength of finger contact between taps to meet a given strength threshold, such as a light or deep pressure threshold (greater than the nominal contact detection strength threshold). Regardless, the length of time between the finger down event and the finger up event is shorter than the predetermined value (eg, 0.1, 0.2, 0.3, 0.4 or 0.5). It is detected according to the judgment (shorter than seconds). This allows the finger tap gesture to meet input criteria configured to be satisfactory even if the characteristic strength of the contact does not meet a given strength threshold. For clarity, finger contact in tap gestures generally needs to meet the nominal contact detection intensity threshold below which no contact is detected in order to detect a finger down event. Similar analysis applies to detecting tap gestures or other contacts with the stylus. If the device is configured to detect finger or stylus contact hovering over the touch-sensitive surface, the nominal contact detection intensity threshold will optionally accommodate physical contact between the finger or stylus and the touch-sensitive surface. do not.

In some embodiments, detecting a finger tap gesture depends on the length of time between detecting a finger down event and detecting a finger up event, but detecting a finger down event. It is not related to the strength of finger contact between doing and detecting a finger-up event. In some embodiments, the tap gesture allows the strength of finger contact between taps to meet a given strength threshold, such as a light or deep pressure threshold (greater than the nominal contact detection strength threshold). Regardless, the length of time between the finger down event and the finger up event is shorter than the predetermined value (eg, 0.1, 0.2, 0.3, 0.4 or 0.5). It is detected according to the judgment (shorter than seconds). Thus, a finger tap gesture can meet a particular input criterion that does not require the characteristic strength of the contact to meet a given strength threshold in order to meet the particular input criterion. For clarity, finger contact in tap gestures generally needs to meet the nominal contact detection intensity threshold below which no contact is detected in order to detect a finger down event. Similar analysis applies to detecting tap gestures or other contacts with the stylus. In cases where the device can detect contact with a finger or stylus hovering over the touch-sensitive surface, the nominal contact detection intensity threshold is optional for physical contact between the finger or stylus and the touch-sensitive surface. Do not correspond.

In a similar fashion, the same concept applies to other types of gestures. For example, swipe gestures, pinch gestures, depinch gestures, and / or long pressing gestures are optional and are not related to the strength of the contact contained in the gesture, or the contact performing the gesture to be recognized reaches the strength threshold. Detected on the basis of meeting criteria that are either not required to do. For example, swipe gestures are detected based on the amount of movement of one or more contacts, pinch gestures are detected based on the movement of two or more contacts towards each other, and depinch gestures are two away from each other. Detected based on the above movement of contact, long pressing gestures are detected based on the duration of contact on the touch-sensitive surface, which is less than the threshold movement amount. Therefore, the statement that a particular gesture recognition criterion does not require that the strength of a contact (s) satisfy each intensity threshold in order to meet a particular gesture recognition criterion is a contact (s) in a gesture. It is possible that certain gesture recognition criteria are met if does not reach the respective intensity thresholds, and one or more of the contacts in the gesture do not reach or exceed the respective intensity thresholds. It means that it is possible to be satisfied in no situation. In some embodiments, the tap gesture detects finger-down and finger-up events within a predetermined time period, regardless of whether the contact is above or below the respective intensity thresholds over a predetermined time period. The swipe gesture is detected based on the determination that the contact movement is greater than a predetermined magnitude, even if the contact exceeds the respective intensity thresholds at the end of the contact movement. Even in embodiments where the detection of the gesture is influenced by the strength of the contact performing the gesture (eg, when the device detects a long press more quickly when the strength of the contact exceeds the strength threshold, or when the strength of the contact is higher. As long as the device delays the detection of tap input) and the criteria for recognizing gestures can be met in situations where the contact does not reach a certain intensity threshold (eg, when the amount of time required to recognize the gesture changes). (Even), the detection of those gestures does not require the contact to reach a particular strength threshold.

Contact strength thresholds, duration thresholds, and movement thresholds are in various different combinations to create heuristics to distinguish two or more different gestures directed to the same input element or region in some situations. Combined with, thereby enabling multiple different dialogues with the same input element to provide a richer set of user dialogues and responses. A statement that a particular set of gesture recognition criteria does not require that the strength of the contact (s) meet each intensity threshold in order for a particular gesture recognition criterion to be met is that the gesture exceeds each intensity threshold. It does not preclude simultaneous evaluation of other strength-dependent gesture recognition criteria to identify other gestures that have criteria that are met when including strong contacts. For example, in some situations, a first gesture for a first gesture that does not require the strength of the contact (s) to meet their respective strength thresholds in order to meet the first gesture recognition criterion. The gesture recognition criteria are in a competitive relationship with the second gesture recognition criteria for the second gesture, which depends on the contact (s) that reach each intensity threshold. In such competition, the gesture is optional, assuming that the first gesture recognition criterion for the first gesture is met if the second gesture recognition criterion for the second gesture is first met. Not recognized. For example, if the contact reaches each intensity threshold before the contact has moved a predetermined amount of movement, a deep pressing gesture is detected instead of a swipe gesture. Conversely, if the contact moves a predetermined amount of movement before the contact reaches each intensity threshold, a swipe gesture is detected instead of a deep pressing gesture. Even in such a situation, the first gesture recognition criterion for the first gesture is that the strength of the contact (s) satisfy the respective strength thresholds so that the first gesture recognition criterion is met. However, this is because if the contact remains below each intensity threshold until the end of the gesture (eg, a swipe gesture with a contact that does not increase to an intensity above each intensity threshold), the gesture This is because it is recognized as a swipe gesture by the first gesture recognition criterion. As such, certain gesture recognition criteria that do not require the strength of the contact (s) to meet their respective intensity thresholds in order for the particular gesture recognition criteria to be met are (A) in some situations. So ignore the strength of the contact with respect to the intensity threshold (eg for tap gestures) and / or (B) in some situations, a particular gesture recognition criterion corresponds to the input gesture (eg for recognition). Before recognizing a competing set of strength-dependent gesture recognition criteria (for example, for deep-pressing gestures), enter as corresponding to a strength-dependent gesture (for example, for deep-pressing gestures). When recognizing, it still depends on the strength of the contact with respect to the intensity threshold in the sense that certain gesture recognition criteria (eg, for long pressing gestures) do not work.

The graphic module 132 includes a touch-sensitive display system 112 or other display that includes components for changing the visual effects of the displayed graphic (eg, brightness, transparency, saturation, contrast, or other visual characteristics). Includes various known software components for rendering and displaying graphics on. As used herein, the term "graphic" includes, but is not limited to, text, web pages, icons (such as user interface objects including softkeys), digital images, videos, and animations. Contains any object that can be displayed against.

In some embodiments, the graphics module 132 stores data representing the graphics that will be used. Each graphic is optionally assigned a corresponding code. The graphic module 132 receives, if necessary, one or more codes specifying the graphic to be displayed together with coordinate data and other graphic characteristic data from an application or the like, and then outputs the code to the display controller 156. Generate screen image data.

The tactile feedback module 133 is used by the tactile output generator (s) 167 to generate tactile outputs at one or more locations on the device 100 in response to user interaction with the device 100. Includes various software components for generating instructions.

The text input module 134 is optionally a component of the graphic module 132 and is a variety of applications (eg, contacts 137, email 140, IM 141, browser 147, and any other application that requires text input. ) Provides a soft keyboard for entering text.

The GPS module 135 determines the position of the device and uses this information for use in various applications (eg, to the phone 138 for use in position-based phone calls, camera 143 as photo / video metadata. To applications that provide location-based services, such as weather widgets, regional yellow page widgets, and map / navigation widgets.

Application 136 optionally includes the following modules (or instruction sets), or subsets or supersets thereof.

Contact Module 137 (sometimes called an address book or contact list),

Telephone module 138,

Video conferencing module 139,

Email client module 140,

Instant Messaging (IM) Module 141,

Training support module 142,

Camera module 143 for still and / or video images,

Image management module 144,

Browser module 147,

Calendar module 148,

Optionally, weather widget 149-1, stock price widget 149-2, computer widget 149-3, alarm clock widget 149-4, dictionary widget 149-5, and other widgets acquired by the user, as well as user-created widget 149. Widget module 149, which contains one or more of -6,

Widget creation module 150 for creating user-created widgets 149-6,

Search module 151,

Optional video and music player module 152, which consists of a video player module and a music player module,

Memo module 153,

Map module 154 and / or

Online video module 155.

Examples of other applications 136 that are optionally stored in memory 102 include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA® compatible applications, encryption, digital writing. Rights management, voice recognition, and voice duplication.

Along with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the contact module 137 (eg, the application internal state 192 of the contact module 137 in the memory 102 or memory 370). Includes executable instructions for managing an address book or contact list (stored in), including adding names (s) to the address book and removing names (s) from the address book. , Phone number (s), email address (s), address (s), or other information associated with a name, image associated with a name, sorted by name, Includes providing a telephone number and / or e-mail address to initiate and / or facilitate communication via telephone 138, video conferencing 139, e-mail 140, or IM 141.

In cooperation with the RF circuit 108, the voice circuit 110, the speaker 111, the microphone 113, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the telephone module 138 becomes a telephone number. Enter the corresponding series of characters to access one or more phone numbers in the address book 137, change the phone numbers entered, dial each phone number, execute the conversation, and the conversation is complete. Includes actionable instructions to hang up or hang up when you do. As mentioned above, wireless communication optionally uses any of a plurality of communication standards, protocols, and technologies.

RF circuit 108, voice circuit 110, speaker 111, microphone 113, touch sensing display system 112, display controller 156, optical sensor (s) 164, optical sensor controller 158, contact module 130, graphic module 132, text input module 134. , Contact list 137, and telephone module 138, the video conferencing module 139 initiates, conducts, and terminates a video conferencing between the user and one or more other participants according to the user's instructions. Includes possible instructions.

In cooperation with the RF circuit 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the email client module 140 responds to user instructions by sending an email. Includes executable instructions to create, send, receive, and manage. Together with the image management module 144, the email client module 140 makes it very easy to compose and send an email with a still or video image taken by the camera module 143.

Together with the RF circuit 108, the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the instant message module 141 inputs the character string corresponding to the instant message and the input characters. And send each instant message (eg, using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocol for phone-based instant messaging, or Internet-based. Includes executable instructions for receiving instant messages and viewing the received instant messages (using XMPP, SIMPLE, Apple Push Notification Services (APNs), or IMPS) for instant messages. In some embodiments, the transmitted and / or received instant message is optionally a graphic, photo, audio file, video such as supported by MMS and / or Enhanced Messaging Service (EMS). Includes files and / or other attachments. As used herein, "instant messages" are telephone-based messages (eg, messages sent using SMS or MMS) and Internet-based messages (eg, XMPP, SIMPLE, APNs, or IMPS). Refers to both messages sent using.

Along with RF circuit 108, touch-sensitive display system 112, display controller 156, contact module 130, graphics module 132, text input module 134, GPS module 135, map module 154, and music player module 146, the training support module 142 is trained. Create (eg, have time, distance, and / or calorie burning targets), communicate with training sensors (in sports devices and smart watches), receive training sensor data, and use to monitor training Includes actionable instructions to calibrate the sensors to be used, select and play music for training, and display, store, and transmit training data.

Along with the touch-sensitive display system 112, display controller 156, optical sensor (s) 164, optical sensor controller 158, contact module 130, graphic module 132, and image management module 144, the camera module 143 is a still image or video (video). Includes executable instructions for capturing (including streams) and storing them in memory 102, changing the characteristics of still images or videos, and / or deleting still images or videos from memory 102.

Along with the touch-sensitive display system 112, display controller 156, contact module 130, graphic module 132, text input module 134, and camera module 143, the image management module 144 arranges and modifies still and / or video images (eg,). , Editing), or otherwise, including executable instructions for labeling, deleting, presenting (eg, in a digital slide show or album), and storing.

Along with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphic module 132, and the text input module 134, the browser module 147 was linked to a web page or a portion thereof, as well as a web page. Includes executable instructions for browsing the Internet according to user instructions, including searching, linking, receiving, and displaying attachments and other files.

Along with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphic module 132, the text input module 134, the email client module 140, and the browser module 147, the calendar module 148 follows the user's instructions. Includes executable instructions for creating, displaying, modifying, and storing the module and the data associated with the module (eg, calendar items, list of things to do, etc.).

Widget module 149, along with RF circuit 108, touch-sensitive display system 112, display system controller 156, contact module 130, graphic module 132, text input module 134, and browser module 147, is optionally downloaded and used by the user. Mini-applications (eg, weather widget 149-1, stock price widget 149-2, computer widget 149-3, alarm clock widget 149-4, and dictionary widget 149-5), or mini-applications created by the user (eg, user). Creation widget 149-6). In some embodiments, the widget includes HTML (Hypertext Markup Language) files, CSS (Cascading Style Sheets) files, and JavaScript files. In some embodiments, the widget includes an XML (Extensible Markup Language) file and a Javascript file (eg, Yahoo! Widget).

In cooperation with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphic module 132, the text input module 134, and the browser module 147, the widget creation module 150 creates a widget (eg, for example. Includes executable instructions (turning the user-specified part of a web page into a widget).

In conjunction with the touch-sensitive display system 112, display system controller 156, contact module 130, graphic module 132, and text input module 134, the search module 151 follows one or more search criteria (eg, one) according to user instructions. Includes executable instructions to search for text, music, sounds, images, videos, and / or other files in memory 102 that match the user-specified search terms above).

In cooperation with the touch-sensitive display system 112, display system controller 156, contact module 130, graphic module 132, audio circuit 110, speaker 111, RF circuit 108, and browser module 147, the video and music player module 152 can be MP3 or AAC. Display, present, or other executable instructions that allow the user to download and play recorded music or other sound files stored in one or more file formats, such as files, as well as videos. Includes executable instructions to play in a manner (eg, on a touch-sensitive display system 112, or on an external display connected wirelessly or via an external port 124). In some embodiments, the device 100 optionally includes features of an MP3 player such as an iPod (trademark of Apple Inc.).

In cooperation with the touch-sensitive display system 112, the display controller 156, the contact module 130, the graphic module 132, and the text input module 134, the memo module 153 creates and manages memos, a list of things to be done, and the like according to user instructions. Includes executable instructions to do.

In cooperation with the RF circuit 108, the touch-sensitive display system 112, the display system controller 156, the contact module 130, the graphic module 132, the text input module 134, the GPS module 135, and the browser module 147, the map module 154 follows the user's instructions. , Receives and displays maps and data associated with maps (eg, driving direction, data about stores at or near a particular location and other points of interest, and other data based on location). Includes executable instructions to modify and remember.

In cooperation with the touch-sensitive display system 112, display system controller 156, contact module 130, graphic module 132, voice circuit 110, speaker 111, RF circuit 108, text input module 134, email client module 140, and browser module 147, The online video module 155 is provided by the user with H. Access, view, receive (eg, by streaming and / or download) and play (eg, on touch screen 112, or wirelessly or external port 124) online video in one or more file formats, such as 264. Includes executable instructions that allow you to send an email with a link to a particular online video (on an external display connected via) and manage it in another way. In some embodiments, an instant messaging module 141 is used instead of an email client module 140 to send a link to a particular online video.

Each of the identified modules and applications has one or more functions described above, as well as the methods described herein (eg, methods performed by a computer, and other information described herein). Corresponds to an executable instruction set that executes (processing method). The modules (ie, instruction sets) do not need to be implemented as separate software programs, procedures, or modules, so various subsets of those modules are optionally combined in different embodiments. Or otherwise relocated. In some embodiments, the memory 102 optionally stores a subset of the modules and data structures identified above. In addition, memory 102 optionally stores additional modules and data structures not described above.

In some embodiments, the device 100 is a device in which a predetermined set of operations on the device is performed exclusively through the touch screen and / or touch pad. By using the touch screen and / or touchpad as the primary input control device for the operation of the device 100, the number of physical input control devices (push buttons, dials, etc.) on the device 100 is optionally reduced. Will be done.

A set of predetermined functions, performed exclusively through the touch screen and / or touchpad, includes, optionally, navigation between user interfaces. In some embodiments, the touchpad navigates the device 100 from any user interface displayed on the device 100 to the main menu, home menu, or root menu when touched by the user. In such an embodiment, the "menu button" is implemented using a touchpad. In some other embodiments, the menu button is not a touchpad, but a physical push button or other physical input control device.

FIG. 1B is a block diagram showing exemplary components for event processing, according to some embodiments. In some embodiments, the memory 102 (in FIG. 1A) or 370 (FIG. 3) is the event sorter 170 (eg, in operating system 126) and the respective application 136-1 (eg, applications 136, 137 described above). 155, 380 to 390).

The event sort unit 170 receives the event information and determines the application view 191 of the application 136-1 and the application 136-1 to which the event information is distributed. The event sort unit 170 includes an event monitor 171 and an event dispatcher module 174. In some embodiments, application 136-1 includes an application internal state 192 that indicates the current application view (s) displayed on the touch-sensitive display system 112 when the application is active or running. .. In some embodiments, the device / global internal state 157 is used by the event sorter 170 to determine which application (s) are currently active, and the application internal state 192 is the event. It is used by the event sorter 170 to determine the application view 191 to which the information is distributed.

In some embodiments, the application internal state 192 indicates resumption information used when application 136-1 resumes execution, information displayed or ready to be displayed by application 136-1. One of user interface state information, a state queue to allow the user to return to the previous state or view of application 136-1, and a redo / undo queue for previous actions taken by the user. Includes additional information such as the above.

The event monitor 171 receives event information from the peripheral device interface 118. Event information includes information about sub-events (eg, user touches on the touch-sensitive display system 112 as part of a multi-touch gesture). Peripheral interface 118 transmits information received from sensors such as the I / O subsystem 106, or proximity sensor 166, accelerometer (s) 168, and / or microphone 113 (via voice circuit 110). The information received by the peripheral interface 118 from the I / O subsystem 106 includes information from the touch-sensitive display system 112 or the touch-sensitive surface.

In some embodiments, the event monitor 171 sends a request to the peripheral interface 118 at predetermined intervals. In response, the peripheral interface 118 sends event information. In other embodiments, the peripheral interface 118 only has a significant event (eg, receiving an input that exceeds a predetermined noise threshold and / or receives an input longer than a predetermined period). Send event information.

In some embodiments, the event sort unit 170 also includes a hit view determination module 172 and / or an active event recognition unit determination module 173.

The hit view determination module 172 provides a software procedure for determining where in one or more views a sub-event has occurred when the touch-sensitive display system 112 displays two or more views. The view is composed of a control unit and other elements that the user can see on the display.

Another aspect of the user interface associated with an application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. .. The application view (for each application) where touch is detected is optional and corresponds to the program hierarchy of the application or the program level within the view hierarchy. For example, the lowest level view where a touch is detected is optional and is called a hit view, and the set of events that are recognized as appropriate input is optional and the initial touch to initiate a touch-based gesture. Judgment is based at least in part on the hit view of.

The hit view determination module 172 receives information related to the touch-based gesture sub-event. When the application has multiple views organized in the hierarchy, the hit view determination module 172 identifies the hit view as the lowest view in the hierarchy to handle subevents. In most situations, a hit view is the lowest level view at which a starting subevent (ie, the first subevent in a series of subevents forming an event or potential event) occurs. When a hit view is identified by the hit view determination module, the hit view typically receives all sub-events associated with the same touch or input source it is identified as a hit view.

The active event recognition unit determination module 173 determines which view (s) in the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognizer determination module 173 determines that only the hit view should receive a particular sequence of sub-events. In another embodiment, the active event recognizer determination module 173 determines that all views, including the physical position of the sub-event, are views that are involved in the active, and therefore all views that are involved in the active are Determines that a particular sequence of subevents should be received. In other embodiments, even if the touch subevent is completely confined to the area associated with a particular view, the views above the hierarchy remain the views that are still actively involved.

The event dispatcher module 174 sends event information to an event recognition unit (for example, event recognition unit 180). In the embodiment including the active event recognition unit determination module 173, the event dispatcher module 174 distributes the event information to the event recognition unit determined by the active event recognition unit determination module 173. In some embodiments, the event dispatcher module 174 stores the event information acquired by each event receiver module 182 in the event queue.

In some embodiments, the operating system 126 includes an event sorter 170. Alternatively, application 136-1 includes an event sort unit 170. In yet another embodiment, the event sorter 170 is part of a stand-alone module or another module stored in memory 102, such as the contact / motion module 130.

In some embodiments, application 136-1 includes a plurality of event handlers 190 and one or more application views 191 each containing instructions for processing touch events that occur within each view of the application's user interface. including. Each application view 191 of application 136-1 includes one or more event recognizers 180. Typically, each application view 191 includes a plurality of event recognizers 180. In other embodiments, one or more of the event recognizers 180 are separate, such as a user interface kit (not shown), or a higher level object from which application 136-1 inherits methods and other properties. Is part of the module. In some embodiments, each event processing unit 190 captures one or more of event data 179 received from data update unit 176, object update unit 177, GUI update unit 178, and / or event sort unit 170. Including. The event processing unit 190 uses or calls the data update unit 176, the object update unit 177, or the GUI update unit 178 in order to update the application internal state 192 at an arbitrary option. Alternatively, one or more of the application views 191 includes one or more respective event processing units 190. Further, in some embodiments, one or more of the data update unit 176, the object update unit 177, and the GUI update unit 178 is included in each application view 191.

Each event recognition unit 180 receives event information (for example, event data 179) from the event sort unit 170, and identifies an event from the event information. The event recognition unit 180 includes an event reception unit 182 and an event comparison unit 184. In some embodiments, the event recognizer 180 also includes at least a subset of metadata 183 and event delivery instructions 188, optionally including sub-event delivery instructions.

The event receiving unit 182 receives event information from the event sorting unit 170. Event information includes information about sub-events, such as touch or touch movement. Depending on the sub-event, the event information also includes additional information such as the location of the sub-event. When the sub-event involves touch movement, the event information is optional and includes the speed and direction of the sub-event. In some embodiments, the event involves rotation of the device from one orientation to another (eg, from portrait to landscape, or vice versa), and the event information is the current orientation of the device. Contains corresponding information about (also called device orientation).

The event comparison unit 184 compares the event information with the definition of the default event or sub-event, determines the event or sub-event based on the comparison, or determines or updates the state of the event or sub-event. In some embodiments, the event comparison unit 184 includes an event definition 186. Event definition 186 includes event definitions (eg, predetermined sequences of sub-events) such as event 1 (187-1), event 2 (187-2), and the like. In some embodiments, the sub-event in event 187 includes, for example, the beginning of a touch, the end of a touch, the movement of a touch, the cancellation of a touch, and a plurality of touches. In one embodiment, the definition for event 1 (187-1) is a double tap on the displayed object. Double tapping is, for example, a first touch (beginning of touch) for a predetermined stage on a displayed object, a first lift-off (end of touch) for a predetermined stage, a displayed object. Includes a second touch (beginning of touch) for the predetermined stage above and a second lift-off (end of touch) for the predetermined stage. In another embodiment, the definition of event 2 (187-2) is a drag on the displayed object. Dragging includes, for example, touching (or touching) a predetermined step on a displayed object, moving the touch across the touch-sensitive display system 112, and lifting off the touch (end of touch). In some embodiments, the event also includes information about one or more associated event handlers 190.

In some embodiments, the event definition 187 includes an event definition for each user interface object. In some embodiments, the event comparison unit 184 performs a hit test to determine the user interface object associated with the subevent. For example, in an application view in which three user interface objects are displayed on the touch-sensitive display system 112, when a touch is detected on the touch-sensitive display system 112, the event comparison unit 184 is among the three user interface objects. Run a hit test to determine which is associated with a touch (sub-event). When each displayed object is associated with the corresponding event processing unit 190, the event comparing unit uses the result of the hit test to determine which event processing unit 190 should be activated. For example, the event comparison unit 184 selects an event processing unit associated with an object that triggers a sub-event and a hit test.

In some embodiments, the definition of each event 187 also includes a delayed action that delays the delivery of event information until it is determined whether a series of sub-events corresponds to the event type of the event recognizer.

When each event recognition unit 180 determines that the series of sub-events does not match any of the events in the event definition 186, each event recognition unit 180 enters the state of event impossible, event failure, or event end. After that, it ignores the next subevent of the touch-based gesture. In this situation, if there are other event recognizers that remain active for the hit view, that event recognizer will continue to track and process sub-events of gestures based on ongoing touches.

In some embodiments, each event recognizer 180 presents configurable properties, flags, and / or to the event recognizer that is actively involved in how the event delivery system should perform sub-event delivery. Alternatively, it includes metadata 183 with a list. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists that indicate how event recognizers interact with each other or are allowed to interact with each other. In some embodiments, the metadata 183 includes configurable properties, flags, and / or lists indicating whether subevents are delivered to various levels in the view hierarchy or program hierarchy.

In some embodiments, each event recognition unit 180 activates the event processing unit 190 associated with an event when one or more specific sub-events of the event are recognized. In some embodiments, each event recognition unit 180 delivers event information associated with the event to the event processing unit 190. Activating the event processor 190 is different from sending (and deferring) a subevent to the corresponding hit view. In some embodiments, the event recognition unit 180 throws a flag associated with the recognized event, and the event processing unit 190 associated with the flag catches the flag and executes a predetermined process.

In some embodiments, the event delivery instruction 188 includes a sub-event delivery instruction that delivers event information about the sub-event without activating the event processing unit. Instead, the sub-event delivery instruction delivers event information to the event processor associated with the set of sub-events or the view involved in the activity. The event processing unit associated with the view involved in the series of sub-events or activities receives the event information and executes a predetermined process.

In some embodiments, the data updater 176 creates and updates the data used in application 136-1. For example, the data update unit 176 updates the telephone number used in the contact module 137 or stores the video file used in the video player module 145. In some embodiments, the object updater 177 creates and updates the objects used in application 136-1. For example, the object update unit 177 creates a new user interface object or updates the position of the user interface object. The GUI update unit 178 updates the GUI. For example, the GUI update unit 178 prepares the display information and sends the display information to the graphic module 132 for display on the touch-sensitive display.

In some embodiments, the event processing unit (s) 190 includes, or has access to, a data update unit 176, an object update unit 177, and a GUI update unit 178. In some embodiments, the data update unit 176, the object update unit 177, and the GUI update unit 178 are included in a single module of the respective application 136-1 or application view 191. In other embodiments, they are included in two or more software modules.

The above description of handling user touch events on a touch-sensitive display also applies to other forms of user input for operating the multifunction device 100 with an input device, all of which are on the touch screen. Please understand that it does not start with. For example, moving the mouse and pressing the mouse button arbitrarily to press or hold one or more keyboards, moving the touch such as tapping, dragging, scrolling on the touchpad, pen stylus input, moving the device, Oral commands Detected eye movements, biometric inputs, and / or any combination thereof are optionally used as inputs corresponding to sub-events that define the event to be recognized.

FIG. 2 shows a portable multifunction device 100 having a touch screen (eg, touch-sensitive display system 112, FIG. 1A), according to some embodiments. The touch screen optionally displays one or more graphics within the user interface (UI) 200. In this embodiment, and the embodiments described below, the user may see, for example, one or more fingers 202 (not drawn at an exact scale in the figure) or one or more stylus 203 (accurate in the figure). One or more of the graphics can be selected by making a gesture on the graphic using (not drawn at a good scale). In some embodiments, when the user breaks contact with one or more graphics, the selection of the one or more graphics occurs. In some embodiments, the gesture is in contact with one or more taps, one or more swipes (left to right, right to left, upward and / or downward), and / or device 100. Includes finger rolling (right-to-left, left-to-right, upward and / or downward), optionally. In some implementations or situations, unforeseen contact with the graphic does not select the graphic. For example, when the gesture corresponding to the selection is a tap, the swipe gesture that sweeps over the application icon is optional and does not select the corresponding application.

Device 100 also optionally includes one or more physical buttons such as "Home" or Menu Button 204. As mentioned above, menu button 204 is optionally used to navigate to any application 136 in a set of applications that is optionally executed on the device 100. Alternatively, in some embodiments, the menu button is implemented as a softkey in the GUI displayed on the touch screen display.

In some embodiments, the device 100 includes a touch screen display, a menu button 204, a push button 206 for turning power on / off the device and locking the device, and a volume control button (s). Includes 208, a subscriber identification module (SIM) card slot 210, a headset jack 212, and an external docking / charging port 124. The push button 206 powers on / off the device by pressing and holding the button for a predetermined time, pressing the button, and before the predetermined time interval elapses. May be used to lock the device and / or unlock the device or initiate the unlocking process by releasing. In some embodiments, the device 100 also accepts verbal input to activate or deactivate some functions through the microphone 113. The device 100 may optionally generate one or more contact strength sensors 165 for detecting contact strength on the touch-sensitive display system 112 and / or one or more tactile outputs for the user of the device 100. Also includes a tactile output generator 167.

FIG. 3 is a block diagram of an exemplary multifunctional device having a display and a touch sensitive surface, according to some embodiments. The device 300 does not have to be portable. In some embodiments, the device 300 is a laptop computer, desktop computer, tablet computer, multimedia playback device, navigation device, educational device (such as a children's learning toy), gaming system, or control device (eg, home). Computer for business use or business use). The device 300 typically comprises one or more processing units (CPUs) 310, one or more networks or other communication interfaces 360, memory 370, and one for interconnecting these components. Includes one or more communication buses 320. The communication bus 320 optionally includes a circuit (sometimes referred to as a chipset) that interconnects and controls communication between system components. The device 300 includes an input / output (I / O) interface 330, including a display 340, which is typically a touch screen display. The I / O interface 330 is optionally a keyboard and / or mouse (or other pointing device) 350, and a touchpad 355 (eg, the tactile output generator described above with reference to FIG. 1A). Tactile output generator 357, sensor 359 (eg, optical sensor, acceleration sensor, proximity sensor, touch sensing sensor, and / or figure) for generating tactile output on device 300 (similar to s) 167. It also includes a contact strength sensor similar to the contact strength sensor (s) 165 described above with reference to 1A. Memory 370 includes high-speed random access memory such as DRAM, SRAM, DDR RAM, or other random access solid state memory device, optionally one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, and the like. Or includes non-volatile memory such as other non-volatile solid state storage devices. The memory 370 optionally includes one or more storage devices located remotely from the CPU (s) 310. In some embodiments, the memory 370 is a program, module, and data structure similar to the program, module, and data structure stored in memory 102 of the portable multifunction device 100 (FIG. 1A), or a subset thereof. Remember. Further, the memory 370 optionally stores additional programs, modules, and data structures that do not exist in the memory 102 of the portable multifunction device 100. For example, the memory 370 of the device 300 optionally stores a drawing module 380, a presentation module 382, a word processing module 384, a website creation module 386, a disk authoring module 388, and / or a spreadsheet module 390, but is portable. The memory 102 of the functional device 100 (FIG. 1A) optionally does not store those modules.

Each of the elements in FIG. 3 identified above is optionally stored in one or more of the previously mentioned memory devices. Each of the identified modules corresponds to an instruction set that performs the functions described above. The identified modules or programs (ie, instruction sets) need not be implemented as separate software programs, procedures, or modules, so various subsets of those modules are optional and various embodiments. Combined or otherwise rearranged in. In some embodiments, memory 370 optionally stores a subset of the modules and data structures identified above. In addition, memory 370 optionally stores additional modules and data structures not described above.

Here, attention will be paid to an embodiment of a user interface (“UI”) that is arbitrarily implemented on the portable multifunction device 100.

FIG. 4A shows an exemplary user interface for menus of applications on the portable multifunction device 100, according to some embodiments. A similar user interface is optionally implemented on the device 300. In some embodiments, the user interface 400 includes the following elements, or a subset or superset thereof.

Signal strength indicator (s) for wireless communication (s) such as cellular and Wi-Fi signals 402,

Time 404,

Bluetooth® Indicator 405,

Battery status indicator 406,

Tray 408, which contains icons for frequently used applications, such as:

Icon 416 for the telephone module 138 labeled "Telephone", optionally including an indicator 414 for the number of missed calls or voicemail messages,

Icon 418 for Email Client Module 140, labeled "Email", which optionally contains an indicator 410 for the number of unread emails,

Icon 420 for browser module 147, labeled "browser", and

Icon 422 for the video and music player module 152, also known as the iPod® module 152, labeled "iPod"

Icons for other applications, such as:

Icon 424 for IM module 141, labeled "Message",

Icon 426 for calendar module 148, labeled "calendar",

Icon 428 for image management module 144, labeled "Photo",

Icon 430 for camera module 143, labeled "Camera",

Icon 432 for Online Video Module 155, labeled "Online Video",

Icon 434 for Stock Widget 149-2, labeled "Stock",

Icon 436 for map module 154, labeled "Map",

Icon 438 for Weather Widget 149-1, labeled "Weather",

Icon 440 for alarm clock widget 149-4, labeled "Clock",

Icon 442 for Training Support Module 142, labeled "Training Support",

Icon 444 for memo module 153, labeled "Memo", and

Icon 446 for a configuration application or module that provides access to settings for device 100 and its various applications 136.

Note that the icon label shown in FIG. 4A is just an example. For example, in some embodiments, the icon 422 for the video and music player module 152 is labeled "Music" or "Music Player". Other labels are optionally used for various application icons. In some embodiments, the label for each application icon includes the name of the application corresponding to each application icon. In some embodiments, the label of a particular application icon is different from the name of the application that corresponds to the particular application icon.

FIG. 4B shows an exemplary user interface on a device (eg, device 300, FIG. 3) having a touch sensing surface 451 (eg, tablet or touchpad 355, FIG. 3) separated from the display 450. The device 300 is optionally for one or more contact strength sensors (eg, one or more of the sensors 357) for detecting the strength of contact on the touch sensing surface 451 and / or for the user of the device 300. Also included is one or more tactile output generators 357 for generating tactile outputs.

FIG. 4B shows an exemplary user interface on a device (eg, device 300, FIG. 3) having a touch sensing surface 451 (eg, tablet or touchpad 355, FIG. 3) separated from the display 450. Many of the following examples are given with reference to the inputs on the touch screen display 112 (when the touch sensitive surface and the display are combined), but in some embodiments the device is shown in FIG. 4B. As shown in, the input on the touch sensing surface separated from the display is detected. In some embodiments, the touch sensitive surface (eg, 451 in FIG. 4B) has a spindle (eg, 452 in FIG. 4B) that corresponds to a spindle on the display (eg, 450) (eg, 453 in FIG. 4B). Have. According to those embodiments, the device contacts the touch sensing surface 451 at a position corresponding to each position on the display (eg, in FIG. 4B, 460 corresponds to 468 and 462 corresponds to 470). (For example, 460 and 462 in FIG. 4B) are detected. Thus, when the touch-sensitive surface is separated from the display, user inputs (eg, contacts 460 and 462, and their movements) detected by the device on the touch-sensitive surface (eg, 451 in FIG. 4B) , Used by the device to manipulate the user interface on the display of a multifunctional device (eg, 450 in FIG. 4B). It should be understood that similar methods are optionally used for the other user interfaces described herein.

In addition, the following examples are given primarily with reference to finger input (eg, finger contact, finger tap gestures, finger swipe gestures, etc.), but in some embodiments, those fingers It should be understood that one or more of the inputs will be replaced with inputs from another input device (eg, mouse-based input or stylus input). For example, a swipe gesture is optionally replaced with a mouse click (eg, instead of a touch), after which the cursor is moved along the swipe path (eg, instead of a touch). As another example, the tap gesture is optional and is replaced by a mouse click while the cursor is hovering over the position of the tap gesture (for example, instead of stopping contact detection followed by contact detection). .. Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice are optionally used simultaneously, or mouse and finger contact is optionally used simultaneously.

As used herein, the term "focus selector" refers to an input element that indicates the current part of the user interface with which the user is interacting. In some embodiments, including a cursor or other location marker, the cursor is touch-sensitive while the cursor is over a particular user interface element (eg, a button, window, slider, or other user interface element). When an input (eg, a pressing input) is detected on a surface (eg, touch pad 355 in FIG. 3 or touch sensing surface 451 in FIG. 4B), certain user interface elements are adjusted according to the detected input. In addition, it functions as a "focus selector". In some embodiments, including a touch screen display that enables direct interaction with user interface elements on the touch screen display (eg, the touch sensitive display system 112 in FIG. 1A, or the touch screen in FIG. 4A), touch. The contact detected on the screen is that the input (eg, touch-pressed input) is detected on the touch screen display at the location of a particular user interface element (eg, a button, window, slider, or other user interface element). It then acts as a "focus selector" so that certain user interface elements are adjusted according to the detected inputs. In some implementations, the focus is (eg, by moving the focus from one button to another using the Tab key or arrow keys) without moving the corresponding cursor or touch on the touch screen display. , Moved from one area of the user interface to another area of the user interface. In these implementations, the focus selector moves as the focus moves between different areas of the user interface. Regardless of the specific form taken by the focus selector, the focus selector is generally used to communicate the user's intended interaction with the user interface (eg, the user interface that the user intends to interact with). By showing the element to the device), a user interface element (or contact on a touch screen display) controlled by the user. For example, when a pressing input is detected on a touch sensitive surface (eg, touchpad or touch screen), the position of the focus selector (eg, cursor, touch or selection box) above the corresponding button is (device). Indicates that the user is trying to activate the corresponding button (rather than the other user interface elements shown on the display).

As used herein and in the claims, the term "strength" of contact on a touch-sensitive surface refers to the force or pressure of contact on a touch-sensitive surface (eg, finger contact or stylus contact). Refers to (force per unit area), or an alternative proxy for contact force or pressure on the touch-sensitive surface. The contact strength has a range of values, including at least 4 different values, and more typically hundreds (eg, at least 256) different values. Contact strength is optionally determined (or measured) using various methods and various sensors or combinations of sensors. For example, one or more force sensors below or adjacent to the touch sensitive surface are optionally used to measure forces at various points on the touch sensitive surface. In some implementations, force measurements from multiple force sensors are combined to determine the estimated contact force (eg, weighted average or sum). Similarly, the pressure sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch sensitive surface. Alternatively, the size and / or its change of the contact area detected on the touch sensing surface, the capacitance and / or its change of the touch sensing surface close to the contact, and / or the resistance of the touch sensing surface close to the contact and / or / Or its change is optionally used as an alternative to the force or pressure of contact on the touch sensitive surface. In some embodiments, an alternative measure of contact force or pressure is used directly to determine if the intensity threshold is exceeded (eg, the intensity threshold is the unit corresponding to the alternative measurement). Described in). In some embodiments, alternative measurements of contact force or pressure are converted to an estimated force or pressure and used to determine if the estimated force or pressure exceeds an intensity threshold. (For example, the intensity threshold is a pressure threshold measured in units of pressure). Using contact strength as an attribute of user input is to display affordances (eg, on a touch-sensitive display) and / or user input (eg, touch-sensitive display, touch-sensitive surface, or knob or button, etc.) Additional device capabilities that may otherwise not be easily accessible by the user in a device of reduced size with a limited area to receive (via physical / mechanical control of). Allows user access to.

In some embodiments, the contact / motion module 130 is one for determining whether an action has been performed by the user (eg, for determining whether the user has "clicked" on an icon). Use the above set of intensity thresholds. In some embodiments, at least a subset of intensity thresholds are determined according to software parameters (eg, intensity thresholds are not determined by the activation threshold of a particular physical actuator and modify the physical hardware of device 100. May be adjusted without). For example, the mouse "click" threshold of a trackpad or touch screen display can be set to any of a wide range of predetermined thresholds without changing the hardware of the trackpad or touch screen display. In addition, some implementations allow the user of the device to adjust multiple intensity thresholds at once (eg, by adjusting individual intensity thresholds and / or by a system-level click "intensity" parameter. By providing software settings for adjusting one or more of the set of intensity thresholds.

As used herein and in the claims, the term "characteristic strength" of a contact refers to the characteristic of the contact based on one or more strengths of the contact. In some embodiments, the characteristic intensity is based on multiple intensity samples. The characteristic strength is optionally for a predetermined number of strength samples, or for a given event (eg, after detecting a contact, before detecting a lift-off of a contact, before or after detecting the start of movement of a contact, or after a contact. A predetermined period of time (eg, 0.05, 0.1, before or after detecting the end, before or after detecting an increase in contact strength, and / or before or after detecting a decrease in contact strength) Based on a set of intensity samples collected within 0.2, 0.5, 1, 2, 5, 10 seconds). The characteristic strength of contact is the maximum value of contact strength, the average value of contact strength (mean value), the average value of contact strength (average value), the top 10% value of contact strength, the maximum half value of contact strength, and the contact. It is optionally based on any one or more of the values of up to 90% of the intensity. In some embodiments, the duration of contact is used to determine the characteristic strength (eg, when the characteristic strength is the average of the contact strength over time). In some embodiments, the characteristic intensity is compared to one or more sets of intensity thresholds to determine if the operation was performed by the user. For example, a set of one or more intensity thresholds can include a first intensity threshold and a second intensity threshold. In this embodiment, the first operation is performed as a result of a contact having a characteristic strength not exceeding the first threshold value, and the contact having a characteristic strength exceeding the first intensity threshold value and not exceeding the second intensity threshold value. As a result, the second operation is performed, and as a result of the contact having a characteristic strength exceeding the second intensity threshold, the third operation is executed. In some embodiments, the comparison between the characteristic intensity and one or more intensity thresholds is not used to determine whether to perform the first or second operation, but 1 It is used to determine whether to perform one or more actions (eg, perform each option or refrain from performing each action).

In some embodiments, some of the gestures are specified for the purpose of determining characteristic intensity. For example, the touch-sensitive surface may receive a continuous swipe contact (eg, drag gesture) that transitions from the start position to reach the end position, at which end position the contact strength increases. In this embodiment, the characteristic strength of the contact at the end position may be based on only a portion of the swipe contact (eg, only the portion at the end position of the swipe contact) rather than the entire continuous swipe contact. In some embodiments, a smoothing algorithm may be applied to the strength of the swipe contact before determining the characteristic strength of the contact. For example, the smoothing algorithm optionally includes one or more of an unweighted moving average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and / or an exponential smoothing algorithm. In some situations, those smoothing algorithms remove a small increase or decrease in the strength of the swipe contact for the purpose of determining the characteristic intensity.

FIG user interface described herein may optionally, one or more intensity threshold (e.g., the contact detection intensity threshold IT _0, light pressure intensity threshold IT _L, deep pressure intensity threshold IT D _(e.g., at least the first shows the current intensity of the contact on the I greater than _L), and / or one or more other intensity threshold (e.g., for low intensity threshold I _H) than I _L, the touch-sensitive surface, a variety of Includes an intensity diagram. This intensity diagram is typically not part of the displayed user interface, but is provided to aid in the interpretation of the diagram. In some embodiments, a light press. The intensity threshold corresponds to the intensity with which the device performs an operation typically associated with clicking a physical mouse button or trackpad. In some embodiments, a deep pressing intensity threshold is a physical mouse. Corresponds to the intensity with which the device performs an action that is different from the action typically associated with clicking a button or trackpad. In some embodiments, it is below a light pressing intensity threshold (eg, and it). When contact is detected at a characteristic strength (above the nominal contact detection strength threshold IT ₀ ), the device performs an action associated with a light or deep pressure threshold. Without moving the focus selector as the contact moves on the touch sensitive surface. In general, these intensity thresholds are consistent across different sets of user interface diagrams, unless otherwise stated.

In some embodiments, the device's response to the input detected by the device depends on a criterion based on the contact strength during the input. For example, for some "light press" inputs, a contact intensity above the first intensity threshold during the input triggers a first response. In some embodiments, the device's response to the input detected by the device depends on a criterion that includes both contact strength and time-based criteria during the input. For example, for some "deep press" inputs, the intensity of the contact above the second intensity threshold, which is greater than the first intensity threshold for the light press during the input, satisfies the first intensity threshold and the second intensity. Only if the delay time elapses between satisfying the thresholds will the second response be triggered. This delay time is typically less than 200 ms in duration (eg, 40 ms, 100 ms, or 120 ms, depending on the magnitude of the second intensity threshold, and the delay time increases the second intensity threshold. Increases as you go). This delay time helps prevent accidental deep press input. As another embodiment, for some "deep press" inputs, there is a desensitized time period that occurs after the time when the first intensity threshold is met. The second intensity threshold increases during the time period during which the sensitivity is reduced. This temporary increase in the second intensity threshold also helps avoid accidental deep pressing inputs. For other deep press inputs, the response to detection of deep press inputs does not depend on time-based criteria.

In some embodiments, one or more of the input intensity thresholds and / or the corresponding outputs are user settings, contact movements, input timings, applications being executed, the rate at which the intensity is applied, and simultaneous inputs. It varies based on one or more factors such as number, user history, environmental factors (eg, ambient noise), focus selector position, and so on. Exemplary factors are described in US Patent Application Publication Nos. 14 / 399,606 and 14 / 624,296, all of which are incorporated herein by reference.

For example, FIG. 4C shows a dynamic intensity threshold 480 that changes over time based in part on the intensity of the touch input 476 over time. The dynamic intensity threshold 480 is the sum of the two components, the first component 474 decays over time after a predetermined delay time p1 from the time the touch input 476 is first detected, and the second component. The 478 tracks the intensity of the touch input 476 over time. The initial high intensity threshold of the first component 474 reduces accidental triggering of a "deep press" response and an immediate "deep press" response if the touch input 476 provides sufficient intensity. Is still possible. The second component, 478, reduces the unintentional triggering of a "deep press" response by the gradual intensity variation of the touch input. In some embodiments, touch on

FIG. 4D shows another dynamic intensity threshold 486 (eg, intensity threshold _ID ). Figure 4D, two other intensity threshold, also the first intensity threshold I _H and a second intensity threshold I _L shown. In FIG. 4D, meet the first intensity threshold I _H and a second intensity threshold I _L in front of the touch input 484 time p2, but not provided a response until the delay time has elapsed p2 is at time 482. Further, in FIG. 4D, the dynamic strength threshold 486 begins to time 488 after the predetermined delay time p1 has elapsed from the time 482 (when the response associated with the second intensity threshold I _L has been triggered) attenuated And it decays over time. This type of dynamic intensity threshold, such as the first intensity threshold I _H or the second intensity threshold I _L, immediately after triggering a response that is associated with a lower intensity threshold, or at the same time, the dynamic intensity threshold I Reduces accidental triggering of the response associated with _D.

FIG. 4E shows yet another dynamic intensity threshold 492 (eg, intensity threshold _ID ). In FIG. 4E, the response associated with the intensity threshold I _L, the delay time p2 is triggered after the elapse from the time when the touch input 490 is first detected. At the same time, the dynamic intensity threshold 492 is attenuated after a predetermined delay time p1 has elapsed since the touch input 490 was first detected. Therefore, without releasing the touch input 490, after triggering a response that is associated with the intensity threshold I _L, reduction of the strength of the touch input 490 increases the strength of the touch input 490 is followed thereafter, touch input 490 another strength threshold intensity of, for example, even when below the intensity threshold I _L, it is possible to trigger a response (e.g., at time 494) associated with the intensity threshold I _D.

From the intensity below the light press intensity threshold IT _L, increase in the characteristic strength of contact to strength between the light press intensity threshold IT _L and deep pressure intensity threshold IT _D may be referred to as "light press" Enter .. Increase in characteristic intensity of the contact from the intensity below the deep pressing intensity threshold IT _D to strength exceeding deep pressure intensity threshold IT _D may be referred to as "deep pressing" Enter. Increasing characteristic strength of contact to the intensity between the intensity below the contact detection intensity threshold IT ₀ contact detection intensity threshold IT ₀ and light pressure intensity threshold IT _L is sometimes referred to as the detection of the contact on the touch surface is there. Reduction characteristics intensity of contact from strength exceeding contact detection intensity threshold IT ₀ to intensity below the contact detection intensity threshold IT ₀ may be referred to as a lift-off detection of the contact from the touch surface. In some embodiments, IT ₀ is zero. In some embodiments, IT ₀ is above zero. In some examples, shaded circles or ellipses are used to represent the strength of contact on the touch sensitive surface. In some examples, unshaded circles or ellipses are used to represent each contact on the touch-sensitive surface without specifying the strength of each contact.

In some embodiments described herein, one or more actions are performed in response to the detection of a gesture containing each pressing input, or at each contact (or multiple contacts). Each press input is executed in response to the detection of each press input, and each press input is at least partially based on the detection of an increase in the intensity of a contact (or multiple contacts) above the press input intensity threshold. Is detected. In some embodiments, each action is performed in response to detecting an increase in the strength of each contact above the press input strength threshold (eg, each action is "for example, each action is a" Performed on "downstroke"). In some embodiments, the press input comprises an increase in the intensity of each contact above the press input intensity threshold and a subsequent decrease in the intensity of the contact below the press input intensity threshold, and each action comprises a press input threshold. It is performed in response to detecting a subsequent decrease in the strength of each contact below (eg, each action is performed on the "upstroke" of each press input).

In some embodiments, the device employs intensity hysteresis to avoid an unforeseen input, sometimes referred to as "jitter," and the device has a hysteresis intensity that has a predetermined association with the press input intensity threshold. Define or select a threshold (eg, the hysteresis intensity threshold has a lower X intensity unit than the press input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some reasonable ratio of the press input intensity threshold. Is). Thus, in some embodiments, the press input comprises an increase in the intensity of each contact above the press input intensity threshold and a subsequent decrease in the intensity of the contact below the hysteresis intensity threshold corresponding to the press input intensity threshold. Each action is performed in response to detecting a subsequent decrease in the strength of each contact below the hysteresis strength threshold (eg, each action is performed on the "upstroke" of each press input. Will be). Similarly, in some embodiments, the press input causes the device to increase the intensity of contact from an intensity below the hysteresis intensity threshold to an intensity greater than or equal to the press input intensity threshold, and optionally to an intensity below the hysteresis intensity. Is detected only when a decrease in the strength of the subsequent contact is detected, and each action detects the pressing input (for example, an increase in the strength of the contact or a decrease in the strength of the contact depending on the situation). In response to that, it is executed.

For ease of explanation, the description of the action performed in response to a press input associated with a press input intensity threshold or in response to a gesture containing a press input is optional and exceeds the press input intensity threshold. Increased contact strength, increased contact strength from strength below the hysteresis strength threshold to strength above the pressing input strength threshold, decreased contact strength below the pressing input strength threshold, or hysteresis strength threshold corresponding to the pressing input strength threshold Triggered in response to the detection of a decrease in contact strength, below. Further, in an embodiment described as performing an action in response to detecting a decrease in contact strength below the press input intensity threshold, the operation corresponds to and is greater than the press input intensity threshold. It is optionally performed in response to the detection of a decrease in contact strength below the low hysteresis strength threshold. As mentioned above, in some embodiments, the triggering of their response also depends on a time-based criterion (eg, the delay time is that the first intensity threshold is met and the second intensity is met. Elapsed between the threshold being met).
User interface and associated processing

Here, a user interface that can be implemented on an electronic device such as a portable multifunction device 100 or device 300, comprising one or more sensors for detecting the strength of contact between the display and the touch sensing surface and the touch sensing surface. Attention is paid to (“UI”) and the embodiments of the associated process.

5A-5BJ show exemplary user interfaces for accessing common device functions, according to some embodiments. The user interfaces in those figures are used to illustrate the processes described below, including the processes in FIGS. 6A-6E, 7A-7E, 8A-8C, and 9A-9E. .. Some of the following examples will be described with reference to inputs on a touch screen display (combining a touch sensitive surface and a display), but in some embodiments the device is shown in FIG. 4B. As shown in, the input is detected on the touch sensing surface 451 separate from the display 450.

5A-1 to 5A-4 show activation of a display (eg, touch-sensitive display 112) in response to a reorientation of device 100. In some embodiments, the device uses one or more sensors (eg, accelerometer, gyro, audio sensor, thermal sensor, and / or optical sensor) to determine if the orientation of the device has changed. use. For example, the device is rotated along the axis of the device, such as whether the device has been rotated beyond a threshold angle (eg, tilted from a position where the device is substantially horizontal to a position where the device is substantially vertical). Taka) Judge whether or not. In FIG. 5A-1, the device is held flat within the user's hand 502 so that the device display is substantially horizontal. In FIG. 5A-2, the device is tilted so that the display is more vertical than FIG. 5A-1. In FIG. 5A-2, the display is not activated because the tilt angle of the device has not increased above the threshold tilt angle. In FIG. 5A-3, the device is tilted so that the display is more vertical than in FIG. 5A-2. In FIG. 5A-3, the display is activated (eg, display content 504 is displayed by the display) because the tilt angle of the device is increasing above the threshold tilt angle. Display content 504 includes, for example, time, date, signal indicators, battery charge level indicators, lock indicators, and / or one or more notifications (such as notification 506). In FIG. 5A-4, the display is substantially vertical. In some embodiments, when the display is activated (eg, as shown in 5A-3), some or all of the display content 504 is smaller than the default size of the display content 504 (eg, more). Narrow width). As the device continues to tilt after the display is activated, the display content 504 gradually increases in size to the default size, as shown in FIG. 5A-4.

5C-5E show a user interface displayed in response to the input provided by contact 508 on physical button 204 (eg, "home" or menu button). In FIG. 5B, the physical button 204 activates the physical button (eg, when the contact 508 rests on the surface of the physical button without pressing the physical button), but does not activate the contact 508. To detect. For example, the physical button 204 does not meet the criteria for activating the physical button 204. The contact 508 at the physical button 204 (eg, as indicated by the actuation meter 510, the characteristic intensity of the contact 508 is the contact detection intensity threshold IT _0. greater than, but according to the detection of not exceeding) the activation threshold IT _a, is activated. In some embodiments, activation of the physical button 204 wakes the device and / or the touch screen display 112. For example, in response to the activation of physical button 204, the device activates the display and displays the wake screen user interface 512, as shown in FIG. 5C. For example, a device is in a waked (display on) state, but is not fully activated in response to an input that activates a physical button but does not activate it.

In some embodiments, the strength of contact 508 with the physical button 204 is the actuation threshold IT, as shown by the actuation meter 510 in FIG. 5D, in response to a contact that meets the criteria for physical button activation (eg, as shown by the actuation meter 510 in FIG. 5D). _{(Exceeding A} ), the device 100 displays a user interface (eg, application springboard user interface 400) that includes a plurality of application icons (eg, application icons 416-446), as shown in FIG. 5E. For example, the device is fully activated in response to an input that activates a physical button.

In some embodiments, the physical button 204 includes a biometric sensor, such as a fingerprint sensor for a user verification scan (eg, Touch ID scan). In some embodiments, activation and / or activation of the physical button unlocks the device in response to successful verification of the user by fingerprint recognition corresponding to contact 508 by the biometric sensor of the physical button. .. For example, the unlocked state of the lock icon 514 is displayed on the wake screen interface 512 of FIG. 5C in response to the activation of the physical button 204, which has been successfully verified by the user.

In some embodiments, device 100 includes a virtual button (eg, instead of a physical button) that functions as a home button or menu button 204. It will be appreciated that as button 204 is described herein, mechanical buttons, virtual buttons, or other types of buttons may be used. For example, the virtual button is activated according to the determination that the characteristic intensity of contact with the virtual button does not exceed the operating threshold, and the virtual button is activated according to the determination that the characteristic intensity of contact with the virtual button exceeds the operating threshold. To. In some embodiments, the virtual button comprises a biometric sensor for user verification.

5F-1 to 5F-4 are sequences of user interfaces displayed in response to inputs detected while device 100 is in lock mode (eg, as indicated by the lock state of lock icon 514). Is shown. The wake screen interface 512 is displayed in FIG. 5F-1. In some embodiments, for example, in response to an input on pushbutton 206 for turning the device on and off to lock the device (eg, a user whose fingerprint is not recognized by the fingerprint scanner on physical button 204). Does not unlock the device (according to) Locking the wake screen interface 512 in response to input on the physical button 204 and / or activating the device display but not unlocking the device The mode is displayed.

FIG. 5F-1 shows a downward swipe gesture in which the contact 516 moves along the path indicated by the arrow 518. In response to the downward swipe gesture, the lock mode of the notification user interface 520 is displayed, as shown in FIG. 5F-2. In some embodiments, when a downward swipe occurs, an animation is displayed in which the notification user interface 520 appears to gradually descend from the top of the display.

As shown in FIG. 5F-3, the user interface 526 with the keyboard 528 is displayed in response to the input by the contact 522 detected at the position corresponding to the search input area 524 in FIG. 5F-2.

In FIG. 5F-3, the search term (“movie”) is entered in the search input area 524 (using, for example, the keyboard 528). In response to the input to initiate the search (eg, the input by the contact 530 detected by the affordance corresponding to the "return" key on the keyboard 528), the search result user interface 532, as shown in FIG. 5F-4. The lock mode is displayed.

The user may want to limit the information displayed on the device when locked so that no one else can see the personal and / or confidential information. In some embodiments, the device is configured so that certain types of notifications, notification content, and / or search results that are displayed when the device is unlocked are not visible when the device is locked. be able to. 5F-5 to 5F-8 show additional content displayed in the unlock mode of the notification user interface 520 and the search result user interface 532.

5F-5 to 5F-8 are a sequence of user interfaces displayed in response to inputs detected while device 100 is in unlock mode (eg, indicated by unlock mode of lock icon 514). Is shown. In some embodiments, the wake screen interface 512 is unlocked in response to an input (eg, by a user whose fingerprint is recognized by the fingerprint scanner of the physical button 204) on the physical button 204 that unlocks the device. The mode is displayed.

FIG. 5F-5 shows a downward swipe gesture in which the contact 534 moves along the path indicated by the arrow 536. In response to the downward swipe gesture, the unlock mode of the notification user interface 520 is displayed, as shown in FIG. 5F-6.

In response to input by contact 538 detected at a position corresponding to the search input area 524 in FIG. 5F-6, a user interface 526 with a keyboard 528 is displayed, as shown in FIG. 5F-7.

In FIGS. 5F-7, the search term (“movie”) is entered in the search input area 524 (using, for example, the keyboard 528). Unlocking the search result user interface 532, as shown in FIG. 5F-8, in response to input to initiate a search (eg, input by contact 540 detected by affordance, "return" key on keyboard 528). The mode is displayed.

FIG. 5G shows the lock mode of the notification user interface 520 (eg, as indicated by the lock state of the lock icon 514). The lock mode of the message notification 542 and the lock mode of the voice mail notification 544 are displayed in the lock mode of the notification user interface 520.

FIG. 5H shows the unlock mode of the notification user interface 520 (eg, as indicated by the unlock state of the lock icon 514). The unlock mode of the message notification mode 542 and the unlock mode of the voice mail notification 544 are displayed in the unlock mode of the notification user interface 520. Compared to the lock mode of the message notification 542, the unlock mode of the message notification 542 includes additional information such as message content (eg, text 546 and / or image 548 from one or more messages). The unlock mode of the notification user interface 520 displays an invitation notification 550 that is not displayed in the lock mode of the notification user interface 520.

FIG. 5I shows the lock mode of the search result user interface 532 (eg, as indicated by the lock state of the lock icon 514). Search results for the search term "movie" entered in the search input area 524 The search results displayed in the lock mode of the user interface 532 are, for example, recommended to be shown in position-based results (eg, near the location of the device). Includes movies, cinema suggestions closer to the location of the device), application results (iMovie®), recommended website results, and more.

FIG. 5J shows the search result user interface 532 (eg, unlock mode as indicated by the unlock state of the lock icon 514). The unlock mode of the notification user interface 532 displays messages 552 and 554 and email 556 that are not displayed in the lock mode of the search result user interface 532.

5K-5N show a sequence in which the user is prompted to provide a password to view the extended view of message notification 542 in response to input received in the lock mode of message notification 542. In FIG. 5K, input is provided by contact 557 to extend message notification 542. Since the device is in lock mode (as indicated by the lock state of lock icon 514), user verification is required before the extended view of message notification 542 is shown. FIG. 5L shows a user verification prompt user interface 558 including a passcode input interface 560. Notification 542, as shown in FIG. 5M, according to the determination that the user verification information is valid (provided, for example, via contact 562 in the passcode input interface 560 and / or input by the fingerprint sensor of the physical button 204). An expanded view of is displayed.

In some embodiments, the extended view of notification 542 is, for example, the message sender's avatar 564 (and / or the message sender's name), the affordance 566 for canceling the notification (eg, "X" affordance), and the like. Message content (eg, a message containing image 548 and a message containing text 546) and / or contextual conversation information (eg, message 568 preceding message 548 in a conversation transcript) can be included. In some embodiments, when a new message is received in a conversation, the extended view of message notification 542 is updated to display the new message (eg, new message 570 is displayed in message notification 542). In some embodiments, the extended view of the message notification 542 is a message notification (in response to an input received by the contact 574, for example, a tap input at a position corresponding to an affordance 572 to display a reply view). Includes affordance 572 for displaying 542 reply views.

FIG. 5N shows a reply view of the message notification 542 that allows the user to reply to the message in the notification (eg, without navigating from the notification user interface to the messaging application user interface). When the reply view of the message notification 542 is displayed, the keyboard 576 is displayed. The input received by the keyboard 576 is displayed in the message input area 578 of the message notification 542.

5O-5Q show navigation from the calendar invitation message notification 550 to the extended view of the calendar invitation message notification 550 and from the extended view of the calendar invitation message notification 550 to the calendar application user interface 594. In FIG. 5O, the input by contact 580 is received at the position corresponding to the invitation message notification 550 of the unlocked view of the notification user interface 520. In response to detecting the input, the device displays an extended view of the calendar invitation message notification 550, as shown in FIG. 5P. The extended view of the calendar invitation message notification 550 includes context calendar information 582 for event 584 corresponding to the invitation, as well as actions including action items 586 (“Accept”), 588 (“Maybe”), and 590 (“Decline”). Contains an item menu. The input by contact 592 is received at the position corresponding to the calendar invitation message notification 550. In response to detecting the input, the device displays the calendar application user interface 594 (eg, the event detail page for event 584 corresponding to the invitation, as shown in FIG. 5Q).

FIG. 5R-1 shows a voice mail notification 544. FIG. 5R-2 shows an expanded view of voicemail notification 544. The extended view of the voicemail notification 544 is, for example, a playback control for playing the voicemail voice (play / pause control 596, playback position slider 598, and / or volume control 5100, etc.), (for example, to the voicemail voice. It can include an action item menu that includes a voicemail transcript 5102 and / or action items 5104 (“Call Back”) and 5106 (“Delate”) of voicemail (which is automatically generated by the device based on).

FIG. 5S-1 shows notification 5108 for another exemplary application (eg, a driving service application). FIG. 5S-2 shows an expanded view of notification 5108. The extended view of notification 5108 includes, for example, map 5112 and / or action item 5118 (“Call Driver”). In some embodiments, map 5112 includes content that is updated in real time and / or near real time. For example, the representation 5114 of the real-time position of the vehicle is displayed on the map 5112 with respect to the representation 5116 of the destination of the vehicle. When the car moves, the car representation 5114 is updated to reflect the movement of the car.

5T-1 to 5T-2 show a sequence of user interfaces displayed in response to inputs detected while device 100 is in lock mode. In FIG. 5T-1, the wake screen interface 512 is displayed in lock mode, as indicated by the lock state of the lock icon 514.

FIG. 5T-1 shows a rightward swipe gesture in which the contact 5117 travels along the path indicated by the arrow 5120. In response to the right swipe gesture, the lock mode of the mini-application user interface 5122 is displayed, as shown in FIG. 5T-2. In some embodiments, when a rightward swipe occurs, an animation is displayed in which the notification user interface 520 appears to gradually slide to the right from the left side of the display.

In some embodiments, the mini-application user interface 5122 includes a search input area 5124. In response to the input received in the search input area 5124, the lock mode of the search result user interface is displayed (eg, similar to the search result user interface 532 shown in FIGS. 5F-4 and 5I).

5T-3 to 5T-4 show a sequence of user interfaces displayed in response to inputs detected while the device 100 is in unlock mode. In FIG. 5T-3, the wake screen interface 512 is displayed in unlock mode, as indicated by the unlock state of the lock icon 514.

FIG. 5T-3 shows a rightward swipe gesture in which the contact 5119 travels along the path indicated by arrow 5121. In response to the right swipe gesture, the unlock mode of the mini-application user interface 5122 is displayed, as shown in FIG. 5T-2. In response to the input received in the search input area 5124, the unlock mode of the search result user interface is displayed (eg, similar to the search result user interface 532 shown in FIGS. 5F-8 and 5J).

FIG. 5U shows the lock mode of the mini-application user interface 5122. The lock mode of the mini-application user interface display displays, for example, the lock mode of the calendar mini-application 5125 and / or the lock mode of the weather mini-application user interface 5126. In some embodiments, the lock and / or unlock modes of the mini-application user interface 5122 are stored and / or stored by (eg, GPS data and / or Wi-Fi data received by device 100, and / or device 100). Includes information 5123, such as information about points of interest near device 100 (as determined using data for points of interest accessed). In some embodiments, the calendar mini-application 5125 includes an identifier (eg, a text 5131 that identifies the corresponding application and / or an icon 5135 that identifies the corresponding application).

In FIG. 5U, input is provided by contact 5128 in the "Show More" affordance 5127 to display an extended view of the calendar mini-application 5125. Since the device is in lock mode (as indicated by the lock icon 514), user validation is required before the extended view of mini-application 5125 is shown. FIG. 5V is a user verification prompt user interface 558 including a passcode input interface 560. A mini-application, as shown in FIG. 5W, according to the determination that the user verification information is valid (eg, provided via contact 5129 in the passcode input interface 560 and / or input by the fingerprint sensor of the physical button 204). An expanded view of 5125 is displayed. In some embodiments, the extended view of the mini-application 5125 is not displayed, for example, in the locked view of the calendar mini-application 5125 (with the locked view and / or unlocked view of the calendar mini-application 5125). Extended time range (by comparison) and calendar event information (eg, "Coffee with Jon", "Team Meeting", "Meet Shelby at Shellfish Market" )"),including. The weather mini-application 5126 is tuned downward in the mini-application user interface 5122 to accommodate extensions to the calendar mini-application 5125.

FIG. 5W corresponds to the affordance 5130 for reducing the extended view of the calendar mini-application 5125 shown in FIG. 5W to the non-extended view of the calendar mini-application 5125, as shown in the unlock mode of the mini-application user interface 5122 of FIG. 5X. An input is provided by contact 5137 at the desired position.

In the unlock mode of the mini-application user interface 5122, an unlocked view of the (unextended) calendar mini-application 5125 is displayed. Compared to the locked view of calendar mini-application 5125, the lock mode of calendar mini-application 5125 is not shown in the unlocked view of calendar mini-application 5125 calendar event information (eg, "Coffee with Jon" and / Or it contains additional information such as "Team Meeting"). The unlock mode of the mini-application user interface 5122 displays the next scheduled mini-application 5132 that is not displayed in the lock mode of the mini-application user interface 5122.

In FIG. 5X, an input is provided by a contact 5139 at a position corresponding to the "Show More" affordance 5141 to display an extended view of the next scheduled mini-application 5132.

FIG. 5Y displays an extended view of the next scheduled mini-application 5132. The extended view of the next upcoming mini-application 5132 includes a map 5134 showing the location of the upcoming upcoming event, as well as action items 5136 (“Directions”), 5138 (“Snooz”), 5140 (“Message Insights”), And 5142 (“Delete Event”), including an action item menu. The input by contact 5143 is detected at the position corresponding to the next scheduled mini-application 5132. In response to detecting an input, the device displays a calendar application user interface 5144 that includes a display of upcoming events 5146, as shown in FIG. 5Z.

5AA-5AD show the changes that occur in the Applications Springboard user interface 400 when an input for displaying a quick action menu is received. In FIG. 5AA, the input by contact 5150 is detected at the position corresponding to the mail application icon 418. The mail application icon 418 corresponds to a mail application (eg, a mail application launched in response to a tap input detected in the mail application icon 418). The characteristic strength of the contact is indicated by the strength level meter 5148. In FIG. 5AB, characteristic intensity of the contact, as shown by the intensity level meter 5148 increases above the tips intensity threshold level IT _H. In some embodiments, characteristic intensity of contact 5149 is in accordance with a determination that has increased above the tips intensity threshold level IT _H, at least part of the user interface 400, is applied blur effect. As shown in FIGS. 5AB-5AD, the blurring effect is enhanced as the characteristic strength of the contact increases. In some embodiments, characteristic strength of contact in accordance with a determination that has increased above the tips intensity threshold level IT _H, aura view 5152 of quick action menu 5154 which corresponds to the mail icon 418 is displayed. As shown in FIGS. 5AB-5AC, the size of precursor view 5152 increases as the characteristic strength of the contact increases.

In FIG. 5AD, the characteristic strength of the contact 5150 increases above the _{deep pressing strength threshold IT D.} In some embodiments, one or more tactile power generators of device 100 output the tactile power shown in 5150 according to the determination that _{the characteristic strength of contact has increased above the deep pressing strength threshold IT D.} .. In some embodiments, a quick action menu 5154 corresponding to the mail application icon 418 is displayed. In some embodiments, the mini-application preview 5156 corresponding to the mail mini-application 5162 (eg, in a position close to the quick action menu 5154 and / or on a platter containing the quick action menu 5154 and / or the mini-application preview 5156). ) Is displayed. In some embodiments, the quick action menu 5154 and / or the mini-application preview 5156 is displayed at least partially overlaid on the application springboard user interface 400.

In FIG. 5AE, the input by contact 5158 is detected in the "Add Widget" affordance 5160 in mini-application preview 5156. In response to detecting the input, the device adds a mail mini-application 5162 corresponding to the mini-application preview 5156 to the mini-application user interface 5122, as shown in FIG. 5AF. In some embodiments, the device displays a mini-application user interface 5122, including a mini-application preview 5156, in response to detecting an input by contact 5158 in the "Add Widget" affordance 5160.

In FIG. 5AF, the mail mini-application 5162 is displayed on the mini-application user interface 5122. In some embodiments, the mail mini-application 5162 is an avatar corresponding to one or more email senders who have recently and / or frequently sent and / or received email from device 100. Includes 5164 (eg, 5164a, 5164b, 5164c and / or 5164d). The badge 5166 at the position corresponding to the avatar 5164a indicates, for example, the number of unread emails received from the email sender corresponding to the avatar 5164a. The input by contact 5170 is detected at the position corresponding to affordance 5168 for displaying the extended view of mail mini-application 5162. In response to detecting the input, the device displays an extended view of the mail mini-application 5162, as shown in FIG. 5AG.

FIG. 5AG shows an extended view of the mail mini-application 5162 displayed in the mini-application user interface 5122. Additional avatars 5164e, 5164f, 5164g, and / or 5164h not shown in the non-extended view of the mail mini-application 5162 are shown in the extended view of the mail mini-application 5162.

FIG. 5AH shows the transition from the state in which the wake screen interface 512 is displayed to the state in which the lock mode of the camera application user interface 5172 is displayed. In a first time t _1, as indicated by arrow 5178, left swipe gesture contact 5176 starts moving along the first portion of the swipe gesture is started. In the second time _{t 2} after the time _{t 1,} as the first portion of the camera application user interface 5172 may be revealed by a first portion of the swipe gesture, the contact 5176 is indicated by an arrow 5180, Continue moving along the second part of the swipe gesture. In a third time _{t 3} after the time _{t 2, the} camera application user interface 5172 is further elucidated by the second portion of the swipe gesture, the contact 5176, as indicated by arrow 5182, a swipe gesture Continue to move along part 3. In the fourth time _{t 4} after the time _{t 3,} the camera application user interface 5172 is further elucidated by the third part of the swipe gesture, the contact 5176 is a swipe gesture as indicated by arrow 5184 5 Continue to move along the part of. In a fifth time t ₅ of the after time t _4, the lock mode of the camera application user interface 5172 is fully revealed.

In some embodiments, the one or more camera controls 5186 are initially hidden in the lock mode of the camera application user interface 5172. For example, the delay from the time t ₁ to left swipe gesture is initiated (e.g., time delay period between the time t ₁ and time t ₃₎ after, in one or more camera control 5186 a camera application user interface 5172 Will be revealed. In some embodiments, one or more camera controls 5186 are revealed in the camera application user interface 5172 after the threshold portion of the camera application user interface 5172 is revealed.

In some embodiments, the lock mode of the camera application user interface 5172 is displayed (e.g., as shown at time t ₅ in FIG 5AH), access to one or more images in the image library device 100 You can't. For example, the unlock mode of the camera application user interface 5172 (e.g., as shown at time t ₅ in FIG 5ai) When displayed, the most recently captured image is displayed in the area 5188, a camera application user interface 5172 lock mode (e.g., as shown in the area 5188 of the time _{t 5} shown in FIG 5AH) When displayed, the area 5188, the image is not displayed.

FIG. 5AI shows the transition from the state in which the wake screen interface 512 is displayed to the state in which the unlock mode of the camera application user interface 5172 is displayed. In a first time t _1, as indicated by arrow 5192, the contact 5190 starts to move along the first portion of the swipe gesture, left swipe gesture is started. At the second time t ₂ after time t ₁ , the first part of the camera application user interface 5172 is revealed by the first part of the swipe gesture and the contact 5190 is indicated by the arrow 5194, as indicated by the arrow 5194. Continue moving along the second part of the swipe gesture. In a third time _{t 3} after the time _{t 2, the} camera application user interface 5172 is further elucidated by the second portion of the swipe gesture, the contact 5190, as indicated by arrow 5196, a swipe gesture Continue to move along the three parts. In the fourth time _{t 4} after the time _{t 3,} the camera application user interface 5172 is further elucidated by the third part of the swipe gesture, the contact 5190, a swipe gesture as shown by arrow 5198 4 Continue to move along the part of. In a fifth time t ₅ of the after time t _4, unlocked mode of the camera application user interface 5172 is fully revealed.

In some embodiments, the one or more camera controls 5186 are initially hidden in the unlock mode of the camera application user interface 5172. For example, the delay from the time t ₁ to left swipe gesture is initiated (e.g., time delay period between the time t ₁ and time t ₃₎ after, in one or more camera control 5186 a camera application user interface 5172 Will be revealed. In some embodiments, one or more camera controls 5186 are revealed in the camera application user interface 5172 after the threshold portion of the camera application user interface 5172 is revealed. In this way, accidental behavior of the camera application user interface is avoided.

In FIG. 5AJ, the input for partially revealing the camera application user interface 5172 is followed by the input for revealing the wake screen interface 512 again. In some embodiments, the input to reveal the camera application user interface 5172 reveals the leftward swipe gesture (eg, all of the camera application user interface 5172 or the threshold portion of the camera application user interface 5172 is revealed. Canceled by (terminating only the first part of the gesture) before it is done. In some embodiments, the input for revealing the camera application user interface 5172 is before all of the camera application user interface 5172 and / or before the threshold portion of the camera application user interface 5172 is revealed. , Canceled by providing an input (eg, rightward swipe gesture). For example, an input that includes a first portion 5178 and a second portion 5180 of a left-pointing gesture and is followed by a right-pointing gesture along the path indicated by arrow 5202 is detected. The right-pointing gesture cancels revealing the camera application user interface 5172. In this way, accidental display and / or operation of the camera application interface is avoided.

In FIG. 5AK, the wake screen interface 512 is displayed in lock mode, as indicated by the lock state of the lock icon 514. FIG. 5AK shows an upward swipe gesture (eg, from the bottom edge of the touch screen 112) in which the contact 5204 travels along the path indicated by the arrow 5206. In response to the upward swipe gesture, the first page of the multi-page control panel user interface 5208 is displayed, as shown in FIG. 5AL. In some embodiments, when an upward swipe occurs, an animation is displayed in which the multi-page control panel user interface 5208 appears to slide gradually upward from the bottom edge of the display.

FIG. 5AL shows the first page of the multi-page control panel user interface 5208. The multi-page control panel user interface 5208 may include, for example, 5210 (airplane mode), 5212 (Wi-Fi), 5214 (Bluetooth®), 5216 (do not disturb mode), 5218 (rotation lock), 5220 ( Multiple control affordances such as Flashlight), 5222 (Timer), 5224 (Nightshift), 5226 (Computer), 5228 (Camera), 5230 (Apple® TV Mirroring), and / or 5232 (Brightness). Including. In some embodiments, one or more control affordances on the first page of the multi-page control panel user interface 5208 are not available in the lock mode of the first page of the multi-page control panel user interface 5208. In some embodiments, one or more appearances of the control affordances on the first page of the multi-page control panel user interface 5208 indicate the state of the control affordances. For example, the Wi-Fi controlled affordance 5212 is not shown shaded, indicating that Wi-Fi is enabled, and the Bluetooth® controlled affordance 5214 is shown shaded, Bluetooth. Indicates that (registered trademark) is invalid.

In some embodiments, the multi-page control panel user interface 5208 comprises a control user interface release affordance 5234. In some embodiments, the page indicator 5236 is used to indicate the page of the currently displayed multi-page control panel. In some embodiments, the multi-page control panel user interface 5208 has another user interface (eg, the lock screen and / / shown in FIG. 5AL) on which inputs for displaying the control user interface are detected. Alternatively, the initial screen, such as the wake screen 5238, the application springboard user interface 400, and / or the application user interface) is displayed with partial or complete overlay. In some embodiments, the appearance of the user interface partially overlaid by the multi-page control panel user interface 5208 (eg, the wake screen user interface 512) is modified, as shown in 5238. For example, the partially overlaid user interface 512 is blurry and / or less bright compared to the previous appearance of the user interface.

In response to input by contact 5241 including movement along the path indicated by arrow 5240 (eg, a leftward swipe), a second page of multi-page control panel user interface 5208 is displayed, as shown in FIG. 5AM. .. In some embodiments, the second page of the multi-page control panel user interface 5208 is a plurality of content playback control affordances such as playback scrubber 5242, previous track control 5246, pause / playback control 5248, next track. Includes control 5250, or volume control control 5252. In some embodiments, the second page of the multi-page control panel user interface 5208 includes content routing destination information 5260 and / or affordance 5262 for displaying / stopping the display of the routing destination menu. In some embodiments, the second page of the multi-page control panel user interface 5208 is an image 5264 and / or (track) representing currently playing and / or recently played content, such as content (such as album art). Includes content identification information 5266 (such as title, album title, and / or artist). In some embodiments, one or more control affordances on the first page of the multi-page control panel user interface 5208 are not available in the lock mode on the second page of the multi-page control panel user interface 5208.

As shown in FIG. 5AN, a third page of multi-page control panel user interface 5208 is displayed in response to input by contact 5268, including movement along the path indicated by arrow 5270 (eg, a leftward swipe). In some embodiments, the third page of the multi-page control panel user interface 5208 is, for example, temperature control 5272, fan control 5274, shade control 5276, lighting control 5278, door control 5280, camera control 5282, smoke alarm control 5284. And / or include multiple remote device control affordances such as sleep control 5286. In some embodiments, one or more remote device controls are not available on the third page of the lock mode of the multi-page control panel user interface 5208. For example, the remote device control affordances 5272-5286 have a multi-page user interface 5208 to indicate that one or more remote device controls corresponding to visually modified remote device control affordances are not available in lock mode. It is shown (eg, shaded) visually modified from the default state of control affordances 5272-5286 to be displayed in unlock mode. In this way, remote device control access is limited to coordination by the user who provided the validation information to unlock the device. In some embodiments, a third page of the multi-page control panel user interface 5208 is a definition area (eg, a "living room"" in which one or more remote devices controlled by control affordances 5272-5286 are located. (Scene such as) identification information (eg, the name of the definition area 5288 and / or the image 5290 corresponding to that area).

In some embodiments, the remote device control affordances 5272-5286 displayed on the third page of the multi-page control panel user interface 5208 are Wi-Fi received by device 100, for example, by GPS and / or device. A remote device control affordance that corresponds (eg, is located) to the currently located defined area (eg, room), as determined by the signal. In some embodiments, the remote device control affordance 5272-5286 displayed on the third page of the multi-page control panel user interface 5208 is the remote device control affordance corresponding to the most recently accessed definition area. In some embodiments, a third page of the multi-page control panel user interface 5208 is an affordance 5292 for displaying / stopping the display of multiple definition area identifiers (eg, multiple definitions, as shown in FIG. 5AQ). Includes a menu 5294) for displaying area identifiers.

In FIG. 5AN, the input is provided by a contact 5296 for operating the illumination control affordance 5278. Since the device is in lock mode (as indicated by the lock icon 514), user verification is required before the optical control affordance 5278 becomes operational. In some embodiments, the device displays a verification prompt user interface 558 in response to detecting an input on the control affordance 5278 while the lock mode is active. FIG. 5AO is a user verification prompt user interface 558 including a passcode input interface 560. Multi-page, as shown in FIG. 5AP, according to the determination that the user verification information (eg, provided via contact 5295 of the passcode input interface 560 and / or input by the fingerprint sensor of the physical button 204) is valid. An unlocked view of the third page of the control panel user interface 5208 is displayed.

In FIG. 5AP, the remote device control affordances 5272-5286 are shown unshaded because the control affordances 5272-5286 are available in the device unlock mode. An input (eg, a tap gesture) is provided by a contact 5296 at a position corresponding to an affordance 5292 for displaying / stopping the display of a plurality of defined area identifiers. In response to this input, a menu 5294 containing the plurality of definition area identifiers is displayed, as shown in FIG. 5AQ. In some embodiments, the appearance of the affordance 5292 for displaying / stopping the display of the plurality of defined area identifiers changes in response to the input provided at the position corresponding to the affordance 5292. For example, the orientation of the chevron icon in affordance 5292 changes from FIG. 5AP to FIG. 5AQ.

In FIG. 5AQ, a plurality of defined area identifiers (eg, “Favorites” 5300, “Current Room” 5302, “Master Bedroom” 5304, “Office” 5306 and / Or the menu 5294 of "Kitchen" 5308) is displayed. Inputs (eg, tap gestures) are provided by contact 5298 at positions corresponding to affordances 5292 for displaying / stopping the display of a plurality of defined area identifiers. In response to this input, menus 5294 with the plurality of definition area identifiers 5300-5308 disappear and the third page of the multi-page control panel user interface 5208 is redisplayed, for example, as shown in FIG. 5AR. In some embodiments, a set of control affordances corresponding to each selected definition area identifier is displayed in response to an input (eg, tap input) that receives each definition area identifier in menu 5294.

As shown in FIGS. 5AR-5AS, the third page of the multi-page control panel user interface 5208 responds to input by contact 5310, including movement along the path indicated by arrow 5312 (eg, a downward swipe). It disappears and the underlying user interface 512 is redisplayed, as shown in FIG. 5AS. (In some embodiments, the input received at the position corresponding to the control user interface release affordance 5234 stops the display of the multi-page control panel user interface 5208 and provides the wake screen interface 512, as shown in FIG. 5AS. Redisplay.)

As shown in FIGS. 5AS-5AT, the multi-page control panel user interface 5208 is redisplayed in response to input by contact 5314 including movement along the path indicated by arrow 5316 (eg, an upward swipe). In some embodiments, as shown in FIG. 5AT, the most recently displayed page of the multi-page control panel user interface 5208 (eg, a third page of the multi-page control panel user interface 5208) responds to input. It will be redisplayed.

FIG. 5AU shows inputs received by contact 5322 (eg, an upward swipe along the path indicated by arrow 5320) in the application user interface (eg, calendar application user interface 5318), as shown in FIG. 5AV. Page Control Panel Displays the first page of the user interface 5208.

In FIG. 5AV, as shown in FIG. 5AW, an input by contact 5235 (eg, a leftward swipe along the path indicated by arrow 5327) is received to display a second page of the multi-page user interface 5208.

In FIG. 5AW, the second page of the multi-page control panel user interface 5208 is displayed with the calendar application user interface 5318 partially overlaid. Multi-page control panel user interface As shown in FIG. 5AX, the multi-page control panel user interface responds to an input that stops the display of the multi-page control panel user interface 5208 (eg, a downward swipe by contact 5329 along the path indicated by arrow 5331). The second page disappears and the calendar application user interface 5318 is redisplayed.

In FIG. 5AX, the calendar application user interface 5318 detects an input for redisplaying the multi-page control panel user interface 5208 (eg, an upward swipe by contact 5333 along the path indicated by arrow 5335). In response to detecting the input, the device redisplays the second page of the redisplay multi-page control panel user interface 5208, as shown in FIG. 5AY.

In FIG. 5AY, the input for displaying the third page of the multi-page control panel user interface 5208 (eg, a horizontal swipe by contact 5339 along the path indicated by arrow 5337) is the multi-page control panel user interface 5208. Detected on the second page. In response to detecting the input, the device displays a third page of the multi-page control panel user interface 5208, as shown in FIG. 5AZ.

In FIG. 5AZ, the input for displaying the second page of the multi-page control panel user interface 5208 (eg, a horizontal swipe by contact 5343 along the path indicated by arrow 5341) is the first page of the multi-page control panel user interface. 5208 detected on page 3. In response to detecting the input, the device displays a second page of the multi-page control panel user interface 5208, as shown in FIG. 5BA.

In FIG. 5BA, as shown in FIG. 5BB, the routing destination menu 5326 responds to an input (eg, tap input) by the contact 5324 at a position corresponding to the affordance 5262 for displaying / stopping the display of the routing destination menu. Is shown.

In FIG. 5BB, the second page of the multi-page control panel user interface 5208 is the routing destination 5328 (“iPhone®”), 5260 (“Airpods®”), and 5332 (“Bluetooth®”). ) Speaker ”), including a routing destination menu 5326. For example, the routing destination menu is extended from the content routing destination information area 5260, and the size of the multi-page control panel user interface 5208 is extended (vertically) to accommodate the routing destination menu 5326. In some embodiments, the one or more routing destinations include information corresponding to the routing destination, such as the current routing destination display 5334 (eg, checkmark icon) and / or battery level display 5336. In some embodiments, the visual appearance of the affordance 5262 changes in response to the input provided at the position corresponding to the affordance 5262. For example, the orientation of the chevron icon in affordance 5262 changes from FIG. 5BA to FIG. 5BB.

In FIG. 5BB, in response to an input (eg, tap input) by the contact 5338 at a position corresponding to the routing destination 5332 (“Bluetooth® Speaker”), the content (eg, currently playing) is shown in FIG. The routing destination of the content) is changed from the routing destination 5260 ("Airpods (registered trademark)") to the routing destination 5332 ("Bluetooth (registered trademark) Speaker").

In FIG. 5BC, for example, the current routing destination display 5334 does not exist in the area corresponding to the routing destination 5260 (“Airpods®”), and the routing destination 5332 (“Bluetooth® Speaker”) Content is routed to destination 5332 (“Bluetooth® Speaker”) and / or routed to 5260 (“AirPods”), as indicated by the presence of the current routed destination display 5334 in the corresponding area. (Registered trademark) ") will not be routed. In response to an input (eg, tap input) by the contact 5340 at a position corresponding to the affordance 5262 for displaying / stopping the display of the routing destination menu 5326, the routing destination menu 5326 disappears, as shown in FIG. 5BD. Will be done.

FIG. 5BD shows an input for changing the content routing destination (eg, a horizontal swipe by contact 5323 along the path indicated by arrow 5345). In response to detecting an input to change the content routing destination, the device indicates the content to indicate the changed content routing destination (eg, "Airpods®"), as shown in FIG. 5BE. The routing destination information area 5260 is updated.

5BF-5BJ show changes in the functionality of the control affordance on the first page of the multi-page control panel user interface 5208, according to some embodiments. In FIG. 5BF, the input by contact 5342 is detected at the position corresponding to the Wi-Fi control affordance 5212 on the first page of the multi-page control panel user interface 5208. As indicated by the intensity meter 5344, characteristic intensity of the contact is below tips pressure intensity threshold IT _H, lighter pressure intensity threshold IT _L, and deep pressure intensity threshold IT _D. According to the judgment that the characteristics intensity of contact satisfy the control toggle criteria (e.g., characteristic intensity of the contact is below the tip pressure intensity threshold IT _H), input by contact 5342 toggles the Wi-Fi control corresponding to the Wi-Fi Toggle the control affordance 5212 from the disabled state to the enabled state.

FIG. 5BG shows the first page of the multi-page control panel user interface 5208 after the Wi-Fi control has been toggled from the disabled state to the enabled state as shown in FIG. 5BG, as shown in FIG. 5BF. The appearance of the Wi-Fi control affordance 5212 has been changed (eg, visually changed from shaded to unshaded) to indicate that Wi-Fi control has been toggled to the enabled state, and / Alternatively, notification 5346 (eg, "Wi-Fi: on") is displayed.

5BH-5BI show inputs for displaying change options for Wi-Fi control. In FIG. 5BH, the input by the contact 5348 at the position corresponding to the Wi-Fi control affordance 5212 meets the enhanced control criteria (eg, the characteristic strength of the contact 5348 is a light pressing force as indicated by the strength meter 5344. increases above the threshold level iT _L, characteristic strength and / or contact 5348 increases above the deep pressing strength threshold level iT _D). In response to this input, change options 5354 ("Disconnect from HomeNetwork"), 5356 ("Turn off for 1 hour"), 5358 ("Off until you leave here (" A change option menu 5350 is displayed, including Turn off until I leave here)), 5360 “Connect to Other Network”) and 5362 (“Wi-Fi Settings”). The input by contact 5352 is detected at the position corresponding to the change option 5356 (“1 hour off”).

In FIG. 5BJ, in response to the input shown in FIG. 5BI, Wi-Fi control is indicated, for example, by the shaded state and notification 5346 (eg, "Wi-Fi: off") of the Wi-Fi control affordance 5212. In addition, it is (temporarily) disabled.

6A-6E show flow diagrams of method 600 for accessing controls from a display-on-user interface, according to some embodiments. Method 600 comprises an electronic device (eg, device 300 of FIG. 3, portable multifunction device 100 of FIG. 1A) having one or more sensors for detecting the intensity of contact by the display, the touch sensing surface, and the touch sensing surface. ) Is executed. In some embodiments, the display is a touch screen display with a touch sensitive surface on or embedded in the display. In some embodiments, the display is separated from the touch sensitive surface. Some actions in method 600 are optionally combined and / or the order of some actions is optionally changed.

As described below, method 600 provides an intuitive way to access the controller from the display-on-user interface. This method reduces the user's cognitive load when accessing the control device from the display-on-user interface, thereby producing a more efficient human-machine interface. Battery-powered electronic devices save power and extend the time between battery charges by enabling users to access controls faster and more efficiently.

During the display off state (eg, as shown in FIGS. 5B and 5D), the device detects the first input (602). In some embodiments, the display off state is a state in which the display is turned off or grayed out, a touch screen is turned off, and / or the touch screen display function is turned off. The touch-sensitive surface in the touch screen remains active, as is the state and at the same time implemented in, for example, sleep / hibernate mode, power saving mode, or another mode with limited functionality. In some embodiments, the first input meets the display activation criteria for activating the display of the device (eg, wakes the device and / or wakes the touch screen display), display activation. Input on affordances (eg, physical or virtual buttons). For example, the first input is the input by contact 508 on the physical button 204, as shown in FIGS. 5B and 5D. In some embodiments, the display activation criteria meet the display activation criteria so that the characteristic intensity of the contact at the detected input meets their respective intensity thresholds (eg, the display activation criteria are by tap gestures). Does not need to be satisfied). In some embodiments, when a contact lift-off is detected before the characteristic intensity of the contact exceeds a predetermined intensity threshold (when the display activation affordance is a virtual button), or the contact physically physicals the display activation affordance. When not activated (when the display activation affordance is a physical button, such as physical button 204), the display activation criteria are met. In some embodiments, the display activation criteria are met by other types of input (eg, voice input, the user picking up and tilting the device, etc.) without the need to detect contact.

In response to detecting the first input, the device activates the device's display (eg, touch screen display 112) and displays on the display a first user interface that corresponds to the device's display on state. (604) (For example, the device displays a wake screen 512 as shown in FIG. 5C, displays the user interface shown immediately after waking the device, or the device activates the display of the device). ..

While displaying the first user interface corresponding to the display on state of the device, the device makes a swipe gesture on the touch-sensitive surface (eg, a downward gesture along the path indicated by arrow 518 in FIG. 5F-1). , Downward gesture along the path indicated by arrow 536 in FIG. 5F-5, rightward gesture along the path indicated by arrow 5120 in FIG. 5T-1, rightward along the path indicated by arrow 5121 in FIG. 5T-3. Gestures: Left-pointing gestures shown in 5AH, left-pointing gestures shown in FIG. 5AI, upward gestures along the path indicated by arrow 5206 in FIG. 5AK, and / or upward gestures along the path indicated by arrow 5314 in FIG. 5AS) Is detected (606).

In response to detecting a swipe gesture on the touch-sensitive surface, the device is in lock mode with the display on (as indicated by the lock state of the lock icon 514) and the swipe gesture is in the first direction (as indicated by the lock state of the lock icon 514). For example, according to the determination to be downward and / or right), the device replaces the display of the first user interface (eg, the wake screen 512 as shown in FIG. 5C) with the display of the second user interface. , The second user interface displays the first content (eg, a notification screen as shown in FIG. 5F-2 and / or a mini-application object screen as shown in FIG. 5T-2) (608). According to the determination that the device is in unlock mode with the display on (indicated by the unlocked state of the lock icon 514) and the swipe gesture is in the first direction (eg downward and / or right), the device , The display of the first user interface is replaced with the display of the second user interface, the second user interface is not displayed when the first content and device are in the display-on state lock mode. Display additional content (eg, restricted notification content and / or restricted mini-application object content). For example, as shown in FIGS. 5F-6 and 5H, the unlock mode of the notification user interface 520 is not shown in the lock mode of the notification user interface 520, the notification 550, the text 546 of the notification 542, and / or the notification 542. Includes additional content, such as image 548 of. As shown in FIGS. 5T-4 and 5X, the unlock mode of the mini-application user interface 5122 is not shown in the locked view of the calendar mini-application 5125, the next scheduled mini-application 5132 and / or the calendar event information ( Includes additional content, such as "Coffee with Jon", "Team Meeting").

Efficiently access information available from the device while maintaining the security of sensitive information available from the device by displaying different content depending on whether the device is in lock or unlock mode. be able to. Providing secure access to the information stored by the device improves the operability of the device (for example, the user accesses the information with the device visible on without fully activating the device). Reduces power usage and improves device battery life by making the user device interface more efficient (by making it possible) and also allowing users to use the device faster and more efficiently. To do.

In some embodiments, the notification includes information corresponding to communications such as telephone calls, video calls, voice mails, emails, SMS, MMS, and / or calendar invitations received by the device. In some embodiments, the notification is a calendar reservation notification, a calendar reservation invitation, a reminder, and / or another type of notification generated by an application installed on the device.

In some embodiments, the first content includes one or more notifications (eg, not including limited notification information) and the first additional content is limited notification information (eg, of the device). Includes notification content and / or other notification information) subject to restricted access by settings such as privacy settings and / or security settings (610). For example, the first additional content is contextual information (as shown in FIG. 5M, received message 548 for which notification 542 is generated, such as initial communication content in a thread containing the first content in a conversation transcript. (For example, message 568 preceding) and / or the calendar information in the time range corresponding to the time associated with the calendar reservation notification in the first content (eg, shown in the extended view of the calendar invitation message notification 550, shown in FIG. 5P). Includes contextual calendar information 582). In some embodiments, when the device is in display-on state unlock mode, a second user interface (eg, FIG. 5F-6, FIG. 5G, FIG. 5M, FIG. 5N, FIG. 5P, FIG. 5R-2 and FIG. 5S-2) includes a longer version of the notification shown in the second user interface when the device is in the display-on state lock mode (eg, shown in FIGS. 5F-2 and 5H). Longer versions of notifications include extended notification content, including, for example, all content in the shorter or standard version of the notification, and additional content not included in the shorter or standard version of the notification. In some embodiments, the enhanced notification content includes a more complete version of the notification content, which is indicated in the short version of the notification. In some embodiments, the enhanced notification content includes selectable options for performing actions related to the notification, which are not included in the image, dialogue control, short version of the notification.

In some embodiments, the first content comprises one or more mini-application objects (eg, one or more "widgets") and the first additional content is limited mini-application object information (devices). Includes mini-application object information and / or other information) subject to restricted access by settings such as privacy and / or security settings (612). For example, the calendar mini-application 5125 shown in the unlock mode of the mini-application user interface 5122 (eg, as shown in FIGS. 5T-4, 5W, and 5X) is shown in (eg, 5T-2 and 5U). (As shown) Contains mini-application object information that is not displayed in the calendar mini-application 5125.

In some embodiments, the restricted mini-application object information and / or restricted notification information may be, for example, contextual content (such as an additional message in a conversation transcript or extended view of the calendar), photo content, video, audio, etc. Live updated content (such as a transport application map view showing real-time vehicle location), controls (eg, action buttons), a list of user's favorite contacts, and / or a keyboard for entering text inline (eg, "Response"). Includes when control is activated with communication notifications). For example, the additional content is a message notification in the extended view of the message notification 542, as shown in FIG. 5M, eg, contextual content (eg, messages 568 and 570) and / or reply affordance 572, as shown in FIG. 5N. Message input area 578 and / or keyboard 576 in the extended view of 542, invitation message notification 550 as shown in FIGS. 5O and 5P (but not present in the lock mode of notification user interface 520), FIG. 5R-2. Playback controls (play / pause control 596, playback position slider 598, and / or volume control 5100, etc.), transcripts, etc. for playing voicemail voicemail voices in the extended view of voicemail notification 544, as shown. 5102 and / or action items 5104 (“Call Back”) and 5106 (“Delate”), map 5112 and / or action item 5118 (“call”) in the expanded view of notification 5108, as shown in FIG. 5S-2. "Call Driver"), calendar event information in the extended view of the calendar mini-application 5125, the next upcoming mini-application 5132, as shown in FIGS. 5W and 5Y (eg, "Coffee with Jon", "Team Meeting". , "Meet Shelvy at Shelfish Market") and extended calendar time ranges, and / or include additional drivers 5164e-5164h in the extended view of the mail mini-application 5162, as shown in FIG. 5AG.

Displaying a device By displaying a mini-application object on an accessible interface while it is on, the user can get information from the application (eg, often) without fully activating the device and / or activating the application. You will be able to see general information), such as user-specified and / or otherwise specified information. Providing access to application information with the device's display on state improves the device's operability (for example, the user can access the device's display on state information without fully activating the device. It makes the user device interface more efficient (by allowing it to), which in turn reduces the power usage of the device by enabling the user to use the device faster and more efficiently. Improves battery life.

In some embodiments, the device is in display-on state lock mode (eg, as shown in FIGS. 5F-1 and / or 5T-1) in response to detecting a swipe gesture on the touch sensitive surface. According to the determination that the swipe gesture is in a second direction different from the first direction (eg, to the right as shown in FIG. 5T-1 and / or downward as shown in FIG. 5F-1). The display of the first user interface is the third user interface, and the third user interface is the display of the third user interface that displays the second content (for example, a mini application as shown in FIG. 5T-2). Replacing the object screen and / or the notification screen as shown in FIG. 5F-2, the device is in unlock mode with the display on (as shown, for example, in FIGS. 5F-5 and / or 5T-3). , The device displays the first user interface according to the determination that the swipe gesture is in the second direction (eg, pointing to the right and / or pointing down as shown in FIG. 5F-5). , A third user interface, the third user interface is a second content and a second additional content that is not displayed when the device is in display-on lock mode (eg, (eg, FIG. 5T-4). Restricted mini-application object content (as shown in) and / or restricted notification content, or other subject to privacy and / or restricted access by security settings (as shown in Figure 5F-6) on the electronic device. It replaces the display of the third user interface that displays the notification information) (614).

In some embodiments, the swipe gesture is in a third direction (eg, leftward, as shown in FIGS. 5AH and 5AI), which is different from the first and second directions. According to the determination that the device is in unlock mode (eg, as shown in FIG. 5AI), the first user interface displays a fourth user interface associated with the camera application (eg, camera application user interface 5172). ) Is replaced. According to the determination that the device is in lock mode (eg, as shown in FIG. 5AH), the first user interface was replaced with the display of the fourth user interface associated with the camera application and associated with the camera application. The image library is limited (eg, only images captured since the first input was received are accessible).

In some embodiments, the swipe gesture is a path indicated by a first direction, a second direction, and a fourth direction (eg, upward, eg, arrow 5206) shown in FIG. 5AK, which is different from the third direction. , And / or along the path indicated by arrow 5314 in FIG. 5AS). According to the determination that the device is in unlock mode (eg, as shown in FIG. 5AS), the first user interface 512 is replaced with the display of the control panel user interface (eg, multi-page control panel user interface 5208). .. In some embodiments, the first user interface 512 is replaced with the display of the multi-page control panel user interface 5208 (eg, according to the determination that the device is in lock mode (eg, shown in FIG. 5AK)). At least one panel and / or control of the control panel user interface 5208 (as described with respect to FIG. 5AN) is not accessible. For example, remote device control, such as a home accessory control device (eg, remote device control affordances 5272-5286), is when the device is in lock mode (eg, as indicated by the lock state of the lock icon 514 in FIG. 5AN). Inoperable and / or not displayed. In some embodiments, according to the determination that the device is in lock mode, the device has a first user interface 512, a second user interface (eg, notification user interface 520 and / or mini-application user interface 5122), first. Lock mode display (eg, notification user interface 520 and / or mini-application user interface 5122), fourth user interface (eg, camera application user interface 5172), and / or control panel user interface 5208 (eg, notification user interface 520 and / or mini-application user interface 5122). For example, a closed Nanjing lock icon 514) is displayed. In some embodiments, depending on the determination that the device is in unlock mode, the device will have a first user interface, a second user interface, a third user interface, a fourth user interface, and / or a control panel. An unlock mode display (eg, an open Nanjing lock icon, such as the unlocked state of the lock icon 514) is displayed on the user interface.

In some embodiments, the second content comprises one or more mini-application objects (eg, calendar mini-application 5125, weather mini-application 5126, upcoming scheduled mini-application 5132, and / or mail mini-application 5162). (616), the second additional content includes restricted mini-application object information.

In some embodiments, the second content includes one or more notifications (eg, calendar invitation notification 550, message notification 542, voicemail notification 544, and / or notification 5108), and the second additional content , Includes restricted notification information (618).

In some embodiments, the second user interface includes a first search input area 524 (eg, as shown in FIGS. 5F-2 to 5F-4 and 5F-6 to 5F-8). , The third user interface includes a second input area 5124 (eg, as shown in FIGS. 5T-2 and 5T-4) (620).

In some embodiments, the device has a search input area (eg, the input term "movie,"" of a first search input area and a second search input area (eg, text input for a filter search). The input in the search input area 524 and / or the search input area 5124) including the above is detected (622).

In response to detecting an input in each search area, the device determines that the device is in lock mode with the display on, and the device is the first in the search results (eg, as shown in FIGS. 5F-4 and 5I). Display a set of (624). According to the determination that it is in the unlock mode with the display on, the device has a second set of search results (eg, as shown in FIGS. 5F-6 and 5J) that is different from the first set of search results. For example, display a set of search results that include additional search results that are restricted by the device security policy and / or privacy policy and are not included in the first set of search results. For example, message 552, message 554, and email 556 are shown in FIG. 5J, which shows the unlock mode of the device, but not in FIG. 5I, which shows the lock mode of the device.

In some embodiments, determining whether the device is in unlock mode or in lock mode with the display on, such as input by contact 508 as shown in FIG. 5D (eg, display activation occurs). In response to detecting the first input (when it is done) (626).

In some embodiments, the device is in display-on lock mode or in display-on unlock mode in response to detecting a swipe gesture (eg, after display activation has occurred). Is determined in response to detecting a swipe gesture on the touch-sensitive surface (eg, as described for 606) (628).

In some embodiments, the electronic device comprises one or more sensors, including, for example, an accelerometer, a gyro, a microphone, a vibration sensor, a thermal sensor, a touch sensor, and / or an optical sensor (display of the device). To detect the first input (to activate) is to use one or more sensors to lift the device by changing the context of the device (eg, as shown in FIGS. 5A-1-5A-4). Includes detecting (630). In some embodiments, the change in the context device is a change in the internal and / or external state of the device, for example, a change in the orientation of the device and / or a change in the environment (eg, a light level). For example, changes in the context of a device include when the user lifts the device from the table and / or removes the device from his pocket and looks at the device held in his hand 502, and / or detects a trigger phrase or a keyword spoken by the user. Occurs in response to detection. In response to detecting a change in the device context, the device determines whether the change in the device context meets the display activation criteria. In some embodiments, the display activation criterion is one or more of tilt criteria (eg, threshold fluctuations from the initial position and / or threshold fluctuations from the horizontal axis), acceleration criteria and / or light level change criteria. including. In some embodiments, the output of one or more sensors is used to prevent false positives. For example, if the light level change criterion is met but the acceleration is below the threshold acceleration level (eg, different from the acceleration reference), the display activation criterion is not met. In some embodiments, as the orientation of the device changes (eg, as shown in FIGS. 5A-3 to 5A-4), the wallpaper and / or notification is extended as the orientation changes.

In some embodiments, detecting the first input (to activate the display of the device) is a display activation affordance (eg, a virtual button on the physical button 204 and / or the touch sensitive surface 112). ) Includes detecting activation (632). In some embodiments, detecting the activation of the display activation affordance comprises detecting the action of the physical button 204, such as the sleep / wake button 206. In some embodiments, detecting the activation of the display activation affordance is a virtual home configured to detect the intensity of contact with the touch-sensitive surface (eg, force, contact area, and / or fingerprint). such as buttons, by one or more sensors for detecting) at a position corresponding to virtual button (e.g., the characteristics of the contact that satisfies IT _a) respective threshold intensity as indicated by the intensity meter 510 in FIG. 5D Includes detecting increased intensity.

In some embodiments, detecting the first input (activating the display of the device) activates the display activation affordance (eg, to activate the display of the device) and activates the display. Includes detecting contact with a display-activated affordance that does not activate the affordance (eg, physical sleep / wake button 206 and / or physical home button 204) (634). In some embodiments, the contact with the display-activated affordance that does not activate the display-activated affordance is the contact with the physical button 204 that moves and / or presses the physical button, as shown in FIG. 5B (eg,). A light touch of a button that includes a fingerprint sensor, such as a physical home button, for Touch ID scanning).

In some embodiments, detecting the first input (for activating the display of the device) is contact with a display activation affordance (eg, a virtual button and / or a physical button 204). To activate the display activation affordance to activate the display of the device (for example, wake the device or wake the touch screen display) and associate the display activation affordance with the display activation affordance without activating it. Includes detecting contact with display activation affordances, performing at least one additional function (636). In some embodiments, the display activation affordance is a virtual or physical button 204 that triggers the execution of different functions in response to different levels of intensity of contact. For example, a first intensity threshold (e.g., IT A as indicated by the intensity meter 510 in FIG. _5B) contacting with a characteristic intensity below the (for example, to activate the display device) associated with the virtual buttons The first function activated, eg, the first intensity threshold (as shown in FIG. 5D), the contact 508 having a characteristic intensity greater than or equal to the device (eg, if the device is locked). Activates a second function associated with the virtual button (to unlock) to display the home screen (eg, the application springboard user interface 400, as shown in FIG. 5E).

In some embodiments, while the first user interface 512 is displayed, the device detects a second input, and in response to detecting the second input, the device receives the first input. The display of the user interface includes multiple application icons corresponding to different applications of multiple applications at a time after the second swipe gesture in the direction opposite to the first direction is detected on the touch sensitive surface. , Replace with the display of the user interface (638). For example, the device includes a contact 508 on a display activation affordance (eg, a physical button 204 or a virtual button) that meets the device unlock criteria for unlocking the device (if the device is locked). Detecting the second input and in response to the second input, the device displays a home screen (eg, an application springboard user interface 400, as shown in FIG. 5E). In some embodiments, the device unlock criteria require that the characteristic intensity of contact at the detected input (eg, when the display activation affordance is a virtual button) meet or exceed their respective intensity thresholds. .. For example, in some embodiments, device unlocking criteria include criteria that are met when _{the characteristic strength of contact increases above the deep pressing strength threshold IT D.} In some embodiments, the unlocking criterion for the device is (when the display activation affordance is the physical button 204), for example, as shown in FIG. 5D, the contact physically activates the display activation affordance. Demand that. In some embodiments, in response to detecting a second input, the device accommodates a display of the first user interface (eg, wake screen user interface 512) for different applications of multiple applications. It replaces the display of a user interface that includes a plurality of application icons (eg, the user interface that includes the plurality of application icons is, for example, the application springboard user interface 400, as shown in FIG. 5E). In some embodiments, the second input involves pressing a physical button (eg, a home button). In some embodiments, the second input is of contact with the touch-sensitive surface (eg, at a position corresponding to the virtual button, such as a virtual home button configured to detect force and / or fingerprints). Increased contact characteristic strength to meet device unlock criteria for unlocking the device and displaying the home screen (if the device is locked) by one or more sensors to detect the strength Is detected. In some embodiments, the device launches the application corresponding to each application icon when the input detected on each application icon in the user interface containing the plurality of application icons meets the application launch criteria. In some embodiments, the application launch criteria are met by the tap input, for example, without requiring the characteristic intensity of the tap input to meet the respective intensity thresholds.

In some embodiments, in response to detecting a second input, the device determines whether the device is in display-on lock mode or display-on unlock mode (in response to the detection of the second input). 640), the device is in lock mode with display on (eg, the device is not unlocked by a UI ID scan that occurs when a second input is provided, and / or the device is a second input. According to the determination that is not unlocked by the previous input provided before the device is provided, the device is such as a user verification prompt user interface 558 including a passcode input interface 560 (eg, as shown in FIG. 5L). Display the authentication user interface (which prompts the user to provide an unlock input to unlock the device).

In some embodiments, detecting the first input comprises detecting the activation of the display activation affordance (642). For example, the first input includes a contact 508 on a display activation affordance (eg, a physical button 204 or a virtual button).

In some embodiments, each mini-application object is configured to perform a subset of the corresponding application features of each mini-application object, less than all (644).

In some embodiments, each mini-application object (eg, mini-application object 5125 shown in FIG. 5U) has an identifier (eg, as shown in FIG. 5U) for the corresponding application of each mini-application object. Display text 5131 identifying the corresponding application and / or icon 5135) identifying the corresponding application (646).

In some embodiments, each mini-application object displays a portion of the content from the corresponding application from each mini-application object (648). For example, the calendar mini-application 5125 is indicated by the calendar application user interface 5144 of FIG. 5Z (eg, a portion of the content includes a subset from the schedule of the day in the calendar application, such as the time closest to the current time). , Includes some of the content from the calendar application.

In some embodiments, a given input on each mini-application object (eg, an input that includes a contact 5143 at a position corresponding to the next scheduled mini-application object 5132, as shown in FIG. 5Y), respectively. Launch the application corresponding to the mini-application object (eg, the calendar application shown by the calendar application user interface 5144 in FIG. 5Z) (650).

In some embodiments, each mini-application object operates as a stand-alone application that also resides in the device's memory, unlike the associated application that also resides in the device's memory (652).

In some embodiments, each mini-application object acts as an extension or component of the associated application on the device (654).

In some embodiments, each mini-application object has a dedicated memory portion for temporary storage of information (656).

In some embodiments, the memory portion is accessible by the corresponding full-featured application of each mini-application object (658).

In some embodiments, the notification is data issued by the application (or server associated with the application) for display outside the application's normal user interface by the operating system (or the operating system's notification management module). It is an object. Notifications can include data retrieved from data stores accessible by both the notification and the application associated with the notification. In some embodiments, the notification may include a programmable component (eg, a mini-application object or extension) that dynamically loads or generates data for display on the device. In some embodiments, the notifications obtained from the application (or the server associated with the application) are shorter, more simplified, viewable versions for display on the device, and longer, more complex notifications. Contains data to generate both viewable versions.

In some embodiments, the mini-application object is configured to perform a less than all subset of the functionality of the corresponding application.

In some embodiments, the mini-application object displays the identifier of the corresponding application.

In some embodiments, the mini-application object displays a portion of the content from the corresponding application. For example, a map mini-application object displays a portion of the map that is displayed in the map application that corresponds to the map mini-application object. For example, a calendar mini-application object displays a portion of the calendar displayed in the corresponding calendar application.

In some embodiments, a given input on a mini-application object launches the corresponding application.

In some embodiments, the mini-application object acts as a stand-alone application that also resides in the device's memory, unlike the associated application that resides in the device's memory. For example, a mini-application object that corresponds to a social networking application behaves as a single-purpose or streamlined application with a subset of all the features of the corresponding application, but is a full-featured social networking application. Be associated. In this example, the mini-application object operates independently of the social networking application, and in scenarios where the social networking application is not running, the mini-application object continues to operate.

In some embodiments, the mini-application object acts as an extension or component of the associated application on the device. For example, a mini-application object for a calendar application is a single-function or behavioral component of a full-featured calendar application. In this example, neither calendar mini-application object will work if the calendar application is not running (eg in the background).

In some embodiments, the mini-application object has a dedicated memory portion for temporary storage of information. In some embodiments, this portion of memory can be accessed by the corresponding full-featured application. For example, the mini-application object of an instant messaging application has a memory portion for temporary storage of a partially written reply message. In this example, when the user opens the corresponding application while writing the reply message, the content of the reply message is retrieved from the temporary storage location and used by the full-featured application to complete the reply message to the user.

In some embodiments, the mini-application is a combination of any of the features described in paragraphs [00298]-[00304].

It should be noted that the particular order described for the operations in FIGS. 6A-6E is merely exemplary and the order described is not intended to indicate that the operation is the only order in which it can be performed. I want to be understood. One of ordinary skill in the art will recognize various methods for reordering the operations described herein. In addition, the details of the other processes described herein with respect to the other methods described herein (eg, methods 700, 800, and 900) are similar to the method 600 described above with respect to FIGS. 6A-6E. Note that the method is also applicable. For example, the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described above with reference to method 600 are other methods described herein (eg, optionally, methods 700, 800, and It has one or more properties of contact, gesture, user interface object, intensity threshold, focus selector, and animation described herein with reference to 900). For the sake of brevity, those details are not repeated here.

7A-7E show a flow diagram of a method 700 for accessing controls from a display-on-user interface, according to some embodiments. Method 700 comprises an electronic device (eg, device 300, FIG. 3, or portable multifunction device 100, FIG. It is executed in 1A). In some embodiments, the display is a touch screen display with a touch sensitive surface on or embedded in the display. In some embodiments, the display is separated from the touch sensitive surface. Some actions of method 700 are optionally combined, and / or the order of some actions is optionally changed.

As described below, Method 700 provides an intuitive way to access controls from the display-on-user interface. The method reduces the user's cognitive load when accessing the control, thereby producing a more efficient human-machine interface. Battery-powered electronic devices save power and extend the time between battery charges by enabling users to access the control faster and more efficiently.

The device is in the display off state (for example, the display is turned off or dimmed, the touch screen is turned off, and / or the touch screen display function is turned off, but the touch While the touch-sensitive surface of the screen remains active, for example, to be performed in sleep / hibernate mode, power saving mode, or another mode with limited functionality, the device First input to activate the display of the device (eg, wake the device or wake the touch screen display), eg, input by contact 508 with physical button 204, as shown in FIG. 5B. Is detected (702).

In response to detecting the first input, the device activates the display of the device and displays on the display a first user interface 512 corresponding to the display on state of the device (eg, in FIG. 5C). Wake screen 512 as shown, user interface shown as soon as the device is waked or the display of the device is activated) (704).

While displaying the first user interface corresponding to the display on state of the device, the device detects a swipe gesture on the touch sensitive surface (706).

In response to detecting the swipe gesture, the device displays the first user interface 512 according to the determination that the swipe gesture is in the first direction (eg, to the left as shown in FIGS. 5AH and 5AI). Replace with the display of the camera application user interface 5172 (708). The swipe gesture is in a second direction different from the first direction (eg, along the path indicated by arrow 5120 in FIG. 5T-1 and / or along the path indicated by arrow 5121 in FIG. 5T-3. According to the determination that it is (rightward), the device displays the display of the first user interface 512 with multiple mini-application objects (eg, calendar mini-application 5125, weather mini-application 5126, next upcoming mini-application 5132, and / or. Replaces the display of the mini-application object user interface 5122 configured to include the mail mini-application 5162). Each mini-application object of a plurality of mini-application objects has a corresponding application stored on the device (eg, a calendar application, a weather application, and / or a mail application). The swipe gesture is upward along the path indicated by arrow 5206 in FIG. 5AK and / or along the path indicated by arrow 5314 in FIG. 5AS, which is different from the first and second directions. According to the determination that it is (upward), the device displays the first page of the multi-page control panel user interface 5208. For example, the device has a control affordance (eg, 520-10232) for adjusting device settings as shown in FIG. 5AL, a first page, as shown in FIG. 5AM, a media player control affordance (eg, 5242). A second page having a second control user interface including ~ 5252) and / or a third having a third control user interface including remote device control affordances (eg, 5272-5286), as shown in FIG. 5AN. Display the multi-page control panel user interface 5208, which includes three pages. The swipe gesture is downward along the path indicated by arrow 518 in FIG. 5F-1 and / or FIG. 5F- in a fourth direction different from the first, second and third directions. According to the determination that it is down along the path indicated by arrow 536 of 5, the device displays multiple notifications (eg, calendar invitation notification 550, message notification 542, voicemail notification 544, and / or notification 5108). Display the notification user interface 520 configured to do so. In some embodiments, the notification includes information corresponding to communications such as telephone calls, video calls, voice mails, emails, SMS, MMS, and / or calendar invitations received by the device. In some embodiments, the notification is a calendar reservation notification, a calendar reservation invitation, a reminder, and / or another type of notification generated by an application installed on the device.

Users by providing a notification user interface that is accessible via input received by the camera application user interface, mini-application object user interface, multi-page control panel user interface, and user interface corresponding to the device's display-on state. Allows you to view information (eg, general information such as application information and device settings) on the device display without fully activating the device. Providing access to such information with the device visible on improves the device's operability and makes the user device interface more efficient (eg, without the user fully activating the device). By enabling access to information while the device is visible), and by enabling faster and more efficient use of the device, it reduces the power usage of the device and the battery of the device. Improve life.

In some embodiments, replacing the display of the first user interface 512 with the display of the camera application user interface 5172 is a display of one or more control affordances associated with the camera application (eg, FIGS. 5AH and FIG. 5 AI's Control Affordance 5186) includes delaying until the control display period elapses (eg, from the time the swipe gesture is detected or from the time the camera application user interface is displayed) (710). For example, the control affordance 5186, after the gesture left is started at time _{t 1,} does not appear until the time _{t 3} of FIG 5AI and FIG 5AH. In some embodiments, replacing the display of the first user interface 512 with the display of the camera application user interface 5172 involves gradually revealing the camera application user interface when a swipe gesture is received. One or more control affordances are displayed according to the determination that a portion of the revealed camera application user interface 5172 has increased above the threshold portion. In some embodiments, by delaying the display of one or more control affordances 5186 associated with the camera application until the control display period elapses or the threshold portion of the camera application user interface 5172 becomes apparent. The user is allowed some time and has the opportunity to revert to the previous user interface (eg, home screen, wake screen, etc.) without lifting the contact 5176. In other words, user interface navigation becomes more fluid and efficient, and users can change their minds after seeing hints on the outcome of swipe gesture continuation in the current swipe direction (eg, current). Continuing a swipe in the direction leads to the camera application user interface). In some embodiments, some camera control functions associated with control affordances (such as some auxiliary light sensors and / or some front-end control functions), and some not associated with control affordances. Control functions (eg, some back-end control functions) take time and power to activate, and activation of these control functions is intended by the user to activate the camera application, but the user swipes. The continuation of the gesture is delayed until it is confirmed by exceeding the relevant threshold.

In some embodiments, the device detects a gesture returning to the first user interface (eg, a swipe gesture in the opposite direction of the swipe gesture displaying the camera application user interface) during the control display period (712). In response to detecting a gesture returning to the first user interface during the control display period, the device displays the display of the camera application user interface (eg, the revealed portion of the camera application user interface). Replace with the display of one user interface (eg, the first user interface is restored) (714).

In some embodiments, the device corresponds to a first page of the multi-page control panel user interface 5208 while the multi-page control panel user interface 5208 is displayed (eg, as shown in FIG. 5AL). Detects gestures (eg, left-pointing gestures along the path indicated by arrow 5240) at positions on the touch-sensitive surface (eg, within and / or adjacent to the first page of the multi-page control panel user interface 5208). (716). In some embodiments, in response to detecting a gesture at a position corresponding to the first page of the multi-page control panel, the device has a second page of the multi-page control panel user interface 5208 shown in FIG. 5AM. Multi-page Control Panel Displays a second page of the user interface, such as, including controls for media playback and / or controls for accessories (such as home device accessories) (718).

In some embodiments, the device corresponds to a second page of the multi-page control panel user interface (eg, multi-page control) while the second page of the multi-page control panel user interface 5208 is displayed. Detect gestures (eg, left-pointing gestures along the path indicated by arrow 5270) at positions on the touch-sensitive surface within and / or adjacent to the first page of the panel user interface (720) and multi-page. In response to detecting a gesture at a location corresponding to the second page of the control panel user interface, the device has a multi-page control panel, such as the third page of the multi-page control panel user interface 5208 shown in FIG. 5AN. A third page of the user interface (including, for example, control of one or more accessory devices (such as a home device) communicatively coupled to an electronic device) is displayed (722).

In some embodiments, while the notification user interface 520 is displayed, the device receives a left swipe or gesture received in an affordance (eg, affordance, eg, "X" affordance (eg, 566 in FIG. 5M)). , Or a quick action item such as a quick action item that "accepts a calendar invitation") on the touch sensitive surface 112 (eg, at the position corresponding to each notification) that corresponds to each notification contained in the notification user interface. Detecting at a location (in the platter and / or adjacent to it) (724) and in response to detecting a release gesture, the device stops displaying each notification in the notification user interface (726). In some embodiments, when the notification user interface is subsequently displayed, the respective notifications are not displayed.

In some embodiments, while the notification user interface is displayed, the device has an activation gesture (eg, shown in FIG. 5P) at a position on the touch sensitive surface corresponding to the first notification included in the notification user interface. Detect (728) tap gestures, such as tap gestures by contact 592 at positions corresponding to notification 550 and / or positions corresponding to extended versions of notifications, and in response to detecting activation gestures. The device launches an application corresponding to the first notification (eg, as shown in FIG. 5Q) (730).

In some embodiments, while the mini-application object user interface is displayed, the device launches a launch gesture at a position on the touch-sensitive surface that corresponds to the first mini-application object contained in the mini-application object user interface. For example, tap gesture) is detected (732). For example, the activation gesture is an input by contact 5143, as shown in FIG. 5Y. In some embodiments, the launch gesture is detected in an extended version of the mini-application object. In some embodiments, the launch gesture is detected in a non-extended version of the mini-application object.

In response to detecting the launch gesture, the device launches the application corresponding to the first mini-application object (734). For example, as shown in FIG. 5Y, the corresponding calendar application 5144 is displayed in response to the activation gesture by contact 5143 at the position corresponding to the next scheduled mini-application object 5132, as shown in FIG. 5Z.

In some embodiments, while the mini-application object user interface 5122 is displayed, the device is information at a position on the touch-sensitive surface that corresponds to a second mini-application object contained in the mini-application object user interface. Detecting an extended gesture (eg, a tap gesture of the "show more" affordance on the platter of a mini-application object) (736), the second mini-application object has mini-application object information (eg, content, features, and / or). Includes input devices (such as control affordance and / or keyboard), for example, the information extension gesture is input by contact 5128 at a position corresponding to the "show object" affordance 5127 of the calendar mini application 5125 as shown in FIG. 5U, and / Or input by contact 5139 at a position corresponding to the "show object" affordance 5141 of the next scheduled mini-application 5132, as shown in FIG. 5X. In response to detecting an information expansion gesture, the device: Extend the second mini-application object (eg, the vertical size of the mini-application object, as shown by the extended view of the calendar mini-application object 5125 in FIG. 5W and the next extended view of the upcoming mini-application object 5132 in FIG. 5Y. (738). The device displays mini-application object information and additional mini-application object information in the extended second mini-application object, for example, as shown in FIG. 5Y: The extended view of 5132 is the information shown in the view of the next scheduled mini-application 5132 shown in FIG. 5X, as well as map 5134 and / or action item 5136 (“Directions”), 5138 (“Snooze”). Includes additional information, such as 5,140 (“Message Invitees”), and / or 5142 (“Delete Event”). In some embodiments, other minis. The application object is a mini-application object user interface Move the bottom of the boot down and / or roll it off (eg, as shown by the weather mini-application object 5126 in FIG. 5Y) to make room for the extended mini-application object.

In some embodiments, each mini-application object is configured to perform a subset of the corresponding application features of each mini-application object, less than all (740).

In some embodiments, each mini-application object has a corresponding application identifier for each mini-application object (eg, text 5131 identifying the corresponding application and / or the corresponding application, as shown in FIG. 5U. The identification icon 5135) is displayed (742).

In some embodiments, each mini-application object displays a portion of content from the corresponding application from each mini-application object (744). For example, the calendar mini-application 5125 may include a portion of the content from the calendar application indicated by the calendar application user interface 5144 in FIG. 5Z (eg, a time schedule of the calendar application, such as a time close to the current time). Includes a subset).

In some embodiments, a given input on each mini-application object (eg, an input that includes contact 5143 at a position corresponding to the next scheduled mini-application object 5132, as shown in FIG. 5Y) Launch the corresponding application of the application object (eg, a calendar application as shown by the calendar application user interface 5144 in FIG. 5Z) (746).

In some embodiments, each mini-application object operates as a standalone application that is also present in the device's memory, unlike the associated application that is present in the device's memory (748).

In some embodiments, each mini-application object acts as an extension or component of the associated application on the device (750).

In some embodiments, each mini-application object has a dedicated memory portion for temporary storage of information (752).

In some embodiments, the memory portion is accessible by the corresponding full-featured application of each mini-application object (754).

The particular order described for the actions in FIGS. 7A-7E is merely exemplary and is not intended to indicate that the order in which the actions can be performed is the only order in which the actions can be performed. I want to be understood. One of ordinary skill in the art will recognize various methods for reordering the operations described herein. In addition, the details of the other processes described herein with respect to the other methods described herein (eg, methods 600, 800, and 900) are also similar to the method 700 described above with respect to FIGS. 7A-7E. Please note that it is applicable in the above way. For example, the contacts, gestures, and user interface objects, and animations described above with reference to method 700 are optionally other methods described herein (eg, methods 600, 800, and 900). It has one or more features of contacts, gestures, and user interface objects, and animations described herein. For the sake of brevity, those details are not repeated here.

8A-8C show a flow diagram of method 800 for adding a mini-application object to the mini-application user interface, according to some embodiments. Method 800 comprises an electronic device having one or more sensors for detecting the intensity of contact with the display, the touch sensitive surface, and the touch sensitive surface (eg, device 300, FIG. 3, or portable multifunction device 100, FIG. It is executed in 1A). In some embodiments, the display is a touch screen display with a touch sensitive surface on or embedded in the display. In some embodiments, the display is separated from the touch sensitive surface. Some actions of Method 800 are optionally combined and / or the order of some actions is optionally changed.

As described below, Method 800 provides an intuitive way to add mini-application objects to the mini-application user interface. The method reduces the cognitive load on the user when adding mini-application objects to the mini-application user interface, thereby producing a more efficient human-machine interface. Battery-powered electronic devices save power and extend the time between battery charges by enabling users to add mini-application objects to the mini-application user interface faster and more efficiently.

The device displays a first user interface 400 that includes a plurality of application icons corresponding to different applications of the plurality of applications installed on the device (eg, the device is a home screen or application spring, as shown in FIG. 5AA). Display board 400 (802).

The device is a first application icon of a plurality of application icons (eg, a mail application icon 418), the first application icon being a first application icon corresponding to the first application of the plurality of applications. A first input (eg, a light or deep press input by the first contact 5150) is detected at a position on the corresponding touch sensitive surface (804). For example, the first input is a contact 5150 input, as shown in FIGS. 5AB-5AD.

In response to detecting the first input, the device previews the first mini-application object or the first mini-application object (eg, preview 5156 of the mini-application object 5162 shown in FIG. 5AF, shown in FIG. 5AD. A representation or reduced version of the first mini-application object, such as, is displayed in the overlay area (eg, overlaid on the application springboard user interface 400) (806). In some embodiments, one or more functional items corresponding to the mini-application object are not activated in the mini-application object preview. For example, when an input is received by the avatar 5164a of the mini-application preview 5156 of FIG. 5AD, no response is generated, but a response is generated in response to the input received by the avatar 5164a of the mini-application object 5162 of FIG. 5AF. (For example, an email received and / or sent from a user corresponding to the avatar 5164a is displayed. For example, the mini-application preview 5156 displays, for example, four avatars 5164a-5164d. 5162 includes an affordance (“Show More” affordance 5168) for extending the mini-application object 5162 to display additional avatars (eg, 5164e-5164h, as shown in FIG. 5AG). The object (eg, preview 5156) corresponds to the first application of a plurality of applications (eg, mail application), and the overlay area corresponds to the first mini-application object (eg, mini-application object 5162 corresponding to preview 5156). A second user interface (eg, including one or more user-selected mini-application objects and / or one or more default mini-application objects) that displays a plurality of mini-application objects (eg, mini-application user interface). 5122) includes an affordance (“Add Wideget” affordance 5160) for addition to.

The device is located on the touch sensitive surface corresponding to the affordance 5160 for adding the first mini-application object 5162 to the second user interface, and the second input (eg, tap input, etc., second contact 5150). by the input, to detect a deep pressure input) to increase light pressure input characteristic intensity of the contact is increased above the light press intensity threshold iT _L, or characteristic intensity of contact exceeds the deep pressing intensity threshold iT _D (808) ..

In response to detecting the second input, the device adds the first mini-application object 5162 to the second user interface to display the plurality of mini-application objects (810). In some embodiments, a plurality of mini-application objects (eg, mail mini-application object 5162, calendar mini-application object 5125, and / or the next scheduled mini shown in FIG. 5AF) within a second user interface (eg, 5122). The application object 5132) is displayed as a vertical stack of platters. In some embodiments, each platter in the vertical stack of platters corresponds to each mini-application object of a plurality of mini-application objects, and each platter has the same initial platter height.

Adding a mini-application object to the mini-application object's user interface in response to the input corresponding to the application icon is an intuitive and efficient way for the user to select the desired mini-application object. Providing the ability to add mini-application objects from an application icon to a collection of mini-application objects improves device operability, makes the user device interface more efficient, and also reduces power usage, allowing the user to device. Improve the battery life of the device by enabling faster and more efficient use of the device.

In some embodiments, while displaying the first user interface (eg, the application springboard user interface 400), the device is a third input on the touch sensitive surface 112, the third of which. Inputs include navigation criteria for navigating from the first user interface to the second user interface (mini-application user interface 5122) (eg, the third input is one or more rightward swipe gestures). , A third input, which meets the first criterion, is detected (812).

In some embodiments, in response to detecting a third input, the device displays a first user interface (eg, an application springboard user interface 400) with a second user interface (mini-application user). Replacing the first user interface with the second user interface by replacing the interface 5122) (814) stops displaying the first user interface (eg, the application springboard user interface 400). And, in the second user interface, to the first mini-application object 5162 (eg, having updated content from the first application), and to the second application of a plurality of applications different from the first application. A second user interface, including displaying at least a second mini-application object (eg, calendar mini-application object 5125 and / or next upcoming mini-application object 5132, as shown in FIG. 5AF) at the same time. Includes displaying the mini-application user interface 5122). In some embodiments, the second user interface provides an "edit" accommodation for rearranging mini-application objects in the mini-application object stack and / or adding new mini-application objects to the mini-application object stack. Including.

In some embodiments, the first mini-application object contains at least one selectable piece of information (eg, avatars 5164a, 5164b, 5164c, and / or 5164d) (816). In some embodiments, the selectable information item is a stock price list item that displays information about the stock price (eg, start price, maximum price, price change), weather forecast information for a location (eg, city). Weather forecast items, recently and / or frequently sent email to device 100 and / or avatar 5164 corresponding to one or more email senders who received email from device 100 and / or calendar events, An item that displays information about a particular type of object, such as. )

While the second user interface is displayed, the device detects a fourth input at a position on the touch sensitive surface corresponding to at least one selectable information item (818).

In response to detecting the fourth input, the device displays selectable information (eg, within a mini-application object that displays multiple stock lists and corresponding stock prices, each stock list provides selectable information Show, Show additional information associated with (820). Additional information about the stock list (such as stock price history) in response to a tap input detected on one of the stock lists in the mini-application object. Appears in the mini-application object). In some embodiments, the information associated with the selectable information item replaces the display of at least one previously displayed portion of the mini-application object. In some embodiments, the mini-application object is extended and additional information is displayed in the mini-application object's extension area.

In some embodiments, the first mini-application object 5162 is a mini-application object (eg, mini-application object 5162, 5125 and / or 5132) in the second user interface (eg, as shown in FIG. 5AF). Is added to the top position in the vertical stack of (822). In some embodiments, the first mini-application object 5162 is at another location within the stack, grid, or other set of mini-application objects (eg, bottom, leftmost and / or rightmost). Will be added to. In some embodiments, the user can reposition one or more mini-application objects within a set of mini-application objects.

In some embodiments, the first mini-application object identifies the identification information that identifies the first application (eg, as shown in FIG. 5U, the text 5131 that identifies the corresponding application and / or the corresponding application. The icon 5135) is included (824).

In some embodiments, the first mini-application object is a subset of the functionality provided by the first application (eg, weather data, stock price data, traffic data, location data for remote devices, and / or maps. Configured to provide data from a remote device (such as data), determine the most recent calendar reservation, and / or determine the travel time to a given location and / or point of interest. Has been (826).

In some embodiments, the first mini-application object is a subset of the content from the first application (eg, text, images, parts of the calendar, maps (electronic device and / or vehicle relative to the location of the electronic device). Includes (such as a live update map showing the location) travel time to a given location and / or target location, weather data, and / or stock data) (828). In some embodiments, the content within the mini-application object is updated at regular intervals and / or in response to user input.

In some embodiments, in response to detecting the first input, the device is selectable corresponding to the first application (eg, in a panel displayed adjacent to the mini-application object). Display (830) menu 5154 for choices (eg, one or more quick action menu items).

In some embodiments, a given input on the first mini-application object 5162 launches the first application (eg, launches a mail application) (832).

In some embodiments, the first mini-application object operates as a stand-alone application that resides in the memory of the device, unlike the first application (834).

In some embodiments, the first mini-application object acts as an extension or component of the first application (836).

In some embodiments, the first mini-application object has a dedicated memory portion for temporary storage of information (838).

In some embodiments, the memory portion is accessible by the first application (840).

It should be noted that the particular order described for the operations in FIGS. 8A-8C is merely exemplary and the order described is not intended to indicate that the operation is the only order in which it can be performed. I want to be understood. One of ordinary skill in the art will recognize various methods for reordering the operations described herein. In addition, details of the other processes described herein with respect to the other methods described herein (eg, methods 600, 700, and 900) can be found in Method 800 described above with respect to FIGS. 8A-8C. Note that it can be applied in a similar way. For example, the contacts, gestures, user interface objects, and intensity thresholds described above with reference to Method 800 may optionally refer to other methods described herein (eg, Methods 600, 700, and 900). It has one or more properties of contact, gesture, user interface object, and intensity threshold described herein. For the sake of brevity, those details are not repeated here.

9A-9E show flow diagrams of method 900 for navigating controls using a multi-page control panel, according to some embodiments. Method 900 comprises an electronic device (eg, device 300, FIG. 3, or portable multifunction device 100, FIG. It is executed in 1A). In some embodiments, the display is a touch screen display with a touch sensitive surface on or embedded in the display. In some embodiments, the display is separated from the touch sensitive surface. Some actions of Method 900 are optionally combined and / or the order of some actions is optionally changed.

As described below, Method 900 provides an intuitive way to navigate controls using a multi-page control panel. The method reduces the user's cognitive load when navigating controls using a multi-page control panel, thereby producing a more efficient human-machine interface. Battery-powered electronic devices save power and extend time between battery charges by allowing users to navigate controls faster and more efficiently using a multi-page control panel.

The initial user interface (eg, application user interface 5318, springboard user interface 400 with multiple application icons, or wake screen user interface 512 displayed when the electronic device wakes from the display off state) is displayed on the display. While doing so, the device receives a first swipe gesture, such as an upward swipe by contact 5322 along the path indicated by the arrow 5320, as shown in FIG. 5AU. ) Is detected (902).

In response to detecting the first gesture, the device displays the first page of the multi-page control panel 5208 on the display, as shown in FIG. 5AV (904). In some embodiments, the first page of the multi-page control panel is displayed as an overlay over a portion of the initial user interface (eg, overlaid on the calendar application user interface 5318). In some embodiments, the first page of the multi-page control panel 5208 gradually rises from the bottom edge of the display as the contact moves upward in the first gesture. The first page of the multi-page control panel includes a plurality of device control affordances (eg, the control affordances 520-10228 described with respect to FIG. 5AL). In some embodiments, control affordances correspond to the control of one or more features of the device and / or the control to initiate the display of an application interface, such as the application interface of an application executed by the device.

While displaying the first page of the multi-page control panel, the device detects a second gesture on the touch-sensitive surface (906). For example, the second gesture is a second swipe gesture, such as a horizontal swipe in a first direction orthogonal to the direction of the first swipe gesture (eg, by contact 5325 along the path indicated by arrow 5327). is there. In some embodiments, the second gesture is not tied to a particular control affordance, for example, the second gesture is a second (inside and / or adjacent to the multi-page control panel user interface). It has the same effect at any position where the two gestures are received.

In response to detecting the second gesture, the device is a second page of the multi-page control panel 5208 (eg, as shown in FIG. 5AW), the second page of the multi-page control panel. The page includes a plurality of content playback control affordances (eg, playback position control sliders, volume control sliders, play / pause, fast forward and / or rewind controls, such as the control affordances 5242-5252 described with respect to FIG. 5AM). Display the second page of the multi-page control panel (908). In some embodiments, the content reproduction control affordance is different from the device control affordance. The second page of the multi-page control panel 5208 replaces the first page of the multi-page control panel 5208 on the display. In some embodiments, an animation (eg, detected) that slides the first page during a transition in which the first page of the multi-page control panel user interface 5208 replaces the first page of the multi-page control panel 5208. A horizontal slide that follows the gesture is displayed.

The multi-page control panel provides more extensive access to common application features and / or device settings than a single page control panel. Typically, the control panel displays the device in multiple device states (eg, display on state, home screen user interface with multiple application icons displayed, fully activated state, or application user interface displayed. It is accessible through the input detected while in any of the fully activated states). Providing a multi-page control panel allows the user to, for example, quit the application, fully activate the device, and / or access the settings menu to access such features. Increasing the number of common features and settings that can be accessed, which improves the operation of the device, makes the user device interface more efficient, and also allows the user to use the device faster and more efficiently. By enabling it, it reduces power usage and improves the battery life of the device.

In some embodiments, while displaying the second page of the multi-page control panel, the device follows the path indicated by arrow 5331 on the touch-sensitive surface (eg, as shown in FIG. 5AW). Detect a third gesture (by contact 5329 moving along or by contact 5339 moving along the path indicated by arrow 5337, as shown in FIG. 5AY) (910).

In some embodiments, in response to detecting a third gesture, the third gesture is the first gesture type (eg, the gesture is a second gesture, eg, as shown in FIG. 5AW). A downward swipe, such as a downward swipe starting near the top of the control user interface, the gesture is a tap outside the second control user interface, and / or the gesture corresponds to the control user interface release affordance 5234. According to the determination (a position tap), the device stops displaying the second page of the multi-page control panel 5208 on the display (as shown in FIG. 5AX, the overlay portion of the initial user interface). To clarify) (912). For example, the multi-page control panel 5208 appears to gradually descend as the vertical length of the multi-page control panel 5208 decreases with downward movement of the contact 5329 within the third gesture.

In some embodiments, the third gesture is a second gesture type different from the first gesture type (eg, the third gesture follows the path indicated by arrow 5337, as shown in FIG. 5AY. The device is a third page of the multi-page control panel and the third page of the multi-page control panel is according to the determination (a horizontal swipe in the first direction, such as an input by a moving contact 5339). Display a third page of the multi-page control panel (eg, as shown in FIG. 5AZ), which includes a plurality of remote device control affordances (914). In some embodiments, the third page of the multi-page control panel is a virtual remote control affordance (eg, 5272-5286) for controlling one or more home automation devices, eg, as described with respect to FIG. 5AN. )including. In some embodiments, the remote device control affordance differs from the content playback control affordance on the second page and the device control affordance on the first page of the multi-page control panel.

In some embodiments, the plurality of device control affordances, the plurality of content reproduction control affordances, and / or the plurality of remote device control affordances are arranged according to a predetermined layout. In some embodiments, the third gesture is not coupled to a particular control affordance, eg, any position (within and / or adjacent to the multi-page control panel user interface) where the third gesture is detected. And the third gesture has the same effect.

In some embodiments, the plurality of remote device control affordances included on the third page of the multi-page control panel 5208 are multiple definitions that can be selected by the user (eg, as described with respect to FIGS. 5AP-5AQ). Each defined area of an area (eg, a room and / or campus building) (eg, a "scene" such as a room and / or building, eg, a "Living Room" as shown in FIG. 5AZ). Corresponds to different devices within (916).

In some embodiments, displaying the third page of the multi-page control panel 5208 will result in a third page of the multi-page control panel when the third page of the multi-page control is last displayed. Includes redisplaying one or more remote device controls that were included (918).

In some embodiments, displaying a third page of the multi-page control panel displays on the third page of the multi-page control panel (eg, a GPS and / or Wi-Fi signal received by the device 100). Includes displaying one or more remote device controls corresponding to the definition area in which the device is currently located (determined using) (920).

In some embodiments, a third page of the multi-page control panel, when activated, is described with respect to a plurality of defined definition area identifiers (eg, FIGS. 5AP-5AQ) that can be selected by the user. Also includes definition area restore affordances (eg, affordances 5292 for displaying / stopping multiple definition area identifiers) configured to cause the display of, for example, a list of "scenes" (922).

In some embodiments, the first page of the multi-page control panel 5208 overlays a portion of the initial user interface 5318 (eg, as shown in FIG. 5AV) (924). For example, the first page of the multi-page control panel 5208 and the initial user interface 5318 are displayed at the same time, and a part of the initial user interface 5318 is visually hidden by the first page of the multi-page control panel 5208. In some embodiments, the first page of the multi-page control panel is translucent, and part of the initial user interface 5318 that resides behind the first page of the multi-page control panel 5208 is the multi-page control panel 5208. Partially visible through the first page of. In some embodiments, the second page of the multi-page control panel 5208 also overlays part of the initial user interface 5318. In some embodiments, one or more pages of the multi-page control panel user interface 5208 have a fixed size that is smaller than the size of the display and the size of the initial user interface 5318.

In some embodiments, the initial user interface is an application user interface (eg, calendar application user interface 5318) (926). The multi-page control panel 5208 and the application user interface 5318 are displayed simultaneously, for example, as shown in FIG. 5AW. In some embodiments, the multi-page control panel user interface is accessible without exiting the application.

In some embodiments, after stopping the display of the second page of the multi-page control panel, the device is subjected to a fourth gesture on the touch-sensitive surface (eg, as shown in FIG. 5AX, by arrow 5335. An upward swipe to redisplay the control user interface, such as an input by contact 5333 moving along the indicated path) is detected (928).

In some embodiments, in response to detecting the fourth gesture, the device redisplays the second page of the multi-page control panel 5208 (eg, as shown in FIG. 5AY) (930). .. In some embodiments, when an input is received to disengage the multi-page control panel user interface and then an input to redisplay the multi-page control panel user interface is received, the multi-page control panel user The most recently displayed page of the interface (displayed before the input to exit the multi-page control panel user interface) is displayed in response to subsequent input to redisplay the multi-page control panel user interface. To. For example, if the third page of the multi-page control panel is displayed just before receiving input to dismiss the multi-page control panel, it responds to subsequent input to redisplay the multi-page control panel user interface. The third page of the multi-page control panel is redisplayed (eg, as shown by FIGS. 5AR-5AT).

In some embodiments, the plurality of device control affordances (eg, the control affordances 5242-5252 described with respect to FIG. 5AM) include at least one toggle control (932). Toggle control is, for example, control for setting the state of components of an electronic device (eg, flashlight control for turning on / off the flashlight of the device, Wi-Fi control, Bluetooth® control), and / or for electronic devices. Controls that set the state of the mode (eg, enable and disable the mode) (eg, night shift control, airplane mode control, incoming call rejection mode control, rotation lock control and / or mirroring mode control (eg, "Apple (eg, Apple) Multiple state control such as (registered trademark) TV Mirror: On "). For example, as shown in FIGS. 5BF to 5BG, the Wi-Fi control 5212 is effective from the invalid state in response to the input by the contact 5342. Toggled to state.

In some embodiments, the second page of the multi-page control panel 5208 shown in FIG. 5BA is the routing destination of the currently playing media, such as "currently playing on AirPods®". Includes a display (eg, content routing destination information area 5260) of a device to be routed or transmitted (934).

In some embodiments, while displaying the second page of the multi-page control panel (eg, as shown in FIG. 5BD), the device is the routing destination for the currently playing media and is currently The routing destination of the media being played is on the touch-sensitive surface corresponding to the display 5260 of the routing destination of the media currently being played, including the first device (Bluetooth® Speaker) corresponding to the first zone. Detects input at position (eg, by swipe gesture, eg, by contact 5323 moving along the path indicated by arrow 5345) (936). In some embodiments, the routing destination corresponding to the first zone is, for example, headphones connected to the electronic device, speakers of the electronic device, and / or one or more remote devices (eg, remote speakers and / or). The remote display in the first room). In response to detecting an input at a position on the touch-sensitive surface that corresponds to the display of the routing destination of the currently playing media, the device outputs the media output to a second device corresponding to the second zone (FIG. 5BE). Route to "Airpods®") as indicated by (938). In some embodiments, the second zone includes, for example, a remote speaker and / or a remote display in the second room. In some embodiments, the display 5260 changes to reflect a change in the routing destination (as shown in FIG. 5BE).

In some embodiments, the second page of the multi-page control panel 5208 is a media routing destination restore affordance (eg, affordance 5262 for displaying / stopping the display of the routing destination menu) of the currently playing media. A chevron placed adjacent to the display of the current destination) is included (940).

In some embodiments, the device detects an input (eg, by contact 5324, as shown in FIG. 5BA) at a position on the touch-sensitive surface corresponding to the media routing destination restore affordance 5262 (942).

In some embodiments, in response to detecting an input at a position corresponding to the media routing destination restore affordance, the device displays a list of media routing destination choices 5236, as shown in FIG. 5BB (944). (For example, replace and / or shift the display position of at least a portion of the content in the second page with the list of media routing destination choices).

In some embodiments, the device populates the list of media routing destination options 5326 with the identifiers of the electronic device and at least one additional device. In some embodiments, populating the list of media routing destination choices with at least one additional device is media routing according to the determination that the electronic device is currently routing media output to one or more remote devices. The list of choices is populated with one or more remote devices that the electronic device is currently routing media output to, and the determination that the electronic device has been previously paired with one or more pairable devices. According to (946), populating a list of media routing destination choices with one or more pairing-enabled devices. In some embodiments, if the number of pairing-enabled devices that the electronic device has recently paired exceeds the threshold number, the list is populated with the latest pairing-enabled device threshold number. In some embodiments, currently undetected pairing capable devices are not included in one or more pairing capable devices. In some embodiments, the electronic device (eg, iPhone® 5328, as shown in FIG. 5BB) is the first media routing destination in the list of media routing destination options, and the electronic device is currently. A pairing-enabled device (if present) followed by a device routing the media (if present) (eg, Airpods® 5260, as shown in FIG. 5BB), followed by an electronic device previously paired. (For example, Bluetooth® Speaker5332, as shown in FIG. 5BB), followed by the signaling device (if present) detected by the electronic device.

In some embodiments, the device lists one or more media routing destination options according to the determination that one or more signal transmission (eg, Bluetooth® signal transmission) devices have been detected in close proximity to the electronic device. Populate with one or more signaling devices (948).

In some embodiments, the electronic device comprises one or more sensors for detecting the strength of contact with the touch sensing surface 112 (950). A device comprises detecting a first contact 5342 at a position on a touch sensing surface 112 corresponding to a first control affordance 5212 of a plurality of device control affordances, as shown in FIGS. 5BF-5BI. The input of 1 is detected (952). In response to the detection of the first input, the first input increases above the _{deep pressing intensity threshold IT D,} as indicated by the second reference (eg, intensity level meter 5344). According to the determination that the second criterion (eg, the enhanced control criterion) is met, the second criterion (eg, the enhanced control criterion) is of the first contact because the second criterion is met. The characteristic intensity requires that the first intensity threshold be met, and the device has one or more modification options for control corresponding to the first control affordance 5212 (eg, 5354, 5356 as shown in FIG. 5BI). , 5358, 5360, 5362) (954). For example, the change options are mode change options (eg, enable Wi-Fi on the selected network) and / or duration change options (eg, enable Wi-Fi for the next time). In some embodiments, the change options are displayed in menu 5350. The first input meets a third criterion (eg, control toggle criterion) (eg, the first input is a tap input), and the third criterion (eg, control toggle criterion) is the first contact. According to the determination that the characteristic intensity of is not required to satisfy the first intensity threshold, the device toggles the control function corresponding to the first control affordance (eg, Wi-as shown in FIGS. 5BF-5BG). Enable / disable Fi).

In some embodiments, the device is one or more (by contact 5352, as shown in FIG. 5BI) while displaying one or more change options for control corresponding to the first control affordance. Detects a second input that activates the first change option of the change option (eg, change option 5356) (956). In some embodiments, the second input is a continuation of the first input by contact. In some embodiments, the second input is a separate tap input for the change options. In response to detecting the second input, the device changes the control corresponding to the first control affordance according to the activated first change option (eg, as shown in FIG. 5BJ). 958).

In some embodiments, each page of the multi-page control panel comprises a display of the total number of pages that the multi-page control panel has (eg, dots corresponding to each page) of the display pages of the multi-page control panel. Indicators corresponding to the currently displayed page are highlighted on each page (eg, as shown in display 5236 in FIGS. 5AL, 5AM, and 5AN) (960). For example, display 5236 indicates the position of the current page among a plurality of pages.

It is understood that the particular order described for the actions in FIGS. 9A-9E is merely exemplary and the order described is not intended to indicate that the actions are the only order in which they can be performed. I want to be. One of ordinary skill in the art will recognize various methods for reordering the operations described herein. Further, details of the other processes described herein with respect to the other methods described herein (eg, methods 600, 700, and 800) can be found in Method 900 described above with respect to FIGS. 9A-9E. Note that similar methods are applicable. For example, the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations described above with respect to Method 900 are optionally one or more of the contacts, gestures, user interface objects, intensity thresholds, focus selectors, and animations. It has the properties described herein with reference to the properties (eg, methods 600, 700, and 800). For the sake of brevity, those details are not repeated here.

According to some embodiments, FIG. 10 shows a functional block diagram of the electronic device 1000 configured according to the principles of the various embodiments described. Functional blocks of this device are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of the various embodiments described. Those skilled in the art will appreciate that the functional blocks described in FIG. 10 are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Accordingly, the description herein supports any possible combination or separation of functional blocks described herein, or any further definition, optionally.

As shown in FIG. 10, the electronic device 1000 includes a display unit 1002, a touch sensing surface unit 1004, and a processing unit 1008 connected to the display unit 1002 and the touch sensing surface unit 1004. In some embodiments, the electronic device comprises one or more sensor units 1006, the processing unit 1008 being coupled to a display unit 1002, a touch sensing surface unit 1004, and one or more sensor units 1006. In some embodiments, the processing unit 1008 includes a detection unit 1010, an activation unit 1012, a display activation unit 1014, a replacement unit 1016, and a determination unit 1018.

The processing unit 1008 is configured to detect the first input (eg, using the detection unit 1010) while the device is in the display off state. In response to detecting the first input, the processing unit 1008 activates the display unit 1002 of the device (eg, using the activation unit 1012) and puts the device display on on the display unit 1002. It is configured to enable the display of the corresponding first user interface (eg, using the display enablement unit 1014). While displaying the first user interface corresponding to the display on state of the device, the processing unit 1008 may detect a swipe gesture on the touch sensing surface unit 1004 (eg, using the detection unit 1010). It is configured. In response to detecting a swipe gesture on the touch-sensitive surface unit 1004, the processing unit 1008 determines that the device is in display-on lock mode and the swipe gesture is in the first direction. The display of the user interface is the second user interface, the second user interface replacing the display of the second user interface displaying the first content (eg, using the replacement unit 1016) and the device. Is in the unlock mode with the display on, and according to the determination that the swipe gesture is in the first direction, the processing unit 1008 displays the display of the first user interface as the second user interface and the second user interface. The user interface replaces the display of the first user interface (eg, using the replacement unit 1016) that displays the first content and the first additional content that is not displayed when the device is in display-on lock mode. It is configured as follows.

In some embodiments, the first content includes one or more notifications and the first additional content includes restricted notification information.

In some embodiments, the first content comprises one or more mini-application objects and the first additional content comprises restricted mini-application object information.

In some embodiments, the processing unit 1008 is in a lock mode with the device in the display on state in response to detecting a swipe gesture on the touch sensitive surface unit 1004, and the swipe gesture is in the first direction. According to the determination that they are in different second directions, the display of the first user interface is the display of the third user interface, and the third user interface is the display of the third user interface that displays the second content. A third user interface display is displayed according to the determination (eg, using replacement unit 1016) that the device is in unlock mode with display on and the swipe gesture is in the second direction. A user interface, the third user interface, is a display of a third user interface that displays a second content and a second additional content that is not displayed when the device is in display-on lock mode (eg,). It is further configured to replace (using the replacement unit 1016).

In some embodiments, the second content comprises one or more mini-application objects and the second additional content comprises restricted mini-application object information.

In some embodiments, the second content includes one or more notifications and the second additional content includes limited notification information.

In some embodiments, the second user interface includes a first search input area and the third user interface includes a second search input area.

In some embodiments, the processing unit 1008 is further configured to detect an input (eg, using the detection unit 1010) in each of the search input areas of the first search input area and the second search input area. Has been done. In response to detecting an input in each search area, the processing unit 1008 displays a first set of search results (eg, a display enablement unit) according to the determination that the device is in display-on locked mode. According to the determination (using 1014) that the device is in unlock mode with display on, display a second set of search results that is different from the first set of search results (eg, display enablement unit). It is configured to be activated (using 1014).

In some embodiments, the determination of whether the device is in display-on state unlock mode or lock mode is made in response to detecting a first input.

In some embodiments, the determination of whether the device is in display-on lock mode or display-on unlock mode responds to detecting a swipe gesture on the touch-sensitive surface unit 1004. Is done.

In some embodiments, the electronic device comprises one or more sensor units 1006, and detecting the first input uses one or more sensor units 1006 to change the context of the device (eg,). Detecting (using the detection unit 1010) and determining whether the change in the context of the device meets the display activation criteria in response to the detection of the change in the context of the device.

In some embodiments, detecting the first input comprises detecting the activation of the display activation affordance (eg, using the detection unit 1010).

In some embodiments, detecting the first input is a contact with the display-activated affordance, with the display-activated affordance, which activates the display-activated affordance and does not activate the display-activated affordance. Includes detecting contact (eg, using detection unit 1010).

In some embodiments, detecting the first input activates the display activation affordance to activate the display of the device and associates it with the display activation affordance without activating the display activation affordance. Includes detecting contact with display activation affordances, performing at least one additional function.

In some embodiments, the processing unit 1008 is configured to detect a second input (eg, using the detection unit 1010) while the first user interface is displayed. In response to detecting the second input, the processing unit 1008 displays the first user interface with a display of the user interface including a plurality of application icons corresponding to different applications of the plurality of applications (eg, for example. It is configured to replace (using the replacement unit 1016).

In some embodiments, in response to detecting a second input, the processing unit 1008 determines whether the device is in display-on state lock mode or display-on state unlock mode. To make a determination (eg, using the determination unit 1018) and enable the display of the authenticated user interface (eg, using the display enablement unit 1014) according to the determination that the device is in lock mode with the display on. It is configured in.

In some embodiments, each mini-application object is configured to perform less than a subset of the functionality of the corresponding application of each mini-application object.

In some embodiments, each mini-application object displays the corresponding application identifier for each mini-application object.

In some embodiments, each mini-application object displays a portion of the content from the corresponding application from each mini-application object.

In some embodiments, a given input on each mini-application object launches an application corresponding to each mini-application object.

In some embodiments, each mini-application object operates as a standalone application that is also present in the device's memory, unlike the associated application that is present in the device's memory.

In some embodiments, each mini-application object acts as an extension or component of the associated application on the device.

In some embodiments, each mini-application object has a dedicated memory portion for temporary storage of information.

In some embodiments, the memory portion is accessible by the corresponding full-featured application of each mini-application object.

The operation in the above information processing method is optional and includes one or more functional modules in the information processing apparatus, such as a general purpose processor (as described above with respect to FIGS. 1A and 3) or a special purpose chip. It is carried out by executing.

The operations described above with reference to FIGS. 6A-6E are optionally performed by the components shown in FIGS. 1A-1B or 10. For example, the detection operations 602 and 606 and the display operations 604 and 608 are optionally performed by the event sort unit 170, the event recognition unit 180, and the event processing unit 190. The event monitor 171 of the event sort unit 170 detects the contact on the touch-sensitive display 112, and the event dispatcher module 174 distributes the event information to the application 136-1. The corresponding event recognition unit 180 of the application 136-1 compares the event information with the corresponding event definition 186, and the first contact (or device rotation) at the first position on the touch sensing surface is determined. Determines whether to respond to a given event or sub-event, such as selecting an object on the user interface or rotating a device from one orientation to another. When the corresponding predetermined event or sub-event is detected, the event recognition unit 180 activates the event processing unit 190 associated with the detection of the event or sub-event. The event processing unit 190 arbitrarily uses or calls the data update unit 176 or the object update unit 177 in order to update the application internal state 192. In some embodiments, the event processor 190 accesses the corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes will be implemented based on the components shown in FIGS. 1A-1B.

According to some embodiments, FIG. 11 shows a functional block diagram of an electronic device 1100 configured according to the principles of various described embodiments. Functional blocks of this device are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of various described embodiments. It will be understood by those skilled in the art that the functional blocks described in FIG. 11 will be optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Let's do it. Accordingly, the description herein supports any possible combination or separation of functional blocks described herein, or any further definition, optionally.

As shown in FIG. 11, the electronic device 1100 includes a display unit 1102, a touch sensing surface unit 1104, and a processing unit 1108 coupled with the display unit 1102 and the touch sensing surface unit 1104. In some embodiments, the processing unit 1108 includes a detection unit 1110, a display activation unit 1112, an activation unit 1114, and a replacement unit 1116.

While the device is in the display off state, the processing unit 1108 detects the first input (eg, using the detection unit 1110) and activates the display of the device, eg, using the activation unit 1114. In response to detecting the first input, the processing unit 1108 activates the display unit 1102 of the device (eg, using the activation unit 1114) to display the device. The processing unit 1108 is configured to enable the display of the first user interface corresponding to the display on state of the device on the unit 1102 (eg, using the display enablement unit 1112). It is configured to detect a swipe gesture on the touch sensing surface unit 1104 (eg, using the detection unit 1110) while displaying the first user interface corresponding to the display on state of the device. In response to detecting the swipe gesture, the processing unit 1108 displays the first user interface with the camera application user interface (eg, using the replacement unit 1116) according to the determination that the swipe gesture is in the first direction. A mini-application object user configured to include multiple mini-application objects in the display of the first user interface according to the determination that the swipe gesture is in a second direction different from the first direction. An interface, each mini-application object of a plurality of mini-application objects, replaces the display (eg, using replacement unit 1116) of a mini-application object user interface, which has a corresponding application stored on the device. According to the determination that the swipe gesture is in a third direction different from the first direction and the second direction, display the first page of the multi-page control panel user interface (eg, display enablement unit 1112). A notification user interface that is enabled (using) and configured to display multiple notifications according to the determination that the swipe gesture is in a fourth direction that is different from the first, second, and third directions. Enables the display of (eg, using the display enablement unit 1112) It is configured as follows.

In some embodiments, replacing the display of the first user interface with the display of the camera application user interface displays one or more control affordances associated with the camera application until the control display period elapses. Including delaying.

In some embodiments, the processing unit 1108 is further configured to detect a gesture returning to the first user interface (eg, using the detection unit 1110) during the control display period. In response to detecting a gesture to return to the first user interface during the control display period, the processing unit 1108 displays the camera application user interface with the display of the first user interface (eg, replacement unit 1116). It is configured to replace (using).

In some embodiments, the processing unit 1108 gestures at the position of the touch sensitive surface unit 1104 corresponding to the first page of the multi-page control panel user interface while the multi-page control panel user interface is displayed. (For example, using the detection unit 1110) and in response to detecting the gesture at the position corresponding to the first page of the multi-page control panel, on the second page of the multi-page control panel user interface. It is further configured to activate the display (eg, using the display activation unit 1112).

In some embodiments, the processing unit 1108 is a touch sensitive surface unit that corresponds to a second page of the multi-page control panel user interface while the second page of the multi-page control panel user interface is displayed. A multi-page control panel user in response to detecting a gesture at position 1104 (using, for example, detection unit 1110) and detecting a gesture at a position corresponding to the second page of the multi-page control panel user interface It is further configured to enable the display of the third page of the interface (eg, using the display enablement unit 1112).

In some embodiments, the processing unit 1108 performs a release gesture at a position on the touch-sensitive surface unit 1104 corresponding to each notification included in the notification user interface while the notification user interface is displayed (eg,). It is further configured to stop displaying each notification in the notification user interface in response to detecting (using the detection unit 1110) and detecting the release gesture.

In some embodiments, the processing unit 1108 performs an activation gesture (eg,) at a position on the touch-sensitive surface unit 1104 corresponding to the first notification included in the notification user interface while the notification user interface is displayed. It is further configured to launch the application corresponding to the first notification in response to the detection (using the detection unit 1110) and the detection of the launch gesture.

In some embodiments, the processing unit 1108 is positioned on the touch sensitive surface unit 1104 corresponding to the first mini-application object included in the mini-application object user interface while the mini-application object user interface is displayed. It is further configured to detect the launch gesture in (eg, using the detection unit 1110) and launch the application corresponding to the first mini-application object in response to the discovery of the launch gesture. ..

In some embodiments, the processing unit 1108 is a second mini-application object contained in the mini-application object user interface while the mini-application object user interface is displayed, and the second mini-application object is , The information expansion gesture at the position on the touch sensing surface unit 1104 corresponding to the second mini-application object, including the mini-application object information, was detected (eg, using the detection unit 1110) and the information expansion gesture was detected. In response to, the second mini-application object is extended to display the mini-application object information and additional mini-application object information in the extended second mini-application object (eg, using the display enablement unit 1112). It is configured to be enabled.

In some embodiments, each mini-application object is configured to perform less than a subset of the functionality of the corresponding application of each mini-application object.

In some embodiments, each mini-application object displays the corresponding application identifier for each mini-application object.

In some embodiments, each mini-application object displays a portion of the content from the corresponding application from each mini-application object.

In some embodiments, a given input on each mini-application object launches an application corresponding to each mini-application object.

In some embodiments, each mini-application object operates as a standalone application that is also present in the device's memory, unlike the associated application that is present in the device's memory.

In some embodiments, each mini-application object acts as an extension or component of the associated application on the device.

In some embodiments, each mini-application object has a dedicated memory portion for temporary storage of information.

In some embodiments, the memory portion is accessible by the corresponding full-featured application of each mini-application object.

The operation in the above-mentioned information processing method optionally executes one or more functional modules in the information processing apparatus, such as a general-purpose processor or a special-purpose chip (as described above with respect to FIGS. 1A and 3). It is carried out by doing.

The operations described above with reference to FIGS. 7A-7E are optionally performed by the components shown in FIGS. 1A-1B or 11. For example, the detection operations 702 and 706 and the display operations 704 and 708 are optionally performed by the event sheet unit 170, the event recognition unit 180, and the event processing unit 190. The event monitor 171 of the event sort unit 170 detects the contact on the touch-sensitive display 112, and the event dispatcher module 174 distributes the event information to the application 136-1. The corresponding event recognition unit 180 of the application 136-1 compares the event information with the corresponding event definition 186, and the first contact (or device rotation) at the first position on the touch sensing surface is determined. Determines whether to respond to a given event or sub-event, such as selecting an object on the user interface or rotating a device from one orientation to another. When the corresponding predetermined event or sub-event is detected, the event recognition unit 180 activates the event processing unit 190 associated with the detection of the event or sub-event. The event processing unit 190 arbitrarily uses or calls the data update unit 176 or the object update unit 177 in order to update the application internal state 192. In some embodiments, the event processor 190 accesses the corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes can be performed based on the components shown in FIGS. 1A and 1B.

According to some embodiments, FIG. 12 shows a functional block diagram of an electronic device 1200 configured according to the principles of various described embodiments. Functional blocks of the device are optionally implemented by hardware, software, or a combination of hardware and software to implement the principles of various described embodiments. Those skilled in the art will appreciate that the functional blocks described in FIG. 12 are optionally combined into or separated into sub-blocks to implement the principles of the various described embodiments. Accordingly, the description herein supports any possible combination or separation of functional blocks described herein, or any further definition, optionally.

As shown in FIG. 12, the electronic device 1200 is configured to display a first user interface that includes a plurality of application icons corresponding to different applications among the plurality of applications installed on the device 1202. And a touch sensing surface unit 1204, and a processing unit 1208 coupled to the display unit 1202 and the touch sensing surface unit 1204. In some embodiments, the processing unit 1208 includes a detection unit 1210, a display activation unit 1212, an additional unit 1214, and a replacement unit 1216.

The processing unit 1208 is a first application icon of a plurality of application icons on the touch sensing surface unit 1204, and the first application icon corresponds to the first application among the plurality of applications. It is configured to detect the first input corresponding to the application icon (eg, using the detection unit 1210). In response to detecting the first input, the processing unit 1208 displays a preview of the first mini-application object or the first mini-application object in the overlay area (eg, using the display enablement unit 1212). It is configured to be enabled. In some embodiments, the first mini-application object corresponds to the first application of the plurality of applications, and the overlay area provides the first mini to the second user interface for displaying the plurality of mini-application objects. Includes affordances for adding application objects. The processing unit 1208 receives a second input at a position on the touch sensing surface unit 1204 corresponding to the affordance for adding the first mini-application object to the second user interface (eg, using the detection unit 1210). It is further configured to detect. In response to detecting the second input, the processing unit 1208 adds the first mini-application object (eg, using the add-on unit 1214) to the second user interface that displays the plurality of mini-application objects. It is configured to do.

In some embodiments, the processing unit 1208 is a third input on the touch sensitive surface unit 1204 while displaying the first user interface, the third input of which is the first user. It is configured to detect a third input (eg, using the detection unit 1210) that meets the first criterion for navigating from the interface to the second user interface. In response to detecting the third input, processing unit 1208 is configured to replace the display of the first user interface with the second user interface (eg, using replacement unit 1216). Replacing the display of the first user interface with the second user interface can stop displaying the first user interface (eg, using the display enablement unit 1212) and second. In the user interface, a first mini-application object and at least a second mini-application object corresponding to a second application among a plurality of applications different from the first application are simultaneously (for example, display enablement unit 1212). Includes enabling the display of the second user interface (eg, using the display enablement unit 1212), including displaying (using).

In some embodiments, the first mini-application object contains at least one selectable information item and the processing unit 1208 has at least one selectable while the second user interface is displayed. A selectable information item is detected in response to detecting a fourth input (eg, using the detection unit 1210) at a position on the touch sensing surface unit 1204 corresponding to the information item and detecting the fourth input. It is configured to enable the display of additional information associated with (eg, using the display enablement unit 1212).

In some embodiments, the first mini-application object is added at the top position in the vertical stack of mini-application objects in the second user interface.

In some embodiments, the first mini-application object contains identifying information that identifies the first application.

In some embodiments, the first mini-application object is configured to provide a subset of the functionality provided by the first application.

In some embodiments, the first mini-application object comprises a subset of content from the first application.

In some embodiments, processing unit 1208 displays a menu of selectable choices corresponding to the first application (eg, display enablement unit 1212) in response to detecting the first input. It is further configured to activate (using).

In some embodiments, a given input on the first mini-application object launches the first application.

In some embodiments, the first mini-application object operates as a stand-alone application that resides in the device's memory, unlike the first application.

In some embodiments, the first mini-application object acts as an extension or component of the first application.

In some embodiments, the first mini-application object has a dedicated memory portion for temporary storage of information.

In some embodiments, the memory portion is accessible by the first application.

The operation in the above-mentioned information processing method optionally executes one or more functional modules in the information processing apparatus, such as a general-purpose processor or a special-purpose chip (as described above with respect to FIGS. 1A and 3). It is carried out by doing.

The operations described above with reference to FIGS. 8A-8C are optionally performed by the components shown in FIGS. 1A, 1B, or 12. For example, the display operations 802 and 806 and the detection operations 804 and 808 are optionally performed by the event sort unit 170, the event recognition unit 180, and the event processing unit 190. The event monitor 171 of the event sort unit 170 detects the contact on the touch-sensitive display 112, and the event dispatcher module 174 distributes the event information to the application 136-1. The corresponding event recognition unit 180 of the application 136-1 compares the event information with the corresponding event definition 186, and the first contact (or device rotation) at the first position on the touch sensing surface is determined. Determines whether to respond to a given event or sub-event, such as selecting an object on the user interface or rotating a device from one orientation to another. When the corresponding predetermined event or sub-event is detected, the event recognition unit 180 activates the event processing unit 190 associated with the detection of the event or sub-event. The event processing unit 190 arbitrarily uses or calls the data update unit 176 or the object update unit 177 in order to update the application internal state 192. In some embodiments, the event processor 190 accesses the corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes will be performed based on the components shown in FIGS. 1A, 1B.

According to some embodiments, FIG. 13 shows a functional block diagram of the electronic device 1300 configured according to the principles of various described embodiments. Functional blocks of this device are optionally implemented by hardware, software, or a combination of hardware and software, and implement the principles of various described embodiments. Those skilled in the art will appreciate that the functional blocks described in FIG. 13 are optionally combined or separated into sub-blocks to implement the principles of various described embodiments. Accordingly, the description herein supports any possible combination or separation of functional blocks described herein, or any further definition, optionally.

As shown in FIG. 13, the electronic device 1300 includes a display unit 1302, a touch sensing surface unit 1304, one or more sensor units 1306, a display unit 1302, a touch sensing surface unit 1304, and one or more sensor units. Includes processing unit 1308 coupled to 1306. In some embodiments, the electronic device comprises one or more sensor units 1006, the processing unit 1008 being coupled to a display unit 1002, a touch sensing surface unit 1004, and one or more sensor units 1006. In some embodiments, the processing unit 1308 includes a detection unit 1310, a display activation unit 1312, a routing unit 1314, a populating unit 1316, a modification unit 1318, and a toggle unit 1320.

The processing unit 1308 is configured to detect the first gesture of the touch sensing surface unit 1304 (eg, using the detection unit 1310) while displaying the initial user interface on the display unit 1302. In response to detecting the first gesture, the processing unit 1308 displays on the display unit 1302 the first page of the multi-page control panel (eg, using the display enablement unit 1312). It is configured to enable. In some embodiments, the first page of the multi-page control panel comprises a plurality of device control affordances. While displaying the first page of the multi-page control panel, the processing unit 1308 is configured to detect a second gesture (eg, using the detection unit 1310) on the touch sensing surface unit 1304. ing. In response to detecting the second gesture, the processing unit 1308 is configured to enable the display of the second page of the multi-page control panel (eg, using the display enablement unit 1312). Has been done. In some embodiments, the second page of the multi-page control panel comprises a plurality of content playback control affordances, and the second page of the multi-page control panel is the first page of the multi-page control panel on the display unit 1302. Replace page.

In some embodiments, while displaying the second page of the multi-page control panel, the processing unit 1308 makes a third gesture on the touch sensitive surface unit 1304 (eg, using the detection unit 1310). ) It is configured to detect, and in response to detecting a third gesture, it displays the second page of the multi-page control panel according to the determination that the third gesture is the first gesture type. Stop displaying on the 1302 (eg, using the display enablement unit 1312).

In some embodiments, the processing unit 1308 displays a third page of the multi-page control panel in response to a determination that the third gesture is a second gesture type different from the first gesture type. It is configured to activate what it does (eg, using the display activation unit 1312). In some embodiments, the third page of the multi-page control panel comprises a plurality of remote device control affordances.

In some embodiments, the plurality of remote device control affordances included on the third page of the multi-page control panel correspond to different devices within each definition area of the plurality of definition areas selectable by the user.

In some embodiments, displaying the third page of the multi-page control panel is included in the third page of the multi-page control panel when the third page of the multi-page control is last displayed. It involves redisplaying one or more remote device controls (eg, using the display enablement unit 1312).

In some embodiments, displaying a third page of the multi-page control panel is one or more remotes on the third page of the multi-page control panel that correspond to the definition area in which the device is currently located. Includes displaying device control (eg, using the display enablement unit 1312).

In some embodiments, a third page of the multi-page control panel comprises a definition area restore affordance configured to cause the display of multiple definition area identifiers selectable by the user when activated.

In some embodiments, the first page of the multi-page control panel overlays part of the initial user interface.

In some embodiments, the initial user interface is the application user interface, and the multi-page control panel and the application user interface are displayed at the same time.

In some embodiments, after stopping the display of the second page of the multi-page control panel, the processing unit 1308 uses a fourth gesture on the touch-sensitive surface unit 1304 (eg, using the detection unit 1310). It is configured to detect and redisplay the second page of the multi-page control panel (eg, using the display enablement unit 1312) in response to the detection of the fourth gesture. ..

In some embodiments, the plurality of device control affordances comprises at least one toggle control.

In some embodiments, the second page of the multi-page control panel includes displaying the routing destination of the currently playing media.

In some embodiments, while displaying the second page of the multi-page control panel, the processing unit 1308 is positioned on the touch-sensitive surface unit 1304 corresponding to the display of the routing destination of the currently playing media. Is configured to detect the input in (eg, using the detection unit 1310). In some embodiments, the routing destination of the currently playing media includes a first device corresponding to the first zone. In response to detecting an input at a position on the touch sensing surface unit 1304 corresponding to the display of the routing destination of the currently playing media, the processing unit 1308 outputs the media output to the second zone corresponding to the second zone. It is configured to route to a device (eg, using a routing unit 1314).

In some embodiments, the second page of the multi-page control panel comprises a media routing destination restore forecast, where the processing unit 1308 inputs at a position on the touch sensitive surface unit 1304 corresponding to the media routing destination restore forecast. In response to detection (using, for example, detection unit 1310) and detection of an input at a position corresponding to the media routing destination restore forecast, the display of a list of media routing destination choices (eg, display enabling unit 1312) is displayed. Is configured to enable (using).

In some embodiments, processing unit 1308 is configured to populate a list of media routing destination choices with identifiers for electronic devices and at least one additional device (eg, in populating unit 1316) and media. Populating the list of routing choices with at least one additional device causes the list of media routing choices to be routed to the electronic device according to the determination that the electronic device is currently routing the media output to one or more remote devices. Populates the media output with one or more devices currently routing (eg, using the popularizing unit 1316), and the electronic device has been previously paired with one or more pairable devices. According to the determination, the list of media routing destination options is populated with one or more pairing compatible devices (for example, to the populating unit 1316).

In some embodiments, the processing unit 1308 lists media routing destination options (eg, using the populating unit 1316) according to the determination that one or more signaling devices have been detected in close proximity to the electronic device. Is further configured to be populated by one or more signal transmitting devices.

In some embodiments, the electronic device comprises one or more sensor units 1306 for detecting the contact strength with the touch-sensitive surface unit 1304 (eg, using the detection unit 1310), the processing unit 1308. The first input (eg, using the detection unit 1310), including detecting the first contact at a position on the touch sensing surface unit 1304 corresponding to the first control affordance of the plurality of device control affordances. In response to the detection and detection of the first input, the first input is the second criterion, the second criterion is the characteristic strength of the first contact, the second criterion is Display of one or more change options for control corresponding to the first control affordance according to the determination that the second criterion is met, which requires the first intensity threshold to be met (eg,). Activated (using the display enablement unit 1312), the first input is a third criterion, which requires that the characteristic intensity of the first contact meet the first intensity threshold. It is configured to toggle the control function corresponding to the first control affordance (eg, using the toggle unit 1320) according to the determination that it does not meet the third criterion.

In some embodiments, the processing unit 1308 displays the first change option for one or more change options while displaying one or more change options for control corresponding to the first control affordance. A second input (eg, using the detection unit 1310) is detected to activate the second input, and in response to the detection of the second input, the first control is performed according to the activated first modification option. It is configured to change the controls corresponding to affordances (eg, using a change unit 1318).

In some embodiments, each page of the multi-page control panel includes an indicator of the total number of pages that the multi-page control panel has, and the indicator corresponding to the currently displayed page of the multi-page control panel is each. Highlighted on the page.

The operation in the above-mentioned information processing method optionally executes one or more functional modules in the information processing apparatus, such as a general-purpose processor or a special-purpose chip (as described above with respect to FIGS. 1A and 3). It is carried out by doing.

The operations described above with reference to FIGS. 9A-9E are optionally performed by the components shown in FIGS. 1A, 1B or 13. For example, the display operations 902, 904 and 908 and the detection operation 906 are optionally performed by the event sort unit 170, the event recognition unit 180, and the event processing unit 190. The event monitor 171 of the event sort unit 170 detects the contact on the touch-sensitive display 112, and the event dispatcher module 174 distributes the event information to the application 136-1. The corresponding event recognition unit 180 of the application 136-1 compares the event information with the corresponding event definition 186, and the first contact (or device rotation) at the first position on the touch sensing surface is the user. Determines whether to respond to a given event or sub-event, such as selecting an object on the interface or rotating a device from one orientation to another. When the corresponding predetermined event or sub-event is detected, the event recognition unit 180 activates the event processing unit 190 associated with the detection of the event or sub-event. The event processing unit 190 arbitrarily uses or calls the data update unit 176 or the object update unit 177 in order to update the application internal state 192. In some embodiments, the event processor 190 accesses the corresponding GUI updater 178 to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes will be performed based on the components shown in FIGS. 1A-1B.

The above description is given with reference to specific embodiments for purposes of explanation. However, the above exemplary description is not intended to be exhaustive or to limit the invention to the exact forms disclosed. In view of the above teachings, many changes and modifications are possible. The principles of the invention and its practical applications will be best described by others skilled in the art with various modifications suitable for the particular application conceived of the present invention and various described embodiments. These embodiments have been selected and described to enable their best use.

Claims (18)

In electronic devices with displays and touch sensitive surfaces
Detecting the first input while the electronic device is in the display off state,
In response to detecting the first input,
Activating the display of the electronic device and
On the display, a first user interface corresponding to the display ON state of the electronic device, when the electronic device in response to said first input is switched to the display ON state from the display OFF state Displaying the first user interface, which is the initial user interface to be displayed,
To detect a swipe gesture on the touch-sensitive surface while displaying the first user interface corresponding to the display-on state of the electronic device.
In response to detecting the swipe gesture on the touch-sensitive surface according to the determination that the swipe gesture is in the first direction,
Replacing the display of the first user interface with the display of the second user interface while maintaining the current lock / unlock mode of the display on state of the electronic device.
The first user interface and the second user interface are different from the home screen user interface including a plurality of application start icons corresponding to a plurality of applications.
The second user interface replaces the display of the second user interface, which has a first search input area.
Detecting the input of the first search input area of the second user interface and
In response to detecting the input in the first search input area of the second user interface,
According to the determination that the electronic device is in the lock mode of the display on state, the first set of search results is displayed on the second user interface.
According to the determination that the electronic device is in the unlock mode of the display on state, displaying a second set of search results on the second user interface, which is different from the first set of search results.
How to include. In response to detecting the swipe gesture on the touch-sensitive surface,
According to the determination that the electronic device is in the lock mode with the display on, and the swipe gesture is in a second direction different from the first direction, the display of the first user interface is displayed in the second direction. Replaces the display of a third user interface that displays content,
According to the determination that the electronic device is in the unlock mode with the display on, and the swipe gesture is in the second direction, the display of the first user interface is displayed with the second content and the electronic. Replacing the display with the display of the third user interface, which is not displayed when the device is in the lock mode with the display on.
The method according to claim 1, wherein the method comprises. The second content includes one or more mini-application objects.
The second additional content includes restricted mini-application object information.
The method according to claim 2. The second content includes one or more notifications.
The second additional content includes restricted notification information.
The method according to claim 2 or 3. The method according to any one of claims 2 to 4, wherein the third user interface includes a second search input area. To detect the input in the second search input area and
In response to detecting the input in the second search input area,
A third set of search results is displayed according to the determination that the electronic device is in the display-on state lock mode.
Displaying a fourth set of search results that is different from the third set of search results, according to the determination that the electronic device is in the unlock mode of the display on state.
5. The method of claim 5. The determination as to whether the electronic device is in the unlock mode or the lock mode in the display-on state is performed in response to the detection of the first input, according to claims 1 to 6. The method according to any one item. The determination of whether the electronic device is in the lock mode in the display-on state or in the unlock mode in the display-on state responds to detecting the swipe gesture on the touch-sensitive surface. The method according to any one of claims 1 to 6, which is carried out. The electronic device includes one or more sensors, and detecting the first input can be

Using the one or more sensors to detect changes in the context of the electronic device,
And that in response to detecting the change in the context of the electronic device, the change in the context of the electronic device to determine whether it satisfies the display activation criteria,
The method according to any one of claims 1 to 8, wherein the method comprises. The method of any one of claims 1-9, wherein detecting the first input comprises detecting activation of the display activation affordance. Detecting the first input is a contact with the display activation affordance, which activates the display activation affordance and does not activate the display activation affordance, and detects contact with the display activation affordance. The method according to any one of claims 1 to 10, wherein the method comprises the above. Detecting the first input is a contact with the display activation affordance, which activates the display activation affordance to activate the display of the electronic device to bring the display of the electronic device. activate the display the first user interface over the face, and detecting the contact between the display activation affordance a method according to any one of claims 1 to 11. Detecting the second input while the first user interface is displayed,
Replacing the display of the first user interface with the display of a user interface containing a plurality of application icons corresponding to different applications of the plurality of applications in response to the detection of the second input.
The method according to any one of claims 1 to 12, comprising the method. In response to detecting the second input,
The electronic device may determine whether in the unlocked mode of the display on-state or in the lock mode of the display ON state,
Displaying the authenticated user interface according to the determination that the electronic device is in the lock mode with the display on.
13. The method of claim 13. In response to detecting each input
Substituted wherein each of the input, in accordance with the determination of the fill device unlock criteria, the display of the first user interface, a display of home screen user inter phosphate scan comprising a plurality of application start icons corresponding to a plurality of application To do,
The method according to any one of claims 1 to 14, comprising the method. Displaying the first set of search results in the second user interface corresponds to the first application in response to the input in the first search input area in the second user interface. At least display the first application icon and
Displaying the second set of search results in the second user interface means representing the first application icon and at least the first content in the second application in the second user interface. To display the representation of the first content in the second application that is not in the first set of search results.
The method according to any one of claims 1 to 15, comprising the method. A computer-readable storage medium that stores one or more programs.
The one or more programs include instructions that, when executed on an electronic device having a display and a touch sensing surface, cause the electronic device to perform the method according to any one of claims 1-16. , Computer readable storage medium. With the display
Touch sensing surface and
With one or more processors
With memory
With one or more programs
It is an electronic device equipped with
The one or more programs are stored in the memory and configured to be executed by the one or more processors.
The electronic device, wherein the one or more programs include instructions for performing the method according to any one of claims 1-16.

Images