JP7765286B2 - Systems, devices, and methods for robotic learning and execution of skills - Google Patents

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to U.S. Provisional Patent Application No. 62/723,694, filed August 28, 2018, and entitled "SYSTEMS, APPARATUS, AND METHODS FOR ROBOTIC LEARNING AND EXECUTION OF SKILLS."

[0002] This application claims priority to U.S. Provisional Patent Application No. 62/723,694, entitled "METHOD AND SYSTEM FOR ROBOTIC LEARNING OF EXECUTION PROCESSES RELATED TO PERCEPTUALLY CONSTRAINED MANIPULATION SKILLS," filed February 25, 2017, and to U.S. Provisional Patent Application No. 62/463,628, entitled "METHOD AND SYSTEM FOR ROBOTIC EXECUTION OF A PERCEPTUALLY CONSTRAINED MANIPULATION SKILL LEARNED VIA HUMAN INTERACTION," filed February 23, 2018, which claims priority to U.S. Provisional Patent Application No. 62/463,630, entitled "METHOD AND SYSTEM FOR ROBOTIC EXECUTION OF A PERCEPTUALLY CONSTRAINED MANIPULATION SKILL LEARNED VIA HUMAN INTERACTION," filed February 25, 2017. This application also claims priority to U.S. patent application Ser. No. 16/456,919, entitled "SYSTEMS, APPARATUS, AND METHODS FOR ROBOTIC LEARNING AND EXECUTION OF SKILLS," filed June 28, 2019, which is a continuation-in-part of International PCT Application No. PCT/US2018/019520, entitled "SYSTEMS, APPARATUS, AND METHODS FOR ROBOTIC LEARNING AND EXECUTION OF SKILLS."

[0003] This application is also a continuation-in-part of International PCT Application No. PCT/US2018/019520.

[0004] The disclosures of each of the above-referenced applications are incorporated herein by reference in their entirety.

government support
[0005] This invention was made with United States government support under Grant Nos. 1621651 and 1738375 awarded by the National Science Foundation under Phase I of the Small Business Innovation Program. The United States government has certain rights in this invention.

Technical Field
[0006] This disclosure relates generally to systems, devices, and methods for robotic learning and performance of skills. More particularly, this disclosure relates to robotic devices capable of learning and performing skills in unstructured environments.

background
[0007] Robots can be used to perform and automate a variety of tasks. Robots can perform tasks by moving through an environment, such as an office building or a hospital. Robots may be equipped with wheels, tracks, or other moving components that enable the robot to move autonomously around the environment. However, robots without arms or other manipulators cannot manipulate objects in the environment. These robots are therefore limited in their ability to perform tasks; for example, such robots may not be able to pick up and deliver objects without a human being present at the pick-up or delivery point to manipulate the object.

[0008] Robots that include arms or other manipulators may be able to pick up objects and deliver them to a location without human assistance. For example, a robot with an arm equipped with an end effector, such as a gripper, can use the gripper to pick up one or more objects from several different locations and deliver the objects to a new location, all without human assistance. These robots can be used to automate certain tasks, thereby allowing human operators to focus on other tasks. However, most commercial robots do not include manipulators due to the challenges and complexity of programming the manipulator's movements.

[0009] Additionally, most commercial robots are designed to operate in structured environments (e.g., factories, warehouses, etc.). Unstructured environments (e.g., human-involved environments such as hospitals and homes) can pose additional challenges for programming robots. In unstructured environments, a robot cannot rely on complete knowledge of its surroundings but must be able to recognize changes in its surroundings and adapt based on those changes. Thus, in unstructured environments, a robot must continuously acquire information about the environment to be able to make autonomous decisions and perform tasks. Often, a robot's motion (e.g., the movement of an arm or end effector within the environment) is also constrained by objects and other obstacles in the environment, further increasing the challenges of robot perception and manipulation. Given the uncertain and dynamic nature of unstructured environments, robots generally cannot be pre-programmed to perform tasks.

[0010] Therefore, there is a need for a robotic system that can perceive and adapt to dynamic, unstructured environments and perform tasks within those environments without relying on pre-programmed operational skills.

overview
Systems, devices, and methods for robotic learning and execution of skills are described. In some embodiments, the device includes a memory, a processor, a manipulative element, and a set of sensors. The processor is operably coupled to the memory, the manipulative element, and the set of sensors and may be configured to: obtain a representation of an environment using a subset of sensors from the set of sensors; identify a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; present information indicative of a position of each marker from the plurality of markers within the representation of the environment; receive a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; obtain, for each position from the plurality of positions associated with movement of the manipulative element within the environment, sensory information associated with the manipulative element, where the movement of the manipulative element is associated with a physical interaction between the manipulative element and the set of physical objects; and generate, based on the sensory information, a model configured to define a behavior of the manipulative element to perform the physical interaction between the manipulative element and the set of physical objects.

[0012] In some embodiments, the operating element may include multiple joints and an end effector. In some embodiments, the operating element may include a base or a carrying element.

[0013] The set of physical objects may be stationary or moving. In some embodiments, the set of physical objects may include fixtures, storage containers, structures (e.g., doors, handles, furniture, walls), etc. In some embodiments, the set of physical objects may include people (e.g., patients, doctors, nurses, etc.).

[0014] In some embodiments, the plurality of markers are fiducial markers and the representation of the environment is a visual representation of the environment.

[0015] In some embodiments, two or more markers from the plurality of markers may be associated with one physical object from the set of physical objects. Alternatively or additionally, in some embodiments, one marker from the plurality of markers may be associated with two or more physical objects from the set of physical objects.

[0016] In some embodiments, a method includes obtaining a representation of an environment using a set of sensors; identifying a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; presenting information indicative of a position of each marker from the plurality of markers within the representation of the environment; receiving, after the presenting, a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; obtaining, for each position from a plurality of positions associated with movement of a control element within the environment, sensory information associated with the control element, where movement of the control element is associated with a physical interaction between the control element and the set of physical objects; and generating, based on the sensory information, a model configured to specify a behavior of the control element to perform the physical interaction between the control element and the set of physical objects.

[0017] In some embodiments, the method further includes receiving a selection of a first subset of features from the set of features, wherein the model is generated based on sensor data associated with the first subset of features and not based on sensor data associated with a second subset of features from the set of features that are not included in the first set of features.

[0018] In some embodiments, a method includes obtaining a representation of an environment using a set of sensors; identifying a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; presenting information indicative of a position of each marker from the plurality of markers within the representation of the environment; identifying a model associated with performing a physical interaction between a manipulative element and the set of physical objects in response to receiving a selection of a set of markers from the plurality of markers associated with the set of physical objects from the plurality of physical objects, where the manipulative element includes a plurality of joints and an end effector; and generating a trajectory for the manipulative element using the model, the trajectory defining movement of the plurality of joints and the end effector associated with performing the physical interaction.

[0019] In some embodiments, the method further includes displaying to a user a trajectory of the manipulative element within the representation of the environment; receiving input from the user after the display; and, in response to the input indicating approval of the trajectory of the manipulative element, performing a movement of the manipulative element (e.g., a plurality of joints, an end effector, a transport element) to perform a physical interaction.

[0020] In some embodiments, a model is associated with (i) a set of stored markers and (ii) sensory information indicating at least one of a position, orientation, or configuration of a manipulative element (e.g., a plurality of joints, an end effector, a transport element) at points along a stored trajectory of the manipulative element associated with the set of stored markers. A method for generating a trajectory of the manipulative element may include, for example, calculating a transformation function between the set of markers and the set of stored markers; transforming at least one of a position or orientation of the manipulative element using the transformation function for each point; determining, for each point, a planned configuration of one or more components of the manipulative element (e.g., a plurality of joints, an end effector, a transport element) based on the configuration of the components along the stored trajectory; and determining, for each point, a portion of the trajectory between the point and successive points based on the planned configuration of the components with respect to the point.

[0021] In some embodiments, the robotic device is configured to learn and perform a skill associated with a transport element. The transport element may be, for example, a set of wheels, a set of tracks, a set of crawlers, or any other suitable device that enables movement of the robotic device within an environment, such as from a first room to a second room or between multiple floors of a building. When the robotic device is moved from a first location to a second location, for example, through a doorway or by an object such as a human, the robotic device may be configured to learn the skill associated with the transport element by acquiring sensory information associated with the transport element, the environment around the robotic device, and/or another component of the robotic device. The sensory information may be recorded at specific points (e.g., at key frames) during operation of the robotic device. Based on the sensory information, the robotic device may be configured to generate a model configured to prescribe the behavior of the transport element (or other component of the robotic device (e.g., a manipulation element)) to execute movement of the robotic device from the first location to the second location.

[0022] In some embodiments, a robotic device is configured to learn a specialized skill using a generic version of the skill. The robotic device can begin performing the generic skill in a particular environment and pause when the robotic device reaches a portion of the performance that requires specialization of the skill in the particular environment. The robotic device can then prompt the user to perform that portion of the skill. The robotic device can continue performing the skill and/or prompt the user to perform specific portions of the skill until the skill is complete. The robotic device then adapts the generic skill based on the performance of the specific portions of the skill to generate a specialized model for performing the skill.

[0023] In some embodiments, the robotic device is configured to learn environmental constraints. The robotic device can record information about its surroundings or objects in that environment to acquire general knowledge about the environment. The robotic device can apply this general knowledge to a set of models related to different skills to be performed in the environment.

[0024] In some embodiments, a robotic device can use behavior arbitration to continuously and autonomously act within a dynamic environment, such as a human social environment. A robotic device may have various resources or components (e.g., a manipulating element, a carrying element, a head, a camera, etc.) and can apply an arbitration algorithm to determine when and how to use these resources. The arbitration algorithm can define a set of rules that allow different actions or behaviors to be prioritized based on information captured by the robotic device. A robotic device may be configured to continuously decide among the execution of different actions and behaviors (and thus use different resources or components) as new information is captured by the robotic device. As part of behavior arbitration, a robotic device may be configured to handle the interruption of an action and switch to a new action. By continuously monitoring itself and its surrounding environment and performing continuous arbitration, a robotic device can continually engage in socially appropriate behaviors over time.

[0025] In some embodiments, the robotic device is configured to receive social context information related to an environment and combine that information with a navigation map of the environment (e.g., overlay the social context information on the navigation map). The robotic device may be provided with the social context information by a user (e.g., a human operator), or the robotic device may capture the social context information through interactions within the environment. As the robotic device engages in interactions with humans within the environment over time, the robotic device may refine the social context information. In some embodiments, the robotic device may be configured to associate certain types of behaviors or actions with particular locations within the environment based on the social context information associated with those locations.

[0026] In some embodiments, the robotic device is configured to interact with a human operator, either locally or via a network connection to a remote device operated by the human operator. The human operator, referred to herein as a "robot supervisor," can use various software-based tools to inform the robotic device about its surroundings and/or instruct the robotic device to perform specific actions. These actions may include, for example, navigation behaviors, manipulation behaviors, head behaviors, sounds, lights, etc. The robot supervisor can control the robotic device to collect information while learning or performing an action and/or tag certain behaviors as good or bad so that the robotic device can use that information to improve its ability to arbitrate among different actions or to perform a particular action in the future.

[0027] Other systems, processes, and features will become apparent to one with skill in the art upon examination of the following drawings and detailed description. All such additional systems, processes, and features are intended to be included within the scope of this specification, be within the scope of the invention, and be protected by the accompanying claims.

BRIEF DESCRIPTION OF THE DRAWINGS
[0028] Those skilled in the art will understand that the drawings are primarily for illustrative purposes and are not intended to limit the scope of the inventive subject matter described herein. The drawings are not necessarily to scale, and in some cases, various aspects of the inventive subject matter disclosed herein may be shown exaggerated or enlarged in the drawings to facilitate an understanding of different features. In the drawings, like reference numerals generally refer to like features (e.g., functionally similar and/or structurally similar elements).

[0029] FIG. 1 is a block diagram illustrating a configuration of a system including a robotic device, according to some embodiments. [0030] FIG. 1 is a block diagram illustrating a configuration of a robotic device, according to some embodiments. [0031] FIG. 1 is a block diagram illustrating the configuration of a control unit associated with a robotic device, according to some embodiments. [0032] FIG. 1 is a schematic diagram of a manipulative element of a robotic device, according to some embodiments. [0033] FIG. 1 is a schematic diagram of a robotic device, according to some embodiments. [0034] FIG. 1 is a schematic diagram of objects in an environment seen by a robotic device, according to some embodiments. [0034] FIG. 1 is a schematic diagram of objects in an environment seen by a robotic device, according to some embodiments. [0035] FIG. 1 is a schematic diagram of objects in an environment seen by a robotic device, according to some embodiments. [0035] FIG. 1 is a schematic diagram of objects in an environment seen by a robotic device, according to some embodiments. [0036] FIG. 1 is a flow diagram illustrating a method for sensing or scanning an environment performed by a robotic device, according to some embodiments. [0037] FIG. 1 is a flow diagram illustrating a method for learning and performing skills performed by a robotic device, according to some embodiments. [0038] FIG. 1 is a flow diagram illustrating a method for learning a skill to be performed by a robotic device, according to some embodiments. [0039] FIG. 10 is a flow diagram illustrating a method for executing a skill performed by a robotic device, according to some embodiments. [0040] FIG. 1 is a block diagram illustrating a system architecture for robot learning and execution, including user actions, according to some embodiments. [0041] FIG. 1 is a flow diagram illustrating the operation of a robotic device within an environment, according to some embodiments. [0042] FIG. 1 is a flow diagram illustrating a method for requesting and receiving input from a user, such as a robot supervisor, according to some embodiments. [0043] FIG. 1 is a flow diagram illustrating a method for learning skills and environmental constraints performed by a robotic device, according to some embodiments. [0044] FIG. 1 is a flow diagram illustrating a method for learning a skill from a generic skill model performed by a robotic device, according to some embodiments. [0045] FIG. 10 is a flow diagram illustrating a method for learning environmental constraints performed by a robotic device, according to some embodiments. [0046] FIG. 1 is a block diagram illustrating an example of components of a robotic device that perform behavior arbitration, according to some embodiments. [0047] FIG. 1 is a schematic diagram of a layer of a map of an environment generated by a robotic device, according to some embodiments. [0048] FIG. 1 is a block diagram illustrating the configuration of a control unit associated with a robotic device, according to some embodiments. [0049] Figure 1 illustrates the flow of information provided by and received from a robotic device to a map maintained by the robotic device, according to some embodiments. [0050] FIG. 10 illustrates a flow diagram illustrating different learning modes of a robotic device, according to some embodiments. [0051] FIG. 1 illustrates a flow diagram illustrating an example learning behavior of a robotic device, according to some embodiments. [0051] FIG. 1 illustrates a flow diagram illustrating an example learning behavior of a robotic device, according to some embodiments. [0051] FIG. 1 illustrates a flow diagram illustrating an example learning behavior of a robotic device, according to some embodiments.

Detailed Description
[0052] Described herein are systems, devices, and methods for robotic learning and performance of skills. In some embodiments, the systems, devices, and methods described herein relate to robotic devices that can learn skills through human demonstration and interaction and perform the learned skills in unstructured environments.

overview
In some embodiments, the systems, devices, and methods described herein relate to robots that can learn skills (e.g., manipulation skills) through a Learning from Demonstration (“LfD”) process, in which a human demonstrates an action to a system through kinesthetic teaching (e.g., a human physically and/or remotely directing the robot to perform the action) and/or by performing the action themselves. Such systems, devices, and methods are designed to avoid the need to pre-program the robot with manipulation skills, and to enable the robot to be adaptive and learn skills through observation. For example, a robot can acquire and perform a manipulation skill using machine learning techniques. After learning a skill, the robot can perform the skill in multiple different environments. A robot can learn and/or perform a skill based on visual data (e.g., sensed visual information). Alternatively or additionally, a robot can learn and/or perform a skill using tactile data (e.g., torque, force, and other non-visual information). Robot training can occur in a factory before the robot is deployed or in the field (e.g., in a hospital) after the robot is deployed. In some embodiments, a robot can be taught skills and/or adapted to operate in an environment by a user untrained in robotics and/or programming. For example, a robot can have learning algorithms that utilize natural human behavior and include tools that can guide a user through a demonstration process.

[0054] In some embodiments, robots can be designed to interact with humans and collaborate with them to perform tasks. In some embodiments, robots can utilize common social behaviors to behave in a socially predictable and acceptable manner around humans. Mobile robots can also be designed to navigate within an environment while interacting with humans within that environment. For example, a robot can be programmed to vocalize specific phrases to navigate around humans, move aside to allow humans to pass, and use gaze to intentionally communicate while navigating. In some embodiments, a robot may be equipped with sensors that enable the robot to sense and track humans within its surrounding environment and use that information to trigger gaze and other social behaviors.

[0055] In some embodiments, the robot can be designed to suggest options for achieving a goal or performing an action during the LfD process. For example, the robot can suggest several different options for achieving a goal (e.g., picking up an object) and indicate which of these options is most likely to be effective and/or efficient for achieving the goal. In some embodiments, the robot can adapt a skill based on user input (e.g., the user indicating relevant features to include in the skill model).

[0056] In some embodiments, a robotic device may be capable of learning and/or performing skills in an unstructured environment (e.g., a dynamic and/or human environment) where the robotic device does not have complete information about the environment a priori. An unstructured environment may include, for example, indoor and outdoor situations and may include one or more humans or other objects that are movable within the environment. Because most natural or real-world environments are unstructured, robotic devices that can adapt and operate in unstructured environments, such as the robotic devices and/or systems described herein, may provide a significant improvement over existing robotic devices that cannot adapt to unstructured environments. Unstructured environments may include indoor situations (e.g., buildings, offices, homes, rooms, etc.) and/or other types of enclosed spaces (e.g., airplanes, trains, and/or other types of movable compartments), as well as outdoor situations (e.g., parks, beaches, outdoor yards, fields). In some embodiments, the robotic devices described herein may operate in an unstructured hospital environment.

[0057] FIG. 1 is a high-level block diagram illustrating a system 100 according to some embodiments. The system 100 can be configured to learn and perform skills, such as operational skills, in an unstructured environment. The system 100 may be implemented as a single device or across multiple devices connected to a network 105. For example, as shown in FIG. 1, the system 100 may include one or more computing devices, such as one or more robotic devices 102 and 110, a server 120, and one or more additional computing devices 150. While four devices are shown, it is understood that the system 100 may include any number of computing devices, including computing devices not specifically shown in FIG. 1.

[0058] Network 105 may be implemented as a wired and/or wireless network and may be any type of network (e.g., a local area network (LAN), a wide area network (WAN), a virtual network, a telecommunications network) used to operatively couple computing devices, including robotic devices 102 and 110, server 120, and one or more computing devices 150. As described in further detail herein, in some embodiments, for example, the computing devices are computers connected to each other by an Internet Service Provider (ISP) and the Internet (e.g., network 105). In some embodiments, a connection may be defined by network 105 between any two computing devices. As shown in FIG. 1, for example, a connection may be defined between robotic device 102 and any one of robotic device 110, server 120, or one or more computing devices 150. In some embodiments, these computing devices may communicate with each other (e.g., send and/or receive data) and with network 105 through intermediate and/or alternative networks (not shown in FIG. 1). Such intermediate and/or alternative networks may be networks of the same type and/or different types than network 105. Each computing device may be any type of device configured to transmit data over network 105 for transmission and/or receive data from one or more of the other computing devices.

[0059] In some embodiments, system 100 includes a single robotic device (e.g., robotic device 102). Robotic device 102 can be configured to sense information about its environment, learn skills through human demonstration and interaction, interact with its environment, and/or learn environmental constraints through human demonstration and input, and/or perform those skills in its environment. In some embodiments, robotic device 102 can self-explore and/or request user input to learn additional information about the skill and/or the environment. A more detailed diagram of an example robotic device is shown in FIG. 2.

[0060] In another embodiment, system 100 includes multiple robotic devices (e.g., robotic devices 102 and 110). Robotic device 102 can transmit data to and/or receive data from robotic device 110 via network 105. For example, robotic device 102 can transmit information sensed by robotic device 102 about its environment (e.g., the location of an object) to robotic device 110 and can receive information about the environment from robotic device 110. Robotic devices 102 and 110 can also transmit information to and/or receive information from each other to learn and/or perform a skill. For example, robotic device 102 can learn a skill in an environment and transmit a model representing the learned skill to robotic device 110, which can then use the model to perform the skill in the same or a different environment. Robotic device 102 may be located in the same location as robotic device 110 or in a different location. For example, robotic devices 102 and 110 may be located in the same room of a building (e.g., a hospital building) so that they can learn and/or perform a skill together (e.g., moving a heavy or large object). Alternatively, robotic device 102 may be located on the first floor of a building (e.g., a hospital building) and robotic device 110 may be located on the second floor of the building, and the two robotic devices may communicate with each other to convey information about the different floors (e.g., where objects are located on these floors, where resources may be located, etc.).

[0061] In some embodiments, system 100 includes one or more robotic devices (e.g., robotic devices 102 and/or 110) and server 120. Server 120 may be a dedicated server that manages robotic devices 102 and/or 110. Server 120 may be located in the same or a different location as robotic devices 102 and/or 110. For example, server 120 may be located in the same building (e.g., a hospital building) as one or more robotic devices and may be managed by a local administrator (e.g., a hospital administrator). Alternatively, server 120 may be located in a remote location (e.g., a location associated with a manufacturer or supplier of the robotic devices).

[0062] In some embodiments, system 100 includes one or more robotic devices (e.g., robotic devices 102 and/or 110) and an additional computing device 150. Computing device 150 may be any suitable processing device configured to initiate and/or perform a particular function. For example, in a hospital setting, computing device 150 may be a diagnostic and/or therapeutic device that can connect to network 105 and communicate with other computing devices, including robotic devices 102 and/or 110.

[0063] In some embodiments, one or more robotic devices (e.g., robotic devices 102 and/or 110) may be configured to communicate with server 120 and/or computing device 150 via network 105. Server 120 may include one or more components located remotely from the robotic devices and/or near the robotic devices on a premises. Computing device 150 may include one or more components located remotely from the robotic devices, near the robotic devices on a premises, and/or integrated into one of the robotic devices. Server 120 and/or computing device 150 may include a user interface that allows a user (e.g., a local user or a robot supervisor) to control the operation of the robotic device. For example, a user can interrupt and/or modify the execution of one or more actions performed by the robotic device. These actions may include, for example, navigation behavior, manipulation behavior, head behavior, sound/lights, and/or other components of the robotic device. In some embodiments, a robot supervisor can remotely monitor the robotic device and control its operation for safety purposes. For example, a robot supervisor can command the robotic device to stop or modify the execution of an action to avoid endangering a human or damaging the robotic device or another object in the environment. In some embodiments, a nearby user can instruct or demonstrate an action to the robotic device. In some embodiments, the robotic device may be configured to request user intervention at certain points during the execution of an action. For example, user intervention may be requested when the robotic device is unable to ascertain certain information about itself and/or the environment around it, when the robotic device is unable to determine a trajectory for completing an action or navigating to a particular location, when the robotic device has been pre-programmed to request user input (e.g., during learning using an interactive learning template, as described further below), etc. In some embodiments, the robotic device may request user feedback at certain points during learning and/or execution. For example, the robotic device may prompt the user to specify what information to collect (e.g., information related to a manipulation or transport element, information related to the surrounding environment) and/or when information should be collected (e.g., the timing of key frames during a demonstration). Alternatively or additionally, the robotic device may request that the user tag past or current behaviors of the robotic device as good or bad examples of an action in a particular context. The robotic device may be configured to use this information to improve future performance of that action in that particular context.

Systems and Devices
2 schematically illustrates a robotic device 200 according to some embodiments. The robotic device 200 includes a control unit 202, a user interface 240, at least one operating element 250, and at least one sensor 270. Additionally, in some embodiments, the robotic device 200 optionally includes at least one transport element 260. The control unit 202 includes a memory 220, storage 230, a processor 204, a graphics processor 205, a system bus 206, and at least one input/output interface (“I/O interface”) 208. The memory 220 may be, for example, a random access memory (RAM), a memory buffer, a hard drive, a database, an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), a read-only memory (ROM), and/or the like. In some embodiments, memory 220 stores instructions that cause processor 204 to perform modules, processes, and/or functions related to sensing or scanning an environment, learning a skill, and/or performing a skill. Storage 230 may be, for example, a hard drive, a database, cloud storage, a network-attached storage device, or other data storage device. In some embodiments, storage 230 may store sensor data including, for example, state information about one or more components of robotic device 200 (e.g., manipulative element 250), learned models, marker position information, etc.

[0065] The processor 204 of the control unit 202 may be any suitable processing device configured to initiate and/or perform functions related to inspecting an environment, learning a skill, and/or performing a skill. For example, the processor 204 may be configured to perform a skill by generating a model for the skill based on sensor information or using the model to generate a trajectory for performing the skill, as described further herein. More particularly, the processor 204 may be configured to execute modules, functions, and/or processes. In some embodiments, the processor 204 may be a general-purpose processor, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), and/or the like.

[0066] Graphics processor 205 may be any suitable processing device configured to drive and/or perform one or more display functions (e.g., functions associated with display device 242). In some embodiments, graphics processor 205 may be a low-power graphics processing unit, such as, for example, a dedicated graphics card or an integrated graphics processing unit.

[0067] The system bus 206 may be any suitable component that enables the processor 204, memory 220, storage 230, and/or other components of the control unit 202 to communicate with each other. The one or more I/O interfaces 208 connected to the system bus 206 may be any suitable component that enables communication between the internal components of the control unit 202 (e.g., the processor 204, memory 220, storage 230) and external input/output devices, such as a user interface 240, one or more operating elements 250, one or more transport elements 260, and one or more sensors 270.

[0068] User interface 240 may include one or more components configured to receive input and provide output to other devices and/or to a user operating a device (e.g., a user operating robotic device 200). For example, user interface 240 may include a display device 242 (e.g., a display, a touchscreen, etc.), an audio device 244 (e.g., a microphone, a speaker), and, optionally, one or more additional input/output devices ("I/O devices") 246 configured to receive input and/or generate output for a user.

[0069] The one or more manipulating elements 250 may be any suitable component capable of manipulating and/or interacting with stationary and/or moving objects, including, for example, a human. In some embodiments, the one or more manipulating elements 250 may include multiple segments coupled to one another by joints capable of providing translation along one or more axes and/or rotation about one or more axes. The one or more manipulating elements 250 may optionally include an end effector capable of engaging and/or otherwise interacting with objects in the environment. For example, the manipulating element may include a gripping mechanism capable of releasably engaging (e.g., grasping) an object in the environment for picking up and/or transporting the object. Other examples of end effectors include, for example, one or more vacuum engagement mechanisms, one or more magnetic engagement mechanisms, one or more suction mechanisms, and/or combinations thereof. In some embodiments, one or more manipulating elements 250 may be retractable within the housing of robotic device 200 when not in use to reduce one or more dimensions of the robotic device. In some embodiments, one or more manipulating elements 250 may include a head or other humanoid component configured to interact with the environment and/or one or more objects within the environment, including humans. In some embodiments, one or more manipulating elements 250 may include a carrier element or base (e.g., one or more carrier elements 260). A detailed view of an example manipulating element is shown in FIG. 4.

[0070] The one or more transport elements 260 may be any suitable component configured for mobility, such as, for example, wheels or tracks. The one or more transport elements 260 may be provided at the base of the robotic device 200 to enable the robotic device 200 to move about an environment. For example, the robotic device 200 may include multiple wheels that enable the robotic device 200 to navigate through a building, such as a hospital. The one or more transport elements 260 may be designed and/or dimensioned to facilitate movement through narrow and/or restricted spaces (e.g., small passageways or hallways, small rooms such as storage rooms, etc.). In some embodiments, the one or more transport elements 260 may be rotatable about an axis and/or movable relative to one another (e.g., along tracks). In some embodiments, the one or more transport elements 260 may be retractable within the base of the robotic device 200 when not in use to reduce one or more dimensions of the robotic device. In some embodiments, one or more of the conveying elements 260 may be part of or form part of an operating element (e.g., one or more of the operating elements 250).

[0071] The one or more sensors 270 may be any suitable component that enables the robotic device 200 to capture information about the environment and/or objects in the environment surrounding the robotic device 200. The one or more sensors 270 may include, for example, an imaging device (e.g., a camera such as an RGB-D (red-green-blue-depth) camera or a webcam), an audio device (e.g., a microphone), an optical sensor (e.g., a light detection and ranging (i.e., lidar) sensor, a color detection sensor), a proprioceptive sensor, a position sensor, a tactile sensor, a force or torque sensor, a temperature sensor, a pressure sensor, a motion sensor, a sound detector, etc. For example, the one or more sensors 270 may include at least one imaging device, such as a camera, for capturing visual information about objects and the environment surrounding the robotic device 200. In some embodiments, the one or more sensors 270 may include a tactile sensor (e.g., a sensor that can communicate force, vibration, touch, and other non-visual information to the robotic device 200).

[0072] In some embodiments, the robotic device 200 may have humanoid features (e.g., a head, torso, arms, legs, and/or a base). For example, the robotic device 200 may include a face with eyes, a nose, a mouth, and other humanoid features. These humanoid features may form part of and/or be part of one or more operational elements. Although not schematically shown, the robotic device 200 may also include actuators, motors, couplers, connectors, power sources (e.g., an internal battery), and/or other components that connect, actuate, and/or drive different portions of the robotic device 200.

[0073] FIG. 3 is a block diagram that schematically illustrates a control unit 302 according to some embodiments. The control unit 302 may include similar components as the control unit 202 and may be structurally and/or functionally similar to the control unit 202. For example, the control unit 302 includes a processor 304, a graphics processor 305, memory 320, one or more I/O interfaces 308, a system bus 306, and storage 330, which may be structurally and/or functionally similar to the processor 204, memory 220, one or more I/O interfaces 208, a system bus 206, and storage 230, respectively.

[0074] Memory 320 stores instructions that can cause processor 304 to execute modules, processes, and/or functions shown as active sensing 322, marker identification 324, learning and model generation 326, trajectory generation and execution 328, and success monitoring 329. Active sensing 322, marker identification 324, learning and model generation 326, trajectory generation and execution 328, and success monitoring 329 may be implemented as one or more programs and/or applications coupled to hardware components (e.g., sensors, operating elements, I/O devices, processors, etc.). Active sensing 322, marker identification 324, learning and model generation 326, trajectory generation and execution 328, and success monitoring 329 may be implemented by a single robotic device or by multiple robotic devices. For example, a robotic device may be configured to implement active sensing 322, marker identification 324, and trajectory generation and execution 328. As another example, the robotic device may be configured to implement active sensing 322, marker identification 324, optionally learning and model generation 326, and trajectory generation and execution 328. As another example, the robotic device may be configured to implement active sensing 322, marker identification 324, trajectory generation and execution 328, and optionally success monitoring 329. Although not shown, memory 320 may also store programs and/or applications related to an operating system and general robotic operation (e.g., power management, memory allocation, etc.).

[0075] Storage 330 stores information related to the learning and/or execution of skills. Storage 330 stores, for example, state information 331, one or more models 334, object information 340, and machine learning library 342. State information 331 may include information about the state of the robotic device (e.g., robotic device 200) and/or the environment in which the robotic device is operating (e.g., a building, such as a hospital). In some embodiments, state information 331 may indicate the location of the robotic device within the environment, such as a room, floor, or enclosed space. For example, state information 331 may include a map 332 of the environment and may indicate the location of the robotic device within map 332. State information 331 may also include one or more locations of one or more objects (or markers representing and/or associated with the objects) within the environment (e.g., within map 332). Thus, state information 331 may identify the location of the robotic device relative to one or more objects. Objects may include any type of physical object located in the environment, including objects that define a space or opening (e.g., a surface or wall that defines a doorway). Objects may be stationary or movable. Examples of objects in an environment such as a hospital include equipment, fixtures, machinery, tools, furniture, and/or people (e.g., nurses, doctors, patients, etc.).

[0076] In some embodiments, the state information 331 may include a representation or map of the environment, such as that shown in FIG. 19. The map of the environment may include, for example, a navigational layer, a static semantic layer, a social layer, and a dynamic layer. Information learned by the robotic device, for example, from demonstrations, user input, sensed sensor information, etc., may be fed into different layers of the map and organized for later reference by the robotic device (and/or other robotic devices). For example, the robotic device may determine how to arbitrate between different behaviors (e.g., waiting for the door of a closed doorway to be opened before passing through the doorway, or requesting help to open the door before passing through the closed doorway) depending on information learned about different objects (e.g., doors) in the environment, as described further herein.

[0077] Object information 340 may include information related to one or more physical objects in the environment. For example, object information may include information that identifies or quantifies different characteristics of an object, such as, for example, location, color, shape, and surface features. Object information may also identify codes, symbols, and other markers associated with a physical object (e.g., quick response or "QR" codes, bar codes, tags, etc.). In some embodiments, object information may include information that characterizes an object in the environment (e.g., a doorway or passageway is closed, a door handle is a type of door handle, etc.). The object information may enable control unit 302 to identify one or more physical objects in the environment.

[0078] State information 331 and/or object information 340 may be examples of environmental constraints, as used and described herein. Environmental constraints may include information about an environment that may be presented or viewed in various dimensions. For example, environmental constraints may vary by location, time, social interaction, specific context, etc. A robotic device may learn environmental constraints from user input, skill demonstrations, interaction with the environment, self-exploration, skill execution, etc.

[0079] Machine learning library 342 may include modules, processes, and/or functions related to different algorithms for machine learning and/or model generation for different skills. In some embodiments, the machine learning library may include methods such as hidden Markov models (i.e., "HMMs"). An example of an existing machine learning library for Python is scikit-learn. Storage 330 may also include additional software libraries related to, for example, robot simulation, motion planning and control, kinematics teaching and perception, etc.

[0080] One or more skill models 334 are models generated for performing different actions and represent skills learned by the robotic device. In some embodiments, each model 334 is associated with a set of markers attached to different physical objects in an environment. Marker information 335 may indicate which markers are associated with a particular model 334. Each model 334 may also be associated with sensory information 336 collected, for example, by one or more sensors of the robotic device, during kinesthetic teaching and/or other demonstration of a skill. Sensory information 336 may optionally include manipulative element information 337 associated with manipulative elements of the robotic device as the robotic device performs actions during the demonstration. Manipulative element information 337 may include, for example, the position and configuration of joints, the position and configuration of end effectors, the position and configuration of bases or transport elements, and/or the forces and torques acting on joints, end effectors, transport elements, etc. Manipulative element information 337 may be recorded at specific points (e.g., key frames) during the performance and/or execution of a skill, or throughout the performance and/or execution of a skill. Sensory information 336 may also include information associated with the environment in which the skill is being performed and/or executed (e.g., the location of markers within the environment). In some embodiments, each model 334 may be associated with success criteria 339. The success criteria 339 can be used to monitor the performance of a skill. In some embodiments, the success criteria 339 may include information associated with visual and tactile data sensed using one or more sensors (e.g., cameras, force/torque sensors, etc.). The success criteria 339 may be tied, for example, to visually detecting the movement of an object, detecting a force acting on a component of the robotic device (e.g., the weight of the object), detecting an engagement between a component of the robotic device and an object (e.g., a change in pressure or force acting on a surface), etc. An example of the use of haptic data in robotic learning of manipulation skills is described in a paper entitled "Learning Haptic Affordances from Demonstration and Human-Guided Exploration" by Chu et al., published at the 2016 IEEE Haptics Symposium (HAPTICS) (Philadelphia, PA, 2016, pp. 119-125), which is incorporated herein by reference and available at http://ieeexplore.ieee.org/document/7463165/. An example of using visual data to teach robots manipulation skills is described in the paper entitled "Simultaneously Learning Actions and Goals from Demonstration" by Akgun et al., published in Autonomous Robots (Volume 40, Issue 2, February 2016, pp. 211-227), available at https://doi.org/10.1007/s10514-015-9448-x (the "Akgun paper"), which is incorporated herein by reference.

[0081] Optionally, sensory information 336 may include transport element information 338. Transport element information 338 may be associated with the movement (e.g., navigation through a doorway, transporting an object, etc.) of a transport element (e.g., one or more transport elements 260) of the robotic device as the robotic device undergoes a skill demonstration. Transport element information 338 may be recorded at specific points during the performance and/or execution of the skill (e.g., keyframes associated with the skill) or throughout the performance and/or execution of the skill.

[0082] In some embodiments, an initial set of environmental constraints (e.g., state information 331, object information 340, etc.) and/or skills (e.g., one or more models 334) may be provided to the robotic device, for example, by a remote manager or supervisor. The robotic device can adapt and/or augment its knowledge of environmental constraints and/or skills based on its own interactions with the environment or humans in the environment (e.g., patients, nurses, doctors, etc.), demonstrations, etc., and/or by additional user input. Alternatively or additionally, the robot supervisor can update the robotic device's knowledge of environmental constraints and/or skills based on new information collected by the robotic device or one or more other robotic devices (e.g., one or more other robotic devices in a similar or same environment (e.g., a hospital)) and/or provided to the robot supervisor by an external party (e.g., a supplier, manager, manufacturer, etc.). Such updates may be provided periodically and/or continuously as new information about the environment or skills is provided to the robotic device and/or robot supervisor.

[0083] FIG. 4 schematically illustrates a manipulating element 350 according to some embodiments. The manipulating element 350 may form part of a robotic device, such as, for example, robotic device 102 and/or 200. The manipulating element 350 may be implemented as an arm including two or more segments 352 joined together by joints 354. The joints 354 may allow one or more degrees of freedom. For example, the joints 354 may provide translation along one or more axes and/or rotation about one or more axes. In some embodiments, the manipulating element 350 may have seven degrees of freedom provided by the joints 354. While FIG. 4 illustrates four segments 352 and four joints 354, one skilled in the art will understand that the manipulating element may include a different number of segments and/or joints.

[0084] Manipulation element 350 includes end effector 356 that can be used to interact with objects in an environment. For example, end effector 356 can be used to engage with and/or manipulate different objects. Alternatively or additionally, end effector 356 can be used to interact with movable or dynamic objects, including, for example, a human. In some embodiments, end effector 356 can be a gripper that can releasably engage or grasp one or more objects. For example, end effector 356 implemented as a gripper can pick up and move an object from a first location (e.g., a storage unit) to a second location (e.g., an office, a room, etc.).

[0085] Sensors 353, 355, 357, and 358 may be disposed on different components of operational element 350 (e.g., segment 352, joint 354, and/or end effector 356). Sensors 353, 355, 357, and 358 may be configured to measure sensory information, including environmental information and/or operational element information. Examples of sensors include position encoders, torque and/or force sensors, contact and/or tactile sensors, imaging devices such as cameras, temperature sensors, pressure sensors, light sensors, etc. In some embodiments, sensor 353 disposed on segment 352 may be a camera configured to capture visual information about the environment. In some embodiments, sensor 353 disposed on segment 352 may be an accelerometer configured to measure the acceleration of segment 352 and/or enable calculation of the speed and/or position of movement of segment 352. In some embodiments, sensor 355 disposed on joint 354 may be a position encoder configured to measure the position and/or configuration of joint 354. In some embodiments, sensor 355 disposed on joint 354 may be a force or torque sensor configured to measure a force or torque applied to joint 354. In some embodiments, sensor 358 disposed on end effector 356 may be a position encoder and/or a force or torque sensor. In some embodiments, sensor 357 disposed on end effector 356 may be a contact or tactile sensor configured to measure engagement of end effector 356 with an object in the environment. Alternatively or additionally, one or more of sensors 353, 355, 357, and 358 may be configured to record information about one or more objects and/or markers in the environment. For example, sensor 358 disposed on end effector 356 may be configured to track the location of an object in the environment and/or the position of the object relative to end effector 356. In some embodiments, one or more of sensors 353, 355, 357, and 358 can also track whether an object, such as a human, moves within the environment. Sensors 353, 355, 357, and 358 can send the sensory information they record to a computing device located on the robotic device (e.g., an on-board control unit (e.g., control units 202 and/or 302)), or sensors 353, 355, 357, and 358 can send the sensory information to a remote computing device (e.g., a server (e.g., server 120)).

[0086] Manipulation element 350 may optionally include coupling elements 359 that allow manipulating element 350 to be removably coupled to a robotic device (such as any of the robotic devices described herein). In some embodiments, manipulating element 350 may be coupled to a robotic device in a fixed position and/or may be coupleable to multiple locations on the robotic device (e.g., to the right or left side of the robotic device's torso as shown in FIG. 5). Coupling elements 359 may include any type of mechanism that allows manipulating element 350 to be coupled to a robotic device, such as, for example, a mechanical mechanism (e.g., a fastener, a latch, a mount), a magnetic mechanism, a friction fit, etc.

20 is a block diagram that schematically illustrates a control unit 1702 of a robotic system, according to some embodiments. The control unit 1702 may include components similar to those of other control units (e.g., control units 202 and/or 302) described herein. For example, the control unit 1702 includes a processor 1704, a graphics processor 1705, memory 1720, one or more I/O interfaces 1708, a system bus 1706, and storage 1730, which may be structurally and/or functionally similar to the processor, memory, one or more I/O interfaces, a system bus, and storage of the control units 202 and/or 302, respectively. The control unit 1702 may be located on the robotic device and/or on a remote server connected to one or more robotic devices.

[0088] Memory 1720 stored instructions that can cause processor 1704 to execute modules, processes, and/or functions, including active sensing 1722, skill and behavior learning 1724, action execution 1726, and, optionally, resource arbitration 1728, and data tracking and analysis 1758. Active sensing 1722, skill and behavior learning 1724, action execution 1726, resource arbitration 1728, and data tracking and analysis 1758 may be implemented as one or more programs and/or applications coupled to hardware components (e.g., sensors, operating elements, I/O devices, processors, etc.). Active sensing 1722, skill and behavior learning 1724, action execution 1726, resource arbitration 1728, and data tracking and analysis 1758 may be implemented by a single robotic device or by multiple robotic devices. In some embodiments, active sensing 1722 may include active sensing or scanning of an environment, as described herein. In other embodiments, active sensing 1722 may include active scanning of an environment and/or detecting or sensing information related to the environment, one or more objects within the environment (including, for example, a human within the environment), and/or one or more conditions associated with the robotic device or system.

[0089] Similar to storage 330, storage 1730 stores information related to an environment and/or objects within the environment, as well as skill learning and/or performance (e.g., tasks and/or social behaviors). Storage 1730 stores, for example, state information 1732, one or more skill models 1734, object information 1740, machine learning libraries 1742, and/or one or more environmental constraints 1754. Optionally, storage 1730 may also store tracking information 1756 and/or one or more arbitration algorithms 1758.

[0090] State information 1732 includes information about the state of a robotic device (such as any of the robotic devices described herein) and/or the environment in which the robotic device is operating. In some embodiments, state information 1732 may include a map of the environment, along with additional static and/or dynamic information related to objects in the environment. For example, state information 1732 may include a navigation map of a building, along with static and/or dynamic information about objects within the building (e.g., fixtures, equipment, etc.), and social context information related to people and/or social situations within the building. FIG. 19 provides a schematic diagram of an example map or representation 1600 of a building. Representation 1600 includes a navigation layer 1610, a static semantic layer 1620, a social layer 1630, and a dynamic layer 1640. The navigation layer 1610 provides a general layout or map of the building, which may identify one or more floors 1612, with one or more walls 1614, one or more staircases 1616, and other elements (e.g., hallways, openings, boundaries) incorporated into the building. The static semantic layer 1620 identifies objects and/or spaces within the building, such as one or more rooms 1622, one or more objects 1624, and one or more doors 1626. The static semantic layer 1620 may identify which one or more rooms 1622 or other spaces are or are not accessible to a robotic device. In some embodiments, the static semantic layer 1620 may provide a three-dimensional map of objects located within a building. The social layer 1630 provides social context information 1632. The social context information 1632 includes information related to people within the building, such as past interactions between one or more robotic devices and one or more people. Social context information 1632 can be used to track interactions between one or more robotic devices and one or more humans, and these interactions can be used to generate and/or adapt existing models of skills involved in one or more interactions between a robotic device and a human. For example, social context information 1632 can indicate that a human is typically at a particular location, thereby enabling a robotic device with knowledge of that information to adapt the execution of a skill that requires the robotic device to move closer to the human's location. Dynamic layer 1640 provides information about one or more objects and other elements within a building that may move and/or change over time. For example, dynamic layer 1640 can track one or more movements 1644 and/or one or more changes 1646 associated with one or more objects 1642. In some embodiments, dynamic layer 1640 can monitor the expiration date of an object 1642 and identify when the object 1642 expires.

[0091] Representation 1600 may be available to and/or managed by, for example, control unit 1602 of a robotic device. Control unit 1602 may include components similar to other control units described herein (e.g., control units 202, 302, and/or 1702). Control unit 1602 may include storage (e.g., similar to other storage elements described herein, such as storage 330 and/or 1730) for storing representation 1600 and state information 1604 including information related to one or more robotic devices (e.g., the configuration of elements of a robotic device, the location of the robotic device within a building, etc.). Control unit 1602 may be located on the robotic device and/or on a remote server connected to one or more robotic devices. As one or more robotic devices collect information about their surrounding environment, one or more robotic devices may be configured to update and maintain state information 1604 including information related to building representation 1600.

[0092] In some embodiments, the control unit 1602 may also optionally include a data tracking and analysis element 1606. The data tracking and analysis element 1606 may be, for example, a computational element (e.g., a processor) configured to perform data tracking and/or analysis of information collected by one or more robotic devices (e.g., information included in representation 1600 and/or other state information 1604). For example, the data tracking and analysis element 1606 may be configured to manage inventory (e.g., tracking expiration dates, monitoring and recording inventory usage, ordering new inventory, analyzing and recommending new inventory, etc.). In a hospital setting, the data tracking and analysis element 1606 may manage the use and/or maintenance of medical supplies and/or equipment. In some embodiments, the data tracking and analysis element 1606 may generate aggregate data displays (e.g., reports, charts, etc.), which may be used to comply with official laws, regulations, and/or standards. In some embodiments, the data tracking and analysis component 1606 may be configured to analyze information related to humans, such as patients in a hospital. For example, a robotic device may be configured to collect information about one or more patients in a hospital and pass that information to the data tracking and analysis component 1606 for analysis and/or summarization for use in various functions, including, for example, diagnostic testing and/or screening. In some embodiments, the data tracking and analysis component 1606 may be configured to perform its data tracking and/or analysis functions by accessing and/or obtaining information from third-party systems (e.g., hospital electronic medical records, security system data, insurance data, vendor data, etc.) and using and/or analyzing that data with or without data collected by one or more robotic devices.

20 , the one or more skill models 1734 are models that can be used to learn and/or perform various actions or skills, including tasks and/or behaviors. The one or more models 1734 may be similar to the one or more models 334, as described herein with reference to FIG. 3 . For example, the one or more models 1734 may include information related to one or more objects involved in the performance of a skill (e.g., objects manipulated by the robotic device, objects with which the robotic device interacts while performing a skill, objects considered by the robotic device while performing a skill). As described above, examples of objects include, for example, stationary and/or movable objects, such as furniture, equipment, people, and/or surfaces defining an opening (e.g., a doorway). The information related to the one or more objects may include, for example, markers that identify the objects and/or features of the objects. Additionally or alternatively, the one or more models 1734 may include sensory information collected by the robotic device, for example, during skill learning and/or performance or active sensing. As described above, the sensory information may include information related to one or more components of the robotic device (e.g., manipulation elements, transport elements) in learning and/or performing the skill, and/or information related to the environment in which the skill is learned and/or performed (e.g., the location of one or more objects in the environment, social context information, etc.). In some embodiments, the model 1734 may be associated with success criteria, such as, for example, visual and/or tactile data sensed using one or more sensors of the robotic device, that indicate successful performance of the skill.

[0094] Object information 1740 may include information related to one or more physical objects (e.g., location, color, shape, surface features, and/or identification codes). Machine learning library 1742 may include modules, processes, and/or functions related to generating models of different machine learning algorithms and/or different skills.

[0095] Environmental constraints 1754 include information related to objects and/or conditions in the environment that may limit the operation of the robotic device within the environment. For example, environmental constraints 1754 may include information related to the size, configuration, and/or location of objects within the environment (e.g., a supply bin, a room, a doorway, etc.) and/or information indicating that access to a particular area (e.g., a room, a hallway, etc.) is restricted. Environmental constraints 1754 may affect the learning and/or execution of one or more actions within the environment. As such, environmental constraints 1754 may be part of each model for a skill that is performed within a context that includes the environmental constraint.

[0096] Tracking information 1756 includes information related to the representation of the environment (e.g., representation 1600) and/or information obtained from third-party systems (e.g., hospital electronic medical records, security system data, insurance data, vendor data, etc.) that is tracked and/or analyzed by a data tracking and analysis component, such as data tracking and analysis component 1606, or processor 1704 executing data tracking and analysis 1758. Examples of tracking information 1756 include inventory data, supply chain data, point-of-use data, and/or patient data, as well as aggregate data compiled from such data.

[0097] The one or more arbitration algorithms 1758 include algorithms for arbitrating or selecting between different actions to perform (e.g., how different resources or components of the robotic device are used). The one or more arbitration algorithms 1758 may be rules and, in some embodiments, may be learned as further described with reference to FIG. 24. These algorithms may be used by a robotic device to select among different actions when the robotic device has multiple resources and/or objectives to manage. For example, a robotic device operating in an unstructured and/or dynamic environment (e.g., an environment that includes humans) may be exposed to several conditions at any one time that may require different behaviors or actions from the robotic device. In such cases, the robotic device may be configured to select among different actions based on one or more arbitration algorithms 1758, which may assign different priorities to various actions based on predefined rules (e.g., emotional state, environmental constraints, socially defined constraints, etc.). In some embodiments, the arbitration algorithm 1758 may assign different scores or values to multiple actions based on information gathered by the robotic device regarding the surrounding environment, current conditions, and/or other factors.

[0098] Similar to the one or more I/O interfaces 208 and/or 308, the one or more I/O interfaces 1708 may be any suitable one or more components that enable communication between the internal components of the control unit 1702 and external devices, such as a user interface, an operating element, a transport element, and/or a computing device. The one or more I/O interfaces 1708 may include a network interface 1760 that can connect the control unit 1702 to a network (e.g., network 105 as shown in FIG. 1). The network interface 1760 enables communication between the control unit 1702 (which may be located on the robotic device or on another network device that communicates with one or more robotic devices) and a remote device, such as a computing device that may be used by a robot supervisor to monitor and/or control one or more robotic devices. The network interface 1760 may be configured to provide a wireless and/or wired connection to a network.

[0099] FIG. 5 schematically illustrates a robotic device 400 according to some embodiments. The robotic device 400 includes a head 480, a torso 488, and a base 486. The head 480 may be connected to the torso 488 by a segment 482 and one or more joints (not shown). The segment 482 may be movable and/or flexible to allow the head 480 to move relative to the torso 488. The head 480, the segment 482, etc. may be examples of one or more operational elements and may include similar functionality and/or structure as one or more other operational elements described herein.

[0100] Head 480 includes one or more imaging devices 472 and/or other sensors 470. Imaging device 472 and/or other sensors 470 (e.g., lidar sensors, motion sensors, etc.) can enable robotic device 400 to scan an environment and obtain a representation (e.g., a visual representation or other semantic representation) of that environment. In some embodiments, imaging device 472 may be a camera. In some embodiments, imaging device 472 may be movable so that it can be used to focus on different areas of the environment around robotic device 400. Imaging device 472 and/or other sensors 470 can collect sensory information and send the sensory information to a computing device or processor (e.g., control unit 202 or 302, etc.) onboard robotic device 400. In some embodiments, head 480 of robotic device 400 may have a human form and include one or more human features (e.g., eyes, nose, mouth, ears, etc.). In such an embodiment, the imaging device 472 and/or other sensors 470 may be implemented as one or more human features. For example, the imaging device 472 may be implemented as an eye on the head 480.

[0101] In some embodiments, the robotic device 400 can use the imaging device 472 and/or other sensors 470 to scan the environment for information about objects (e.g., physical structures, devices, items, people, etc.) within the environment. The robotic device 400 can engage in active sensing, or the robotic device 400 can begin sensing or scanning in response to a trigger (e.g., input from a user, a detected event or change in the environment).

[0102] In some embodiments, the robotic device 400 can engage in adaptive sensing, which can perform sensing based on stored knowledge and/or user input. For example, the robotic device 400 can identify areas in an environment to scan for objects based on prior information the robotic device 400 has about the objects. With reference to FIG. 6A , the robotic device 400 can scan a scene (e.g., an area of a room) and obtain a representation 500 of the scene. In representation 500, the robotic device 400 identifies that a first object 550 is located in area 510 and a second object 560 is located in areas 510 and 530. The robotic device 400 can store the locations of objects 550 and 560 in its internally stored map of the environment so that the robotic device 400 can use that information to locate objects 550 and 560 when performing future scans. For example, when the robotic device 400 returns to the scene and scans it a second time, the robotic device 400 may obtain a different view of the scene, as shown in FIG. 6B. When performing this second scan, the robotic device 400 may obtain a representation 502 of the scene. To locate objects 550 and 560 in representation 502, the robotic device 400 may refer to information that it previously stored regarding the location of these objects when it obtained representation 500 of the scene. The robotic device 400 may take into account that its location in the environment may have changed and may recognize that objects 550 and 560 may be located in different areas of representation 502. Based on this information, the robotic device 400 may know that it sees area 510 for object 550, but sees areas 520 and 540 for object 560. By using previously stored information regarding the locations of objects 550 and 560, the robotic device 400 can automatically identify areas to carefully scan (e.g., by zooming in, slowly moving the camera within the area) for objects 550 and 560.

[0103] In some embodiments, the robotic device 400 may also know to more carefully sense or scan different areas of a scene based on human input. For example, a human may indicate to the robotic device 400 that certain areas of a scene contain one or more objects of interest, and the robotic device 400 may scan those areas more carefully to identify those objects. In such embodiments, the robotic device 400 may include an input/output device 440, such as a display with a keyboard or other input device and/or a touchscreen, as shown schematically in FIG. 5.

[0104] In some embodiments, the robotic device 400 can scan the environment and identify that an object, such as a human, is moving within the environment. For example, as shown in FIGS. 7A and 7B, an object 660 may be moving within the environment while an object 650 remains stationary. FIG. 7A shows a representation 600 of a scene showing that object 660 is within areas 610 and 630, and FIG. 7B shows a representation 602 of a scene showing that object 660 is within areas 620 and 640. In both representations 600 and 602, object 650 may remain in the same location in area 610. The robotic device 400 can identify that object 660 has moved within the scene and adjust its actions accordingly. For example, if the robotic device 400 is to interact with the object 660, the robotic device 400 may change its trajectory (e.g., move closer to the object 660) and/or change the trajectory of a manipulative element or other component configured to interact with the object 660. Alternatively or additionally, if the robotic device 400 is to interact with the object 650 (and/or another object in the scene), the robotic device 400 may take the movement of the object 660 into account while planning a course for interacting with the object 650. In some embodiments, the robotic device 400 may engage in active sensing such that the actions of the robotic device 400 can be adjusted in near real time.

5, base 486 may optionally include one or more transport elements implemented as wheels 460. Wheels 460 may enable robotic device 400 to move about within an environment (e.g., a hospital). Robotic device 400 also includes at least one manipulating element implemented as arm 450. Arm 450 may be structurally and/or functionally similar to other manipulating elements described herein (e.g., manipulating element 350). Arm 450 may be fixedly attached to torso 488 of robotic device 400, or optionally, manipulating element 450 may be removably coupled to torso 488 by a coupling element (e.g., coupling element 359) that may attach to coupling portion 484 of robotic device 400. Coupling portion 484 may be configured to engage coupling element 359 and provide an electrical connection between arm 450 and an on-board computing device (e.g., control unit 202 or 302) so that the on-board computing device can power and/or control components of arm 450 and receive information collected by sensors (e.g., sensors 353, 355, 357, and 358) located on operating element 450.

[0106] Optionally, the robotic device 400 may also include one or more additional sensors 470 located on the segments 482, the torso 488, the base 486, and/or other portions of the robotic device 400. The one or more sensors 470 may be, for example, an imaging device, a force or torque sensor, a motion sensor, an optical sensor, a pressure sensor, and/or a temperature sensor. The sensors 470 may enable the robotic device 400 to capture visual and non-visual information about the environment.

method
8-11 are flow diagrams illustrating a method 700 that may be performed by a robotic system (e.g., robotic system 100) including one or more robotic devices, according to some embodiments. For example, all or a portion of method 700 may be performed by a single robotic device (such as any of the robotic devices described herein). Alternatively, all of method 700 may be performed sequentially by multiple robotic devices, each performing a portion of method 700 in turn. Alternatively, all or a portion of method 700 may be performed simultaneously by multiple robotic devices.

8, the robotic device may scan an environment and obtain a representation of the environment at 702. The robotic device may scan the environment using one or more sensors (e.g., one or more sensors 270 or 470 and/or one or more imaging devices 472). In some embodiments, the robotic device may scan the environment using a movable camera, and the position and/or focus of the camera may be adjusted to capture multiple areas within the environmental scene. At 704, based on the information collected during sensing, the robotic device may identify one or more markers within the captured representation of the environment by analyzing the data. The markers may be associated with one or more objects within the scene marked with visual or fiducial markers (e.g., visible markers such as QR codes, bar codes, tags, etc.). Alternatively or additionally, the robotic device may identify markers associated with one or more objects within the environment through object recognition using object information (e.g., object information 340) stored in memory (e.g., storage 330) on the robotic device. The object information may include information that describes different characteristics of the object, such as location, color, shape, and surface features. In some embodiments, the object information may be organized as a set of numerical values that represent different characteristics of the object, sometimes referred to as a feature space.

[0109] After identifying the one or more markers, at 706, the robotic device may optionally present the one or more markers within a representation of the environment. In some embodiments, the representation of the environment may be a visual representation, such as, for example, an augmented view of the environment. In such embodiments, the robotic device may display the visual representation of the environment, for example, on a display screen, and display the locations of the one or more markers within the visual representation of the environment. Alternatively or additionally, the representation of the environment may be a semantic representation of the environment, with the locations of the one or more markers indicated by semantic markers within the environment.

[0110] In some embodiments, the robotic device may present a representation of the environment with one or more markers to the user and, optionally, at 708, may prompt the user, e.g., using a user interface or other type of I/O device, to accept or reject one or more markers in the representation of the environment. If the user does not accept one or more markers (708: NO), method 700 may return to 702, where the robotic device may obtain a second representation of the environment by rescanning the environment. If the user accepts one or more markers (708: YES), method 700 may proceed to 708, where the robotic device may store information associated with the markers (e.g., locations, features, etc.) in memory (e.g., storage 330). For example, the robotic device may store the locations of one or more markers in an internal map of the environment (e.g., map 332).

[0111] In some embodiments, the robotic device may identify one or more markers at 704 and immediately begin saving the locations of the one or more markers and/or other information associated with the markers at 710 without prompting the user to approve the one or more markers. In such embodiments, the robotic device may analyze the locations of the one or more markers before saving them. For example, the robotic device may have previously stored information regarding the locations of one or more markers (e.g., obtained during a previous scan of the environment and/or input to the robotic device by a user or computing device) and may compare the locations of the one or more markers to the previously stored information to check accuracy and/or identify changes in the marker locations. Specifically, if the previously stored information indicates that a particular marker should be located in a location different from the location identified by the robotic device, the robotic device may verify the location by initiating an additional scan of the environment before saving the marker location. Alternatively or additionally, the robotic device may send a notification to the user indicating that the marker location has changed. In such cases, the robotic device may store the new location of the marker, but may also store a message indicating that the marker's location has changed, so that the user or computing device can later review the message and reconcile the change in the marker's location.

9 , method 700 optionally proceeds to 712, where the robotic device can prompt a user, e.g., using a user interface or other type of I/O device, to select a set of markers from one or more markers identified in the representation of the environment. At 714, the user can make a selection, and the robotic device can receive the selection from the user. Alternatively, in some embodiments, instead of prompting the user to make a selection, the robotic device can automatically select a set of markers. The robotic device may be programmed to select markers based on certain predefined or learned rules and/or conditions. For example, the robotic device may be instructed to select markers associated with a particular type of object (e.g., supplies) during a particular time of day or when foot traffic in the building is low. In the latter case, the robotic device can determine when foot traffic in the building is low by actively moving through the building (e.g., patrolling and monitoring hallways and rooms) and sensing or scanning the environment. That way, the robotic device knows to select the marker associated with a particular object when there is less foot traffic in the building than at most other times.

[0113] After the robotic device receives a selection of a set of markers from a user and/or automatically selects a set of markers, method 700 may proceed to learning the skill at 716 or performing the skill at 718.

[0114] For any particular skill, the robotic device can be taught the skill before it can execute or perform the skill. For example, to acquire a manipulation skill, the robotic device can be taught through LfD (e.g., kinesthetic teaching), in which a user (e.g., a nearby user or a remotely located robot supervisor) or another robotic device can demonstrate the skill to the robotic device. For example, a manipulation element such as an arm of the robotic device can move through a series of waypoints to interact with an object. As another example, a mobile base of the robotic device (e.g., a base with transport elements such as wheels, tracks, crawlers, etc.) can be navigated around objects in an environment by a user (e.g., by using a joystick, user interface, or other type of physical or virtual control device, or by physically directing or moving (e.g., pulling, pushing) the robotic device).

[0115] With kinesthetic teaching, a user can physically demonstrate a skill to a robotic device. Training or teaching can occur in a mass-production setting (e.g., a manufacturing environment) where the robotic device can be taught using an aggregate model that represents the generic behavior of the skill. Alternatively or additionally, teaching can occur in the field after the robotic device is deployed (e.g., in a hospital) so that the robotic device can learn to perform the skill in a specific field environment. In some embodiments, a robotic device can be taught in the field setting and then send information related to the learned skill to one or more other robotic devices so that these other robotic devices also have knowledge of the taught skill when operating in the same field setting. Such an embodiment can be useful when multiple robotic devices are deployed at a single field setting, where each robotic device can send and receive information to the other robotic devices so that the other robotic devices can collectively learn a set of skills related to the field environment.

[0116] In the learning mode shown in FIG. 10, method 700 proceeds to 720-724, where a user can teach a skill to a robotic device using an LfD teaching process. In one embodiment, a skill may be defined as grasping an object at a particular location, picking up the object, moving the object to another location, and setting the object down at the other location. In another embodiment, a skill may involve interacting with the robotic device's surrounding environment (e.g., a door).

[0117] At 720, a user (or another robotic device) can teach a robotic device, including a manipulation element (e.g., manipulation element 250, 350, or 450) and/or a transport element (e.g., one or more transport elements 260, 460), actions. For example, a user can teach a robotic device's manipulation element (and/or other components of the robotic device (e.g., a transport element)) through demonstrations related to performing a particular skill (e.g., interacting with a human, engaging with and/or manipulating an object, and/or other interactions with one or more humans and/or the surrounding environment). In some embodiments, a user can demonstrate to a robotic device how to interact with and/or navigate through a door. The user can demonstrate how the robotic device, for example, interacts with a door handle using one or more of its manipulation elements. For example, a robotic device's manipulation element, such as an arm, can be taught a series of movements relative to a door handle (e.g., entered by visual input or identified by a fiducial marker). For example, while or after a door handle is turned using a manipulating element, the robotic device may be demonstrated to push the door open, for example, by moving one or more conveying elements. As described further below, a combination of the movements of one or more manipulating elements and one or more conveying elements may be used to build a model for performing a skill or behavior for future interactions with that door and/or similar doors.

[0118] In some embodiments, the robotic device can be supervised by a locally present user (e.g., by a user physically moving the robotic device and/or providing input to the robotic device). In some embodiments, the robotic device can be supervised by a user (e.g., a robot supervisor) located remotely from the robotic device, for example, using a remote or cloud interface.

[0119] While directing a manipulative element (and/or other components of the robotic device) to move, a user can indicate to the robotic device when to capture information about the state of the manipulative element (e.g., joint configuration, joint forces and/or joint torques, end effector configuration, end effector position), other components of the robotic device, and/or the environment (e.g., location of an object associated with a selected marker and/or other objects in the environment). For example, the robotic device can receive a signal from the user at 722 at a waypoint or keyframe during movement of the manipulative element to capture information about the manipulative element and/or the environment. In response to receiving the signal, the robotic device can capture information about the manipulative element, other components of the robotic device, and/or the environment at that keyframe at 724. Manipulative element information can include, for example, joint configuration, joint torque, end effector position, and/or end effector torque. Environmental information can include, for example, the position of a selected marker relative to the end effector, indicating to the robotic device when objects in the environment may have moved. If the operation is still continuing (728: NO), the robotic device may wait to capture information about the operational element and/or the environment at additional key frames. In some embodiments, the robotic device may be programmed to capture key frame information without receiving a signal from the user. For example, while the operational element is being moved by the user, the robotic device may monitor changes in the segments and joints of the operational element. When these changes exceed a threshold or when there is a directional change in the trajectory of a segment or joint, the robotic device may autonomously select that point in time as a key frame and record information about the operational element and/or the environment at that key frame.

[0120] During operation of the manipulative element (and/or other components of the robotic device (e.g., the transport element)), the robotic device may also continuously or periodically record sensory information (e.g., information about the manipulative element, one or more other elements of the robotic device (e.g., the transport element), and/or the environment) at 730, without receiving a signal from the user. For example, as the user moves the manipulative element throughout a demonstration, the robotic device may record information about the trajectories of the segments and joints and their configurations. During the demonstration, the robotic device may also record information about one or more environmental constraints (e.g., static information about one or more objects in the environment (e.g., location or size of doorways, supply bins, etc.) and/or dynamic information about one or more objects in the environment (e.g., traffic levels in a room, user movement around the robotic device, etc.)). In some embodiments, the sensory information recorded by the robotic device during the demonstration can be added to and/or modified in one or more layers of a map of the environment, such as that shown in FIG. 19.

[0121] In some embodiments, the robotic device may include an audio device (e.g., 244), such as a microphone, and the definition of key frames may be controlled by voice commands. For example, a user can indicate to the robotic device that they are going to demonstrate by saying, "I will guide you." A demonstration may begin when the user indicates the first key frame by saying, "start here." An intermediate key frame may be indicated by saying, "go here." And the final key frame, representing the end of the demonstration, may be indicated by saying, "end here." Suitable examples of demonstration teaching are provided in Akgun's paper.

[0122] In some embodiments, while demonstrating a skill to a robotic device, a user can indicate, e.g., through one or more inputs to the robotic device, which one or more portions of the skill are generic and which one or more portions of the skill are more specific to a particular environment or situation. For example, while demonstrating to a robotic device how to move furniture from a first location to a second location (e.g., a room) and then drop the furniture off at the second location, the user can indicate to the robotic device that the action of navigating from the first location to the second location is generic, but the action of dropping the furniture off at the second location is specific and requires additional information specific to the environment to perform (e.g., a specific tag or marker for dropping the furniture off). When the robotic device later uses the model for the skill to move equipment between different locations, the robotic device may know to request and/or scan specific information regarding unloading before performing the skill (e.g., request and/or scan information regarding a specific tag to determine the location before unloading).

[0123] In some embodiments, after demonstrating a skill to the robotic device, the user can indicate which portions of the skill are generic or specific and/or change which portions of the skill the user previously indicated as generic or specific. In some embodiments, the robotic device can determine whether portions of a set of skills are generic or specific after learning a set of skills. In some embodiments, the robotic device can recommend to the user that portions of the set of skills are generic or specific and request confirmation from the user. Based on the user's confirmation, the robotic device can store this information for reference when learning and/or performing future skills related to this set of skills. Alternatively, the robotic device can automatically determine and classify different portions of a skill as generic or specific without user input.

[0124] Upon completion of the action or demonstration (728: YES), the robotic device may generate a model of the demonstrated skill based on a subset of all recorded sensory information (e.g., manipulation element information, transport element information, environmental information). For example, at 732, the robotic device may optionally prompt the user, e.g., using a user interface or other type of I/O device, to select features relevant to learning the skill, and at 734, the robotic device may receive the feature selection from the user. Alternatively or additionally, the robotic device may know to select specific features to use in generating the model based on previous instructions from the user. For example, the robotic device may recognize, e.g., based on the sensory information, that object picking is being demonstrated, and may automatically select one or more features of the sensory information (e.g., joint configuration, joint torque, end effector torque) to include as relevant features for generating the skill model, e.g., based on previous demonstrations of picking up the same or different objects.

[0125] At 736, the robotic device can generate a model of the skill using the selected features. The model may be generated using a stored machine learning library or algorithm (e.g., machine learning library 342). In some embodiments, the model may be represented as an HMM algorithm including multiple parameters, such as, for example, a number of hidden states, a feature space (e.g., features included in a feature vector), and an emission distribution for each state modeled as a Gaussian distribution. In some embodiments, the model may be represented as a support vector machine, or "SVM," model, which may include parameters, such as, for example, a kernel type (e.g., linear, radial, polynomial, sigmoidal), a cost parameter or function, weights (e.g., equal, class balanced), a loss type or function (e.g., hinge, rectangular hinge), and an answer type or problem type (e.g., dual, primal). The model may be associated with relevant sensory information and/or other sensory information recorded by the robotic device during the performance of the skill. The model may also be associated with marker information indicating the set of markers manipulated during the performance of the skill and/or features associated with one or more physical objects bound to those markers. The robotic device may store the model in memory (e.g., storage 230 or 330) at 738.

[0126] In some embodiments, information related to the performance (e.g., sensory information, models, etc.) can be used to add to and/or modify layers of a map or representation of the environment, such as the representation 1600 shown in FIG. 19. For example, FIG. 21 shows

[0127] Optionally, at 740, the robotic device may determine whether the user will perform another demonstration of the skill. If another demonstration is to be performed (740: YES), method 700 may return to 720, where the user (or another robotic device) may direct the robotic device for additional demonstrations. If the demonstration is complete (740: NO), method 700 may optionally return to the beginning and perform a new scan of the environment. Alternatively, in some embodiments, method 700 may end.

[0128] In another embodiment, the skill may be a navigation behavior, such as navigation between two locations or navigation around and/or through objects in an environment. Similar to learning a skill using a manipulative element as described herein, at 720, a user (or other robotic device) can teach a navigation behavior to a robotic device including a set of transport elements (e.g., one or more transport elements 260, 460). For example, the user can use a joystick or other control device to control the movement of the set of transport elements (and/or other components of the robotic device (e.g., manipulative elements)) and/or physically guide such elements (e.g., push or pull the robotic device) to enable the robotic device to perform a navigation behavior. While controlling the movement of the set of transport elements (and/or other components of the robotic device), the user can signal the robotic device when to capture sensory information, such as information about the state of the set of transport elements (e.g., the angle of each transport element, the configuration of each transport element, the spacing between transport elements if movable relative to one another, etc.), other components of the robotic device, and/or the environment (e.g., the location of the robotic device in a map, the location and/or boundaries of objects in the environment). The user can signal the robotic device at key frames during the movement of the set of transport elements (and/or other components of the robotic device), e.g., at the beginning, end, and/or transition points between moving the set of transport elements in a first direction and moving the set of transport elements in a second direction. In response to receiving a signal from the user at 722, the robotic device can capture snapshots of the movement (including information about the set of transport elements, other components of the robotic device, and/or the environment) at the key frames. Alternatively or additionally, the robotic device may be configured to autonomously select points during the movement of the set of transport elements (and/or other components of the robotic device) at which to capture snapshots of the movement. For example, the robotic device may monitor the angle and/or configuration of the set of transport elements (e.g., when a user takes over control of the robotic device (e.g., by controlling the robotic device suing a control device, by pushing the robotic device, etc.)) and may automatically capture a snapshot of that information each time the robotic device detects a change in angle and/or configuration. In some embodiments, the robotic device may be configured to continuously collect information about the set of transport elements, other components of the robotic device, and/or the environment while a user controls the movement of the robotic device.

[0129] Similar to learning a skill using a manipulative element as described herein, the robotic device may continue to capture sensory information related to the operation of the transport element (and/or other components of the robotic device) until the operation is completed (728: YES). At 732-734, the robotic device may optionally receive a selection of features relevant to learning the navigation behavior and/or autonomously identify relevant features in the collected sensory information (e.g., by recognizing that a particular navigation skill is being demonstrated and identifying the features relevant to learning that skill). The robotic device may then generate a model for the navigation behavior based on the sensory information related to the relevant features at 736 and store the model and sensory information at 738 for later use in generating a trajectory for the robotic device to execute.

[0130] An example of a navigation behavior is navigating through a doorway. While a robotic device can be configured to navigate through standard doorways in a building, the robotic device may be unable to navigate through non-standard doorways (e.g., small or irregularly shaped doorways). Thus, the robotic device may prompt a user to demonstrate to the robotic device how it can safely navigate through a doorway. The robotic device may navigate to one side of a doorway and then have the user demonstrate to the robotic device how to pass through the doorway to the other side. During the demonstration, the robotic device may passively record information such as the location of the robotic device within a map and/or sensor data (e.g., door boundaries, configuration of transport and/or operating elements, and other sensory information as described herein). In some embodiments, the robotic device may learn how to navigate through a doorway using an interactive learning template, as further described herein with reference to FIG. 16 .

[0131] In some embodiments, the user may be on-site near the robotic device. In other embodiments, the user may be located remotely and control the robotic device from a computing device (e.g., server 120, computing device 150) via a network connection to the robotic device (e.g., as shown in FIG. 1 and described above).

[0132] In some embodiments, as described herein, a robotic device may actively scan its surrounding environment to monitor changes in the environment. Thus, during learning and/or execution, the robotic device may engage in continuous sensing or scanning of the environment and update its representation of the environment and the environmental information stored by the robotic device accordingly.

[0133] In some embodiments, a robotic device can be configured to learn socially appropriate behaviors, i.e., actions that take into account interactions with humans. For example, a robotic device can be configured to learn manipulation or navigation skills to be performed around humans. The robotic device can learn socially appropriate behaviors through demonstration by a human operator (e.g., a nearby user or a remotely located robot supervisor). In some embodiments, a robotic device may be configured to learn behaviors in an interactive context (e.g., by a human operator intervening in the robotic device's autonomous performance of a skill and demonstrating to the robotic device how to perform the skill). Upon detecting human operator intervention (e.g., a nurse pushing the robotic device to one side), the robotic device may be configured to switch to a learning mode in which the robotic device is controlled by the human operator and passively records information related to the demonstration and the perceptual context in which the demonstration takes place. The robotic device can use this information to generate a revised model of the skill and associate the model with the appropriate social context in which the robotic device should later perform the skill. As an example, the robotic device can detect when a human operator intervenes in the autonomous execution of the navigation plan and, in response to the detected intervention, switch to a learning model controlled by the human operator. In the presence of a human, the robotic device can record information about its surroundings and its own behavior as the human operator demonstrates how the navigation plan should be modified (e.g., that instead of waiting for a clear aisle, the robotic device should move to the side of the aisle to let the human through). Further details regarding interactive learning are described herein with reference to FIG. 16.

11 , the robotic device can optionally prompt the user at 750, e.g., using a user interface or other type of I/O device, to select a model, such as a model previously generated by the robotic device in learn mode. The robotic device can receive the model selection at 752. In some embodiments, the robotic device can receive the model selection from the user, or alternatively, the robotic device can automatically select a model based on certain rules and/or conditions. For example, the robotic device may be programmed to select a model when it is in a particular area of a building (e.g., when it is in a particular room or floor), during a particular time or day of the week, etc. Alternatively or additionally, the robotic device can know to select a particular model based on a selected set of markers. At 754, the robotic device can determine whether to move near the selected markers before generating a trajectory and performing a skill on the selected markers. For example, the robotic device can determine whether to move to be more desirable for performing the skill (e.g., closer or closer to the markers, facing the markers from a particular angle) based on the selected set of markers and the selected model. The robotic device may make this determination based on sensory information recorded during the performance of the skill. For example, the robotic device may recognize that it was positioned closer to a marker when the skill was performed and adjust its position accordingly.

[0135] If the robotic device determines to move relative to the selected marker (754: YES), the robotic device may move its position (e.g., adjust its location and/or orientation) at 756, and method 700 may return to 702, where the robotic device again scans the environment to obtain a representation of the environment. Method 700 may proceed through various steps and return to 754. If the robotic device determines not to move relative to the selected marker (754: NO), the robotic device may generate a trajectory of action, for example, for a manipulative element of the robotic device.

[0136] Specifically, at 758, the robotic device can calculate a function that transforms (e.g., translates) between the selected set of markers and the set of markers associated with the selected model (e.g., one or more markers selected when the robotic device learned the skill, i.e., generated the selected model), referred to herein as the "saved markers" or "saved set of markers." For example, a robotic device can be taught a skill using a first set of markers that were present in one or more particular locations and/or one or more orientations relative to a portion of the robotic device, such as an end effector. The robotic device can later perform the skill using a second set of markers that are present in one or more different locations and/or one or more different orientations relative to the manipulative element. In such a case, the robotic device can calculate a transformation function that transforms between one or more positions and/or one or more orientations of the first set of markers and one or more positions and/or one or more orientations of the second set of markers.

[0137] At 760, the robotic device may use the calculated transformation function at each keyframe recorded as the skill was taught to transform the position and orientation of a portion of the manipulative element (e.g., the end effector of the manipulative element). Optionally, at 762, the robotic device may take into account any environmental constraints, e.g., characteristics of objects and/or areas in the environment, such as size, configuration, and/or location. The robotic device may limit the movement of the manipulative element based on the environmental constraints. For example, if the robotic device recognizes that it is attempting to perform a skill in a supply room, the robotic device may take into account the size of the supply room when transforming the position and orientation of the manipulative element to avoid contact of a portion of the manipulative element with walls or other physical structures within the supply closet. The robotic device may also take into account other environmental constraints related to the supply room, such as the size of bins in the supply room, the location of shelves within the supply room, etc. The robotic device may be provided with information about and/or taught one or more environmental constraints prior to performing a skill in a situation having one or more environmental constraints, as further described herein with reference to FIG. 17 .

Optionally, at 762, the robotic device may use inverse kinematics equations or algorithms to determine 762 the joint configurations of the manipulative elements for each keyframe. The positions and orientations of the end effector and marker set may be provided in task space (e.g., the Cartesian space in which the robotic device is operating), and the joint orientations may be provided in joint or configuration space (e.g., an n-dimensional space associated with the configuration of the manipulative elements, where the robotic device is represented by a point and n is the number of degrees of freedom of the manipulative elements). In some embodiments, the inverse kinematics calculations may be guided by joint configuration information recorded when the robotic device was taught a skill (e.g., joint configurations recorded during a teaching demonstration using the manipulative elements). For example, the inverse kinematics calculations may be seeded (e.g., provided as an initial guess for the calculations or provided with a bias) using the joint configurations recorded at each keyframe. Additional conditions may also be imposed on the inverse kinematics calculations, such as requiring that the calculated joint configurations not deviate from the joint configurations of adjacent keyframes by more than a predefined amount. At 764, the robotic device can plan a trajectory between joint configurations, e.g., in joint space, from one key frame to the next, to generate a complete trajectory for the manipulative element to perform the skill. Optionally, the robotic device can take into account environmental constraints as described herein.

[0139] In some embodiments, after the robotic device transforms the position and orientation of a portion of a manipulative element (e.g., an end effector), the robotic device can plan a trajectory for the manipulative element in task space. In such embodiments, method 700 can proceed directly from 760 to 764.

[0140] At 766 and 768, the robotic device can optionally present the trajectory to the user and prompt the user to approve or reject the trajectory, for example, using a user interface or other I/O device. Alternatively, the robotic device may approve or reject the trajectory based on internal rules and/or conditions and by analyzing associated sensory information. If the trajectory is rejected (768: NO), the robotic device can optionally modify one or more parameters of the selected model at 770 and generate a second trajectory at 758-764. The model parameters can be modified, for example, by selecting different features (e.g., different sensory information) to include in the model generation. In some embodiments where the model is an HMM model, the robotic device can change the model parameters based on the determined success or failure, where the robotic device tracks the log-likelihood of different models with different parameters and selects the model with a higher log-likelihood than the other models. In some embodiments where the model is an SVM model, the robotic device can change the parameters by changing the feature space or configuration parameters (e.g., kernel type, cost parameters or cost functions, weights) as described herein.

[0141] If the trajectory is approved (768: YES), the robotic device may move the manipulative element to execute the generated trajectory at 772. While the manipulative element executes the planned trajectory, the robotic device may record and/or store sensory information, e.g., information about the manipulative element and/or the environment, at 774, e.g., using one or more sensors on the manipulative element and other components of the robotic device.

Optionally, at 774, the robotic device can determine whether the execution of the skill was successful (e.g., whether the interaction with the object met predefined success criteria). For example, the robotic device can scan the environment, determine the current state of the environment and the robotic device, including, for example, the location of one or more objects and/or the position or orientation of those objects relative to a manipulative element or another component of the robotic device, and determine whether the current state matches the predefined success criteria. The predefined and/or learned success criteria may be provided by a user or, in some embodiments, may be provided by a different robotic device and/or computing device. The predefined and/or learned success criteria may indicate information about different characteristics of the environment and/or robotic device that are associated with success. In some embodiments, the user can also provide input to the robotic device indicating that the execution was successful.

[0143] In a particular example where a skill is defined as grasping an object at a particular location and picking up the object, success of the skill may be taught and/or defined as detecting one or more markers associated with the object in a particular relationship relative to each other and/or to the robotic device, or detecting that sufficient force or torque is (or was) experienced by an end effector or joint (e.g., a wrist joint) of a manipulative element (meaning that the manipulative element is supporting the weight of the object and thus has picked up the object). If the execution was unsuccessful (776: NO), the robotic device may optionally modify model parameters at 770 and/or generate a new trajectory at 758-764. If the execution was successful (776: YES), data related to the successful interaction (e.g., data indicating that the execution was successful and how it was successful) may be recorded, and method 700 may, optionally, start over and perform a new scan of the environment. Alternatively, in some embodiments, method 700 may end.

[0144] In some embodiments, the robotic device is configured to perform a skill involving one or more movements of a manipulating element, a transport element, and/or another component of the robotic device (e.g., head, eyes, sensors, etc.). The robotic device may be configured to plan a trajectory for the skill, similar to that described above with respect to the manipulating element. For example, at 750-752, the robotic device may prompt and receive a user selection of a model for the skill, or alternatively, may autonomously select a model for the skill. At 758, the robotic device may calculate a function that transforms between a set of markers currently identified in the environment and a set of stored markers associated with the selected model (e.g., one or more markers identified when the robotic device learned the skill and stored with the model of the skill). Such transformations may include transforming key frames associated with the transport element or other component of the robotic device. At 760, the robotic device may use the calculated function to transform the configuration of one or more components of the robotic device at each key frame. At 764, the robotic device may plan a trajectory for one or more components of the robotic device between each transformed key frame. While transforming the key frames and/or planning the trajectory, the robotic device can optionally take into account any environmental constraints related to the context in which the skill is being performed. At 772, the robotic device can perform movement of one or more components of the robotic device according to the planned trajectory. In some embodiments, the robotic device may determine the joint configuration at 762, present the planned trajectory to the user at 766, and/or perform other optional steps as shown in FIG. 11 . An example of a skill involving the movement of a transport element may be opening a door. A user (e.g., a nearby user or a remotely located robot supervisor) can instruct the robotic device to open the door using the robot's base (e.g., by pushing the door open). The user can instruct the robotic device, for example, using a control device such as a joystick or by physically instructing the robotic device to perform the action of opening the door using the base (e.g., pushing or pulling the robotic device). The robotic device can sense and record information while the user is instructing the movement and use that information to generate a skill model. The robotic device can later use the skill model to perform actions to open that door or other doors in the environment, for example, by transforming keyframes associated with transport elements during the user's skill demonstration.

[0145] FIG. 12 is a block diagram illustrating a system architecture for robot learning and execution, including actions performed by a user, according to some embodiments. System 800 can be configured for robot learning and execution. System 800 may include one or more robotic devices, such as any of the robotic devices described herein, that can perform the modules, processes, and/or functions illustrated in FIG. 12 as active sensing 822, marker identification 824, learning and model generation 826, trajectory generation and execution 828, and success monitoring 829. Active sensing 822, marker identification 824, learning and model generation 826, trajectory generation and execution 828, and success monitoring 829 may correspond to one or more steps performed by the robotic device, as described with reference to method 700 shown in FIGS. 8-11. For example, active sensing 822 may include step 702 of method 700, marker identification 824 may include one or more of steps 704-710 of method 700, learning and model generation 826 may include one or more of steps 712-738, trajectory generation and execution 828 may include one or more of steps 712, 714, 718, and 750-774, and success monitoring 829 may include one or more of steps 774 and 776.

[0146] The system 800 can be connected to (e.g., in communication with) one or more devices, including, for example, one or more cameras 872, an arm 850 (including a gripper 856 and one or more sensors 870), a display device 842, and a microphone 844. The system 800 can receive input from a user using the display device 842, the microphone 844, and/or other I/O devices (not shown) related to one or more user actions 880. User actions 880 can include, for example, a user approving a marker or requesting a rescan of the environment 882, a user selecting one or more markers 884, a user selecting relevant information and generating a model 886, a user selecting a model for performing a skill 888, a user approving a trajectory for performing a skill 890, a user confirming the success of the performed skill 892, and a user teaching a skill through kinesthetic learning 894.

[0147] With respect to active sensing 822, system 800 scans the environment using one or more cameras 872 and records sensory information about the environment, including information associated with one or more markers in the environment. With respect to marker identification 824, system 800 may analyze the sensory information to identify one or more markers in the environment and receive one or more inputs from the user, for example, using display device 842 (shown 822 with the user approving the marker or requesting a rescan of the environment). With respect to learning and model generation 826, system 800 may receive sensory information collected by one or more cameras 872 and/or one or more sensors 870 on arm 850 and use that information to generate a model of the skill. As part of learning and model generation 826, system 800 can receive one or more inputs from the user, for example, using display device 842 and/or microphone 844 (showing user selection of one or more sets of markers for teaching the skill 884, user selection of particular features of recorded sensory information for use in generating a model 886, and/or user demonstrating the skill 894). Regarding trajectory generation and execution 828, system 800 can generate a planned trajectory and control the movement of arm 850 to execute the trajectory. Regarding trajectory generation and execution 828, system 800 can receive one or more inputs from the user, for example, using display device 842 (showing user selection of a model for generating a trajectory 888, and/or user approval or rejection of the generated trajectory 890). Regarding success monitoring 829, system 800 can determine whether the skill was successfully performed by analyzing sensory information recorded by one or more sensors 870 during the skill's performance. As part of success monitoring 829, system 800 may receive one or more inputs from the user (shown 892 with user confirmation that execution was successful), for example, using display device 842 and/or microphone 844.

[0148] While particular one or more devices and/or connections between system 800 and such one or more devices are shown in FIG. 12, it is understood that any of the embodiments described herein may enable one or more additional devices (not shown) to communicate with system 800 to receive information from and/or transmit information to system 800.

[0149] Figures 13-17 are flow diagrams illustrating methods 1300 and 1400 that may be performed by a robotic system (e.g., robotic system 100) including one or more robotic devices, according to embodiments described herein. For example, methods 1300 and/or 1400 may be performed by a single robotic device and/or multiple robotic devices.

[0150] As shown in FIG. 13 , the robotic device is configured to operate in an execution mode at 1301. In execution mode, the robotic device can autonomously plan and execute actions within an environment. To determine which actions to perform and/or plan how to perform the actions, the robotic device can scan the environment and collect information about the environment and/or objects within the environment at 1304 and use this information to build and/or modify a representation or map of the environment (e.g., representation 1600) at 1305. The robotic device can repeatedly (e.g., at predefined times and/or time intervals) or continuously scan the environment and update the representation of the environment based on the information it collects about the environment. Similar to the methods described above, the robotic device can collect information about the environment using one or more sensors (e.g., one or more sensors 270 or 470 and/or one or more imaging devices 472).

[0151] In some embodiments, the robotic device can repeatedly and/or continuously scan an environment at 1304 for data tracking and/or analysis at 1307. For example, the robotic device can move through an environment (e.g., a building such as a hospital) autonomously or remotely driven (e.g., by a robot supervisor) and collect data about the environment, objects within the environment, etc. This information can be used, for example, by a data tracking and analysis component (e.g., data tracking and analysis component 1606) that manages hospital supplies and/or equipment. In some embodiments, the robotic device can collect information about various people (e.g., patients) to track the behavior of such patients (e.g., compliance with medications and/or treatments), perform diagnostic tests, and/or conduct screenings, etc.

[0152] In some embodiments, the robotic device can repeatedly and/or continuously scan its environment at 1304 for modeling and learning purposes at 1309. For example, the robotic device can collect data about its environment and/or objects within that environment (e.g., including humans and their behavior in response to robot actions) and use that information to develop new behaviors and/or actions. In some embodiments, the robotic device can collect a large amount of information about an environment, which can be used by the robotic device to further refine and/or generate a model specific to that environment. In some embodiments, the robotic device can provide this information to a robot supervisor (e.g., a remote user), who can use this information to further adapt the robotic device to a particular environment (e.g., by generating and/or modifying skill models and/or behaviors within that environment). The robot supervisor can repeatedly (e.g., at specific time intervals) and/or continuously (e.g., in real time) fine-tune the information collected by the robotic device and/or other robotic devices and/or the parameters of the models the robot is using. In some embodiments, this active exchange of information with the robotic device allows the robotic supervisor to repeatedly and/or continuously adapt the robotic device to a particular environment. For example, the robotic supervisor can modify a planned path that the robotic device may be using to navigate through an environment, which in turn can change the information and/or one or more models used by the robotic device to generate the movements of its transport elements. These changes, both provided by the robotic supervisor and/or implemented by the robotic device, can be fed into one or more layers (e.g., the social layer or semantic layer of the map) of the environment (e.g., map 1600).

[0153] At 1306, the robotic device may determine which one or more actions to perform in the environment, i.e., arbitrate among a set of resources or components on which a particular action can be performed. The robotic device may select among different actions based on one or more arbitration algorithms (e.g., one or more arbitration algorithms 1758). The robotic device may arbitrate autonomously and/or with user input. For example, the robotic device may be configured to request user input when it is unable to determine the current state of one or more of its components and/or objects in the environment, or when it is unable to determine which action to perform and/or plan how to perform a selected skill. In some embodiments, the robotic device may be configured to autonomously select from among a set of actions when it is familiar with a particular situation (e.g., has previously learned and/or performed actions in that situation) and to request user input when it encounters a new situation. When the robotic device encounters a new situation, the robotic device may request that the user select an appropriate action for the robotic device to perform, or alternatively, the robotic device may prompt the user to select an action and confirm the selection of the action.

[0154] In some embodiments, a human operator (e.g., a robot supervisor) may monitor the robotic device and send a signal to the robotic device when the human operator wishes to intervene in the performance of an action by the robotic device. The human operator may be located near the robotic device and/or on a remote computing device connected via a network to the robotic device or a nearby device that may be used to monitor the robotic device. For example, the human operator may decide to intervene in the performance of an action by the robotic device when the human operator wishes to teach the robotic device a new skill or behavior for safety reasons (e.g., to prevent harm to the environment surrounding the human or the robotic device) and/or to prevent damage to the robotic device.

[0155] In response to determining that user input is required (e.g., due to an unfamiliar situation or in response to a signal from the user) (1312: YES), the robotic device may optionally prompt the user to provide user input at 1314. For example, the robotic device may display a prompt to the user requesting user input on an on-board display or a display located on a remote device (e.g., using a remote or cloud interface). The robotic device may receive the user input at 1315 and perform arbitration based on the user input. If user input is not required (1312: NO), the robotic device continues to perform arbitration autonomously.

[0156] After the robotic device selects an action to perform, the robotic device can plan and execute the action at 1308. As described above, the action can be associated with a task, such as a manipulation action (e.g., involving a manipulating element, such as a manipulating element described herein), or a movement (e.g., involving a transport element, such as a transport element described herein), and/or a social behavior. While performing the action, the robotic device can continue to scan its surrounding environment at 1304. If the robotic device detects a change in its current state and/or the state of the environment (e.g., the location of an object within the environment), the robotic device can evaluate the change at 1310 to determine whether the robotic device should abort the execution of the action. For example, the robotic device may decide to abort the execution of the action in response to detecting physical engagement of one or more of its components with a human or other object in the environment (e.g., when a human operator contacts a manipulating element, transport element, etc.), or when it detects the presence of a human or other object in the immediate vicinity. Additionally or alternatively, the robotic device may decide to suspend execution of an action in response to receiving a signal from a user (e.g., a robot supervisor). In some embodiments, the robotic device may be pre-configured to suspend execution of an action at a particular point during execution of the action, for example, when the robotic device may require the user to demonstrate part of the action as defined in the interactive learning template. Further details regarding learning using interactive learning templates are described herein with reference to FIG. 15.

[0157] If the robotic device determines to abort the execution of the action (1310: YES), the robotic device may determine at 1312 whether user input is required. As described above, if the robotic device determines that user input is required (1312: YES), the robotic device may optionally prompt and/or receive user input at 1314-1315. The robotic device may then return to sensing or scanning the surrounding environment at 1304, arbitrating for a set of resources at 1306, and/or performing actions at 1308. Optionally, as shown in FIGS. 14 and 15 , once the robotic device determines that user input is required, the robotic device may switch to a learning mode at 1402 and then proceed to learning a skill at 1404 or learning environmental constraints at 1406. If the robotic device determines that user input is not required (1312: NO), the robotic device can return to sensing or scanning the surrounding environment at 1304, arbitrating for a set of resources at 1306, and/or performing an action at 1308. If the robotic device determines that the action does not need to be interrupted (1310: NO), the robotic device can continue performing the action at 1308.

[0158] In some embodiments, the robotic device may be configured to determine when it may need to ask for user input, for example, to help it perform a particular task or behavior. Such assistance may be requested both during the performance of a particular action and/or while the robotic device is navigating an environment. The robotic device may be configured to use its learned model and information about the environment and objects within that environment to determine 1312 when it needs to ask for user input and how to ask for that user input. For example, the robotic device may use knowledge of the types of doors and different users in its environment to determine when and how to ask for help (e.g., asking different users to help open different types of doors (e.g., doors with keypads that require specific permissions or knowledge), and knowing to ask certain types of users who may be more likely to provide help than others (e.g., nurses vs. doctors)).

[0159] In some embodiments, for example, the user input received by the robotic device at 1315 may include feedback from the user regarding whether the selection and/or execution of the skill was appropriate and/or successful. For example, the user may tag an action (e.g., a task or behavior) being performed by the robotic device (or previously performed by the robotic device) as a good or bad example of the action. The robotic device may store this feedback (e.g., as success criteria associated with this action and/or other actions) and use it to adjust the selection and/or execution of that or other actions in the future.

[0160] In some embodiments, the robotic device is configured to engage in socially appropriate behaviors. The robotic device may be designed to operate in an environment where humans are present. When operating around humans, the robotic device may be configured to continually plan 1306-1308 how its actions may be perceived by humans. For example, when the robotic device is moving, the robotic device may monitor its surroundings in search of humans in 1304 and engage in social interactions with humans it encounters in 1308. When the robotic device is not performing any tasks (e.g., is stationary), the robotic device may continue to monitor its surroundings in search of humans in 1304 and determine 1306 whether the robotic device may need to perform one or more socially appropriate behaviors based on how humans may perceive the robotic device's presence. In such embodiments, the robotic device may be configured with an underlying framework (e.g., one or more arbitration algorithms) to plan and execute socially appropriate behaviors given a particular context or situation. The robotic device may be configured to manage multiple resources (e.g., manipulation elements, carrying elements, humanoid components such as heads or eyes, sound generators, etc.) and generate appropriate behaviors based on one or more of these resources.

[0161] FIG. 18 provides an example of a component of a robotic device that arbitrates regarding an attention mechanism (e.g., an eye element). For example, a robotic device may have a camera or other element that can be perceived by a human as having a line of sight (e.g., an eye element). A human near the robotic device may recognize the direction the eye element is pointed as the robotic device's attention. Thus, the robotic device may be configured to determine where the eye element is pointed when the robotic device is operating and/or near a human. The robotic device may continually arbitrate the use of the eye element so that the robotic device maintains socially appropriate behavior when operating around and/or in the presence of humans.

[0162] As shown in FIG. 18 , different components of the robotic device may request a gaze target (e.g., pointing the eye element in a particular direction). A first component associated with active sensing (e.g., a camera, laser, or other sensor) may determine 1510 that an object is within the field of view of the eye element. The first component may further determine 1514 that the object is a human. In response to determining that the object is a human, the first component may send a request 1516 to the central resource manager 1508 (e.g., a control unit (e.g., one or more control units 202, 302, and/or 1702) and/or components of the control unit) to point the eye element toward the human's face. A second component associated with navigation actions (e.g., a camera, laser, or other sensor) may determine 1520 that a particular location (e.g., an intermediate or final navigation destination) is within the field of view of the eye element. In response to determining that the location is within line of sight, the second component can send a request 1522 to the central resource manager 1508 to direct an eye element to the location. A third component associated with the manipulation action (e.g., a sensor on the manipulation element, a camera, or other sensor) can determine 1530 that the robotic device is engaged with an object (e.g., the robotic device is carrying an object, a human has touched the robotic device). In response to determining that the robotic device is engaged with the object, the third component can send a request 1532 to the central resource manager 1508 to direct an eye element to the object.

[0163] The central resource manager 1508 receives requests 1516, 1522, and 1532 from the components of the robotic device and can arbitrate 1540 to determine in which direction the eye element should be pointed. When determining in which direction the eye element should be pointed, the central resource manager 1508 can take into account specified rules or constraints (e.g., socially appropriate constraints). These specified rules can include, for example, not moving the eye element twice for a predefined period of time (e.g., approximately 5 seconds), having the eye element point in a direction for at least a predefined minimum period of time (e.g., approximately 5 seconds), moving the eye element after a predefined maximum period of time, which can differ for non-human objects and humans, etc. The specified rules can be encoded into an arbitration algorithm for managing gaze resources.

[0164] In some embodiments, the central resource manager 1508 can arbitrate gaze resources based on social context and other information collected about the environment. For example, a robotic device may be configured to associate certain behaviors with particular locations (e.g., waiting for a person to pass through a crowded doorway or hallway before attempting to navigate through the doorway or hallway, operating quieter in quiet areas (e.g., refraining from using sound and/or speech functionality), etc.). The robotic device may be configured to capture social context information associated with such locations and add it to a representation of the environment (as described herein with reference to FIG. 19) used for navigation. In some embodiments, a human operator may also provide social context information to the robotic device, and the robotic device can adapt this information over time as it learns more about the environment in which it operates.

[0165] FIGS. 15-17 show flow diagrams of a robotic device operating in learn mode. As described above, the robotic device may operate in run mode and switch to operate in learn mode, for example, at 1312, when the robotic device determines that it requires user input. Alternatively or additionally, the robotic device may be configured to operate in learn mode, for example, when the robotic device is initially deployed to a new environment (e.g., a new area, building, etc.). The robotic device may operate in learn mode until a user indicates to the robotic device that it is capable of switching to operate in run mode and/or until the robotic device determines that it is capable of switching to operate in run mode.

[0166] When operating in the learning mode, the robotic device may learn a skill at 1404 and/or learn environmental constraints at 1406. The robotic device may be configured to learn a skill with or without an existing model of the skill (e.g., a generic model of the skill). When learning a skill without an existing model (e.g., learning a skill without relying on prior knowledge), the robotic device may generate a model of the skill after being taught performance of the skill, as described herein with reference to FIG. 10. When learning a skill using an existing model (e.g., a generic model of the skill), the robotic device may begin performing the skill using the existing model and request user input when the robotic device requires the user to demonstrate a portion of the skill. The existing model of the skill may be part of, or form part of, an interactive learning template (i.e., a template for teaching the robotic device to learn a skill with input from a user at specified points during performance of the skill).

[0167] FIG. 16 illustrates a process for learning a skill using an interactive learning template. A robotic device may be deployed in a particular context (e.g., a building). The robotic device may select an existing model of the skill at 1410. The existing model of the skill may be a generic model of the skill that is not specific to the environment or context in which the robotic device is operating. At 1412, the robotic device may generate a plan to perform the skill using the skill model, following a process similar to that described herein with reference to FIG. 11. At 1414, the robotic device may begin performing the skill by performing one or more steps or portions of the skill that do not require user input and/or specialization. Upon reaching a portion of the skill (e.g., a portion of the skill that the robotic device cannot determine how to perform or that requires specialization to the context in which the robotic device is performing the skill) (1416: YES), the robotic device may prompt the user to provide a demonstration of that portion of the skill at 1418. In some embodiments, the interactive learning template may indicate to the robotic device when the robotic device should prompt the user for a demonstration. Alternatively or additionally, the robotic device may autonomously determine that user input is required before the robotic device can perform a portion of a skill, for example, if the robotic device is unable to determine the state of objects in the environment and/or unable to generate a plan to perform a particular portion of a skill given constraints imposed by the environment. In some embodiments, a user (e.g., a robot supervisor) may also monitor the robotic device's performance of a skill and send a signal to the robotic device when the user wants the robotic device to demonstrate a portion of a skill. Upon receiving a signal from the user, the robotic device may decide to proceed to 1418.

At 1420, a user can teach the robotic device an action. For example, the user can demonstrate a portion of a skill by moving one or more components (e.g., a manipulation element, a transport element, etc.) of the robotic device. While teaching the robotic device an action, the user can optionally indicate to the robotic device at 1422 when it should capture information about the state of one or more of the components of the robotic device and/or the environment. For example, the robotic device can receive a signal from the user at key frames during the robotic device's movement to capture sensory information, including information about the manipulation element, the transport element, and/or the environment. Alternatively, the robotic device can autonomously determine when to capture sensory information. For example, while the robotic device is being moved by the user, the robotic device can monitor changes in one or more of its components, and when those changes exceed a threshold or a directional change occurs in the trajectory of a component, the robotic device can autonomously select that time as a key frame and record information about the robotic device and/or the environment at that key frame. In response to receiving a signal from a user or determining to autonomously capture sensory information, the robotic device can capture sensory information using one or more of its sensors at 1424. During operation of the robotic device, the robotic device can also record sensory information continuously or periodically at 1430 without receiving a signal from a user.

[0169] Upon completion of the action or performance (1426: YES), the robotic device may optionally receive, at 1432, a selection of features relevant to learning the portion of the skill that was demonstrated. In some embodiments, the robotic device may autonomously perform the feature selection and/or prompt the user to confirm the selection made by the robotic device, at 1432. At 1436, the robotic device may save information related to the performance. If skill performance is not complete (1417: NO), the robotic device may continue performing the skill, at 1414, and, if necessary, prompt the user for further performance of portions of the skill, at 1418. The robotic device may continue to repeat the interactive learning process until action performance is complete (1417: YES), at which point the robotic device may generate, at 1438, a model of the skill having portions specific to the context in which the robotic device performed the skill. The robotic device may then learn another skill and/or environmental constraints. Alternatively, if the robotic device does not need to learn additional skills and/or environmental constraints, the robotic device may switch to execution mode at 1301 and begin sensing or scanning the environment, performing arbitration, and/or performing actions.

[0170] Using interactive learning templates, robotic devices are taught and/or provided with an initial set of models for skills. The initial set of models may be developed before the robotic devices are deployed in the field in a particular environment (e.g., a hospital). For example, this initial set of models may be developed and made available to the robotic devices in a factory setting or at a training site. Once deployed in the field, the robotic devices can adapt or specialize the initial set of models to the environment, for example, through interactive learning sessions as described herein. Additionally, as new models are developed (e.g., off-site at a factory or training site), the new models may be made available to the robotic devices, for example, via a network connection. Thus, the systems and methods described herein enable users and/or entities to continue developing new models of skills and providing them to robotic devices even after those robotic devices are deployed in the field.

[0171] An example of an interactive learning session may involve adapting a generic model for moving an item into a room. A robotic device may be equipped with the generic model and deployed on-site at a hospital. Once deployed at the hospital, the robotic device may autonomously begin performing the skill of moving an item into a patient's room, 1412-1414. For example, the robotic device may autonomously navigate to the patient's room using a hospital map. Once the robotic device has navigated to the patient's room, the robotic device may determine that the robotic device needs a user to demonstrate where in the patient's room the item should be dropped off (1416: YES). The robotic device may prompt the user to move it to a specific drop-off location, 1418. The user may control the operation of the robotic device, 1420, using, for example, a joystick or other type of control device. As described above, the user may be on-site near the robotic device or may be located remotely. While the user moves the robotic device to the unloading location, the robotic device can capture sensory information about its current state and/or its surrounding environment at 1424 and/or 1430. Upon arriving at the unloading location, the robotic device can switch back to autonomous execution at 1414. For example, the robotic device can perform known arm movements (e.g., locating an item in a container loaded on the robotic device, grasping the item with a manipulative element, and positioning the manipulative element in a generic position to release the item). The robotic device can make a second determination (1416: YES) that the robotic device needs the user to demonstrate a portion of a skill (e.g., lowering an item onto a shelf). The robotic device can prompt the user for another demonstration at 1418, and the user can move the manipulative element to place the item in place on the shelf. The robotic device can again capture sensory information during the operation of the manipulative element at 1424 and/or 1430. The robotic device may regain control at 1414 and autonomously open the gripper of the manipulative element to place the item on the shelf, then retract the manipulative element back to its resting position. The robotic device may then determine that execution is complete (1417: YES) and generate a specialized model of the skill at 1438 based on the information captured by the robotic device during the two user demonstrations.

[0172] Other examples of interactive learning sessions may include adapting navigation actions to navigate through certain corridors and/or doorways, for example, as described above.

[0173] In some embodiments, a user (or robotic device) can modify a model of a skill after the skill is demonstrated and/or after the model of the skill is generated. For example, a user can iterate through keyframes captured during a skill performance and decide whether to retain, modify, and/or delete these keyframes. By modifying and/or deleting one or more keyframes, the user can modify the model of the skill generated based on these keyframes. An example of using iterative and adaptive versions of keyframe-based performances is described in a paper entitled "Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective" by Akgun et al., published in Proceedings of the 7th Annual ACM/IEEE International Conference on Human-Robot Interaction (2012), pp. 391-98, which is incorporated herein by reference. As another example, an interactive graphical user interface ("GUI") can be used to display keyframes captured during a skill performance to a user so that the user can indicate how much variance is acceptable associated with the position, orientation, etc., of the robotic device's components during the planned execution. Examples of using a GUI with keyframes are described in a paper by Kurenkov et al. entitled "An Evaluation of GUI and Kinesthetic Teaching Methods for Constrained-Keyframe Skills," published at the IEEE/RSJ International Conference on Intelligent Robots and Systems (2015), available at http://sim.ece.utexas.edu/static/papers/kurenkov_iros2015.pdf, which is incorporated herein by reference. These examples enable modification of a skill model after the skill has been learned. The systems and methods described herein further provide learning templates that enable a robotic device to plan and execute certain portions of a skill while leaving other portions of the skill to user demonstration during ongoing execution of the skill. The systems and methods described herein also provide a robotic device that can determine, for example, autonomously or based on information provided to the robotic device prior to the learning process, when to collect data and/or when to request the demonstration of a portion of a skill.

17 illustrates a process for learning environmental constraints. As described above, a robotic device can be configured to learn environmental constraints, which can be applicable to a set of skills to be learned and/or performed in a situation that includes the environmental constraints. At 1450, the robotic device can optionally prompt the user to select a type of constraint and/or an object associated with the constraint. The type of constraint may be, for example, a barrier (e.g., a wall, a surface, a boundary), a size and/or dimension, a restricted area, an object location, a time constraint, etc. At 1452, the robotic device can receive the selection of the type of constraint and/or object. Optionally, at 1454, a user (or another robotic device) can direct the robotic device in actions to demonstrate the environmental constraints. For example, a user can demonstrate the size of an equipment container to the robotic device by moving a manipulative element of the robotic device along one or more edges of the container. As another example, a user can demonstrate a shelf location by moving a manipulative element of the robotic device along the surface of the shelf. During operation of the robotic device, the robotic device may capture 1456 information related to the constraints using one or more sensors (e.g., cameras, lasers, haptics, etc.). At 1458, the robotic device may store the captured information related to the environmental constraints for use with models of skills to be performed in the situation that include the environmental constraints. In some embodiments, the robotic device may include information related to the environmental constraints in each model of a skill to be performed in the situation. Alternatively, the robotic device may be configured to reference information related to the environmental constraints when planning and/or performing a skill in the situation. In some embodiments, the environmental constraints may be added to a representation of the environment (e.g., representation 1600).

[0175] In some embodiments, the robotic device can learn environmental constraints without requiring user demonstration or robotic device movement. For example, after receiving a constraint type and/or object selection at 1452, the robotic device may be configured to scan the environment for relevant information related to the environmental constraints at 1456. In some embodiments, the robotic device can present the information it has captured and/or determined to be relevant to the environmental constraints to the user so that the user can review and/or modify which information is relevant to the environmental constraints. The robotic device can then store the relevant information at 1458.

[0176] In some embodiments, a robotic device can use transport elements (e.g., wheels or tracks) to navigate an environment and learn one or more environmental constraints. For example, a robotic device can navigate a hallway or corridor while, e.g., receiving demonstrations and/or performing skills, and learn that the hallway is crowded during a particular time frame. In some embodiments, a robotic device can learn various environmental constraints and/or behaviors and associate them with different conditions (e.g., time, location, etc.). For example, a robotic device can recognize that a hallway is crowded based on information collected by one or more of its sensors, user input, and/or information derived from sensed information (e.g., during the demonstration and/or performance of a skill). As another example, a robotic device can determine whether the robotic device should respond "hello" or "good night" when a user enters a particular room at various times of day.

[0177] In some embodiments, the robotic device can learn environmental constraints autonomously or interactively. For example, the robotic device can acquire an initial set of environmental constraints by moving through an environment (e.g., navigating a hallway or corridor) at 1458. A user (e.g., a robot supervisor or local user) can review the constraints and determine whether to adjust them at 1459 by directly modifying the constraints and/or by demonstration. For example, some constraints may be provided through interaction with the robotic device (e.g., demonstration), while other constraints (e.g., object locations (e.g., the distance a shelf extends from a wall)) may be provided by input (e.g., to a user interface (e.g., user interface 240)).

[0178] FIG. 21 illustrates an example of a map 2120 (e.g., a map of a building (e.g., a hospital, etc.)) for entry and exit information. The map 2120 may be stored and/or maintained by one or more robotic devices, such as any of the robotic devices described herein. The map 2120 may be similar to the map 1600 shown in FIG. 19. For example, the map 2120 may include one or more layers 2122, such as, for example, a navigational layer, a static layer, a dynamic layer, and a social layer.

[0179] Map 2120 can provide information that the robotic device can use while operating in learn mode 2102 or run mode 2112. For example, a robotic device operating in learn mode 2102 can access map 2120 to obtain information about one or more environmental constraints, one or more objects, and/or one or more social contexts (e.g., similar to state information 331, 1732, object information 340, 1740, one or more environmental constraints 1754, social context 1632, etc.). Such information can enable the robotic device to determine the location of one or more objects, identify one or more characteristics of one or more objects, analyze one or more environmental constraints, select one or more skill models to use, prompt one or more users for one or more inputs, etc., as described herein. Additionally or alternatively, a robotic device operating in execution mode 2112 may access map 2120 to obtain information about one or more environmental constraints, one or more objects, and/or one or more social contexts and use that information to assess the environment, arbitrate between different behaviors or skills, determine which skills to perform, and/or adapt behaviors or skills to suit a particular environment.

[0180] A robotic device operating in learn mode 2102 can also provide information that adds to and/or modifies information in map 2120. For example, the robotic device can incorporate sensed information 2104 (e.g., information collected by one or more sensors of the robotic device) and/or derived information 2106 (e.g., information derived by the robotic device based on, for example, an analysis of sensed information 2104) into map 2120. The robotic device can incorporate such information by adding information to map 2120 and/or adapting existing information in map 2120. Additionally, a robotic device operating in execute mode 2112 can provide information (e.g., sensed information 2114, derived information 2116) that adds to and/or modifies information in map 2120.

[0181] As an example, a robotic device undergoing a demonstration of how to navigate through doorways (e.g., such as the demonstrations shown in FIGS. 10 and 16) can collect information during the demonstration that is provided to one or more layers 2122 of map 2120. The doorways may be specific doorways located in various locations throughout a building, as shown in map 2122, for example. The characteristics and/or properties of the doorways may be derived by the robotic device based on information recorded and/or sensed by various sensors on the robotic device. For example, the robotic device may sense the size of the doorway and determine that it is a narrow doorway. The robotic device can add this information about the doorway to map 2120, such as using one or more semantic labels, as raw or processed sensor information, as derived rules, etc. The robotic device can also use this information to adapt the context layer of the map (e.g., the social or behavioral layer of the map (e.g., the social layer 1630 shown in FIG. 19)). For example, based on the doorway being a narrow doorway, the robotic device may determine to engage in particular behavior(s) and/or action(s), including, for example, passing through the doorway when opened by the user (or one or more other robotic devices) instead of following and/or parallelly passing through the doorway with the user (or one or more other robotic devices), helping the robotic device navigate through the doorway (e.g., by holding the doorway open and/or directing the robotic device to perform various one or more actions), seeking out a particular user familiar with the doorway, etc.

[0182] In some embodiments, map 2120 may be maintained centrally for one or more robotic devices described herein. For example, map 2120 may be stored on a remote computing device (e.g., a server) and centrally hosted for a group of robotic devices operating together, for example, in a hospital. Map 2120 may be updated as information (e.g., sensed information 2104, 2114 and derived information 2106, 2116) is received from the group of robotic devices as they operate in learn mode 2102 and/or perform mode 2112, and provided to each robotic device as needed to perform actions, behaviors, etc. This information may be shared with other robotic devices as they encounter similar environments and/or perform similar skills (e.g., actions, behaviors), as individual robotic devices in the group learn new information so that one or more layers 2122 of map 2120 may be adapted. In some embodiments, a local copy of the map 2120 may be stored on each robotic device, and this local copy may be updated or synchronized (e.g., at predetermined intervals, during rest or downtime) with a centrally maintained copy of the map 2120. By periodically updating the map (e.g., with newly collected information such as sensed information 2104, 2114 and derived information 2106, 2116) and/or sharing the map among multiple robotic devices, each robotic device may have access to a more comprehensive map that provides the robotic device with more accurate information for interacting with and/or performing skills within its surrounding environment.

[0183] In some embodiments, map 2120 may be a mixed-initiative map in which information (e.g., one or more environmental constraints, one or more rules, one or more skills, etc.) may be learned (e.g., during learning 2102 and/or execution 2112) and/or provided by a user (e.g., with user input 2130). For example, the robotic device may construct map 2120 by incorporating information provided directly by a user (e.g., a robot supervisor or local user), information learned through demonstration and/or execution, or information provided interactively by a combination of demonstration and/or execution and user input (e.g., requested by the robotic device in real time or retrospectively in connection with an interaction). For example, a user may indicate to the robotic device that the robotic device should proceed more slowly through a certain area on map 2120 (e.g., a narrow or crowded passage). In some embodiments, the robotic device presents the map to the user, and the user may draw an area on the map and indicate that the robot should proceed more slowly in that area. The robotic device may attempt to navigate the area on the map 2120 multiple times and learn that the area should be avoided at certain times of day (e.g., due to overcrowding). Alternatively or additionally, the robotic device may encounter a new situation (e.g., a partitioned passage) that the robotic device is not accustomed to dealing with autonomously. In such cases, the robotic device may communicate with a user (e.g., a robot supervisor and/or a local user) to obtain more information about the area and/or determine how to navigate around the area. When requesting user input, the robotic device may broadly inquire about the situation and/or provide a list of examples that the user labels (e.g., to enable the robotic device to autonomously derive appropriate behavior). In some embodiments, the robotic device may detect and/or derive information about an area without user input and propose one or more rules associated with the area to the user for confirmation. In some embodiments, the user, upon viewing the proposed rule and/or other information from the robotic device, may modify the rule and/or other information before accepting it. Figure 24, described below, provides a more detailed description of this process.

[0184] 22-25 are flow diagrams illustrating different means by which any of the robotic devices described herein, including, for example, robotic devices 102, 110, 200, 400, etc., may learn one or more skills, one or more environmental constraints, etc. In some embodiments, a robotic device operating according to the methods described herein may be constantly learning (e.g., operating in a learning mode). For example, the robotic device may continuously collect, store, analyze, and/or update information as it navigates through an environment and/or engages in particular behaviors or actions, adapting and/or adding to its library of learned information (e.g., skills, behaviors, environmental constraints, maps, etc.). In other embodiments, the robotic device may switch between operating in a learning mode and an execution mode. While operating in a learning mode 2202, the robotic devices described herein may learn by demonstration 2204, by execution 2206, by exploration 2208, by derivation/user input 2210, and/or any combination thereof.

[0185] In some embodiments, the robotic device can learn by demonstration 2204, for example, as shown and described above with reference to FIGS. 8-10. For example, the robotic device can learn new skills or environmental constraints by demonstrating a skill using the LfD teaching process. The robotic device can analyze and extract information from past demonstrations of a skill in a human environment. As an example, a user can demonstrate to the robotic device how to move a control element (e.g., an arm) of the robotic device in an equipment room with a human and the robotic device. From one or more demonstrations, the robotic device can define one or more environmental constraints for the movements of one or more existing demonstrations that constrain the movements of the robotic device when planning and/or executing future skills. The robotic device can generate movements and build a graph (in which environmental constraints are encoded) using existing demonstrations (e.g., a sequence of one or more keyframes), each of which provides a path through an n-dimensional space of nodes and movements. Subsequently, when the robotic device is presented with a new environment and needs to adapt its skills to the new environment, it can use the constructed graph to efficiently sample from its existing set of demonstrated moves to plan moves for the new environment. By incorporating such environmental constraint learning into the initial skill demonstration, the robotic device can quickly adapt to the new environment without requiring new environmental constraints to be defined. In the new environment, the robotic device can scan the environment to obtain additional information about the environment and/or one or more objects within the environment (e.g., people, furniture, doors, etc.), and then use the existing skill model to plan a motion trajectory through the new environment without significantly deviating from its existing set of demonstrated moves.

[0186] In some embodiments, the robotic device can learn through demonstration 2204 and user input 2210. For example, the robotic device can engage in interactive learning with a user, e.g., using an interactive learning template. As described above, with interactive learning, an initial set of robotic device and/or skill models is provided. The initial set of models may be developed by other robotic devices in a factory setting and/or by the robotic device itself in one or more settings. While operating in a particular environment, the robotic device can adapt or specialize this initial set of skills through interactive learning sessions in which the robotic device can autonomously perform certain portions of the skills while leaving others to user demonstration. Further details of such an interactive learning process are described with reference to FIG. 16.

[0187] In some embodiments, the robotic device can learn skills as it executes 2206 them. For example, the robotic device can collect, store, analyze, and/or update information as the robotic device executes behaviors, actions, etc. (e.g., at 774 in FIG. 11 and 1304, 1305, and 1307 in FIG. 13). In some embodiments, the robotic device can execute a skill that requires the robotic device to navigate a hallway several times a day. The robotic device can learn that a hallway has higher traffic during certain times of day. The robotic device can then adapt its behavior, for example, to avoid traversing that hallway during those high traffic times (e.g., by taking an alternate route and/or by waiting to traverse that hallway at a time with less traffic).

23 is a flow diagram illustrating an exemplary method for learning by exploration. The robotic device may scan an environment and collect information about the environment and/or objects within the environment at 2302. Optionally, based on the information collected by the robotic device, the robotic device may determine to explore the environment by executing a skill. To execute the skill, the robotic device may select an existing skill model at 2304, plan the execution of the skill using the skill model at 2306, and execute the skill at 2308. For example, the robotic device may scan the environment and identify a doorway. The robotic device may determine the execution of a skill that includes steps of opening the door and moving through the doorway. At 2304, the robotic device may select an existing skill model previously provided and/or learned to navigate through the doorway. At 2306, the robotic device may generate a plan to open the door and move through the doorway using the existing skill (e.g., following a process similar to the process described with reference to FIG. 11 ). The robotic device can then execute the skill according to the generated plan at 2308. While executing the skill, the robotic device can collect information about the environment and/or the execution of the skill and compare that information with information collected during previous interactions with the doorway at 2310. Based on this comparison, the robotic device can evaluate whether its interaction with the doorway was successful or unsuccessful (e.g., based on success criteria such as those described above). The robotic device can optionally generate a skill model specific to the doorway at 2312 based on its execution of the skill.

[0189] In some embodiments, a robotic device may begin performing a skill and determine that user input is required to further complete the skill. For example, as described above with reference to FIG. 16, when a robotic device begins performing a skill and reaches a portion of the skill that the robotic device cannot determine how to perform, it may request user input to proceed with the skill.

[0190] In some embodiments, the robotic device can execute a skill and determine that the execution failed. For example, the robotic device may plan and/or execute a movement through a doorway, but detect that it was unable to move through the doorway (e.g., due to a failed attempt to open the door or navigate through a narrow doorway). The robotic device may decide to scan the environment again (2314: YES) and re-plan and re-execute the skill at 2306-2308. In some embodiments, the robotic device may select a different skill at 2304 (e.g., a skill more specific to the narrow doorway) and then re-plan and re-execute based on the different skill. The robotic device may continue to learn by exploration by resensing, re-planning, and re-executing the skill until the robotic device determines that it has met a certain objective (e.g., successfully executed the skill a predetermined number of times or in a predetermined number of ways, learned sufficient information about the environment, etc.). At 2316, the robotic device may store information it has collected about the environment and/or objects in the environment, and optionally, models it has adapted or generated based on the performance of skills in the environment.

[0191] In some embodiments, the robotic device can engage in learning by exploration 2208 using user input 2210. For example, the robotic device can perform one or more skills in a particular environment to explore and/or learn more information about the environment. While the robotic device is exploring, a user (e.g., a remote supervisor or a local user) can provide input to the robotic device based on information the user perceives and/or learns about the environment (e.g., directly by being in the environment and/or through the robotic device). The user can then provide one or more inputs to the robotic device that further guide its exploration of the environment. For example, a robotic device that encounters a doorway can plan one or more actions to navigate through the doorway. Upon executing those actions, the robotic device may fail to navigate through the doorway. The user, upon observing the failed attempt by the robotic device, can provide one or more inputs to the robotic device to guide the robotic device in a second attempt to navigate through the doorway. For example, a user may determine that a doorway is a narrow doorway and indicate to the robotic device that the robotic device should first ask a nearby user or robotic device to open the door before navigating through the doorway. Upon receiving the user input, the robotic device may ask the user to open the door before navigating through the doorway.

[0192] In some embodiments, the robotic device can learn through user input 2210 by receiving information from the user about the environment and/or objects in the environment, or by receiving an initial set of models, rules, etc. In some embodiments, the robotic device may initially learn a skill by demonstration (e.g., as described above with reference to FIG. 10 ) and learn to adapt the skill based on user input. For example, the robotic device can learn the skill of moving equipment from a first location to a second location and then dropping off the equipment at the second location. During user demonstration of the skill, the user can indicate to the robotic device that dropping off the equipment at the second location requires specific information specific to the environment or situation. The user can later provide input to the robotic device specifying specific information for the robotic device to observe while dropping off the equipment for different pieces of equipment and/or different locations. For example, the user can provide information regarding a tag or marker that identifies where the robotic device should drop off the equipment.

[0193] In some embodiments, the robotic device can learn behaviors or rules (e.g., one or more arbitration algorithms 1858, as described above) based on stored information and/or user input 2210. FIG. 24 is a flowchart of an exemplary method of robot learning of behaviors and/or rules. The robotic device can analyze stored information (e.g., one or more maps, one or more models, one or more user inputs) at 2402. The robotic device can derive behaviors and/or rules based on the analysis of the stored information at 2404. For example, a user (e.g., a robot supervisor or local user) can define a rule for the robotic device to be more interactive when more humans are near the robotic device. The robotic device may determine, based on the analysis of the stored information, that more humans are near it during certain times of the day (e.g., around lunchtime). Thus, the robotic device may further derive that it should be more interactive during that time of day. In some embodiments, the robotic device can automatically implement this behavior as a new rule. Alternatively, the robotic device can suggest 2406 a new rule to the user to be more interactive at that time of day. The robotic device can receive 2408 user input in response to the rule suggestion. For example, the user can view the rule on a local and/or remote display of the robotic device and choose to accept or reject it. Based on the user input, the robotic device can modify 2410 its behavior and/or rule. For example, if the user indicates that the rule is acceptable, the robotic device can save the new rule and implement it in the future. Alternatively, if the user indicates that the rule is acceptable but with further tweaking or modification (e.g., to be more interactive only in the cafeteria around lunchtime), the robotic device can adapt the rule and save the adapted rule for future implementation.

[0194] In some embodiments, a robotic device can learn skills and/or environmental constraints by adapting existing skills and/or environmental constraints based on user input. FIG. 25 is a flow diagram of an example method for adapting skills and/or environmental constraints based on user input. The robotic device can receive user input at 2502. The robotic device can associate the user input with one or more skills and/or environmental constraints at 2504. Such association may be based, for example, on further input by the user (e.g., the user specifying the skill and/or environmental constraint to which the input pertains) and/or derivation by the robotic device (e.g., the robotic device determining that such input is relevant to a particular skill or environmental constraint based on the input and/or one or more rules, attributes, etc. associated with the particular skill or environmental constraint). For example, the user may provide the robotic device with new information for identifying an object, such as a QR code, barcode, tag, etc. The user may specify that the new information pertains to a particular object, and the robotic device may associate the new information with the object. The robotic device may further associate the new information about the object with one or more skills (e.g., a grasping skill or a movement skill) involved in interacting with the object. The robotic device may optionally modify and/or create the skill or environmental constraint based on the input at 2506-2508.

[0195] In some embodiments, the robotic device may allow a user to retroactively label information related to skills and/or environmental constraints. For example, the robot may present skills or environmental constraints to the user, for example, using a user interface (e.g., user interface 240) and/or a remote interface. The user may choose to add, modify, and/or remove labels associated with the skills or environmental constraints, and such input may be received at the robotic device at 2502. The robotic device may then associate the input with the skills or environmental constraints at 2504 and may modify and/or create new skills or environmental constraints based on the input at 2506-2508.

Example
[0196] It will be appreciated that the present disclosure may include any one up to all of the following examples.

[0197] Example 1: A device having a memory, a processor, a control element, and a set of sensors, the processor being operatively coupled to the memory, the control element, and the set of sensors and configured to: obtain a representation of an environment using a subset of sensors from the set of sensors; identify a plurality of markers within the representation of the environment, where each marker from the plurality of markers is associated with a physical object from a plurality of physical objects located in the environment; present information indicative of a position of each marker from the plurality of markers within the representation of the environment; receive a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; obtain, for each position from a plurality of positions associated with movement of the control element within the environment, sensory information associated with the control element, where movement of the control element is associated with a physical interaction between the control element and the set of physical objects; and generate, based on the sensory information, a model configured to specify a behavior of the control element to perform the physical interaction between the control element and the set of physical objects.

[0198] Example 2: The apparatus of Example 1, wherein the set of physical objects includes a person.

[0199] Example 3: The apparatus of Examples 1 or 2, wherein the manipulating element includes an end effector configured to engage a subset of physical objects from the set of physical objects.

[0200] Example 4: The device of any one of Examples 1 to 3, wherein the subset of sensors is a subset of a first sensor, and the processor is configured to acquire sensory information using a second subset of sensors from the set of sensors, the second subset of sensors being different from the first subset of sensors.

[0201] Example 5: The apparatus of any one of Examples 1 to 4, wherein the operating element includes a plurality of movable components connected by a plurality of joints, and the set of sensors includes at least one of a sensor configured to measure a force acting on a joint from the plurality of joints, or a sensor configured to detect engagement of a movable component from the plurality of movable components with a physical object from the set of physical objects.

[0202] Example 6: The device of Example 5, wherein the set of sensors further includes a sensor configured to measure the position of a joint or moving component relative to a portion of the device.

[0203] Example 7: The apparatus of Example 5, wherein the set of sensors further includes at least one of a light sensor, a temperature sensor, an audio capture device, and a camera.

[0204] Example 8: The apparatus of Example 1, wherein the manipulating element includes (i) a plurality of joints and (ii) an end effector configured to move a physical object from the set of physical objects, and the set of sensors includes a sensor configured to measure a force on the end effector or at least one of the joints from the plurality of joints coupled to the end effector when the end effector moves the physical object.

[0205] Example 9: The apparatus of any one of Examples 1 to 8, wherein the sensory information includes sensor data associated with a set of features, the processor is further configured to receive a selection of a first subset of features from the set of features, and the processor is configured to generate the model based on the sensor data associated with the first subset of features and not based on sensor data associated with a second subset of features from the set of features that is not included in the first set of features.

[0206] Example 10: The apparatus of Example 9, wherein the processor is further configured to prompt the user to select at least one feature from the set of features in response to a selection made by the user, such that the processor receives a selection of the first subset of features.

[0207] Example 11: The device described in any one of Examples 1 to 10, wherein the plurality of markers are fiducial markers and the representation of the environment is a visual representation of the environment.

[0208] Example 12: The device of any one of Examples 1 to 11, wherein the processor is further configured to store a model and information associated with the model in memory, the information associated with the model including (i) a set of markers and (ii) sensory information.

[0209] Example 13: The device described in Example 1, wherein the operating element includes a plurality of joints, and the sensory information includes, for each position from a plurality of positions associated with movement of the operating element, information indicating a current state of each joint from the plurality of joints.

[0210] Example 14: The device of any one of Examples 1 to 13, wherein the processor is further configured to prompt the user to select at least one marker from the plurality of markers after the presentation, such that in response to a selection made by the user, the processor receives a selection of the set of markers.

[0211] Example 15: The apparatus of any one of Examples 1 to 14, wherein the processor is configured to obtain a representation of the environment by sensing or scanning an area of interest within the environment using a set of sensors.

[0212] Example 16: A method comprising: acquiring a representation of an environment using a set of sensors; identifying a plurality of markers within the representation of the environment, each marker from the plurality of markers associated with a physical object from a plurality of physical objects located in the environment; presenting information indicative of a position of each marker from the plurality of markers within the representation of the environment; and after the presenting, receiving a selection of a set of markers from the plurality of markers associated with a set of physical objects from the plurality of physical objects; acquiring, for each position from a plurality of positions associated with movement of a control element within the environment, sensory information associated with a control element, the movement of the control element being associated with a physical interaction between the control element and the set of physical objects; and generating, based on the sensory information, a model configured to define a behavior of the control element to perform a physical interaction between the control element and the set of physical objects.

[0213] Example 17: The method of Example 16, wherein the sensory information includes sensor data associated with a set of features, the method further comprising receiving a selection of a first subset of features from the set of features, and wherein generating includes generating the model based on sensor data associated with the first subset of features and not based on sensor data associated with a second subset of features from the set of features that is not included in the first set of features.

[0214] Example 18: The method of Example 17, further comprising prompting the user to select at least one feature from the set of features, such that, in response to a selection made by the user, a selection of the first subset of features is received.

[0215] Example 19: The method of any one of Examples 16-18, wherein the plurality of markers are fiducial markers and the representation of the environment is a visual representation of the environment.

[0216] Example 20: The method of any one of Examples 16 to 19, wherein the operating element includes a plurality of joints, and the sensory information includes, for each position from a plurality of positions associated with movement of the operating element, information indicating a current state of each joint from the plurality of joints.

[0217] Example 21: The method of any one of Examples 16-20, further comprising, after the presentation, prompting the user to select at least one marker from the plurality of markers, such that a selection of the set of markers is received in response to a selection made by the user.

[0218] Example 22: The method of any one of Examples 16-21, wherein obtaining a representation of the environment includes sensing or scanning an area of interest within the environment using a set of sensors.

[0219] Example 23: A non-transitory processor-readable medium storing code representing instructions to be executed by a processor, the code causing the processor to: obtain a representation of an environment using a set of sensors; identify a plurality of markers within the representation of the environment, each marker from the plurality of markers associated with a physical object from a plurality of physical objects located in the environment; present information indicative of a position of each marker from the plurality of markers within the representation of the environment; identify a model associated with performing a physical interaction between a manipulative element and the set of physical objects in response to receiving a selection of a set of markers from the plurality of markers associated with the set of physical objects from the plurality of physical objects, the manipulative element including a plurality of joints and an end effector; and generate a trajectory for the manipulative element defining a movement of the plurality of joints and the end effector associated with performing the physical interaction using the model.

[0220] Example 24: The non-transitory processor-readable medium of Example 23, wherein the code for causing the processor to identify a model associated with performing a physical interaction comprises code for causing the processor to prompt a user to identify the model.

[0221] Example 25: The non-transitory processor-readable medium of Example 23 or 24, wherein the code for causing the processor to identify a model associated with performing a physical interaction comprises code for causing the processor to identify the model based on a selection of a set of markers.

[0222] Example 26: The non-transitory processor-readable medium of any one of Examples 23-25, further comprising code that causes a processor to: display a trajectory of the manipulative element to a user within a representation of the environment; receive input from the user after the display; and, in response to the input indicating approval of the trajectory of the manipulative element, perform movement of a plurality of joints and an end effector to perform a physical interaction.

[0223] Example 27: The non-transitory processor-readable medium of any one of Examples 23-26, wherein the trajectory is a first trajectory, and the non-transitory processor-readable medium further comprises code that causes a processor to: generate a modified model by modifying a set of parameters associated with the model in response to an input that does not indicate approval of the trajectory of the manipulative element; and generate a second trajectory of the manipulative element using the modified model.

[0224] Example 28: The non-transitory processor-readable medium of any one of Examples 23-26, wherein the trajectory is a first trajectory, the model is a first model, and the non-transitory processor-readable medium further comprises code that causes a processor to, in response to an input that does not indicate approval of the trajectory of the manipulative element, generate a second model based on sensor data associated with a set of features different from the set of features used to generate the first model, and generate a second trajectory of the manipulative element using the second model.

[0225] Example 29: The non-transitory processor-readable medium of any one of Examples 23 to 28, further comprising code that causes a processor to: perform movements of a plurality of joints and an end effector to perform a physical interaction; acquire sensory information related to the performance of the physical interaction; and determine, based on the sensory information, whether the performance of the physical interaction meets pre-defined and/or learned success criteria.

[0226] Example 30: The non-transitory processor-readable medium of Example 29, further comprising code that causes a processor to: generate a signal indicating that the physical interaction was successful in response to a determination that the performance of the physical interaction meets pre-defined and/or learned success criteria; and generate a modified model by modifying the model based on the sensory information in response to a determination that the performance of the physical interaction does not meet the pre-defined and/or learned success criteria; and generate a second trajectory for the manipulative element using the modified model.

[0227] Example 31: The non-transitory processor-readable medium of any one of Examples 23-30, wherein the model is associated with (i) a set of stored markers; (ii) sensory information indicative of at least one of a position or orientation of the manipulative element at points along a stored trajectory of the manipulative element associated with the set of stored markers; and (iii) sensory information indicative of a configuration of a plurality of joints at the points along the stored trajectory, and the code for causing the processor to generate a trajectory for the manipulative element includes code for causing the processor to: calculate a transformation function between the set of markers and the stored set of markers; transform, for each point, at least one of a position or orientation of the manipulative element using the transformation function; determine, for each point, a planned configuration of the plurality of joints based on the configuration of the plurality of joints at the point along the stored trajectory; and determine, for each point, a portion of the trajectory between the point and successive points based on the planned configuration of the plurality of joints relative to the point.

[0228] Example 32: The non-transitory processor-readable medium of any one of Examples 23-30, wherein the model is associated with (i) a set of stored markers and (ii) sensory information indicative of at least one of a position or orientation of the manipulative element at points along the stored trajectory of the manipulative element associated with the set of stored markers, and the code for causing the processor to generate a trajectory for the manipulative element includes code for causing the processor to: calculate a transformation function between the set of markers and the set of stored markers; transform, for each point, at least one of the position or orientation of the manipulative element using the transformation function; determine, for each point, a planned configuration of a plurality of joints; and determine, for each point, a portion of the trajectory between the point and successive points based on the planned configuration of the plurality of joints relative to the point.

[0229] Example 33: The non-transitory processor-readable medium of any one of claims 23 to 32, further comprising code that causes a processor to: determine whether to change a location of the manipulative element based on a distance between a first location of the manipulative element and a location of a physical object from a set of physical objects; and, in response to determining to change the location of the manipulative element, move the manipulative element from the first location to a second location that is closer to the location of the physical object; and the code that causes the processor to generate a trajectory for the manipulative element includes code that causes the processor to generate the trajectory based on the location of the physical object relative to the second location of the manipulative element after the movement.

[0230] While various inventive embodiments have been described and illustrated herein, those skilled in the art will readily envision numerous other means and/or structures for performing the functions described herein and/or obtaining one or more of the results and/or advantages described herein, and each such variation and/or modification is deemed to be within the scope of the inventive embodiments described herein. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are intended to be exemplary, and that the actual parameters, dimensions, materials, and/or configurations will depend on the specific application or applications for which the teachings of the present invention are used. Those skilled in the art will recognize, or will be able to ascertain using no more than routine experimentation, many equivalents to the specific inventive embodiments described herein. Accordingly, it is to be understood that the above-described embodiments are presented by way of example only, and that, within the scope of the appended claims and their equivalents, inventive embodiments may be practiced otherwise than as specifically described and claimed. The inventive embodiments of the present disclosure are directed to each individual feature, system, product, material, kit, and/or method described herein. Furthermore, any combination of two or more such features, systems, products, materials, kits, and/or methods is within the inventive scope of the present disclosure, provided those features, systems, products, materials, kits, and/or methods are not mutually inconsistent.

[0231] Also, various inventive concepts may be embodied as one or more methods, examples of which have been provided. Acts performed as part of a method may be ordered in any suitable manner. Accordingly, embodiments may be constructed in which acts are performed in an order different from that illustrated, and these embodiments may include performing some acts simultaneously even though, in the illustrative embodiment, acts are shown as sequential.

[0232] Some embodiments and/or methods described herein may be performed by other software (executed on hardware), hardware, or a combination thereof. Hardware modules may include, for example, general-purpose processors, field programmable gate arrays (FPGAs), and/or application-specific integrated circuits (ASICs). Software modules (executed on hardware) may be expressed in various software languages (e.g., computer code), including C, C++, Java™, Ruby, Visual Basic™, and/or other object-oriented, procedural, or other programming languages and development tools. Examples of computer code include, but are not limited to, microcode or microinstructions, machine instructions such as those produced by a compiler, code used to create web services, and files containing high-level instructions executed by a computer using an interpreter. For example, embodiments may be implemented using imperative programming languages (e.g., C, Fortran, etc.), functional programming languages (e.g., Haskell, Erlang, etc.), logic programming languages (e.g., Prolog), object-oriented programming languages (e.g., Java, C++, etc.), or other suitable programming languages and/or development tools. Additional examples of computer code include, but are not limited to, control signals, encryption code, and compression code.

[0233] While terms such as "first," "second," and "third" may be used herein to describe various elements, it will be understood that these elements are not limited by these terms. These terms are used only to distinguish one element from another. Thus, a first element described herein could be termed a second element without departing from the teachings of the present disclosure.

[0234] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising" or "includes" and/or "including" as used herein specify the presence of stated features, regions, integers, steps, operations, elements, and/or components, but do not exclude the presence or addition of one or more other features, regions, integers, steps, operations, elements, components, and/or groups thereof.

Claims (18)

an operating element including a plurality of joints ;
a conveying element configured to move along a surface;
A set of sensors and
Memory and
a processor operatively coupled to the memory, the operating element, the transport element, and the set of sensors,
acquiring, using the set of sensors, information about an environment, which is an unstructured environment, and a current state of the robotic device , wherein the current state of the robotic device is a current state of the transport element and the manipulating element, and the information acquired using the set of sensors includes information about positions of physical objects in the environment and information about configurations of the plurality of joints of the manipulating element;
determining, based on the information of the environment, that user input is required; and
In response to determining that the user input is required, requesting the user input from a user;
selecting, using an arbitration algorithm, a skill to execute from a plurality of skills based on the information of the environment, the current state of the robotic device , and the user input, wherein the execution of the skill includes a physical interaction between the operational element and the physical object in the environment ;
generating a plan for executing at least a portion of the skill based on at least one of the information of the environment, the current state of the robotic device, and the user input using a model for executing at least a portion of the skill, the plan including one or more movements of at least one of the manipulation element or the transport element;
moving the at least one of the operating element or the conveying element based on the plan;
a processor configured to:
A robotic device comprising: the portion of the skill is a first portion, and the processor:
determining that a second portion of the skills requires specialization for the environment;
In response to determining that the second portion of the skill requires specialization, requesting second user input from a user;
In response to receiving the second user input, generating a plan to perform the second portion of the skill based on the second user input;
moving the at least one of the operating element and the transport element based on the plan to perform the second portion of the skill;
The robotic device of claim 1 , further configured to: the portion of the skill is a first portion, and the processor:
determining that a second portion of the skills requires specialization for the environment;
In response to determining that the second portion of the skill requires specialization, requesting a performance from the user;
acquiring, for each location from a plurality of locations associated with the demonstration, information collected by the set of sensors and relating to at least one of the environment, the operating element, or the transport element;
generating a specialized model for performing the skill in the environment using the model and the information collected by the set of sensors ; and
The robotic device of claim 1 , further configured to: the processor:
acquiring information of at least one of the environment, the operation element, or the transport element using the set of sensors when the at least one of the operation element or the transport element is moving based on the plan;
determining, based on the information of the at least one of the environment, the operating element, or the transport element, that the operation of the at least one of the operating element or the transport element failed to meet one or more success criteria;
modifying the plan to perform at least a portion of the skill in response to determining that the action failed to meet the one or more success criteria;
moving the at least one of the operating element or the conveying element based on the correction plan;
The robotic device of claim 1 , further configured to: the processor:
acquiring information of at least one of the environment, the operation element, or the transport element using the set of sensors when the at least one of the operation element or the transport element is moving based on the plan;
updating a map of the environment based on the information of the at least one of the environment, the operating element, or the transport element;
The robotic device of claim 1 , further configured to: the map of the environment includes at least a semantic layer and a context layer, and the processor:
storing information related to one or more objects in the environment in the semantic layer;
determining one or more behaviors to perform near the one or more objects based on the information related to the one or more objects;
storing the one or more behaviors in the context layer;
6. The robotic device of claim 5, configured to update the map of the environment by: the processor:
displaying the plan to execute the at least some of the skills to a user; and
receiving, after the display, a third user input indicating a confirmation to initiate performance of the at least part of the skill;
and further configured to:
The robotic device of claim 1 , wherein the processor is configured to move the at least one of the manipulation element or the transport element in response to receiving the third user input. the processor:
displaying the plan to execute the at least some of the skills to a user; and
receiving a fourth user input after said display indicating a change to said plan;
modifying the plan based on the fourth user input; and
and further configured to:
The robotic device of claim 1 , wherein the processor is configured to move the at least one of the manipulation element or the transport element based on the plan after modifying the plan. the operating element includes a plurality of movable components connected by a plurality of joints;
the set of sensors
a sensor configured to measure a torque acting on a joint from the plurality of joints; or a sensor configured to detect engagement of a moving component from the plurality of moving components with a physical object from the set of physical objects;
The robotic device of claim 1 , comprising at least one of: The robotic device of claim 1, wherein the manipulative element includes an end effector configured to engage one or more objects in the environment. the processor:
presenting the information of the environment to a user;
receiving a fifth user input, the fifth user input including an instruction to perform a social behavior, the portion of the skill involving an interaction with at least one human;
identifying the at least one human in the environment;
performing the social behavior in proximity to the at least one human; and
The robotic device of claim 1 , further configured to: the processor:
acquiring information about another portion of the environment that is different from the portion of the environment using the set of sensors;
In response to determining that user input is required to perform another portion of the skill in the other portion of the environment, prompting a user to provide a sixth user input related to performing the other portion of the skill;
generating another plan to execute the other portion of the skill based on the sixth user input; and
The robotic device of claim 1 , further configured to: the processor:
moving the manipulation element and the transport element based on the plan to perform the portion of the skill while acquiring additional information about the environment;
suspending the execution of the skill in response to identifying a change in the environment based on the additional information of the environment;
prompting the user to provide feedback based on the changes in the environment; and
generating another plan to execute another portion of the skill based on the feedback; and
The robotic device of claim 1 , further configured to: acquiring, using a set of sensors of a robotic device, information of an environment, the environment being an unstructured environment, and a current state of the robotic device, the current state of the robotic device being a current state of a transport element and a manipulation element of the robotic device, the information acquired using the set of sensors including information of positions of physical objects in the environment and information of configurations of the plurality of joints of the manipulation element;
determining, based on the information of the environment, that user input is required; and
In response to determining that the user input is required, requesting the user input from a user;
selecting, using an arbitration algorithm, a skill to execute from a plurality of skills based on the information of the environment, the current state of the robotic device , and the user input, wherein the execution of the skill includes a physical interaction between the operational element and the physical object in the environment ;
generating a plan for executing the portion of the skill based on at least one of the information of the environment, the current state of the robotic device, and the user input using a model for executing the portion of the skill, the plan including one or more movements of at least one of a manipulation element or a transport element of the robotic device;
moving the at least one of the operating element or the conveying element based on the plan;
A method comprising: acquiring information of at least one of the environment, the operating element, or the transport element using the set of sensors of the robotic device when the at least one of the operating element or the transport element is moving based on the plan;
updating a map of the environment stored in the memory based on the information of the at least one of the environment, the operating element, or the transport element;
sending information indicative of the update to the map of the environment to a computing device;
receiving information from the computing device indicative of updates to the map of the environment received by the computing device from a set of robotic devices other than the robotic device;
The method of claim 14 further comprising: acquiring information about another portion of the environment that is different from the portion of the environment using the set of sensors;
In response to determining that user input is required to perform another portion of the skill in the other portion of the environment, prompting a user to provide a second user input related to performing the other portion of the skill;
generating another plan to execute the other portion of the skill based on the second user input;
The method of claim 14 further comprising: moving the manipulation element and the transport element based on the plan to perform the portion of the skill while acquiring additional information about the environment;
suspending the execution of the skill in response to identifying a change in the environment based on the additional information of the environment;
prompting the user to provide feedback based on the changes in the environment; and
generating another plan to execute another portion of the skill based on the feedback; and
The method of claim 14 further comprising: acquiring information of at least one of the environment, the operation element, or the transport element using the set of sensors when the at least one of the operation element or the transport element is moving based on the plan;
determining, based on the information of the at least one of the environment, the operating element, or the transport element, that the operation of the at least one of the operating element or the transport element failed to meet one or more success criteria;
modifying the plan to perform at least a portion of the skill in response to determining that the action failed to meet the one or more success criteria;
moving the at least one of the operating element or the conveying element based on the correction plan;
16. The method of claim 15, further comprising: