JP7769126B2 - Method and system for controlling robots using reinforcement learning-based algorithms and path planning-based algorithms, and buildings equipped with robots - Google Patents

Description

The following description relates to methods and systems for controlling the movement of robots using reinforcement learning-based algorithms and path planning-based algorithms, and to buildings in which such robots are deployed.

Autonomous robots are robots that can independently assess their surroundings, sense obstacles, and use their wheels and legs to navigate to their destination along the optimal route. They are being developed in a variety of fields, including autonomous vehicles, logistics, hotel services, and robot vacuum cleaners.

When robots are deployed into spaces such as buildings to provide services, they must be controlled to appropriately avoid predictable and unpredictable obstacles within the space, and in some cases may be required to navigate narrow areas such as corridors and hallways within the building.

It is important for such robots to be efficiently controlled so as to avoid collisions and interference with other robots and obstacles, while also avoiding unnecessary detours or unnecessary movements such as traveling long distances.

Korean Patent Publication No. 10-2005-0024840 is a technology relating to a path planning method for an autonomous mobile robot, and discloses a method for planning an optimal path for a mobile robot operating autonomously in a home or office to safely and quickly move to a target point while avoiding obstacles.

The information provided above is for ease of understanding only, and the present invention may include material that does not form part of the prior art, or may not include material that the prior art would suggest to one skilled in the art.

The objective of this invention is to provide a robot control method that controls the movement of a robot to a destination based on a first algorithm learned in advance through reinforcement learning, and determines whether the robot satisfies predetermined obstacle avoidance conditions based on a second algorithm for controlling the robot's movement according to a spatial path plan while the robot is moving to the destination. Depending on whether the avoidance conditions are met, the method either controls the robot's movement using the first algorithm or stops the robot's movement.

The objective of this method is to provide an efficient method for controlling a robot by using a path planning-based algorithm to determine whether the robot needs to deviate from a predetermined avoidance range to avoid an obstacle identified while the robot is moving to a destination, and by stopping the robot based on the determination result.

In one aspect, a robot control method executed by a robot moving within a space or a robot control system for controlling a robot includes the steps of: controlling the movement of the robot to a destination based on a first algorithm previously learned by reinforcement learning for controlling the movement of the robot; determining whether the robot satisfies a predetermined obstacle avoidance condition while moving to the destination based on a second algorithm for controlling the movement of the robot according to a path plan based on a map constructed for the space; and, if the avoidance condition is satisfied, controlling the movement of the robot to move to the destination in accordance with control by the first algorithm; and, if the avoidance condition is not satisfied, controlling the robot to stop moving.

The first algorithm and the second algorithm may be executed in parallel in the robot or the robot control system to control the movement of the robot.

The robot is a service robot that provides a service within the space, and the destination may be the robot's final destination for providing the service, or a stopover point through which the robot passes on its way to the final destination.

The first algorithm may be configured to control the robot so that the robot moves to the destination while avoiding obstacles during movement, and the second algorithm may be configured to control the movement of the robot according to the path plan based on a path to the destination generated based on a map constructed for the space.

The map constructed for the space may be an occupancy grid map for the space.

The determining step may include determining, based on the path plan, whether the robot can avoid an obstacle identified during its movement to the destination without deviating from a predetermined avoidance range and move to the destination; if the robot can avoid the obstacle without deviating from the avoidance range and move to the destination, the robot may be controlled to move to the destination in accordance with control by the first algorithm; and if the robot cannot avoid the obstacle and move to the destination without deviating from the avoidance range, the robot may be controlled to stop in front of the obstacle.

The determining step may determine that the robot cannot avoid the obstacle without departing from the avoidance range if the robot must move a predetermined first distance or more to avoid the obstacle according to the path plan based on a path to the destination generated based on a map constructed for the space.

The determining step may involve determining, based on a map constructed for the space, whether the robot can move to the destination while avoiding the obstacle without deviating by a predetermined second distance or more from a straight line from the robot's current location or starting location to the destination, and controlling the robot to stop in front of the obstacle if the robot cannot move to the destination while avoiding the obstacle without deviating by the second distance or more.

If the obstacle is small enough that the robot can avoid it without detouring to another path, the robot may be controlled according to control by the first algorithm to avoid the obstacle and move to the destination without detouring to another path or stopping in front of the obstacle.

The determining step includes determining, based on the path plan, whether the robot can move to the destination by avoiding an obstacle identified during the robot's movement to the destination without entering a predetermined prohibited area. If the robot can move to the destination by avoiding the obstacle without entering the prohibited area, the robot may be controlled to move to the destination in accordance with control by the first algorithm. If the robot cannot move to the destination by avoiding the obstacle without entering the prohibited area, the robot may be controlled to stop in front of the obstacle.

The determining step may include a step of determining, based on the path plan, whether the robot can avoid an obstacle identified during the robot's movement to the destination and move to the destination without moving backward, and if it is determined that the robot needs to move backward to avoid the obstacle, the robot may be controlled to stop in front of the obstacle.

Depending on the result of the determination, if the obstacle in the path along which the robot travels to the destination is small enough to be avoided without detouring into another path, the robot may be controlled to avoid the obstacle and travel to the destination without detouring into another path according to control by the first algorithm, and if another robot is identified as the obstacle ahead of the robot in the path, the robot may be controlled to wait behind the other robot without overtaking the other robot.

The passage may be an area within the space where the robots are required to be aligned, or an area too narrow for two or more robots to travel side by side.

In another aspect, a computer system for a robot is provided, comprising at least one processor configured to execute computer-readable instructions, wherein the at least one processor controls the movement of the robot to a destination based on a first algorithm previously learned by reinforcement learning for controlling the movement of the robot, determines whether the robot satisfies a predetermined obstacle avoidance condition based on a second algorithm for controlling the movement of the robot according to a path plan based on a map constructed for the space while the robot is moving to the destination, and controls the movement of the robot to move to the destination in accordance with control by the first algorithm if the avoidance condition is satisfied, and controls the robot to stop moving if the avoidance condition is not satisfied.

In another aspect, a building is provided in which at least one robot that moves within the building is disposed, the building including at least one processor that executes computer-readable instructions, and the robot's movement within the building is controlled by a computer system that is included in the robot or a server, the computer system including at least one processor that executes computer-readable instructions, the at least one processor controlling the movement of the robot to a destination based on a first algorithm that has been learned in advance by reinforcement learning to control the movement of the robot, determining whether the robot satisfies a predetermined obstacle avoidance condition during the robot's movement to the destination based on a second algorithm for controlling the robot's movement according to a path plan based on a map constructed for the space, and if the avoidance condition is satisfied, controlling the movement of the robot to move to the destination according to control by the first algorithm, and if the avoidance condition is not satisfied, controlling the robot to stop its movement.

By controlling the robot's movement to its destination using a reinforcement learning-based algorithm and a path planning-based algorithm executed in parallel, it is possible to limit the range of obstacle avoidance that could not be adjusted using the reinforcement learning-based algorithm. This makes it possible to minimize inefficient movement of the robot, such as when the robot deviates significantly from the path to the destination (e.g., the global path) when traveling through narrow passages or encountering obstacles.

Furthermore, when performing actions required to provide a service, such as when robots need to line up in a narrow passage, the robot can be controlled to position itself at an appropriate distance from the preceding robot while avoiding minor obstacles.

FIG. 1 illustrates a method for controlling a robot's movement to a destination using reinforcement learning-based and path planning-based algorithms in one embodiment. FIG. 1 is a block diagram illustrating a robot providing services within a space in one embodiment. FIG. 1 is a block diagram illustrating a robot control system for controlling a robot according to an embodiment. FIG. 1 is a block diagram illustrating a robot control system for controlling a robot according to an embodiment. FIG. 1 is a block diagram illustrating a processor of a computer system for a robot that uses reinforcement learning-based algorithms and path planning-based algorithms to control movement of the robot to a destination, in one embodiment. 1 is a flowchart illustrating a method for controlling movement of a robot to a destination using reinforcement learning-based algorithms and path planning-based algorithms in one embodiment. 10 is a flowchart illustrating a method for determining whether a robot satisfies a predetermined obstacle avoidance condition when controlling the movement of the robot to a destination in one example. FIG. 10 is a diagram illustrating a method for controlling the movement of a robot to a destination based on whether the robot deviates from a predetermined avoidance range for an obstacle, in one example. FIG. 1 illustrates how, in one example, a robot's movement to a destination is controlled depending on whether the robot must enter a predetermined prohibited area to avoid an obstacle. FIG. 10 is a diagram illustrating a method for controlling the movement of a robot to a destination in one example, depending on whether the robot deviates from a predetermined avoidance range relative to the path to the destination in order to avoid an obstacle. FIG. 1 illustrates a method for controlling robot movement in an area where robot alignment is required, in one example. FIG. 1 illustrates an occupancy grid map as a map for space used by a path planning-based algorithm in one example.

The following describes the embodiments in detail with reference to the attached drawings.

Figure 1 illustrates a method for controlling a robot's movement to a destination using a reinforcement learning-based algorithm and a path planning-based algorithm in one embodiment.

Figure 1 shows how a robot 100 configured to provide a service within a space 10 moves toward a destination while avoiding obstacles 30 in accordance with the execution of algorithms (i.e., a first algorithm and a second algorithm described below) configured to control the movement of the robot 100 toward the destination.

The algorithm configured to control the movement of the robot 100 to its destination may be executed within the robot 100 or within the robot control system 120.

When an algorithm is executed within the robot 100, i.e., when the algorithm is deployed and executed on the robot 100, the robot 100 may be controlled to move toward a destination while avoiding obstacles 30 as defined by the algorithm. On the other hand, when such an algorithm is executed within the robot control system 120, the robot 100 may be controlled based on control signals (commands), such as speed control signals and/or direction control signals, from the robot control system 120 according to the algorithm.

The space 10 in which the robot 100 moves (or travels) is the location where the robot 100 provides services, and may represent, for example, indoor and/or outdoor spaces within a building. A building includes a space where multiple personnel (hereinafter referred to as users) work or reside, and may include, for example, multiple partitioned spaces. The space 10 may represent a portion of a building (a specific floor or a subspace within a floor).

The robot 100 may be a service robot used to provide services within the space 10. The robot 100 may be configured to provide services on at least one floor of the space 10. Although only one robot 100 is shown in FIG. 1, multiple robots 100 may be arranged and operated within the space. Within the space 10, each robot 100 may move around and provide services at an appropriate location within the space 10, or provide services to an appropriate user.

The services provided by the robot 100 may include, for example, at least one of a parcel delivery service, a service for delivering ordered drinks (such as coffee), a cleaning service, and a service for providing other information/content.

The robot 100 may be configured to provide services to predetermined locations in the space 10 or to predetermined users by autonomous navigation, and the (respective) movements of the robot 100 and the provision of services may be controlled by the robot control system 120. The movement of the robot 100 to a destination controlled by the above-described algorithm may be the movement of the robot 100 to a predetermined location for the provision of such services.

In the detailed description below, the "destination" to which the robot 100 travels may be the robot 100's final destination for providing a service, or may be a "waypoint" through which the robot 100 passes on its way to the final destination. As an example, if the service provided by the robot 100 is a parcel delivery service, the destination may be a location for receiving a parcel or a location for delivering the received parcel, or a location through which the robot 100 passes on its way to the receiving location or delivery location.

The structure of the robot 100 and robot control system 120 will be described in more detail with reference to Figures 2-5.

As shown in the figure, the robot 100 may encounter an obstacle 30 while traveling to the destination, and may be controlled to travel to the destination while avoiding the obstacle 30.

An obstacle 30 is an object that exists temporarily or non-temporarily within the space 10, and may be a moving object such as a human or another robot, or may be a fixed object that does not move within the space 10.

In an embodiment, the robot 100 may be controlled by a first algorithm (i.e., a reinforcement learning-based algorithm) that is pre-trained by reinforcement learning to control the movement of the robot 100, and a second algorithm (i.e., a path planning-based algorithm) that controls the movement of the robot 100 according to a path plan based on a map constructed for the space 100.

For example, the movement of the robot 100 to the destination may be controlled by the first algorithm, and when it is determined by the second algorithm that the obstacle 30 it faces can be avoided while satisfying predetermined avoidance conditions (or avoidance ranges), the robot 100 may be controlled to avoid the obstacle 30 by the first algorithm. In this case, if it is determined that the obstacle 30 cannot be avoided while satisfying the predetermined avoidance conditions (or avoidance ranges), the robot 100 may be controlled to stop in front of the obstacle 30.

In other words, in the embodiment, the movement of the robot 100 can be controlled by using the first algorithm to maintain the ability to avoid the obstacle 30, while also using the second algorithm to limit the avoidance range for the obstacle 30 and minimize dangerous and inefficient movement of the robot 100.

Thus, the robot 100 is controlled by a first algorithm that has excellent performance in avoiding the obstacle 30, but the avoidance range for avoiding the obstacle 30 is limited based on the second algorithm, so that the robot can be controlled to avoid dangerous and inefficient movements.

The control method of the robot 100 using the first and second algorithms will be explained in more detail with reference to Figures 5 to 12.

Figure 2 is a block diagram showing a robot providing services within a space in one embodiment.

As described above, the robot 100 may be a service robot used to provide services within the space 10. The robot 100 may travel autonomously and provide services at a predetermined location within the space 10, or provide services to a predetermined user.

The robot 100 is a physical device and may include a control unit 104, a drive unit 108, a sensor unit 106, and a communication unit 102, as shown in the figure.

The control unit 104 may be a physical processor built into the robot 100, and although not shown, may include a path planning processing module, a mapping processing module, a drive control module, a localization processing module, a data processing module, and a service processing module. In this case, the path planning processing module, the mapping processing module, and the localization processing module may be selectively included in the control unit 104 depending on the embodiment so that the robot 100 can continue to navigate autonomously indoors even if communication with the robot control system 120 is lost.

The communication unit 102 may be a component that enables the robot 100 to communicate with other devices (such as other robots or the robot control system 120). In other words, the communication unit 102 may be a hardware module, such as an antenna, data bus, network interface card, network interface chip, or network interface port, of the robot 100, that transmits and receives data and/or information to and from other devices, or a software module, such as a network device driver or a networking program.

The drive unit 108 is configured to control the movement of the robot 100 and enable it to move, and may include equipment for doing so.

The sensor unit 106 may be configured to collect data required for the autonomous navigation and service provision of the robot 100. The sensor unit 106 does not need to include an expensive sensing device and may include sensors such as low-cost ultrasonic sensors and/or low-cost cameras. The sensor unit 106 may include sensors for identifying other robots or humans in front and/or behind the robot. For example, the camera of the sensor unit 106 may identify other robots, humans, and other objects as obstacles 30. The sensor unit 106 may also include an infrared sensor (or an infrared camera). In addition to the camera, the sensor unit 106 may further include sensors for recognizing/identifying nearby users, other robots, or objects. In this way, the sensor unit 106 may be configured to identify obstacles 30.

As an example, when an algorithm for autonomous navigation of the robot 100 is executed within the robot control system 120, in which the robot 100 is controlled by the robot control system 120, the data processing module of the control unit 104 may transmit sensing data including sensor output values of the sensor unit 106 to the robot control system 120 from the communication unit 102. The robot control system 120 may transmit path data generated using an indoor map of the space 10 to the robot 100. The path data may be transmitted from the communication unit 102 to the data processing module. The data processing module may immediately transmit the path data to the drive control module, and the drive control module may control the drive unit 108 in accordance with the path data to control the indoor autonomous navigation of the robot 100. This allows the robot 100 to perform autonomous navigation based on the second algorithm described above. On the other hand, when the first algorithm is executed in the robot control system 120, the robot control system 120 may generate control signals (e.g., speed and/or direction control signals) for controlling the robot 120 according to the first algorithm based on sensing data received from the robot 100, and may control the robot 100 based on the generated control signals. This allows the robot 100 to perform autonomous driving based on the first algorithm.

Alternatively, when the robot 100 and the robot control system 120 cannot communicate or when an algorithm for autonomous navigation is executed within the robot 100, the data processing module may directly process the indoor autonomous navigation of the robot 100 by sending sensing data to the localization processing module and generating path data using the path planning processing module and mapping processing module. This allows the robot 100 to navigate autonomously based on the second algorithm described above. On the other hand, when the robot 100 executes the first algorithm described above, the data processing module of the robot 100 may generate control signals (e.g., speed and/or direction control signals) for controlling the robot 120 according to the first algorithm based on the sensing data, and may control the robot 100 based on the generated control signals. This allows the robot 100 to navigate autonomously based on the first algorithm.

The robot 100 may be a different entity from a mapping robot used to generate an indoor map of the space 10. Because the robot 100 does not include expensive sensing equipment, it may perform indoor autonomous navigation using output values from sensors such as low-cost ultrasonic sensors and/or low-cost cameras. On the other hand, if the robot 100 has previously performed indoor autonomous navigation through communication with the robot control system 120, it can accurately perform indoor autonomous navigation even using low-cost sensors by further utilizing mapping data including path data previously received from the robot control system 120.

However, in some embodiments, the robot 100 may also serve as the mapping robot.

The service processing module may receive commands from the robot control system 120 via the communication unit 102 or via the communication unit 102 and the data processing module. The driving unit 108 may include not only equipment for moving the robot 100, but also equipment related to the service provided by the robot 100. For example, to perform a drink/delivery service, the driving unit 108 of the robot 100 may include a configuration for loading drinks/deliveries and a configuration for handing over the drinks/deliveries to the user (e.g., a robot arm). The robot 100 may also include a speaker and/or a display for providing information/content. The service processing module may transmit driving commands for the service to be provided to the driving control module, and the driving control module may control the components included in the robot 100 and the driving unit 108 in accordance with the driving commands to provide the service.

The robot 100 may be controlled to move to the destination while avoiding the obstacles 30 in accordance with the control of the first and second algorithms described above.

As described above, if the robot 100 only provides sensing data for controlling the robot 100 to the robot control system 120, and the first and second algorithms for controlling the robot 100 are executed by the robot control system 120, the robot 100 may be considered a brainless robot.

On the other hand, each robot 100 may be formed in different sizes and shapes depending on the model, services provided, etc.

The configuration and operation of the robot control system 120 that controls the robot 100 will be described in more detail with reference to Figures 3 to 5.

The technical features explained above with reference to Figure 1 also apply to Figure 2, so duplicate explanations will be omitted.

Figures 3 and 4 are block diagrams showing a robot control system that controls a robot in one embodiment.

The robot control system 120 may be a device that controls the movement (i.e., running) of the robot 100 within the space 10 described above and the provision of services by the robot 100 within the space 10. The robot control system 120 may control the movement of each of the multiple robots 100 and the provision of services by each of the robots 100. The robot control system 120 may set a route for the robot 100 to provide a service by communicating with the robot 100, and may transmit information about such a route to the robot 100. The robot 100 may run based on the received information about the route, and may provide a service at a predetermined location or to a predetermined user. The robot control system 120 may control the movement of the robot so that the robot moves (runs) along the set route.

The robot control system 120 may include at least one computing device.

As described above, the robot control system 120 may be a device that sets a path for the robot 100 to travel and controls the movement of the robot 100. The robot control system 120 may include at least one computing device and may be realized as a server located within the space 10 or outside the space 10.

Setting a path for the robot 100 to travel and controlling the movement of the robot 100 may include controlling the robot 100 using the second algorithm described above.

The robot control system 120 may include a memory 330, a processor 320, a communication unit 310, and an input/output interface 340, as shown.

Memory 330 is a computer-readable recording medium and may include RAM (random access memory), ROM (read-only memory), and a persistent mass storage device such as a disk drive. Here, the ROM and the persistent mass storage device may be included as separate persistent storage devices distinct from memory 330. Memory 330 may also store an operating system and at least one program code. Such software components may be loaded from a computer-readable recording medium separate from memory 330. Such separate computer-readable recording media may include a computer-readable recording medium such as a floppy drive, disk, tape, DVD/CD-ROM drive, or memory card. In other embodiments, software components may be loaded into memory 330 via communication unit 310, which is not a computer-readable recording medium.

The processor 320 may be configured to process computer program instructions by performing basic arithmetic, logic, and input/output operations. The instructions may be provided to the processor 320 by the memory 330 or the communication unit 310. For example, the processor 320 may be configured to execute instructions received according to program code loaded into the memory 330. Such a processor 320 may include components 410-440 as shown in FIG. 4.

Each of the components 410-440 of the processor 320 may be software and/or hardware modules as part of the processor 320, and may represent functions (functional blocks) implemented by the processor. The components 410-440 of the processor 320 will be described with reference to Figure 4.

The communication unit 310 may be a component that enables the robot control system 120 to communicate with other devices (such as the robot 100 or other servers). In other words, the communication unit 310 may be a hardware module, such as an antenna, data bus, network interface card, network interface chip, or networking interface port, of the robot control system 120 that transmits/receives data and/or information to/from other devices, or a software module, such as a network device driver or networking program.

The input/output interface 340 may be a means for interfacing with input devices such as a keyboard or mouse, and devices such as a display and speakers.

Also, in other embodiments, the robot control system 120 may include more components than those shown in the figure.

Referring to Figure 4, the components 410-440 of the processor 320 will be described in more detail. As shown in the figure, the processor 320 may include a map generation module 410, a localization processing module 420, a route planning processing module 430, and a service operation module 440. The components included in the processor 320 may represent different functions executed by at least one processor included in the processor 320 in accordance with control instructions from the operating system code and the code of at least one computer program.

The map generation module 410 may be a component for generating an indoor map of a target facility (e.g., the interior of space 10) using sensing data generated by a mapping robot (not shown) autonomously traveling within space 10.

At this time, the localization processing module 420 may determine the position of the robot 100 within the target facility using sensing data received from the robot 100 via the network and the indoor map of the target facility generated by the map generation module 410.

The path planning processing module 430 may generate control signals for controlling the indoor autonomous movement of the robot 100 using the sensing data received from the robot 100 and the generated indoor map. For example, the path planning processing module 430 may generate a path (i.e., path data) for the robot 100. The generated path (path data) may be set for the robot 100 so that the robot 100 moves along this path. The robot control system 120 may transmit information about the generated path to the robot 100 via a network. For example, the information about the path may include information indicating the current location of the robot 100, information for mapping the current location to the indoor map, and path planning information. The information about the path may include information about a path that the robot 100 will travel to provide a service at a predetermined location within the space 10 or to a predetermined user. The path planning processing module 430 may set a path (i.e., path data) for the robot 100. The robot control system 120 may control the movement of the robot 100 so that the robot 100 moves along such a set path (i.e., according to the set path).

Setting a path for the robot 100 to travel and controlling the movement of the robot 100 may include controlling the robot 100 according to the second algorithm.

The service operation module 440 may include a function for controlling the services provided by the robot 100 within the space 10. For example, the service provider operating the robot control system 120 or the space 10 may provide the user or creator of the robot 100 with an IDE (Integrated Development Environment) for the services (e.g., cloud services) provided by the robot control system 120. In this case, the user or creator of the robot 100 may create software using the IDE for controlling the services provided by the robot 100 within the space 10 and register it in the robot control system 120. In this case, the service operation module 440 may control the services provided by the robot 100 using the software registered in association with the robot 100. As a specific example, assuming that the robot 100 provides a service of delivering an item requested by a user (e.g., a drink or a parcel) to the user's location, the robot control system 120 not only controls the indoor autonomous movement of the robot 100 so that the robot 100 moves to the user's location, but also transmits related commands to the robot 100 so that the robot 100 provides a series of services, such as handing over the item to the user upon arriving at the destination and outputting a corresponding voice message to the user.

The robot control system 120 is a computer system for controlling the robot 100 and may be a server. The robot control system 120 may be a server located outside the space 10 or a building and may be a cloud server. Alternatively, in some embodiments, the robot control system 120 may be located inside the space 10 or a building.

The technical features explained above with reference to Figures 1 and 2 also apply to Figures 3 and 4, so duplicate explanations will be omitted.

Figure 5 is a block diagram showing the processor of a computer system for a robot that controls the robot's movement to a destination using a reinforcement learning-based algorithm and a path planning-based algorithm in one embodiment.

The processor 500 shown in the figure is a component included in a computer system for the robot 100, and may be, for example, the processor of the control unit 104 of the robot 100 or the processor 320 of the robot control system 120 described above.

In other words, the processor 500 is configured to execute control using the first and second algorithms for the robot 100 to avoid the obstacle 30 and move to the destination, and may be included in the robot 100 or the robot control system 120.

The processor 500 may include a first algorithm processing unit 510 and a second algorithm processing unit 520. The first algorithm processing unit 510 may be configured to execute a first algorithm to control the movement of the robot 100, and the second algorithm processing unit 520 may be configured to execute a second algorithm to control the movement of the robot 100.

Each of the components 510 and 520 of the processor 500 is part of the processor 500 and may be a software and/or hardware module, and may represent a function (functional block) realized by the processor.

The first algorithm processing unit 510 and the second algorithm processing unit 520 may execute the first algorithm and the second algorithm in parallel. That is, the first algorithm and the second algorithm may be executed in parallel or simultaneously by the robot 100 or the robot control system 120 to control the movement of the robot 100.

The first and second algorithms are explained in more detail below.

The first algorithm may be a reinforcement learning-based algorithm for controlling the autonomous navigation of the robot 100.

Reinforcement learning is a type of machine learning, a learning method that selects the optimal action for a given situation (or state), and the computer program that is the subject of reinforcement learning may be called an agent. The agent establishes a policy that indicates the action it will take in a given situation, and may train a model to establish a policy that will maximize reward.

As an example, reinforcement learning may involve an artificial intelligence agent interacting in a simulation or the real world to learn how to control the robot 100 in a way that maximizes a reward specified by the developer, rather than a human creating an algorithm to control the robot 100.

The first algorithm may be realized as an algorithm for controlling an autonomous vehicle or an autonomous robot through such reinforcement learning. The reinforcement learning may include deep reinforcement learning (DRL), which is a model that performs reinforcement learning using a deep neural network (DNN), and the first algorithm may be realized through deep reinforcement learning.

When controlling the autonomous movement of the robot 100, such a reinforcement learning-based algorithm, i.e., the first algorithm, can achieve higher performance and robustness in obstacle avoidance, etc., compared to algorithms based on path planning for a map constructed for the space 10.

The first algorithm may be implemented to learn optimal parameters for mapping inputs from sensors included in the robot 100 to the velocity of the robot 100 through interaction with the environment of the space 10.

Therefore, the first algorithm may be configured to control the robot 100 so that the robot 100 moves to a destination while avoiding obstacles during movement.

The first algorithm may be indifferent to means or methods for avoiding obstacles 30 and moving to a destination. For example, if the robot 100 is controlled solely based on the first algorithm, the robot 100 may not make special attempts to protect the user or obstacles from actions that, while relatively rare, have serious consequences, even if such protection results in little or no performance loss.

In addition, if the robot 100 is controlled based solely on the first algorithm, the avoidance range for the robot 100 to avoid the obstacle 30 is not limited, and the robot 100 may make an excessively long detour to avoid the obstacle 30. Furthermore, even in a situation where the robots 100 must line up in a narrow passage and move in order, the robot 100 controlled based solely on the first algorithm may try to overtake other robots in front of it to move to its destination, or may turn around on the spot if it is unable to avoid the other robots.

As such, if the robot 100 is controlled based solely on the first algorithm, it becomes difficult to control the robot 100 to wait without avoiding the obstacle 30 as required for safety or the services provided by the robot 100, and the robot 100 may enter a prohibited area within the space 10 in order to avoid the obstacle 30.

In this embodiment, the movement of the robot 100 is controlled using the first algorithm to maintain the ability to avoid the obstacle 30, but the second algorithm is also used to control the movement of the robot 100 in order to limit the range of the obstacle 30 to minimize dangerous and inefficient movement of the robot 100.

The second algorithm may be a path-planning-based algorithm for controlling the autonomous movement of the robot 100 according to a path plan based on a map (e.g., an indoor map) constructed for the space 10. For example, the second algorithm may be configured to control the movement of the robot 100 according to a path plan based on a path to a destination generated based on a map constructed for the space 10.

The second algorithm may be an algorithm for determining an optimal path for the robot 100 to travel to a destination based on a map constructed for the space 10, and for controlling the robot 100 to travel along this path while avoiding obstacles 30. The map for the space 10 may be generated by an individual mapping robot. The map constructed for the space 100 may be, for example, an occupancy grid map for the space 100.

In this regard, Figure 12 shows an example of an occupancy grid map, which serves as a map of space used by a path-planning-based algorithm.

The occupancy grid map 100 shown in the figure is for a space 10 within a building and may be generated by a mapping robot. The path 1210 shown in the figure indicates, in one example, the path of the robot 100 from its current location or starting point to its destination. The path 1210 may be composed of multiple nodes and edges connecting the nodes. Each node may indicate a waypoint (or destination) that the robot 100 will pass through on the path. The nodes may be set based on pre-set principles.

When controlling the robot 100 using only an occupancy grid-based planner, which is a path planning-based algorithm that uses an occupancy grid map, it is possible to avoid the obstacle 30 and adjust the avoidance range for the obstacle 30. However, such an algorithm is vulnerable to errors in the sensing data or positioning of the robot 100, and when errors occur, the movement of the robot 100 to its destination may not be effectively controlled.

In this embodiment, the first algorithm and the second algorithm are executed in parallel within a computer system, so that the first algorithm controls the movement of the robot 100 to maintain its ability to avoid obstacles 30, and the second algorithm limits the range within which the robot 100 can avoid obstacles 30, minimizing dangerous and inefficient movement of the robot 100.

That is, in the embodiment, the first algorithm and the second algorithm are executed in parallel on a computer system, and the second algorithm may be used as an auxiliary planner. Such a second algorithm may be used to determine whether the robot 100 can avoid the obstacle 30 while satisfying a predetermined avoidance condition (or avoidance range) according to a path plan based on a map constructed for the space 10.

Therefore, in an embodiment, when the robot 100 faces an obstacle 30, if the robot 100 must move a certain distance or more from the path (global path) according to the path planning in order to avoid the obstacle 30 based on the determination made by the second algorithm, the robot 100 may be controlled to stop and wait without avoiding the obstacle 30. As such, in an embodiment, the first algorithm and the second algorithm are executed in parallel, and if the second algorithm determines that it is impossible to avoid the obstacle 30 while satisfying a predetermined avoidance condition (or avoidance range), the robot 100 may be stopped by ignoring the command from the first algorithm. On the other hand, the robot 100 may be controlled to avoid the obstacle 30 according to the first algorithm if the obstacle 30 is small enough to be avoided or can be avoided without moving a certain distance away.

The control method of the robot 100 using the first and second algorithms will be explained in more detail with reference to Figures 6 to 11.

The technical features described above with reference to Figures 1 to 4 also apply to Figures 5 and 12, so duplicate explanations will be omitted.

For ease of explanation, the following detailed description will describe a method for controlling movement of a robot to a destination as operations performed by a computer system including the processor 500 described with reference to Figure 5.

Figure 6 is a flowchart illustrating a method for controlling a robot's movement toward a destination using a reinforcement learning-based algorithm and a path planning-based algorithm in one embodiment.

In step 610, the computer system may control the movement of the robot 100 to the destination based on a first algorithm previously trained by reinforcement learning to control the movement of the robot 100. That is, the robot 100 may be controlled to move while avoiding obstacles identified during movement to the destination based on the reinforcement learning-based algorithm.

The destination may be the robot's 100 final destination for providing service, or a stopover point through which the robot 100 passes on its way to the final destination.

In step 620, the computer system may determine whether the robot 100 satisfies a predetermined avoidance condition for the obstacle 30 using a second algorithm for controlling the movement of the robot 100 according to a path plan based on a map constructed for the space 10 while the robot 100 is moving to its destination.

In the computer system, the first algorithm and the second algorithm are executed in parallel or simultaneously to control the movement of the robot 100, so the determination at step 620 may be performed continuously while the robot 100 is moving to its destination. In other words, the determination at step 620 may be performed in real time while the robot 100 is moving.

The avoidance conditions, including the avoidance range and prohibited areas described below, may be set by an administrator of the robot 100 or the robot control system 120. The administrator may set the avoidance conditions as conditions for path planning using the second algorithm.

As a result, the computer system may determine whether the avoidance conditions are met when the robot 100 identifies or encounters an obstacle 30 while the robot 100 is moving.

The specific method for determining whether the obstacle 30 meets the avoidance conditions and controlling the robot 100 will be explained in more detail with reference to Figures 7 to 11.

In step 630, the computer system may control the robot 100 depending on whether the avoidance condition is met. For example, if the avoidance condition is met, the computer system may control the movement of the robot 100 to move to the destination according to control by the first algorithm. Furthermore, if the avoidance condition is not met, the computer system may control the robot 100 to stop the movement of the robot 100. When the computer system controls the robot 100 to stop the movement of the robot 100, it may transmit an interrupt to the robot 100 to stop the robot 100.

Therefore, the robot 100 can be controlled to avoid the obstacle 30 only if the avoidance conditions or avoidance range for the obstacle 30 limited by the second algorithm are met, and can be controlled to stop without avoiding the obstacle 30 if the avoidance conditions or avoidance range are not met. The computer system may again control the robot 100 to move to the destination after overcoming the obstacle 30 (e.g., after the obstacle 30 has moved to another position).

The technical features described above with reference to Figures 1 to 5 and Figure 12 also apply to Figure 6, so duplicate explanations will be omitted.

Figure 7 is a flowchart showing an example method for determining whether a robot satisfies predetermined obstacle avoidance conditions when controlling the robot's movement to a destination.

In step 710, if an obstacle 30 is identified while the robot 100 is moving to the destination, the computer system may determine whether the robot 100 can move to the destination while avoiding the obstacle 30 without deviating from the predetermined avoidance range based on the path planning using the second algorithm.

The avoidance conditions correspond to the avoidance conditions described above and may be set by an administrator of the robot 100 or the robot control system 120. The avoidance conditions may include the distance traveled by the robot 100 and/or the radius from the current location of the robot 100.

If it is determined that the robot 100 can avoid the obstacle 30 and move to the destination without departing from the avoidance range, the robot 100 may be controlled to move to the destination according to control by the first algorithm. On the other hand, if the robot 100 cannot avoid the obstacle 30 and move to the destination without departing from the avoidance range, the robot 100 may be controlled to stop in front of the obstacle 30.

For example, if the robot 100 must travel a predetermined first distance or more to avoid the obstacle 30 (e.g., a detour must be made a predetermined first distance or more) according to a path plan based on a path to a destination (e.g., a global path) generated based on a map constructed for the space 10, the computer system may determine that the robot 100 cannot avoid the obstacle 30 without leaving the avoidance range. If the path plan based on the second algorithm requires the robot 100 to travel a certain radius or more from its current location to avoid the obstacle 30, the computer system may determine that the robot 100 cannot avoid the obstacle 30 without leaving the avoidance range. In this case, the control command based on the first algorithm may be ignored, and the robot 100 may be controlled to stop without avoiding the obstacle 30.

In this regard, Figure 8 shows a method for controlling the movement of a robot to a destination based on whether the robot deviates from a predetermined obstacle avoidance range, in one example.

In the example shown in the figure, the robot 100 may identify an obstacle 800 ahead when moving to the destination. The computer system 100 may determine whether a detour path to avoid the identified obstacle 30 deviates from a predetermined avoidance range based on path planning using the second algorithm. For example, the computer system 100 may determine that the detour path has deviated from the predetermined avoidance range if the additional distance traveled by the robot 100 when moving along the detour path is equal to or greater than a predetermined value, or if the robot 100 deviates from its current location by more than a certain radius when moving along the detour path. In this case, the control command from the first algorithm may be ignored, and the robot 100 may be controlled to stop without avoiding the obstacle 30.

Alternatively, the computer system may determine that the robot 100 cannot avoid the obstacle 30 without deviating from the avoidance range if, in order for the robot 100 to avoid the obstacle 30, the path to the destination must deviate by more than a predetermined value from the path connecting the robot 100's current location or the starting location to the destination, ignoring the obstacle 30, according to a path plan based on a map constructed for the space 100.

For example, the computer system may determine, based on a map constructed for the space 10, whether the robot 100 can move to the destination while avoiding the obstacle 30 without deviating by more than a predetermined second distance from a straight line from the robot's current location or starting point to the destination. The straight line may correspond to a global path from the robot's current location or starting point to the destination. If the computer system determines that the robot 100 cannot move to the destination while avoiding the obstacle 30 without deviating by more than the second distance, it may control the robot 100 to stop in front of the obstacle 30.

In this regard, Figure 10 is a diagram showing, in one example, a method for controlling a robot's movement to a destination based on whether the robot deviates from a predetermined avoidance range on the path to the destination in order to avoid an obstacle.

Point 1010 shown in the figure may be the starting point or the current location of robot 100, and point 1020 may be the destination. The line connecting point 1010 and point 1020 may correspond to the line described above. A predetermined second distance may be determined by dropping a perpendicular line from such a line. The second distance is an avoidance range, which is a value set by an administrator of robot 100 or robot control system 120, and may be, for example, 1 m.

If it is determined that the robot can move to the destination 1020 while avoiding the obstacle 1030 without deviating by the second distance from the straight line, the robot 100 may be controlled to move to the destination based on the first algorithm; otherwise, the robot 100 may be controlled to stop in front of the obstacle 1030 without avoiding it. On the other hand, even if a detour path 1 exists to bypass the obstacle 1030, the use of such detour path 1 may not be considered because it deviates by the second distance from the straight line.

In the illustrated embodiment, the obstacle 1030 within the avoidance range that can be avoided under control by the first algorithm may be considered to be an obstacle that is small enough that the robot 100 can easily avoid it under control by the first algorithm.

In step 720, if an obstacle 30 is identified while the robot 100 is moving to the destination, the computer system may determine, based on path planning using the second algorithm, whether the robot 100 can move to the destination while avoiding the obstacle 30 without entering a predetermined prohibited area. The prohibited area is set by an administrator of the robot 100 or the robot control system 120, and may be an area within the space 10 into which the robot 100 is not permitted to enter.

If the robot 100 can move to the destination by avoiding the obstacle 30 without entering the prohibited area, the computer system may control the robot 100 to move to the destination according to the first algorithm. If the robot 100 cannot move to the destination by avoiding the obstacle 30 without entering the prohibited area, or if the robot 100 must make a detour of more than a predetermined distance to avoid entering the prohibited area, the computer system may control the robot 100 to stop in front of the obstacle 30.

In this regard, Figure 9 shows a method for controlling the movement of a robot to a destination in one example, depending on whether the robot enters a predetermined prohibited area to avoid an obstacle.

The prohibited area 900 is set by an administrator of the robot 100 or the robot control system 120 and may be, for example, a toilet, a non-working space for the robot 100, or other restricted area. The prohibited area 900 may be identified on a map generated for the space 10. Therefore, based on the path planning by the second algorithm, it can be determined whether the robot 100 will enter the prohibited area 900 to avoid the obstacle 800. As shown in the figure, if the robot 100 cannot avoid the obstacle 800 and move to the destination without entering the prohibited area 900, the robot 100 may be controlled to stop in front of the obstacle 800.

In step 730, if an obstacle 30 is identified while the robot 100 is moving to the destination, the computer system may determine, based on the path planning by the second algorithm, whether the robot 100 can move to the destination while avoiding the obstacle 30 without moving backward. If it is determined that the robot 100 must move backward to avoid the obstacle 30, the computer system may control the robot 100 to stop in front of the obstacle 30. Thus, in an embodiment, if a detour path to avoid the obstacle 30 requires the robot 100 to move backward, such a detour path may not be taken into consideration in the movement control of the robot 100.

The technical features described above with reference to Figures 1 to 6 and Figure 12 also apply to Figures 7 to 10, so duplicate explanations will be omitted.

Figure 11 shows an example of a method for controlling robot movement in an area where robot alignment is required.

As described above, if the obstacle 1120 is small enough that the robot 100 can avoid it without detouring into another passage, the robot 100 may be controlled by the first algorithm to avoid the obstacle 1120 and move to the destination without detouring into another passage or stopping in front of the obstacle 1120.

For example, using the method described with reference to FIG. 10, an obstacle 1120 that can be avoided according to control by the first algorithm may be determined to be a sufficiently small obstacle 1120. As a result, an obstacle 1120 that is determined to be sufficiently small according to the second algorithm may be avoided according to control by the first algorithm, and for obstacles that are not, control of the robot 100 may be interrupted so that they cannot be avoided.

Alternatively, whether the obstacle 1120 is sufficiently small may be determined by a computer system through sensing by the sensor unit 106 of the robot 100.

The illustrated passage 1110 may be an area within the space 10 where the robots 100 are required to line up. Alternatively, the passage 1110 may be a confined/narrow area, e.g., an area too narrow for two or more robots to travel side by side (laterally). For example, the passage 1110 may be a section within the space 10 where multiple robots are required to pass through in a line in sequence.

For example, the aisle 1110 may be an area where robots 100 providing parcel delivery services line up to move to a position to receive parcels.

In accordance with the determination result of step 620 described with reference to FIG. 6, when an obstacle 1120 in the passage 110 is small enough that the robot 100 can avoid it without detouring to another passage, the computer system 100 may control the robot 100 to avoid the obstacle 1120 and move to the destination without detouring to another passage, in accordance with control by the first algorithm.

In this case, when another robot is identified in front of the robot 100 in the passage 1110, the computer system 100 may control the robot 100 to wait behind the other robot without overtaking the other robot. In other words, when another robot is identified as an obstacle 30 in the passage 1110, the computer system 100 determines that such other robot does not fall under the category of a sufficiently small obstacle 1120, and therefore the robot 100 may be controlled to wait behind the other robot, i.e., in front of the other robot. Therefore, within the passage 1110, the robots can be lined up without cutting in line or overtaking other robots and forcibly moving towards their destinations, and each robot can move to its destination in turn.

This embodiment can achieve superior alignment performance of the robot 100 in the passage 1110 compared to when the robot 100 entering the passage 1110 is controlled by a trajectory following algorithm. When the robot 100 is controlled by a trajectory following algorithm, the robot 100 stops when an obstacle is identified, which can cause a problem of the robot 100 stopping even when a small obstacle 1120 that can be avoided is identified. Furthermore, positioning errors can occur for the robot 100, which can cause a problem of the robot 100's movement being unclear when the robot 1110 approaches the passage 1110.

In contrast, in an embodiment, the computer system 100 can control the robot 100 to avoid small obstacles 1120 according to control by the first algorithm, and to stop in front of other obstacles (e.g., other robots) without avoiding them, in accordance with the determination by the second algorithm for the identified obstacles 30.

The technical features described above with reference to Figures 1 to 10 and Figure 12 also apply to Figure 11, so duplicate explanations will be omitted.

The devices described above may be implemented using hardware components, software components, or a combination of hardware and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications that run on the OS. The processing device may also access, record, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the description may refer to the use of a single processing device; however, those skilled in the art will recognize that a processing device may include multiple processing elements and/or multiple types of processing elements. For example, a processing unit may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or may independently or collectively instruct the processing device. The software and/or data may be embodied in any type of machine, component, physical device, virtual device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored and executed in a distributed manner across computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

Methods according to embodiments may be embodied in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the medium may be specially designed for the embodiments or may be readily available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language code, such as that generated by a compiler, but also high-level language code executed by a computer using an interpreter, for example.

While the embodiments have been described above based on limited examples and drawings, those skilled in the art will appreciate that various modifications and variations may be made in light of the above description. For example, the techniques described may be performed in an order different from that described, and/or the components of the described systems, structures, devices, circuits, etc. may be coupled or combined in a manner different from that described, or may be substituted or replaced by other components or equivalents, and still achieve suitable results.

Therefore, different embodiments that are equivalent to the claims are within the scope of the appended claims.

Claims (16)

A robot control method executed by a robot that moves in a space or a robot control system that controls a robot, comprising:
controlling the movement of the robot to a destination based on a first algorithm previously learned by reinforcement learning for controlling the movement of the robot;
a step of determining whether the robot satisfies a predetermined avoidance condition for an obstacle based on a second algorithm for controlling the movement of the robot according to a path plan based on a map constructed for the space while the robot is moving to the destination; and a step of controlling the movement of the robot to move to the destination in accordance with control by the first algorithm if the avoidance condition is satisfied, and controlling the robot to stop moving if the avoidance condition is not satisfied , wherein the first algorithm is configured to control the movement of the robot without limiting an avoidance range for the obstacle . the first algorithm and the second algorithm are executed in parallel in the robot or the robot control system to control movement of the robot.
The robot control method according to claim 1 . the robot is a service robot that provides a service within the space,
The destination is a final destination of the robot for providing the service or a waypoint through which the robot passes on its way to the final destination.
The robot control method according to claim 1 . The first algorithm:
The robot is configured to control the robot so that the robot moves to the destination while avoiding obstacles during movement,
The second algorithm:
and controlling movement of the robot according to the path plan based on a path to the destination generated based on a map constructed for the space.
The robot control method according to claim 1 . the map constructed for the space is an occupancy grid map for the space;
The robot control method according to claim 1 . The determining step includes:
a step of determining, based on the path plan, whether or not the robot can avoid an obstacle identified during the robot's movement to the destination without deviating from a predetermined avoidance range and move to the destination,
When the robot can move to the destination while avoiding the obstacle without departing from the avoidance range, the robot is controlled to move to the destination in accordance with control by the first algorithm;
When the robot cannot avoid the obstacle and move to the destination without leaving the avoidance range, the robot is controlled to stop in front of the obstacle.
The robot control method according to claim 1 . The determining step includes:
and determining that the robot cannot avoid the obstacle without departing from the avoidance range when the robot must move a predetermined first distance or more to avoid the obstacle according to the path plan based on a path to the destination generated on the basis of a map constructed for the space.
The robot control method according to claim 6. The determining step includes:
determining whether the robot can move to the destination while avoiding the obstacles without deviating by more than a second predetermined distance from a straight line from the current location or the starting location of the robot to the destination based on the map constructed for the space;
When the robot cannot move to the destination while avoiding the obstacle without deviating by the second distance or more, the robot is controlled to stop in front of the obstacle.
The robot control method according to claim 6. When the obstacle is small enough that the robot can avoid it without detouring to another path, the robot is controlled to move to the destination while avoiding the obstacle without detouring to another path or stopping in front of the obstacle, according to control by the first algorithm.
The robot control method according to claim 6. The determining step includes:
and determining, based on the path plan, whether or not the robot can move to the destination while avoiding an obstacle identified during the robot's movement to the destination without entering a predetermined prohibited area,
If the robot can move to the destination while avoiding the obstacle without entering the prohibited area, the robot is controlled to move to the destination in accordance with control by the first algorithm;
If the robot cannot move to the destination while avoiding the obstacle without entering the prohibited area, the robot is controlled to stop in front of the obstacle.
The robot control method according to claim 1 . The determining step includes:
determining, based on the path plan, whether the robot can avoid an obstacle identified during the robot's movement to the destination without moving backward and move to the destination;
When it is determined that the robot needs to move backward to avoid the obstacle, the robot is controlled to stop in front of the obstacle.
The robot control method according to claim 1 . Depending on the result of the above judgment,
When an obstacle is small enough to be avoided within a path along which the robot travels to the destination without detouring to another path, the robot is controlled in accordance with the first algorithm so as to avoid the obstacle and travel to the destination without detouring to another path;
When another robot is identified as the obstacle ahead of the robot in the passage, the robot is controlled to wait behind the other robot without overtaking the other robot.
The robot control method according to claim 1 . The passage is an area in the space where the robots are required to be aligned, or an area of a width that does not allow two or more robots to travel side by side.
The robot control method according to claim 12. A non-transitory computer-readable recording medium on which a program for causing the robot or robot control system, which is a computer system, to execute the method of claim 1 is recorded. A computer system for a robot that moves in space,
at least one processor implemented to execute computer readable instructions;
The at least one processor
A computer system that controls the movement of the robot to a destination based on a first algorithm learned in advance by reinforcement learning to control the movement of the robot, determines whether the robot satisfies a predetermined avoidance condition for an obstacle based on a second algorithm that controls the movement of the robot according to a path plan based on a map constructed for the space while the robot is moving to the destination, and controls the movement of the robot to move to the destination in accordance with control by the first algorithm if the avoidance condition is satisfied, and controls the robot to stop moving if the avoidance condition is not satisfied, wherein the first algorithm is configured to control the movement of the robot without limiting the avoidance range for the obstacle . A building,
At least one robot is disposed to move through the space within the building;
The robot's movement within the building is controlled by a computer system that is included in the robot or is a server;
The computer system includes:
at least one processor implemented to execute computer readable instructions;
The at least one processor
A building, comprising: a first algorithm for controlling the movement of the robot to a destination based on a first algorithm learned in advance by reinforcement learning for controlling the movement of the robot; a second algorithm for controlling the movement of the robot according to a path plan based on a map constructed for the space during the movement of the robot to the destination based on which it is determined whether the robot satisfies a predetermined avoidance condition for an obstacle; if the avoidance condition is satisfied, the movement of the robot is controlled to move to the destination in accordance with control by the first algorithm; and if the avoidance condition is not satisfied, the robot is controlled to stop moving; and the first algorithm is configured to control the movement of the robot without limiting the avoidance range for the obstacle .