JP7668152B2 - ROBOT CONTROL DEVICE, ROBOT SYSTEM, AND ROBOT CONTROL METHOD - Google Patents

Description

The present invention relates to a robot control device, a robot system, and a robot control method.

A robot control device is known in which a robot is equipped with a ranging device such as a camera or LiDAR (Laser Imaging Detection and Ranging) based on environmental map data that describes the positions (at least two-dimensional positions) of structures, obstacles, etc. present in the robot's workspace, and an operator or the like operates the robot along a specified path within the robot's workspace while using the ranging device to perform self-position estimation processing.

Patent document 1 discloses a method for determining moving objects from their movement vectors in order to mask the moving objects when performing self-location estimation.

JP 2016-157197 A

However, the above-mentioned Patent Document 1 does not disclose how to eliminate the influence of an object that affects the self-location estimation results as much as possible when the object is present in the robot's workspace.

An example of an object that affects the self-location estimation result is described with reference to Figures 8 and 9.

Figure 8 shows an example of an object that affects the self-location estimation result and an example of the effect it has on the self-location estimation result, where Figure 8(a) is a top view of the object and Figure 8(b) is a side view of the object.

In Figures 8(a) and (b), 601 denotes a pillar present in the robot workspace, and 600 denotes a protective net covering the side of this pillar 601. The protective net 600 is wrapped around a support 602 that is placed at a certain distance from the pillar 601.

In a robot workspace where such pillars 601 and protective net 600 are provided, as shown in FIG. 8(c), when the robot 100 performs self-location estimation using its own ranging device, the light (including laser) emitted from the ranging device may or may not pass through the protective net 600 depending on the relative positions of the robot 100 and the protective net 600.

More specifically, in the relative positional relationship shown in FIG. 8(c), the light emitted from the robot's distance measuring device passes through the protective net 600 located at the bottom of FIG. 8(c) and reaches the pillar 601, where it is reflected by the pillar 601 and returns to the distance measuring device. On the other hand, the protective net 600 located on the right side of FIG. 8(c) is at an acute angle when viewed from the robot 100, so the light emitted from the robot's distance measuring device is reflected by the protective net 600 and returns to the distance measuring device. In other words, the robot 100 may mistake the protective net 600 located on the right side of FIG. 8(c) for the right side of the pillar 601 in the environmental map data, which may result in the robot 100 being unable to correctly perform self-location estimation operations.

The possibility that self-location estimation operation may not be performed correctly will be further explained with reference to Figure 9.

In the self-location estimation process, the position of the robot 100 itself is estimated from the characteristic shape of the environmental map and its relative position (l, θ). As shown in FIG. 9(a), if the light from the distance measuring device of the robot 100 hits the corner of the pillar 601 correctly (the lower right corner in the figure), the correct position of the robot 100 itself can be obtained from the relative position (l, θ) between the corner of the pillar 601 and the robot 100 obtained from the reflected light. This is the correct value for self-location estimation. On the other hand, as shown in FIG. 9(b), if the light from the distance measuring device of the robot 100 hits and reflects off the right side of the protective net 600 instead of the corner of the pillar 601 (the lower right corner in the figure), the distance measuring device of the robot 100 will mistakenly recognize the right side of the protective net 600 as the right side of the pillar 601, which may result in a deviation in the self-location estimation result.

As such, if an object such as the protective net 600, which may or may not be transparent depending on its relative position to the robot 100, is present in the robot's workspace, there is a possibility that errors will occur in the self-position estimation.

The present invention has been made in consideration of the above problems, and aims to provide a robot control device, a robot system, and a robot control method that can determine specific objects that may affect the self-position estimation results and eliminate their influence as much as possible.

In order to solve the above problem, a robot control device according to one aspect of the present invention is a robot control device that controls work performed by a robot, the robot having an imaging device that images objects present in the area surrounding the robot, a ranging device that measures the distance to the object , and a working unit that causes the robot to work, the robot control device having a processor and a memory, the memory storing object recognition information for recognizing a mesh-like object that may or may not be recognized depending on the angle at which the light emitted from the ranging device hits and that may affect the self-position estimation result, the processor receives an imaging signal output from the imaging device , detects a mesh-like object in the area surrounding the robot based on the received imaging signal and the object recognition information, performs self-position estimation by excluding the ranging signal of the area where the mesh-like object is detected from the ranging signal of the area surrounding the robot output from the ranging device , and controls the working unit to cause the robot to perform work.

The present invention makes it possible to realize a robot control device, a robot system, and a robot control method that can identify specific objects that may affect the self-location estimation results and eliminate their influence as much as possible.

1 is a diagram showing an overview of a robot system according to a first embodiment. 1 is a perspective view showing a robot used in a robot system according to a first embodiment. FIG. FIG. 2 is a diagram illustrating a functional configuration of the robot system according to the first embodiment. FIG. 2 is a diagram illustrating a machine learning machine and an object detector in the robot control device according to the first embodiment. 5A to 5C are diagrams illustrating an example of a rotating table control operation in the robot control device according to the first embodiment. 5A to 5C are diagrams illustrating an example of a rotating table control operation in the robot control device according to the first embodiment. 13A to 13C are diagrams for explaining an example of a masking operation in the robot control device according to the second embodiment. FIG. 1 is a diagram illustrating an example of a self-position estimation operation in a typical robot control device. FIG. 1 is a diagram illustrating an example of a self-position estimation operation in a typical robot control device.

The following describes an embodiment of the present invention with reference to the drawings. Note that the embodiment described below does not limit the invention as claimed, and not all of the elements and combinations thereof described in the embodiment are necessarily essential to the solution of the invention.

In addition, in the figures explaining the embodiments, parts having the same functions are given the same reference numerals, and repeated explanations are omitted.

In the following explanation, expressions such as "xxx data" may be used as an example of information, but the information may have any data structure. That is, to show that the information does not depend on the data structure, "xxx data" may be referred to as a "xxx table." Furthermore, "xxx data" may simply be referred to as "xxx." In the following explanation, the structure of each piece of information is an example, and the information may be divided and stored, or combined and stored.

In the following explanation, the process may be described with a "program" as the subject, but since a program is executed by a processor (e.g., a CPU (Central Processing Unit)) to perform a defined process using storage resources (e.g., memory) and/or communication interface devices (e.g., ports) as appropriate, the subject of the process may also be the program. Processes described with a program as the subject may also be processes performed by a processor or a computer having that processor.

The robot control device of this embodiment has the following configuration, as an example:

As already explained, there are some places in the robot's workspace where protective netting is installed around pillars, and when the robot passes nearby, there may be "skips" in the self-location estimation. This is presumably because the protective netting may or may not be recognized by the ranging device depending on the robot's position and the angle at which the light emitted from the ranging device hits the pillar and protective netting.

Therefore, the robot control device of this embodiment detects specific objects near the robot, including mesh-like objects such as protective nets, and estimates the robot's self-position by pointing the LiDAR sensor in a direction to avoid the object.

More specifically, the robot control device of this embodiment mounts a monocular camera with a 360-degree rotation mechanism and a LiDAR sensor on the robot, and while rotating the monocular camera, it recognizes specific objects using an object recognizer such as YOLO (You Only Look Once) (this is trained in advance). Then, it points the LiDAR sensor in a direction where there is no recognized object and performs self-location estimation.

In addition to pointing the LiDAR sensor, it is also possible to perform self-location estimation while ignoring the sensing results in the area where the measurement area of the LiDAR sensor coincides with the bounding box obtained by object recognition.

The above-mentioned technique can also be applied to objects that LiDAR sensors are not good at detecting, such as mesh objects (protective nets, gratings, handrails), reflective objects (exposed metal ducts, gratings, delivery equipment, other robots), and black objects (black walls).

The robot control device of this embodiment uses a classifier to detect a specific object that causes self-location estimation to fail, making it possible to perform self-location estimation excluding the area that contains the object, thereby preventing self-location estimation failure.

The robot system according to the first embodiment will be described with reference to Figs. 1 to 6. Fig. 1 is a diagram showing the overall configuration of the robot system according to the first embodiment.

The robot system of this embodiment is mainly composed of a robot 100, a server (robot control device) 101, a WiFi (registered trademark) router 104, a display 102, and an input device 103 including a keyboard and a joystick.

The server 101 is a device capable of various types of information processing, for example an information processing device such as a computer. The server 101 has a processor 101a represented by a CPU and a memory 101b, and further has a storage 101c, a network 101d, and a power source 101e.

The processor 101a is, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or an FPGA (Field-Programmable Gate Array). The memory 101b includes, for example, a magnetic storage medium such as a HDD (Hard Disk Drive), a RAM (Random Access Memory), a ROM (Read Only Memory), or a semiconductor storage medium such as an SSD (Solid State Drive). In addition, a combination of an optical disk such as a DVD (Digital Versatile Disk) and an optical disk drive is also used as memory. Other well-known storage media such as magnetic tape media are also used as memory.

Memory 101b stores programs such as firmware. When server 101 starts operating (e.g., when the power is turned on), the programs such as firmware are read from this memory and executed to control the entire server 101. In addition to programs, memory 101b also stores data necessary for each process of server 101. Details of the programs stored in memory 101b will be described later with reference to FIG. 3.

The display 102 displays a predetermined screen based on a display control signal sent from the server 101. The input device 103 sends an input instruction signal to the server 101 based on an input instruction operation from an operator (not shown) of the server 101. The WiFi router 104 works in conjunction with the network 101d of the server 101 to perform communication between the server 101 and its peripheral devices.

Note that the server 101 in this embodiment may be configured as a so-called cloud in which multiple information processing devices are configured to be able to communicate with each other via a communication network.

The robot 100 has a PC 105, a LiDAR sensor 100a, a rotating table 100b, a crawler 100c, an arm 100d, an RGB camera (imaging device) 100e, a battery 100f, and a motor drive circuit 100g.

Like server 101, PC 105 is a device capable of various types of information processing, for example an information processing device such as a computer. PC 105 has a processor 105a represented by a CPU and memory 105b, and further has storage 105c, network 105d, and power supply 105e. The various components that make up PC 105 are the same as those of server 101, so a description thereof will be omitted here.

Among the components of the robot 100 other than the PC 105, the detailed configuration other than the motor drive circuit 100g will be described later with reference to FIG. 2.

The motor drive circuit 100g is a circuit that drives the motors and servo motors (not shown) mounted on the rotating table 100b, the crawler 100c, and the arm 100d. The LiDAR sensor 100a and the RGB camera 100e are mounted on the rotating table 100b, and can each change direction independently.

The Linux (registered trademark) OS runs on the PC 105 and the server 101 (see FIG. 3), and under the control of the OS, the PC 105 and the server 101 are connected to each other via a TCP/IP-based Wi-Fi network.

Figure 2 is a perspective view showing the appearance of the robot 200 used in the robot system of this embodiment.

The robot 200 has crawlers 211 on the left and right sides of the robot casing 210, and a robot arm 220 and a rotating table 212 on the top of the robot casing 210. A LiDAR sensor 213 and an RGB camera 214 are mounted on the top of the rotating table 212. In addition, the PC 105, battery 100f, and motor drive circuit 100g (not shown in FIG. 2) described in FIG. 1 are mounted inside the robot casing 210.

The LiDAR sensor 213 is installed so that it can irradiate a laser backwards (diagonally downward to the right in FIG. 2) from the rear of the vehicle (the direction marked "back" in FIG. 2) (so that it can measure distances). As an example, the LiDAR sensor 213 can measure distances in a range of 210 degrees left and right and 40 degrees up and down.

The LiDAR sensor 213 and RGB camera 214 are mounted on a rotating table 212 equipped with a rotation mechanism (not shown), allowing the orientation of the LiDAR sensor 213 and RGB camera 214 to be changed, enabling measurements over a wider area.

As an example of the LiDAR sensor 213, there are sensors that scan space in two dimensions, such as the product name Hokuyo Denki UST-10/20LX, and those called 3D LiDAR (3D scanning), such as the product name Hokuyo Denki YVT35-LX, and either type of LiDAR sensor can be applied to the robot system of this embodiment. Similar distance measuring devices include those that use infrared rays instead of lasers (Microsoft® Kinect® 2), and distance measuring devices that use infrared rays can also be applied to the robot system of this embodiment.

Figure 3 is a diagram showing the software configuration of the robot system of this embodiment. In Figure 3, an example is explained in which the robot system is composed of one robot and one server.

In FIG. 3, 300 is software installed in the robot, and 301 is software installed in the server. In the robot system of this embodiment, the overall operation of both the robot and the server is managed by the Linux OS and robot control middleware (ROS: Robot Operating System) that runs on the Linux OS (collectively indicated by the reference symbol 310).

Each program that uses ROS (runs on ROS) performs processing by sending and receiving named data called topics. Specifically, a program receives input data as a topic and outputs the processing results as a topic. Programs repeatedly send and receive topics between each other. ROS employs a distributed processing architecture.

Running on the robot are a LiDAR sensor data transmission program 311, a turntable drive program 312, a crawler drive program 313, an arm drive program 314, and an RGB camera image distribution program 315. Also running on the server are a point cloud map distribution program 320, a self-position estimation program 321, a turntable control program 322, a path generation program 323, a crawler control program 324, an arm control program 325, an object detector 326, and a screen display program 327.

The point cloud map delivery program 320 uses a point cloud map stored in a file (not shown in FIG. 3, stored in memory 101b of the server 101) as input data, and outputs the point cloud map as an output topic. The point cloud map delivery program 320 reads the point cloud map of the robot workspace provided from the file, and delivers it to the self-position estimation program 321. Here, a point cloud map is a map made up of a point cloud represented by an infinite number of points (x, y, z).

The self-location estimation program 321 uses a point cloud map and measurement data from a LiDAR sensor as input topics, and outputs the position and posture of the robot as output topics. The self-location estimation program 321 performs self-location estimation processing. The self-location estimation processing is performed by matching the point cloud map with the measurement data (point cloud) from the LiDAR sensor using a method called scan matching. As a scan matching algorithm, well-known algorithms such as NDT (Normal Distributions Transform) and ICP (Iterative Closest Point) are preferably used. The position and posture of the robot, which are the output topics of the self-location estimation program 321, are expressed by seven parameters consisting of position (x, y, z) and posture parameters (qx, qy, qz, qw) called quaternions.

The turntable control program 322 uses the object detection result, which is the output topic of the object detector 326 described below, as an input topic, and outputs the orientation of the robot's turntable as an output topic. From the object detection result, which is the output topic of the object detector 326, the turntable control program 322 determines and distributes the orientation of the turntable in which no object is captured or in which the area in which the object is captured is the smallest. Details of the operation of the turntable control program 322 will be described later with reference to Figures 5 and 6.

The route generation program 323 uses the point cloud map as an input topic, and the position and posture of the robot, which are the output topics of the self-location estimation program 321. The route generation program 323 also uses as input data the destination input by the operator operating (clicking) an input device (mouse) connected to the server. The route generation program 323 also outputs the route from the current position to the destination (a list of points spaced at 10 cm intervals) as an output topic.

When the server operator inputs the robot's destination on a point cloud map, specified by performing an operation such as clicking the mouse, the path generation program 323 outputs a list of routes from the current position to the destination, connected by dots spaced 10 cm apart. At this time, the path generation program 323 considers areas on the point cloud map where dots are densely packed as obstacles (walls) that the robot cannot pass through. The path generation program 323 then uses known algorithms such as the Dijkstra algorithm and the Easter algorithm to find a route in a space without obstacles. The operation of the path generation program 323 is known, so further detailed explanation will be omitted.

The crawler control program 324 uses as input topics the route, which is the output topic of the route generation program 323, and the position and posture of the robot, which are the output topics of the self-location estimation program 321, and outputs as output topics the movement amount and rotation angle of the robot. The crawler control program 324 determines the movement amount and rotation angle required to move the robot from the current position and posture to the next point on the route. The operation of the crawler control program 324 is known, so further detailed explanation will be omitted.

The arm control program 325 accepts operational input from an input device such as a joystick connected to the server, and outputs the rotation angle of the joint of the robot's arm as an output topic. The arm control program 325 specifies the angle of the arm joint based on the operational input from an input device such as a joystick. The operation of the arm control program 325 is known, so further detailed explanation is omitted.

The object detector 326 receives the image signal output as a result of capturing an image by the RGB camera, and outputs the object detection result as an output topic. Based on the image signal output from the RGB camera, the object detector 326 detects a specific object captured in the image of the RGB camera, and outputs the object detection result. The specific object to be detected by the object detector 326 is trained in advance. Details of the operation of the object detector 326 will be described later with reference to FIG. 4.

The screen display program 327 uses the point cloud map, the robot's position and posture, the route, the orientation of the turntable, the RGB camera images, and the object detection results as input topics. The screen display program 327 superimposes and displays information such as the point cloud map, the robot's position and posture, the route, the orientation of the turntable, the RGB camera images, and the object detection results on the screen of a display connected to the server.

The LiDAR sensor data transmission program 311 acquires data measured by a LiDAR sensor mounted on the robot and outputs this measurement data as an output topic.

The turntable drive program 312 uses the turntable orientation, which is the output topic of the turntable control program 322, as an input topic, and rotates the turntable based on this turntable orientation.

The crawler drive program 313 uses the robot's movement amount and rotation angle, which are output topics of the crawler control program 324, as input topics, and converts the robot's movement amount and rotation angle into rotation amounts for the left and right crawlers to rotate the left and right crawlers.

The arm drive program 314 uses the arm joint rotation angle, which is the output topic of the arm control program 325, as an input topic, and rotates the arm joint to the specified arm joint rotation angle.

The RGB camera image distribution program 315 acquires an image signal from the RGB camera mounted on the robot and outputs this image signal as an output topic.

Figure 4 is a diagram for explaining the method of generating the machine learning results that are input to the object detector 326 in the robot system of this embodiment.

Hundreds to tens of thousands of pairs of training images 410 and annotation information 411 are prepared, and machine learning is performed by the machine learning device 400 to obtain a training result 420. The training image 410 is an image that contains at least a part of a specific object to be detected by the object detector 326 of this embodiment, and the annotation information is information that indicates the position in the training image 410 where the specific object is reflected.

It is preferable to determine in advance the specific object to be detected. In this embodiment, a protective net 600 is used as the specific object. Other examples of specific objects include objects that the LiDAR sensor 213 is not good at detecting, such as net-like objects (protective nets, gratings, handrails), reflective objects (exposed metal ducts, gratings, delivery equipment, other robots), and black objects (black walls).

Then, by setting the learning result 420 in the object detector 326 and inputting the RGB image 430 obtained from the RGB camera, the object detection result (object area) 440 can be obtained.

Here, the machine learning device 400 and the object detector 326 can suitably use OSS (Open Source Software) such as Yolo and SSD (Single Shot MultiBox Detector).

Next, the operation of the turntable control program 322 will be described with reference to Figures 5 and 6.

The turntable control program 322 calculates the object detection rate ratio (520), as shown in FIG. 5(b). How to calculate the object detection rate ratio will be described later. Next, the turntable control program 322 determines whether the object detection rate ratio calculated in 520 is equal to or greater than a certain threshold (511). The threshold may be determined as appropriate, but it is preferable that the threshold be a threshold that can determine that the presence of a specific object detected by the object detector 326 in a position where the specific object is expected to affect the distance measurement results by the LiDAR sensor 213 is inferred from the image of the RGB camera 214 (i.e., inferred from the relative positional relationship between the robot 100 and the specific object).

If the result of the judgment shows that the object detection rate ratio is equal to or greater than the threshold value (YES in 511), it is determined that further control is necessary and the process proceeds to 530. If the result shows that the object detection rate ratio is below the threshold value (NO in 511), it is determined that further control is unnecessary and the process returns to 520, and the process of calculating the object detection rate ratio continues. In 530, the turntable control program 322 performs turntable drive processing.

The procedure for calculating the object detection rate (ratio) by the turntable control program 322 is described below with reference to FIG. 5(c).

First, the turntable control program 322 obtains an image from the RGB camera 214 (521). Next, the turntable control program 322 obtains an object detection result from the object detector 326 (522). As an example, as shown in FIG. 5(a), it is assumed that the object detection result 501 is output from the object detector 326 based on the image 500 output from the RGB camera 214. In the example shown in FIG. 5(a), a specific object, a safety net 600, is reflected in the image from the RGB camera 214.

Next, the turntable control program 322 calculates the object detection rate ratio based on the following formula (523).

Object detection rate ratio=number of pixels in object detection area/number of pixels in the image of the RGB camera 214

The turntable drive process by the turntable control program 322 will be described with reference to Figure 6.

In this embodiment, it is assumed that the rotation range of the turntable 212 is -90 degrees to 90 degrees. First, the turntable control program 322 sets the initial value of the rotation angle i of the turntable 212, the minimum value min_ratio of the object detection rate ratio, and the rotation angle min_i of the turntable 212 at that time (531). After this, the turntable 212 is rotated in 1 degree increments from -90 degrees to 90 degrees with the rotation angle set as variable i, and the object detection rate ratio at each rotation angle is obtained.

The turntable control program 322 determines whether the variable i is equal to or less than 90 degrees (532). If the variable i is equal to or less than 90 degrees (YES in 532), the program proceeds to 533. If the variable i is greater than 90 degrees (NO in 532), the program proceeds to 538.

In step 533, the turntable control program 322 sets the rotation angle of the turntable 212 to the variable i. Next, the turntable control program 322 calculates the object detection rate ratio according to the procedure shown in FIG. 5(c) (534).

The turntable control program 322 then determines whether the object detection rate ratio calculated in 534 falls below the minimum value min_ratio (535). If the determination results in that the object detection rate ratio falls below the minimum value min_ratio (YES in 535), the program proceeds to 536, and if the object detection rate ratio is equal to or greater than the minimum value min_ratio (NO in 535), the program proceeds to 537. In 536, the program rewrites the value of the minimum value min_ratio to the object detection rate ratio calculated in 534, and rewrites the value of the rotation angle min_i to the rotation angle i at that time.

In step 537, the rotation angle i of the turntable 212 is increased by 1 degree, and then the process returns to step 532 and the subsequent steps are repeated.

On the other hand, in 538, the rotation angle of the rotation table 212 is set to min_i. This allows the rotation table 212 to rotate to the rotation angle min_i when the object detection rate ratio is at its minimum min_ratio, and allows the LiDAR sensor 213 to be oriented in a direction that minimizes the effect of a specific object on the distance measurement operation by the LiDAR sensor 213.

As described above in detail, the robot system of this embodiment makes it possible to identify specific objects that may affect the self-location estimation results and eliminate their influence as much as possible.

In the robot system of this embodiment, the influence of a specific object detected by the object detector 326 is minimized by excluding (masking) the area in which the specific object is present from the distance measurement results of the LiDAR sensor 213.

As shown in FIG. 7(a), the object detector 326 outputs an object detection result 501 that detects a specific object based on an image 500 of the RGB camera 214. Also, as shown in FIG. 7(b), the LiDAR sensor 213 obtains a measurement point cloud 700 as a distance measurement result.

Based on the object detection result 501 and the measurement point cloud 700, the self-location estimation program 321 creates a point cloud 701 by masking (removing) the measurement point cloud 700 in the area where a specific object is detected, and performs self-location estimation operation based on this point cloud 701.

Therefore, this embodiment also makes it possible to identify specific objects that may affect the self-location estimation results and eliminate their influence as much as possible.

The above-mentioned embodiments are detailed descriptions of the configurations in order to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described. In addition, it is possible to add, delete, or replace part of the configuration of each embodiment with another configuration.

The above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example by designing them as integrated circuits. The present invention can also be realized by software program code that realizes the functions of the embodiments. In this case, a storage medium on which the program code is recorded is provided to a computer, and a processor of the computer reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiments, and the program code itself and the storage medium on which it is stored constitute the present invention. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, SSDs (Solid State Drives), optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

In addition, the program code that realizes the functions described in this embodiment can be implemented in a wide range of program or script languages, such as assembler, C/C++, perl, Shell, PHP, Java (registered trademark), Python, etc.

Furthermore, all or part of the program code of the software that realizes the functions of each embodiment may be stored in memory 101b in advance, or, if necessary, may be stored in memory 101b from a non-transitory storage device of another device connected to the network, or from a non-transitory storage medium via an external I/F (not shown) provided in server 101.

Furthermore, the program code of the software that realizes the functions of the embodiment may be distributed over a network and stored in a storage means such as a computer's hard disk or memory, or in a storage medium such as a CD-RW or CD-R, and the processor of the computer may read and execute the program code stored in the storage means or storage medium.

In the above examples, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

100, 200...Robot 100a, 213...LiDAR sensor 100b, 212...Rotary base 100e, 214...RGB camera 101...Server 101a...Processor 101b...Memory 311...LiDAR sensor data transmission program 312...Rotary base driving program 313...Crawler driving program 314...Arm driving program 315...RGB camera image distribution program 320...Point cloud map distribution program 321...Self-position estimation program 322...Rotary base control program 323...Path generation program 324...Crawler control program 325...Arm control program 326...Object detector 327...Screen display program 400...Machine learning machine 600...Protective net 601...Pillar

Claims (8)

A robot control device for controlling a task performed by a robot,
The robot has an imaging device that images an object present in an area surrounding the robot, a distance measuring device that measures a distance to the object , and a working unit that causes the robot to perform a task;
The robot control device has a processor and a memory,
the memory stores object recognition information for recognizing a net-like object that may or may not be recognized depending on the angle at which light emitted from the distance measuring device hits the object and that may affect a self-location estimation result;
The processor,
receiving an imaging signal output from the imaging device ;
Detecting the net-like object within a region surrounding the robot based on the received imaging signal and the object recognition information;
performing self-location estimation by excluding the ranging signal of the area in which the net-like object is detected from the ranging signal of the area around the robot outputted from the ranging device ;
A robot control device that controls the working unit so as to cause the robot to perform a task. the imaging device includes an RGB camera that images an area around the robot;
The distance measurement device includes a LiDAR that measures the distance to an object by irradiating a light beam to an area around the robot and detecting the light reflected from the object.
The robot control device according to claim 1 . 3. The robot control device according to claim 2, wherein the memory stores position identification information for identifying a position of the object that is present in the area surrounding the robot and in a workspace of the robot. the working unit has a rotation mechanism that rotates an imaging range of the imaging device at least in a horizontal direction,
the imaging device and the distance measuring device are mounted on a rotation platform equipped with the rotation mechanism;
The robot control device according to claim 1 , wherein the processor performs the self-position estimation by controlling the rotation mechanism so that the net- like object is least likely to be captured within the imaging range. The robot control device according to claim 1 , wherein the processor performs self-location estimation by masking the distance measurement signal in the area where the net-like object is detected. The robot control device according to claim 1, characterized in that the object recognition information is generated by machine learning. A robot system having a robot and a robot control device that controls an operation performed by the robot,
The robot has an imaging device that images an object present in an area surrounding the robot, a distance measuring device that measures a distance to the object , and a working unit that causes the robot to perform a task;
The robot control device has a processor and a memory,
the memory stores object recognition information for recognizing a net-like object that may or may not be recognized depending on the angle at which light emitted from the distance measuring device hits the object and that may affect a self-location estimation result;
The processor,
receiving an imaging signal output from the imaging device ;
Detecting the net-like object within a region surrounding the robot based on the received imaging signal and the object recognition information;
performing self-location estimation by excluding the ranging signal of the area in which the net-like object is detected from the ranging signal of the area around the robot outputted from the ranging device ;
A robot system comprising: a working unit that controls the working unit so as to cause the robot to perform a task. A robot control method executed by a robot control device that controls a task performed by a robot, comprising:
The robot has an imaging device that images an object present in an area surrounding the robot, a distance measuring device that measures a distance to the object , and a working unit that causes the robot to perform a task;
The robot control device has a processor and a memory,
the memory stores object recognition information for recognizing a net-like object that may or may not be recognized depending on the angle at which light emitted from the distance measuring device hits the object and that may affect a self-location estimation result;
The processor,
receiving an imaging signal output from the imaging device ;
Detecting the net- like object based on the received imaging signal and the object recognition information;
performing self-location estimation by excluding the ranging signal of the area in which the net-like object is detected from the ranging signal of the area around the robot outputted from the ranging device ;
A robot control method comprising: controlling the working unit so as to cause the robot to perform a task.