JP7536697B2 - Unmanned Vehicles - Google Patents

Description

The present invention relates to an unmanned moving body that moves by following a target.

Unmanned mobile objects such as robots that move automatically by following a target (such as a worker who is the target to be followed or another unmanned mobile object that is the target to be followed). For example, Patent Document 1 describes a specific configuration of this type of unmanned mobile object.

The unmanned mobile body described in Patent Document 1 is provided with a lead-type input device. The lead-type input device is a device that physically connects the operator and the unmanned mobile body with a lead such as a string or chain. The operator can operate the unmanned mobile body by taking a specific action, such as pulling the lead of the lead-type input device a predetermined number of times.

In Patent Document 1, the unmanned mobile body calculates the position of the worker based on the direction of the lead being pulled by the worker and the length of the lead (or the force applied to the lead), and follows the worker by moving to the calculated position. By calculating the worker's position using the lead, it is possible to avoid misidentifying the worker who is the target of following, even when there are many people around the unmanned mobile body, for example.

International Publication No. 2017/56334

In Patent Document 1, in order to detect the direction and length of the lead and the force acting on the lead, the lead must be pulled taut by the worker. If this is not the case, the direction and length of the lead cannot be detected with high accuracy, and since the force acting on the lead cannot be detected, it is difficult to calculate the position of the worker.

Patent Document 1 states that "It would be even better if the lead-type input device was configured to detect both θ (i.e., the lead direction) and L (i.e., the lead length) regardless of whether the lead 31 is pulled or not." However, the specific configuration is unclear. It is difficult to understand from the contents of Patent Document 1 the configuration for detecting the lead direction, length, and force acting on the lead when the lead is not pulled.

The present invention has been made in consideration of the above circumstances, and aims to provide an unmanned mobile body that can calculate the position of a target to be tracked regardless of whether the target is pulling a lead.

An unmanned mobile body according to one embodiment of the present invention is a device that moves by following a target to be followed, and includes a main body, a lead having one end attached to the main body and the other end attached to the target to be followed, a camera provided on the main body for capturing the lead within the field of view, a lead detection unit that detects the other end of the lead attached to the target to be followed from an image captured by the camera unit, and a position calculation unit that calculates the position of the target to be followed based on the other end of the lead detected by the lead detection unit, and moves to the position of the target to be followed calculated by the position calculation unit.

The unmanned moving body according to one embodiment of the present invention further includes a tracking target determination unit that determines whether the tracking target has been lost, a lead direction detection unit that detects the extension direction of the lead shown in the captured image, and a destination estimation unit that, when the tracking target determination unit determines that the tracking target has been lost, estimates the destination of the tracking target based on the extension direction detected by the lead direction detection unit, and may be configured to fit the destination estimated by the destination estimation unit within the angle of view of the camera unit.

The image captured by the camera unit is, for example, a color image. In this case, the lead direction detection unit sets a threshold value for the captured image based on the color of the lead, binarizes the captured image based on the set threshold value, and detects the extension direction of the lead based on the binarized captured image.

In one embodiment of the present invention, if the target to be tracked is not captured at the destination within the angle of view, the position calculation unit may calculate a tentative position based on the extension direction detected by the lead direction detection unit as the position of the target to be tracked.

In one embodiment of the present invention, the lead direction detection unit determines whether or not the lead shown in the captured image is loose, and if it determines that the lead is loose, may detect the extension direction of the line segment connecting one end of the lead to the other end as the extension direction of the lead.

In one embodiment of the present invention, the lead is, for example, elastic.

The unmanned mobile body according to one embodiment of the present invention may further include a reed storage section that reels out and winds up the reed in response to an external force associated with the movement of the object to be tracked.

The unmanned moving body according to one embodiment of the present invention may further include a detachment determination unit that determines whether the lead has been detached from the target to be followed, and may be configured to stop following the target to be followed when the detachment determination unit determines that the lead has been detached from the target to be followed.

The unmanned moving body according to one embodiment of the present invention may further include a stair determination unit that determines whether the target to be tracked moves up stairs, a shape determination unit that determines the shape of the stairs when the stair determination unit determines that the target to be tracked moves up stairs, and a position correction unit that corrects the position of the target to be tracked on the stairs calculated by the position calculation unit according to the shape of the stairs determined by the shape determination unit.

In one embodiment of the present invention, the staircase determination unit may determine whether the target to be tracked is moving up stairs based on whether the extension direction of the lead detected by the lead direction detection unit is a direction pointing toward the stairs.

According to one embodiment of the present invention, an unmanned mobile object is provided that can calculate the position of a target to be tracked regardless of whether the target is pulling a lead.

1 is a diagram showing a schematic diagram of an unmanned moving body according to an embodiment of the present invention; 2 is a diagram showing a schematic diagram of an internal structure of a lead portion of an unmanned moving body according to one embodiment of the present invention; FIG. 1 is a block diagram showing components built into the fuselage of an unmanned vehicle according to one embodiment of the present invention; FIG. 4 is a flowchart showing a following movement process executed by the control device in one embodiment of the present invention. 5 is a flowchart showing a process of identifying a person to be followed that is executed by a control device according to an embodiment of the present invention. FIG. 6 is a diagram showing an example of a binary image generated in step S204 of FIG. 5 . 5 is a diagram supplementing the explanation of steps S102 to S107 in FIG. 4. 5 is a diagram for explaining an example of a method for calculating a virtual position of a worker in step S109 in FIG. 4; 5 is a flowchart showing a staircase movement process executed by the control device in one embodiment of the present invention.

Below, an unmanned mobile body according to one embodiment of the present invention will be described with reference to the drawings. In this embodiment, a multi-legged robot is taken as an example of an unmanned mobile body, but the unmanned mobile body according to the present invention is not limited to this. The unmanned mobile body may be anything that is capable of moving by following a target to be followed, and may be, for example, a cart-type unmanned mobile body equipped with caterpillar tracks or wheels, or an unmanned flying object such as a drone. The unmanned mobile body may also be equipped with an imaging device (camera) required for patrol, or a loading platform or cart for transporting materials.

In this embodiment, it is assumed that the unmanned mobile body will move by following a target to be tracked in locations such as construction sites and civil engineering sites where the layout of the ground, floors, walls, ceilings, materials, etc. changes daily. However, the locations where the unmanned mobile body moves to be tracked are not limited to this. The unmanned mobile body may also be moved by following a target to be tracked in other locations such as farmland, theme parks, commercial facilities, warehouses, disaster sites, etc.

Figure 1 is a schematic diagram of an unmanned mobile body 1 according to one embodiment of the present invention. The unmanned mobile body 1 is a robot capable of walking on four legs, and as shown in Figure 1, comprises a main body unit 10, a camera unit 20, a fixture 30, a lead unit 40, and a power button 50. Note that Figure 1, which illustrates the components of the unmanned mobile body 1, and Figures 2 and 3 described below, illustrate the main components necessary for explaining this embodiment. For example, illustrations of some components, such as the battery and drive mechanism, which are essential components of the unmanned mobile body 1, are omitted as appropriate.

In general terms, the unmanned mobile body 1 is configured to calculate the position of the worker 2 to be followed based on the lead 41 (described in detail later) that appears in the image captured by the camera unit 20, and to follow the worker 2 by moving to the calculated position. With this configuration, the position of the worker 2 can be calculated regardless of whether the lead 41 is being pulled or not.

Note that the target to be followed is not limited to a person. In another embodiment, the target to be followed may be another unmanned mobile body. Furthermore, the number of unmanned mobile bodies following the target to be followed is not limited to one, and may be multiple. As an example, unmanned mobile body 1 may be moved by following worker 2, and further, another unmanned mobile body may be moved by following unmanned mobile body 1.

In the SLAM (Simultaneous Localization and Mapping) process described later, a world coordinate system (X, Y, Z) and a local coordinate system (x, y, z) are defined. In the world coordinate system, whose origin is fixed, the vertical direction is defined as the Z-axis direction, and the two horizontal directions that are perpendicular to the Z-axis direction and perpendicular to each other are defined as the X-axis direction and the Y-axis direction, respectively.

In contrast, the local coordinate system with the unmanned mobile body 1 as its base point has an origin that moves as the unmanned mobile body 1 moves. In the local coordinate system, the vertical direction is the z-axis direction, and the two horizontal directions that are perpendicular to the z-axis direction and perpendicular to each other are the x-axis direction and the y-axis direction, respectively. These three mutually perpendicular axial directions form a right-handed system. For ease of explanation, the positive x-axis direction is also referred to as the front side, the negative x-axis direction is also referred to as the rear side, the positive y-axis direction is also referred to as the left side, the negative y-axis side is also referred to as the right side, the positive z-axis side is also referred to as the upper side, and the negative z-axis side is also referred to as the lower side.

Note that the names of directions in the local coordinate system are used for convenience to explain the relative positional relationships of components, and do not indicate absolute directions. For example, if the attitude of the unmanned mobile body 1 is tilted, the x-axis and y-axis directions will be inclined relative to the horizontal plane according to that attitude, and the z-axis direction will be inclined relative to the vertical plane according to that attitude.

The main body 10 comprises a trunk 11 and four legs 12. Two legs 12 are provided on each side of the trunk 11. The legs 12 are provided at intervals in the x-axis direction. The main body 10 is configured so that the four legs 12 can be driven individually, and can move freely in each direction by driving each leg 12 individually.

The shape of the torso 11, the number of legs 12, the number of joints, the attachment positions, etc. are not limited to those shown in FIG. 1. There is a degree of freedom in the configuration of the main body 10, and various design changes are possible.

The camera unit 20 and the fixture 30 are provided at the front of the body unit 11.

The camera unit 20 is a stereo camera that captures RGB-D images. The camera unit 20 captures RGB color images and simultaneously captures distance information (three-dimensional point cloud data) to each object within the field of view based on the distance (baseline length) between the pair of cameras, the focal length of the cameras, and the parallax.

The image sensor of the camera unit 20 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor. This image sensor may be another imager, such as a CCD (Charge Coupled Device) image sensor. In addition, in this embodiment, the color filter of the image sensor is an RGB primary color filter, but may be another color filter, such as a CMYG complementary color filter.

In this embodiment, distance information is acquired using the camera unit 20, but the scope of the present invention also includes configurations in which distance information is acquired using other means, such as a scanner-type laser distance sensor.

The fixture 30 is provided below the camera unit 20 in a position that fits within the angle of view of the camera unit 20. More specifically, the fixture 30 is provided at the front of the body unit 11 so as to be located on a line that extends vertically downward from the midpoint of the baseline of the camera unit 20. As a result, the fixture 30 is always captured at a fixed position (the bottom center in this embodiment) in the captured image.

The fixing device 30 has, for example, a rod-shaped member attached to the front of the body 11. A spherical member is formed at the tip of this rod-shaped member. One end of the lead 41 of the lead portion 40 (hereinafter referred to as "lead end 41a") is fastened so as to be hooked onto this spherical member.

The shape, mounting position, orientation, etc. of the fixing device 30 are not limited to those shown in FIG. 1. As an example, the fixing device 30 may be installed on the upper surface of the body 11. In the example of FIG. 1, the fixing device 30 is installed in a position in which the longitudinal direction of the rod-shaped member faces the x-axis direction, but the fixing device 30 may be installed in a position in which the longitudinal direction of the rod-shaped member faces another direction (for example, the z-axis direction). A fixing device that is easy to attach and detach, such as a carabiner or snap hook, may be attached to the lead end 41a. In this case, the fixing device 30 may be a mounting bracket that can be attached to the fixing device attached to the lead end 41a. In other words, there is a degree of freedom in the form of the fixing device 30, and various design changes are possible.

Figure 2 is a diagram showing a schematic internal structure of the lead portion 40. As shown in Figure 2, the lead portion 40 includes a lead 41 and a lead housing portion 42.

The reed 41 is a string made of an elastic material, such as a rubber string. The total length of the reed 41 is, for example, 5 m to 20 m. The total length of the reed 41 may be shorter than 5 m or longer than 20 m. The reed 41 is, for example, a flat string, has a suitable width (for example, 10 mm or more) and is less likely to twist. The reed 41 may also be coated with luminous paint, fluorescent paint, phosphorescent paint, or the like. This allows the reed 41 to be photographed by the camera unit 20 even when the site is dimly lit or in a dark area.

The lead storage section 42 is a device that stores the lead 41 so that it can be reeled out and wound up, and has a structure similar to that of a device that stores a lead used when walking a pet, for example.

The lead storage unit 42 includes a housing 43. An attachment unit 44 is formed on the exterior surface of the housing 43 for attaching a metal fitting such as a carabiner. For example, the metal fitting attached to the attachment unit 44 is attached to a belt loop or harness of the worker 2, thereby fixing the lead storage unit 42 to the waist of the worker 2.

The lead 41 is wound around a reel 45 provided inside the housing 43. An internal gear 46 is formed on the reel 45. An external gear 47 that meshes with the internal gear 46 is provided inside the internal gear 46. The external gear 47 is biased in the positive direction (clockwise in FIG. 2, in other words, in the direction in which the lead 41 is wound) by a biasing member 48 such as a coil spring.

An external force caused by the movement of the worker 2 is applied to the lead 41. If this external force is stronger than the biasing force of the biasing member 48, the external gear 47 and the internal gear 46 rotate in the opposite direction (counterclockwise in FIG. 2, in other words, in the direction in which the lead 41 is unwound) against this biasing force. As a result, the lead 41 wound around the reel 45 is unwound from the opening 49 of the housing 43.

When the external force is weaker than the force stored by unwinding the reed 41, the external gear 47 and the internal gear 46 rotate in the forward direction. This causes the reed 41 to enter the housing 43 through the opening 49 and be wound onto the reel 45.

In this way, one end (lead end 41a) of the lead 41 is attached to the main body 10 via the fixing device 30, and the other end is attached to the worker 2 via the lead storage section 42. In addition, the lead storage section 42 reels out and winds up the lead 41 in response to an external force that accompanies the movement of the object to be followed.

The lead portion 40 may be configured without the lead storage portion 42. In this case, the other end of the lead 41 is directly attached to a belt loop or harness of the worker 2. A fastener that is easy to attach and detach, such as a carabiner or snap hook, may be attached to the other end of the lead 41. In this case, the other end of the lead 41 is attached to a belt loop or harness of the worker 2 by this fastener.

Figure 3 is a block diagram showing the components built into the torso 11. In addition to the camera unit 20, the torso 11 is provided with an IMU (Inertial Measurement Unit) 50 and a control device 100.

The IMU 50 is a sensor module equipped with, for example, a nine-axis sensor (three-axis acceleration, three-axis gyro, and three-axis compass). The IMU 50 sequentially collects various information (for example, acceleration, angular velocity, angle, etc.) required to estimate the self-position of the unmanned mobile body 1. The self-position may be estimated using a GPS (Global Positioning System), a BLE (Bluetooth (registered trademark) Low Energy) beacon, Wi-Fi (registered trademark), a geomagnetic sensor, a barometric pressure sensor, an ultrasonic sensor, etc.

The control device 100 is a device that controls the unmanned vehicle 1. The control device 100 is, for example, a microcomputer, and includes, as its hardware configuration, a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), an interface for communicating with peripheral circuits, and a bus line connecting these ports.

As shown in FIG. 3, the control device 100 has, as its functional configuration, a drive instruction unit 102, a drive control unit 104, a self-position estimation unit 106, a lead detection unit 108, a position calculation unit 110, a recording unit 112, a movement destination estimation unit 114, a lead direction detection unit 116, a follow-up target distance determination unit 118, a removal determination unit 120, a follow-up target determination unit 122, a staircase determination unit 124, a form determination unit 126, a position correction unit 128, and a landing determination unit 130.

The drive instruction unit 102 generates an instruction signal for moving the unmanned mobile body 1 to the position of the worker 2 calculated (detected) based on the lead 41 shown in the captured image, and transmits the generated instruction signal to the drive control unit 104.

The drive control unit 104 drives and controls the torso unit 11 and each leg unit 12 according to the instruction signal received from the drive instruction unit 102. This causes the unmanned mobile body 1 to walk and move to the position where the worker 2 is detected. Note that, during following movement, the unmanned mobile body 1 may be sequentially driven and controlled to a posture that captures the worker 2 in front, or may be sequentially driven and controlled to a posture that faces the next destination (described in detail later).

The self-position estimation unit 106 performs SLAM processing, i.e., estimating the self-position of the unmanned mobile body 1 and creating an environmental map at the same time, based on the three-dimensional point cloud data acquired by the camera unit 20. In this embodiment, in order to improve the accuracy of the SLAM processing, the self-position estimation result by the odometry method using various information such as acceleration, angular velocity, and angle collected by the IMU 50 is also used.

Figure 4 is a flowchart showing the following movement processing executed by the control device 100 in one embodiment of the present invention. Figure 5 is a flowchart showing the following target person identification processing executed by the control device 100 in one embodiment of the present invention.

For example, when a start instruction operation is performed to instruct the worker 2 to start following movement, the following movement process and the following target person identification process are started to be executed in parallel. Also, when a stop instruction operation is performed to instruct the worker 2 to stop following movement, the following movement process and the following target person identification process are terminated. The worker 2 can perform the start instruction operation and the stop instruction operation using an operation button (not shown) provided on the remote controller or the main body unit 10.

When a start instruction operation is performed, the following movement process is started. As shown in FIG. 4, in the following movement process, first, the self-position estimation unit 106 starts executing SLAM processing (step S101). During the following movement process, SLAM processing is executed continuously to estimate the self-position of the unmanned mobile body 1 and create an environmental map.

When the execution of the SLAM process is started, the self-position estimation unit 106 defines the origin of the environmental map created by the SLAM process (i.e., the origin of the world coordinate system) and the origin of a local coordinate system with the unmanned mobile body 1 as the base point. As an example, the origin of the first acquired distance image frame (the point where the distances in each of the X, Y and Z axial directions with the camera unit 20 are zero, i.e., the position of the camera unit 20) is defined as the origin of the world coordinate system and the origin of the local coordinate system.

The origin of the world coordinate system is fixed. During the tracking movement process, the origin of the world coordinate system does not move from the origin of the first acquired distance image frame. In contrast, the origin of the local coordinate system moves. During the tracking movement process, the origin of the local coordinate system is updated to the origin of the latest distance image frame. In other words, the origin of the local coordinate system moves as the unmanned mobile body 1 moves.

When the start instruction operation is performed, the process of identifying the person to be followed also starts. As shown in FIG. 5, in the process of identifying the person to be followed, first, an initial value is set by the lead detection unit 108 (step S201). Specifically, in order to detect a specific line (specifically, the lead 41) that appears in the captured image, the lead detection unit 108 sets the position of the lead end 41a in the captured image and sets a threshold value for the captured image based on the color of the line to be detected.

The fixture 30 to which the lead end 41a is fastened is always captured at a fixed position (bottom center) in the captured image. Therefore, the lead end 41a is also always captured at a fixed position in the captured image. Therefore, in this embodiment, the position of the tip of the fixture 30 to which the lead end 41a is fastened is set as the position of the lead end 41a. The position of the tip of the fixture 30 is known from the positional relationship between the camera unit 20 and the fixture 30.

The lead detection unit 108 defines, for example, in the YCbCr color space, a predetermined area (e.g., an area in the YCbCr color space that can be regarded as red) centered on a plot corresponding to the color of the line to be detected (i.e., the color of the lead 41, e.g., red) The lead detection unit 108 sets the value of the Cr component that defines this area as a threshold value for detecting the line.

The lead detection unit 108 acquires an RGB image from the RGB-D image captured by the camera unit 20 (step S202). The lead detection unit 108 converts the acquired RGB image into a YCbCr image (step S203). The lead detection unit 108 binarizes the YCbCr image based on the threshold value (step S204).

Figure 6 shows an example of a binary image generated in step S204. In the example of Figure 6, pixels whose Cr component values are within the threshold (values that fall within the above-mentioned specified range) have their pixel values converted to a white value (e.g., 1). Also, pixels whose Cr component values are outside the threshold (values that do not fall within the above-mentioned specified range) have their pixel values converted to a black value (e.g., 0).

The color of the lead 41 shown in the captured image may change depending on the light source. Therefore, the threshold value used in the binarization process may be dynamically changed depending on, for example, the color of the lead end 41a shown in the captured image.

The lead detection unit 108 detects a line L (see FIG. 6) extending from the position of the lead end 41a set in step S201 as a start end position L _S to a terminal end position L _E from the binary image generated in step S204 by using a Hough transform (step S205). That is, the lead detection unit 108 detects a line L including the other end (terminal position L _E ) of the lead 41 from within the captured image.

The end position _LE of the line L is the portion of the lead 41 that is drawn out from the opening 49 to the outside of the housing 43 that is closest to the opening 49. In this case, the worker 2 with the lead housing 42 attached to his/her waist should be present at the end position _LE of the line L. The position calculation unit 110 then determines whether or not a person is present at the end position _LE (step S206). For example, the position calculation unit 110 estimates the shape and size of an object that is present at the end position _LE based on the distance information acquired by the camera unit 20, and determines whether or not the object is a person by pattern matching.

When it is determined that the object shown in the terminal position _LE is a person (i.e., the worker 2) (step S206: YES), the position calculation unit 110 calculates the position of the worker 2 and records it in the recording unit 112 (step S207). For example, the position calculation unit 110 calculates and records the intersection of the foot of the worker 2 shown in the terminal position _LE and the walking surface (ground or floor surface) as the position of the worker 2.

In this way, the position calculation unit 110 calculates and records the position of the worker 2 based on the other end (terminal position L _E ) of the lead 41 detected by the lead detection unit 108. The position of the worker 2 is recorded using X and Y coordinate values in the world coordinate system. For convenience, the position of the worker 2 recorded in the recording unit 112 is referred to as a "walking recorded position."

If it is determined that the object at the terminal position _LE is not a human (or no object is detected at the terminal position _LE ) (step S206: NO), the walking record position is not calculated and recorded, and the process of identifying the person to be followed proceeds to step S208.

As will be described in more detail later, if the unmanned mobile body 1 loses sight of the worker 2, the destination estimation unit 114 estimates the destination of the worker 2 based on the extension direction of the lead 41 shown in the captured image. Therefore, in the process of identifying the person to be followed, the extension direction of the lead 41 is detected by the lead direction detection unit 116.

Here, the lead 41 is elastic and is stored in the lead storage section 42 so that it can be reeled out and wound up in response to external forces associated with the movement of the worker 2. Therefore, the lead 41 is basically in a taut state while the unmanned mobile body 1 is moving in a following manner. Therefore, the line L detected by the Hough transform is usually a straight line. In this case, the lead direction detection section 116 detects the extension direction of the line L as the extension direction of the lead 41.

In contrast, in a situation where the distance between the unmanned vehicle 1 and the worker 2 is rapidly reduced, for example, the reed storage unit 42 cannot keep up with the winding speed of the reed 41, and the reed 41 may slacken in the direction of gravity. In this case, the line L detected by the Hough transform is not detected as a straight line but as a curve. In this case, the reed direction detection unit 116 cannot detect the extension direction of the reed 41.

Then, the lead direction detection unit 116 judges whether or not the lead 41 shown in the captured image is loose (step S208). For example, if the line L detected by the Hough transform is a straight line, it is judged that the lead 41 is not loose. If the line L detected by the Hough transform is not a straight line, it is judged that the lead 41 is loose.

If it is determined that the lead 41 is loose (step S208: YES), the lead direction detection unit 116 detects the extension direction of the line segment connecting the start position L _S and the end position L _E as the extension direction of the lead 41 (step S209).

If it is determined that the lead 41 is not loose (step S208: NO), the lead direction detection unit 116 detects the extension direction of the straight line L as the extension direction of the lead 41 (step S210).

The lead direction detection unit 116 records the extension direction of the lead 41 detected in step S209 or step S210 as three-dimensional XYZ vector data in the recording unit 112 in association with the walking record position recorded in step S207 (step S211). Note that if the walking record position has not been calculated and recorded, the process of step S211 is not performed. Next, the following target person identification process returns to step S202, and the processes from step S202 onwards are executed.

Returning to the explanation of the following movement process in FIG. 4, the following target distance determination unit 118 determines whether the distance between the object determined to be the worker 2 in step S206 in FIG. 5 and the unmanned mobile unit 1 is greater than or equal to a predetermined distance (e.g., 50 cm) based on the distance information acquired by the camera unit 20 (step S102).

When it is determined that the distance from the operator 2 is 50 cm or more (step S102: YES), the drive instruction unit 102 designates the next walking record position as the next destination of the unmanned mobile body 1 and generates an instruction signal to move to the designated destination (step S103). In accordance with this instruction signal, the drive control unit 104 controls the drive of the torso unit 11 and each leg unit 12, causing the unmanned mobile body 1 to move toward the next destination (step S104). Here, the "next walking record position" is the walking record position that is recorded in the recording unit 112 earliest in the time series among the walking record positions that have not yet been designated as the destination of the unmanned mobile body 1.

The following target distance determination unit 118 constantly detects the distance between the unmanned mobile body 1 and the worker 2 while the following movement process is being executed. The drive instruction unit 102 adjusts the speed instruction information included in the instruction signal to the drive control unit 104 so that the distance detected by the following target distance determination unit 118 is always kept at 50 cm or more. This speed instruction information controls the movement speed of the unmanned mobile body 1 so that the distance to the worker 2 is kept at 50 cm or more.

In addition, the movement speed of the unmanned mobile body 1 not only changes according to the movement speed of the worker 2, but also changes according to the distance detected by the following target distance determination unit 118. As an example, the movement speed of the unmanned mobile body 1 increases as the distance to the worker 2 increases, and decreases as the distance to the worker 2 decreases. In addition, the movement speed of the unmanned mobile body 1 increases when the unmanned mobile body 1 loses sight of the worker 2. When the unmanned mobile body 1 loses sight of the worker 2, it may be configured to move at its maximum speed.

When it is determined that the distance to the worker 2 is less than 50 cm (step S102: NO), the drive instruction unit 102 determines whether or not a stop instruction operation has been performed (step S112). When it is determined that a stop instruction operation has been performed (step S112: YES), the following movement process ends. When it is determined that a stop instruction operation has not been performed (step S112: NO), the following movement process proceeds to step S103.

Depending on the movement conditions of the unmanned mobile body 1 and the worker 2, the distance between them may temporarily become quite close. In order to avoid a collision between the two, the unmanned mobile body 1 may be forced to stop when the distance between it and the worker 2 becomes 30 cm or less.

The detachment determination unit 120 also constantly determines whether the lead 41 (more precisely, the lead portion 40) has been detached from the worker 2 while the following movement process is being performed. When the lead portion 40 is detached from the worker 2, the lead 41 is wound onto the reel 45 by the biasing force of the biasing member 48. By winding the lead 41 onto the reel 45, the lead storage portion 42 is suspended in front of the body portion 11, and the distance between the unmanned mobile body 1 and the lead storage portion 42 becomes, for example, 30 cm or less.

Therefore, when the distance between the end position _LE of the line L detected by the lead detection unit 108 and the unmanned mobile body 1 is 30 cm or less and the worker 2 is not detected by pattern matching by the position calculation unit 110, the detachment determination unit 120 determines that the lead 41 has been detached from the worker 2. In this case, the unmanned mobile body 1 stops following the worker 2.

During the movement to the destination designated in step S103, the follow-up target determination unit 122 determines whether or not the worker 2 has been lost (step S105). For example, the follow-up target determination unit 122 determines whether or not the worker 2 has been lost based on whether or not the next walking record position has been recorded in the recording unit 112.

If it is determined that the operator 2 has not been lost (step S105: NO) and that the distance between the operator 2 and the operator 2 is 50 cm or more (step S102: YES), the drive instruction unit 102 designates the next walking record position recorded in the recording unit 112 as the next destination, and generates an instruction signal to move to this destination (step S103). As a result, the unmanned mobile body 1 moves toward the destination designated in the current step S103 after reaching the destination designated in the previous step S103 (step S104). By repeatedly executing steps S102 to S105, the unmanned mobile body 1 moves through the walking record positions in order to follow the operator 2.

If it is determined that the worker 2 has been lost (step S105: YES), the recording unit 112 records the position of the worker 2 only up to the next walking record position (next destination).

When the unmanned mobile body 1 reaches its next destination, the destination estimation unit 114 estimates the destination of the worker 2 based on the extension direction of the lead 41 recorded in the recording unit 112 in association with the walking record position corresponding to the reached destination (step S106). The drive control unit 104 rotates the unmanned mobile body 1 according to the instruction signal from the drive instruction unit 102 so that the estimated destination is within the angle of view of the camera unit 20 (step S107). The instruction signal for rotation includes control information for controlling the attitude angle (yaw angle, roll angle, pitch angle) of the unmanned mobile body 1, for example.

Here, Fig. 7 shows a diagram supplementing the explanation of steps S102 to S107. In Fig. 7, symbols _P1 to _P4 indicate walking record positions. These walking record positions are recorded in the recording unit 112 in the order of walking record positions _P1 , _P2 , _P3 , and _P4 . Therefore, in Fig. 7A, walking record position _P1 is the next movement destination of the unmanned mobile body 1.

By repeatedly executing steps S102 to S105, the unmanned mobile body 1 moves through walking record positions _P1 , _P2 , and _P3 in order to follow the worker 2. Here, the worker 2 is ahead of a corner that is a blind spot for the unmanned mobile body 1. Therefore, the recording unit 112 has not recorded a walking record position next to walking record position P4. Therefore, during the movement to walking record position _P4 , the following target determination unit 122 determines that the worker ₂ has been lost (step S105: YES).

When the unmanned mobile body 1 reaches the walking record position _P4 (see FIG. 7B), the destination estimation unit 114 estimates the destination of the worker 2 based on the extension direction of the lead 41 recorded in association with the walking record position _P4 (step S106). Here, it is assumed that the direction diagonally forward to the right is recorded in the recording unit 112 as the extension direction of the lead 41, similar to the line L in FIG. 6. In this case, in step S107, the drive control unit 104 rotates the unmanned mobile body 1 according to the instruction signal from the drive instruction unit 102 so that the destination estimated in step S106 is within the angle of view of the camera unit 20 (see FIG. 7C).

After the unmanned mobile body 1 turns, as shown in Figure 7C, when the worker 2 is in a position that fits within the angle of view of the camera unit 20, this position is recorded in the recording unit 112 as a walking record position by the following target person identification process that is being executed in parallel with the following movement process.

The follow-up target determination unit 122 determines whether or not the worker 2 is present at the destination estimated in step S106 based on whether or not the next walking record position has been recorded (step S108). If the worker 2 is present, the walking record position is recorded by the follow-up target person identification process as described above. This results in a YES determination. In this case, the follow-up movement process returns to step S102, and the processes from step S102 onwards are executed. If the worker 2 is not present, in other words, if the worker 2 is not captured at the destination captured within the angle of view (step S108: NO), the follow-up movement process proceeds to step S109.

In step S109, the position calculation unit 110 calculates the tentative position of the worker 2 based on the extension direction of the lead 41 detected at this time by the tracking target person identification process as a walking record position and records it in the recording unit 112.

An example of a method for calculating the tentative position of the worker 2 will be described with reference to Fig. 8. In the example of Fig. 8, the worker 2 is around a corner that is a blind spot for the unmanned mobile body 1. The captured image does not show the part of the lead 41 that is around the corner. In other words, the captured image shows only the part of the lead 41 that corresponds to the corner AA of the wall. Therefore, in step S205 of the processing for identifying the person to be followed, the part of the lead 41 that corresponds to the corner AA of the wall is detected as the end position _LE of the line L.

The position calculation unit 110 calculates a tentative position _P0 as a position on a line extending in a predetermined angular direction (for example, a direction forming 135 degrees with respect to wall surface BB as shown in FIG. 8 ) with respect to corner AA of the wall (in other words, terminal position L _{E )} , and which is a distance D away from corner AA, which is the sum of the avoidance distance, width, and a predetermined margin of the unmanned mobile body 1. Note that the unmanned mobile body 1 moves with at least a predetermined distance from objects such as walls so as not to collide with them, while referring to distance information acquired by the camera unit 20, a scanner-type laser distance sensor, etc. This predetermined distance is the "avoidance distance."

The provisional position calculated as the walking record position corresponds to the next walking record position. Therefore, the drive instruction unit 102 designates the provisional position as the next movement destination of the unmanned mobile body 1, and generates an instruction signal for moving to the provisional position (step S110). In accordance with this instruction signal, the drive control unit 104 controls the drive of the body unit 11 and each leg unit 12, so that the unmanned mobile body 1 moves toward the provisional position (step S111). When the unmanned mobile body 1 reaches the provisional position, the following movement process returns to step S106, and the processes from step S106 onwards are executed.

The unmanned mobile body 1 can ascend and descend stairs by individually driving each leg 12. Figure 9 is a flowchart showing the stair movement process executed by the control device 100 when the unmanned mobile body 1 moves up and down stairs following the worker 2.

The staircase movement process is also executed in parallel with the following movement process and the following target person identification process. That is, when a start instruction operation is performed, the staircase movement process is also started, and when a stop instruction operation is performed, the staircase movement process is also ended.

The staircase determination unit 124 determines whether or not there are stairs in the vicinity of the unmanned mobile body 1 (step S301). For example, whether or not there are stairs in the vicinity of the unmanned mobile body 1 is determined based on whether or not stairs are captured in the captured image. Whether or not an object captured in the captured image is a staircase can be determined based on, for example, the self-position estimated by SLAM processing and the created environmental map.

When it is determined that there are stairs near the unmanned mobile body 1 (step S301: YES), the staircase determination unit 124 determines whether the worker 2 will move up or down the stairs (step S302). For example, it determines whether the worker 2 will move up the stairs depending on whether the extension direction of the lead 41 detected by the lead direction detection unit 116 at the time of this determination process is a direction pointing to the stairs. In addition, it may be determined whether the worker 2 will move up the stairs depending on whether the walking record position specified as the next destination of the unmanned mobile body 1 in step S103 of FIG. 4 is located on the stairs.

When it is determined that the worker 2 is moving up the stairs (step S302: YES), the drive control unit 104 controls the drive of the torso unit 11 and each leg unit 12 according to the instruction signal received from the drive instruction unit 102 so that the unmanned mobile body 1 faces the stairs (step S303). The unmanned mobile body 1 changes its posture and direction by rotating and moving in parallel through the drive control by the drive control unit 104, and faces the stairs.

Stairs in which the depth dimension of the tread surface is constant in the width direction of the board surface (straight stairs, folded stairs, return stairs, etc.) are referred to as "normal stairs". Stairs in which the depth dimension of the tread surface is not constant in the width direction of the board surface (spiral stairs, etc.) are referred to as "non-normal stairs". For example, in a spiral staircase, the depth of the tread surface is narrower on the inner side of the spiral and wider on the outer side of the spiral. Therefore, if the unmanned mobile unit 1 tries to walk on the inner side of the spiral, there is a risk of it stepping off the step of the stairs.

Then, the form determination unit 126 determines the form of the stairs (step S304). More specifically, the form determination unit 126 determines whether the stairs facing the unmanned mobile body 1 are non-normal stairs or not, based on the shape of the stairs on the environmental map created by the SLAM process.

When it is determined that the staircase that the unmanned mobile body 1 faces is a normal staircase (step S304: NO), the unmanned mobile body 1 moves up the normal staircase by passing through each walking record position on the normal staircase calculated and recorded in step S207 of Figure 5 in order (step S305).

On the other hand, if the staircase facing the unmanned mobile body 1 is a non-normal staircase (step S304: YES), the position correction unit 128 corrects each walking record position on the non-normal staircase calculated and recorded in step S207 of FIG. 5 (step S306). For example, the position correction unit 128 changes each walking record position on the non-normal staircase to a position with a larger depth on the tread surface. For example, in the case of a spiral staircase, the position is changed to a position closer to the outer periphery of the spiral. More specifically, the position is changed from the outermost periphery of the spiral to a position closer to the inner periphery of the spiral by a distance that takes into account the avoidance distance, width, and a specified margin of the unmanned mobile body 1, so as not to collide with an object (such as a wall on the outer periphery). As a result, the unmanned mobile body 1 moves through each corrected walking record position in order on the non-normal staircase (step S307).

The staircase determination unit 124 determines whether the unmanned mobile body 1 has finished moving up the stairs based on the self-position estimated by the SLAM process and the created environmental map (step S308). If it is determined that the unmanned mobile body 1 is moving up the stairs (step S308: NO), the processes of steps S304 to S308 are repeatedly executed.

When it is determined that the unmanned mobile body 1 has finished moving up the stairs (step S308: YES), the tracking target determination unit 122 determines whether or not the worker 2 has been lost, similar to step S105 in FIG. 4 (step S309). When it is determined that the worker 2 has not been lost (step S309: NO), the stair movement process returns to step S301. The process of step S301 is repeatedly executed until it is determined that there are stairs in the vicinity of the unmanned mobile body 1.

When it is determined that the worker 2 has been lost (step S309: YES), the destination estimation unit 114 estimates the destination of the worker 2 (step S310), similar to step S106 in FIG. 4. The drive control unit 104 rotates the unmanned mobile unit 1 so that the estimated destination is within the angle of view of the camera unit 20 (step S311), similar to step S107 in FIG. 4.

In light of the situation where the unmanned mobile body 1 loses sight of the worker 2 when it finishes moving up the stairs, there is a possibility that the current position of the unmanned mobile body 1 is a landing. Therefore, the landing determination unit 130 determines whether the current position of the unmanned mobile body 1 is a landing (step S312).

For example, the landing determination unit 130 determines whether the extension direction of the lead 41 in the Z-axis direction detected by the lead direction detection unit 116 at the time of this determination process satisfies a predetermined condition. For example, the predetermined condition is satisfied when the extension direction of the lead 41 in the Z-axis direction is a direction that forms an angle of a predetermined degree or more upward with respect to the horizontal plane, or a direction that forms an angle of a predetermined degree or more downward with respect to the horizontal plane. If the extension direction of the lead 41 in the Z-axis direction is a direction that forms an angle of a predetermined degree or more upward with respect to the horizontal plane, it is presumed that the worker 2 is ascending the stairs. If the extension direction of the lead 41 in the Z-axis direction is a direction that forms an angle of a predetermined degree or more downward with respect to the horizontal plane, it is presumed that the worker 2 is descending the stairs.

If the extension direction of the lead 41 in the Z-axis direction meets a predetermined condition, the landing determination unit 130 determines that the landing between the staircase that the unmanned mobile body 1 has just traveled and the next staircase is the current position of the unmanned mobile body 1 (step S312: YES). In this case, the drive control unit 104 changes the posture and direction of the unmanned mobile body 1 by rotating or translating it, as in step S303, so that the unmanned mobile body 1 faces the next staircase to be traveled (step S313). Next, the staircase travel process returns to step S304, and the processes from step S304 onwards are executed.

If it is determined that the extension direction of the lead 41 in the Z-axis direction does not satisfy the predetermined condition (step S312: NO), it is presumed that the current position of the unmanned mobile body 1 is not a landing, but a floor after stair movement. In this case, the stair movement process returns to step S301. The process of step S301 is repeatedly executed until it is determined that there are stairs near the unmanned mobile body 1.

As described above, according to the unmanned mobile body 1 of this embodiment, the position of the worker 2 is calculated based on the other end of the lead 41 that appears in the captured image. This makes it possible to accurately calculate the position of the worker 2 regardless of whether the lead 41 is pulled or not, or whether the lead 41 is taut or not.

According to the unmanned mobile body 1 of this embodiment, even if a large number of people are captured in a captured image, the worker 2 can be identified by the lead 41. This prevents erroneous recognition of the worker 2.

The unmanned mobile body 1 according to this embodiment moves by following the worker 2 whose position has been calculated with high accuracy, so there is no need to operate the movement using a controller. The worker 2 can move around the construction site or civil engineering site with a high level of attention without being distracted by the movement operation.

According to the unmanned mobile body 1 of this embodiment, even if the worker 2 is lost, the destination of the worker 2 can be estimated with high accuracy based on the extension direction of the lead 41. Therefore, even if the worker 2 is lost, the unmanned mobile body 1 can continue to move by following the worker 2.

The above is an explanation of exemplary embodiments of the present invention. The embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical concept of the present invention. For example, the embodiments of the present invention also include appropriate combinations of embodiments explicitly shown as examples in the specification or obvious embodiments.

1: Unmanned mobile body 10: Main body 11: Torso 12: Legs 20: Camera 30: Fixing device 40: Lead 41: Lead 42: Lead storage 43: Housing 44: Attachment 45: Reel 46: Internal gear 47: External gear 48: Pressing member 50: Power button 100: Control device 102: Drive instruction unit 104: Drive control unit 106: Self-position estimation unit 108: Lead detection unit 110: Position calculation unit 112: Recording unit 114: Movement destination estimation unit 116: Lead direction detection unit 118: Target distance determination unit 120: Removal determination unit 122: Target determination unit 124: Staircase determination unit 126: Form determination unit 128: Position correction unit 130: Landing determination unit

Claims (10)

In an unmanned moving body that moves by following a target,
A main body portion,
A lead having one end attached to the main body and the other end attached to the target to be followed;
a camera unit provided on the main body unit and configured to capture the lead within an angle of view;
a lead detection unit that detects the other end of the lead attached to the tracking target from within the captured image taken by the camera unit;
a position calculation unit that calculates a position of the tracking target based on the other end of the lead detected by the lead detection unit,
The target object is moved to the position calculated by the position calculation unit.
Unmanned mobile vehicle. a tracking target determination unit that determines whether the tracking target has been lost;
a lead direction detection unit for detecting an extension direction of the lead shown in the captured image;
a destination estimation unit that estimates a destination of the target to be tracked based on the extension direction detected by the lead direction detection unit when the target to be tracked is determined to have been lost by the target to be tracked.
The destination estimated by the destination estimation unit is placed within an angle of view of the camera unit.
The unmanned vehicle according to claim 1. The image captured by the camera unit is a color image,
the lead direction detection unit sets a threshold value for the photographed image based on a color of the lead, binarizes the photographed image based on the set threshold value, and detects an extension direction of the lead based on the binarized photographed image.
The unmanned vehicle according to claim 2. When the target object is not captured in the movement destination included in the angle of view, the position calculation unit calculates a tentative position based on the extension direction detected by the lead direction detection unit as the position of the target object.
4. An unmanned moving body according to claim 2 or 3. the lead direction detection unit determines whether the lead shown in the captured image is loose, and when it is determined that the lead is loose, detects an extension direction of a line segment connecting the one end and the other end of the lead as an extension direction of the lead.
An unmanned moving body according to any one of claims 2 to 4. The lead is elastic.
An unmanned moving body according to any one of claims 1 to 5. a lead housing section that reels out and reels in the lead in response to an external force caused by the movement of the object to be followed;
An unmanned moving body according to any one of claims 1 to 6. a detachment determination unit that determines whether the lead has been detached from the tracking target,
when the detachment determination unit determines that the lead has been detached from the tracking target, the tracking of the tracking target is stopped.
An unmanned moving body according to any one of claims 1 to 7. a staircase determination unit that determines whether the target to be tracked moves on a staircase;
a shape determination unit that determines a shape of the stairs when the staircase determination unit determines that the target to be tracked moves on the stairs;
a position correction unit that corrects the position of the target on the stairs calculated by the position calculation unit in accordance with the shape of the stairs determined by the shape determination unit,
An unmanned moving body according to any one of claims 1 to 8. the staircase determination unit determines whether the target to be tracked moves on the staircase depending on whether an extension direction of the lead detected by the lead direction detection unit is a direction pointing to the staircase.
An unmanned vehicle according to claim 9 which recites claim 2.

Images