JP7495293B2 - Autonomous mobile work device and program - Google Patents

Description

The present invention relates to an autonomous mobile work device and a program.

Conventionally, robot devices that set a work area by teaching are known (see, for example, Patent Document 1). The robot device described in Patent Document 1 uses a stereo camera to detect an operator moving forward, and acquires teaching data for a travel route by following the operator's movements. When the robot device travels autonomously according to the teaching data for the travel route, an ultrasonic sensor detects an obstacle, and the teaching data for the travel route is corrected so as to avoid the obstacle. A work area in which the robot device can travel autonomously is automatically set according to the corrected travel route.

There is also known a navigation device that remotely sets a work area for a mobile object (see, for example, Patent Document 2). In the navigation device described in Patent Document 2, a webcam installed indoors is wirelessly connected to a mobile object moving within a floor. The webcam captures an image of the floor, and a floor image is displayed on the monitor of the navigation device. Markers representing the travel route of the mobile object can be placed on the floor image, and a navigation image in which the mobile object travels autonomously is created based on the placement of the markers. A work area in which the mobile object can travel autonomously is set by this navigation image.

JP 2006-185438 A International Publication No. 2010/044277

However, the robot device described in Patent Document 1 requires the operator to repeatedly move around to acquire teaching data, and in the case of a large floor area such as a commercial facility, the operator has to move long distances, which places a heavy burden on the operator. The navigation device described in Patent Document 2 allows the operator to instruct the movement of a moving object while checking it remotely, but it is difficult to place markers on the floor image while the moving object is traveling. In addition, the operator cannot take his or her eyes off the moving object displayed on the monitor, which places a heavy burden on the operator.

Therefore, the present invention has been made in consideration of these points, and aims to provide an autonomous mobile work device and program that can easily set a work area while reducing the workload of the operator.

The autonomous mobile work device of the present invention is an autonomous mobile work device that performs work on a work surface while autonomously traveling, and includes an acquisition unit that acquires an image from an imaging unit that images the work surface, a recognition unit that recognizes markers present on the work surface from the image, a creation unit that creates a movement trajectory of the marker from chronologically consecutive captured images, and a setting unit that sets the inside of the movement trajectory of the marker as a work area.

According to this configuration, by moving the marker on the work surface, the imaging unit captures an image of the marker moving on the work surface. A marker movement trajectory is created from chronologically consecutive captured images, and a work area is automatically set inside the marker movement trajectory. Thus, the work area can be set by the simple task of moving the marker on the work surface, reducing the workload of the operator.

The autonomous mobile work device of the present invention is equipped with a correction unit that corrects the perimeter of the work area so that the distance from the barrier is equal to or greater than the safe distance. This configuration makes it possible to avoid collisions between the autonomous mobile work device and the barrier.

The autonomous mobile work device of the present invention is equipped with a determination unit that determines a travel route that passes through the entire work area by repeating round trip travel. With this configuration, the travel route within the work area is automatically determined, reducing the workload of the operator.

In the autonomous mobile work device of the present invention, the setting unit can set the inside of the movement trajectory of the marker as a non-work area. With this configuration, it is possible to set the work area and the non-work area according to the condition of the work surface.

In the autonomous mobile work device of the present invention, the setting unit can reset a new work area by combining multiple work areas, and can reset a new work area by combining a work area and a non-work area. With this configuration, a new work area can be set according to the condition of the work surface by combining work areas together and by combining a work area and a non-work area.

In the autonomous mobile work device of the present invention, the marker is provided on a manual work tool. With this configuration, the marker provided on the work tool moves when the operator works using the work tool. Therefore, while the operator is working, the autonomous mobile work device can create a movement trajectory of the marker and automatically set the work area.

In the autonomous mobile work device of the present invention, the marker is a light source or a preregistered characteristic shape. With this configuration, the autonomous mobile work device can easily recognize the marker.

In the autonomous mobile work device of the present invention, the recognition unit recognizes the marker during autonomous travel. With this configuration, the autonomous mobile work device recognizes the marker while traveling autonomously, and thus it is possible to set a work area in parallel with the work of the autonomous mobile work device, thereby improving work efficiency.

In the autonomous mobile work device of the present invention, when the setting unit sets multiple work areas on the work surface, a different work plan is applied to each work area. With this configuration, work suitable for the conditions of the work area can be performed.

In the autonomous mobile work device of the present invention, the imaging unit captures images in all directions. This configuration makes it possible to eliminate blind spots when capturing images of the surface to be cleaned.

The autonomous mobile work device of the present invention includes a storage unit that stores an environmental map of the work surface that has been created in advance, the recognition unit recognizes the position of the marker on the work surface from a captured image, associates feature points of the environmental map with feature points of the captured image, and reflects the position of the marker on the environmental map, the creation unit creates a movement trajectory of the marker by connecting the positions of the multiple markers reflected on the environmental map, and the setting unit sets the inside of the movement trajectory of the marker on the environmental map as the work area. With this configuration, by reflecting the movement trajectory of the marker on the environmental map, the work area can be easily set on the environmental map.

The program of the present invention is a program for an autonomous mobile work device that performs work on a work surface while traveling autonomously, and causes the autonomous mobile work device to execute the steps of acquiring an image from an imaging unit that images the work surface, recognizing markers present on the work surface from the image, creating a movement trajectory of the marker from chronologically consecutive captured images, and setting the inside of the movement trajectory of the marker as a work area. With this configuration, the autonomous mobile work device can install the program, thereby giving the autonomous mobile work device a function for setting a work area, thereby reducing the workload of the operator.

The present invention makes it possible to easily set the work area while reducing the workload of the operator.

FIG. 2 is a perspective view of the cleaning device according to the embodiment. FIG. 2 is a schematic diagram of a cleaning device according to the present embodiment. FIG. 2 is a control block diagram of the device main body of the embodiment. FIG. 2 is a diagram showing an example of a cleaning tool according to the present embodiment. 11 is a flowchart of a first setting example of the present embodiment. FIG. 2 is a diagram showing an example of a captured image according to the present embodiment. FIG. 11 is a transition diagram of a first setting example of the present embodiment. 11 is a flowchart of a second setting example of the present embodiment. FIG. 11 is a transition diagram of a second setting example of the present embodiment. FIG. 11 is a transition diagram of a third setting example of the present embodiment. FIG. 11 is a transition diagram of a fourth setting example of the present embodiment.

The cleaning device of this embodiment will be described below with reference to the drawings. FIG. 1 is a perspective view of the cleaning device of this embodiment. FIG. 2 is a schematic diagram of the cleaning device of this embodiment. In the following description, an autonomously traveling cleaning device will be used as an example of the autonomously traveling work device, but the autonomously traveling work device may be a work device that performs various tasks such as cleaning, polishing, polishing, and waxing on a work surface by autonomous traveling.

As shown in FIG. 1, the cleaning device 1 is configured to be able to autonomously clean floor surfaces (work surfaces) in commercial facilities, manufacturing facilities, concourses, etc. Front wheels 12 as drive wheels and rear wheels 13 as auxiliary wheels are provided at the bottom of the device body 10 of the cleaning device 1. Cleaning members 16 such as cleaning pads and cleaning brushes are provided between the front wheels 12 and rear wheels 13, and a squeegee 17 is provided behind the cleaning members 16. A handle 28 is provided at the upper rear side of the device body 10, and a display unit 22 is provided in front of the handle 28. An imaging unit 21 is provided at the upper front side of the device body 10, and an obstacle sensor 26 is provided in a recess below the imaging unit 21.

As shown in FIG. 2, the device body 10 of the cleaning device 1 is provided with a running unit 11, a cleaning unit 15, a wireless communication unit 19, an imaging unit 21, a display unit 22, a measuring unit 25, an operation unit 27, a memory unit 50, a power supply unit 29, and a control unit 30. The running unit 11 is composed of the above-mentioned front wheels 12, rear wheels 13, a running motor (not shown), an encoder (not shown), etc. The running unit 11 drives the front wheels 12 with the running motor to run the cleaning device 1. Note that, although a front-wheel-drive cleaning device 1 is exemplified in this embodiment, only the rear wheels 13 may be driven by the running unit 11, or both wheels may be driven by the running unit 11.

The cleaning unit 15 cleans the floor surface F, for example, by wet cleaning, and is composed of the above-mentioned cleaning member 16 and squeegee 17, a cleaning motor (not shown), an actuator (not shown), a cleaning liquid supply unit (not shown), a wastewater recovery unit (not shown), etc. The cleaning unit 15 presses the cleaning member 16 against the floor surface F using the actuator, and rotates the cleaning member 16 relative to the floor surface F using the cleaning motor. At this time, cleaning liquid is sprayed from a cleaning liquid tank onto the floor surface F by the cleaning liquid supply unit, and wastewater collected by the squeegee 17 is collected in a wastewater tank by the wastewater recovery unit.

The wireless communication unit 19 connects the device main body 10 and the management terminal (not shown) via wireless communication. For example, the device main body 10 and the management terminal are connected via a wireless LAN such as WiFi (registered trademark). The imaging unit 21 is a dome-shaped camera capable of capturing an image of the floor surface F. The imaging unit 21 is provided with a rotation mechanism and a height adjustment mechanism, and captures images in all directions with the imaging unit 21 at the center, eliminating blind spots on the floor surface F. This makes it possible to capture images of objects regardless of the traveling direction of the cleaning device 1. The imaging unit 21 may be an omnidirectional camera capable of capturing panoramic images, or a fixed camera capable of capturing images in front of the cleaning device 1.

The display unit 22 has a touch panel type display screen. The display screen of the display unit 22 displays the captured image captured by the imaging unit 21, as well as an operation screen that accepts operations by the operator for the cleaning device 1. The operation screen allows the operation mode of the cleaning device 1 to be selected, and various settings to be input for the cleaning device 1. Note that a tablet terminal or a mobile terminal such as a smartphone that is detachable from the device main body 10 can also be used as the display unit 22. When a mobile terminal is used as the display unit 22, the wireless communication function of the mobile device may be used instead of the wireless communication unit 19.

The measuring unit 25 includes the obstacle sensor 26 described above. The obstacle sensor 26 is, for example, configured with an LRF (Laser Range Finder) that measures the distance and angle from an obstacle or a wall surface. Laser light is emitted from the obstacle sensor 26 into the surroundings, and the distance and angle from the obstacle sensor 26 to an obstacle is measured by receiving reflected light from an object, making it possible to recognize the surrounding environment of the cleaning device 1. Furthermore, the obstacle sensor 26 may be configured to be able to perform obstacle detection using a captured image of the surrounding environment instead of obstacle detection using laser light.

The operation unit 27 receives manual operation by the operator and is composed of the above-mentioned handle 28 (see FIG. 1), throttle, etc. The operation unit 27 converts the steering amount of the handle 28 and the twist amount of the throttle into an electrical signal, which is output to the control board of the travel motor of the device main body 10 to drive the travel motor. The operation of the operation unit 27 adjusts the travel speed of the device main body 10 and allows left and right turns. In addition, a power switch (not shown) for turning the power ON/OFF and an emergency stop switch (not shown) for completely stopping the operation of the device main body 10 are provided near the operation unit 27 of the device main body 10.

The memory unit 50 is composed of one or more storage media such as a ROM (Read Only Memory), a RAM (Random Access Memory), a HDD (Hard Disk Drive), a flash memory, etc., depending on the application. The ROM stores, for example, a program for controlling the cleaning device 1. The HDD or flash memory stores, for example, a cleaning plan (operation plan) that sets the autonomous driving route, driving speed, amount of cleaning liquid sprayed, contact pressure of the cleaning member 16, etc., and other parameters. The power supply unit 29 is composed of a battery (not shown), a charging circuit, etc. Power is supplied to each part of the device from the power supply unit 29.

The control unit 30 controls each part of the device. The various processes of the control unit 30 may be realized by software using a processor, or may be realized by a logic circuit (hardware) formed in an integrated circuit or the like. When a processor is used, the processor reads out and executes programs stored in the storage unit 50 to perform various processes. For example, a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) is used as the processor. The control blocks of the control unit 30 and the storage unit 50 will be described in detail later.

Such a cleaning device 1 is provided with a learning mode in which a cleaning plan is taught to the cleaning device 1, and a reproduction mode in which the cleaning device 1 reproduces the cleaning plan. In the learning mode, the operator manually operates the cleaning device 1 to make the cleaning device 1 memorize the cleaning area (work area). In the reproduction mode, the cleaning device 1 cleans while autonomously traveling within the cleaning area. However, in order to set the cleaning area in the learning mode, the cleaning device 1 must be operated so that the cleaning members 16 pass through the entire cleaning area, which is a cumbersome task for the operator.

However, it is difficult for the cleaning device 1 to safely clean walls and corners in order to avoid collisions between the cleaning device 1 and barriers such as walls and pillars. Usually, these areas must be cleaned manually using a manual cleaning tool 60 (see FIG. 4) such as a mop. Since the cleaning tool 60 is moved so as to surround the cleaning area, if the movement trajectory of the cleaning tool 60 can be recognized, it is possible to set the inside of the movement trajectory as the cleaning area. Therefore, the cleaning device 1 of this embodiment is provided with an area setting mode that recognizes a marker 63 (see FIG. 4) attached to the cleaning tool 60 and sets the inside of the marker movement trajectory as the cleaning area.

The detailed configuration of the control unit and memory unit of the device main body will be described below. Figure 3 is a control block diagram of the device main body of this embodiment. Note that this mainly shows the control block when the cleaning device operates in the area setting mode. Also, the symbols in Figures 1 and 2 will be used appropriately in the explanation.

As shown in FIG. 3, the control unit 30 includes a cleaning control unit 31, a driving control unit 32, a SLAM control unit 33, a driving path creation unit 34, and an environmental map creation unit 35. The control unit 30 also includes an image acquisition unit 36 that functions in the area setting mode, a marker recognition unit 37, a marker search unit 38, a marker movement trajectory creation unit 39, a cleaning area setting unit 41, a cleaning area correction unit 42, and a cleaning plan creation unit 43. The memory unit 50 includes a driving path memory unit 51, an environmental map memory unit 52, a marker memory unit 53, a marker search setting memory unit 54, a marker movement trajectory memory unit 55, a cleaning area memory unit 56, a cleaning plan memory unit 57, and a device setting memory unit 58.

The cleaning control unit 31 controls the manual cleaning of the cleaning device 1 according to the manual operation of the operator, or the automatic cleaning of the cleaning device 1 according to a cleaning plan. The control contents of the manual cleaning and automatic cleaning include, for example, the amount of cleaning liquid sprayed, the contact pressure of the cleaning member 16, and the suction force of dirty water. The driving control unit 32 controls the manual driving of the cleaning device 1 according to the manual operation of the operator, or the autonomous driving of the cleaning device 1 according to a cleaning plan. The control contents of the manual driving and autonomous driving include, for example, the driving speed. Furthermore, the control contents of the autonomous driving include the driving route set in the cleaning area.

The SLAM control unit 33 executes SLAM (Simultaneous Localization Mapping) in real time during manual operation to estimate the self-position of the cleaning device 1 and create a local map based on the distance and angle from obstacles around the cleaning device 1 measured by the measurement unit 25. The travel path creation unit 34 creates a travel path for the cleaning device 1 by connecting multiple self-positions arranged in chronological order during manual operation. The environmental map creation unit 35 creates an environmental map by connecting multiple local maps arranged in chronological order during manual operation. The travel path is stored in the travel path memory unit 51, and the environmental map is stored in the environmental map memory unit 52. Note that the travel path obtained during the creation of the environmental map is not used in the area setting mode described below.

The area setting mode of the cleaning device 1 is enabled when an environmental map has been created. If no environmental map is stored in the environmental map storage unit 52, the display unit 22 or the like prompts the operator to create an environmental map. When the area setting mode is selected by the operator, the imaging unit 21 captures an image of the surrounding floor surface F, and the captured image is periodically output from the imaging unit 21 to the captured image acquisition unit 36. The captured image acquisition unit (acquisition unit) 36 periodically acquires captured images from the imaging unit 21 that captured an image of the floor surface F during the area setting mode. The captured image acquisition unit 36 may acquire captured images in response to an operation by the operator. The captured images may also be displayed in real time on the display screen of the display unit 22.

The marker recognition unit (recognition unit) 37 recognizes markers 63 (see FIG. 4) that exist on the floor surface F from the captured image during the area setting mode. The markers 63 in the captured image are recognized by performing various image processing and pattern matching processing on the captured image. At this time, if the height and orientation of the imaging unit 21 are fixed to the device body 10, the distance and angle of the marker 63 relative to the imaging unit 21 are found by recognizing the marker position within the frame of the captured image. The marker position is reflected on the environmental map by adding the found distance and angle of the marker 63 to the self-position of the cleaning device 1.

On the other hand, when the imaging unit 21 is driven (pan, tilt, roll) to also perform a patrol function, the marker position may be reflected on the environmental map by associating feature points of the captured image with feature points of the environmental map. The marker recognition unit 37 reads the environmental map from the environmental map storage unit 52 and compares feature points of the environmental map with feature points of the captured image. For example, the captured image is associated with the environmental map by treating a corner of a wall in the captured image as a feature point and associating it with the same feature point in the environmental map, and the marker position determined from the captured image is reflected on the environmental map by converting the marker position in the captured image system (camera coordinate system) to the marker position in the coordinate system of the environmental map.

Alternatively, the distance from the imaging unit 21 to the marker 63 may be measured based on the difference between the actual size of the marker 63 and the size of the marker 63 captured in the captured image. This allows the marker recognition unit 37 to recognize the marker position (coordinates) in the imaging coordinate system (camera coordinate system). Furthermore, a stereo camera may be used for the imaging unit 21, and the marker position may be recognized from the parallax information of the stereo camera. When the marker recognition unit 37 recognizes the marker position (coordinates) in the imaging coordinate system (camera coordinate system), the marker position in the imaging image system (camera coordinate system) is converted to the marker position in the coordinate system of the environmental map by associating the feature points of the captured image with the feature points of the environmental map as described above, and the marker position determined from the captured image is reflected on the environmental map.

Each time the captured image acquisition unit 36 acquires an image from the imaging unit 21, the marker recognition unit 37 performs marker recognition processing on the captured image. Marker positions are recognized from multiple captured images, and the multiple marker positions are reflected on the environmental map.

In this embodiment, the marker 63 is attached to a cleaning tool (work implement) 60 such as a mop. Examples of the marker 63 include light sources of visible light or infrared light, reflective markers for motion capture, and characteristic shapes of pre-registered objects or patterns. The marker 63 is recognized by performing various image processing and pattern matching processing on the captured image. When the characteristic shape of an object or pattern is used as the marker 63, the characteristic shape is registered in advance in the marker storage unit (storage unit) 53 as a template image. The marker storage unit 53 also stores the actual size of the marker 63 for measuring the distance to the marker 63.

When the marker 63 is not recognized in the captured image by the marker recognition unit 37 during the area setting mode, the marker search unit 38 drives the imaging unit 21 to move up and down and rotate to search for the marker 63 on the floor surface F. When the marker 63 is not recognized even when the imaging unit 21 is driven, the marker search unit 38 controls the autonomous driving of the cleaning device 1 to search for the marker 63 on the floor surface F. The driving speed during marker search by autonomous driving is stored in advance in the marker search setting memory unit 54. In addition, when searching for a marker by autonomous driving, a sufficient distance from a barrier such as a wall or pillar is ensured so that the cleaning device 1 does not collide with the barrier.

The marker movement trajectory creation unit 39 creates a marker movement trajectory from chronologically consecutive captured images during the area setting mode. As described above, since multiple marker positions are reflected on the environmental map, the marker movement trajectory is created on the environmental map by connecting the multiple marker positions. When the cleaning tool 60 is moved in small increments, the marker 63 attached to the cleaning tool 60 moves finely. For this reason, the marker movement trajectory creation unit 39 calculates an approximation curve from the moving average of the multiple marker positions to create the marker movement trajectory. The marker movement trajectory is stored in the marker movement trajectory storage unit 55.

When the marker 63 is hidden behind the operator during the area setting mode, the marker 63 will not be included in the captured image captured by the imaging unit 21. As a result, the marker movement trajectory created by the marker movement trajectory creation unit 39 will be interrupted and will become intermittent. In this case, the marker movement trajectory may be complemented to connect the interrupted parts of the marker movement trajectory. If the marker movement trajectory does not form a closed area, the marker movement trajectory may be complemented to connect the start point and end point of the marker movement trajectory to form a closed area. Furthermore, the marker movement trajectory may be created along the shape of the surroundings so as to be parallel to the wall surface of the barrier on the environmental map.

The cleaning area setting unit (setting unit) 41 sets the inside of the marker movement trajectory on the environmental map as the cleaning area during the area setting mode. A closed area that is completely or partially surrounded by the marker movement trajectory is automatically set as the cleaning area. The cleaning area correction unit (correction unit) 42 corrects the periphery of the cleaning area so that the distance from the barrier is equal to or greater than the safety distance. The cleaning area correction unit 42 reads the safety distance from the device setting memory unit 58, and offsets the periphery of the cleaning area when the distance from the barrier to the periphery of the cleaning area is less than the safety distance. The distance from the barrier to the periphery of the cleaning area is found from the environmental map.

In addition, the cleaning area setting unit 41 can set the inside of the marker movement trajectory on the environmental map as a non-cleaning area (non-working area) during the area setting mode in response to the operator's operation. For example, if an obstacle is present inside the marker movement trajectory, the inside of the marker movement trajectory is set as a non-cleaning area to avoid a collision between the obstacle and the cleaning device 1. Furthermore, the cleaning area setting unit 41 can set a new cleaning area by combining multiple cleaning areas, and can set a new cleaning area by combining a cleaning area and a non-cleaning area. The cleaning areas and non-cleaning areas are stored in the cleaning area memory unit 56.

The cleaning plan creation unit (decision unit) 43 determines a travel route that causes the cleaning device 1 to pass through the entire cleaning area by repeating back and forth travel. When the start point and end point of the travel route are set in the cleaning area, a zigzag travel route is determined such that the cleaning member 16 passes through the entire cleaning area by repeating back and forth travel from the start point to the end point. The back and forth direction of the travel route is set to be, for example, the longitudinal direction of the cleaning area, and the interval between the straight path in this back and forth direction is set so that the passing surfaces of the cleaning member 16 overlap. The width dimension of the cleaning member 16 is stored in the device setting memory unit 58 as a device setting.

The start and end points of the travel route may be set automatically or manually by the operator. When either the start or end point is set, the other point may be automatically set at a symmetrical position relative to the other point. The operator sets cleaning conditions such as the amount of cleaning liquid to be sprayed at each point from the start to the end point of the travel route, the contact pressure of the cleaning member 16, the suction power of the wastewater, and the travel speed to create a cleaning plan. The cleaning conditions stored in the device setting memory unit 58 may be automatically reflected in the travel route. A plurality of cleaning plans are stored in the cleaning plan memory unit 57, and a cleaning plan can be selected as appropriate.

Now, a first setting example of the cleaning area will be described with reference to Figs. 4 to 7. The first setting example is an example of setting the cleaning area on the environmental map when the cleaning device is stopped. In the following description, it is assumed that the environmental map has been created in advance. Fig. 4 is a diagram showing an example of a cleaning tool of this embodiment. Fig. 5 is a flowchart of the first setting example of this embodiment. Fig. 6 is a diagram showing an example of a captured image of this embodiment. Fig. 7 is a transition diagram of the first setting example of this embodiment, where (A) shows the initial state, (B) shows the creation state of the marker movement trajectory, (C) shows the setting state of the cleaning area, and (D) shows the setting state of the travel route. Here, the symbols of Figs. 1 to 3 will be used appropriately for the description.

As shown in FIG. 4, in this embodiment, an operator OP uses a cleaning tool 60 to clean areas such as the edge of a wall that cannot be cleaned by the cleaning device 1. The cleaning tool 60 is configured by connecting a head 62 to the tip of a handle 61 via a universal joint. The head 62 is formed in a rectangular shape in a plan view, and a wiping member (not shown) is attached to the entire underside of the head 62, and a pair of LED (Light Emitting Diode) light sources are attached as markers 63 to both ends of the upper surface of the head 62 in the longitudinal direction. When the operator OP cleans using the cleaning tool 60, the marker 63 draws a marker movement trajectory along the edge of the wall.

As shown in FIG. 5, when an environmental map M (see FIG. 7(A)) has been created and an area setting mode is selected by an operator OP (step S01), an image of a floor surface F is acquired by the image acquisition unit 36 from the imaging unit 21 (step S02). For example, the captured image includes the operator OP cleaning using a cleaning tool 60 (see FIG. 6). Next, a marker recognition process is performed on the captured image by the marker recognition unit 37 (step S03). In this embodiment, a pair of markers 63 is provided on the head 62 of the cleaning tool 60, and therefore at least one of the pair of markers 63 is recognized.

If the marker 63 is not recognized from the captured image (No in step S03), the marker search unit 38 performs a marker search process (step S04). As described above, in the marker search process, the imaging unit 21 is driven to move up and down and rotate to search for the marker 63 on the floor surface F. If the marker 63 is found within a predetermined time (Yes in step S05), the process returns to step S02, and the captured image acquisition unit 36 acquires an image from the imaging unit 21. If the marker 63 is not found within the predetermined time (No in step S05), it is determined that the operator OP has finished cleaning and left the cleaning area, and the marker recognition process is stopped.

When the marker 63 is recognized from the captured image (Yes in step S03), the marker position in the captured image system is obtained. Then, the feature points of the environmental map M are matched with the feature points of the captured image, and the self-position P1 of the cleaning device 1 and the marker position P2 are reflected in the coordinate system of the environmental map M (see FIG. 7(A)). If the operator OP does not perform a stop operation for the marker recognition process (No in step S06), the processes from step S02 to step S05 are repeated. If the operator OP performs a stop operation (Yes in step S06), cleaning using the cleaning tool 60 is deemed to have ended, and the marker recognition process is stopped.

After the marker recognition process is stopped, the marker movement trajectory creation unit 39 performs a movement trajectory creation process (step S07). In the movement trajectory creation process, a marker movement trajectory 71 is created by connecting multiple marker positions P2 on the environmental map M (see FIG. 7B). Since a pair of markers 63 is provided on the cleaning tool 60, the marker movement trajectory 71 may be created for either one of the pair of markers 63, or a marker movement trajectory 71 may be created that passes through the middle position of the pair of markers 63. If the marker movement trajectory 71 has an interruption or the like, the marker movement trajectory 71 is appropriately supplemented to form a closed area.

Next, the cleaning area setting unit 41 performs an area setting process (step S08). In the area setting process, the cleaning area 72 is set inside the marker movement trajectory 71. If the marker movement trajectory 71 does not form a closed area, the cleaning area 72 is not set. Next, the cleaning area correction unit 42 performs an area correction process (step S09). In the area correction process, the distance from the barrier of the environmental map M to the outer periphery of the cleaning area 72 is corrected to be equal to or greater than the safety distance. In this embodiment, the outer periphery of the cleaning area 72 is corrected over the entire circumference (see FIG. 7(C)).

Next, the cleaning plan creation unit 43 performs a plan creation process (step S10). In the plan creation process, a start point 76 and an end point 77 of a travel route 75 are set in the cleaning area 72, and a zigzag travel route 75 from the start point 76 to the end point 77 is determined (see FIG. 7(D)). In addition, cleaning conditions for the cleaning operation are set at each point of the travel route 75 to create a cleaning plan. Then, in response to an instruction from the operator OP, the cleaning device 1 is driven according to the cleaning plan to perform a cleaning operation on the cleaning area 72 (step S11). The cleaning operation by the cleaning device 1 continues until an end operation is performed by the operator OP (step S12).

As described above, in the first setting example, the cleaning area 72 is automatically set by the cleaning device 1 while the floor surface F is being cleaned using the cleaning tool 60. This allows the cleaning area 72 to be easily set while reducing the workload of the operator OP.

A second example of setting a cleaning area will be described with reference to Figs. 8 and 9. The second example of setting is an example of setting another cleaning area in the environmental map in the background while the cleaning device is cleaning a cleaning area. In the following description, it is assumed that the environmental map has been created in advance. Fig. 8 is a flowchart of the second example of setting in this embodiment. Fig. 9 is a transition diagram of the second example of setting in this embodiment, in which (A) shows the initial state, (B) shows the state in which the marker movement trajectory is created, (C) shows the state in which the cleaning area is set, and (D) shows the state in which the travel route is set. Here, the symbols in Figs. 1 to 4 will be used as appropriate for the description. Also, the same processes as in the first example of setting will be omitted.

As shown in FIG. 8, when the operator OP selects the area setting mode (step S21) after the environmental map M (see FIG. 9A) has been created, the cleaning device 1 performs a cleaning operation on the cleaning area 82 (step S22). In parallel with the cleaning operation on the cleaning area 82, the cleaning device 1 performs a setting process for another cleaning area 83 in the background. In the setting process for the other cleaning area 83, the captured image acquisition unit 36 acquires an image of the floor surface F from the imaging unit 21 (step S23). That is, while the cleaning device 1 is traveling autonomously, the imaging unit 21 captures an image of the operator OP during cleaning.

Next, the marker recognition unit 37 performs a marker recognition process while the cleaning device 1 is autonomously traveling (step S24). In the marker recognition process, the marker position in the captured image system is obtained, and the marker position P2 is reflected in the coordinate system of the environmental map M based on the self-position P1 of the cleaning device 1 (see FIG. 9A). Note that since the second setting example is performed during the cleaning operation of the cleaning device 1, the search process for the marker 63 is not performed even if the marker 63 is not recognized by the marker recognition unit 37. When the cleaning operation of the cleaning device 1 is completed and the operator OP performs a stop operation (Yes in step S25), the marker recognition process is stopped as it is determined that cleaning using the cleaning tool 60 has ended.

Next, the marker movement trajectory creation unit 39 performs a movement trajectory creation process (step S26). In the movement trajectory creation process, a marker movement trajectory 81 is created by connecting multiple marker positions P2 on the environmental map M (see FIG. 9B). At this time, the start point and the end point of the marker movement trajectory 81 are not connected, but the start point and the end point are connected to complete the marker movement trajectory 81. Next, the cleaning area setting unit 41 performs an area setting process (step S27). In the area setting process, another cleaning area 83 is set inside the marker movement trajectory 81 so as to avoid the obstacle C. The cleaning area correction unit 42 performs an area correction process (step S28). In the area correction process, the distance from the barrier on the environmental map M to the outer periphery of the other cleaning area 83 is corrected to be equal to or greater than the safe distance (see FIG. 9C).

Next, the cleaning plan creation unit 43 performs a plan creation process (step S29). In the plan creation process, a zigzag travel route 85 from a start point 86 to an end point 87 is determined, and cleaning conditions for the cleaning operation are set at each point on the travel route 85 to create a cleaning plan (see FIG. 9 (D)). Then, at the instruction of the operator OP, the cleaning device 1 is driven according to the cleaning plan to perform cleaning operations for other cleaning areas 83 (step S30). The cleaning operation by the cleaning device 1 continues until an end operation is performed by the operator OP (step S31).

As described above, in the second setting example, the marker recognition unit 37 recognizes the marker 63 while the cleaning device 1 is traveling autonomously. This improves work efficiency by adding another cleaning area 83 in parallel with the cleaning operation of the cleaning device 1 in the cleaning area 82. In addition, by setting the cleaning areas 82 and 83 on the environmental map M, an appropriate cleaning plan can be created for each area even when an obstacle C such as cardboard is present on the floor surface F or when there is partial dirt.

A third example of setting the cleaning area will be described with reference to FIG. 10. The third example of setting is an example of resetting a new cleaning area by combining a cleaning area and a non-cleaning area. In the following explanation, it is assumed that an environmental map has been created in advance. FIG. 10 is a transition diagram of the third example of setting in this embodiment, where (A) shows the initial state, (B) shows the state in which the marker movement trajectory has been created, (C) shows the state in which the cleaning area has been set, and (D) shows the state in which the travel route has been set. Here, the explanation will be given using the symbols in FIG. 1 to FIG. 4 as appropriate. Also, the explanation will be omitted for processes similar to those in the first example of setting.

As shown in FIG. 10(A), in the third setting example, a cleaning area 92 is set on the environmental map M by performing processing similar to that of the first setting example. As the operator uses the cleaning tool 60 to clean the edge of the wall, the cleaning area 92 is formed in a rectangular shape along the barrier of the environmental map M. If an obstacle C such as cardboard is placed on the floor surface F, the cleaning device 1 will interfere with the obstacle C and the cleaning operation of the cleaning area 92 will be hindered by the obstacle C. The cleaning device 1 cannot perform the cleaning operation of the cleaning area 92 until the area where the obstacle C is installed is removed from the cleaning area 92.

As shown in FIG. 10B, in order to remove obstacle C from cleaning area 92, a non-cleaning area 93 including obstacle C is set inside cleaning area 92. In this case, operator OP moves cleaning tool 60 so as to separate the installation area of obstacle C from cleaning area 92. A marker movement trajectory 91 of cleaning tool 60 is created by cleaning device 1, and a triangular closed area is formed by marker movement trajectory 91 and two sides (the vertical and horizontal marker movement trajectory of cleaning area 92) of cleaning area 92. A non-cleaning area 93 is set in this closed area by the operation of the operator. As a result, a new cleaning area 94 is re-set by removing non-cleaning area 93 from cleaning area 92, as shown in FIG. 10C.

As shown in FIG. 10(D), a zigzag travel route 95 is re-determined for the new cleaning area 94, going from a start point 96 to an end point 97. Furthermore, cleaning conditions for the cleaning operation are set for each point on the travel route 95, and a cleaning plan is re-created. The cleaning device 1 is then driven according to the cleaning plan, and a cleaning operation is performed for the new cleaning area 94. Note that in the third setting process, the new cleaning area 94 is re-defined by combining the cleaning area 92 and the non-cleaning area 93, but a new cleaning area may also be re-defined by combining multiple cleaning areas.

As described above, in the third setting process, a new cleaning area 94 that avoids the obstacle C is reset by combining the cleaning area 92 and the non-cleaning area 93. This prevents the cleaning device 1 from colliding with the obstacle C during autonomous traveling.

With reference to FIG. 11, a fourth example of setting a cleaning area will be described. The fourth example of setting is an example in which multiple cleaning areas are set on an environmental map, and a cleaning plan is set for each cleaning area. In the following explanation, it is assumed that an environmental map has been created in advance. FIG. 11 is a transition diagram of the fourth example of setting in this embodiment, where (A) shows the initial state, (B) shows the state in which a marker movement trajectory has been created, (C) shows the state in which a cleaning area has been set, and (D) shows the state in which a travel route has been set. Here, the explanation will be given using the symbols in FIG. 1 to FIG. 4 as appropriate. Also, the explanation will be omitted for processes similar to those in the first example of setting.

As shown in FIG. 11 (A), in the fourth setting example, a cleaning area 102 is set in approximately half of the environmental map M by performing processing similar to that of the first setting example. A cleaning plan has already been created for the cleaning area 102. Next to the cleaning area 102 is an area 104 that is heavily soiled. This area 104 needs to be cleaned by the cleaning device 1, but it is difficult to sufficiently remove the dirt from the area 104 under the same cleaning conditions as the cleaning area 102. For this reason, a cleaning plan different from that for the cleaning area 102 needs to be created for the area 104.

As shown in FIG. 11(B), while the cleaning device 1 is cleaning the cleaning area 102, another cleaning area 103 is set in the heavily soiled area 104. When the operator OP notices that the area 104 is dirty, the operator OP moves the cleaning tool 60 so as to surround the area 104. The cleaning device 1 creates a marker movement trajectory 101 for the cleaning tool 60, and a closed area is formed by the marker movement trajectory 101 and the cleaning area 102. As shown in FIG. 11(C), another cleaning area 103 is set in this closed area by the operation of the operator OP, and the other cleaning area 103 is added next to the cleaning area 102 on the environmental map M.

As shown in FIG. 11(D), a zigzag travel route 105 is determined for the other cleaning area 103, going from a start point 106 to an end point 107. In addition, cleaning conditions for the cleaning operation are set for each point on the travel route 105 to create a cleaning plan. Then, the cleaning device 1 is driven according to the cleaning plan to perform the cleaning operation for the other cleaning area 103. Note that in the fourth setting example, the other cleaning area 103 is added while the cleaning device 1 is autonomously traveling in the cleaning area 102, but the other cleaning area 103 may be set while the cleaning device 1 is stopped after cleaning the cleaning area 102 or before cleaning.

As described above, in the fourth setting example, when multiple cleaning areas 102, 103 are set on the environmental map, a cleaning plan suitable for the conditions of the cleaning areas 102, 103 is applied. Therefore, the heavily soiled area 104 is cleaned well according to a cleaning plan suitable for cleaning the dirt.

As described above, according to this embodiment, by moving the marker 63 on the floor surface F, the imaging unit 21 captures an image of the marker 63 moving on the floor surface F. A marker movement trajectory 71 is created from chronologically consecutive captured images, and a cleaning area is automatically set inside the marker movement trajectory 71. Thus, the simple task of moving the marker 63 on the floor surface F can set a cleaning area, reducing the workload of the operator.

In this embodiment, a cleaning device that performs cleaning work is exemplified as the autonomous traveling work device, but the autonomous traveling work device is not limited to a cleaning device. The autonomous traveling work device may be a work device that performs other work such as dry cleaning work, polishing work, polishing work, wax application work, etc. In other words, the autonomous traveling work device may be a robotic cleaning machine, polisher, buffing machine, or wax application machine. Therefore, instead of cleaning members such as pads, cleaning brushes, polishing brushes, polishing brushes, and application members may be used in the autonomous traveling work device, and further, abrasives, polishing agents, and coating agents may be used as cleaning fluids.

In addition, in this embodiment, a floor surface is exemplified as the surface to be worked on, but the surface to be worked on may also be a wall surface, ceiling surface, window surface, mirror surface, etc.

The cleaning device of this embodiment is provided with a learning mode, a reproduction mode, and an area setting mode as operation modes, but it is sufficient if at least the reproduction mode and the area setting mode are provided.

In addition, in this embodiment, the environmental map is created using SLAM, but the environmental map may also be created using V-SLAM or LiDAR-SLAM, or the environmental map may be prepared in advance by the operator.

In addition, in this embodiment, a mop or other cleaning tool is used as an example of a work tool, but the work tool may be any tool that an operator can hold by hand when working.

In addition, in this embodiment, light sources such as LEDs and the characteristic shapes of objects and patterns are exemplified as markers, but an illuminated spot pointed to by a laser pointer may also be used as a marker. Furthermore, by using a transmitter of sound waves, radio waves, etc. as a marker and grasping the level of attenuation using a receiver instead of an imaging unit, the marker position (angle and distance) based on the autonomous mobile work device can be identified. By converting the marker position into the coordinate system of the environmental map, the marker position is reflected on the environmental map.

In addition, in this embodiment, the imaging unit is provided in the cleaning device, but the imaging unit may be provided in a location separate from the cleaning device. For example, the imaging unit may be installed on the ceiling, and the imaging unit and the cleaning device may be connected via wireless communication.

In addition, in the above-described embodiment and modified examples, a program may be installed in the cleaning device to give the cleaning device a function for setting the cleaning area, thereby reducing the workload of the operator. This program is stored in a storage medium. The storage medium is not particularly limited, but may be a non-transitory storage medium such as an optical disk, a magneto-optical disk, or a flash memory.

Although this embodiment has been described, other embodiments may be combinations of the above embodiments and variations in whole or in part.

The technology of the present invention is not limited to the above-described embodiments, and may be modified, substituted, or altered in various ways without departing from the spirit of the technical idea. Furthermore, if the technical idea can be realized in a different way due to technological advances or other derived technologies, it may be implemented using that method. Therefore, the claims cover all embodiments that may fall within the scope of the technical idea.

As explained above, this is useful for industrial robots suitable for automated work in work areas, such as autonomous cleaning equipment.

1: Cleaning device (autonomous mobile work device)
21: Imaging unit 22: Display unit 36: Image acquisition unit (acquisition unit)
37: Marker recognition unit (recognition unit)
39: Marker movement trajectory creation unit (creation unit)
41: cleaning area setting unit (setting unit)
42: cleaning area correction unit (correction unit)
43: Cleaning plan creation unit (decision unit)
52: Environmental map memory unit (memory unit)
60: Cleaning tools (work tools)
63: Marker 71, 81, 91, 101: Marker movement trajectory 72, 83, 92, 103: Cleaning area 75, 85, 95, 105: Travel route 93: Non-cleaning area F: Floor surface M: Environmental map

Claims (12)

An autonomous mobile work device that performs work on a work surface while autonomously traveling,
an acquisition unit that acquires an image from an imaging unit that images the work surface;
a recognition unit that recognizes a marker present on the work surface from a captured image;
a creating unit that creates a movement trajectory of the marker from time-series consecutive captured images;
a setting unit that sets an area inside the movement trajectory of the marker as a working area ,
An autonomous mobile work device , wherein the marker is attached to a cleaning tool which is a manually operated work tool . The autonomous mobile work device according to claim 1, further comprising a correction unit that corrects the perimeter of the work area so that the distance from the barrier is equal to or greater than the safety distance. The autonomous mobile work device according to claim 1 or 2, characterized in that it is provided with a determination unit that determines a travel route that passes through the entire work area by repeating round trip travel. The autonomous mobile work device according to any one of claims 1 to 3, characterized in that the setting unit is capable of setting the inside of the movement trajectory of the marker as a non-work area. The autonomous mobile work device according to claim 4, characterized in that the setting unit is capable of resetting a new work area by combining multiple work areas, and is also capable of resetting a new work area by combining a work area and a non-work area. An autonomous mobile work device as described in any one of claims 1 to 5, characterized in that while the autonomous mobile work device is working in a work area, the setting unit sets another work area by working using a work tool with the marker in the background . The autonomous mobile work device according to any one of claims 1 to 6, characterized in that the marker is a light source or a pre-registered characteristic shape. The autonomous mobile work device according to any one of claims 1 to 7, characterized in that the recognition unit recognizes the marker during autonomous mobile. The autonomous mobile work device according to any one of claims 1 to 8, characterized in that when the setting unit sets multiple work areas on the work surface, a different work plan is applied to each work area. The autonomous mobile work device according to any one of claims 1 to 9, characterized in that the imaging unit captures images in all directions. A storage unit is provided that stores an environmental map of the work surface that is created in advance,
the recognition unit recognizes the position of the marker present on the work surface from the captured image, associates feature points of an environmental map with feature points of the captured image, and reflects the position of the marker on the environmental map;
the creation unit creates a movement trajectory of the marker by connecting the positions of the plurality of markers reflected on the environmental map;
9. The autonomous mobile work device according to claim 1, wherein the setting unit sets an area inside a movement trajectory of the marker on an environmental map as a work area. A program for an autonomous mobile work device in which a marker is attached to a cleaning tool, which is a manual work tool, and which performs work on a work surface while traveling autonomously, comprising:
acquiring an image from an imaging unit that images the work surface;
Recognizing the marker present on the work surface from a captured image;
creating a movement trajectory of the marker from time-series consecutive captured images;
and setting an area inside a movement trajectory of the marker as a working area.

Images