JP7637555B2 - Cleaning system, unmanned flying device and autonomous cleaning device - Google Patents

Description

The present invention relates to a cleaning system equipped with an autonomously traveling cleaning device capable of performing autonomously traveling and cleaning, an unmanned flying device capable of linking with this autonomously traveling cleaning device, and this autonomously traveling cleaning device.

Conventionally, some autonomous cleaning devices perform autonomous cleaning by autonomously traveling through a cleaning area and automatically collecting and cleaning up debris such as dust and fallen leaves on the floor or ground of the cleaning area. Some autonomous cleaning devices also perform autonomous cleaning in cooperation with an unmanned flying device.

For example, in the cleaning system of Patent Document 1, a self-propelled robotic cleaning device that cleans the floor surface and an aircraft device that flies through the air perform cleaning operations in coordination with each other. The aircraft device uses a cleaning aid to drop objects to be cleaned that are higher than the floor surface onto the floor surface at a coordination location within the cleaning area, and transmits a notification regarding the operation of dropping the objects to be cleaned onto the floor surface to the robotic cleaning device via a communication unit. The robotic cleaning device performs the coordinated operation by traveling over the floor surface of the coordination location and cleaning after a predetermined time has elapsed since receiving the notification regarding the operation of dropping the objects to be cleaned onto the floor surface at the coordination location within the cleaning area.

JP 2019-91326 A

In conventional technology such as the cleaning system of Patent Document 1, an aircraft device uses a cleaning aid to drop cleaning objects such as garbage that are higher than the floor surface at a linked location within a cleaning area onto the floor surface. However, cleaning objects that are higher than the floor surface do not necessarily fall into the linked location, so cleaning objects that do not fall into the linked location end up remaining on the floor surface without being cleaned.

Also, for example, in parks or parking lots where trees are planted adjacent to each other, many fallen leaves may fly off and scatter over a wide area depending on the season. In such a wide-area cleaning area, it is difficult to clean and collect fallen leaves and other objects to be collected, so efficient automatic cleaning is required. Furthermore, in parking lots, vehicles are parked randomly, and fallen leaves and other objects to be collected may be scattered so as to be hidden in the blind spots of the vehicles, so it is very time-consuming to visually search for objects to be collected in the parking lot. Therefore, it is very difficult to automatically collect and clean objects to be collected that are scattered over a wide area in the parking lot.

The present invention has been made in consideration of the problems described above, and the objective of the present invention is to provide a cleaning system, an unmanned flying device, and an autonomously traveling cleaning device that can easily, reliably, and automatically collect and clean items scattered over a wide area.

In order to solve the above problems, a first cleaning system of the present invention is a cleaning system including an autonomous traveling cleaning device capable of performing automatic traveling cleaning by traveling autonomously and cleaning automatically, and further including an unmanned flying device, the autonomous traveling cleaning device including a device main body, a traveling unit that travels the device main body, a cleaning unit that cleans on the traveling path of the device main body, an environmental map acquisition unit that acquires an environmental map of a cleaning area in which the automatic traveling cleaning is performed, and a first communication unit that wirelessly communicates with the unmanned flying device, the unmanned flying device including an imaging unit and a second communication unit that wirelessly communicates with the autonomous traveling cleaning device, the imaging unit being configured to capture images of the unmanned flying device flying above the cleaning area. When the autonomous driving cleaning device is started, the cleaning area is imaged from the sky to generate image data, and the second communication unit communicates wirelessly with the first communication unit to transmit the image data to the autonomous driving cleaning device, and the autonomous driving cleaning device and/or the unmanned flying device are equipped with an object discrimination unit that discriminates collection targets and non-collection targets on the floor or ground of the cleaning area based on the image data captured by the imaging unit, and the autonomous driving cleaning device is equipped with a data reflection unit that reflects the image data on the environmental map by associating the collection targets and non-collection targets discriminated by the object discrimination unit with the environmental map.

According to the first cleaning system of the present invention, by reflecting the image data of the cleaning area captured by the unmanned flying device on the environmental map, it is possible to obtain information on blind spots that the autonomous cleaning device cannot see, and to obtain an environmental map showing the cleaning area in more detail. Therefore, the autonomous cleaning device can automatically collect and clean collection objects scattered over a wide area in the cleaning area easily and reliably.

In order to solve the above problem, in the second cleaning system of the present invention, the autonomous cleaning device is characterized by having a cleaning plan creation unit that creates a cleaning plan for performing the autonomous cleaning so as to clean the collection objects based on the environmental map, and an autonomous cleaning control unit that performs the autonomous cleaning based on the cleaning plan.

The second cleaning system of the present invention can obtain information about blind spots that the autonomous cleaning device cannot see, and can create a cleaning plan that can more reliably collect items without leaving any items behind in the cleaning area.

In order to solve the above problems, in the third cleaning system of the present invention, the unmanned flying device is equipped with a flight control unit that controls a collection operation that uses downwash during flight to blow and collect the collection objects at a predetermined collection position within the cleaning area, the imaging unit captures an image of the cleaning area including the collection position after performing the collection operation to generate the imaging data, the data reflection unit reflects the collection position captured in the imaging data on the environmental map, and the cleaning plan creation unit creates the cleaning plan for performing the automatic driving cleaning so as to preferentially clean the collection objects at the collection position based on the environmental map.

According to the third cleaning system of the present invention, the autonomous cleaning device can clean the majority of the collection objects in the cleaning area by cleaning the collection location where the collection objects were collected, thereby improving cleaning efficiency. Therefore, collection objects in the cleaning area can be collected more easily and reliably so that none are left behind.

To solve the above problem, in the fourth cleaning system of the present invention, the autonomous cleaning device includes an obstacle recognition unit that recognizes obstacles based on the measurement results of the positional relationship of obstacles around the device body, and the object discrimination unit includes a collection position setting unit that distinguishes the non-collection objects on the floor or ground of the cleaning area based on the recognition information of the obstacles recognized by the obstacle recognition unit, and sets the collection position on one side of the non-collection objects distinguished by the object discrimination unit based on the imaging data or the recognition information.

According to the fourth cleaning system of the present invention, by utilizing non-collectible objects at the collection location, the objects to be collected can be blown together more reliably, and can be collected and cleaned more easily and reliably so that no objects to be collected are left behind in the cleaning area.

In order to solve the above problem, the first unmanned flying device of the present invention includes an imaging unit that images the cleaning area from above when flying above the cleaning area to generate imaging data, an object discrimination unit that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit, a collection position setting unit that sets a collection position on one side of the objects not to be collected that are discriminated based on the imaging data by the object discrimination unit, a flight control unit that controls a collection operation that uses downwash during flight to blow the objects to the collection position in the cleaning area, and a communication unit that wirelessly communicates with an autonomously traveling cleaning device, and the imaging unit images the cleaning area including the collection position after the collection operation is performed to generate the imaging data, and the communication unit wirelessly communicates with the autonomously traveling cleaning device to transmit the imaging data captured by the imaging unit after the collection operation is performed to the autonomously traveling cleaning device.

According to the first unmanned flying device of the present invention, the collection location where the collection objects were collected can be provided to the autonomously traveling cleaning device, so that the autonomously traveling cleaning device can clean the majority of the collection objects in the cleaning area by cleaning this collection location, thereby improving cleaning efficiency. Therefore, collection objects in the cleaning area can be collected more easily and reliably so that none are left behind.

To solve the above problem, the second unmanned flying device of the present invention further includes a cleaning device detection unit that detects the position of the autonomously traveling cleaning device based on the imaging data, and the collection position setting unit sets the collection position to one side of the non-collection objects that are closer to the autonomously traveling cleaning device among the non-collection objects identified by the object discrimination unit.

According to the second unmanned flying device of the present invention, by using non-collection objects close to the autonomous cleaning device as collection locations, the objects to be collected can be blown together more reliably, and collection can be made easier and more reliable so that no objects to be collected are left behind in the cleaning area.

To solve the above problem, in the third unmanned flying device of the present invention, the collection position setting unit is erected on the floor or ground of the cleaning area, recognizes landmarks that separate the cleaning area based on the imaging data, and sets the collection position on one side of the recognized landmark.

According to the third unmanned flying device of the present invention, even if there are no non-collection objects in the cleaning area to collect the collection objects, a landmark for collecting the collection objects can be easily installed in the cleaning area, making it possible to collect the collection objects by blowing them together.

To solve the above problem, in the fourth unmanned flying device of the present invention, the cleaning device detection unit detects the position of the autonomously traveling cleaning device by detecting a beacon signal sent out by the autonomously traveling cleaning device.

In addition, in the fifth unmanned flying device of the present invention, the cleaning device detection unit detects the position of the autonomous cleaning device by detecting infrared rays emitted by a laser range finder as the beacon signal to measure the positional relationship of surrounding obstacles.

According to the fourth or fifth unmanned flying device of the present invention, the position of the autonomous cleaning device can be grasped more easily and reliably, and non-collection objects close to the autonomous cleaning device can be used as collection locations for collection objects, so that collection objects can be more reliably gathered together.

To solve the above problem, the sixth unmanned flying device of the present invention is characterized in that it includes a moving object discrimination unit that discriminates moving objects moving on the floor or ground of the cleaning area, and the flight control unit suspends flight when the moving object discrimination unit discriminates the moving object.

According to the sixth unmanned flying device of the present invention, when the unmanned flying device is flying to take aerial images of a cleaning area or to collect collection items, if a moving object such as a person is present, the flight is interrupted, thereby improving safety.

In order to solve the above problems, the first autonomous driving cleaning device of the present invention is an autonomous driving cleaning device capable of performing automatic driving cleaning by autonomously driving and automatically cleaning, and is characterized in that it comprises an apparatus main body, a driving unit that drives the apparatus main body, a cleaning unit that cleans along the driving path of the apparatus main body, an environmental map acquisition unit that acquires an environmental map of the cleaning area where the automatic driving cleaning is performed, a communication unit that receives image data generated by an unmanned flying device flying above the cleaning area and imaging the cleaning area from above by wireless communication with the unmanned flying device, an object discrimination unit that discriminates collection objects and non-collection objects on the floor or ground of the cleaning area based on the imaging data received from the unmanned flying device by the communication unit, and a data reflection unit that reflects the imaging data on the environmental map by associating the collection objects and non-collection objects discriminated by the object discrimination unit with the environmental map.

According to the first autonomous cleaning device of the present invention, by reflecting the image data of the cleaning area captured by the unmanned flying device on the environmental map, it is possible to obtain information on blind spots that the autonomous cleaning device cannot see, and to obtain an environmental map showing the cleaning area in more detail. Therefore, the autonomous cleaning device can automatically collect and clean collection objects scattered over a wide area in the cleaning area easily and reliably.

According to the present invention, objects to be collected that are scattered over a wide area can be collected and cleaned automatically, easily and reliably.

1 is a schematic diagram showing a configuration of a cleaning system according to an embodiment of the present invention. 1 is a block diagram showing a schematic configuration of a cleaning system according to an embodiment of the present invention. FIG. 2 is a block diagram showing the electrical configuration of the cleaning system according to the embodiment of the present invention. A plan view showing an example of a cleaning area that is subject to manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. A plan view showing an example of a case in which collection items are scattered in a cleaning area that is the subject of manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. A plan view showing an example of a case in which an unmanned flying device is manually flown over a cleaning area littered with collection items in a cleaning system according to an embodiment of the present invention. A plan view showing an example of a cleaning system according to an embodiment of the present invention, in which an unmanned flying device performs manual flight to collect collection items at a collection position. 13 is a flowchart illustrating an example of the operation of manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. A flowchart showing an example of the automatic traveling and cleaning operation of an autonomous traveling cleaning device after manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. A flowchart showing an example of the automatic flight operation of an unmanned flying device and the automatic driving and cleaning operation of an autonomous driving cleaning device in a cleaning system according to an embodiment of the present invention. A flowchart showing another example of the automatic flight operation of an unmanned flying device and the automatic driving and cleaning operation of an autonomous driving cleaning device in a cleaning system according to an embodiment of the present invention. A plan view showing an example of the measurement range of the measurement unit of an autonomously moving cleaning device relative to the cleaning area that is the target of automatic flight of the unmanned flying device in a cleaning system according to an embodiment of the present invention. A plan view showing an example of the imaging range of an unmanned flying device relative to a cleaning area that is the target of automatic flight of the unmanned flying device in a cleaning system according to an embodiment of the present invention. A plan view showing an example of an unmanned flying device automatically flying over a cleaning area littered with collection items in a cleaning system according to an embodiment of the present invention. A plan view showing an example of a cleaning system according to an embodiment of the present invention, in which an unmanned flying device performs automatic flying to collect collection items at a collection position. 1 is a plan view showing an example of a cleaning area where no non-collection objects are to be set in a cleaning system according to an embodiment of the present invention. FIG. 11 is a plan view showing an example in which landmarks for setting collection positions are provided in a cleaning area in which there are no non-collection targets and in which collection positions are to be set in a cleaning system according to an embodiment of the present invention. FIG.

The following describes embodiments of the present invention with reference to the drawings. The following embodiments are preferred specific examples of the present invention and disclose various preferred techniques, but the technical scope of the present invention is not limited to these aspects.

A cleaning system 1 according to an embodiment of the present invention will be described. As shown in FIG. 1, the cleaning system 1 includes an unmanned flying device 2, a control device 3 for controlling the unmanned flying device 2, and an autonomously traveling cleaning device 4 capable of performing autonomous traveling and cleaning by autonomously traveling and cleaning automatically. The unmanned flying device 2 and the autonomously traveling cleaning device 4 are configured to be able to cooperate with each other.

The unmanned flying device 2 of this embodiment will be described.

The unmanned flying device 2 is configured, for example, as a drone, which performs flying operations by being controlled by a control device 3 or according to a preset program. As shown in FIG. 1, the unmanned flying device 2 includes a device main body 10 for housing each component, a flying unit 11 for flying the device main body 10, and an imaging unit 12 for capturing images of the surroundings.

As shown in FIG. 2, the unmanned flying device 2 also includes a positioning unit 13 that acquires position information of the unmanned flying device 2, a flight detection unit 14 that detects the flight status of the unmanned flying device 2, and a power supply unit 15 that supplies power to each part of the unmanned flying device 2 and controls the monitoring of the remaining charge of the battery (not shown) and charging.

Furthermore, the unmanned flying device 2 includes a control unit 16 that provides overall control over each part and function of the unmanned flying device 2 (such as flight by the flying unit 11 and image capture by the image capture unit 12), and a memory unit 17 that stores a flight plan, which is the flight program of the flying unit 11, and image data captured by the image capture unit 12. The unmanned flying device 2 also includes a communication unit 18 (second communication unit) for wireless communication with external devices such as the pilot device 3 and the autonomous cleaning device 4.

Next, each part of the unmanned flying device 2 will be described. The flying unit 11 has multiple (e.g., four) flying wings 11a on the top of the device body 10, and also has multiple flight motors 11b that rotate and drive each of the multiple flying wings 11a. The flying unit 11 controls the flight altitude, flight direction, flight attitude, flight speed, etc. of the unmanned flying device 2 by controlling the rotation speed of each flight motor 11b using the control unit 16.

The imaging unit 12 is provided at the bottom of the device body 10, and captures an aerial image of the area below the unmanned flying device 2 to generate imaging data. The imaging unit 12 is composed of, for example, a CCD (Charge Coupled Device) camera. The imaging unit 12 generates and outputs, as imaging data, the image of the captured aerial image itself and/or image analysis data of the captured aerial image.

The positioning unit 13 acquires position information of the unmanned flying device 2 using a satellite positioning system such as the Global Navigation Satellite System (GNSS) or the Global Positioning System (GPS). The flight detection unit 14 is composed of a gyro sensor and the like, and detects the flight state of the unmanned flying device 2, such as changes in attitude and acceleration during flight.

The power supply unit 15 includes a battery (power source) and a charging circuit mounted inside the device main body 10, and when connected to an external power source, the battery is charged and power is supplied to each part of the unmanned flying device 2.

The control unit 16 is composed of a computer device such as a CPU (Central Processing Unit), and is connected to a storage unit 17 including a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), a flash memory, etc. The control unit 16 is also connected to each part of the unmanned flying device 2, such as the flight unit 11, the imaging unit 12, the positioning unit 13, the flight detection unit 14, and the power supply unit 15.

The control unit 16 is also connected to be able to communicate with external devices via the communication unit 18. The communication unit 18 performs wireless communication with, for example, the pilot device 3 and the autonomously traveling cleaning device 4 using a communication standard such as Bluetooth (registered trademark). The communication unit 18, for example, receives flight instructions from the pilot device 3, receives instructions from the pilot device 3 to the imaging unit 12 to capture aerial images, and transmits imaging data of the aerial images captured by the imaging unit 12 to the pilot device 3 and the autonomously traveling cleaning device 4.

The memory unit 17 stores programs and data for controlling each part and various functions of the unmanned flying device 2, and the control unit 16 performs calculations based on the programs and data stored in the memory unit 17, thereby providing overall control of each part and various functions of the unmanned flying device 2. For example, by executing the programs stored in the memory unit 17, the control unit 16 operates as a flight control unit 50, a flight plan creation unit 51, an image capture control unit 52, an object discrimination unit 53, a collection position setting unit 54, a cleaning device detection unit 55, and a moving object discrimination unit 56, as shown in FIG. 3.

When the communication unit 18 receives a flight instruction from the pilot device 3, the flight control unit 50 controls the flight motors 11b of the flight unit 11 in accordance with the flight instruction to rotate and drive the flight wings 11a, and controls the manual flight of the flight unit 11 to control the flight altitude, flight direction, flight attitude, flight speed, etc. of the unmanned flying device 2. When manually flying the flight unit 11, the flight control unit 50 can collect collection objects on the floor or ground of the cleaning area toward a specified collection direction (collection position) by using the downwash generated by the rotation of the flight wings 11a of the flight unit 11.

The flight control unit 50 also controls the flight unit 11 to perform automatic flight according to a flight plan, which is a flight program pre-stored in the memory unit 17. The flight plan includes information such as the flight path of the unmanned flying device 2, as well as the flight altitude, flight direction, flight attitude, and flight speed at each position on the flight path.

For example, the flight plan includes a pre-flight plan that is created in advance to confirm the cleaning area to be cleaned by the autonomously traveling cleaning device 4. The pre-flight plan is created to perform a pre-flight in which the unmanned flying device 2 hovers at a predetermined position in the cleaning area to a flight altitude that overlooks the cleaning area in order to capture aerial images of the cleaning area using the imaging unit 12 of the unmanned flying device 2. The pre-flight plan sets the imaging position and imaging altitude on the flight path of the unmanned flying device 2 as the imaging timing of the imaging unit 12. The pre-flight plan may be created to perform a single hover to capture an aerial image at one imaging position in the cleaning area, or may be created to perform a moving flight to each imaging position and hovering at each imaging position in order to capture aerial images at multiple imaging positions in the cleaning area.

When the communication unit 18 receives an automatic flight instruction from the pilot device 3, the flight control unit 50 controls the flight unit 11 based on the pre-flight plan to perform a pre-flight of the unmanned flying device 2.

The flight plan creation unit 51 creates a collection flight plan to perform an automatic collection flight that assists in the collection of objects on the floor or ground of a cleaning area by using downwash generated by the rotation of the flying wings 11a of the flight unit 11. Since the topography and size of a cleaning area differ from one cleaning area to another, and the scattering of objects on a cleaning area differs from one cleaning to the next, the flight plan creation unit 51 creates a collection flight plan for each cleaning.

The flight plan creation unit 51 creates a collection flight plan based on the collection objects and non-collection objects identified by the object discrimination unit 53 and the collection positions set by the collection position setting unit 54. Specifically, the flight plan creation unit 51 creates a collection flight plan that flies a flight path that blows the collection objects to the collection positions by downwash and avoids non-collection objects.

Furthermore, the flight plan creation unit 51 creates a collection flight plan so that, at the end of the automatic collection flight, a post-flight is performed in which the unmanned flying device 2 hovers at a specified position in the cleaning area to a flight altitude that allows a view of the cleaning area, in order to capture aerial images of the cleaning area using the imaging unit 12 of the unmanned flying device 2 and generate imaging data for analyzing the cleaning area. Note that the post-flight may operate in the same manner as the pre-flight.

When the flight control unit 50 performs a pre-flight of the unmanned flying device 2 to generate imaging data of the cleaning area, the flight plan creation unit 51 creates a collection flight plan based on this imaging data. Also, when the flight plan creation unit 51 creates a collection flight plan, the flight control unit 50 controls the flight unit 11 based on this collection flight plan to perform an automatic collection flight of the unmanned flying device 2.

The imaging control unit 52 controls the imaging unit 12 to capture aerial images below the unmanned flying device 2 so that the pilot device 3 can check the flight status while the unmanned flying device 2 is flying manually or automatically. At this time, the imaging control unit 52 causes the imaging unit 12 to periodically capture still images or videos to generate imaging data for confirmation, and transmits the same to the pilot device 3 via the communication unit 18.

When the imaging control unit 52 receives an imaging instruction from the pilot device 3, the imaging control unit 52 controls the imaging unit 12 in accordance with the imaging instruction to capture an aerial image below the unmanned flying device 2 in order to generate imaging data for analyzing the cleaning area. In addition, when the unmanned flying device 2 flies automatically in accordance with the preliminary flight plan, the imaging control unit 52 controls the imaging unit 12 in accordance with the imaging position and imaging altitude set in the preliminary flight plan to capture an aerial image below the unmanned flying device 2 in order to generate imaging data for analyzing the cleaning area. At this time, the imaging control unit 52 causes the imaging unit 12 to capture a still image to generate imaging data for analysis, and causes the communication unit 18 to transmit the imaging data to the pilot device 3 and the autonomously traveling cleaning device 4.

The object discrimination unit 53 discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the image data for analysis captured by the imaging unit 12 in order to create a collection flight plan by the flight plan creation unit 51. For example, the object discrimination unit 53 uses image analysis techniques such as semantic segmentation techniques based on the differences in shape and color between the floor or ground, objects to be collected such as fallen leaves, and objects not to be collected such as walls to distinguish between objects to be collected and objects not to be collected and further distinguish between objects to be collected and objects not to be collected. The object discrimination unit 53 calculates the position information of the distinguished objects to be collected and objects not to be collected in the cleaning area.

The collection position setting unit 54 determines and sets a collection position for collecting the collection objects in the cleaning area based on the results of the discrimination of collection objects and non-collection objects discriminated by the object discrimination unit 53. For example, the collection position setting unit 54 sets a collection position on one side of the non-collection objects that are close to the autonomous traveling cleaning device 4 among the discriminated non-collection objects, and on the side closer to the autonomous traveling cleaning device 4. At this time, the collection position setting unit 54 determines the non-collection objects for which a collection position should be set based on the position of the autonomous traveling cleaning device 4 detected by the cleaning device detection unit 55.

Alternatively, the collection position setting unit 54 may determine the concentration position where the collection objects are most concentrated, and set the collection position on one side of the non-collection objects that is closest to the concentration position. The collection position setting unit 54 may set one collection position for the cleaning area, or may set multiple collection positions.

The cleaning device detection unit 55 detects the position of the autonomously traveling cleaning device 4 relative to the unmanned flying device 2. For example, the unmanned flying device 2 is equipped with a beacon receiver and receives beacon signals such as infrared rays or Bluetooth (registered trademark) sent out by the autonomously traveling cleaning device 4, and the cleaning device detection unit 55 detects the beacon signals sent out by the autonomously traveling cleaning device 4.

The cleaning device detection unit 55 detects the beacon signal to recognize the position of the autonomous cleaning device 4 relative to the unmanned flying device 2, and recognizes the position of the autonomous cleaning device 4 in the cleaning area. For example, the cleaning device detection unit 55 detects infrared rays sent by the LRF 34a (described later) of the measurement unit 34 of the autonomous cleaning device 4 as a beacon signal to recognize the position of the autonomous cleaning device 4. The cleaning device detection unit 55 may reflect the detected position of the autonomous cleaning device 4 in the imaging data captured by the imaging unit 12.

The moving object discrimination unit 56 discriminates moving objects such as people that are moving or present around (within a specified distance from) the unmanned flying device 2. For example, the moving object discrimination unit 56 may be composed of a human presence sensor or the like to discriminate moving objects, or may discriminate moving objects using image analysis technology based on imaging data captured by the imaging unit 12. For example, if the moving object discrimination unit 56 operates while the unmanned flying device 2 is performing automatic flight and discriminates a moving object, the flight control unit 50 temporarily suspends automatic flight and hovers in place. Thereafter, when the moving object discrimination unit 56 no longer discriminates a moving object, the flight control unit 50 resumes automatic flight.

The control device 3 of this embodiment will be described.

The control device 3 controls the unmanned flying device 2 via wireless communication, and is configured with a control unit 20 and a display unit 21, for example, as shown in FIG. 2. The control device 3 also has a control unit 22 that provides overall control of each unit and function of the control device 3, and a memory unit 23 that stores various information. The control device 3 also has a communication unit 24 for wireless communication with external devices such as the unmanned flying device 2.

The control unit 22 is composed of a computer device such as a CPU, and is connected to a storage unit 23 including a ROM, a RAM, an SSD, a flash memory, etc. The control unit 22 is also connected to each part of the control device 3, such as the control unit 20 and the display unit 21.

The control unit 22 is also connected to be able to communicate with external devices via the communication unit 24. The communication unit 24, for example, performs wireless communication with the unmanned flying device 2 using a communication standard such as Bluetooth (registered trademark). The communication unit 24, for example, transmits flight instructions corresponding to the operation of the control unit 20 and instructions to capture aerial images of the cleaning area to the unmanned flying device 2, and receives imaging data of aerial images captured by the imaging unit 12 from the unmanned flying device 2.

The memory unit 23 stores programs and data for controlling each part and various functions of the control device 3, and the control unit 22 performs calculations based on the programs and data stored in the memory unit 23, thereby providing overall control of each part and various functions of the control device 3.

The control unit 20 is composed of, for example, left and right joysticks, and outputs flight instructions for the unmanned flying device 2 in response to the operation of the joysticks, and transmits them to the unmanned flying device 2 via the communication unit 24. The flight instructions indicate, for example, the flight altitude, flight direction, flight attitude, flight speed, etc. of the unmanned flying device 2.

The display unit 21 is composed of a touch panel or the like, and displays information related to the operation of the unmanned flying device 2. For example, the display unit 21 receives imaging data of an aerial image of the cleaning area captured by the imaging unit 12 of the unmanned flying device 2 from the unmanned flying device 2 via the communication unit 24 and displays it. This allows the operator to control the flight of the unmanned flying device 2 using the control unit 20 while checking the aerial image of the cleaning area displayed on the display unit 21.

In addition, in order to obtain imaging data for analysis of the cleaning area during manual flight of the unmanned flying device 2, the display unit 21 displays an imaging button that can be operated to instruct the imaging unit 12 to capture an image, and causes the communication unit 24 to transmit an imaging instruction to the unmanned flying device 2 in response to operation of the imaging button.

Furthermore, in order to perform automatic flight (pre-flight or automatic collection flight) of the unmanned flying device 2, the display unit 21 displays an automatic flight button that can be operated to instruct automatic flight in the cleaning area, and transmits an automatic flight instruction to the unmanned flying device 2 via the communication unit 24 in response to operation of the automatic flight button. The display unit 21 may also be provided with buttons that can individually instruct the unmanned flying device 2 to perform a pre-flight or automatic collection flight.

The control device 3 may be replaced by a tablet terminal or a smartphone, in which case the control unit 20 and the image capture button may be configured as a GUI (Graphical User Interface) on a touch panel. Furthermore, the control device 3 may be configured to be operable using voice input operation technology.

This section describes an autonomous cleaning device 4 according to an embodiment of the present invention.

The autonomous traveling cleaning device 4 travels through a wide area such as a park or parking lot as a cleaning area, collecting and cleaning up debris such as dust and fallen leaves scattered on the floor or ground of the cleaning area as collection items. As an example, as shown in FIG. 4, a parking lot, which is a cleaning area, has multiple parking spaces, in which vehicles, which are not collection items, are parked, and the parking lot has walls, which are not collection items. In such a parking lot, the autonomous traveling cleaning device 4 cleans the cleaning area where fallen leaves, which are collection items, are scattered, as shown in FIG. 5.

As shown in FIG. 1, the autonomously traveling cleaning device 4 includes a device main body 30 for housing each component, a travel unit 31 for traveling the device main body 30, a cleaning unit 32 for cleaning the floor or ground below the device main body 30, and a travel operation unit 33 for accepting manual operation of the travel unit 31.

The autonomous cleaning device 4 also includes a measurement unit 34 that measures the positional relationship between the device body 30 and surrounding walls, vehicles, obstacles (e.g., ornaments, etc.), and other non-collection targets, and an imaging unit 35 that captures images of the device body 30. As shown in FIG. 2, the autonomous cleaning device 4 also includes an operation display unit 36 for operating various functions of the autonomous cleaning device 4 and displaying various information, and a power supply unit 37 that supplies power to each unit of the autonomous cleaning device 4 and controls the remaining charge and charging of the battery (not shown).

Furthermore, the autonomous traveling cleaning device 4 includes a control unit 38 that controls each part and each function of the autonomous traveling cleaning device 4 (traveling by the traveling unit 31, cleaning work by the cleaning unit 32, measurement by the measuring unit 34, etc.), and a memory unit 39 that stores a cleaning plan consisting of the traveling data of the traveling unit 31 and the cleaning data of the cleaning unit 32. The autonomous traveling cleaning device 4 also includes a communication unit 40 (first communication unit) for wireless communication with external devices such as the unmanned flying device 2.

Next, we will provide an overview of the operation of the autonomous cleaning device 4.

Before performing automatic traveling cleaning of a cleaning area, the autonomous traveling cleaning device 4 acquires an environmental map of the cleaning area in advance, and creates a cleaning plan including traveling data and cleaning data for the automatic traveling cleaning based on the environmental map. The autonomous traveling cleaning device 4 associates the environmental map and cleaning plan with the cleaning area and stores them in the memory unit 39. The traveling data indicates the traveling path and traveling state of the automatic traveling cleaning, and the cleaning data indicates the cleaning state of the automatic traveling cleaning. The autonomous traveling cleaning device 4 sets a specified collection position in the cleaning area, and when a collection object is collected at this collection position, creates a cleaning plan so that the collection object at this collection position is cleaned preferentially. The autonomous traveling cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 by the control unit 38 to perform automatic traveling cleaning based on the environmental map and cleaning plan associated with the cleaning area to be cleaned.

A cleaning plan may be composed of, for example, multiple steps along a travel route, with each step being associated with travel data and cleaning data. The interval between steps may be set to a predetermined time interval (e.g., 25 ms) or may be set to a predetermined travel distance (e.g., 0.5 m). The cleaning plan may also be stored in association with the time elapsed since the start of cleaning for each step.

The driving data includes, for example, self-position data on the driving route of the driving unit 31 (X and Y coordinates indicating the self-position on the environmental map, and the angle relative to the direction of the starting position), steering flags (straight ahead, left turn, right turn), driving speed in the direction of travel [m/s], and turning speed [deg/s], and the driving route can be illustrated based on the driving data. Alternatively, the driving speed may be switched to one of a number of speed stages, and the driving data may be stored by converting the stepwise speed of the driving speed into a number (for example, 8 stages from 0 to 7).

The cleaning data includes, for example, the rotation speed of the brush 32a of the cleaning unit 32 and the suction volume of the suction unit 32b. The rotation speed of the brush 32a and the suction volume of the suction unit 32b may be switched to one of multiple levels of strength, and the cleaning data may be stored by converting the levels of strength of the rotation speed and suction volume into numbers (for example, three levels from 0 to 2, or five levels from 0 to 4, etc.). The cleaning data may also include the operation/stop and rotation speed of the brush 32a and the suction unit 32b.

Next, we will explain each part of the autonomous cleaning device 4.

The running unit 31 is provided at the bottom of the device body 30, and includes, for example, a pair of front wheels 31a as drive wheels and rear wheels 31b as auxiliary wheels. The front wheels 31a are provided on both sides (left and right sides) of the device width direction at the front side in the traveling direction, and include a pair of left and right running drive motors (not shown) and a front wheel rotation encoder (not shown). The running unit 31 drives the running drive motor to rotate the front wheels 31a to move the device body 30 forward, and stops the device body 30 by stopping the rotation of the front wheels 31a. The running speed of the autonomous running cleaning device 4 (running unit 31) is adjusted (accelerated and decelerated) by controlling the drive of the running drive motor. Note that the running unit 31 may move the device body 30 backward by the running drive motor reversing the front wheels 31a.

The pair of front wheels 31a are also equipped with a steering rotation encoder (not shown), and the device body 30 is pivoted to the left or right by rotating the pair of front wheels 31a in opposite directions. For example, the device turns left by rotating the right front wheel 31a forward and the left front wheel 31a in reverse, while the device turns right by rotating the right front wheel 31a in reverse and the left front wheel 31a in forward. The turning radius of the autonomous cleaning device 4 (traveling unit 31) is adjusted by controlling the rotation of each of the pair of front wheels 31a.

The rear wheels 31b are rotatably provided at the rear of the traveling direction in the center of the width direction of the device. The rear wheels 31b rotate in response to the movement of the device body 30 caused by the drive of the pair of front wheels 31a. Note that in this embodiment, an example is described in which the front wheels 31a are driving wheels and the rear wheels 31b are auxiliary wheels, but the present invention is not limited to this example. For example, in other embodiments, the front wheels 31a are auxiliary wheels and the rear wheels 31b are driving wheels.

When the autonomous cleaning device 4 performs autonomous cleaning of a cleaning area, the driving unit 31 operates according to the control of the control unit 38 based on the driving data of the environmental map and cleaning plan associated with the cleaning area.

The cleaning unit 32 is provided at the bottom of the device body 30 and is configured to clean the floor or ground below the device body 30. When the autonomous cleaning device 4 performs autonomous cleaning of a cleaning area, the cleaning unit 32 operates according to the control of the control unit 38 based on the environmental map associated with the cleaning area and the cleaning data of the cleaning plan.

The cleaning unit 32 is composed of a dry cleaning mechanism that collects and cleans garbage such as dust and fallen leaves on the floor or ground as collection objects. The cleaning unit 32 is equipped with a brush 32a that sweeps the floor or ground below the device body 30 to collect collection objects on the floor or ground, a suction unit 32b that sucks up the collection objects collected by the brush 32a, and a collection unit 32c that collects and accumulates the collection objects sucked up by the suction unit 32b. The cleaning unit 32 also includes a brush motor (not shown) that rotates the brush 32a on the floor or ground, and a suction motor (not shown) that operates suction by the suction unit 32b.

The brushes 32a are arranged side by side on both sides (left and right) in the width direction of the device, and the left and right brushes 32a are configured to be large enough to come into contact at the center of the width direction of the device and protrude outward in the width direction of the device. The left brush 32a rotates clockwise when viewed from above, and the right brush 32a rotates counterclockwise when viewed from above, rotating from the front to the rear at the center of the width direction. This causes the objects to be collected in front of the brush 32a to be collected toward the center of the width direction and then toward the rear. The suction unit 32b is arranged behind the brush 32a and is configured, for example, by a suction blower. The suction unit 32b is connected to the collection unit 32c and discharges the objects to be collected that it has sucked into the collection unit 32c.

The driving operation unit 33 is provided on the upper rear side of the device body 30 and includes a handle and a throttle that can be manually operated by an operator. Note that the driving operation unit 33 accepts manual operation by an operator, for example, when creating an environmental map, but does not accept manual operation other than the operation of the emergency stop button when performing automatic driving and cleaning.

The driving operation unit 33 converts the steering amount of the steering wheel by the operator into an electrical signal, and outputs the electrical signal to the driving motor to drive it, thereby rotating the pair of front wheels 31a and making the device main body 30 turn left (left turn) or right (right turn). The turning radius of the autonomous driving cleaning device 4 (driving unit 31) is adjusted according to the steering amount of the driving operation unit 33.

The driving operation unit 33 also converts the amount of throttle rotation (opening) by the operator into an electrical signal, and outputs the electrical signal to the driving motor to drive it, thereby rotating the pair of front wheels 31a and moving the device body 30 forward in the travel direction. The driving speed of the autonomous driving cleaning device 4 (driving unit 31) is adjusted according to the amount of throttle rotation of the driving operation unit 33.

The measurement unit 34 includes a laser range finder (LRF) 34a that measures position information (e.g., angle and distance with respect to the traveling direction of the device body 30) between the device body 30 and non-collection objects such as surrounding walls and obstacles. The measurement unit 34 measures position information between the device body 30 and non-collection objects at predetermined timings (e.g., every 25 ms) while the device body 30 is traveling. The LRF 34a is disposed in a cutout portion formed by cutting out the front side of the device body 30 in the left-right direction, and has a measurement range on the front side and both the left and right sides as shown in FIG. 12. Note that in FIG. 12, the measurement range of the LRF 34a is shown as a sector around the LRF 34a.

The measurement unit 34 also includes an obstacle detection unit (not shown) such as an ultrasonic sensor or an infrared sensor that detects the presence or absence of non-collectible objects such as walls or obstacles within a predetermined distance from the device main body 30. The obstacle detection unit may be in operation at all times while the device main body 30 is traveling, and detects non-collectible objects that may come into contact with the device main body 30 as the device continues to travel, and outputs an alarm signal.

The imaging unit 35 is provided on the top of the device body 30, and captures images of the surroundings of the autonomous cleaning device 4 to generate surrounding image data. The imaging unit 35 is composed of, for example, a CCD camera.

The operation display unit 36 is provided on the upper rear side of the device main body 30, and for example, the operation display unit 36 includes a key switch, an emergency stop button, and a touch panel. The operation display unit 36 also includes an environmental map creation button, an environmental map input button, and an automatic traveling and cleaning execution button. The operation display unit 36 may be configured as an operation display panel attached to the device main body 30, or may be configured as a tablet terminal or the like that is detachably attached to the device main body 30. The operation display unit 36 configured as a tablet terminal or the like is connected to the control unit 38 via wireless communication by the communication unit 40, making it possible to remotely control the autonomous traveling and cleaning device 4.

The key switch is configured to be switchable between on and off. By switching the key switch on, power is supplied from the power supply unit 37 to each part and the autonomous traveling cleaning device 4 operates, whereas by switching the key switch off, the power supply from the power supply unit 37 to each part is stopped and the operation of the autonomous traveling cleaning device 4 is stopped. By operating the emergency stop button, the operation of each part of the autonomous traveling cleaning device 4, in particular the automatic traveling and cleaning, is forcibly stopped (braked).

The touch panel displays various screens in response to control signals from the control unit 38, and transmits operation signals based on touch operations on each screen to the control unit 38. For example, when the key switch is turned on to operate the autonomous cleaning device 4, the touch panel displays an operation screen that allows operation of the autonomous cleaning device 4's automatic traveling and cleaning. The operation screen identifiably displays, for example, an environmental map of the cleaning area where the automatic traveling and cleaning is to be performed, the traveling route, and the traveling data and cleaning data of the cleaning plan. The operation screen may also be configured to allow operation of the traveling speed of the traveling unit 31, the rotation speed of the brush 32a of the cleaning unit 32, the suction amount of the suction unit 32b, etc.

When the environmental map creation button is operated, it causes the control unit 38 to create an environmental map of the cleaning area based on the measurement results of the measurement unit 34 while the autonomously traveling cleaning device 4 is traveling in the cleaning area. At this time, the autonomously traveling cleaning device 4 may manually travel in the cleaning area in response to the operation of the travel operation unit 33, or may simply automatically travel in the cleaning area based on the measurement results of the measurement unit 34. When the environmental map input button is operated, it causes the control unit 38 to input environmental map data from an external device. When the automatic traveling and cleaning execution button is operated, it causes the control unit 38 to execute automatic traveling and cleaning of the autonomously traveling cleaning device 4. Note that the environmental map creation button, environmental map input button, and automatic traveling and cleaning execution button may be displayed operably on the operation screen of the touch panel.

The power supply unit 37 includes a battery (power source) and a charging circuit mounted inside the device main body 30, and charges the battery by connecting to an external power source, and also supplies power to each part of the autonomously traveling cleaning device 4. The power supply unit 37 may output a signal indicating the remaining battery charge to the control unit 38.

The control unit 38 is composed of a computer such as a CPU, and is connected to a storage unit 39 including a ROM, a RAM, an SSD, a flash memory, etc., as shown in FIG. 2. The control unit 38 is also connected to each part of the autonomous traveling cleaning device 4, such as the traveling unit 31, the cleaning unit 32, the traveling operation unit 33, the measuring unit 34, the operation display unit 36, and the power supply unit 37.

The control unit 38 is also connected to be able to communicate with external devices via the communication unit 40. The communication unit 40 wirelessly communicates with external devices such as the operation display unit 36, which is configured as a tablet terminal or the like that is separate from the device main body 30, and the operator's control device 3, using a communication standard such as Bluetooth (registered trademark).

The memory unit 39 stores programs and data for controlling each part and various functions of the autonomous traveling cleaning device 4, and the control unit 38 executes calculation processing based on the programs and data stored in the memory unit 39 to centrally control each part and various functions. For example, the control unit 38 executes a program stored in the memory unit 39 to operate as an environmental map acquisition unit 60, a data reflection unit 61, an object discrimination unit 62, an obstacle recognition unit 63, a collection position setting unit 64, a cleaning plan creation unit 65, an automatic traveling cleaning control unit 66, and a moving object discrimination unit 67, as shown in FIG. 3. This allows the autonomous traveling cleaning device 4 to autonomously travel and work automatically according to a cleaning plan stored in advance. The memory unit 39 stores an environmental map and a cleaning plan in association with a cleaning area to be subjected to automatic traveling cleaning, and may store an environmental map and a cleaning plan for each of a plurality of cleaning areas.

The environmental map acquisition unit 60 acquires an environmental map of the cleaning area where the autonomous cleaning is to be performed and stores it in the memory unit 39. For example, in response to the operation of the environmental map creation button on the operation display unit 36, the environmental map acquisition unit 60 estimates its own position and creates an environmental map in real time using a technique such as SLAM (Simultaneous Localization and Mapping) while the autonomous cleaning device 4 is traveling in the cleaning area. Note that the autonomous cleaning device 4 may be manually driven in response to the operation of the driving operation unit 33 when creating the environmental map, or may be simply automatically driven in the cleaning area based on the measurement results of the measurement unit 34.

At this time, while traveling in the cleaning area, the environmental map acquisition unit 60 acquires position information of the device main body 30 and obstacles around the device main body 30 as the measurement result of the measurement unit 34, and creates a local map of the surroundings of the device main body 30 at predetermined time intervals or predetermined distance intervals based on the measurement result of the measurement unit 34. The environmental map acquisition unit 60 estimates the self-position (coordinates) of the autonomous traveling cleaning device 4 in the local map based on the local map and the detection results (movement amount of the traveling unit 31) by each encoder of the traveling unit 31. The environmental map acquisition unit 60 creates an environmental map of the cleaning area by connecting (combining) each local map.

Alternatively, the environmental map acquisition unit 60 may input environmental map data, such as CAD data, from an external device in response to the operation of an environmental map input button on the operation display unit 36.

The data reflection unit 61 associates the imaging data of the aerial images taken by the unmanned flying device 2 for the cleaning area to be subjected to automatic cleaning with the environmental map and stores it in the memory unit 39, and reflects the imaging data on the environmental map. For example, when the environmental map acquisition unit 60 has acquired the environmental map of the cleaning area and the data reflection unit 61 receives imaging data of the aerial images of the cleaning area from the unmanned flying device 2 via the communication unit 40, the data reflection unit 61 associates the imaging data with the environmental map.

The data reflection unit 61 reflects on the environmental map the data on the position (coordinates) and size (range) of the non-collected object acquired from the imaging data by the object discrimination unit 62, and the data on the position (coordinates) and size (range) of the collection position set by the collection position setting unit 64 based on the data of the non-collected object, and similarly superimposes on the environmental map the data on the position (coordinates) and size (range) of the collection object acquired from the imaging data. Alternatively, the data reflection unit 61 may reflect on the environmental map the data on the position (coordinates) and size (range) of the non-collected object acquired from the imaging data by the object discrimination unit 53 in the unmanned flying device 2, and the data on the position (coordinates) and size (range) of the collection position set by the collection position setting unit 54, and superimpose the data on the position (coordinates) and size (range) of the collection object on the environmental map.

The object discrimination unit 62 discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data of the aerial image of the cleaning area received from the unmanned flying device 2 by the communication unit 40. For example, the object discrimination unit 62 uses image analysis techniques such as semantic segmentation techniques based on the differences in shape and color between the floor or ground, objects to be collected such as fallen leaves, and objects not to be collected such as walls to discriminate between objects to be collected and objects not to be collected and discriminate between objects to be collected and objects not to be collected. The object discrimination unit 62 calculates data on the position (coordinates) and size (range) of the discriminated objects to be collected and objects not to be collected in the cleaning area as the discrimination result.

The object discrimination unit 62 also discriminates non-collectible objects on the floor or ground of the cleaning area based not only on the imaging data of the aerial image of the cleaning area, but also on the recognition information of obstacles recognized by the obstacle recognition unit 63 based on the measurement results of the measurement unit 34. The object discrimination unit 62 may also discriminate between collectible objects and non-collectible objects around the autonomously traveling cleaning device 4 based on surrounding image data captured by the imaging unit 35.

The obstacle recognition unit 63 recognizes obstacles around the device body 30, for example, recognizes the shape of the obstacles, based on the measurement results obtained by the measurement unit 34 measuring the positional relationship between the device body 30 and the obstacles around the device body 30. The obstacle recognition unit 63 also acquires position information of the obstacle relative to the device body 30. The obstacle recognition unit 63 regards the recognized shape and position information of the obstacle as recognition information. The obstacle recognition unit 63 may also recognize obstacles around the autonomously traveling cleaning device 4 based on surrounding image data captured by the imaging unit 35.

The collection position setting unit 64 determines and sets a collection position for collecting the objects in the cleaning area based on the image data of the cleaning area and/or the recognition information of obstacles in the cleaning area, or based on the discrimination result of objects to be collected or objects not to be collected discriminated by the object discrimination unit 62. For example, the collection position setting unit 64 sets a collection position on one side of the non-collection objects close to the autonomously traveling cleaning device 4 among the discriminated non-collection objects, the side closest to the autonomously traveling cleaning device 4.

Alternatively, the collection position setting unit 64 may determine the concentration position where the objects to be collected are most concentrated, and set the collection position on one side of the objects to be collected that is closest to the concentration position and is close to the concentration position. The collection position setting unit 64 may set one collection position for the cleaning area, or may set multiple collection positions.

When the data on non-collectible objects acquired from the image data of the cleaning area by the data reflecting unit 61 is reflected on the environmental map, the cleaning plan creating unit 65 creates a cleaning plan for performing automatic traveling cleaning to clean dirt, such as dirt and sand, on the floor or ground of the cleaning area, and stores the plan in association with the cleaning area and the environmental map in the storage unit 39. Furthermore, when the data on the collectible objects is superimposed on the environmental map, the cleaning plan creating unit 65 creates a special cleaning plan for performing automatic traveling cleaning to clean the collectible objects on the floor or ground of the cleaning area based on the environmental map, and stores the plan in association with the cleaning area and the environmental map in the storage unit 39 separately from the normal cleaning plan. Note that when the collection position is set by the collection position setting unit 64, the cleaning plan creating unit 65 creates a special cleaning plan for performing automatic traveling cleaning to preferentially clean the collectible objects at the collection position. The cleaning plan creating unit 65 creates a cleaning plan for a travel route that maintains a safe distance from non-collectible objects and obstacles in the environmental map. The cleaning plan creation unit 65 may create a cleaning plan to stop cleaning in locations where there are no objects to be collected. The cleaning plan creation unit 65 may display the created normal cleaning plan or special cleaning plan on the operation display unit 36, and may make the plan editable in response to an operation of the operation display unit 36.

In response to the operation of the automatic travel and cleaning execution button on the operation display unit 36, the automatic travel and cleaning control unit 66 reads out from the memory unit 39 the environmental map and the cleaning plan or special cleaning plan associated with the cleaning area to be subjected to automatic travel and cleaning, and controls the travel unit 31 and the cleaning unit 32 to perform automatic travel and cleaning based on the travel data and cleaning data of the environmental map and the cleaning plan or special cleaning plan. During automatic travel and cleaning of the cleaning area with the cleaning plan, the automatic travel and cleaning control unit 66 may change the cleaning plan during automatic travel and cleaning to the special cleaning plan at the position where the collection object is determined, and continue the automatic travel and cleaning. In other words, the automatic travel and cleaning control unit 66 may switch and update to either the cleaning plan or the special cleaning plan depending on the distribution situation of the collection object reflected in the environmental map, and continue the automatic travel and cleaning.

At this time, the automatic traveling and cleaning control unit 66 performs automatic traveling and cleaning while estimating the self-position of the autonomous traveling and cleaning device 4 on the environmental map corresponding to the cleaning plan or special cleaning plan. For example, the automatic traveling and cleaning control unit 66 uses the environmental map acquisition unit 60 to create a local map using a technology such as SLAM, estimates the self-position of the autonomous traveling and cleaning device 4 in the local map, and matches the local map with the environmental map to estimate the self-position on the environmental map. Then, the automatic traveling and cleaning control unit 66 controls the traveling unit 31 by matching the self-position data for each step of the traveling data for the cleaning plan or special cleaning plan with the self-position estimated on the environmental map, while controlling the cleaning work for each step based on the cleaning data of the cleaning unit 32.

Furthermore, during the automatic traveling and cleaning, the automatic traveling and cleaning control unit 66 detects non-collectible objects based on the measurement results of the measurement unit 34, controls the traveling unit 31 to decelerate when the distance between the autonomous traveling and cleaning device 4 and the non-collectible object approaches a predetermined safe distance, and controls the traveling unit 31 to stop when the distance between the autonomous traveling and cleaning device 4 and the non-collectible object falls below the predetermined safe distance.

The autonomous traveling and cleaning control unit 66 temporarily suspends the autonomous traveling and cleaning operation if the moving object discrimination unit 67 discriminates a moving object that is moving or present around the autonomous traveling and cleaning device 4. After that, when the moving object discrimination unit 67 no longer discriminates a moving object, the autonomous traveling and cleaning control unit 66 resumes the autonomous traveling and cleaning operation.

The moving object discrimination unit 67 discriminates moving objects such as people that are moving or present around (within a specified distance from) the autonomous cleaning device 4. For example, the moving object discrimination unit 67 may be configured with a human presence sensor or the like to discriminate moving objects, or may discriminate moving objects using image matching technology based on surrounding image data captured by the imaging unit 35.

In the cleaning system 1 of this embodiment, the automatic traveling and cleaning operation of the autonomous traveling cleaning device 4 when an operator manually flies the unmanned flying device 2 by operating it will be described with reference to the flowcharts in Figures 8 and 9.

First, as shown in FIG. 8, the operator operates the control device 3 to start the unmanned flying device 2 (step S1), and operates the control unit 20 of the control device 3 to control the unmanned flying device 2 to manually fly it above the cleaning area to be cleaned by automatic cleaning. At this time, the control device 3 transmits flight instructions to the unmanned flying device 2 (step S2), and the flight control unit 50 controls the flight unit 11 of the unmanned flying device 2 to manually fly in accordance with the received flight instructions (step S3). During manual flight, the unmanned flying device 2 has the imaging control unit 52 control the imaging unit 12 to capture an aerial image below the unmanned flying device 2, generate imaging data for confirmation, and transmit it to the control device 3 (step S4). The unmanned flying device 2 may take and transmit a video as the imaging data for confirmation, but it is not limited to a video, and may also take still images every few seconds and transmit them one by one.

The control device 3 receives the image data for confirmation from the unmanned flying device 2 and displays it on the display unit 21 (step S5), and the worker can control the unmanned flying device 2 while checking the aerial image displayed on the display unit 21. For example, the worker can check the collection target such as fallen leaves on the floor or ground of the cleaning area from the aerial image on the display unit 21, as shown in Figures 6 and 7, and use the downwash of the flight unit 11 to control the unmanned flying device 2 to fly so as to blow and collect the collection target toward a specified collection direction (collection position) (step S6). In addition, the worker can check the non-collection target such as a wall on the floor or ground of the cleaning area from the aerial image on the display unit 21, as shown in Figure 7, and control the unmanned flying device 2 to fly so as to blow and collect the collection target with the adjacent position of the non-collection target as the collection position.

After collecting the items to be collected at the collection position, while manually flying the unmanned flying device 2 above the cleaning area, the worker operates the image capture transmission button on the control device 3 to cause the unmanned flying device 2 to capture an aerial image of the cleaning area to obtain imaging data for analysis of the cleaning area. At this time, the control device 3 transmits an image capture transmission command to the unmanned flying device 2 (step S7), and in the unmanned flying device 2, the imaging control unit 52 controls the imaging unit 12 in response to the received image capture transmission command to capture an aerial image below the unmanned flying device 2 and generate imaging data (step S8). The unmanned flying device 2 also transmits the captured imaging data for analysis to the autonomously traveling cleaning device 4. The unmanned flying device 2 only needs to capture and transmit at least one still image as imaging data for analysis.

When the operation of acquiring the imaging data for analyzing the cleaning area is completed, the operator operates the control unit 20 of the control device 3 to control the unmanned flying device 2 and end the manual flight. At this time, the control device 3 transmits a control instruction to end the flight to the unmanned flying device 2 (step S9), and in response to the received control instruction to end the flight, the flight control unit 50 of the unmanned flying device 2 controls the flight unit 11 to end the manual flight (step S10), for example, to return to the starting position of the manual flight. Alternatively, the operator may perform all flights of the unmanned flying device 2 until the flight ends by manually flying the unmanned flying device 2 by operating the control device 3. The operator may also keep the unmanned flying device 2 hovering without ending the flight.

On the other hand, as shown in FIG. 9, after the autonomous cleaning device 4 is started (step S11), the environmental map acquisition unit 60 acquires an environmental map of the cleaning area where the autonomous cleaning is to be performed, and stores the map in the memory unit 39 (step S12). The autonomous cleaning device 4 also receives image data for analysis of the cleaning area captured by the unmanned flying device 2 from the unmanned flying device 2 (step S13), and performs image analysis of the image data using the object discrimination unit 62 to acquire data on the position (coordinates) and size (range) of non-collectible objects and data on the position (coordinates) and size (range) of collectible objects (step S14). Furthermore, the data reflection unit 61 reflects the acquired data on non-collectible objects on the environmental map, and superimposes the acquired data on collectible objects on the environmental map (step S15).

The autonomously traveling cleaning device 4 uses the object discrimination unit 62 to discriminate between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the data on objects to be collected and objects not to be collected and their collection positions obtained from the imaging data received from the unmanned flying device 2 (step S16).

If there are no objects to be collected in the cleaning area (step S16: No), the autonomously traveling cleaning device 4 creates a normal cleaning plan for performing autonomous traveling cleaning to clean up dirt, sand, and other debris using the cleaning plan creation unit 65 (step S17), and stores the plan in the memory unit 39 in association with the cleaning area and the environmental map.

On the other hand, if there are objects to be collected in the cleaning area (step S16: Yes), the autonomously traveling cleaning device 4 creates a special cleaning plan for performing autonomously traveling cleaning so as to prioritize and focus on cleaning the objects to be collected on the floor or ground of the cleaning area based on the environmental map on which the data of the objects to be collected in the cleaning area is reflected and the data of the objects to be collected is superimposed by the cleaning plan creation unit 65 (step S18), and stores the plan in the memory unit 39 in association with the cleaning area and the environmental map.

Next, the autonomous traveling cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 to perform autonomous traveling cleaning by the autonomous traveling cleaning control unit 66 based on the environmental map and the cleaning plan or special cleaning plan associated with the cleaning area to be subjected to autonomous traveling cleaning (step S19).

In the cleaning system 1 of this embodiment, the automatic traveling and cleaning operation of the autonomous traveling cleaning device 4 when the unmanned flying device 2 is automatically flown will be explained with reference to the flowchart in Figure 10.

First, the worker operates the control device 3 to start the unmanned flying device 2 (step S21), and operates the automatic flight button of the control device 3 to automatically fly the unmanned flying device 2 in the cleaning area to be cleaned by automatic cleaning (step S22). At this time, the control device 3 transmits an automatic flight instruction to the unmanned flying device 2, and the unmanned flying device 2 starts automatic flight in response to the received automatic flight instruction. First, the flight control unit 50 controls the flight unit 11 based on the pre-flight plan to perform a pre-flight. During the pre-flight, the imaging control unit 52 of the unmanned flying device 2 controls the imaging unit 12 to capture aerial images of the cleaning area and generate imaging data for analysis of the cleaning area. In other words, imaging data of the cleaning area before the collection of the collection objects is generated (step S23).

When the image data of the cleaning area is generated, the object discrimination unit 53 of the unmanned flying device 2 performs image analysis on the generated image data (step S23), acquires data on non-collectible objects on the floor or ground of the cleaning area, and transmits the data to the autonomously traveling cleaning device 4 (step S24). Furthermore, the object discrimination unit 53 acquires data on collectible objects on the floor or ground of the cleaning area by performing image analysis on the generated image data (step S25). After that, the object discrimination unit 53 discriminates collectible objects and non-collectible objects on the floor or ground of the cleaning area based on the image data in order to create a collection flight plan by the flight plan creation unit 51. In addition, the collection position setting unit 54 of the unmanned flying device 2 determines and sets the collection position where the collectible objects should be collected in the cleaning area based on the acquired data on the collectible objects and non-collectible objects (step S26), as shown in FIG. 14.

When data on the objects to be collected and the objects not to be collected is acquired by image analysis of the image data of the cleaning area, the flight plan creation unit 51 of the unmanned flying device 2 creates a collection flight plan based on the identified objects to be collected and the objects not to be collected and the set collection positions (step S27). At this time, the flight plan creation unit 51 creates a collection flight plan in which the unmanned flying device 2 flies while avoiding the objects not to be collected and flies in a manner that blows the objects to be collected to the collection positions by using downwash.

When the collection flight plan is created, the flight control unit 50 controls the flight unit 11 based on the collection flight plan to perform an automatic collection flight (step S28), as shown in Figure 14. Through this automatic collection flight, the unmanned flying device 2 flies in a manner that utilizes the downwash of the flight unit 11 to perform a collection assistance operation that blows and gathers the objects to be collected on the floor or ground of the cleaning area toward the collection position, as shown in Figures 14 and 15.

At the end of the automatic collection flight, the flight control unit 50 controls the flight unit 11 to perform a post-flight based on the collection flight plan, and the imaging control unit 52 controls the imaging unit 12 to capture aerial images of the cleaning area and generate imaging data for analysis of the cleaning area. That is, imaging data of the cleaning area after collection of the collection objects is generated, and the object discrimination unit 53 performs image analysis on the generated imaging data (step S29). The unmanned flying device 2 obtains data on the collection objects in the imaging data from the results of the image analysis, and transmits this data to the autonomously traveling cleaning device 4 together with data on the collection position that has already been set (step S30).

When the acquired data on the objects to be collected and the data on the collection location that has already been set are transmitted to the autonomously traveling cleaning device 4, the unmanned flying device 2 ends the automatic flight. The flight control unit 50 controls the flying unit 11 to end the automatic flight, for example, to return to the starting position of the automatic flight. Alternatively, the flight control unit 50 may allow the unmanned flying device 2 to continue hovering over the cleaning area.

On the other hand, after the autonomous cleaning device 4 is started (step S31), the environmental map acquisition unit 60 acquires an environmental map of the cleaning area where the autonomous cleaning is to be performed, and stores the map in the memory unit 39 (step S32). The environmental map acquisition unit 60 creates an environmental map based on the measurement results of the LRF 34a of the measurement unit 34, for example. At this time, as shown in FIG. 12, the measurement range of the LRF 34a in the cleaning area is shown as a sector around the autonomous cleaning device 4 (LRF 34a), but randomly parked vehicles may become obstacles from the perspective of the LRF 34a, blocking the measurement range, creating blind spots in the measurement of the LRF 34a, and non-collectable objects may exist that cannot be detected from the measurement results of the measurement unit 34.

The autonomously traveling cleaning device 4 also receives data on non-collectible objects in the cleaning area acquired by the unmanned flying device 2 from the unmanned flying device 2, and reflects the data on the non-collectible objects on the environmental map by the data reflecting unit 61 (step S33). At this time, the imaging range of the unmanned flying device 2 is wide, as shown by the dashed line in FIG. 13, and the imaging data of the cleaning area captured includes a range that cannot be grasped by the measurement results of the measurement unit 34. Therefore, by reflecting the non-collectible objects acquired by image analysis of the imaging data on the environmental map, it is possible to reflect on the environmental map the existence of cleaning areas that cannot be detected from the measurement results of the measurement unit 34 due to blind spots of obstacles such as vehicles. After that, it is sufficient to reflect the collection objects and collection positions. By reflecting the entire cleaning area without blind spots on the environmental map, it is possible to reliably clean the collection objects that exist in the blind spots.

The autonomously traveling cleaning device 4 superimposes the collection objects on the floor or ground of the cleaning area onto an environmental map based on the data of the collection objects and collection locations received from the unmanned flying device 2 using the data reflection unit 61, and reflects the collection locations on the environmental map (step S35).

If there are no objects to be collected in the cleaning area (step S36: No), the autonomously traveling cleaning device 4 creates a normal cleaning plan for performing autonomous traveling cleaning to clean up dirt, sand, and other debris using the cleaning plan creation unit 65 (step S37), and stores the plan in the memory unit 39 in association with the cleaning area and the environmental map.

On the other hand, if there are objects to be collected in the cleaning area (step S36: Yes), the autonomously traveling cleaning device 4 creates a special cleaning plan for performing autonomous traveling cleaning to clean the objects to be collected on the floor or ground of the cleaning area, as shown in FIG. 15, based on the environmental map on which the data of the objects to be collected and the data of the objects to be collected in the cleaning area is reflected by the cleaning plan creation unit 65 and superimposed (step S38), and stores the plan in the memory unit 39 in association with the cleaning area and the environmental map.

Next, the autonomous traveling cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 to perform autonomous traveling cleaning by the autonomous traveling cleaning control unit 66 based on the environmental map and the cleaning plan or special cleaning plan associated with the cleaning area to be subjected to autonomous traveling cleaning (step S39).

As described above, in a configuration in which the unmanned flying device 2 is equipped with an object discrimination unit 53 and a collection position setting unit 54, a large computing capacity is required for the control unit 16 of the unmanned flying device 2, and the unmanned flying devices 2 that can be implemented are limited. Therefore, the operation of setting collection objects, non-collection objects, and collection positions in image data using the object discrimination unit 62, obstacle recognition unit 63, and collection position setting unit 64 of the autonomously traveling cleaning device 4 will be described with reference to the flowchart in Figure 11.

The worker operates the control device 3 to start the unmanned flying device 2 (step S41), and operates the automatic flight button on the control device 3 to automatically fly the unmanned flying device 2 in the cleaning area to be cleaned by automatic cleaning (step S42). At this time, the control device 3 transmits an automatic flight instruction to the unmanned flying device 2, and the unmanned flying device 2 starts preliminary automatic flight in response to the received automatic flight instruction. During automatic flight, the imaging control unit 52 of the unmanned flying device 2 controls the imaging unit 12 to capture aerial images of the cleaning area, generate imaging data for analysis of the cleaning area, and transmit it to the autonomous cleaning device 4 (step S43).

On the other hand, the autonomous traveling cleaning device 4, after starting (step S51), similarly to the above, acquires an environmental map of the cleaning area where the autonomous traveling cleaning is performed by the environmental map acquisition unit 60 and stores it in the storage unit 39 (step S52). In addition, when the autonomous traveling cleaning device 4 receives imaging data from the unmanned flying device 2 (step S53), the object discrimination unit 62 performs image analysis based on the imaging data, and acquires and discriminates data of the collection object and the non-collection object on the floor surface or ground of the cleaning area (step S54). On the other hand, the data of the non-collection object is reflected in the environmental map by the data reflection unit 61 (step S55), and the cleaning plan creation unit 65 creates a normal cleaning plan in advance for performing the autonomous traveling cleaning so as to clean up garbage such as soil and sand in the cleaning area (step S56). After that, based on the data of the non-collection object and the collection object acquired and discriminated from the result of the image analysis, the collection position setting unit 64 sets a collection position where the collection object should be collected in the cleaning area, as shown in FIG. 14. The autonomously traveling cleaning device 4 transmits data on the identified non-collection targets and collection targets, as well as data on the set collection location, to the unmanned flying device 2 (step S57).

When the unmanned flying device 2 receives data on non-collectible objects and collectible objects and data on the set collection location (step S44), the flight plan creation unit 51 creates a collection flight plan based on the determined collectible objects and non-collectible objects and the set collection location (step S45). The flight control unit 50 of the unmanned flying device 2 controls the flight unit 11 based on the created collection flight plan to perform an automatic collection flight, and collects the collectible objects toward the collection location (step S46). At the end of the automatic collection flight, the flight control unit 50 controls the flight unit 11 based on the collection flight plan to perform a post-flight, and the imaging control unit 52 controls the imaging unit 12 to capture an aerial image of the cleaning area and generate imaging data for analysis of the cleaning area. That is, imaging data of the cleaning area after collection of the collection objects is generated and transmitted to the autonomously traveling cleaning device 4 (step S47).

When the autonomous traveling cleaning device 4 receives imaging data for analysis from the unmanned flying device 2 that performed the automatic collection flight, the object discrimination unit 62 performs image analysis based on the imaging data to acquire and discriminate data on the objects to be collected on the floor or ground of the cleaning area (step S58). If the image analysis result shows that there is no object to be collected in the received imaging data (step S59: No), the autonomous traveling cleaning control unit 66 controls the traveling unit 31 and the cleaning unit 32 to perform autonomous traveling cleaning based on the environmental map associated with the cleaning area and the traveling data and cleaning data of a normal cleaning plan for performing autonomous traveling cleaning to clean up dirt such as soil and sand that has already been created by the cleaning plan creation unit 65 (step S62).

On the other hand, if the image analysis results in the presence of a collection target in the received image data (step S59: Yes), the data reflecting unit 61 superimposes the acquired data of the collection target on the environmental map and reflects and sets the data of the collection location on the environmental map (step S60), and the cleaning plan creating unit 65 creates a special cleaning plan for performing automatic traveling and cleaning to clean the collection target on the floor or ground of the cleaning area, and stores the special cleaning plan in association with the cleaning area and the environmental map in the storage unit 39. After that, the cleaning plan creating unit 65 updates the normal cleaning plan for automatic traveling and cleaning of the cleaning area to the special cleaning plan (step S61). Next, the autonomous traveling and cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 to perform automatic traveling and cleaning based on the environmental map and the special cleaning plan associated with the cleaning area to be subjected to automatic traveling and cleaning by the autonomous traveling and cleaning control unit 66 (step S62).

For example, the autonomous driving cleaning device 4 may perform autonomous driving cleaning immediately after creating a cleaning plan in advance by reflecting data on non-collectible objects obtained by image analysis of the imaging data transmitted from the above-mentioned unmanned flying device 2 in the environmental map in advance. Furthermore, if the autonomous driving cleaning device 4 performs image analysis of the imaging data transmitted from the unmanned flying device 2 while performing autonomous driving cleaning according to a normal cleaning plan to determine data on collectible objects, it may create a special cleaning plan based on the data on the collectible objects, update the cleaning plan to the special cleaning plan, and continue autonomous driving cleaning.

As described above, according to the present embodiment, the cleaning system 1 equipped with the autonomously traveling cleaning device 4 capable of performing automatic traveling cleaning, which autonomously travels and automatically cleans, further includes the unmanned flying device 2. The autonomously traveling cleaning device 4 includes a device main body 30, a traveling unit 31 that travels the device main body 30, a cleaning unit 32 that cleans on the travel path of the device main body 30, a control unit 38 that functions as an environmental map acquisition unit 60 that acquires an environmental map of the cleaning area in which the automatic traveling cleaning is performed, and a communication unit 40 (first communication unit) that wirelessly communicates with the unmanned flying device 2. The unmanned flying device 2 includes an imaging unit 12 and a communication unit 18 (second communication unit) that wirelessly communicates with the autonomously traveling cleaning device 4. When the unmanned flying device 2 flies above the cleaning area, the imaging unit 12 images the cleaning area from the sky and generates imaging data. The communication unit 18 wirelessly communicates with the communication unit 40 to transmit the imaging data to the autonomously traveling cleaning device 4. The control unit 38 of the autonomous traveling cleaning device 4 functions as a data reflection unit 61 that reflects the imaging data captured by the imaging unit 12 on the environmental map by associating the imaging data with the environmental map.

With this configuration, the cleaning system 1 can obtain information on blind spots that the autonomous cleaning device 4 cannot see by reflecting the image data of the cleaning area captured by the unmanned flying device 2 on the environmental map, and can obtain an environmental map that shows the cleaning area in more detail. Therefore, the autonomous cleaning device 4 can recognize the entire cleaning area.

In addition, in this embodiment, the control unit 38 of the autonomous traveling cleaning device 4 functions as an object discrimination unit 62 that discriminates between objects to be collected and objects to be not collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit 12. The data reflection unit 61 reflects the objects to be collected and objects to be not collected that have been discriminated by the object discrimination unit 62 on the environmental map. The control unit 38 of the autonomous traveling cleaning device 4 functions as a cleaning plan creation unit 65 that creates a cleaning plan for performing autonomous traveling cleaning to clean objects to be collected based on the environmental map, and functions as an autonomous traveling cleaning control unit 66 that performs autonomous traveling cleaning based on the cleaning plan.

This allows the autonomous cleaning device 4 to obtain information about blind spots that it cannot see, and allows for the creation of a cleaning plan that ensures collection is more reliable and does not leave any items left behind in the cleaning area.

In addition, in this embodiment, the unmanned flying device 2 is equipped with a control unit 16 that functions as a flight control unit 50 that controls the collection operation that utilizes downwash during flight to blow and collect collection objects at a predetermined collection position within the cleaning area. After performing the collection operation, the imaging unit 12 captures an image of the cleaning area including the collection position to generate imaging data. The data reflection unit 61 reflects the collection position captured in the imaging data on the environmental map. The cleaning plan creation unit 65 creates a cleaning plan for performing automatic driving cleaning so as to give priority to cleaning collection objects at the collection position based on the environmental map.

As a result, the autonomous cleaning device 4 can clean most of the objects to be collected in the cleaning area by cleaning the collection location where the objects were collected, improving cleaning efficiency. Therefore, objects to be collected in the cleaning area can be collected more easily and reliably, without leaving any behind.

In addition, in this embodiment, the control unit 38 of the autonomous traveling cleaning device 4 functions as an obstacle recognition unit 63 that recognizes obstacles based on the measurement results of the positional relationship of obstacles around the device main body 30. The object discrimination unit 62 discriminates non-collection objects on the floor or ground of the cleaning area based on the recognition information of the obstacles recognized by the obstacle recognition unit 63. The control unit 38 of the autonomous traveling cleaning device 4 also functions as a collection position setting unit 64 that sets a collection position on one side of the non-collection object discriminated by the object discrimination unit 62 based on the imaging data or recognition information.

This allows non-collection items to be used at the collection location, so that collection items can be more reliably blown together, making it easier and more reliable to collect and clean the cleaning area without leaving any collection items behind.

In other words, in this embodiment, the unmanned flying device 2 includes an imaging unit 12 that captures an aerial image of the cleaning area from above when flying over the cleaning area to generate imaging data, a control unit 16, and a communication unit 18 that performs wireless communication with the autonomously traveling cleaning device 4. The control unit 16 functions as an object discrimination unit 53 that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit 12, functions as a collection position setting unit 54 that sets a collection position on one side of the objects not to be collected that are determined based on the imaging data by the object discrimination unit 53, and functions as a flight control unit 50 that controls the collection operation of blowing and collecting objects to a collection position in the cleaning area by using downwash during flight. After the collection operation is performed, the imaging unit 12 captures the cleaning area including the collection position to generate imaging data. The communication unit 18 wirelessly communicates with the autonomously traveling cleaning device 4 to transmit the imaging data captured by the imaging unit 12 to the autonomously traveling cleaning device 4 after the collection operation is performed.

As a result, the unmanned flying device 2 can provide the autonomously traveling cleaning device 4 with the collection location where the collection objects were collected, so that by cleaning this collection location, the autonomously traveling cleaning device 4 can clean most of the collection objects in the cleaning area, improving cleaning efficiency. This makes it easier and more reliable to collect the collection objects in the cleaning area, so that none are left behind.

In addition, in this embodiment, the unmanned flying device 2 further includes a cleaning device detection unit 55 that detects the position of the autonomously traveling cleaning device 4 based on the imaging data. The collection position setting unit 54 sets a collection position on one side of the non-collection objects that are close to the autonomously traveling cleaning device 4 among the non-collection objects identified by the object identification unit 53.

By using non-collection objects close to the autonomous cleaning device 4 as collection locations, the objects can be more reliably blown together, making collection easier and more reliable so that no objects are left behind in the cleaning area.

In addition, in this embodiment, the cleaning device detection unit 55 detects the beacon signal sent by the autonomous cleaning device 4 to detect the position of the autonomous cleaning device 4.

For example, the cleaning device detection unit 55 detects the position of the autonomous cleaning device 4 by detecting infrared rays emitted by a laser range finder as a beacon signal to measure the relative positions of surrounding obstacles.

This allows the position of the autonomous cleaning device 4 to be grasped more easily and reliably, and non-collection objects close to the autonomous cleaning device 4 can be used as collection locations for collection objects, making it possible to more reliably gather and collect the collection objects.

In addition, in this embodiment, the unmanned flying device 2 is equipped with a moving object discrimination unit 56 that discriminates moving objects moving on the floor or ground of the cleaning area, and the flight control unit 50 suspends flight when the moving object discrimination unit 56 discriminates a moving object.

This allows the unmanned flying device 2 to improve safety by suspending flight if a moving object such as a person is present when flying to capture aerial images of a cleaning area or to collect collection items.

In the above embodiment, an example was described in which, when there is a non-collection target in the cleaning area, the collection position setting unit 54 of the unmanned flying device 2 and the collection position setting unit 64 of the autonomously traveling cleaning device 4 set a collection position to one side of the non-collection target, but the present invention is not limited to this example. For example, as shown in FIG. 16, when there is no non-collection target in the cleaning area, the collection position setting unit 54 and the collection position setting unit 64 cannot determine the non-collection target for which to set a collection position.

As shown in FIG. 17, the worker may erect landmarks such as folding screens or partitions that separate the cleaning area on the floor or ground of the cleaning area, and the collection position setting unit 54 or the collection position setting unit 64 may recognize the landmark based on the imaging data of the cleaning area captured by the imaging unit 12, and set the collection position on one side of the recognized landmark. Note that the landmarks may be configured to have a repeating shape so that they can be identified from the imaging data captured by the imaging unit 12 of the unmanned flying device 2 and the measurement results measured by the measurement unit 34 of the autonomously traveling cleaning device 4.

For example, if the object discrimination unit 53 of the unmanned flying device 2 or the object discrimination unit 62 of the autonomously traveling cleaning device 4 does not discriminate any non-collectable objects based on the image data of the cleaning area, the unmanned flying device 2, the control device 3 or the autonomously traveling cleaning device 4 may notify the worker that there are no non-collectable objects and prompt the worker to install a landmark.

This means that even if there are no non-collectible objects in the cleaning area to blow the collectible objects together, landmarks for blowing the collectible objects together can be easily placed in the cleaning area, making it possible to blow the collectible objects together.

The present invention may be modified as appropriate without going against the gist or concept of the invention as can be read from the claims and the entire specification, and cleaning systems, unmanned flying devices, and autonomous cleaning devices that incorporate such modifications are also included in the technical concept of the present invention.

The present invention can be suitably used in a cleaning robot that uses an autonomously traveling cleaning device capable of performing autonomously traveling and cleaning automatically to clean a wide area such as a park or parking lot, and travels through the cleaning area to collect and clean debris such as dust and fallen leaves scattered on the floor or ground of the cleaning area. The present invention can also be suitably used in a cleaning system that includes an autonomously traveling cleaning device such as a cleaning robot that performs autonomously traveling and cleaning, and an unmanned flying device such as a drone equipped with an imaging unit such as a camera.

REFERENCE SIGNS LIST 1 Cleaning system 2 Unmanned flying device 3 Pilot device 4 Autonomous traveling cleaning device 11 Flying section 12 Imaging section 16 Control section 18 Communication section (second communication section)
31 Traveling unit 32 Cleaning unit 34 Measurement unit 38 Control unit 40 Communication unit (first communication unit)
50 Flight control unit 51 Flight plan creation unit 52 Imaging control unit 53 Object discrimination unit 54 Collection position setting unit 55 Cleaning device detection unit 56 Moving object discrimination unit 60 Environmental map acquisition unit 61 Data reflection unit 62 Object discrimination unit 63 Obstacle recognition unit 64 Collection position setting unit 65 Cleaning plan creation unit 66 Automatic travel cleaning control unit 67 Moving object discrimination unit

Claims (11)

A cleaning system including an autonomously traveling cleaning device capable of performing autonomously traveling and automatically cleaning,
Further comprising an unmanned flying device,
The autonomous cleaning device includes:
A device body,
A running unit that runs the device body;
a cleaning unit that performs cleaning on a travel path of the device body;
an environmental map acquisition unit that acquires an environmental map of a cleaning area in which the automatic cleaning is performed;
A first communication unit that wirelessly communicates with the unmanned flying device,
The unmanned flying device is
An imaging unit;
A second communication unit that wirelessly communicates with the autonomously traveling cleaning device,
The imaging unit captures an image of the cleaning area from above when the unmanned flying device flies above the cleaning area to generate imaging data,
The second communication unit wirelessly communicates with the first communication unit to transmit the imaging data to the autonomous traveling cleaning device,
The autonomous cleaning device and/or the unmanned flying device includes an object discrimination unit that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit,
A cleaning system characterized in that the autonomously traveling cleaning device is equipped with a data reflection unit that reflects the imaging data in the environmental map by associating the collection objects and non-collection objects identified by the object discrimination unit with the environmental map. The autonomous cleaning device includes:
a cleaning plan creation unit that creates a cleaning plan for performing the automatic traveling cleaning so as to clean the collection objects based on the environmental map;
The cleaning system according to claim 1 , further comprising: an automatic traveling and cleaning control unit that executes the automatic traveling and cleaning based on the cleaning plan. The unmanned flying device is equipped with a flight control unit that controls a collection operation to blow and collect the collection objects to a predetermined collection position within the cleaning area by utilizing downwash during flight,
The imaging unit, after the collection operation is performed, captures an image of the cleaning area including the collection position to generate the imaging data;
The data reflecting unit reflects the collection position captured in the imaging data on the environmental map,
The cleaning system according to claim 2, characterized in that the cleaning plan creation unit creates the cleaning plan for performing the automatic cleaning so as to prioritize cleaning of the collection objects at the collection location based on the environmental map. The autonomous traveling cleaning device includes an obstacle recognition unit that recognizes an obstacle based on a measurement result of a positional relationship of the obstacle around the device body,
The object discrimination unit discriminates the non-collection objects on a floor surface or a ground surface of the cleaning area based on the recognition information of the obstacle recognized by the obstacle recognition unit,
The cleaning system according to claim 3 , further comprising a collection position setting unit that sets the collection position on one side of the non-collection object determined by the object determination unit based on the imaging data or the recognition information. An unmanned flying device,
an imaging unit that captures an image of a cleaning area from above when flying over the cleaning area to generate imaging data;
an object discrimination unit that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit;
a collection position setting unit that sets a collection position on one side of the non-collection object determined by the object determination unit based on the imaging data;
A flight control unit that controls a collection operation to blow and collect the collection objects to the collection position within the cleaning area by using downwash during flight;
A communication unit that wirelessly communicates with the autonomous cleaning device;
Equipped with
The imaging unit, after the collection operation is performed, captures an image of the cleaning area including the collection position to generate the imaging data;
An unmanned flying device characterized in that the communication unit transmits the imaging data captured by the imaging unit after the collection operation is performed to the autonomously traveling cleaning device via wireless communication with the autonomously traveling cleaning device. a cleaning device detection unit that detects a position of the autonomously traveling cleaning device based on the imaging data;
The unmanned flying device described in claim 5, characterized in that the collection position setting unit sets the collection position on one side of the non-collection objects identified by the object discrimination unit that are closer to the autonomously moving cleaning device. The unmanned flying device according to claim 5, characterized in that the collection position setting unit is erected on the floor or ground of the cleaning area, recognizes landmarks that separate the cleaning area based on the imaging data, and sets the collection position on one side of the recognized landmark. The unmanned flying device according to claim 6, characterized in that the cleaning device detection unit detects the position of the autonomously traveling cleaning device by detecting a beacon signal transmitted by the autonomously traveling cleaning device. The unmanned flying device according to claim 8, characterized in that the cleaning device detection unit detects the position of the autonomous cleaning device by detecting infrared rays emitted by a laser range finder as the beacon signal to measure the positional relationship of surrounding obstacles. a moving object discrimination unit that discriminates a moving object moving on a floor surface or a ground surface of the cleaning area,
The unmanned flying device according to claim 5, wherein the flight control unit suspends flight when the moving object determination unit determines that the moving object is a moving object. An autonomous cleaning device capable of performing autonomous cleaning by autonomously traveling and automatically cleaning,
A device body,
A running unit that runs the device body;
a cleaning unit that performs cleaning on a travel path of the device body;
an environmental map acquisition unit that acquires an environmental map of a cleaning area in which the automatic cleaning is performed;
A communication unit that receives imaging data generated by an unmanned flying device flying above the cleaning area and imaging the cleaning area from above by wireless communication with the unmanned flying device;
An object discrimination unit that discriminates collection objects and non-collection objects on the floor or ground of the cleaning area based on the imaging data received from the unmanned flying device by the communication unit;
An autonomous cleaning device comprising: a data reflection unit that reflects the imaging data in the environmental map by associating the collection objects and non-collection objects identified by the object discrimination unit with the environmental map.

Images