JP7779192B2 - Work support device and work system equipped with the same - Google Patents

Description

The present invention relates to a work support device and a work system equipped with the same.

Conventionally, hydraulic excavators have been known as construction machinery, having a machine body and a work attachment supported on the machine body so that it can be raised or lowered. Patent Document 1 discloses technology in which an autonomous air vehicle equipped with a camera captures images of a work site from above and displays the images on a display device, allowing the operator operating the hydraulic excavator to check the images. In this technology, when the upper rotating body of the hydraulic excavator rotates, the position and orientation of the air vehicle are adjusted so that the upper rotating body is displayed in the same orientation on the display device.

Patent No. 6938389

The technology described in Patent Document 1 has a problem in that if the rotating body rotates while capturing an image of a specific part of the work attachment, a valid image cannot be acquired. Specifically, in the above technology, the air vehicle is positioned at an imaging position so that the entire hydraulic excavator fits within the camera's angle of view (imaging range). However, at actual work sites, operators often want to be able to zoom in and check the tip of the work attachment (work mechanism) while working. If the upper rotating body rotates while the air vehicle is positioned close to the work attachment, the tip of the work attachment will move out of the camera's angle of view, making it impossible to capture a valid image. In this case, it would be possible to adjust the camera orientation to follow the rotation of the rotating body, but this process is cumbersome and there is a possibility of a delay between the rotation and the adjustment of the camera orientation, making it similarly difficult to capture a valid image.

The object of the present invention is to provide a work support device and a work system equipped with the same that can acquire valid images even if the rotating body rotates while imaging a specific part of the work mechanism.

The present invention provides a work support device. The work support device supports a work machine including a lower main body, a rotating body that can rotate relative to the lower main body, and a work mechanism supported on the rotating body, using an air vehicle that has an imaging device and can move to a position in accordance with a generated command signal. The work support device includes a control unit that generates the command signal. When the air vehicle is located at a specific imaging position and the rotating body performs a rotation operation that satisfies a predetermined condition, the control unit generates a movement command signal, which is the command signal for moving the air vehicle to an effective imaging position. The specific imaging position is a position where the imaging device captures an image of a specific part of the work mechanism, and which changes depending on the rotation position of the rotating body. The effective imaging position is a position where the imaging device can capture an image of a specified imaging target, and which does not change depending on the rotation position of the rotating body.

With this configuration, when the flying object is positioned at a specific imaging position, the imaging device captures images of specific parts of the work mechanism, allowing for a detailed understanding of the working status of the work machine. However, if the rotating body rotates under specified conditions in this state, the imaging device will be unable to effectively capture images of the specific parts because the specific imaging position changes depending on the rotation position of the rotating body. Even in such a case, the control unit generates a movement command signal to move the flying object to an effective imaging position where the imaging device can capture the specified imaging target and does not change depending on the rotation position of the rotating body, thereby providing effective images to the worker. Furthermore, compared to adjusting the orientation of the imaging device to follow the rotation of the rotating body, this prevents processing complexity and delays.

The above configuration may further include an image display unit capable of displaying images captured by the imaging device.

With this configuration, when the flying object is positioned at a specific imaging position, the worker can check the image of the specific area captured by the imaging device on the image display unit. Furthermore, even if the rotating body rotates under specified conditions in this state, the flying object moves to an effective imaging position, allowing the worker to view an image of the target to be captured via the image display unit.

In the above configuration, the effective imaging position may be a position where the imaging device can capture an image of the entire work machine as the imaging target.

With this configuration, the imaging device can capture an image of the entire work machine at the effective imaging position, regardless of the rotation position of the rotating body.

In the above configuration, the rotating body may be supported by the lower main body so as to be able to rotate around a central axis of rotation, and the effective imaging position may be above the work machine and on the central axis of rotation.

With this configuration, the imaging device can capture images of the relative positional relationship between the lower body and the rotating body at the effective imaging position, regardless of the rotation position of the rotating body.

In the above configuration, the effective imaging position is a position where the imaging device can capture an image of a specific area as the imaging target, and the specific area may be an area set relative to the lower body.

With this configuration, the imaging device can capture a specific area at the effective imaging position, regardless of the rotation position of the rotating body. Because this specific area is relative to the lower body, it can continue to capture the imaging area even when the lower body is moving.

In the above configuration, the lower body may be a lower running body that can run in the fore-and-aft direction on a running surface, and the specific area may include at least one of the areas in front of and behind the lower running body in the fore-and-aft direction.

With this configuration, the imaging device can capture images of the area in front or behind the lower body at the effective imaging position, regardless of the rotation position of the rotating body.

In the above configuration, the work mechanism may be capable of performing work on a work object, and the effective imaging position may be a position where the imaging device can capture an image of the work object as the imaging target.

With this configuration, the imaging device can capture images of the work object at the effective imaging position, regardless of the rotation position of the rotating body.

The above configuration may further include an input unit capable of receiving information regarding candidate positions to which the aircraft can next move, and the effective imaging position may be the candidate position to which the aircraft can next move input to the input unit.

With this configuration, the imaging device can capture images of the imaging target at the proposed movement position, regardless of the rotation position of the rotating body.

The above configuration may further include a candidate position display unit capable of displaying information regarding the candidate movement positions.

With this configuration, the operator can set the next movement position of the aircraft using the input unit based on the information displayed in the candidate position display unit.

In the above configuration, the candidate movement positions may include the specific imaging position.

With this configuration, the aircraft can be moved again to a specific imaging position based on the operator's intentions.

In the above configuration, the control unit may be capable of receiving information corresponding to the amount of operation for rotating the rotating body, and the condition may be that the amount of operation exceeds a predetermined threshold.

With this configuration, when an operation is input in which the amount of operation exceeds a predetermined threshold, the aircraft can be moved to an effective imaging position. Therefore, even with a slight turning movement, the aircraft can remain at a specific imaging position.

In the above configuration, the control unit may be capable of determining whether the specific part is included in the image captured by the imaging device, and the condition may be that the specific part changes from being included in the image to not being included in the image.

With this configuration, if a specific part moves out of the imaging range of the imaging device due to a rotational movement, the aircraft can be moved to an effective imaging position. Therefore, even with a slight rotational movement, the aircraft can remain at the specific imaging position.

In the above configuration, the control unit may be capable of receiving information corresponding to the rotation angle of the rotating body relative to the lower body, and the condition may be that the amount of change in the rotation angle exceeds a predetermined threshold.

With this configuration, when the change in the rotation angle of the rotating body exceeds a predetermined threshold, the flying body can be moved to an effective imaging position. Therefore, even with a slight rotation, the flying body can remain at a specific imaging position.

In the above configuration, the specific imaging positions may include a right-side specific imaging position where the imaging device images the specific part from the right side of the working mechanism, and a left-side specific imaging position where the imaging device images the specific part from the left side of the working mechanism.

This configuration allows specific areas to be imaged from a more desirable direction depending on the structure of the work machine and the conditions at the work site.

In the above configuration, the control unit may input the generated command signal to a remote control device capable of sending and receiving signals to and from the aircraft, thereby moving the aircraft to a position corresponding to the command signal.

With this configuration, the aircraft can be moved to an effective imaging position using a remote control device capable of sending and receiving signals with the aircraft.

The above configuration may further include a transmitter capable of transmitting a command signal generated by the control unit to the aircraft.

With this configuration, a command signal can be sent directly to the aircraft via the transmitter, allowing the aircraft to move to an effective imaging position.

The present invention provides a work system. The work system comprises a work machine including a lower body, a rotating body that can rotate relative to the lower body, and a work mechanism supported by the rotating body, and the work support device described above.

With this configuration, even if the rotating body rotates while capturing an image of a specific part of the work mechanism, it is possible to obtain valid images using a work system including a work support device and a work machine.

The present invention also provides a work system. The work system includes an aircraft having an imaging device and capable of moving to a position in response to a generated command signal, and the work support device described above.

With this configuration, even if the rotating body rotates while capturing images of a specific part of the work mechanism, it is possible to obtain valid images using a work system including a work support device and an aircraft.

This invention provides a work support device and a work system equipped with the same that can acquire valid images even if the rotating body rotates while capturing an image of a specific part of the work mechanism.

1 is a side view showing a work machine according to an embodiment of the present invention. 1 is a block diagram of a work machine and an air vehicle according to an embodiment of the present invention. 1 is a plan view showing a state in which an aircraft according to an embodiment of the present invention is placed at a specific imaging position. 1 is an enlarged perspective view showing an image of a specific portion of a working mechanism captured by an imaging device according to an embodiment of the present invention; 1 is a plan view showing a state in which an aircraft according to an embodiment of the present invention is placed at an effective imaging position. 1 is a flowchart of a process executed by a work assistance device according to an embodiment of the present invention. FIG. 10 is a plan view showing a state in which the flying object according to a modified embodiment of the present invention is placed at an effective imaging position. FIG. 10 is a perspective view showing a modified embodiment of the flying object according to the present invention capturing an image of a work object at an effective imaging position. 10A and 10B are diagrams showing images displayed on a display device in a modified embodiment of the present invention. 10A and 10B are diagrams showing images displayed on a display device in a modified embodiment of the present invention. 10A and 10B are diagrams showing images displayed on a display device in a modified embodiment of the present invention. FIG. 10 is a diagram showing the relationship between an aircraft, a work machine, a cloud, and a site management side in a modified embodiment of the present invention.

A preferred embodiment of the present invention will now be described with reference to the drawings. Figure 1 is a side view showing a demolition machine 100 (work machine) equipped with a work support device 100S (Figure 2) according to one embodiment of the present invention. Figure 2 is a block diagram of the demolition machine 100 and unmanned aerial vehicle 50 according to this embodiment.

The demolition machine 100 comprises a crawler-type lower traveling body 1 (lower main body), an upper rotating body 2, a cab 3, and a work attachment 4 (work mechanism). The lower traveling body 1 is capable of traveling in a forward and backward direction on a traveling surface such as the ground. The direction of travel of the lower traveling body 1 can be directed left and right by changing the rotation speed of the left and right crawlers on the lower traveling body 1. The upper rotating body 2 is mounted on the lower traveling body 1 so that it can rotate around a rotation center axis CL that extends vertically. The cab 3 is located at the front end of the upper rotating body 2 and allows an operator (worker) to board. Inside the cab 3, operating units and the like, which will be described below, are located.

The work attachment 4 is a member used to perform a specific task. The work attachment 4 is attached to the front end of the upper rotating body 2, adjacent to the cab 3 in the left-right direction of the upper rotating body 2. The work attachment 4 includes a boom 5 supported on the upper rotating body 2 so as to be able to be raised and lowered about a horizontal axis in the left-right direction, an inter-boom 6 supported at the tip of the boom 5 so as to be rotatable about the horizontal axis in the left-right direction, an arm 7 supported at the tip of the inter-boom 6 so as to be rotatable about the horizontal axis in the left-right direction, and a work unit 8 attached to the tip of the arm 7. In the example shown in Figure 1, the work unit 8 is an open-close crushing device, but the work unit 8 may also be an excavation bucket or a crushing breaker.

Note that in Figure 1, the front-rear and left-right directions of the lower running body 1 and the front-rear and left-right directions of the upper rotating body 2 match, but when the upper rotating body 2 rotates relative to the lower running body 1, the front-rear and left-right directions of the two differ.

The demolition machine 100 also has cylinders (hydraulic cylinders) that operate the work attachment 4, including a boom cylinder 9 that raises and lowers the boom 5, an inter-boom cylinder 10 that rotates the inter-boom 6, an arm cylinder 11 that rotates the arm 7, and a work cylinder 12 that operates the work unit 8. The work unit 8 operates in response to the extension and contraction of the work cylinder 12 via a link 13 attached to the tip of the arm 7. The link 13 has a first link 14 and a second link 15.

The boom 5 has a lower main boom 5a and an upper front boom 5b. The main boom 5a and front boom 5b are detachable, and when transporting or storing the demolition machine 100, it is disassembled into a set of the upper rotating body 2 and main boom 5a (base machine set) and a set of the work attachment 4 excluding the main boom 5a (attachment set). Note that the structure of the work attachment 4 is not limited to the above.

As shown in Figure 1, the work attachment 4 of the demolition machine 100 is long, making it difficult for the operator in the cab 3 to check the posture of the work attachment 4 or the state of the working unit 8. In this embodiment, an unmanned aerial vehicle 50 (aerial vehicle) flies around the demolition machine 100 and acquires image information, thereby notifying the operator of the status of the work site and supporting the demolition machine 100. This support includes direct support for the operation of the demolition machine 100, support for the operator operating the demolition machine 100, and support for workers, managers, etc. in remote locations. To provide such support, the demolition machine 100 is equipped with a work support device 100S.

Also, referring to FIG. 2, the demolition machine 100 further includes an operation unit 21, an input unit 22, a position information detection unit 23, an attitude information detection unit 24, a rotation angle detection unit 25, a travel drive unit 31, a rotation drive unit 32, an attachment drive unit 33, a display device 40, and a controller 101. The work support device 100S is composed of the above-mentioned input unit 22, controller 101, and display device 40.

The operating unit 21 is located inside the cab 3 and is operated by the operator. The operating unit 21 includes multiple operating levers that are used to rotate the upper rotating body 2, raise and lower the boom 5, rotate the inter-boom 6, rotate the arm 7, and operate the working unit 8. When the operator operates the operating levers, a signal corresponding to the amount of operation is input to the controller 101.

The input unit 22 is located inside the cab 3. The operator can input various types of information through the input unit 22. As an example, the operator inputs information for using the unmanned aerial vehicle 50 from the input unit 22 (aircraft use switch). The operator also inputs information about the object to be captured by the camera 503 of the unmanned aerial vehicle 50 from the input unit 22. Note that the information about the object to be captured may be, for example, coordinate specification, or may be selected from objects identified by image recognition. A signal corresponding to the information input to the input unit 22 is input to the controller 101.

The position information detection unit 23 acquires position information of the demolition machine 100 at the work site. As an example, the position information detection unit 23 can acquire main body coordinate information, which is information regarding the absolute coordinates at the work site of a main body reference point that is pre-installed on the upper rotating body 2. The main body reference point functions as a GNSS (Global Navigation Satellite System) mobile station. Meanwhile, a GNSS reference station (not shown) is installed to acquire the above-mentioned main body coordinate information. The GNSS reference station is a reference station that is located at the work site or in a position closest to the work site. In addition to the well-known Global Positioning System (GPS), satellite positioning systems such as the Global Navigation Satellite System (GLONASS), Galileo, and the Quasi-Zenith Satellite System (QZSS) may also be used as the GNSS.

The posture information detection unit 24 includes four sensors attached to the boom cylinder 9, inter-boom cylinder 10, arm cylinder 11, and work cylinder 12, respectively, and detects the stroke (extension amount, length) of each cylinder. The posture information detection unit 24 can calculate the posture of the work attachment 4 based on the detected stroke of each cylinder. The lengths of each component, the boom 5, inter-boom 6, arm 7, and work unit 8, used in this calculation are stored in advance in the controller 101. The angles of each component corresponding to the stroke of each cylinder are also stored in the same way. The posture of the work attachment 4 detected by the posture information detection unit 24 is input to the controller 101.

The slewing angle detection unit 25 detects the slewing angle of the upper rotating body 2 relative to the lower running body 1. In this embodiment, the state in which the forward direction of the lower running body 1 and the forward direction of the upper rotating body 2 coincide is defined as a slewing angle of zero degrees, and the slewing angle is detected by considering a right turn of the upper rotating body 2 as positive and a left turn as negative. The slewing angle detected by the slewing angle detection unit 25 is input to the controller 101.

The travel drive unit 31, swivel drive unit 32, and attachment drive unit 33 drive each component of the demolition machine 100. These drive units drive each component at a drive speed that corresponds to the amount of operation received by the operating lever of the operating unit 21. These drive units include hydraulic circuits such as hydraulic pumps and hydraulic motors. The travel drive unit 31 controls the travel of the lower travel unit 1, the swivel drive unit 32 controls the rotation of the upper swivel unit 2, and the attachment drive unit 33 controls the raising, lowering, and operation of the work attachment 4.

The display device 40 is positioned inside the cab 3 so that it can be viewed by the operator. The display device 40 receives a specified display command signal and displays various information to be notified to the operator in response to the display command signal.

The controller 101 is composed of a CPU (Central Processing Unit), a ROM (Read Only Memory) that stores a control program, and a RAM (Random Access Memory) used as the CPU's work area. As shown in FIG. 2, the controller 101 is connected to an operation unit 21, an input unit 22, a position information detection unit 23, an attitude information detection unit 24, a rotation angle detection unit 25, a travel drive unit 31, a rotation drive unit 32, an attachment drive unit 33, and a display device 40. Also, as shown in FIG. 2, the controller 101 can send and receive signals to and from the unmanned aerial vehicle 50 via a wireless communication mechanism (not shown). The controller 101 and the unmanned aerial vehicle 50 may be connected via a network. In this case, the controller 101 may be a server (server computer). The network may have any configuration, but may include a wireless communication network, the Internet, a VPN (Virtual Private Network), a WAN (Wide Area Network), a wired network, or any combination of these.

When the CPU executes a control program stored in ROM, the controller 101 functions as having a drive control unit 102, mode switching unit 103, aircraft control unit 104, image processing unit 105, and display command unit 106. These functional units do not have physical entities, but correspond to units of function executed by the program. In other words, the control executed by these functional units can be said to be essentially executed collectively by the controller 101. Note that each functional unit may be distributed across multiple controllers.

The drive control unit 102 inputs drive command signals to the travel drive unit 31, slewing drive unit 32, and attachment drive unit 33 depending on the operation received by the operating unit 21. As a result, the operation of the lower travel unit 1, upper slewing unit 2, work attachment 4, etc. is controlled.

The mode switching unit 103 has the function of switching the flight mode of the unmanned aerial vehicle 50 according to predetermined conditions. In this embodiment, the mode switching unit 103 switches the flight mode between standby mode, ATT tip confirmation mode (also called monitoring mode or work support mode), and bird's-eye view mode.

The aircraft control unit 104 generates command signals to be input to the unmanned aircraft 50. Specifically, the aircraft control unit 104 controls the position and orientation of the unmanned aircraft 50 according to the flight mode set by the mode switching unit 103. The aircraft control unit 104 sends command signals to the unmanned aircraft 50 (aircraft drive unit 501) to move the unmanned aircraft 50 to a predetermined position and adjust the orientation (attitude) of the unmanned aircraft 50. The target position of the unmanned aircraft 50 is set based on latitude, longitude, and altitude. In addition, the position information of the unmanned aircraft 50 can be obtained from a GPS sensor. The attitude information of the unmanned aircraft 50 includes, for example, information regarding the rotation of the unmanned aircraft 50 around the yaw axis, roll axis, and pitch axis of the unmanned aircraft 50. This attitude information of the unmanned aerial vehicle 50 can be obtained from a sensor such as an inertial measurement unit (IMU) mounted on the unmanned aerial vehicle 50. Furthermore, the aerial vehicle control unit 104 adjusts the orientation of the camera 503 mounted on the unmanned aerial vehicle 50 by sending a command signal to the camera drive unit 502.

The image processing unit 105 accepts image information captured by the camera 503 of the unmanned aerial vehicle 50 and processes the image information so that it can be displayed on the display device 40.

The display command unit 106 inputs the image information processed by the image processing unit 105 to the display device 40. As a result, the operator can view the image from the camera 503 on the display device 40.

The unmanned aerial vehicle 50 is capable of flying above the work site and acquires information about the surrounding area of the work site to support the work of the demolition machine 100. The unmanned aerial vehicle 50 can move to a position according to the command signal generated. When the unmanned aerial vehicle 50 is not in use, it waits at a home point HP (Figure 1) located on the ground. The unmanned aerial vehicle 50 has an aerial vehicle drive unit 501, a camera drive unit 502, and a camera 503 (imaging device).

The aircraft drive unit 501 is used to fly the unmanned aircraft 50, and as an example, has four (or more) propellers, a motor that rotates the propellers, and a battery that supplies power to the motor. By adjusting the rotation and speed of each propeller, the ascent, descent, forward movement, backward movement, turning, hovering, etc. of the unmanned aircraft 50 are controlled.

The camera drive unit 502 includes a motor that adjusts the orientation of the camera 503. In this embodiment, the camera 503 can rotate around a horizontal axis extending left and right so that the angle of view of the camera 503 faces vertically downward from the unmanned aerial vehicle 50 and faces forward.

The camera 503 acquires image information from the unmanned aerial vehicle 50 in a specified imaging area. The camera 503 may be of any type, such as a wide-angle camera. The camera 503 acquires images of the forward environment ahead of the unmanned aerial vehicle 50 using an imaging element such as a CCD (Charge-Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). The camera 503 may, for example, acquire images of the forward environment in real time and supply them to the controller 101 in a stream format at a specified frame rate. The camera 503 also preferably includes a gimbal (not shown). The gimbal functions to keep the optical axis of the camera 503 in a constant direction (for example, a specified direction in a horizontal plane) even if the attitude of the unmanned aerial vehicle 50 changes.

As mentioned above, in this embodiment, the mode switching unit 103 of the controller 101 switches the flight mode of the unmanned aerial vehicle 50 between standby mode, ATT tip confirmation mode, and bird's-eye view mode. The standby mode is a mode in which the unmanned aerial vehicle 50 waits or returns to the home point HP in Figure 1.

The ATT tip confirmation mode is a mode in which the camera 503 of the unmanned aerial vehicle 50 captures a magnified image of the tip of the work attachment 4, i.e., the working unit 8 (specific portion), allowing the operator or a worker located near the demolition machine 100 to monitor the working status of the work attachment 4. The position of the unmanned aerial vehicle 50 at this time is referred to as the ATT (attachment) tip photographing position (also referred to as the specific photographing position). Figure 3 is a plan view showing the unmanned aerial vehicle 50 of this embodiment positioned at the ATT tip photographing position in the ATT tip confirmation mode. The ATT tip photographing position includes a right ATT tip photographing position (dashed line in Figure 3) (also referred to as the right effective photographing position) where the camera 503 photographs the working unit 8 from the right side of the work attachment 4 in the left-right direction of the upper rotating body 2, and a left ATT tip photographing position (solid line in Figure 3) (also referred to as the left effective photographing position) where the camera 503 photographs the working unit 8 from the left side of the work attachment 4 in the left-right direction. Figure 4 is an enlarged perspective view showing the camera 503 of the unmanned aerial vehicle 50 capturing an image of the working unit 8 of the work attachment 4 at the left ATT tip capturing position in Figure 3. In Figure 4, the angle of view A (imaging range) of the camera 503 is shown by a dashed line.

On the other hand, the overhead mode is a mode in which the operator and surrounding workers can view a specified overhead image. The position of the unmanned aerial vehicle 50 in this overhead mode is referred to as the overhead shooting position (also referred to as the effective shooting position). Figure 5 is a plan view showing the unmanned aerial vehicle 50 positioned at the overhead shooting position in the overhead mode. As shown in Figure 5, in this embodiment, the overhead shooting position is set so that the unmanned aerial vehicle 50 is positioned above the demolition machine 100 and the entire demolition machine 100 fits within the angle of view A. At such an overhead shooting position, even if the upper rotating body 2 rotates in any direction relative to the lower running body 1, the entire demolition machine 100 is included within the angle of view A, allowing the operator to easily check the status of the demolition machine 100 based on the image captured by the camera 503.

When the flight mode of the unmanned aerial vehicle 50 is set to the ATT tip confirmation mode and an image like that shown in Figure 4 is displayed on the display device 40, if the operator operates the operation unit 21 (Figure 2) to rotate the upper rotating body 2, the work attachment 4 will rotate together with the upper rotating body 2, causing the work unit 8 to move out of (disappear from) the angle of view A in Figure 4. At this time, if the unmanned aerial vehicle 50 is attempted to move to follow the movement of the work unit 8, not only will the process become complicated, but control delays may occur. Such control delays not only cause confusion to the operator viewing the display device 40, but may also cause a collision between the work attachment 4 and the unmanned aerial vehicle 50 depending on the flight position of the unmanned aerial vehicle 50.

To solve the above problems, in this embodiment, the work support device 100S (controller 101) controls the flight of the unmanned aerial vehicle 50. Figure 6 is a flowchart of the processing executed by the work support device 100S. The flight control processing of the unmanned aerial vehicle 50 by the work support device 100S is described in detail below.

With an operator aboard the demolition machine 100 and the unmanned aerial vehicle 50 positioned at the home point HP (Figure 1), the operator turns on the aerial vehicle use switch on the input unit 22 as needed. The controller 101 determines whether the aerial vehicle use switch has been turned on (step S1). If the aerial vehicle use switch has been turned on (YES in step S1), the mode switching unit 103 changes the flight mode of the unmanned aerial vehicle 50 from standby mode to bird's-eye view mode, and the aerial vehicle control unit 104 moves the unmanned aerial vehicle 50 to the bird's-eye view shooting position shown in Figure 5 (step S2). Note that if the aerial vehicle use switch is OFF in step S1 (NO in step S1), the unmanned aerial vehicle 50 continues to wait at the home point HP (step S10).

When the unmanned aerial vehicle 50 reaches the overhead shooting position, the camera 503 begins transmitting overhead video (step S3). At this overhead shooting position, the unmanned aerial vehicle 50 is attitude-controlled so that it faces forward of the lower running structure 1. This allows images to be acquired in a fixed orientation. The operator can check the conditions around the demolition machine 100 by looking at the image displayed on the display device 40.

While the unmanned aerial vehicle 50 is positioned at the bird's-eye view photography position, the controller 101 periodically checks whether the aircraft use switch is ON (step S4). If the use switch is OFF (NO in step S4), the unmanned aerial vehicle 50 returns to the home point HP (step S10).

On the other hand, if the aircraft use switch remains ON in step S4 (YES in step S4) and the operator instructs the input unit 22 to transition to ATT tip confirmation mode (YES in step S5), the mode switching unit 103 transitions from bird's-eye view mode to ATT tip confirmation mode, and the aircraft control unit 104 moves the unmanned aircraft 50 to the ATT tip photographing position on the left side of Figure 3 (step S6). The command signal for moving to the ATT tip photographing position can be set by the aircraft control unit 104 calculating the coordinates of the working unit 8 from the rotation angle of the upper rotating body 2 and the attitude of the work attachment 4, so that the camera 503 can photograph the working unit 8. Note that a marker (not shown) may be attached to the working unit 8, and the command signal to the unmanned aircraft 50 may be set so that the marker is included in the photographing range of the camera 503.

When the unmanned aerial vehicle 50 reaches the ATT tip position photography position, the aerial vehicle control unit 104 adjusts the attitude of the unmanned aerial vehicle 50 and the orientation of the camera 503 so that the working unit 8 is included within the angle of view A as shown in Figure 4 (step S7). Then, the camera 503 begins transmitting an image of the ATT tip (step S8). The operator can view the image displayed on the display device 40 and check the condition around the working unit 8 while working.

Furthermore, the mode switching unit 103 determines whether the upper rotating body 2 has performed a rotation operation that satisfies a predetermined condition while the ATT tip confirmation mode is being executed. In other words, the mode switching unit 103 determines whether the ATT tip (working unit 8) has moved out of the angle of view A of the camera 503 (step S9). In this embodiment, the rotation operation of the upper rotating body 2 is determined based on the above condition of whether the rotation operation lever included in the operation unit 21 has been operated beyond a predetermined operation amount (threshold). If the rotation operation lever has been operated beyond the threshold, i.e., if the ATT tip has moved out of the angle of view A (YES in step S9), the mode switching unit 103 transitions from the ATT tip confirmation mode to the bird's-eye view mode. As a result, the aircraft control unit 104 sends a command signal (movement command signal) to the unmanned aircraft 50, causing the unmanned aircraft 50 to move to the bird's-eye view shooting position shown in FIG. 5 (step S2). After that, the camera 503 starts transmitting the overhead video (step S3), and the processing from step S4 onwards is repeated.

In step S9, if the turning operation lever has not been operated beyond the threshold, i.e., if the ATT tip has not deviated from the angle of view A (NO in step S9), the mode switching unit 103 maintains the ATT tip confirmation mode.

As described above, in this embodiment, when the unmanned aerial vehicle 50 is positioned at the ATT tip photographing position, the camera 503 photographs the working part 8 of the work attachment 4, allowing for detailed understanding of the working status of the demolition machine 100 (Figure 4). On the other hand, if the upper rotating body 2 rotates under specified conditions in this state, the camera 503 will be unable to effectively capture images of the working part 8 because the ATT tip photographing position changes depending on the rotation position of the upper rotating body 2. Even in such a case, the controller 101 generates a movement command signal, causing the unmanned aerial vehicle 50 to move to an overhead photographing position where the camera 503 can photograph the entire demolition machine 100 (predetermined imaging target) from above. Note that the overhead photographing position is a position that does not change depending on the rotation position of the upper rotating body 2. This makes it possible to provide effective images to the operator (worker). In this case, processing complexity and delays can be prevented compared to when the orientation of the camera 503 or unmanned aerial vehicle 50 is adjusted in accordance with the rotation of the upper rotating body 2.

In this embodiment, the work support device 100S is equipped with a display device 40 (image display unit). Therefore, when the unmanned aerial vehicle 50 is positioned at the ATT tip imaging position, the operator can view the image of the working unit 8 captured by the camera 503 on the display device 40. Even if the upper rotating body 2 rotates under certain conditions in this state, the unmanned aerial vehicle 50 moves to the overhead imaging position, allowing images of the target, such as the demolition machine 100, to be provided to the operator via the display device 40. In this case, compared to adjusting the orientation of the camera 503 or the unmanned aerial vehicle 50 in response to the rotation of the upper rotating body 2, it is possible to prevent unnecessary information from being displayed on the display device 40 due to processing delays, etc. The display device 40 is not limited to being located within the cab 3. It may be held by a worker located near the demolition machine 100, or it may be located in a remote location so that it can be viewed by a remote operator or work manager located in the remote location.

In this embodiment, as mentioned above, the overhead shooting position is a position where the camera 503 can capture an image of the entire demolition machine 100 as the imaging target. Therefore, regardless of the rotation position of the upper rotating body 2, the camera 503 can stably capture an image of the entire demolition machine 100 and its surroundings at the overhead shooting position. As a result, the operator can monitor the conditions around the demolition machine 100. In other words, the camera 503 of the unmanned aerial vehicle 50 can be used as a fixed camera.

In addition, in this embodiment, the condition for determining the rotation operation of the upper rotating body 2 is that when an operation in which the amount of operation exceeds a predetermined threshold is input to the operation unit 21, the unmanned aerial vehicle 50 is moved to the bird's-eye view shooting position. Therefore, for slight rotation operations, images of the working unit 8 can be continuously acquired by keeping the unmanned aerial vehicle 50 at the ATT tip shooting position. As a result, frequent changes in the image displayed on the display device 40 can be prevented, reducing the burden on the operator. Note that the rotation operation may also be determined according to a characteristic value that takes into account the amount of operation input to the operation unit 21 and the operation time (timer).

In this embodiment, the ATT tip imaging positions include the right ATT tip imaging position (dashed line in Figure 3) and the left ATT tip imaging position (solid line in Figure 3). Therefore, the working unit 8 can be imaged from a more desirable direction depending on the structure of the demolition machine 100 and the conditions of the work site (the relative positional relationship between the work object and the demolition machine 100).

In this embodiment, the work system of the present invention may be configured by the demolition machine 100 (Figure 1) and the work support device 100S (Figure 2). Alternatively, the work system may be configured by the unmanned aerial vehicle 50 (Figure 1) and the work support device 100S. In this case, the unmanned aerial vehicle 50 may include the controller 101 and generate command signals by itself. In either configuration, even if the upper rotating body 2 rotates while capturing images of the working unit 8 of the work attachment 4, the unmanned aerial vehicle 50 can be moved to a bird's-eye view shooting position by using each work system, and useful images can be acquired.

The work support device 100S according to the present invention and a work system equipped with the same have been described above, but the present invention is not limited to this and can take on modified embodiments such as those described below.

<Variant 1> Figure 7 is a plan view showing an unmanned aerial vehicle 50 according to a variant embodiment of the present invention positioned at a bird's-eye view shooting position. The bird's-eye view shooting position to which the unmanned aerial vehicle 50 moves is a position where the camera 503 can capture an image of a specific area as the imaging target, and the specific area may be an area set relative to the undercarriage 1 as shown in Figure 7. In this case, the camera 503 can capture an image of the specific area at the bird's-eye view shooting position regardless of the rotation position of the upper rotating body 2. Furthermore, because the specific area is an area relative to the undercarriage 1, it can continue to capture the imaging area even when the undercarriage 1 is moving.

In particular, as shown in Figure 7, the specific area may include at least one of the area in front of and the area behind the undercarriage 1. In this case, regardless of the rotation position of the upper rotating body 2, the area in front of or behind the undercarriage 1 can be imaged from the bird's-eye view shooting position. Therefore, the presence of obstacles or workers in the direction of travel of the undercarriage 1 can be confirmed in advance.

<Variant 2> Figure 8 is a perspective view showing an unmanned aerial vehicle 50 according to another variant embodiment of the present invention capturing an image of a work object T at a bird's-eye view shooting position. The work attachment 4 (Figure 1) is capable of performing demolition work (predetermined work) on a work object T such as a building, and the bird's-eye view shooting position may be a position where the camera 503 can capture an image of the work object T as the imaging target, as shown in Figure 8. With this configuration, the work object T can be captured at the bird's-eye view shooting position regardless of the rotation position of the upper rotating body 2, making it possible to check the progress of the work.

<Variation 3> Furthermore, the overhead photography position may be a position above the demolition machine 100 and on the central axis of rotation CL of the upper rotating body 2. With this configuration, the relative positional relationship between the lower traveling body 1 and the upper rotating body 2 can be captured at the overhead photography position, regardless of the rotation position of the upper rotating body 2. Furthermore, because the unmanned aerial vehicle 50 is positioned on the central axis of rotation CL, contact between the work attachment 4 and the unmanned aerial vehicle 50 can be prevented.

<Variation 4> In the previous embodiment, the work support device 100S was described as being included in the demolition machine 100, as shown in FIG. 2. However, the present invention is not limited to this. The work support device 100S may be a device independent of the demolition machine 100. Furthermore, when a commercially available aerial vehicle such as a drone is used as the unmanned aerial vehicle 50, a remote command device (remote controller) capable of transmitting and receiving signals between the unmanned aerial vehicle 50 (aircraft body) and the unmanned aerial vehicle 50 is prepared. In this case, the controller 101 may input a generated command signal (movement command signal) to the remote command device, thereby moving the unmanned aerial vehicle 50 to a position corresponding to the command signal. With this configuration, the unmanned aerial vehicle 50 can be moved from the ATT tip shooting position to the bird's-eye shooting position using a remote command device capable of transmitting and receiving signals with the unmanned aerial vehicle 50. In either embodiment, the controller 101 may be included in a terminal device, such as a tablet or smartphone, carried by a worker or work manager located near the demolition machine 100. The worker may also be someone who remotely operates the demolition machine 100. In this case, the remote command device is provided at the location where the worker remotely operates the machine.

<Variant 5> On the other hand, the work assistance device 100S may further include a transmitter (not shown) capable of transmitting command signals generated by the controller 101 to the unmanned aerial vehicle 50. With this configuration, the unmanned aerial vehicle 50 can be moved from the ATT tip shooting position to the overhead shooting position by sending a command signal directly to the unmanned aerial vehicle 50 via the transmitter.

<Variation 6> Furthermore, the specific imaging position according to the present invention is not limited to the imaging position at the tip of the ATT. The specific part may be the entire work attachment 4, or a part of the work attachment 4 that is different from the working unit 8. Furthermore, the overhead imaging position as an effective imaging position may be the home point HP (Figure 1), or another position such as a position that corresponds to a blind spot of the demolition machine 100 as seen by the operator.

<Variation 7> When the upper rotating body 2 rotates under predetermined conditions in step S9 of Figure 6, the input unit 22 may receive information regarding a candidate movement position, which is a position to which the unmanned aerial vehicle 50 can next move. In this case, the aircraft control unit 104 of the controller 101 sets the candidate movement position input into the input unit 22 as the bird's-eye photography position and generates a command signal to move the unmanned aerial vehicle 50 to the candidate movement position. With this configuration, the operator can capture an image of the desired subject at the candidate movement position, regardless of the rotation position of the upper rotating body 2. The candidate movement position may be a position where the entire demolition machine 100 can be imaged, or another position. The candidate movement position may be input into the input unit 22 manually by the operator (worker) or automatically by the controller 101, etc.

<Variant 8> If the above-mentioned candidate movement positions are input manually, the display device 40 (candidate movement position display unit) may be able to display information about the candidate movement positions (see Figures 9 to 11 below). In this case, the operator can set the next movement position of the unmanned aerial vehicle 50 based on the information displayed on the display device 40. The display device 40 may display the names of the movement destinations as candidate movement positions in a list, or may illustrate them on an overall map of the work site. Note that while the candidate movement positions are displayed, the image from the camera 503 does not need to be displayed. In this case, it is possible to prevent the operator from receiving unnecessary information.

<Variant 9> The above candidate movement positions may include the ATT tip photographing position. With this configuration, the unmanned aerial vehicle 50 can be moved again to the ATT tip photographing position based on the operator's intentions. At this time, the rotation of the upper rotating body 2 may be stopped as a condition so that the unmanned aerial vehicle 50 can stably move to the ATT tip photographing position. As an example, the mode switching unit 103 may allow transition to the ATT tip photographing mode (step S5 in Figure 6) when the amount of change in the rotation angle detected by the rotation angle detection unit 25 is less than a predetermined threshold.

<Variant 10> Furthermore, the conditions for determining the rotation of the upper rotating body 2 are not limited to those in the above embodiment. If the controller 101 is able to determine whether the working unit 8 is included in the image captured by the camera 503 based on the image processing results of the image processing unit 105, the condition may be that the working unit 8 changes from being included in the image to not being included. With this configuration, if the rotation causes the working unit 8 to move out of the imaging range of the camera 503, the unmanned aerial vehicle 50 can be moved to a bird's-eye view shooting position. Therefore, with a slight rotation, the unmanned aerial vehicle 50 can remain at the ATT tip shooting position.

<Variant 11> Furthermore, if the controller 101 is capable of receiving information corresponding to the rotation angle of the upper rotating body 2 relative to the lower running body 1 from the rotation angle detection unit 25, the condition may be that the amount of change in the rotation angle exceeds a predetermined threshold. With this configuration, when the amount of change in the rotation angle of the upper rotating body 2 exceeds the predetermined threshold, the unmanned aerial vehicle 50 can be moved to a bird's-eye view shooting position. In this case, too, a slight rotation movement can keep the unmanned aerial vehicle 50 at the ATT tip shooting position. Furthermore, because the unmanned aerial vehicle 50 does not move frequently, it is possible to prevent unnecessary information from being provided to the operator, and there is no visual burden on the operator.

<Variant 12> In the previous embodiment, in Figure 6, when the aircraft use switch is turned ON (YES in step S1), the mode switching unit 103 changes the flight mode of the unmanned aircraft 50 from standby mode to bird's-eye view mode, and the aircraft control unit 104 moves the unmanned aircraft 50 to the bird's-eye view shooting position in Figure 5 (step S2). However, the flight mode of the unmanned aircraft 50 may also transition directly from standby mode to ATT tip confirmation mode.

<Variation 13> Furthermore, after a YES determination is made in step S9 of FIG. 6 , when the unmanned aerial vehicle 50 reaches the overhead photography position again, the mode switching unit 103 may cause the display device 40 to display information regarding the next position to which the unmanned aerial vehicle 50 can move (candidate movement position) via the display command unit 106. FIG. 9 shows an example of such information, showing an image displayed on the display device 40 in this variation. In addition to the unmanned aerial vehicle 50 and demolition machine 100 positioned at the overhead photography position, the display device 40 also shows arrows indicating multiple candidate movement positions. Note that the unmanned aerial vehicle 50 does not need to be shown. Furthermore, the bottom of the display device 40 displays "In overhead mode" to notify the operator of the current mode, and also displays the message "Please select an imaging position. If not selected, the overhead mode will be maintained." Note that the display of "5 seconds remaining" indicates the countdown time until the mode switching unit 103 decides to maintain the overhead mode.

The operator can select a desired position from the candidate movement positions displayed on the display device 40 and input the next candidate movement position by touching the position on the screen of the touch panel display device 40, for example. Furthermore, if the operator wishes to maintain the overhead photography position, the state shown in Figure 9 can be maintained until the countdown time reaches zero. As a result, the operator can easily set the destination of the unmanned aerial vehicle 50.

10 and 11 are diagrams showing other images displayed on the display device 40 in this modified example. Unlike in FIG. 9, the example in FIG. 10 displays the candidate destinations by name. Similarly, the operator can input the next candidate destination by touching each name on the screen of the touch-panel display device 40. Also, in this example, when the countdown time reaches zero, the list of candidate destinations in FIG. 10 is hidden, and a "Destination List" is displayed in the lower right corner of the display device 40, as shown in FIG. 11. When the operator touches this "Destination List," the image in FIG. 10 is displayed again, allowing the operator to input a candidate destination. Note that the display on the display device 40 is not limited to those shown in FIGS. 9, 10, and 11. Coordinates based on the aircraft's position information or mode names (e.g., work support mode, surveillance mode) may also be displayed. The destination may also include the home point HP (FIG. 1).

As shown in Figures 9 and 10, the operator may move the unmanned aerial vehicle 50 from the overhead shooting position to the ATT tip shooting position again (ATT left side or ATT right side in Figures 9 and 10). When moving the unmanned aerial vehicle 50 to the ATT tip shooting position again, it is desirable that the operator confirm that the rotation operation of the upper rotating body 2 has finished, and then touch the "ATT left side" or "ATT right side" display. The mode switching unit 103 may permit transition to the ATT tip confirmation mode, provided that the amount of operation received by the rotation control lever or the change in the rotation angle detected by the rotation angle detection unit 25 is below a predetermined threshold. In this case, the "ATT right side" and "ATT left side" displays may be hidden or highlighted while the above condition is not met.

<Variant 14> Figure 12 is a diagram showing the relationship between the unmanned aerial vehicle 50, demolition machine 100, cloud C, and site management side 110 in a variant of the present invention. In the previous embodiment, the display device 40 was described as being located in the cab 3 so that it could be viewed by the operator and displaying various information to notify the operator, but the present invention is not limited to this. The display device 40 may be provided on the terminal of the work manager (site supervisor, etc.) so that the work manager can check it, or it may be displayed on cloud C. In other words, not only the operator but also the work manager, etc. can obtain various information to support the work of the demolition machine 100. Furthermore, images captured by the camera 503 (Figure 2) of the unmanned aerial vehicle 50 may be transmitted directly to the display device 40 or may be transmitted to cloud C. In this case, the images may be transmitted from cloud C to the operator in the demolition machine 100, workers located around the demolition machine 100, the manager in the site management side 110, etc. Note that the workers and managers may be located remotely. Additionally, anyone who wishes to view the image may access cloud C of their own volition and obtain the image.

1 Lower running body (lower body)
10 Inter-boom cylinder 100 Demolition machine 100S Work support device 101 Controller (control unit)
102 Drive control unit 103 Mode switching unit 104 Aircraft control unit 105 Image processing unit 106 Display command unit 11 Arm cylinder 12 Work cylinder 2 Upper rotating body 21 Operation unit 22 Input unit 23 Position information detection unit 24 Attitude information detection unit 25 Swing angle detection unit 3 Cab 31 Travel drive unit 32 Swing drive unit 33 Attachment drive unit 4 Work attachment (work mechanism)
40 Display device (image display section, candidate position display section)
5. Boom 50 Unmanned Aerial Vehicle (Aerial Vehicle)
501 Aircraft drive unit 502 Camera drive unit 503 Camera 5a Main boom 5b Front boom 6 Inter-boom 7 Arm 8 Working unit (specific part)
9 Boom cylinder HP Home point

Claims (18)

An apparatus for supporting a work machine including a lower body, a rotating body that can rotate relative to the lower body, and a work mechanism supported on the rotating body, using an aircraft that has an imaging device and is movable to a position in response to a generated command signal,
a control unit that generates the command signal,
the control unit generates a movement command signal, which is the command signal for moving the flying object to an effective image capture position, when the flying object performs a rotation operation that satisfies a predetermined condition while the flying object is located at a specific image capture position;
the specific imaging position is a position at which the imaging device images a specific portion of the working mechanism and changes depending on the rotation position of the rotating body,
The effective imaging position is a position where the imaging device can capture an image of a predetermined imaging target and does not change depending on the rotation position of the rotating body. The work assistance device according to claim 1, further comprising an image display unit capable of displaying images captured by the imaging device. The work support device described in claim 1, wherein the effective imaging position is a position where the imaging device can capture an image of the entire work machine as the imaging target. The rotating body is supported by the lower body so as to be rotatable about a rotation center axis,
The work support device according to claim 1 , wherein the effective image capturing position is a position above the work machine and on the central axis of rotation. the effective imaging position is a position where the imaging device can image a specific area as the imaging target,
The work support device according to claim 1 , wherein the specific area is an area set relatively with the lower body as a reference. the lower body is a lower running body that can run in a front-rear direction on a running surface,
The work assistance device according to claim 5 , wherein the specific area includes at least one of an area in front of the lower traveling structure and an area behind the lower traveling structure in the front-rear direction. the working mechanism is capable of performing work on a work object,
The work support device according to claim 1 , wherein the effective image capturing position is a position where the image capturing device can capture an image of the work object as the image capturing target. An input unit capable of receiving information regarding a candidate position to which the flying object can move next,
The work support device according to claim 1 , wherein the effective image capture position is the movement candidate position input to the input unit. The work assistance device of claim 8, further comprising a candidate position display unit capable of displaying information regarding the candidate movement positions. The work assistance device of claim 8, wherein the candidate movement positions include the specific imaging position. The work assistance device of claim 1, wherein the control unit is capable of receiving information corresponding to an amount of operation for rotating the rotating body, and the condition is that the amount of operation exceeds a predetermined threshold. The work assistance device of claim 1, wherein the control unit is capable of determining whether the specific part is included in the image captured by the imaging device, and the condition is that the specific part changes from being included in the image to not being included in the image. The work assistance device of claim 1, wherein the control unit is capable of receiving information corresponding to the rotation angle of the rotating body relative to the lower body, and the condition is that the amount of change in the rotation angle exceeds a predetermined threshold. The work support device of claim 1, wherein the specific imaging positions include a right-side specific imaging position at which the imaging device images the specific part from the right side of the work mechanism, and a left-side specific imaging position at which the imaging device images the specific part from the left side of the work mechanism. The work assistance device described in claim 1, wherein the control unit inputs the generated command signal to a remote control device capable of sending and receiving signals to and from the air vehicle, thereby moving the air vehicle to a position corresponding to the command signal. The work assistance device of claim 1, further comprising a transmitter capable of transmitting a command signal generated by the control unit to the aircraft. a work machine including a lower body, a rotating body that is rotatable relative to the lower body, and a work mechanism supported on the rotating body;
The work support device according to claim 1 ;
A working system comprising: a flying object having an imaging device and capable of moving to a position in response to a generated command signal;
The work support device according to claim 1 ;
A working system comprising: