JP7596738B2 - Work vehicles - Google Patents

Description

The present invention relates to a work vehicle.

Conventionally, when transporting cargo using a mobile crane, which is a work vehicle, the operator of the mobile crane needs to operate multiple operating tools, such as a slewing operating tool for operating the crane's boom, a raising and lowering operating tool, and a winch operating tool for lifting and lowering the cargo. In addition, the operator of the mobile crane must operate the multiple operating tools simultaneously to operate the mobile crane so that the cargo does not come into contact with structures at the work site. With a mobile crane configured in this way, it can be difficult to transport the cargo to the target position depending on the operator's skill and the conditions of the work site. Therefore, a crane that displays information about the crane operation based on the cargo's movement path displayed on a 3D map of the work site is known. For example, see Patent Document 1.

The crane described in Patent Document 1 extracts three-dimensional information of a suspended load (luggage) based on three-dimensional information of the work site acquired by a stereo camera or the like at the end of the boom. Furthermore, the crane creates a plan view based on the three-dimensional information of the work site. Next, the crane calculates a movement path on the plan view. The crane displays information regarding the operations required to move the suspended load along the calculated movement path.

The crane described in Patent Document 1 displays information about the operations required to move the load as numerical values, such as the amount of change in the rotation angle of the swivel base, the amount of change in the telescopic length of the boom, and the amount of change in the hoisting angle. The operator of the crane operates each of the control tools, which are assigned actuators with different movement speeds, so that the swivel base and the boom assume postures based on the displayed numerical values. However, operators with low skill levels find it difficult to intuitively determine the amount of operation of each control tool that corresponds to the movement speed of the assigned actuator. Therefore, even when the crane described in Patent Document 1 displays information about the operations required to move the load, operation can be difficult depending on the operator's level of skill.

JP 2018-95369 A

The object of the present invention is to provide a work vehicle that can move the tip of the boom along a predetermined movement path regardless of the operator's level of skill.

The problem that the present invention aims to solve is as described above, and the means for solving this problem will be explained next.

A work vehicle according to one embodiment of the present invention comprises a work vehicle having a boom that can be raised and lowered via a swivel platform attached to a running body, a camera that captures images of the surroundings of the work vehicle from above, a display device that displays images captured by the camera and is capable of outputting the coordinates of any position on the display screen specified from outside, and a control device that controls the actuators of the work vehicle.

The control device acquires the position of the tip of the boom or the position of the boom's rotation center relative to the imaging device, attitude information of the boom, information on the angle of view of the image, and an image of the surroundings of the work vehicle captured from above by the imaging device, displays the image on the display device, calculates the coordinates of the tip of the boom in the coordinate system of the display screen and the coordinates of the boom's rotation center in the coordinate system of the display screen, calculates the coordinates of the target position of the tip of the boom set on the image in the coordinate system of the display screen, and displays the current operating position and the target operating position of each operating tool of the actuator so that the tip of the boom reaches the target position in a straight line.

In the above configuration, the control device of the work vehicle calculates the coordinates of the center of rotation of the work vehicle and the coordinates of the tip of the boom in an image including the surroundings of the work vehicle captured by the imaging device. By calculating the coordinates of the center of rotation of the work vehicle and the coordinates of the tip of the boom, the control device can convert the coordinate system of the display screen, which is an orthogonal coordinate system with a predetermined position on the display screen of the display device as its origin, into a crane coordinate system based on the center of rotation of the mobile crane.

Therefore, the operator of the work vehicle can set the target position of the boom tip on the image while checking the situation around the work vehicle on the image in real time. The control device also displays the current operation position and the target operation position of each operating tool, which is information related to the operation of each operating tool that operates the actuator so that the boom tip reaches the target position along a predetermined movement path. The operator can move the boom tip to the target position with a simple operation based on the information related to the operation of each operating tool while grasping the situation of the work site in real time using the work vehicle. Therefore, even an operator with low proficiency can visually grasp the operation amount of each operating tool to which the actuator with different movement speeds is assigned. This allows the work vehicle to move the boom tip along the predetermined movement path regardless of the operator's level of proficiency.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration: The control device displays the operation sequence of each operating tool of the actuator.

In the above-described configuration, the control device further displays the operation sequence of each of the operating tools that operate the actuator so that the tip of the boom reaches the target position along a predetermined movement path. The operator can move the tip of the boom to the target position with a simple operation that follows the operation sequence in addition to the amount of operation of each of the operating tools while grasping the situation at the work site in real time using the work vehicle. This allows the work vehicle to move the tip of the boom along the predetermined movement path regardless of the operator's level of skill.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration. The image capturing device is provided at the tip of the boom. The control device sets the center position of the image to the tip position of the boom, and calculates the coordinates of the boom tip in the coordinate system of the display screen and the coordinates of the boom rotation center in the coordinate system of the display screen from the boom attitude information including the boom hoisting angle, the boom extension length, the boom rotation angle, and the shape and dimensions of the boom.

In the above configuration, since the image is captured from the tip position of the boom, the control device can calculate the coordinates of the boom tip in the coordinate system of the display screen and the coordinates of the boom's rotation center in the coordinate system of the display screen from the angle of view information, including information such as the ratio between the unit length in the image and the actual length of the corresponding part for each shooting distance, and the boom attitude information. Therefore, the operator can move the boom tip to the target position with simple operations based on information related to the operation of each operating tool, while grasping the situation at the work site in real time using the work vehicle. This allows the work vehicle to move the boom tip along the specified movement path regardless of the operator's level of skill.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration: The control device detects an image of the tip of the boom from the image captured by the imaging device, and calculates the coordinates of the boom's rotation center in the coordinate system of the display screen and the coordinates of the boom tip in the coordinate system of the display screen from the orientation and size of the image of the tip of the boom and the boom attitude information.

In the above configuration, the control device automatically detects the coordinates of the boom's rotation center and the coordinates of the boom's tip in the coordinate system of the display screen from an image including the work vehicle captured by the imaging device, by comparing the coordinates with an image of the boom's tip previously acquired. The control device can easily calculate the coordinates of the target position of the boom's tip specified on the image based on the coordinates of the boom's rotation center. In addition, the operator can move the boom's tip to the target position with a simple operation based on information related to the operation of each of the operating tools while grasping the situation of the work site in real time using the work vehicle. This allows the work vehicle to move the boom's tip along the specified movement path regardless of the operator's level of skill.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration. The work vehicle includes a first GNSS receiver that detects the absolute coordinates of the boom's rotation center or the absolute coordinates of the boom's tip, a direction sensor that detects the direction of the boom's extension direction, and a second GNSS receiver that detects the absolute coordinates of the imaging device. The control device acquires the detection value of the first GNSS receiver, the detection value of the second GNSS receiver, the detection value of the direction sensor, and the boom's attitude information, and calculates the absolute coordinates of the boom's rotation center and the absolute coordinates of the boom's tip.

In the above configuration, the control device calculates the absolute coordinates of the boom's rotation center or the absolute coordinates of the boom's tip from the absolute coordinates of the boom's rotation center or the absolute coordinates of the boom's tip, the orientation of the boom's extension direction, the absolute coordinates of the imaging device, the angle of view information of the imaging device, and the boom's attitude information. The control device also calculates the coordinates of the rotation center and the coordinates of the boom's tip in the coordinate system of the display screen based on the absolute coordinates of the imaging device and the angle of view information of the imaging device. The operator can move the boom's tip to the target position with a simple operation based on information on the operation of each of the operating tools while grasping the situation of the work site in real time using the work vehicle. This allows the work vehicle to move the boom's tip along the specified movement path regardless of the operator's level of skill.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration: The target operating position of the operating tool of each actuator is displayed by a graphic.

In the above-described configuration, the control device transmits information related to the operation of each of the operating tools to the operator by means of a graphic. Specifically, the control device displays the current operating position, operating sequence, and target operating position of each of the operating tools of the actuator by the position of the graphic. The operator can move the tip of the boom to the target position by operating each of the operating tools according to the target operating position and the operating sequence for the current operating position of each of the operating tools displayed, while grasping the situation at the work site in real time using the work vehicle. This allows the work vehicle to move the tip of the boom along the predetermined movement path, regardless of the operator's level of skill.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration: The display device is configured to be capable of operating an operating tool for the actuator.

In the above-described configuration, the display device allows the operator to operate each of the operating tools while visually checking information related to the operation of each of the operating tools. The operator can easily and accurately change the current operating position of each of the operating tools to the target operating position of each of the operating tools displayed on the display device while grasping the situation of the work site in real time using the work vehicle. This allows the work vehicle to move the tip of the boom along the predetermined movement path regardless of the operator's level of skill.

From another perspective, it is preferable that the work vehicle of the present invention includes the following configuration: When the control device acquires the coordinates of another work vehicle, the control device sets the position of the other work vehicle as the target position.

In the above-described configuration, the control device displays on the display device information regarding the operation of each of the operating devices required to move the tip of the boom along the predetermined movement path with the position of the other work vehicle as a target position. The operator can move the tip of the boom to the other work vehicle by operating each of the operating devices according to the target operation position and the operation order relative to the current operation position of each of the operating devices displayed, while grasping the situation at the work site in real time using the work vehicle. This allows the work vehicle to move the tip of the boom along the predetermined movement path regardless of the operator's level of skill.

The terminology used in this specification is used for the purpose of defining only specific embodiments, and is not intended to limit the invention. Unless otherwise defined, all terms (including technical and scientific terms) used in this specification have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with the meaning in the context of the relevant technology and this disclosure, and should not be interpreted in an ideal or overly formal sense unless explicitly defined in this specification.

In the description of the present invention, it is understood that a number of techniques and processes are disclosed. Each of these has distinct advantages and may be used with one or more, or in some cases all, of the other disclosed techniques. Thus, for the sake of clarity, the description of the present invention refrains from unnecessarily repeating all possible combinations of the individual steps. However, the specification and claims should be read with the understanding that all such combinations are within the scope of the present invention.

In the following description, numerous specific examples are set forth to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention can be practiced without these specific examples. Therefore, the following disclosure should be considered as an example of the present invention, and is not intended to limit the present invention to the specific embodiments illustrated in the following drawings or description.

According to one embodiment of the present invention, the work vehicle can move the tip of the boom along a predetermined movement path regardless of the operator's level of skill.

1 is an overall view showing the overall configuration of a crane according to the present invention; FIG. 2 is a block diagram showing a control configuration of the crane according to the first embodiment of the present invention. FIG. 1 is a schematic diagram showing a control configuration of an image processing apparatus according to a first embodiment of the present invention. FIG. 4 is a block diagram showing a screen displaying the moving direction of the boom tip of the crane according to the first embodiment of the present invention and a target operation position of the crane equipment operating tool. FIG. 2 is a block diagram showing a state in which the boom tip of the crane according to the first embodiment of the present invention is moved by operating the crane equipment operation tool, and a screen showing the operation state of the crane equipment operation tool; FIG. 11 is a diagram showing a crane and a camera for the crane according to a second embodiment of the present invention. FIG. 11 is a block diagram showing a control configuration of a crane according to a second embodiment of the present invention. FIG. 11 is a block diagram showing a control configuration of an image processing apparatus according to a second embodiment of the present invention. FIG. 11 is a block diagram showing a control configuration of a crane according to a third embodiment of the present invention. FIG. 13 is a block diagram showing a control configuration of an image processing apparatus according to a third embodiment of the present invention.

Below, a crane 1, which is a work vehicle according to a first embodiment of the present invention, will be described with reference to Figures 1 to 4. Note that in this embodiment, a rough terrain crane will be described, but any work vehicle equipped with a boom that can be raised and lowered via a swivel base, such as an all-terrain crane, a truck crane, or an aerial work vehicle, can be used.

As shown in Figures 1 and 2, the crane 1 is a mobile crane that can be moved to any location. The crane 1 has a vehicle 2, a crane device 6, a boom camera 9b that is an image capture device, a display device 21 (see Figure 2), an input device 22, an image processing device 23, and a control device 24 (see Figure 2).

The vehicle 2 is a traveling body that transports the crane device 6. The vehicle 2 has multiple wheels 3 and runs using an engine 4 as a power source. The vehicle 2 is provided with outriggers 5. The outriggers 5 are composed of an extension beam that can be hydraulically extended on both sides in the width direction of the vehicle 2, and a hydraulic jack cylinder that can be extended in a direction perpendicular to the ground.

The crane device 6 is a work device that hoists a load W using a wire rope. The crane device 6 includes a swivel base 7, a boom 9, a main hook block 10, a sub-hook block 11, a hydraulic cylinder for raising and lowering the load 12, a main winch 13, a main wire rope 14, a sub-winch 15, a sub-wire rope 16, a cabin 20, and a crane device sensor 17.

The swivel table 7 is a rotating device that allows the crane device 6 to rotate. The swivel table 7 is mounted on the frame of the vehicle 2 via annular bearings. The swivel table 7 is configured to be freely rotatable around the center of the annular bearing. The swivel table 7 is provided with a hydraulic swivel motor 8, which is an actuator. The swivel table 7 is configured to be swivelable in one direction and the other direction by the swivel hydraulic motor 8.

The boom 9 is a movable support that supports the main wire rope 14 and the sub wire rope 16 so that the load W can be lifted. The boom 9 is composed of multiple boom members. The base end of the base boom member of the boom 9 is swingably mounted at approximately the center of the swivel base 7. The boom 9 is configured to be freely extendable in the axial direction by moving each boom member with an actuator, an extension hydraulic cylinder 9a (see FIG. 2). The boom 9 is also provided with a boom camera 9b, which is an imaging device, and a jib 9c. Images captured by the boom camera 9b are displayed on the display device 21.

The main hook block 10 and the sub-hook block 11 are used to suspend cargo W. The main hook block 10 is provided with multiple hook sheaves around which the main wire rope 14 is wound, and a main hook 10a for suspending cargo W. The sub-hook block 11 is provided with a sub-hook 11a for suspending cargo W.

The hoisting hydraulic cylinder 12 is an actuator that raises and lowers the boom 9 and maintains the position of the boom 9. The end of the cylinder section of the hoisting hydraulic cylinder 12 is connected to the swivel base 7 so that it can be raised and lowered, and the end of the rod section is connected to the base boom member of the boom 9 so that it can swing freely.

The main winch 13 and the sub winch 15 are used to wind up (hoist up) and unwind (lower down) the main wire rope 14 and the sub wire rope 16. The main winch 13 is configured so that the main drum around which the main wire rope 14 is wound is rotated by a main hydraulic motor (not shown) which serves as an actuator. The sub winch 15 is configured so that the sub drum around which the sub wire rope 16 is wound is rotated by a sub hydraulic motor (not shown) which serves as an actuator.

The crane apparatus 6 has multiple crane apparatus sensors 17 required to control the swing hydraulic motor 8, the extension hydraulic cylinder 9a, the elevation hydraulic cylinder 12, the main winch 13, and the sub winch 15. The multiple crane apparatus sensors 17 are provided on the corresponding actuators or the control valves of the corresponding actuators. The multiple crane apparatus sensors 17 detect the amount of operation of each actuator, etc. In addition, the detection values of the crane apparatus sensors 17 are included in the attitude information Ip of the boom 9.

The cabin 20 is a housing that covers the operator's seat. The cabin 20 is mounted on the rotating platform 7. A operator's seat (not shown) is provided. The operator's seat is provided with a travel operating tool 18 for operating the vehicle 2, a rotation operating tool for operating the rotation hydraulic motor 8 of the crane device 6, a telescopic operating tool for operating the telescopic hydraulic cylinder, a hoisting operating tool for operating the hoisting hydraulic cylinder 12, crane device operating tools 19 including a main operating tool for operating the main winch 13 and a sub operating tool for operating the sub winch 15, a display device 21, an input device 22, etc. (see FIG. 2).

The display device 21 displays the image P captured by the boom camera 9b. The display device 21 is composed of a liquid crystal monitor or the like having a touch panel function that allows input from the screen. The display device 21 is arranged in a position where it can be seen by the operator of the crane 1. The display device 21 is configured to be able to acquire the image P from the boom camera 9b. The display device 21 displays the image P captured by the boom camera 9b, information about the crane 1 calculated based on the image P captured by the boom camera 9b, and information about the operation of the crane device operating tool 19 (see FIG. 4). The information about the operation of the crane device operating tool 19 is information about the current operating position, which is the operating position of the crane device operating tool 19 in the current attitude of the boom 9, and the target operating position, which is the operating position of the crane device operating tool 19 for moving the tip of the boom 9 to the target position. The information about the operation of the crane device operating tool 19 may also include information about the operating order of the crane device operating tool 19.

The input device 22 inputs the target position of the tip of the boom 9 and the operation signal of the crane device operation tool 19. The input device 22 also inputs the operation signal of the boom camera 9b. In this embodiment, the input device 22 is constituted by the display device 21, which is configured as a touch panel that can be input from the screen. The input device 22 can input any position within the range in which the image P is displayed on the screen of the display device 21 as the target position of the tip of the boom 9. In other words, the display device 21, which is the input device 22, is configured to be able to output the coordinates of any position within the range in which the image P is displayed on the screen to the image processing device 23, etc. The input device 22 can also input any position within the range in which information on the operation of the crane device operation tool 19 on the screen of the display device 21 is displayed as the target operation position of the crane device operation tool 19. In other words, the display device 21, which is the input device 22, is configured to be able to operate the crane device operation tool 19.

The image processing device 23 processes the image P (see FIG. 4) captured by the boom camera 9b. The image processing device 23 may be configured with a CPU, ROM, RAM, HDD, etc. connected via a bus, or may be configured with a one-chip LSI, etc. In this embodiment, the image processing device 23 is provided in the display device 21. The image processing device 23 stores various programs and data for controlling the operation of the display device 21, the input device 22, etc., and for processing image data. The image processing device 23 is connected to the control device 24. In other words, the image processing device 23 is considered to be electrically included in the control device 24. The image processing device 23 may be configured integrally with the control device 24.

3, the image processing device 23 is connected to the display device 21. The image processing device 23 can acquire the position of the target position input from the display screen of the display device 21, which is the input device 22, as a target coordinate T in the coordinate system of the display screen (hereinafter simply referred to as the "coordinate system of the display screen"), which is an orthogonal coordinate system with a predetermined position on the display screen as the origin. Furthermore, the image processing device 23 can acquire the current operation position of the crane device operating tool 19 (see FIG. 2) displayed on the display screen as a current operation position coordinate Pa in the coordinate system of the display screen. The image processing device 23 can acquire a current operation signal Sa corresponding to the current operation position of the crane device operating tool 19 from the control device 24. Furthermore, the image processing device 23 can acquire a target operation signal Sb corresponding to the target operation position of the crane device operating tool 19 required to move the tip of the boom 9 (see FIG. 1) to the target position along a predetermined movement path R (see FIG. 4). The image processing device 23 can acquire boom 9 attitude information Ip, including the boom 9 elevation angle, boom 9 extension length, boom 9 rotation angle, and boom 9 shape and dimensions, from the control device 24. The image processing device 23 can acquire the operation signal of the boom camera 9b from the display device 21.

The image processing device 23 can generate a current operation signal Sa of the crane operating device 19 (see FIG. 2) based on the current operation position coordinate Pa of the crane operating device 19 in the coordinate system of the display screen that has been acquired. The image processing device 23 can calculate the current operation position coordinate Pa of the crane operating device 19 in the coordinate system of the display screen based on the current operation signal Sa of the crane operating device 19 that has been acquired. Similarly, the image processing device 23 can calculate the target operation position coordinate Pb of the crane operating device 19 in the coordinate system of the display screen based on the target operation signal Sb of the crane operating device 19 that has been acquired. The image processing device 23 can generate a control signal for the boom camera 9b based on the operation signal of the boom camera 9b that has been acquired.

The image processing device 23 can transmit the generated current operation signal Sa of the crane operating tool 19 (see FIG. 2) to the control device 24. The image processing device 23 can display a figure showing the crane operating tool 19 or a figure showing the operating position of the crane operating tool 19 at the current operating position coordinate Pa and the target operating position coordinate Pb of the crane operating tool 19 in the calculated coordinate system of the display screen. In other words, the image processing device 23 can display the current operating position of the crane operating tool 19, which is information related to the operation of the crane operating tool 19, and the target operating position of the crane operating tool 19 on the display device 21. The image processing device 23 can transmit the generated operation signal of the boom camera 9b to the boom camera 9b. The image processing device 23 may be configured to display the operation sequence of the crane operating tool 19 on the display device 21.

As shown in FIG. 3, the image processing device 23 has angle of view information Ii including information such as the ratio between a unit length in the image P captured by the boom camera 9b and the actual length of the corresponding part for each shooting distance. The image processing device 23 can calculate the transformed tip coordinate Bc of the boom 9, the transformed rotation center coordinate Cc of the boom 9, and the transformed target coordinate Tc of the tip of the boom 9 in the crane coordinate system based on the rotation center of the crane 1 from the angle of view information Ii of the boom camera 9b and the attitude information Ip of the boom 9.

Since the boom camera 9b is provided at the tip of the boom 9 (see FIG. 1), the image processing device 23 calculates the tip coordinate B of the boom 9 in the coordinate system of the display screen, with the center position of the image P being the tip position of the boom 9. Next, the image processing device 23 calculates the height of the boom camera 9b based on the posture information Ip of the boom 9, with the installation surface of the crane 1 as the reference. The image processing device 23 calculates the rotation center coordinate C of the boom 9 in the coordinate system of the display screen from the angle of view information Ii of the boom camera 9b, the height of the boom camera 9b, and the posture information Ip of the boom 9. The image processing device 23 superimposes the calculated positions of the tip coordinate B and the rotation center coordinate C on the image P displayed on the display device 21 (see FIG. 2). Note that if the rotation center coordinate C is not included in the range of the image P, it is not displayed on the image P.

The image processing device 23 converts the boom 9 tip coordinate B and boom 9 rotation center coordinate C in the coordinate system of the display screen from the height of the boom camera 9b and the boom camera 9b angle of view information Ii into a converted tip coordinate Bc and a converted rotation center coordinate Cc in a crane coordinate system with the boom 9 rotation center as the origin.

The image processing device 23 also acquires the target coordinate T in the coordinate system of the display screen of the target position input from the display device 21. Furthermore, the image processing device 23 superimposes on the image P a predetermined movement path R (see FIG. 4) from the tip coordinate B to the target coordinate T. The image processing device 23 converts the target coordinate T into a converted target coordinate Tc, which is a coordinate in the crane coordinate system, based on the height of the boom camera 9b and the viewing angle information Ii of the boom camera 9b. The image processing device 23 transmits the converted tip coordinate Bc, the converted rotation center coordinate Cc, and the converted target coordinate Tc to the control device 24.

The image processing device 23 calculates the current operation position coordinate Pa and the target operation position coordinate Pb of the crane operating device 19 in the coordinate system of the display screen based on the current operation signal Sa and the target operation signal Sb of the crane operating device 19 acquired from the control device 24. The image processing device 23 displays a figure representing the crane operating device 19 at the current operation position coordinate Pa of the crane operating device 19 in the calculated coordinate system of the display screen. Similarly, the image processing device 23 displays a figure representing the crane operating device 19 at the target operation position coordinate Pb of the crane operating device 19 in the calculated coordinate system of the display screen. In other words, the image processing device 23 displays the operation amount, operation direction, and movement order of the crane operating device 19 for moving the tip of the boom 9 from the current position to the target position along a predetermined movement path R.

As shown in Figures 2 and 3, the control device 24 controls each actuator of the crane 1. The control device 24 is provided in the cabin 20. The control device 24 may be configured with a CPU, ROM, RAM, HDD, etc. connected via a bus, or may be configured with a one-chip LSI, etc. The control device 24 stores various programs and data for controlling the operation of the actuators, switching valves, crane device sensors 17, etc., and for processing image data. The control device 24 includes an image processing device 23.

The control device 24 is connected to the traveling operation tool 18 and the crane operation tool 19. The control device 24 can acquire the operation signals of the traveling operation tool 18 and the crane operation tool 19. The control device 24 generates a control signal for the vehicle 2 based on the operation signal generated by the operation of the traveling operation tool 18. Similarly, the control device 24 generates a control signal for the crane 6 based on the operation signal generated by the operation of the crane operation tool 19. The control device 24 can transmit the generated control signal to each actuator of the crane 6. The control device 24 is connected to the crane sensor 17. The control device 24 can control each actuator based on the detection value of the crane sensor 17. The control device 24 can also acquire the attitude information Ip of the boom 9 by the crane sensor 17.

The control device 24 is connected to the image processing device 23 by wire or wirelessly. The control device 24 can transmit the attitude information Ip of the boom 9 to the image processing device 23. The control device 24 can acquire from the image processing device 23 the converted tip coordinate Bc, which is the coordinate of the tip of the boom 9 in the crane coordinate system, the converted rotation center coordinate Cc, which is the coordinate of the rotation center of the boom 9, and the converted target coordinate Tc of the tip of the boom 9.

The control device 24 can generate a current operation signal Sa of the crane device operating tool 19 in the current attitude of the boom 9 and a target operation signal Sb of the crane device operating tool 19 for the tip of the boom 9 to reach the conversion target coordinate Tc along a predetermined movement path R at a predetermined speed from the acquired conversion tip coordinate Bc and conversion target coordinate Tc. The control device 24 can transmit the generated current operation signal Sa and target operation signal Sb of the crane device operating tool 19 to the image processing device 23. In other words, the control device 24 can calculate the operation amount and operation direction of the crane device operating tool 19 for moving the luggage W along the predetermined movement path R at a predetermined speed. In this embodiment, the predetermined movement path R is a path connecting the conversion tip coordinate Bc to the conversion target coordinate Tc with a straight line (see FIG. 4). The control device 24 may also be configured to calculate the operation order of each operating tool of the crane device operating tool 19.

The crane 1 configured in this manner can move the vehicle 2 to any position by operating the travel operating device 18. The crane 1 can also transport cargo W to any position by rotating, raising, lowering, and extending the boom 9 by operating the crane device operating device 19. The crane 1 can also hoist or lower cargo W with the main winch 13 or the like by operating the crane device operating device 19. Meanwhile, the crane 1 can generate an operating signal for the crane device operating device 19 to move the tip of the boom 9 along a predetermined movement path R to a target position set based on the image P.

The input device 22, the display device 21, and the image processing device 23 may be configured as a remote control terminal that is detachable from the cabin 20. The remote control terminal further includes a communication device and the like, and is configured to be capable of communicating between the image processing device 23 and the control device 24. This allows the crane 1 to operate the boom 9 from a position away from the cabin 20.

The image processing by the image processing device 23 and the control device 24 will be described below with reference to Figures 3 to 5. Note that the image P captured by the boom camera 9b includes the work site around the crane 1. The crane 1 is in a state in which the cargo W is hoisted to a predetermined height by the sub-winch 15.

As shown in Figures 4 and 5, the display device 21 (see Figure 3) has an image display area in which the image P captured by the boom camera 9b is displayed, and an operating tool display area that displays the operating positions of the rotation operating tool that operates the rotation hydraulic motor 8, the extension operating tool that operates the extension hydraulic cylinder, the hoisting operating tool that operates the hoisting hydraulic cylinder 12, the main operating tool that operates the main winch 13, and the sub operating tool that operates the sub winch 15.

The operating tool display area displays the range of motion for each crane operating tool 19, and the current operating position P1a of the swivel operating tool, the current operating position P2a of the telescopic operating tool, the current operating position P3a of the hoisting operating tool, the current operating position P4a of the main operating tool, and the current operating position P5a of the sub operating tool, which constitute the crane operating tool 19 (see black circles). The operating tool display area can also display the target operating position P1b of the swivel operating tool, the target operating position P2b of the telescopic operating tool, the target operating position P3b of the hoisting operating tool, the target operating position P4b of the main operating tool, and the target operating position P5b of the sub operating tool, which are used to move the tip of the boom 9 to the target position, as graphics (see dashed double-dashed circles).

As shown in FIG. 3, the image processing device 23 acquires an image P of the tip of the boom 9 of the crane 1 captured by the boom camera 9b every unit time. In addition, the image processing device 23 acquires posture information Ip of the boom 9 from the control device 24 every unit time. The image processing device 23 displays the acquired image P (see FIG. 4 and FIG. 5) in the image display area of the display device 21. The operator inputs the target position of the tip of the boom 9 from the image P. The image processing device 23 calculates the target coordinate T in the coordinate system of the display screen of the input target position.

The image processing device 23 calculates the tip coordinate B of the boom 9 in the coordinate system of the display screen, with the center position of the image P being the tip position of the boom 9. Next, the image processing device 23 calculates the rotation center coordinate C of the boom 9 in the coordinate system of the display screen. As shown in Figures 4 and 5, the image processing device 23 superimposes the calculated tip coordinate B, rotation center coordinate C, and target coordinate T on the image P. Furthermore, the image processing device 23 superimposes on the image P a predetermined movement path R from the tip coordinate B to the target coordinate T superimposed on the image P. Note that in this embodiment, the image P does not include the rotation center coordinate C. Therefore, the rotation center coordinate C is not displayed on the image P.

As shown in FIG. 3, the image processing device 23 calculates the converted tip coordinate Bc, the converted rotation center coordinate Cc, and the converted target coordinate Tc in the crane coordinate system from the height of the boom camera 9b and the angle of view information Ii of the boom camera 9b. The image processing device 23 transmits the converted rotation center coordinate Cc, the converted target coordinate Tc, and the converted target coordinate Tc to the control device 24. Note that, although the crane coordinate system is an orthogonal coordinate system in this embodiment, it may be a polar coordinate system.

The control device 24 generates a target operation signal Sb of the crane device operating tool 19 for moving the tip of the boom 9 to the converted target coordinate Tc along a predetermined movement path R based on the converted rotation center coordinate Cc and the converted tip coordinate Bc. The control device 24 transmits the generated target operation signal Sb of the crane device operating tool 19 to the image processing device 23. The image processing device 23 calculates the target operation position coordinate Pb of the crane device operating tool 19 in the coordinate system of the display screen based on the target operation signal Sb of the crane device operating tool 19 acquired from the control device 24.

The image processing device 23 displays figures indicating the target operating position P1b of the rotation operating tool, the target operating position P2b of the extension operating tool, the target operating position P3b of the elevation operating tool, the target operating position P4b of the main operating tool, and the target operating position P5b of the sub operating tool in the operating tool display area.

The image processing device 23 may display the operation order of the rotation operating tool, the telescopic operating tool, the elevation operating tool, the main operating tool, and the sub operating tool. For example, the image processing device 23 displays the operation order of each operating tool in numbers such as (1), (2), (3), etc. in the operating tool display area of each operating tool. When multiple operating tools are operated simultaneously, the image processing device 23 displays the same operation order for the operating tools that are operated simultaneously. In this embodiment, the image processing device 23 displays the same operation order "(2)" for the telescopic operating tool and the main operating tool that are operated simultaneously.

By configuring in this way, the crane 1 can set the target position of the boom 9 while checking the surrounding situation with the image P. The crane 1 can also visually recognize the amount and direction of operation of the crane device operating tool 19 required to move the tip of the boom 9 to the target position. The crane 1 also displays the operation sequence of the crane device operating tool 19, allowing the user to visually recognize the operation sequence in addition to the amount and direction of operation of each operating tool.

As shown in FIG. 5, the control device 24 acquires a current operation signal Sa for each crane operating tool 19 included in the crane operating tool 19 when the operating tool is operated. The control device 24 transmits the acquired current operation signal Sa to the image processing device 23. The image processing device 23 calculates the current operating position coordinate Pa of the crane operating tool 19 in the coordinate system of the display screen based on the current operation signal Sa of the crane operating tool 19 acquired from the control device 24. The image processing device 23 sets the calculated current operating position coordinate Pa of the crane operating tool 19 as the new current operating position coordinate Pa of the crane operating tool 19 after operation, and displays a figure showing the crane operating tool 19 at the current operating position coordinate Pa.

By configuring the crane 1 in this manner, the figure representing the crane equipment operating tool 19 displayed on the display device 21 moves as the crane equipment operating tool 19 is operated. This allows the operator of the crane 1 to operate the crane equipment operating tool 19 while checking the situation around the crane 1 and the difference between the current operating position of the crane equipment operating tool 19 and the target operating position.

When at least one of the figures showing the current operating position P1a of the rotation operating tool, the current operating position P2a of the extension operating tool, the current operating position P3a of the elevation operating tool, the current operating position P4a of the main operating tool, and the current operating position P5a of the sub operating tool displayed in the operating tool display area is moved, the image processing device 23 generates a current operating signal Sa with the coordinates of the moved figure in the coordinate system of the display screen as the current operating position coordinates Pa of the corresponding operating tool. The image processing device 23 transmits the generated current operating signal Sa to the control device 24. The control device 24 transmits a control signal to the corresponding actuator based on the acquired current operating signal Sa.

By configuring in this manner, the tip of the boom 9 of the crane 1 moves in accordance with the movement of the figure on the screen of the display device 21. By moving the figure indicating the current operating position P1a of the telescopic operating tool, the figure indicating the current operating position P2a of the telescopic operating tool, the figure indicating the current operating position P3a of the hoisting operating tool, the figure indicating the current operating position P4a of the main operating tool, and the figure indicating the current operating position P5a of the sub operating tool to the target operating position coordinates Pb on the screen of the display device 21, the crane 1 can move the tip of the boom 9 to the target position along a predetermined movement path R while checking the situation around the crane 1.

Therefore, the operator of the crane 1 can set the target position of the tip of the boom 9 on the image P displayed on the display device 21 while checking the situation around the crane 1 in real time. In addition, the image processing device 23 calculates the converted tip coordinate Bc, the converted rotation center coordinate Cc, and the converted target coordinate Tc in the crane coordinate system, with the center position of the image P photographed by the boom camera 9b provided at the tip of the boom 9 being the tip position of the boom 9. Furthermore, the image processing device 23 displays the current operating position and the target operating position of the crane device operating tool 19 that operates each actuator so that the tip of the boom 9 reaches the target position on a linear movement path R. The operator can move the tip of the boom 9 to the target position with easy operation based on information on the operation of the crane device operating tool 19 while grasping the situation of the work site in real time using the crane 1. Therefore, the crane 1 allows even an operator with low skill level to visually grasp the operation amount of each operating tool to which the actuators with different movement speeds are assigned. The image processing device 23 may be configured to display the operation order of the crane device operating tool 19. This allows the crane 1 to move the tip of the boom 9 along a linear movement path R, regardless of the operator's level of skill.

Next, a crane 1A according to a second embodiment of the present invention will be described with reference to Figures 6 to 8. Note that the crane 1A according to the following embodiments will refer to the same crane 1 shown in Figures 1 to 5, and the names, drawing numbers, and symbols used in the description will be used to refer to the same thing. Therefore, in the following embodiments, specific descriptions of the same points as in the embodiments already described will be omitted, and the description will focus on the differences.

As shown in FIG. 6, the crane 1A has an imaging device 25, a remote control terminal 28 (see FIG. 7), and an image processing device 30 (see FIG. 7). The imaging device 25 captures images of the crane 1A and the surroundings of the crane 1A at the work site. The imaging device 25 includes an unmanned aerial vehicle 26 and a camera 27.

The unmanned aerial vehicle 26 is an autonomous unmanned aerial vehicle (drone) that can be remotely controlled and fly autonomously by a control signal. The unmanned aerial vehicle 26 is, for example, a multicopter with multiple propellers. The unmanned aerial vehicle 26 is configured to be able to send and receive control signals from the remote control terminal 28. As a result, the unmanned aerial vehicle 26 is configured to be able to move to any position by the remote control terminal 28. The unmanned aerial vehicle 26 may be configured to follow a beacon attached to the boom 9 of the crane 1A, an image of the boom 9 detected by the image processing device 30, an AR tag attached to the boom 9, etc.

The camera 27 is provided on the unmanned aerial vehicle 26. The camera 27 is configured to capture images vertically below the unmanned aerial vehicle 26. The camera 27 is configured to be able to acquire control signals and the like from the remote control terminal 28. The camera 27 is also configured to be able to transmit captured image data to the image processing device 30. This allows the camera 27 to start and stop capturing images, determine the capturing direction, zoom, and perform other operations using the remote control terminal 28.

The remote control terminal 28 is an operation terminal that operates the unmanned aerial vehicle 26 and the camera 27. The remote control terminal 28 is arranged in the cabin 20. The remote control terminal 28 is provided with an operation tool for the unmanned aerial vehicle 26 and an operation tool for the camera 27 in a portable housing. In other words, the remote control terminal 28 is configured as a portable operation terminal that can be carried outside the cabin 20. The remote control terminal 28 can place the unmanned aerial vehicle 26 above the crane 1A by operating the operation tool, and can hover at any position. The remote control terminal 28 is configured to be able to transmit and receive control signals to the unmanned aerial vehicle 26 and the camera 27. The remote control terminal 28 is also configured to be able to transmit and receive input signals, operation signals, etc. to the control device 24 of the crane 1A. The remote control terminal 28 is provided with a display device 21. The display device 21 may be arranged in the cabin 20 separately and independently from the remote control terminal 28.

As shown in FIG. 7, the communication device 29 transmits and receives image data and control signals. The communication device 29 is provided in the remote control terminal 28. The communication device 29 can transmit and receive control signals for the unmanned aerial vehicle 26 and the camera 27 of the photographing device 25 to and from the image processing device 30. The communication device 29 can also transmit images P captured by the camera 27 to the image processing device 30. The communication device 29 can transmit and receive control signals between the unmanned aerial vehicle 26 and the camera 27. The communication device 29 can receive images P captured by the camera 27.

The image processing device 30 is provided in the remote control terminal 28. The image processing device 30 is connected to the display device 21. The image processing device 30 can acquire operation signals of the unmanned aerial vehicle 26 and the camera 27 from the display device 21. The image processing device 30 generates a control signal for the unmanned aerial vehicle 26 based on the operation signal for the unmanned aerial vehicle 26 input from the display device 21. Similarly, the image processing device 30 generates a control signal for the camera 27 based on the operation signal for the camera 27 input from the display device 21. The image processing device 30 is connected to the communication device 29. The image processing device 30 transmits the generated control signal to the unmanned aerial vehicle 26 or the camera 27 via the communication device 29.

The image processing device 30 detects the tip of the boom 9 from the image P captured by the camera 27 based on a learning model stored in advance. In this embodiment, the image processing device 30 uses a combination of vectorization using a color histogram of the image P, vectorization using an autoencoder, and vectorization using local features to calculate features by each vectorization. Vectorization using a color histogram is a method of vectorizing the color intensity and occurrence frequency as features. Vectorization using an autoencoder is a method of using the intermediate layer of the autoencoder as a feature representing the image P. The autoencoder is a three-layer neural network in which supervised learning is performed using the same data for the input layer and output layer. Vectorization using local features is a method of vectorizing the image P using KAZE local features.

The image processing device 30 calculates three feature amounts by vectorization of the image P using a color histogram, vectorization using an autoencoder, and vectorization using local features. The feature amounts may be the luminance values contained in the image P, an edge histogram, a correlation function between color and edges, etc. The image processing device 30 may also calculate local feature amounts of the image P, such as HOG (Histogram of Oriented Gradients) and SIFT (Scale Invariant Feature Transform).

Next, detection of the tip of the boom 9 and the extension direction of the boom 9 in the image processing device 30 of the crane 1A will be described using Figure 8. The image processing device 30 has a reference image holding unit 30a, a judgment image detection unit 30b, and a posture judgment unit 30c. The image processing device 30 has a learning model that has undergone supervised learning of a predetermined range of the tip of the boom 9. The image processing device 30 can detect the tip of the boom 9 and the extension direction of the boom 9 from three feature amounts of the image P captured by the camera 27.

As shown in FIG. 8, the reference image storage unit 30a is part of the image processing device 30, and stores a reference image Is of the tip of the boom 9 in advance. The reference image Is is a reference image for determining the tip of the boom 9. The reference image Is is composed of at least a part of an image of the tip of the boom 9 in an arbitrarily determined reference posture. In other words, the reference image Is is an image that shows the reference posture of the tip of the boom 9. In this embodiment, the reference posture is the boom 9 in a specified posture. In other words, the reference image Is is composed of at least a part of an image taken by the imaging device 25 in a specified posture from above at a specified distance away of the tip of the boom 9 at a specified elevation angle, extension length, and rotation angle.

The judgment image detection unit 30b is part of the image processing device 30, and detects an image of a portion of the current image P that corresponds to the reference image Is as a judgment image Ij. The judgment image Ij is an image of the tip portion of the boom 9 in the current image P relative to the reference posture of the tip portion of the boom 9. In other words, the judgment image Ij is an image for calculating the relative posture of the tip portion of the boom 9 in the current image P relative to the reference image Is. The judgment image Ij is composed of an image of at least a portion of the tip portion of the boom 9 in the current image P.

The judgment image detection unit 30b can calculate feature points of the selected reference image Is. Furthermore, the judgment image detection unit 30b can calculate feature points from the current image P. The judgment image detection unit 30b can compare the feature points of the reference image Is with the feature points of the current image P, and detect the part of the current image P that corresponds to the reference image Is as the judgment image Ij.

The attitude determination unit 30c is part of the image processing device 30, and determines the current orientation D of the extension direction of the boom 9. The attitude determination unit 30c is configured to be able to calculate the shape, direction, and inclination of the judgment image Ij relative to the reference image Is by comparing the reference image Is with the judgment image Ij. The attitude determination unit 30c can calculate the tip coordinates B of the boom 9 in the current coordinate system of the display screen relative to the reference attitude of the tip of the boom 9 and the orientation D of the extension direction of the boom 9 in the coordinate system of the display screen from the coordinates of each part of the reference image Is in the coordinate system of the display screen and the coordinates of each part of the judgment image Ij in the coordinate system of the display screen.

The image processing device 30 can calculate the rotation center coordinate C, which is the coordinate of the rotation center of the boom 9 in the coordinate system of the display screen, from the attitude information Ip of the boom 9, such as the size in the image P of the detected tip portion of the boom 9, the extension direction of the boom 9, the boom hoisting angle of the boom 9, the extension length of the boom 9, the rotation angle of the boom 9, and the shape and dimensions of the boom 9. The image processing device 30 superimposes the calculated tip coordinate B, rotation center coordinate C, and direction D of the boom 9 on the image P displayed on the display device 21.

The image processing device 30 can convert the coordinate system of the display screen into a crane coordinate system with the boom 9's rotation center as the origin, based on the boom 9's tip coordinate B, boom 9's rotation center coordinate C, and boom 9's posture information Ip in the coordinate system of the display screen. The image processing device 30 converts the tip coordinate B and rotation center coordinate C in the coordinate system of the display screen into the converted tip coordinate Bc and converted rotation center coordinate Cc in the crane coordinate system. The image processing device 30 transmits the converted tip coordinate Bc, converted rotation center coordinate Cc, and converted target coordinate Tc to the control device 24.

When performing image processing using the image processing device 30, the crane 1A hovers the unmanned aerial vehicle 26 of the imaging device 25 above the tip of the boom 9 before transport begins (see Figure 6). Even if the boom 9 moves, the unmanned aerial vehicle 26 maintains the hovering position it had before the boom 9 moved. At this time, the image processing device 30 performs sway correction so that the position on the screen of a target object such as a structure that is automatically detected or specified by the operator does not fluctuate.

The image processing device 30 acquires the image P from above the crane 1A captured by the camera 27 and information on the position, posture, etc. of the image capturing device 25 every unit time via the communication device 29 (see FIG. 4). In addition, the image processing device 30 acquires posture information Ip of the boom 9 from the control device 24 every unit time. The image processing device 30 displays the acquired image P on the display device 21.

The image processing device 30 detects a judgment image Ij, which is an image of the tip of the boom 9, from the latest image P captured by the camera 27. The image processing device 30 prestores a reference image Is, which is an image of the tip of the boom 9 detected using a convolutional neural network (CNN) from images of the tip of the boom 9 of various cranes 1A (see FIG. 4).

The image processing device 30 detects a judgment image Ij, which is an image of the tip of the boom 9, from the latest image P by image classification using CNN based on the reference image Is. From the detected judgment image Ij and the attitude information Ip of the boom 9, the image processing device 30 calculates the direction D of the axis of the boom 9 in the judgment image Ij, the tip coordinate B indicating the position of the tip of the boom 9 on the axis in the coordinate system of the display screen, and the rotation center coordinate C indicating the rotation center of the boom 9 in the coordinate system of the display screen.

Specifically, the image processing device 30 calculates the length from the tip to the base of the boom 9 in the coordinate system of the display screen in which the boom 9 is photographed from above, from the boom 9 attitude information Ip, which is the width of the boom 9 tip, the boom hoisting angle, and the boom 9 extension length. The image processing device 30 calculates the rotation center coordinate C from the calculated tip coordinate B, axis direction D, and the length from the tip to the base of the boom 9. As a result, the image processing device 30 can convert the coordinate system of the display screen into a crane coordinate system with the boom 9 rotation center as the origin, from the tip coordinate B, rotation center coordinate C, axis direction D, and boom 9 attitude information Ip. The image processing device 30 converts the tip coordinate B and rotation center coordinate C in the coordinate system of the display screen into the converted tip coordinate Bc and converted rotation center coordinate Cc in the crane coordinate system. In this embodiment, the crane coordinate system is an orthogonal coordinate system, but it may be a polar coordinate system.

The operator inputs the target position of the tip of the boom 9 onto the image P. The operator sets the target position while checking the status of the boom 9 of the crane 1A displayed in the image P, as well as the lifting position of the load W and the placement position of the load W, from above the crane 1A. This allows the operator to intuitively set a transport route that avoids obstacles while taking into account the layout of structures within the work site.

With this configuration, the image processing device 30 automatically calculates the rotation center coordinate C of the crane 1A and the tip coordinate B of the boom 9 by, for example, comparing the image P including the crane 1A and the surroundings of the crane 1A captured by the camera 27 with the image P of the tip of the boom 9 previously acquired. By calculating the rotation center coordinate C and tip coordinate B of the boom 9 using the image processing device 30, the control device 24 can convert the coordinate system of the display screen into a crane coordinate system based on the rotation center of the crane 1A.

In the above embodiment, the image P may include an AR marker. The image processing device 30 calculates the tip coordinate B and turning center coordinate C of the boom 9 from the shape of the boom 9 included in the image P. However, if the image P includes an image of an AR marker provided on a part of the boom 9, the image processing device 30 can calculate the tip coordinate B and turning center coordinate C of the boom 9 from the position of the AR marker in the image P. The image processing device 30 can also calculate the turning center coordinate C from the yaw angle of the AR marker. The image processing device 30 can also calculate the elevation angle from the roll angle and pitch angle of the AR marker.

Next, a crane 1B according to a third embodiment of the present invention will be described with reference to Figures 9 and 10. Note that the crane 1B according to the following embodiments is the same as the crane 1 and A1 shown in Figures 1 to 8, and the names, drawing numbers, and symbols used in the description will be used to refer to the same thing.

As shown in FIG. 9, the crane 1B is a mobile crane that can be moved to any location. The crane 1B has a vehicle 2, a crane device 6, an imaging device 25, a display device 21, a first GNSS receiver 31, a direction sensor 32, a second GNSS receiver 33, a communication device 34, an image processing device 30, and a control device 24.

The first GNSS receiver 31 is a receiver that constitutes the Global Navigation Satellite System. The first GNSS receiver 31 receives ranging radio waves from satellites and calculates the absolute coordinates of the receiver, which are latitude, longitude, and altitude. The first GNSS receiver 31 is provided at the center of rotation of the boom 9. The crane 1B can obtain the absolute coordinates of the center of rotation of the boom 9 by the first GNSS receiver 31. In this embodiment, the first GNSS receiver 31 detects the absolute coordinates of the center of rotation of the boom 9, but it is sufficient that the position is such that the attitude of the boom 9 can be specified. For example, the first GNSS receiver 31 may be configured to detect the absolute coordinates of the tip of the boom 9.

The orientation sensor 32 detects the orientation. The orientation sensor 32 is composed of a geomagnetic sensor such as an MR sensor or an MI sensor, a GNSS receiver having two antennas, etc. The orientation sensor 32 detects the orientation in the extension direction of the boom 9. The orientation sensor 32 is provided at the center of rotation of the boom 9 or at the tip of the boom 9. The orientation sensor 32 may also be configured to be shared by the first GNSS receiver 31.

The second GNSS receiver 33 is a receiver that constitutes a global positioning satellite system. The second GNSS receiver 33 receives ranging radio waves from satellites and calculates the absolute coordinates of the receiver, which are latitude, longitude, and altitude. The second GNSS receiver 33 is provided in the unmanned air vehicle 26 of the imaging device 25. The crane 1B can obtain the absolute coordinates of the camera 27 provided in the unmanned air vehicle 26 by using the second GNSS receiver 33.

The communication device 34 can receive the absolute coordinates calculated by the first GNSS receiver 31. The communication device 34 can receive the absolute coordinates calculated by the second GNSS receiver 33.

As shown in FIG. 10, the image processing device 30 processes the image P captured by the camera 27. The image processing device 30 can convert the coordinate system of the display screen into an absolute coordinate system from the absolute rotation center coordinate Ca, which is the absolute coordinate of the rotation center received by the first GNSS receiver 31, the direction D of the extension direction of the boom 9 calculated by the direction sensor 32, the absolute camera coordinate Ia, which is the absolute coordinate of the camera 27 received by the second GNSS receiver 33, and the attitude information Ip of the boom 9. The image processing device 30 can calculate the absolute tip coordinate Ba, which is the absolute coordinate of the tip of the boom 9. In addition, the image processing device 30 can calculate the absolute target coordinate Ta from the target coordinate T of the tip of the boom 9 input to the display device 21 using a conversion formula. The image processing device 30 transmits the calculated absolute target coordinate Ta to the control device 24.

When performing image processing using the image processing device 30, the crane 1B hovers the unmanned aerial vehicle 26 of the imaging device 25 above the tip of the boom 9 before transport begins (see Figure 6). Even if the boom 9 moves, the unmanned aerial vehicle 26 maintains the hovering position it had before the boom 9 moved. At this time, the image processing device 30 performs sway correction so that the position on the screen of a target object such as a structure that is automatically detected or specified by the operator does not fluctuate.

The image processing device 30 of the crane 1B acquires an image P from above the crane 1B taken by the camera 27 via the communication device 34 every unit time. In addition, the image processing device 30 acquires the attitude information Ip of the boom 9 from the control device 24 every unit time. The image processing device 30 displays the acquired image P on the display device 21.

The image processing device 30 acquires the absolute rotation center coordinate Ca from the first GNSS receiver 31, acquires the direction D of the extension direction of the boom 9 from the direction sensor 32, and acquires the absolute camera coordinate Ia from the second GNSS receiver 33. The image processing device 30 calculates the absolute tip coordinate Ba from the absolute rotation center coordinate Ca, the direction D of the extension direction of the boom 9, and the attitude information Ip of the boom 9 based on the positional relationship between the absolute camera coordinate Ia and the absolute rotation center coordinate Ca.

The image processing device 30 calculates the target coordinate T input by the operator from the screen of the display device 21. The image processing device 30 calculates the absolute target coordinate Ta from the calculated target coordinate T based on the absolute camera coordinate Ia and the angle of view information Ii of the camera 27. The image processing device 30 transmits the calculated absolute target coordinate Ta to the control device 24.

With this configuration, the control device 24 calculates the absolute tip coordinate Ba from the absolute rotation center coordinate Ca of the boom 9, the orientation D of the extension direction of the boom 9, the absolute camera coordinate Ia, the angle of view information Ii of the camera 27, and the attitude information Ip of the boom 9. The control device 24 also calculates a conversion formula for converting coordinates in the coordinate system of the display screen into absolute coordinates based on the absolute camera coordinate Ia and the angle of view information Ii of the camera 27. This makes it possible to convert the coordinate system of the display screen into an absolute coordinate system.

In each of the above-described embodiments, the cranes 1, 1A, and 1B are configured to set the target position on the image P displayed on the display device 21, but when position information including the coordinates of another work vehicle is acquired, the coordinates of the other work vehicle may be set as the target coordinates T.

In another embodiment of the present invention, when the other work vehicle (e.g., a crane, aerial work platform, truck, etc.) is equipped with a GNSS receiver, the control device 24 of the cranes 1, 1A, 1B may be configured to acquire position information from the GNSS receiver of the other work vehicle via a communication device or the like. In addition, when the other work vehicle has an AR marker, the control device 24 of the cranes 1, 1A, 1B may be configured to acquire position information of the other work vehicle by photographing the AR marker with the photographing device 25. The control device 24 calculates the transformation target coordinate Tc, which is the coordinate of the other work vehicle in the crane coordinate system of the cranes 1, 1A, 1B, from the acquired position information of the other work vehicle. In addition, the control device 24 generates an operation signal for the crane device operating tool 19 for the tip of the boom 9 to reach the transformation target coordinate Tc along the predetermined movement path R at a predetermined speed.

The control device 24 displays on the display device 21 information regarding the operation of the crane apparatus operating tool 19 required to move the tip of the boom 9 along a predetermined movement path R with the other work vehicle as the target position. The operator can operate the cranes 1, 1A, 1B while grasping the situation at the work site in real time using the imaging device 25. The operator can also move the tip of the boom 9 to the other work vehicle by operating the crane apparatus operating tool 19 according to the target operation position and operation sequence relative to the current operation position of the crane apparatus operating tool 19. This allows the work vehicle to move the tip of the boom 9 along the predetermined movement path R regardless of the operator's level of skill.

In each of the above-described embodiments, the control device 24 of the cranes 1, 1A, and 1B transmits control signals to each actuator of the crane device 6 while the crane device operating tool 19 is being operated or the graphic representing the position of the crane device operating tool 19 on the display device 21 is being moved. However, the control device 24 may also be configured to stop the movement of the boom 9 or to notify the user of this when the tip of the boom 9 moves away from the specified movement path R by a predetermined distance or more.

In addition, in each of the above-mentioned embodiments, the image processing device of the cranes 1, 1A, 1B displays the current operating position and the target operating position of each operating tool as a graphic on the display device 21, but this is not limited to this. The image processing device may be configured to display the current operating position and the target operating position of each operating tool in a form that allows the current operating position and the target operating position to be distinguished and visually recognized, such as by displaying them with a level indicator, by displaying them with an arrow, or by changing the shape and color of the graphic.

In addition, in each of the above-mentioned embodiments, the control device 24 of the cranes 1, 1A, and 1B may be configured to inform the operator that the current operating position of each operating tool matches the target operating position by displaying a figure, color, letter, icon, etc., an alarm by sound, vibrating the operating tool, fixing the operating tool, increasing the operating resistance of the operating tool, etc.

In addition, in each of the above-described embodiments, the control device 24 displays on the display device 21 information regarding the operation of the crane device operating tool 19 required to move the tip of the boom 9 along a predetermined movement path R with the other work vehicle as the target position. However, the control device 24 may set a person or a feature as the target position as long as it can recognize the positional relationship with the cranes 1, 1A, 1B using an AR marker or the like.

The above-mentioned embodiment is merely a representative example, and various modifications can be made without departing from the gist of the embodiment. Of course, the present invention can be implemented in various other forms, and the scope of the present invention is indicated by the claims, and further includes the equivalent meanings set forth in the claims, and all modifications within the scope of the claims.

REFERENCE SIGNS LIST 1 Crane 6 Crane device 9 Boom 9b Boom camera 17 Sensor for crane device 18 Traveling operation tool 19 Operation tool for crane device 21 Display device 22 Input device 23 Image processing device 24 Control device P Image C Swing center coordinate B Tip coordinate T Target coordinate Sa Current operation signal Sb Current operation signal Pa Current operation position coordinate Pb Target operation position coordinate

Claims (8)

A work vehicle having a boom that can be raised and lowered via a swivel on a traveling body;
An imaging device that images the surroundings of the work vehicle from above;
a display device that displays the image captured by the image capture device and is capable of outputting coordinates of any position on the display screen that is specified from outside;
A control device for controlling an actuator of the work vehicle,
The control device includes:
a work vehicle that acquires the position of the tip of the boom or the position of the boom's rotation center relative to the imaging device, attitude information of the boom, angle of view information of the image, and an image of the surroundings of the work vehicle captured from above by the imaging device, displays the image on the display device, calculates the coordinate of the tip of the boom in the coordinate system of the display screen and the coordinate of the boom's rotation center in the coordinate system of the display screen, calculates the coordinate in the coordinate system of the display screen of a target position of the tip of the boom set on the image, and displays current operating positions and target operating positions of each operating tool of the actuator so that the tip of the boom reaches the target position via a predetermined movement path. The work vehicle according to claim 1, wherein the control device displays the operation sequence of each operating tool of the actuator. The imaging device is
Provided at the tip of the boom,
The control device includes:
3. A work vehicle as described in claim 1 or claim 2, wherein the center position of the image is set to the tip position of the boom, and the coordinates of the tip of the boom in the coordinate system of the display screen and the coordinates of the center of rotation of the boom in the coordinate system of the display screen are calculated from the boom posture information. The control device includes:
3. The work vehicle according to claim 1, wherein an image of the tip of the boom is detected from an image captured by the photographing device, and the coordinates of the boom's center of rotation in the coordinate system of the display screen and the coordinates of the boom tip in the coordinate system of the display screen are calculated from the orientation and size of the image of the tip of the boom and attitude information of the boom. A first GNSS receiver that detects absolute coordinates of a rotation center of the boom or absolute coordinates of a tip of the boom;
An orientation sensor for detecting an orientation of the extension direction of the boom;
A second GNSS receiver that detects absolute coordinates of the imaging device,
The control device includes:
3. The work vehicle according to claim 1 or 2, further comprising: a detection value of the first GNSS receiver, a detection value of the second GNSS receiver, a detection value of the orientation sensor, and attitude information of the boom; and an absolute coordinate of a rotation center of the boom and an absolute coordinate of a tip of the boom are calculated by acquiring the detection value of the first GNSS receiver, the detection value of the second GNSS receiver, the detection value of the orientation sensor, and attitude information of the boom. A work vehicle according to any one of claims 1 to 5, in which the operating positions of the operating tools of each actuator are displayed graphically. The display device includes:
The work vehicle according to claim 1 , wherein an operating tool for the actuator is operable. The control device includes:
The work vehicle according to claim 1 , wherein when coordinates of another work vehicle are acquired, the position of the other work vehicle is set as the target position.

Images