JP7633899B2 - Image Evaluation Device - Google Patents

Description

The present invention relates to an image evaluation device.

Continuous operation of a work machine can cause parts that make up the machine to wear out. In order to properly manage the wear state of parts that wear out, the amount of wear in those parts may be measured. One method for measuring the amount of wear is to take an image of the part that wears out using an imaging device such as a camera, identify the outline of that part from the image, and compare it with the outline of that part before it wore out, thereby measuring the amount of wear in that part.

In the above measurement method, it is important that the captured image of the part that wears is suitable for measuring the amount of wear. For example, in a crawler-type hydraulic excavator, the sprocket of the undercarriage can wear out. When measuring the amount of wear on a sprocket, it is difficult to ensure the measurement accuracy of the amount of wear unless the image is taken directly in front of the sprocket by facing the imaging device directly in front of the sprocket.

As a technology for acquiring an image captured from a specific direction, such as a front image, for example, the technology disclosed in Patent Document 1 is known. Patent Document 1 discloses an image processing device that includes an imaging means for capturing an image of a subject plane, a plane measurement means for measuring the orientation of the subject plane relative to the imaging means, and an image distortion correction means for correcting distortion caused by oblique shooting of the image captured by the imaging means based on the measured orientation of the subject plane.

JP 2000-307947 A

However, when using the technology of Patent Document 1 to measure the amount of wear, it is necessary to prepare a special means called a planar measuring means before imaging, and to use the planar measuring means to image the part that wears out when imaging. In this case, since the special means called a planar measuring means must be distributed to maintenance personnel and users of the work machine, it is not realistic to use the technology of Patent Document 1 to measure the amount of wear. In particular, even if the technology of Patent Document 1 is used, it is difficult to casually image the part that wears out using a smartphone or the like and measure the amount of wear in that part.

In addition, the technology of Patent Document 1 requires the use of a special means called a planar measurement means when capturing images of the worn areas, and therefore the technology of Patent Document 1 cannot be applied to images that have already been captured without using this means.

For these reasons, there is a demand for technology that can accurately and easily evaluate whether images of parts of a work machine that are subject to wear are suitable for measuring the amount of wear.

In view of the above circumstances, the present invention aims to simplify and streamline the work of measuring the amount of wear by appropriately and easily evaluating whether an image of a part of a work machine that wears is suitable for measuring the amount of wear.

In order to solve the above problem, the image evaluation device of the present invention is an image evaluation device that evaluates whether a captured image of a specific part of a work machine is suitable as a measurement image used to measure the amount of wear of the specific part, and is characterized in that a learning image including the contour of the specific part and a plurality of feature points for identifying the contour of the specific part included in the learning image are machine-learned as teacher data, and the image evaluation device is characterized in that it includes: a feature point extraction unit that extracts feature points that identify the contour of the specific part included in the captured image; a reference value storage unit that stores a reference value that is calculated from the feature points in the learning image captured from a direction suitable for measuring the amount of wear and serves as a reference for evaluating the captured image; an evaluation value calculation unit that calculates an evaluation value for evaluating whether the captured image is suitable as the measurement image based on the feature points in the captured image extracted by the feature point extraction unit and the reference value stored in the reference value storage unit; and an image judgment unit that judges whether the captured image can be used as the measurement image according to the evaluation value calculated by the evaluation value calculation unit.

According to the present invention, it is possible to appropriately and easily evaluate whether an image of a part of a work machine that is subject to wear is suitable for measuring the amount of wear, thereby simplifying and streamlining the work of measuring the amount of wear.

1 is a diagram illustrating a schematic configuration of a wear amount measuring system including an image evaluation device according to a first embodiment. FIG. 2 is a diagram showing a configuration of the work machine shown in FIG. 1 . FIG. 2 is a block diagram showing a functional configuration of the image evaluation device shown in FIG. 1 . FIG. 4 is a block diagram showing a detailed configuration of a reference value calculation unit and an evaluation value calculation unit shown in FIG. 3 . FIG. 13 is a schematic diagram showing a front image suitable for measuring the amount of wear of a sprocket. FIG. 4 is a diagram illustrating characteristic points that identify the contour of a sprocket. 5 is a flowchart showing the operation of the image evaluation device in a reference value learning mode. 5 is a flowchart showing the operation of the image evaluation device in an image extraction mode. FIG. 11 is a block diagram showing the detailed configuration of a reference value calculation unit and an evaluation value calculation unit included in an image evaluation device according to a second embodiment. FIG. 4 is a diagram showing a schematic configuration of a tip portion of a bucket. FIG. 13 is a schematic diagram showing an oblique image suitable for measuring the amount of wear of teeth. FIG. 13 is a diagram illustrating characteristic points that identify the contours of the tooth and adapter. FIG. 11 is a block diagram showing the functional configuration of an image evaluation device according to a third embodiment. 5 is a flowchart showing the operation of the image evaluation device in an automatic imaging mode.

Embodiments of the present invention will be described below with reference to the drawings. Components with the same reference numerals in each embodiment have the same functions in each embodiment unless otherwise specified, and the description thereof will be omitted.

Using current AI technology, it is possible to automatically measure the amount of wear from a captured image 461 that includes parts of the work machine 6 that are subject to wear, such as the sprocket 63 (see Figure 2) and teeth 693 (see Figure 10). However, high accuracy in the order of millimeters may be required to measure the amount of wear. To ensure highly accurate measurements, the images used to measure the amount of wear (hereinafter also referred to as "measurement images") differ for each part of the work machine 6.

For example, the measurement image of the sprocket 63 is a front image captured by facing the imaging device 31 directly to the front 633 of the sprocket 63 (see FIG. 5). The sprocket 63 has a measurement gauge image that fits the teeth 631 and tooth roots 632 (see FIG. 5) of the sprocket 63 without any gaps before wear. The amount of wear of the sprocket 63 can be easily measured by measuring the size of the gap between the teeth 631 and tooth roots 632 included in the image 461 of the sprocket 63 and the measurement gauge image by image processing. In order to measure the amount of wear of the sprocket 63 with high accuracy from the two-dimensional image 461 and the measurement gauge image, it is important to fit the measurement gauge image to the front image that has almost no depth. Therefore, the measurement image of the sprocket 63 is a front image captured by facing the imaging device 31 directly to the front 633 of the sprocket 63.

On the other hand, the measurement image of the tooth 693 is an image (hereinafter also referred to as an "oblique image") captured by directing the image capture device 31 at a moderate angle to the front 694 of the tooth 693 so that the front 694 and side 695 of the adapter 692 and tooth 693, which will be described later, are captured (see FIG. 11). Since the tooth 693 does not have a plane that serves as a reference for the front, it is difficult to evaluate how close the captured image 461 of the tooth 693 is to a front image. Rather, information on the side 695 of the tooth 693 should be used. This is because if the boundary 696 between the tooth 693 and the adapter 692 is used as the measurement standard, even if the bucket 69 is rotating, there is no problem in measuring the amount of wear. Through repeated experiments, the inventors have succeeded in proving that by using the boundary 696 as the measurement standard, the difference between the measured amount of wear from the image and the actual measured value can be made small. Therefore, the measurement image of tooth 693 is an oblique image captured from a moderately oblique direction relative to the front 694 of tooth 693 so that the front 694 and side 695 of each of the adapter 692 and tooth 693 are visible.

The image evaluation device 40 is a device that evaluates whether a captured image 461 including a worn part of the work machine 6 is suitable as a measurement image. The image evaluation device 40 can evaluate the captured images 461 of various parts by performing processing appropriate for each part included in the captured image 461 to be evaluated.

[Embodiment 1]
An image evaluation device 40 according to the first embodiment will be described with reference to FIGS.

In the first embodiment, an image evaluation device 40 is described that evaluates whether a captured image 461 of a sprocket 63 is suitable as a measurement image when the part of the work machine 6 that wears is the sprocket 63.

Figure 1 is a schematic diagram showing the configuration of a wear amount measurement system 1 equipped with an image evaluation device 40 according to embodiment 1.

The wear amount measurement system 1 is a system that measures the amount of wear of a worn part of a work machine 6 from an image of the part. The wear amount measurement system 1 is a system in which a server device 2 that manages the wear amount measurement system 1 and a plurality of communication terminals 3, each equipped with an image capture device 31, are connected via a network so as to be capable of wireless communication. In the wear amount measurement system 1 of this embodiment, the image evaluation device 40 is provided in the communication terminal 3, but all or part of the functions of the image evaluation device 40 may be provided in the server device 2.

The communication terminal 3 is a mobile communication terminal such as a smartphone or a tablet PC. The communication terminal 3 includes an imaging device 31 capable of capturing moving or still images, a display device 32 and an input device 33 that constitute a touch panel display, and an image evaluation device 40.

The image evaluation device 40 includes a calculation processing device 401 including a CPU and the like, and a storage device 402 including a ROM and a RAM. The storage device 402 includes data storage areas including a teacher data storage unit 41, a reference value storage unit 45, and a captured image storage unit 46, which will be described later. The storage device 402 includes a program storage area that stores various programs that realize the functions of a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, an evaluation value calculation unit 47, an image determination unit 48, and an image extraction unit 49, which will be described later. The calculation processing device 401 can realize various functions of the image evaluation device 40 by executing various programs stored in the program storage area of the storage device 402 using various data stored in the data storage area of the storage device 402.

Figure 2 is a diagram showing the configuration of the work machine 6 shown in Figure 1.

The work machine 6 is not particularly limited, but may be, for example, a construction machine such as a hydraulic excavator. In this embodiment, the work machine 6 is described as being a crawler-type hydraulic excavator.

The work machine 6 includes a lower running body 61, an upper rotating body 65 rotatably attached to the lower running body 61 via a rotating device 651, and a multi-joint front work machine 66 attached to the front of the upper rotating body 65.

The lower running body 61 includes an idler 62 provided at the front end of the lower running body 61, a sprocket 63 provided at the rear end of the lower running body 61 and driven by a hydraulic motor, and a belt-shaped crawler 64 stretched over the idler 62 and sprocket 63. The sprocket 63 may wear out due to friction with the crawler 64 or friction with soil or sand that has gotten between the sprocket 63 and the crawler 64.

The upper rotating body 65 includes a cab 652 provided at the front of the upper rotating body 65 and a machine room 653 provided at the rear of the upper rotating body 65 to house the drive unit of the work machine 6.

The front work machine 66 comprises a boom 67 rotatably supported on the front of the upper rotating body 65, an arm 68 rotatably supported on the tip of the boom 67, and a bucket 69 rotatably supported on the tip of the arm 68. The boom 67, arm 68, and bucket 69 are connected to a boom cylinder 671, an arm cylinder 681, and a bucket cylinder 691 via links, respectively, and rotate when these hydraulic cylinders extend and retract. Teeth 693 are attached to the tip 69a of the bucket 69 at intervals in the width direction. The teeth 693 may wear out due to friction with the soil and sand being excavated.

Figure 3 is a block diagram showing the functional configuration of the image evaluation device 40 shown in Figure 1. Figure 4 is a block diagram showing the detailed configuration of the reference value calculation unit 44 and the evaluation value calculation unit 47 shown in Figure 3. Figure 5 is a schematic diagram showing a front image G1 suitable for measuring the amount of wear of the sprocket 63. Figure 6 is a diagram explaining feature points that identify the contour of the sprocket 63.

The image evaluation device 40 extracts feature points that identify the contour of a part that wears, such as a sprocket 63, from a captured image 461 that includes the part, and calculates an evaluation value that evaluates the degree to which the captured image 461 is suitable as a measurement image. The image evaluation device 40 determines whether the captured image 461 can be used as a measurement image based on the calculated evaluation value. The feature point extraction function is constructed as a model generated by machine learning. The captured image 461 to be evaluated has already been captured and is stored in advance in the captured image storage unit 46. The image evaluation device 40 can evaluate whether multiple captured images 461 stored in the captured image storage unit 46 are suitable as measurement images, and extract a measurement image from the multiple captured images 461.

The image evaluation device 40 includes a teacher data storage unit 41, a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, a reference value storage unit 45, a captured image storage unit 46, an evaluation value calculation unit 47, an image determination unit 48, and an image extraction unit 49.

The functions of the image evaluation device 40 can be broadly divided into functions that are used or executed in the learning stage, in which machine learning is performed to generate a feature point extraction function and the reference value X described below is calculated, and functions that are used or executed in the utilization stage, in which the captured image 461 is evaluated as to whether it is suitable as a measurement image.

First, the functions used or executed in the learning stage of the image evaluation device 40 will be described.
The teacher data memory unit 41 stores, in association with each other, a learning image 411 showing the contours of a part of the work machine 6 that is subject to wear, and feature point data 412 indicating the coordinates of a number of feature points contained in the learning image 411 for identifying the contours of that part.

The learning unit 42 uses the learning images 411 and feature point data 412 stored in the teacher data storage unit 41 as teacher data, performs machine learning to extract feature points, and generates a feature point extraction unit 43. The machine learning algorithm possessed by the learning unit 42 is not particularly limited, and may be, for example, a random forest. When evaluating whether a captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image, the machine learning-prepared feature point extraction unit 43 extracts feature points that identify the contour of a part included in the captured image 461.

The learning image 411 is an image including the contours of the front 633 and back 634 of the tooth bottom 632 that forms the outer edge of the sprocket 63. The learning image 411 is acquired by capturing images of the teeth 631 and tooth bottom 632 of the sprocket 63 from multiple imaging directions including a direction suitable for measuring the amount of wear. The learning image 411 may be an image captured by the imaging device 31 of the communication terminal 3, or may be an image captured by another imaging device different from the imaging device 31. The direction suitable for measuring the amount of wear is, for example, an imaging direction in which the optical axis of the imaging device 31 is perpendicular to a plane including the front 633 of the sprocket 63. The image captured from the direction suitable for measuring the amount of wear is a true-on image G1 as shown in FIG. 5. The image captured from the imaging direction other than the direction suitable for measuring the amount of wear is an image G2 as shown in FIG. 6. The image G2 shown in FIG. 6 is an image including the contours of the front 633 and back 634 of the tooth bottom 632 of the sprocket 63.

The front surface 633 faces outward in the left-right direction of the work machine 6, the back surface 634 faces inward in the left-right direction of the work machine 6, and the side surface 635 is a surface that is continuous with the front surface 633 and the back surface 634 and runs along the circumferential direction of the sprocket 63.

6, in an image including the contours of the tooth bottom 632 of the sprocket 63, a front circle F approximating the contour of the front surface 633 of the tooth bottom 632 and a back circle B approximating the contour of the back surface 634 of the tooth bottom 632 can be drawn. In an image including the contours of the tooth bottom 632 of the sprocket 63, the center Cf of the front circle F and the center Cb of the back circle B coincide with each other, which is almost equivalent to the image being a true front image G1. In an image in which the contour of the back surface 634 is completely hidden, the center Cb of the back surface 634 is unknown. When acquiring an image including the contours of the tooth bottom 632 of the sprocket 63, if the attitude of the imaging device is changed so that the center Cf of the front circle F and the center Cb of the back circle B overlap from a situation in which the center Cf of the front circle F and the center Cb of the back circle B are captured, the two will approach each other. Eventually, the center Cf of the front circle F and the center Cb of the back circle B coincide with each other and the center Cb of the back circle B disappears. The image captured at the moment the center Cb of the back circle B disappears can be considered the front image G1.

Due to the principle of perspective, the radius Rb of the back circle B is smaller than the radius Rf of the front circle F. Therefore, it is impossible to overlap the contour of the front surface 633 of the tooth base 632 with the contour of the back surface 634. To obtain a true-front image G1 without using the technique of drawing the front circle F and the back circle B, the contour of the front surface 633 of the tooth base 632 and the contour of the back surface 634 must be viewed simultaneously in parallel, while taking into consideration the balance of the deviation between the two, and the image must be captured intuitively. This makes it very difficult to obtain a true-front image G1. However, by adopting the technique of drawing the front circle F and the back circle B, a more accurate true-front image G1 can be obtained by a simple operation of simply overlapping the center Cf of the front circle F with the center Cb of the back circle B as described above.

For this reason, the image evaluation device 40 of this embodiment sets the coordinates of the points on each contour of the tooth base 632 of the sprocket 63 at which the front circle F and the back circle B can be drawn as feature point data 412. P1 to P4 in Figure 6 are feature points that specify the contour of the front 633 of the tooth base 632 of the sprocket 63. P5 to P8 in Figure 6 are feature points that specify the contour of the back 634 of the tooth base 632 of the sprocket 63.

The reference value calculation unit 44 calculates a reference value X that serves as a criterion for evaluating whether the captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image. As shown in FIG. 4, the reference value calculation unit 44 includes a center radius calculation unit 441 and a ratio calculation unit 442.

As described above, in an image including the contours of the tooth bottom 632 of the sprocket 63, the center Cf of the front circle F and the center Cb of the back circle B coincide with each other, which is almost equivalent to the image being a true front image G1. However, since the radius Rf of the front circle F and the radius Rb of the back circle B are different, the points to be coincident are, to be precise, points closer to the tooth bottom 632 than the center Cf and points closer to the tooth bottom 632 than the center Cb. These points to be coincident are reference points (hereinafter also referred to as "reference points") for evaluating whether the captured image 461 to be evaluated is suitable for a measurement image. The reference point of the front circle F and the reference point of the back circle B should be set to a point away from the center Cf and the center Cb on the tooth bottom 632 side by a distance calculated by multiplying the radius Rf and the radius Rb, respectively, depending on the width and thickness of the tooth bottom 632, and therefore depending on the type of the sprocket 63. In the image evaluation device 40 of this embodiment, before evaluating the captured image 461, the reference value calculation unit 44 calculates this ratio as a reference value X and stores it in the reference value storage unit 45 for each type of sprocket 63. In an image that includes the contours of the tooth bottoms 632 of the sprocket 63, the fact that the reference point of the front circle F and the reference point of the back circle B coincide is completely equivalent to the image being a true front image G1.

For this reason, the image evaluation device 40 of this embodiment determines the point closer to the tooth base 632 than the center Cf of the front circle F as the reference point of the front circle F, and the point closer to the tooth base 632 than the center Cb of the back circle B as the reference point of the back circle B. Then, the image evaluation device 40 of this embodiment determines the image in which the reference point of the front circle F and the reference point of the back circle B coincide as the true front image G1.

When calculating the reference value X, the reference value calculation unit 44 uses feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear. The learning image 411 used to calculate the reference value X may be, for example, a learning image 411 stored in the teacher data storage unit 41, or a learning image 411 newly captured by the imaging device 31 from a direction suitable for measuring the amount of wear in order to calculate the reference value X. The reference value calculation unit 44 inputs these learning images 411 to the feature point extraction unit 43 that has been machine-learned. The feature point extraction unit 43 extracts feature points that specify the contour of the front surface 633 of the tooth bottom 632 and feature points that specify the contour of the back surface 634. The feature point extraction unit 43 specifies the front circle F and the back circle B from the extracted feature points.

The center radius calculation unit 441 calculates the center Cf and radius Rf of the front circle F identified by the feature point extraction unit 43, and the center Cb and radius Rb of the back circle B identified by the feature point extraction unit 43. The ratio calculation unit 442 of the reference value calculation unit 44 calculates a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B. The ratio calculation unit 442 calculates X on the calculated straight line that satisfies the following formula (1).
Distance from center Cf of front circle F (radius Rf × X) = distance from center Cb of back circle B (radius Rb × X) (1)

The reference value calculation unit 44 determines a point on a line passing through the center Cf of the front circle F and the center Cb of the back circle B, which is a distance from the center Cf to the tooth bottom 632 side by the distance obtained by multiplying the radius Rf by X, as the reference point of the front circle F. The reference value calculation unit 44 determines a point on a line passing through the center Cf of the front circle F and the center Cb of the back circle B, which is a distance from the center Cb to the tooth bottom 632 side by the distance obtained by multiplying the radius Rb by X, as the reference point of the back circle B. The reference value calculation unit 44 stores the calculated X in the reference value storage unit 45 as the reference value X. At this time, the reference value storage unit 45 stores the reference value X calculated by the reference value calculation unit 44 in association with the part number that identifies the type of the sprocket 63.

In this way, the reference value storage unit 45 stores the reference value X, which is calculated from the feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear, and serves as a standard for evaluating the captured image 461.

Next, functions used or executed in the utilization stage of the image evaluation device 40 will be described.
The captured image storage unit 46 stores a captured image 461. The captured image 461 stored in the captured image storage unit 46 is a moving image or a plurality of still images captured in advance. The moving image is composed of a plurality of frame images captured at a predetermined frame rate. When evaluating the moving image stored in the captured image storage unit 46, the image evaluation device 40 can evaluate each of the plurality of frame images that constitute the moving image.

When evaluating whether a captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image, the feature point extraction unit 43 extracts a number of feature points for identifying the contour of a specific portion contained in the captured image 461. Specifically, the captured image 461 stored in the captured image storage unit 46 is input to the feature point extraction unit 43. The feature point extraction unit 43 extracts feature points that identify the contour of the front surface 633 and feature points that identify the contour of the back surface 634 of the tooth bottom 632 of the sprocket 63 contained in the captured image 461. The feature point extraction unit 43 identifies a front circle F and a back circle B from the extracted feature points.

The evaluation value calculation unit 47 calculates an evaluation value for evaluating whether the captured image 461 is suitable as a measurement image based on the feature points in the captured image 461 extracted by the feature point extraction unit 43 and the reference value X stored in the reference value storage unit 45. As shown in FIG. 4, the evaluation value calculation unit 47 includes a reference value average calculation unit 471, a center radius calculation unit 472, a difference length calculation unit 473, and a difference ratio calculation unit 474.

The reference value average calculation unit 471 calculates the average value Xm of multiple reference values X that are linked to the part number of the sprocket 63 and stored in the reference value storage unit 45. The center radius calculation unit 472 calculates the center Cf and radius Rf of the front circle F identified by the feature point extraction unit 43, and the center Cb and radius Rb of the back circle B identified by the feature point extraction unit 43.

The differential length calculation unit 473 calculates a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B. The differential length calculation unit 473 calculates a differential length D on the calculated straight line, which is expressed by the following equation (2).
Differential length D = {point separated by a distance (radius Rf x Xm) from the center Cf of the front circle F} - {point separated by a distance (radius Rb x X) from the center Cb of the back circle B} (2)

The difference ratio calculation unit 474 calculates the difference ratio Δ expressed by the following equation (3).
Difference ratio Δ=difference length D/(radius Rf of front circle F+radius Rb of rear circle B) (3)

The evaluation value calculation unit 47 uses the difference ratio Δ calculated by the difference ratio calculation unit 474 as an evaluation value for evaluating whether the captured image 461 is suitable as a measurement image. The evaluation value calculation unit 47 inputs the difference ratio Δ calculated as the evaluation value to the image determination unit 48.

The image determination unit 48 determines whether the captured image 461 can be used as a measurement image according to the evaluation value calculated by the evaluation value calculation unit 47. Specifically, the image determination unit 48 compares the difference ratio Δ calculated as the evaluation value with the threshold value T. The threshold value T is a threshold value for determining whether the captured image 461 can be used as a measurement image, and is determined in advance for each type of sprocket 63. The threshold value T is stored in advance in a threshold value storage unit (not shown) of the image evaluation device 40 in association with the part number of the sprocket 63. If the difference ratio Δ is smaller than the threshold value T, the image determination unit 48 determines that the captured image 461 can be used as a measurement image. If the difference ratio Δ is equal to or greater than the threshold value T, the image determination unit 48 determines that the captured image 461 cannot be used as a measurement image. The image determination unit 48 inputs the determination result to the image extraction unit 49.

The image extraction unit 49 extracts the measurement image from the captured images 461 stored in the captured image storage unit 46 according to the judgment result of the image judgment unit 48. Specifically, the image extraction unit 49 extracts the captured image 461 corresponding to the measurement image from the multiple frame images or multiple still images constituting the moving image stored in the captured image storage unit 46 according to the judgment result of the image judgment unit 48. When the multiple captured images 461 correspond to the measurement image in the judgment result of the image judgment unit 48, the image extraction unit 49 may extract all the captured images 461 corresponding to the measurement image, may extract only the captured image 461 with the smallest difference ratio Δ, or may extract only a predetermined number of captured images 461 (for example, the top five captured images 461) in order from the smallest difference ratio Δ. The image extraction unit 49 outputs the captured images 461 corresponding to the extracted measurement image to the display device 32 and displays the image on the display device 32.

After judging whether the captured image 461 stored in the captured image storage unit 46 can be used as a measurement image, the image evaluation device 40 may link the judgment result to the captured image 461 and store it in the captured image storage unit 46 or output it to the display device 32. In this case, the image evaluation device 40 does not need to extract the captured image 461 that corresponds to the measurement image from the captured images 461 stored in the captured image storage unit 46. In this case, the image evaluation device 40 may omit the image extraction unit 49.

The image evaluation device 40 may simply extract the captured image 461 with the smallest difference ratio Δ as the measurement image, without determining whether or not all of the captured images 461 stored in the captured image storage unit 46 can be used as the measurement image. In this case, the image evaluation device 40 may omit the image determination unit 48.

Next, the operation of the image evaluation device 40 will be described. The image evaluation device 40 can operate in a machine learning mode that generates a feature point extraction function, a learning mode that calculates and stores the reference value X in advance, and an image extraction mode that extracts a measurement image from the captured image 461 stored in the captured image storage unit 46.

Figure 7 is a flowchart showing the operation of the image evaluation device 40 in the learning mode of the reference value X.

In step S701, the image evaluation device 40 accepts the start of a learning mode for the reference value X in response to a user's input operation on the input device 33.

In step S702, the image evaluation device 40 accepts the part number of the sprocket 63 in response to a user's input operation on the input device 33.

In step S703, the image evaluation device 40 determines whether or not it has accepted the erasure (initialization) of the reference value X in response to a user's input operation on the input device 33. If the image evaluation device 40 has not accepted the erasure of the reference value X, it proceeds to step S705. If the image evaluation device 40 has accepted the erasure of the reference value X, it proceeds to step S704.

In step S704, the image evaluation device 40 erases the reference value X. Specifically, the image evaluation device 40 erases the reference value X stored in the reference value storage unit 45, or the reference value X for which an error message was output in step S713.

In step S705, the image evaluation device 40 determines whether or not the end of the learning mode of the reference value X has been accepted in response to an input operation by the user on the input device 33. If the image evaluation device 40 accepts the end of the learning mode of the reference value X, it ends this process shown in FIG. 7. If the image evaluation device 40 has not accepted the end of the learning mode of the reference value X, it proceeds to step S706.

In step S706, the image evaluation device 40 reads the learning image 411 to be used to calculate the reference value X.

In step S707, the image evaluation device 40 extracts feature points that identify the contour of the front surface 633 of the tooth bottom 632 of the sprocket 63 and feature points that identify the contour of the back surface 634. The image evaluation device 40 identifies the front circle F and the back circle B from the extracted feature points.

In step S708, the image evaluation device 40 calculates the center Cf and radius Rf of the front circle F and the center Cb and radius Rb of the back circle B.

In step S709, the image evaluation device 40 calculates a reference value X. Specifically, the image evaluation device 40 calculates a reference value X that is on a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B and satisfies the above formula (1).

In step S710, the image evaluation device 40 determines whether or not the reference value X has already been stored in the reference value storage unit 45. If the reference value X has not already been stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S713. If the reference value X has already been stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S711.

In step S711, the image evaluation device 40 determines whether the difference between the currently calculated reference value X and the reference value X already stored in the reference value storage unit 45 is within the allowable range. If the difference between the currently calculated reference value X and the reference value X already stored in the reference value storage unit 45 is within the allowable range, the image evaluation device 40 proceeds to step S713. If the difference between the currently calculated reference value X and the reference value X already stored in the reference value storage unit 45 is not within the allowable range, the image evaluation device 40 proceeds to step S712. Note that whether or not it is within the allowable range is based on a threshold value that is set and stored in advance. The threshold value can be set arbitrarily.

In step S712, the image evaluation device 40 outputs an error message to the display device 32 and proceeds to step S703.

In step S713, the image evaluation device 40 stores the currently calculated reference value X in the reference value storage unit 45 as a new reference value X.

In this way, the image evaluation device 40 can calculate and store in advance the reference value X that serves as the standard for evaluating the captured image 461.

Figure 8 is a flowchart showing the operation of the image evaluation device 40 in image extraction mode.

In step S801, the image evaluation device 40 accepts the start of the image extraction mode in response to a user's input operation on the input device 33.

In step S802, the image evaluation device 40 accepts the part number of the sprocket 63 in response to a user's input operation on the input device 33.

In step S803, the image evaluation device 40 determines whether or not the reference value X is stored in the reference value storage unit 45. If the reference value X is stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S805. If the reference value X is not stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S804.

In step S804, the image evaluation device 40 outputs a message to the display device 32 indicating that the reference value X is not stored. The image evaluation device 40 then ends this process shown in FIG. 8.

In step S805, the image evaluation device 40 calculates the average value Xm of the reference values X stored in the reference value storage unit 45.

In step S806, the image evaluation device 40 reads the captured image 461 stored in the captured image storage unit 46.

In step S807, the image evaluation device 40 extracts feature points that identify the contour of the front surface 633 of the tooth bottom 632 of the sprocket 63 and feature points that identify the contour of the back surface 634. The image evaluation device 40 identifies the front circle F and the back circle B from the extracted feature points.

In step S808, the image evaluation device 40 calculates the center Cf and radius Rf of the front circle F and the center Cb and radius Rb of the back circle B.

In step S809, the image evaluation device 40 calculates the difference length D expressed by the above formula (2) on the straight line passing through the center Cf of the front circle F and the center Cb of the back circle B.

In step S810, the image evaluation device 40 calculates the difference ratio Δ expressed by the above formula (3).

In step S811, the image evaluation device 40 determines whether the currently loaded captured image 461 can be used as a measurement image by determining whether the calculated difference ratio Δ is smaller than a predetermined threshold value T.

In step S812, the image evaluation device 40 determines whether the judgment result in step S811 indicates that the image can be used as a measurement image. If the judgment result does not indicate that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S814. If the judgment result indicates that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S813.

In step S813, the image evaluation device 40 extracts the captured image 461 that was just read as a measurement image.

In step S814, the image evaluation device 40 determines whether or not all captured images 461 stored in the captured image storage unit 46 have been evaluated. If the image evaluation device 40 has not evaluated all captured images 461 stored in the captured image storage unit 46, the image evaluation device 40 proceeds to step S806. If the image evaluation device 40 has evaluated all captured images 461 stored in the captured image storage unit 46, the image evaluation device 40 proceeds to step S815.

In step S815, the image evaluation device 40 determines whether or not a measurement image has been extracted. If a measurement image has been extracted, the image evaluation device 40 proceeds to step S817. If a measurement image has not been extracted, the image evaluation device 40 proceeds to step S816.

In step S816, the image evaluation device 40 outputs a message to the display device 32 indicating that a measurement image was not extracted. The image evaluation device 40 then ends this process shown in FIG. 8.

In step S817, the image evaluation device 40 outputs the extracted measurement image to the display device 32. The image evaluation device 40 ends this process shown in FIG. 8.

As described above, the image evaluation device 40 of the first embodiment is an image evaluation device that evaluates whether the captured image 461 of a specific part (a part that wears, for example, the sprocket 63) of the work machine 6 is suitable as a measurement image used to measure the amount of wear. The feature point extraction unit 43 machine-learns the learning image 411 including the contour of the specific part and the feature point data 412 indicating the coordinates of a plurality of feature points for identifying the contour of the specific part included in the learning image 411 as teacher data, and extracts feature points that identify the contour of the specific part included in the captured image 461. The reference value storage unit 45 stores the reference value X that is calculated from the feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear and serves as a reference for evaluating the captured image 461. The evaluation value calculation unit 47 calculates an evaluation value that evaluates whether the captured image 461 is suitable as a measurement image based on the feature points in the captured image 461 extracted by the feature point extraction unit 43 and the reference value X stored in the reference value storage unit 45. The image determination unit 48 determines whether or not the captured image 461 can be used as a measurement image, depending on the evaluation value calculated by the evaluation value calculation unit 47.

As a result, the image evaluation device 40 of the first embodiment can evaluate with high accuracy whether the captured image 461 of a specific part of the work machine 6 is suitable for measuring the amount of wear, without using a special means such as the planar measurement means disclosed in Patent Document 1. Furthermore, the image evaluation device 40 of the first embodiment can evaluate with high accuracy whether the captured image 461 is suitable for measuring the amount of wear, regardless of whether the captured image 461 has already been captured. Therefore, the image evaluation device 40 of the first embodiment can appropriately and easily evaluate whether the captured image 461 is suitable for measuring the amount of wear, even if the captured image 461 is an image captured by a user casually using a smartphone or the like of a part that is subject to wear. By introducing the image evaluation device 40, the user can measure the amount of wear from the two-dimensional captured image 461 without using an expensive means such as a three-dimensional shape measuring machine that acquires a three-dimensional point cloud of the target part. The image evaluation device 40 of the first embodiment can simplify and improve the efficiency of the wear amount measurement work.

The image evaluation device 40 of the first embodiment further includes an image extraction unit 49 that extracts a measurement image. The captured image 461 to be evaluated in the first embodiment is a moving image or a plurality of still images captured in advance and stored in the captured image storage unit 46. The image extraction unit 49 extracts a measurement image from the plurality of frame images or the plurality of still images constituting the moving image stored in the captured image storage unit 46 according to the judgment result of the image judgment unit 48.

As a result, the image evaluation device 40 of embodiment 1 can automatically extract images suitable for measuring the amount of wear from a collection of moving images or still images that have already been captured. In particular, the image evaluation device 40 of embodiment 1 can automatically extract images suitable for measuring the amount of wear even from a collection of moving images or still images that have been captured without any purpose. Thus, the image evaluation device 40 of embodiment 1 can easily obtain images suitable for measuring the amount of wear, further simplifying and improving the efficiency of the work of measuring the amount of wear.

In the above-mentioned wear amount measurement system 1, the communication terminal 3 has all the functions of the image evaluation device 40, but the server device 2 may have all or part of the functions of the image evaluation device 40. For example, the server device 2 may have the teacher data storage unit 41 and the learning unit 42, and the communication terminal 3 may have the feature point extraction unit 43 that has been machine-learned. Alternatively, the server device 2 may also have the feature point extraction unit 43 that has been machine-learned, and the communication terminal 3 may receive the extraction result of the feature point extraction unit 43 transmitted from the server device 2. In addition, the server device 2 may have the teacher data storage unit 41, the learning unit 42, the feature point extraction unit 43, the reference value calculation unit 44, the reference value storage unit 45, the evaluation value calculation unit 47, the image judgment unit 48, and the image extraction unit 49, and the communication terminal 3 may only have the captured image storage unit 46. In addition, the server device 2 may also have the captured image storage unit 46, and the communication terminal 3 may receive the extraction result of the image extraction unit 49 transmitted from the server device 2 and output it to the display device 32.

[Embodiment 2]
The image evaluation device 40 of the second embodiment will be described with reference to Figures 9 to 12. In the image evaluation device 40 of the second embodiment, the description of the same configuration and operation as in the first embodiment will be omitted.

In the second embodiment, an image evaluation device 40 is described that evaluates whether a captured image 461 of a tooth 693 is suitable as a measurement image when the tooth 693 is a part of the work machine 6 that wears.

Figure 9 is a block diagram showing the detailed configuration of the reference value calculation unit 44 and evaluation value calculation unit 47 provided in the image evaluation device 40 of embodiment 2. Figure 10 is a diagram showing a schematic configuration of the tip portion 69a of the bucket 69. Figure 11 is a diagram showing a schematic oblique image G3 suitable for measuring the amount of wear of the tooth 693. Figure 12 is a diagram explaining characteristic points that identify the contours of the tooth 693 and the adapter 692. Note that the dashed lines shown in Figures 11 and 12 indicate the contour of the tooth 693 before wear.

As shown in FIG. 10, the teeth 693 are attached to adapters 692 provided at the tip 69a of the bucket 69 at intervals in the width direction. The adapters 692 are not easily worn. In this embodiment, the length and width of the adapter 692 are fixed. The front surface 694 of the adapter 692 and the tooth 693 shown in FIG. 11 is a surface that continues with the inner surface of the bucket 69. The side surface 695 of the adapter 692 and the tooth 693 shown in FIG. 11 is a surface that continues with the back surface that continues with the outer surface of the bucket 69 and the front surface 694, and is a surface that follows the dump/crowd direction of the bucket 69.

The amount of wear (wear rate) of tooth 693 can be measured as follows. That is, as shown in Fig. 11, the length of worn tooth 693 on the image is Lt, and the length of adapter 692 on the image is La. The actual length of adapter 692 is la (fixed value), and the length of tooth 693 before wear on the image is Lt0. In this case, the actual length lt of worn tooth 693 can be expressed by the following formula (4). The amount of wear (wear rate) of tooth 693 can be expressed by the following formula (5).
Actual length lt of worn tooth 693=(length Lt of worn tooth 693 on the image)×(length La of adapter 692 on the image)/(actual length la of adapter 692) (4)
Wear amount of tooth 693 (wear rate)=1−{(length Lt of worn tooth 693 on image)/(length Lt0 of tooth 693 before wear on image)} (5)

However, in an image close to being captured from the front 694 of the tooth 693, the area of the adapter 692 and the side 695 of the tooth 693 is narrow. Therefore, in an image close to being captured from the front 694 of the tooth 693, the length La of the adapter 692 on the image and the lengths Lt and Lt0 of the tooth 693 on the image cannot be accurately measured. On the other hand, in an image captured from an excessively oblique direction with respect to the front 694 of the tooth 693, the adapter 692 and the tooth 693 on the front side of the image overlap with the adapter 692 and the tooth 693 on the back side of the image, and a part of the side 695 of the adapter 692 and the tooth 693 on the back side of the image is not captured. Therefore, in an image captured from an excessively oblique direction with respect to the front 694 of the tooth 693, the length La of the adapter 692 on the image and the lengths Lt and Lt0 of the tooth 693 on the image cannot be accurately measured. Therefore, the image captured from a direction suitable for measuring the amount of wear is an image captured from a direction that allows accurate measurement of the length La on the image of the adapter 692 and the lengths Lt and Lt0 on the image of the tooth 693. The image captured from a direction suitable for measuring the amount of wear is an oblique image G3 captured from a moderately oblique direction with respect to the front surface 694 of the tooth 693, as shown in FIG. 11.

The oblique image G3 is an image including the contours of the front 694 and side 695 of the adapter 692 and the contours of the front 694 and side 695 of the tooth 693. In the oblique image G3, the ratio of the width Wf of the front 694 to the width Ws of the side 695 at the boundary 696 between the adapter 692 and the tooth 693 can be defined as a predetermined value (at least within a predetermined range) as the length on the image. In an image including the contours of the front 694 and side 695 of the adapter 692 and the contours of the front 694 and side 695 of the tooth 693, the ratio of the width Wf of the front 694 to the width Ws of the side 695 at the boundary 696 being a predetermined value (at least within a predetermined range) is equivalent to the image being an oblique image G3.

For this reason, in the image evaluation device 40 of embodiment 2, the coordinates of the vertices that specify the contours of the front 694 and side 695 of the adapter 692 and the coordinates of the vertices that specify the contours of the front 694 and side 695 of the tooth 693 are taken as feature point data 412. Image G4 shown in FIG. 12 is an image that includes the contours of the front 694 and side 695 of the adapter 692 and the contours of the front 694 and side 695 of the tooth 693. P11 to P14 in FIG. 12 are feature points that specify the contour of the front 694 of the tooth 693. P12, P14, and P15 in FIG. 12 are feature points that specify the contour of the side 695 of the tooth 693. P13, P14, P16, and P17 in FIG. 12 are feature points that specify the contour of the front 694 of the adapter 692. P14, P15, P17, and P18 in FIG. 12 are feature points that identify the contour of the side 695 of the adapter 692.

In the image evaluation device 40 of the second embodiment, an image including the contours of the front surface 694 and the side surface 695 of the adapter 692 and the contours of the front surface 694 and the side surface 695 of the tooth 693 is used as the learning image 411. The learning image 411 is obtained by capturing images of the front surface 694 and the side surface 695 of each of the adapter 692 and the tooth 693 from multiple imaging directions, including a direction suitable for measuring the amount of wear.

In addition, in the image evaluation device 40 of embodiment 2, the ratio of the width Wf of the front surface 694 to the width Ws of the side surface 695 at the boundary 696 between the adapter 692 and the tooth 693 is set as the reference value X.

Specifically, the reference value calculation unit 44 of the second embodiment inputs the learning image 411 to the machine-learned feature point extraction unit 43. The feature point extraction unit 43 of the second embodiment extracts feature points that specify the contours of the front 694 and side 695 of the adapter 692 included in the learning image 411, and feature points that specify the contours of the front 694 and side 695 of the tooth 693. The feature point extraction unit 43 specifies the width Wf of the front 694 and the width Ws of the side 695 at the boundary 696 between the adapter 692 and the tooth 693 from the extracted feature points.

As shown in FIG. 9, the reference value calculation unit 44 of the second embodiment includes a width ratio calculation unit 443. The width ratio calculation unit 443 calculates the ratio (Wf/Ws) between the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary portion 696 identified by the feature point extraction unit 43. The width ratio calculation unit 443 stores the calculated ratio (Wf/Ws) in the reference value storage unit 45 as the reference value X. At this time, the reference value storage unit 45 stores the reference value X calculated by the reference value calculation unit 44 in association with the part number that identifies the type of tooth 693.

When evaluating whether the captured image 461 stored in the captured image storage unit 46 is suitable as a measurement image, the feature point extraction unit 43 of the second embodiment extracts feature points that specify the contours of the front 694 and side 695 of the adapter 692 contained in the captured image 461, and feature points that specify the contours of the front 694 and side 695 of the tooth 693. From the extracted feature points, the feature point extraction unit 43 specifies the width Wf of the front 694 and the width Ws of the side 695 at the boundary 696 between the adapter 692 and the tooth 693.

As shown in FIG. 9, the evaluation value calculation unit 47 of the second embodiment includes a reference value average calculation unit 471, a width ratio calculation unit 475, and a width ratio difference calculation unit 476. As in the first embodiment, the reference value average calculation unit 471 calculates the average value Xm of multiple reference values X associated with the part number of the tooth 693 and stored in the reference value storage unit 45. The width ratio calculation unit 475 calculates the ratio X (=Wf/Ws) between the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary portion 696 identified by the feature point extraction unit 43. The width ratio difference calculation unit 476 calculates the difference value |X-Xm| between the ratio X calculated by the width ratio calculation unit 475 and the average value Xm calculated by the reference value average calculation unit 471.

In the second embodiment, the evaluation value calculation unit 47 uses the difference value |X-Xm| calculated by the width ratio difference calculation unit 476 as an evaluation value for evaluating whether the captured image 461 is suitable as a measurement image. The evaluation value calculation unit 47 inputs the difference value |X-Xm| calculated as the evaluation value to the image determination unit 48.

The image determination unit 48 of the second embodiment, like the first embodiment, determines whether or not the captured image 461 can be used as a measurement image, depending on the evaluation value calculated by the evaluation value calculation unit 47. The image extraction unit 49 of the second embodiment, like the first embodiment, extracts a measurement image from the captured images 461 stored in the captured image storage unit 46, depending on the determination result of the image determination unit 48.

As described above, the image evaluation device 40 of the second embodiment can appropriately and easily evaluate whether the captured image 461 is suitable for measuring the amount of wear, as in the first embodiment, even when a specific part of the work machine 6 does not have a directly frontal reference plane, such as the tooth 693. The image evaluation device 40 of the second embodiment can automatically extract images suitable for measuring the amount of wear from a collection of captured moving images or still images, as in the first embodiment.

[Embodiment 3]
13 and 14, the image evaluation device 40 of the third embodiment will be described. In the image evaluation device 40 of the third embodiment, the description of the same configuration and operation as in the first embodiment will be omitted.
FIG. 13 is a block diagram showing the functional configuration of an image evaluation device 40 according to the third embodiment.

The imaging device 31 has a live view function that displays an image of the subject on the display device 32 in real time when the image is captured. The image evaluation device 40 of the third embodiment does not evaluate the captured image 461 stored in the captured image storage unit 46, but evaluates the captured image 461 (hereinafter also referred to as a "live view image 501") acquired by the live view function of the imaging device 31. The image evaluation device 40 of the third embodiment guides the position or attitude of the imaging device 31 via the display device 32 so that the imaging device 31 can capture a measurement image suitable for measuring the amount of wear when the user uses the imaging device 31 to capture an image of a part of the work machine 6 that is subject to wear.

As shown in FIG. 13, the image evaluation device 40 of the third embodiment includes a live view image acquisition unit 50 instead of the captured image storage unit 46, and an image capture guide unit 51 instead of the image extraction unit 49.

The live view image acquisition unit 50 acquires the live view image 501 acquired by the live view function of the imaging device 31 from the imaging device 31, and inputs it to the learned feature point extraction unit 43. When the image determination unit 48 determines that the live view image 501 cannot be used as a measurement image, the imaging guidance unit 51 outputs a message to the display device 32 to guide the position or posture of the imaging device 31 so that the live view image 501 can be used as a measurement image. The imaging guidance unit 51 may output the message to a speaker (not shown) of the communication terminal 3. When the image determination unit 48 determines that the live view image 501 can be used as a measurement image, the imaging guidance unit 51 automatically captures the live view image 501 and stores it in a specified storage unit. The time for which automatic imaging is continued is arbitrary and continues as long as the live view image 501 is suitable as a measurement image, but it may be stopped at any timing by an operation instructing "stop". The imaging time may also be changed according to the progress of wear. In this case, a reference value for determining the progress of wear may be set in advance, and if the reference value is exceeded, it may be assumed that wear is progressing, and automatic imaging may be repeated at least two times, with the operator being asked to select the best image result. This makes it possible to grasp the progress of wear more reliably and with greater accuracy.

In this way, the image evaluation device 40 of embodiment 3 can operate in an automatic imaging mode in which the live view image 501 is automatically captured when it is suitable as a measurement image.

Figure 14 is a flowchart showing the operation of the image evaluation device 40 in automatic imaging mode.

In step S1401, the image evaluation device 40 accepts the start of the automatic imaging mode in response to a user's input operation on the input device 33.

In step S1402, the image evaluation device 40 determines whether or not a change to the threshold T has been accepted in response to an input operation by the user on the input device 33. The threshold T is a threshold for determining whether or not the live view image 501 can be used as a measurement image, and is stored in advance in a threshold storage unit (not shown) of the image evaluation device 40. If the image evaluation device 40 has not accepted a change to the threshold T, the image evaluation device 40 proceeds to step S1404. If the image evaluation device 40 has accepted a change to the threshold T, the image evaluation device 40 proceeds to step S1403.

In step S1403, the image evaluation device 40 resets the received threshold value T.

In step S1404, the image evaluation device 40 accepts the part number of the sprocket 63 in response to a user's input operation on the input device 33.

In step S1405, the image evaluation device 40 determines whether or not the reference value X is stored in the reference value storage unit 45. If the reference value X is stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S1407. If the reference value X is not stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S1406.

In step S1406, the image evaluation device 40 outputs a message to the display device 32 indicating that the reference value X is not stored. The image evaluation device 40 ends this process shown in FIG. 14.

In step S1407, the image evaluation device 40 calculates the average value Xm of the reference values X stored in the reference value storage unit 45.

In step S1408, the image evaluation device 40 reads the live view image 501 acquired by the live view image acquisition unit 50.

In steps S1409 to S1413, the image evaluation device 40 performs the same processing as steps S807 to S811 shown in FIG. 8.

In step S1414, the image evaluation device 40 determines whether the judgment result in step S1413 indicates that the image can be used as a measurement image. If the judgment result does not indicate that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S1417. If the judgment result indicates that the image can be used as a measurement image, the image evaluation device 40 proceeds to step S1415.

In step S1415, the image evaluation device 40 captures the currently loaded live view image 501 and stores it as a measurement image.

In step S1416, the image evaluation device 40 outputs the extracted measurement image to the display device 32. The image evaluation device 40 ends this process shown in FIG. 14.

In step S1417, the image evaluation device 40 outputs to the display device 32 a message guiding the position or attitude of the imaging device 31 so that the live view image 501 can be used as a measurement image. For example, if the reference point of the front circle F calculated in step S1410 is diagonally above and to the right of the reference point of the rear circle B, the image evaluation device 40 outputs to the display device 32 a message guiding the user to "move the imaging device 31 further diagonally above and to the right and capture an image from that position."

In step S1418, the image evaluation device 40 determines whether the position or attitude of the imaging device 31 has changed. The image evaluation device 40 can determine whether the position or attitude of the imaging device 31 has changed by checking a change in the live view image 501 or checking a detection signal from the attitude sensor of the communication terminal 3. If the position or attitude of the imaging device 31 has changed, the image evaluation device 40 proceeds to step S1408. If the position or attitude of the imaging device 31 has not changed, the image evaluation device 40 proceeds to step S1419.

In step S1419, the image evaluation device 40 determines whether or not the end of the automatic imaging mode has been accepted in response to an input operation by the user on the input device 33. If the image evaluation device 40 has not accepted the end of the automatic imaging mode, the image evaluation device 40 proceeds to step S1418. If the image evaluation device 40 has accepted the end of the automatic imaging mode, the image evaluation device 40 ends this process shown in FIG. 14.

As described above, the image evaluation device 40 of the third embodiment includes an image capture guide unit 51 that provides guidance on the position or posture of the image capture device 31 that captures images of parts of the work machine 6 that are subject to wear. The captured image 461 to be evaluated in the third embodiment is a live view image 501 acquired by the live view function of the image capture device 31. When the image determination unit 48 determines that the live view image 501 cannot be used as a measurement image, the image capture guide unit 51 outputs a message that provides guidance on the position or posture of the image capture device 31 so that the live view image 501 can be used as a measurement image.

As a result, the image evaluation device 40 of embodiment 3 can easily enable a user who is about to capture an image of a worn part of the work machine 6 to capture an image suitable for measuring the amount of wear. Therefore, the image evaluation device 40 of embodiment 3 can further simplify and improve the efficiency of the wear amount measurement work. Also, when capturing an image, due to the characteristics of the subject to be captured, there are cases where object recognition is not possible due to the presence of mud, etc. This is an obstacle to accurate measurement of the amount of wear. By capturing an image of the part to be captured in real time using a live view image, it becomes possible to obtain an even more accurate image suitable for measuring the amount of wear.

Although the third embodiment has been described taking as an example a case where the part of the work machine 6 that wears is the sprocket 63, the image evaluation device 40 of the third embodiment can also be applied to a case where the part of the work machine 6 that wears is the tooth 693. Furthermore, the image evaluation device 40 of the third embodiment is provided with a live view image acquisition unit 50 instead of the captured image storage unit 46, and an image capture guide unit 51 instead of the image extraction unit 49. However, the image evaluation device 40 may be provided with both the captured image storage unit 46 and the live view image acquisition unit 50, and may be provided with both the image extraction unit 49 and the image capture guide unit 51.

[Other embodiments]
The wear amount measurement system 1 measures the amount of wear using the measurement image extracted by the image evaluation device 40. The measurement of the amount of wear may be performed by the communication terminal 3 or by the server device 2. The wear amount measurement system 1 can identify the level of the measured amount of wear, create a wear amount indicator image including an indication according to the identified level, and output the image to the display device 32 of the communication terminal 3.

The wear level may be, for example, a first level where the worn part has reached a wear level that requires replacement, a second level where the worn part is likely to reach a wear level that requires replacement after a predetermined amount of work has been performed, and a third level where the worn part has not yet reached a wear level that requires replacement even after a predetermined amount of work has been performed.

The wear amount indicator image is created for each part by changing at least one of the shape, size, color, and pattern according to the level of wear. For example, if the wear amount level is the first level, the wear amount indicator image may be created using red, if the wear amount level is the second level, the wear amount indicator image may be created using yellow, and if the wear amount level is the third level, the wear amount indicator image may be created using green. Note that the levels and colors are not limited to three levels or three colors, and may be set arbitrarily. Alternatively, for example, the wear amount indicator image may be created so that the length of the wear amount indicator image is the longest when the wear amount level is the first level, the length of the wear amount indicator image is medium when the wear amount level is the second level, and the length of the wear amount indicator image is the shortest when the wear amount level is the third level. In other words, the indicator image may be created so that the length of the indicator image increases as the degree of recommendation for replacement of the worn part increases.

Furthermore, the wear amount measurement system 1 can predict future changes in the amount of wear from the current operating status of the work machine 6. The wear amount measurement system 1 can then create an image that displays the predicted changes in the amount of wear superimposed on the indicator image, and output the image to the display device 32. Alternatively, the wear amount measurement system 1 can create an animation in which the length of the indicator image changes according to the predicted changes in the amount of wear, and output the animation to the display device 32.

Furthermore, the wear amount measurement system 1 can aggregate the wear amounts of parts that wear in multiple work machines 6 in the server device 2, and identify multiple work machines 6 that have similar wear amounts. Then, the wear amount measurement system 1 can transmit information on multiple work machines 6 that have similar wear amounts to the users' communication terminals 3, allowing the information to be shared between users.

With this configuration, the wear amount measurement system 1 can notify the user of information such as the wear amount level, the timing for part replacement, and the predicted progress of the wear amount in a format that is easy to understand intuitively.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the present invention described in the claims. The present invention allows the configuration of one embodiment to be added to the configuration of another embodiment, the configuration of one embodiment to be replaced with another embodiment, or part of the configuration of one embodiment to be deleted.

1...Wear amount measurement system, 2...Server device, 3...Communication terminal, 31...Imaging device, 32...Display device, 33...Input device, 40...Image evaluation device, 401...Calculation processing device, 402...Storage device, 41...Teacher data storage unit, 411...Learning image, 412...Feature point data, 42...Learning unit, 43...Feature point extraction unit, 44...Reference value calculation unit, 441...Center radius calculation unit, 442...Ratio calculation unit, 443...Width ratio calculation unit, 45...Reference value storage unit, 46...Captured image storage unit, 461...Captured image, 47...Evaluation value calculation unit, 471...Reference value average calculation unit, 472...Center radius calculation unit, 473...Differential length calculation unit, 474...Differential ratio calculation unit, 475...Width ratio calculation unit, 476...Width ratio difference calculation unit, 48...Image determination unit, 49...Image extraction unit, 50...Live view image acquisition unit, 501...Live view image, 51...Image capture guide unit, 6...Work machine, 61...Lower travel unit, 62...Idler, 63...Sprocket, 631...Tooth, 632...Tooth bottom, 633...Front, 634...Rear, 635...Side, 64...Crawler, 65...Upper rotating body, 651...Swivel device, 652...Cab, 653...Machine room, 66...Front work machine, 67...Boom, 671...Boom cylinder, 68...Arm, 681...Arm cylinder, 69...Bucket, 69a...Tip, 691...Bucket cylinder, 692...Adapter, 693...Tooth, 694...Front, 695...Side, 696...Boundary

Claims (3)

1. An image evaluation device for evaluating whether a captured image of a specific portion of a work machine is suitable as a measurement image used for measuring an amount of wear of the specific portion,
a feature point extraction unit that extracts feature points that specify the contour of the specific part included in the captured image, the feature points being machine-trained as teacher data based on a learning image including the contour of the specific part and a plurality of feature points for specifying the contour of the specific part included in the learning image;
a reference value storage unit configured to store a reference value that is a reference for evaluating the captured image and that is calculated from the feature points in the learning image captured from a direction suitable for measuring the amount of wear; and
an evaluation value calculation unit that calculates an evaluation value for evaluating whether the captured image is suitable as the measurement image based on the feature points in the captured image extracted by the feature point extraction unit and the reference value stored in the reference value storage unit;
an image determination unit that determines whether or not the captured image can be used as the measurement image according to the evaluation value calculated by the evaluation value calculation unit;
An image evaluation device comprising: An image extraction unit that extracts the measurement image,
The captured image is a moving image or a plurality of still images captured in advance,
2. The image evaluation device according to claim 1, wherein the image extraction section extracts the measurement image from among a plurality of frame images constituting the moving image or the plurality of still images according to a determination result of the image determination section. An imaging guide unit that guides a position or a posture of an imaging device that images the specific part,
the captured image is a live view image acquired by the imaging device,
2. The image evaluation device according to claim 1, wherein, when the image determination unit determines that the live view image cannot be used as the measurement image, the imaging guidance unit outputs a message providing guidance on the position or attitude of the imaging device so that the live view image can be used as the measurement image.

Images