JP2023023802A - Image evaluation apparatus - Google Patents

Abstract

To properly and easily evaluate whether or not a captured image obtained by capturing an abrasion part of a work machine is an image properly suited for measurement of an abrasion amount to thereby achieve a simplified and efficient measurement work of the abrasion amount.SOLUTION: An image evaluation apparatus 40 according to the present invention evaluates whether or not a captured image 461 of a specific part of a work machine 6 is suitable for a measurement image for use in measurement of an abrasion amount. A feature point extracting unit 43 has carried out machine learning of a learning image including a contour of the specific part and a plurality of feature points for use in specifying the contour of the specific part included in the learning image as teacher data. A reference value storing unit 45 stores a reference value X used as a reference for evaluation of the captured image 461. An evaluation value calculating unit 47, based on the feature points of the captured image 461 and the reference value X thereof, calculates an evaluation value for use in evaluating whether or not the captured image 461 is suitable for the measurement image. An image determining unit 48 determines, in response to the calculated evaluation value, whether or not the captured image 461 can be used as the measurement image.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image evaluation device.

Due to continuous operation of the work machine, parts constituting the work machine may wear out. In order to appropriately manage the state of wear of a part to be worn, the amount of wear of the part may be measured. As a method of measuring the amount of wear, an imaging device such as a camera is used to image the part to be worn, the contour of the part is specified from the captured image of the part, and the contour of the part before wear is compared. There is a method of measuring the amount of wear of the part concerned.

In the above measurement method, it is important that the captured image of the worn portion is an image suitable for measuring the amount of wear. For example, in crawler hydraulic excavators, the sprockets of the undercarriage may wear. When measuring the amount of wear of a sprocket, it is difficult to ensure the accuracy of the measurement of the amount of wear unless the image is taken directly from the front of the sprocket with an imaging device facing the front of the sprocket.

For example, the technique disclosed in Patent Document 1 is known as a technique for acquiring an image captured from a specific direction, such as a straight-on image. Japanese Patent Application Laid-Open No. 2002-200000 discloses an imaging device for imaging an object plane, a plane measurement device for measuring the orientation of the object plane with respect to the imaging device, and an oblique photographing of an image taken by the imaging device based on the measured orientation of the object plane. and image distortion correcting means for correcting distortion caused by the image processing apparatus.

JP-A-2000-307947

However, when the technique of Patent Document 1 is adopted for measuring the amount of wear, a special means called plane measurement means is prepared in advance before imaging, and the plane measurement means is used at the time of imaging to image the abraded portion. There is a need. In this case, it is not realistic to employ the technique of Patent Document 1 to measure the amount of wear, since special means, plane measurement means, must be distributed to maintenance personnel and users of the work machine. In particular, even if the technique disclosed in Patent Document 1 is adopted, it is difficult to easily image a worn portion with a smartphone or the like and measure the amount of wear of the portion.

In the technique of Patent Document 1, since it is necessary to use a special means called plane measurement means at the time of imaging to image the worn part, patent The technique of Document 1 cannot be applied.

For this reason, there is a demand for a technique that can appropriately and easily evaluate whether a captured image of a worn portion of a working machine is suitable for measuring the amount of wear.

In view of the above circumstances, the present invention appropriately and easily evaluates whether a captured image of a worn portion of a working machine is suitable for measuring the amount of wear, thereby simplifying and improving the efficiency of measuring the amount of wear. The purpose is to

In order to solve the above-described problems, an image evaluation apparatus of the present invention provides image evaluation for evaluating whether a captured image of a specific portion of a working machine is suitable as a measurement image used to measure the wear amount of the specific portion. A device, wherein a learning image including the contour of the specific part and a plurality of feature points for specifying the contour of the specific part included in the learning image are machine-learned as teacher data. , a feature point extraction unit that extracts feature points that identify the contour of the specific part included in the captured image; a reference value storage unit for storing a reference value used as a reference for evaluating the captured image; and the feature points in the captured image extracted by the feature point extraction unit and stored in the reference value storage unit. an evaluation value calculation unit that calculates an evaluation value for evaluating whether the captured image is suitable for the measurement image based on the reference value that has been obtained; and an image determination unit that determines whether the captured image can be used as the measurement image.

According to the present invention, it is possible to appropriately and easily evaluate whether an image of a portion of a working machine that wears out is suitable for measuring the amount of wear, thereby simplifying and improving the efficiency of the work of measuring the amount of wear. can be done.

1 is a diagram schematically showing the configuration of a wear amount measurement system including the image evaluation device of Embodiment 1. FIG. The figure which shows the structure of the working machine shown in FIG. FIG. 2 is a block diagram showing the functional configuration of the image evaluation apparatus shown in FIG. 1; FIG. 4 is a block diagram showing the detailed configuration of a reference value calculation unit and an evaluation value calculation unit shown in FIG. 3; The figure which shows typically the front image suitable for the measurement of the wear amount of a sprocket. FIG. 4 is a diagram for explaining feature points that specify the outline of a sprocket; 4 is a flowchart showing the operation of the image evaluation device in the reference value learning mode; 4 is a flowchart showing the operation of the image evaluation device in image extraction mode; FIG. 8 is a block diagram showing the detailed configuration of a reference value calculator and an evaluation value calculator included in the image evaluation apparatus of the second embodiment; The figure which shows typically the structure of the front-end|tip part of a bucket. The figure which shows typically the perspective image suitable for the measurement of the wear amount of a tooth. FIG. 4 is a diagram illustrating feature points that specify contours of a tooth and an adapter; FIG. 11 is a block diagram showing the functional configuration of an image evaluation apparatus according to Embodiment 3; 4 is a flowchart showing the operation of the image evaluation device in automatic imaging mode;

An embodiment of the present invention will be described below with reference to the drawings. Components denoted by the same reference numerals in each embodiment have the same function in each embodiment unless otherwise specified, and the description thereof will be omitted.

Using current AI technology, it is possible to automatically measure the amount of wear from a captured image 461 that includes the parts of the work machine 6 that wear, such as the sprocket 63 (see FIG. 2) and tooth 693 (see FIG. 10). . However, the measurement of the amount of wear may require high accuracy in units of mm. An image used for measuring the amount of wear (hereinafter also referred to as a “measurement image”) to ensure highly accurate measurement differs for each part of the work machine 6 .

For example, the image for measurement of the sprocket 63 is a straight front image captured with the imaging device 31 directly facing the front 633 of the sprocket 63 (see FIG. 5). The sprocket 63 has a measurement gauge image that fits tightly on the teeth 631 and roots 632 (see FIG. 5) of the sprocket 63 before wear. The amount of wear of the sprocket 63 can be easily measured by measuring the size of the gap between the tooth 631 and tooth root 632 included in the captured image 461 of the sprocket 63 and the measurement gauge image by image processing. In order to measure the wear amount of the sprocket 63 from the two-dimensional captured image 461 and the measurement gauge image with high accuracy, it is important to fit the measurement gauge image to the straight-on image, which hardly captures the depth. Therefore, the image for measurement of the sprocket 63 is a straight-on image captured with the imaging device 31 directly facing the front 633 of the sprocket 63 .

On the other hand, the image for measurement of the tooth 693 is captured with the image pickup device 31 directed at an appropriate oblique direction to the front surface 694 of the tooth 693 so that the front surface 694 and the side surface 695 of the adapter 692 and the tooth 693, which will be described later, are captured. 11 (see FIG. 11). Since the tooth 693 does not have a straight front reference plane, it is difficult to evaluate how close the captured image 461 of the tooth 693 is to the straight front image. Rather, the information on side 695 of tooth 693 should be used. This is because if the boundary portion 696 between the tooth 693 and the adapter 692 is used as a measurement reference, even if the bucket 69 is rotating, there is no problem in measuring the amount of wear. Through repeated experiments, the inventors have succeeded in proving that by using the boundary portion 696 as a measurement reference, the difference between the image-based wear amount measurement and the actual measurement can be minimized. Therefore, the image for measurement of the tooth 693 is a perspective image taken from an appropriately oblique direction with respect to the front surface 694 of the tooth 693 so that the front surface 694 and side surface 695 of the adapter 692 and tooth 693 are captured.

The image evaluation device 40 is a device that evaluates whether or not a captured image 461 including a worn portion of the work machine 6 is suitable as a measurement image. The image evaluation device 40 can perform processing suitable for each part included in the captured image 461 to be evaluated, and evaluate the captured images 461 of various parts.

[Embodiment 1]
An image evaluation device 40 according to the first embodiment will be described with reference to FIGS. 1 to 8. FIG.

In the first embodiment, the image evaluation device 40 that evaluates whether the captured image 461 of the sprocket 63 is suitable for the measurement image when the sprocket 63 is the part of the work machine 6 that wears will be described.

FIG. 1 is a diagram schematically showing the configuration of a wear amount measurement system 1 including an image evaluation device 40 of Embodiment 1. As shown in FIG.

The wear amount measurement system 1 is a system that measures the wear amount of a portion of the work machine 6 that wears from an image. The wear amount measurement system 1 is a system in which a server device 2 that manages the wear amount measurement system 1 and a plurality of communication terminals 3 each having an imaging device 31 are connected via a network so as to be capable of wireless communication. . In the wear amount measurement system 1 of the present embodiment, the image evaluation device 40 is provided in the communication terminal 3 , but all or part of the functions of the image evaluation device 40 may be provided in the server device 2 .

The communication terminal 3 is a mobile communication terminal such as a smart phone or a tablet PC. The communication terminal 3 includes an imaging device 31 capable of capturing moving images or still images, a display device 32 and an input device 33 that constitute a touch panel display, and an image evaluation device 40 .

The image evaluation device 40 includes an arithmetic processing device 401 configured by a CPU and the like, and a storage device 402 configured by a ROM, a RAM and the like. The storage device 402 is provided with data storage areas such as a teacher data storage unit 41, a reference value storage unit 45, and a captured image storage unit 46, which will be described later. The storage device 402 stores various programs for realizing the functions of a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, an evaluation value calculation unit 47, an image determination unit 48, an image extraction unit 49, etc., which will be described later. A program storage area is provided. The arithmetic processing unit 401 executes various programs stored in the program storage area of the storage device 402 using various data stored in the data storage area of the storage device 402, thereby performing various functions of the image evaluation device 40. can be realized.

FIG. 2 is a diagram showing the configuration of the work machine 6 shown in FIG. 1. As shown in FIG.

The working machine 6 is not particularly limited, but may be, for example, a construction machine such as a hydraulic excavator. In this embodiment, the work machine 6 is described as a crawler hydraulic excavator.

The work machine 6 includes a lower traveling body 61 , an upper revolving body 65 attached to the lower traveling body 61 via a revolving device 651 so as to be able to turn, and an articulated front part attached to the front part of the upper revolving body 65 . and a working machine 66 .

The lower traveling body 61 includes an idler 62 provided at the front end of the lower traveling body 61, a sprocket 63 provided at the rear end of the lower traveling body 61 and driven by a hydraulic motor, and the idler 62 and the sprocket 63. A strip-shaped crawler 64 is provided. The sprocket 63 may wear out due to friction with the crawler 64 or friction with sand or the like that has entered between the sprocket 63 and the crawler 64 .

The upper revolving body 65 includes a cab 652 provided in the front part of the upper revolving body 65 and a machine room 653 provided in the rear part of the upper revolving body 65 and accommodating the driving device of the work machine 6 .

The front work machine 66 includes a boom 67 rotatably supported on the front part of the upper swing body 65 , an arm 68 rotatably supported on the tip of the boom 67 , and the tip of the arm 68 . and a bucket 69 rotatably supported with respect to the part. The boom 67, the arm 68 and the bucket 69 are connected to the boom cylinder 671, the arm cylinder 681 and the bucket cylinder 691 via links, respectively, and rotate by extending and contracting these hydraulic cylinders. Teeth 693 are attached to the tip portion 69a of the bucket 69 at intervals in the width direction. The tooth 693 may wear due to friction with excavated soil or the like.

FIG. 3 is a block diagram showing the functional configuration of the image evaluation device 40 shown in FIG. 1. As shown in FIG. FIG. 4 is a block diagram showing detailed configurations of the reference value calculator 44 and the evaluation value calculator 47 shown in FIG. FIG. 5 is a diagram schematically showing a front image G1 suitable for measuring the wear amount of the sprocket 63. As shown in FIG. FIG. 6 is a diagram for explaining feature points that specify the contour of the sprocket 63. As shown in FIG.

The image evaluation device 40 extracts feature points that specify the outline of a captured image 461 including a portion to be worn such as the sprocket 63, and evaluates how well the captured image 461 is suitable as a measurement image. Calculate the value. The image evaluation device 40 determines whether or not the captured image 461 can be used as a measurement image according to the calculated evaluation value. The feature point extraction function is constructed as a model generated by machine learning. The captured image 461 to be evaluated has already been captured and stored in advance in the captured image storage unit 46 . The image evaluation device 40 can evaluate whether the plurality of captured images 461 stored in the captured image storage unit 46 are suitable for the measurement image, and extract the measurement image from the plurality of captured images 461. .

The image evaluation device 40 includes a teacher data storage unit 41, a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, a reference value storage unit 45, a captured image storage unit 46, and an evaluation value calculation unit. 47 , an image determination unit 48 , and an image extraction unit 49 .

The functions provided in the image evaluation device 40 are a function used or executed in a machine learning for generating a feature point extraction function and a learning stage for calculating a reference value X to be described later, and a function to determine whether the captured image 461 is suitable for a measurement image. It can be roughly divided into functions that are used or executed in the utilization stage, which evaluates whether the

First, the functions used or executed during the learning stage of the image evaluation device 40 will be described.
The training data storage unit 41 stores a learning image 411 showing the contour of a worn portion of the work machine 6, and feature points indicating the coordinates of a plurality of feature points included in the learning image 411 for specifying the contour of the portion. data 412 are associated with each other and stored.

Using the learning image 411 and the feature point data 412 stored in the teacher data storage unit 41 as teacher data, the learning unit 42 machine-learns feature point extraction to generate the feature point extraction unit 43 . A machine learning algorithm that the learning unit 42 has is not particularly limited, and may be, for example, a random forest. The machine-learned feature point extracting unit 43 identifies the contour of the part included in the captured image 461 when evaluating whether the captured image 461 stored in the captured image storage unit 46 is suitable for the measurement image. Extract feature points.

The learning image 411 is an image including contours of the front face 633 and the back face 634 of the root 632 forming the outer edge of the sprocket 63 . The learning image 411 is acquired by imaging the tooth 631 and the tooth bottom 632 of the sprocket 63 from a plurality of imaging directions including directions suitable for measuring the amount of wear. The learning image 411 may be an image captured by the imaging device 31 of the communication terminal 3 or may be an image captured by another imaging device different from the imaging device 31 . The direction suitable for measuring the wear amount is, for example, an imaging direction in which the optical axis of the imaging device 31 is orthogonal to the plane including the front surface 633 of the sprocket 63 . An image captured from a direction suitable for measuring the amount of wear is a front image G1 as shown in FIG. An image captured from an imaging direction other than the direction suitable for measuring the amount of wear is an image G2 as shown in FIG. An image G2 shown in FIG. 6 is an image including contours of the front surface 633 and the rear surface 634 of the root 632 of the sprocket 63 .

The front surface 633 is a surface facing outward in the left-right direction of the working machine 6 , the rear surface 634 is a surface facing inward in the left-right direction of the working machine 6 , and the side surface 635 is continuous with the front surface 633 and the rear surface 634 and continues to the sprocket 63 . is a surface along the circumferential direction of

As shown in FIG. 6, in the image including each contour of the tooth bottom 632 of the sprocket 63, there is a front circle F that approximates the contour of the front face 633 of the tooth bottom 632 and a back face circle F that approximates the contour of the back face 634 of the tooth bottom 632. B can be drawn. In the image including each contour of the tooth bottom 632 of the sprocket 63, the fact that the center Cf of the front circle F and the center Cb of the back circle B coincide is almost equivalent to the image being the front image G1. In an image in which the outline of the back surface 634 is completely hidden, the center Cb of the back surface 634 is unknown. When acquiring an image including each contour of the tooth bottom 632 of the sprocket 63, the center Cf of the front circle F and the center Cb of the back circle B are obtained from the situation where the center Cf of the front circle F and the center Cb of the back circle B are captured. When the posture of the imaging device is changed so that the two overlap each other, the two approach each other. Before long, the center Cf of the front circle F and the center Cb of the back circle B coincide with each other, and the center Cb of the back circle B disappears. An image captured at the moment when the center Cb of the back circle B disappears can be regarded as the front image G1.

The radius Rb of the back circle B is smaller than the radius Rf of the front circle F according to the principle of perspective. Therefore, it is impossible to overlap the contours of the front face 633 and the back face 634 of the root 632 . In order to obtain the straight front image G1 without using the method of drawing the front circle F and the back circle B, the contour of the front face 633 and the contour of the back face 634 of the tooth base 632 are observed simultaneously and the balance between the two is considered. However, you have to take an image intuitively. This makes it very difficult to obtain the front image G1. However, by adopting the method of drawing the front circle F and the back circle B, a more accurate front circle can be obtained by a simple operation of overlapping the center Cf of the front circle F and the center Cb of the back circle B as described above. Image G1 can be acquired.

For this reason, the image evaluation apparatus 40 of the present embodiment uses the coordinates of points on each contour of the tooth bottom 632 of the sprocket 63 that can draw the front circle F and the back circle B as the feature point data 412. do. P1 to P4 in FIG. 6 are characteristic points that specify the profile of the front surface 633 of the tooth bottom 632 of the sprocket 63 . P5 to P8 in FIG. 6 are feature points that specify the contour of the back surface 634 of the tooth root 632 of the sprocket 63 .

The reference value calculation unit 44 calculates a reference value X that serves as a reference for evaluating whether the captured image 461 stored in the captured image storage unit 46 is suitable for the measurement image. The reference value calculator 44 includes a center radius calculator 441 and a ratio calculator 442, as shown in FIG.

As described above, in the image including each contour of the tooth bottom 632 of the sprocket 63, the fact that the center Cf of the front circle F and the center Cb of the back circle B coincide means that the image is the front image G1, Almost equivalent. However, since the radius Rf of the front circle F and the radius Rb of the back circle B are different, precisely, the points to be matched are points closer to the tooth root 632 than the center Cf and points closer to the tooth root than the center Cb. It is the point on the side closer to the bottom 632 . This point to be matched is a reference point (hereinafter also referred to as a “reference point”) for evaluating whether the captured image 461 to be evaluated is suitable for the measurement image. The reference point of the front circle F and the reference point of the back circle B are respectively set at points separated from the center Cf and the center Cb toward the root 632 by a ratio of the radius Rf and the radius Rb. Whether it should be done depends on the width and thickness of the root 632, and therefore depends on the type of the sprocket 63. In the image evaluation device 40 of the present embodiment, before evaluating the captured image 461, the reference value calculation unit 44 calculates this ratio as the reference value X, and stores it in the reference value storage unit 45 for each type of sprocket 63. . In the image containing each contour of the tooth root 632 of the sprocket 63, matching the reference point of the front circle F and the reference point of the back circle B is completely equivalent to the image being the front image G1. .

For this reason, the image evaluation apparatus 40 of the present embodiment uses a point on the side closer to the tooth bottom 632 than the center Cf of the front circle F as a reference point of the front circle F, and the tooth A point on the side closer to the bottom 632 is set as a reference point of the back circle B. FIG. Then, the image evaluation device 40 of this embodiment regards an image in which the reference point of the front circle F and the reference point of the back circle B coincide as the front image G1.

When calculating the reference value X, the reference value calculator 44 uses feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear. The learning image 411 used for calculating the reference value X is, for example, the learning image 411 stored in the teacher data storage unit 41, or the image 411 suitable for measuring the wear amount by the imaging device 31 for calculating the reference value X. It may be a learning image 411 that is newly captured from a different direction. The reference value calculation unit 44 inputs these learning images 411 to the feature point extraction unit 43 that has undergone machine learning. The feature point extraction unit 43 extracts feature points that specify the contour of the front face 633 of the root 632 and feature points that specify the contour of the back face 634 . The feature point extraction unit 43 identifies the front circle F and the back circle B from the extracted feature points.

The center radius calculator 441 calculates the center Cf and radius Rf of the front circle F identified by the feature point extraction unit 43 and the center Cb and radius Rb of the back circle B identified by the feature point extraction unit 43 . The ratio calculator 442 of the reference value calculator 44 calculates a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B. FIG. The ratio calculator 442 calculates X that is on the calculated straight line and that satisfies the following equation (1).
Distance from the center Cf of the front circle F (radius Rf×X)=Distance from the center Cb of the back circle B (radius Rb×X) (1)

The reference value calculator 44 is a point on a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B, and is located on the side of the root 632 from the center Cf by a distance obtained by multiplying the radius Rf by X. Let the distant point be the reference point of the front circle F. The reference value calculator 44 is a point on a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B, and is located on the side of the tooth bottom 632 from the center Cb by a distance obtained by multiplying the radius Rb by X. Let the distant point be the reference point of the back circle B. The reference value calculation unit 44 stores the calculated X as the reference value X in the reference value storage unit 45 . At this time, the reference value storage unit 45 stores the reference value X calculated by the reference value calculation unit 44 in association with the part number that identifies the type of the sprocket 63 .

In this way, the reference value storage unit 45 stores the reference value X serving as a reference for evaluating the captured image 461, which is calculated from the feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear. Remember.

Next, functions that are used or executed during the utilization stage of the image evaluation device 40 will be described.
The captured image storage unit 46 stores a captured image 461 . The captured image 461 stored in the captured image storage unit 46 is a moving image or a plurality of still images captured in advance. A moving image is composed of a plurality of frame images captured at a predetermined frame rate. When evaluating the moving image stored in the captured image storage unit 46, the image evaluation device 40 can evaluate each of the plurality of frame images that make up the moving image.

When evaluating whether or not the captured image 461 stored in the captured image storage unit 46 is suitable for the measurement image, the feature point extraction unit 43 extracts the contour of a specific part included in the captured image 461. Extract multiple feature points. Specifically, the captured image 461 stored in the captured image storage unit 46 is input to the feature point extraction unit 43 . The feature point extraction unit 43 extracts feature points that specify the contour of the front face 633 and feature points that specify the contour of the back face 634 in the root 632 of the sprocket 63 included in the captured image 461 . The feature point extraction unit 43 identifies the front circle F and the back circle B from the extracted feature points.

Based on the feature points in the captured image 461 extracted by the feature point extraction unit 43 and the reference value X stored in the reference value storage unit 45, the evaluation value calculation unit 47 determines that the captured image 461 is suitable for the measurement image. Calculate an evaluation value that evaluates whether the The evaluation value calculator 47 includes a reference value average calculator 471, a center radius calculator 472, a difference length calculator 473, and a difference ratio calculator 474, as shown in FIG.

The reference value average calculation unit 471 calculates an average value Xm of a plurality of reference values X stored in the reference value storage unit 45 in association with the part number of the sprocket 63 . The center radius calculator 472 calculates the center Cf and radius Rf of the front circle F identified by the feature point extraction unit 43 and the center Cb and radius Rb of the back circle B identified by the feature point extraction unit 43 .

The difference length calculator 473 calculates a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B. FIG. The difference length calculator 473 calculates a difference length D represented by the following equation (2) on the calculated straight line.
Differential length D = {point separated by distance (radius Rf × Xm) from center Cf of front circle F}−{point separated by distance (radius Rb × X) from center Cb of back circle B} (2 )

The difference ratio calculator 474 calculates the difference ratio Δ represented by the following equation (3).
Difference ratio Δ=difference length D/(radius Rf of front circle F+radius Rb of back circle B) (3)

The evaluation value calculation unit 47 uses the difference ratio Δ calculated by the difference ratio calculation unit 474 as an evaluation value for evaluating whether the captured image 461 is suitable for the measurement image. The evaluation value calculation unit 47 inputs the difference ratio Δ calculated as the evaluation value to the image determination unit 48 .

The image determination section 48 determines whether or not the captured image 461 can be used as a measurement image according to the evaluation value calculated by the evaluation value calculation section 47 . Specifically, the image determination unit 48 compares the difference ratio Δ calculated as the evaluation value with the threshold T. The threshold T is a threshold for determining whether or not the captured image 461 can be used as a measurement image, and is predetermined for each type of sprocket 63 . The threshold T is stored in advance in a threshold storage unit (not shown) of the image evaluation device 40 in association with the part number of the sprocket 63 . If the difference ratio Δ is smaller than the threshold value T, the image determination unit 48 determines that the captured image 461 can be used as a measurement image. If the difference ratio Δ is equal to or greater than the threshold value T, the image determination unit 48 determines that the captured image 461 cannot be used as a measurement image. The image determination section 48 inputs the determination result to the image extraction section 49 .

The image extraction unit 49 extracts the measurement image from the captured image 461 stored in the captured image storage unit 46 according to the determination result of the image determination unit 48 . Specifically, the image extracting unit 49 extracts the image for measurement from a plurality of frame images or a plurality of still images constituting the moving image stored in the captured image storage unit 46 according to the determination result of the image determining unit 48 . A captured image 461 corresponding to the image is extracted. When the determination result of the image determining unit 48 indicates that a plurality of captured images 461 correspond to the images for measurement, the image extracting unit 49 may extract all the captured images 461 corresponding to the images for measurement. Only the captured image 461 with the smallest ratio Δ may be extracted, or only a predetermined number of captured images 461 (for example, top five captured images 461) with the smallest difference ratio Δ may be extracted. . The image extraction unit 49 outputs the captured image 461 corresponding to the extracted measurement image to the display device 32 and causes the display device 32 to display the image.

After determining whether or not the captured image 461 stored in the captured image storage unit 46 can be used as a measurement image, the image evaluation device 40 associates the determination result with the captured image 461 and captures the image. It may be stored in the image storage unit 46 or output to the display device 32 . In this case, the image evaluation device 40 does not have to extract the captured image 461 corresponding to the measurement image from the captured images 461 stored in the captured image storage unit 46 . In this case, the image evaluation device 40 may omit the image extractor 49 .

In addition, the image evaluation device 40 does not determine whether or not all of the captured images 461 stored in the captured image storage unit 46 can be used as images for measurement, and measures the captured image 461 with the smallest difference ratio Δ. It is also possible to simply extract it as an image for use. In this case, the image evaluation device 40 may omit the image determination section 48 .

Next, the operation of the image evaluation device 40 will be described. The image evaluation device 40 has a machine learning mode for generating a feature point extraction function, a learning mode for pre-calculating and storing the reference value X, and a captured image 461 stored in the captured image storage unit 46. It can operate with an image extraction mode that extracts the measurement image.

FIG. 7 is a flow chart showing the operation of the image evaluation device 40 in the reference value X learning mode.

In step S<b>701 , the image evaluation device 40 accepts the start of the learning mode for the reference value X according to the user's input operation on the input device 33 .

In step S<b>702 , the image evaluation device 40 receives the part number of the sprocket 63 according to the user's input operation on the input device 33 .

In step S<b>703 , the image evaluation device 40 determines whether or not deletion (initialization) of the reference value X has been accepted in accordance with the user's input operation on the input device 33 . If the image evaluation apparatus 40 has not received the deletion of the reference value X, the process proceeds to step S705. When the image evaluation device 40 accepts deletion of the reference value X, the process proceeds to step S704.

In step S704, the image evaluation device 40 erases the reference value X. Specifically, the image evaluation device 40 erases the reference value X stored in the reference value storage unit 45 or the reference value X for which the error message was output in step S713.

In step S<b>705 , the image evaluation device 40 determines whether or not termination of the learning mode for the reference value X has been received according to the user's input operation on the input device 33 . When the image evaluation device 40 accepts the end of the learning mode for the reference value X, it ends the processing shown in FIG. If the image evaluation apparatus 40 has not received the end of the learning mode for the reference value X, the process proceeds to step S706.

In step S706, the image evaluation device 40 reads the learning image 411 used for calculating the reference value X. FIG.

In step S<b>707 , the image evaluation device 40 extracts feature points specifying the contour of the front face 633 of the root 632 of the sprocket 63 and feature points specifying the contour of the back face 634 . The image evaluation device 40 identifies the front circle F and the back circle B from the extracted feature points.

In step S708, the image evaluation device 40 calculates the center Cf and radius Rf of the front circle F and the center Cb and radius Rb of the back circle B. FIG.

In step S709, the image evaluation device 40 calculates the reference value X. FIG. Specifically, the image evaluation device 40 calculates a reference value X that is on a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B and that satisfies the above equation (1).

In step S<b>710 , the image evaluation device 40 determines whether or not the reference value X is already stored in the reference value storage unit 45 . If the reference value X is not already stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S713. When the reference value X is already stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S711.

In step S711, the image evaluation device 40 determines whether the difference between the reference value X calculated this time and the reference value X already stored in the reference value storage unit 45 is within the allowable range. If the difference between the reference value X calculated this time and the reference value X already stored in the reference value storage unit 45 is within the allowable range, the image evaluation device 40 proceeds to step S713. If the difference between the reference value X calculated this time and the reference value X already stored in the reference value storage unit 45 is not within the allowable range, the image evaluation device 40 proceeds to step S712. Whether or not it is within the allowable range is based on a preset and stored threshold value. Any threshold can be set.

In step S712, the image evaluation device 40 outputs an error message to the display device 32, and proceeds to step S703.

In step S<b>713 , the image evaluation device 40 stores the reference value X calculated this time as a new reference value X in the reference value storage unit 45 .

In this manner, the image evaluation device 40 can calculate and store in advance the reference value X that serves as a reference for evaluating the captured image 461 .

FIG. 8 is a flow chart showing the operation of the image evaluation device 40 in the image extraction mode.

In step S<b>801 , the image evaluation device 40 accepts the start of the image extraction mode according to the user's input operation on the input device 33 .

In step S<b>802 , the image evaluation device 40 receives the part number of the sprocket 63 according to the user's input operation on the input device 33 .

In step S<b>803 , the image evaluation device 40 determines whether or not the reference value X is stored in the reference value storage unit 45 . When the reference value X is stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S805. If the reference value X is not stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S804.

In step S804, the image evaluation device 40 outputs a message to the display device 32 to the effect that the reference value X is not stored. The image evaluation device 40 ends this process shown in FIG.

In step S<b>805 , the image evaluation device 40 calculates the average value Xm of the reference values X stored in the reference value storage unit 45 .

In step S<b>806 , the image evaluation device 40 reads the captured image 461 stored in the captured image storage unit 46 .

In step S<b>807 , the image evaluation device 40 extracts feature points specifying the contour of the front face 633 of the root 632 of the sprocket 63 and feature points specifying the contour of the back face 634 . The image evaluation device 40 identifies the front circle F and the back circle B from the extracted feature points.

In step S808, the image evaluation device 40 calculates the center Cf and radius Rf of the front circle F and the center Cb and radius Rb of the back circle B. FIG.

In step S809, the image evaluation device 40 calculates the difference length D represented by the above equation (2) on a straight line passing through the center Cf of the front circle F and the center Cb of the back circle B.

In step S810, the image evaluation device 40 calculates the difference ratio Δ represented by Equation (3) above.

In step S811, the image evaluation device 40 determines whether or not the calculated difference ratio Δ is smaller than a predetermined threshold value T, thereby determining whether or not the captured image 461 read this time can be used as a measurement image. judge.

In step S812, the image evaluation device 40 determines whether or not the determination result of step S811 indicates that the image can be used as the image for measurement. If the determination result does not indicate that the image can be used as the image for measurement, the image evaluation device 40 proceeds to step S814. When the determination result indicates that the image can be used as the measurement image, the image evaluation device 40 proceeds to step S813.

In step S813, the image evaluation device 40 extracts the captured image 461 read this time as a measurement image.

In step S814, the image evaluation device 40 determines whether or not all the captured images 461 stored in the captured image storage unit 46 have been evaluated. If the image evaluation device 40 has not evaluated all the captured images 461 stored in the captured image storage unit 46, the process proceeds to step S806. When the image evaluation device 40 evaluates all the captured images 461 stored in the captured image storage unit 46, the process proceeds to step S815.

In step S815, the image evaluation device 40 determines whether or not the image for measurement has been extracted. When the image for measurement is extracted, the image evaluation device 40 proceeds to step S817. When the image for measurement is not extracted, the image evaluation device 40 proceeds to step S816.

In step S816, the image evaluation device 40 outputs to the display device 32 a message to the effect that the image for measurement has not been extracted. The image evaluation device 40 ends this process shown in FIG.

In step S<b>817 , the image evaluation device 40 outputs the extracted measurement image to the display device 32 . The image evaluation device 40 ends this process shown in FIG.

As described above, the image evaluation device 40 of the first embodiment determines whether the captured image 461 of a specific portion of the work machine 6 (a portion to be worn, such as the sprocket 63) is suitable as a measurement image used to measure the amount of wear. It is an image evaluation device that evaluates the The feature point extraction unit 43 obtains a learning image 411 including the contour of the specific part, and feature point data indicating the coordinates of a plurality of feature points for specifying the contour of the specific part included in the learning image 411. 412 is machine-learned as teaching data, and extracts feature points that specify the contour of the specific part included in the captured image 461 . The reference value storage unit 45 stores a reference value X serving as a reference for evaluating the captured image 461, which is calculated from the feature points in the learning image 411 captured from a direction suitable for measuring the amount of wear. Based on the feature points in the captured image 461 extracted by the feature point extraction unit 43 and the reference value X stored in the reference value storage unit 45, the evaluation value calculation unit 47 determines that the captured image 461 is suitable for the measurement image. Calculate an evaluation value that evaluates whether the The image determination section 48 determines whether or not the captured image 461 can be used as a measurement image according to the evaluation value calculated by the evaluation value calculation section 47 .

As a result, the image evaluation device 40 of the first embodiment can obtain the captured image 461 of the specific portion of the work machine 6 without using special means such as the plane measurement means disclosed in Patent Document 1. It is possible to highly accurately evaluate whether the image is suitable for measuring the amount of wear. Furthermore, the image evaluation device 40 of the first embodiment can highly accurately evaluate whether the captured image 461 is suitable for measuring the amount of wear regardless of whether or not the captured image 461 has been captured. Therefore, the image evaluation apparatus 40 of the first embodiment can appropriately determine whether the captured image 461 is an image suitable for measuring the amount of wear, even if the captured image 461 is a casual image captured by a user using a smartphone or the like of a portion to be worn. and can be easily evaluated. By introducing the image evaluation device 40, the user can estimate the amount of wear from the two-dimensional captured image 461 without using expensive means such as a three-dimensional shape measuring machine for acquiring a three-dimensional point cloud of the target site. measurements can be made. The image evaluation device 40 of the first embodiment can simplify and improve the efficiency of the work of measuring the amount of wear.

Furthermore, the image evaluation device 40 of Embodiment 1 includes an image extraction unit 49 that extracts the measurement image. The captured image 461 to be evaluated in the first embodiment is a moving image or a plurality of still images captured in advance and stored in the captured image storage unit 46 . The image extraction unit 49 extracts measurement images from among the plurality of frame images or the plurality of still images forming the moving image stored in the captured image storage unit 46 according to the determination result of the image determination unit 48 .

As a result, the image evaluation apparatus 40 of the first embodiment can automatically extract images suitable for measuring the amount of wear from already-captured moving image collections and still image collections. In particular, the image evaluation apparatus 40 of Embodiment 1 can automatically extract images suitable for measuring the amount of wear even from a collection of moving images and still images taken without any purpose. Therefore, the image evaluation apparatus 40 of Embodiment 1 can easily acquire an image suitable for measuring the amount of wear, and can further simplify and improve the efficiency of the operation of measuring the amount of wear.

In the wear amount measurement system 1 described above, the communication terminal 3 has all the functions of the image evaluation device 40, but the server device 2 may have all or part of the functions of the image evaluation device 40. . For example, the server device 2 may include the teacher data storage unit 41 and the learning unit 42, and the communication terminal 3 may include the feature point extracting unit 43 which has undergone machine learning. Alternatively, the server device 2 may also include a machine-learned feature point extraction unit 43 , and the communication terminal 3 may receive the extraction result of the feature point extraction unit 43 transmitted from the server device 2 . The server device 2 also includes a teacher data storage unit 41, a learning unit 42, a feature point extraction unit 43, a reference value calculation unit 44, a reference value storage unit 45, an evaluation value calculation unit 47, an image determination unit 48, and an image extraction unit 49. , and the communication terminal 3 may include only the captured image storage unit 46 . The server device 2 may also include the captured image storage unit 46 , and the communication terminal 3 may receive the extraction result of the image extraction unit 49 transmitted from the server device 2 and output it to the display device 32 .

[Embodiment 2]
The image evaluation device 40 of the second embodiment will be described with reference to FIGS. 9 to 12. FIG. In the image evaluation apparatus 40 of the second embodiment, descriptions of the same configurations and operations as those of the first embodiment are omitted.

In the second embodiment, an image evaluation device 40 that evaluates whether the picked-up image 461 of the tooth 693 is suitable for the image for measurement when the tooth 693 is the part to be worn of the work machine 6 will be described.

FIG. 9 is a block diagram showing the detailed configuration of the reference value calculator 44 and the evaluation value calculator 47 included in the image evaluation device 40 of the second embodiment. FIG. 10 is a diagram schematically showing the configuration of the tip portion 69a of the bucket 69. As shown in FIG. FIG. 11 is a diagram schematically showing a perspective image G3 suitable for measuring the wear amount of the tooth 693. As shown in FIG. FIG. 12 is a diagram illustrating feature points that specify the contours of tooth 693 and adapter 692 . 11 and 12 indicate the contour of the tooth 693 before wear.

As shown in FIG. 10, the tooth 693 is attached to an adapter 692 which is provided at the tip portion 69a of the bucket 69 at intervals in the width direction. The adapter 692 is a part that is less prone to wear. In this embodiment, the length and width of adapter 692 remain unchanged. A front surface 694 of the adapter 692 and tooth 693 shown in FIG. 11 is a surface that is continuous with the inner surface of the bucket 69 . A side surface 695 of the adapter 692 and the tooth 693 shown in FIG. 11 is a surface continuous with the rear surface continuous with the outer surface of the bucket 69 and the front surface 694, and is the surface along the dump/crowd direction of the bucket 69 .

The wear amount (wear rate) of the tooth 693 can be measured as follows. That is, as shown in FIG. 11, the image length of the worn tooth 693 is Lt, and the image length of the adapter 692 is La. Also, the actual length of the adapter 692 is la (fixed value), and the image length of the tooth 693 before wear is Lt0. In this case, the actual length lt of the worn tooth 693 can be expressed by the following equation (4). The amount of wear (wear rate) of the tooth 693 can be expressed by the following equation (5).
Actual length lt of worn tooth 693=(Length of worn tooth 693 on image Lt)×(Length of adapter 692 on image La)/(Actual length la of adapter 692) . . . (4)
Wear amount (wear rate) of tooth 693=1-{(length Lt of worn tooth 693 on image)/(length Lt0 on image of tooth 693 before wear)} (5)

However, in an image similar to that taken from the front face 694 of the tooth 693, the area of the adapter 692 and the side face 695 of the tooth 693 is narrow. Therefore, in an image that is almost taken from the front face 694 of the tooth 693, the image length La of the adapter 692 and the image lengths Lt and Lt0 of the tooth 693 cannot be measured accurately. On the other hand, in an image taken from an excessively oblique direction with respect to the front face 694 of the tooth 693, the adapter 692 and the tooth 693 appearing on the front side of the image overlap with the adapter 692 and the tooth 693 appearing on the back side of the image. A part of the rear side adapter 692 and the side surface 695 of the tooth 693 are not captured. Therefore, in an image taken from an excessively oblique direction with respect to the front face 694 of the tooth 693, the length La on the image of the adapter 692 and the lengths Lt and Lt0 on the image of the tooth 693 cannot be measured accurately. . Therefore, an image captured from a direction suitable for measuring the amount of wear is an image captured from a direction in which the length La of the adapter 692 on the image and the lengths Lt and Lt0 on the image of the tooth 693 can be accurately measured. be. An image captured from a direction suitable for measuring the amount of wear is a perspective image G3 captured from a direction moderately oblique to the front face 694 of the tooth 693, as shown in FIG.

The perspective image G3 is an image including contours of the front face 694 and side faces 695 of the adapter 692 and contours of the front face 694 and side faces 695 of the tooth 693 . In the perspective image G3, as the length on the image, the ratio of the width Wf of the front face 694 to the width Ws of the side face 695 at the boundary 696 between the adapter 692 and the tooth 693 can be defined as a predetermined value (at least a predetermined value). can be defined as falling within a range). In the image including the contours of the front face 694 and side faces 695 of the adapter 692 and the contours of the front face 694 and side faces 695 of the tooth 693, the ratio between the width Wf of the front face 694 and the width Ws of the side faces 695 at the boundary 696 is a predetermined value. (at least within a predetermined range) is equivalent to the image being the perspective image G3.

For this reason, in the image evaluation apparatus 40 of the second embodiment, the coordinates of the vertices that specify the contours of the front face 694 and the side faces 695 of the adapter 692 and the coordinates of the vertices that specify the contours of the front face 694 and the side faces 695 of the tooth 693 and the coordinates of are set as feature point data 412 . An image G4 shown in FIG. 12 is an image including contours of the front face 694 and side faces 695 of the adapter 692 and contours of the front face 694 and side faces 695 of the tooth 693 . P11 to P14 in FIG. 12 are feature points that specify the contour of the front face 694 of the tooth 693. FIG. P12, P14, and P15 in FIG. 12 are characteristic points that specify the contour of the side surface 695 of the tooth 693. FIG. P13, P14, P16, P17 in FIG. 12 are feature points that identify the contour of the front face 694 of the adapter 692. FIG. P 14 , P 15 , P 17 , P 18 in FIG. 12 are feature points that identify the contour of side 695 of adapter 692 .

In the image evaluation apparatus 40 of the second embodiment, an image including contours of the front face 694 and side face 695 of the adapter 692 and contours of the front face 694 and side face 695 of the tooth 693 is used as the learning image 411 . The learning image 411 is obtained by imaging the front face 694 and side face 695 of the adapter 692 and the tooth 693, respectively, from a plurality of imaging directions including directions suitable for measuring the amount of wear.

In the image evaluation apparatus 40 of the second embodiment, the reference value X is the ratio of the width Wf of the front face 694 to the width Ws of the side face 695 at the boundary 696 between the adapter 692 and the tooth 693 .

Specifically, the reference value calculation unit 44 of the second embodiment inputs the learning image 411 to the feature point extraction unit 43 that has undergone machine learning. The feature point extraction unit 43 of the second embodiment specifies feature points that specify the contours of the front face 694 and side faces 695 of the adapter 692 included in the learning image 411 and the contours of the front face 694 and the side faces 695 of the tooth 693 . Extract feature points. The feature point extractor 43 identifies the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary 696 between the adapter 692 and the tooth 693 from the extracted feature points.

The reference value calculator 44 of the second embodiment includes a width ratio calculator 443 as shown in FIG. The width ratio calculator 443 calculates the ratio (Wf/Ws) between the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary 696 specified by the feature point extraction unit 43 . The width ratio calculator 443 stores the calculated ratio (Wf/Ws) as the reference value X in the reference value storage unit 45 . At this time, the reference value storage unit 45 stores the reference value X calculated by the reference value calculation unit 44 in association with the part number that identifies the type of tooth 693 .

When evaluating whether the captured image 461 stored in the captured image storage unit 46 is suitable for the measurement image, the feature point extraction unit 43 of the second embodiment extracts the front 694 and the side surface of the adapter 692 included in the captured image 461. The feature points that identify each contour of tooth 695 and the feature points that identify each contour of front face 694 and side face 695 of tooth 693 are extracted. The feature point extractor 43 identifies the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary 696 between the adapter 692 and the tooth 693 from the extracted feature points.

The evaluation value calculator 47 of the second embodiment includes a reference value average calculator 471, a width ratio calculator 475, and a width ratio difference calculator 476, as shown in FIG. The reference value average calculation unit 471 calculates the average value Xm of the plurality of reference values X stored in the reference value storage unit 45 in association with the part number of the tooth 693, as in the first embodiment. The width ratio calculator 475 calculates a ratio X (=Wf/Ws) between the width Wf of the front surface 694 and the width Ws of the side surface 695 at the boundary 696 specified by the feature point extraction unit 43 . The width ratio difference calculation unit 476 calculates a difference value |X−Xm| between the ratio X calculated by the width ratio calculation unit 475 and the average value Xm calculated by the reference value average calculation unit 471 .

The evaluation value calculation unit 47 of the second embodiment uses the difference value |X−Xm| The evaluation value calculation unit 47 inputs the difference value |X−Xm| calculated as the evaluation value to the image determination unit 48 .

As in the first embodiment, the image determination unit 48 of the second embodiment determines whether or not the captured image 461 can be used as a measurement image according to the evaluation value calculated by the evaluation value calculation unit 47 . As in the first embodiment, the image extraction unit 49 of the second embodiment extracts the measurement image from the captured image 461 stored in the captured image storage unit 46 according to the determination result of the image determination unit 48 .

As described above, the image evaluation apparatus 40 of the second embodiment can perform the following operations in the same manner as in the first embodiment, even when a specific portion of the work machine 6 does not have a straight front reference plane such as the tooth 693 . It is possible to appropriately and easily evaluate whether the captured image 461 is an image suitable for measuring the amount of wear. As in the first embodiment, the image evaluation device 40 of the second embodiment can automatically extract images suitable for measuring the amount of wear from captured moving image collections and still image collections.

[Embodiment 3]
The image evaluation device 40 of the third embodiment will be described with reference to FIGS. 13 and 14. FIG. In the image evaluation apparatus 40 of Embodiment 3, the description of the same configuration and operation as those of Embodiment 1 is omitted.
FIG. 13 is a block diagram showing the functional configuration of the image evaluation device 40 of Embodiment 3. As shown in FIG.

The imaging device 31 has a live view function for displaying an image of a subject on the display device 32 in real time during imaging. The image evaluation apparatus 40 of the third embodiment evaluates the captured image 461 acquired by the live view function of the imaging device 31 (hereinafter referred to as "live (also referred to as "view image 501") is to be evaluated. The image evaluation device 40 of the third embodiment is configured so that when the user uses the imaging device 31 to capture an image of a worn portion of the working machine 6, the imaging device 31 can capture a measurement image suitable for measuring the amount of wear. The position or orientation of the imaging device 31 is guided through the display device 32 .

The image evaluation device 40 of the third embodiment includes a live view image acquisition unit 50 instead of the captured image storage unit 46 and an image capture guide unit 51 instead of the image extraction unit 49, as shown in FIG.

The live-view image acquisition unit 50 acquires a live-view image 501 acquired by the live-view function of the imaging device 31 from the imaging device 31 and inputs it to the learned feature point extraction unit 43 . When the image determination unit 48 determines that the live view image 501 cannot be used as the measurement image, the imaging guidance unit 51 guides the position or orientation of the imaging device 31 so that the live view image 501 can be used as the measurement image. Output the message to the display device 32 . The imaging guide section 51 may output the message to a speaker (not shown) of the communication terminal 3 . When the image determination unit 48 determines that the live view image 501 can be used as a measurement image, the imaging guide unit 51 automatically captures the live view image 501 and stores it in a predetermined storage unit. The automatic imaging can be continued for any length of time as long as the live view image 501 is suitable as an image for measurement. Also, the imaging time may be changed according to the progress of wear. In this case, a reference value for determining the progress of wear is set in advance, and when the reference value is exceeded, it is assumed that wear is progressing, and automatic imaging is repeated at least two times, and the operator is instructed to A configuration may be adopted in which the best imaging result is selected. This makes it possible to grasp progress of wear more reliably and with higher accuracy.

Thus, the image evaluation apparatus 40 of Embodiment 3 can operate in an automatic imaging mode for automatically imaging when the live view image 501 is suitable as a measurement image.

FIG. 14 is a flow chart showing the operation of the image evaluation device 40 in automatic imaging mode.

In step S<b>1401 , the image evaluation device 40 accepts the start of the automatic imaging mode according to the user's input operation on the input device 33 .

In step S<b>1402 , the image evaluation device 40 determines whether or not the change of the threshold value T has been accepted according to the user's input operation on the input device 33 . A threshold value T is a threshold value for determining whether or not the live view image 501 can be used as a measurement image, and is stored in advance in a threshold storage unit (not shown) of the image evaluation device 40 . If the image evaluation apparatus 40 has not received the change of the threshold value T, the process proceeds to step S1404. When the image evaluation apparatus 40 accepts the change of the threshold value T, the process proceeds to step S1403.

In step S1403, the image evaluation device 40 resets the accepted threshold value T. FIG.

In step S<b>1404 , the image evaluation device 40 receives the part number of the sprocket 63 according to the user's input operation on the input device 33 .

In step S<b>1405 , the image evaluation device 40 determines whether or not the reference value X is stored in the reference value storage unit 45 . When the reference value X is stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S1407. If the reference value X is not stored in the reference value storage unit 45, the image evaluation device 40 proceeds to step S1406.

In step S1406, the image evaluation device 40 outputs to the display device 32 a message to the effect that the reference value X is not stored. The image evaluation device 40 ends this process shown in FIG.

In step S<b>1407 , the image evaluation device 40 calculates the average value Xm of the reference values X stored in the reference value storage unit 45 .

In step S<b>1408 , the image evaluation device 40 reads the live-view image 501 acquired by the live-view image acquisition unit 50 .

In steps S1409 to S1413, the image evaluation device 40 performs the same processing as in steps S807 to S811 shown in FIG.

In step S1414, the image evaluation device 40 determines whether the determination result in step S1413 indicates that the image can be used as the image for measurement. If the determination result does not indicate that the image can be used as the measurement image, the image evaluation device 40 proceeds to step S1417. When the determination result indicates that the image can be used as the measurement image, the image evaluation device 40 proceeds to step S1415.

In step S1415, the image evaluation device 40 captures the live view image 501 read this time and stores it as a measurement image.

In step S<b>1416 , the image evaluation device 40 outputs the extracted measurement image to the display device 32 . The image evaluation device 40 ends this process shown in FIG.

In step S1417, the image evaluation device 40 outputs to the display device 32 a message guiding the position or orientation of the imaging device 31 so that the live view image 501 can be used as a measurement image. For example, when the reference point of the front circle F calculated in step S1410 is located diagonally above the reference point of the back circle B, the image evaluation device 40 determines that "the position where the imaging device 31 is moved diagonally upward to the right". Please take an image from" is output to the display device 32.

In step S1418, the image evaluation device 40 determines whether or not the position or orientation of the imaging device 31 has been changed. The image evaluation device 40 determines whether or not the position or orientation of the imaging device 31 has been changed by checking changes in the live view image 501 or checking the detection signal of the orientation sensor of the communication terminal 3 . be able to. When the position or orientation of the imaging device 31 is changed, the image evaluation device 40 proceeds to step S1408. If the position or orientation of the imaging device 31 has not been changed, the image evaluation device 40 proceeds to step S1419.

In step S<b>1419 , the image evaluation device 40 determines whether or not termination of the automatic imaging mode has been accepted according to the user's input operation on the input device 33 . If the image evaluation apparatus 40 has not received the end of the automatic imaging mode, the process proceeds to step S1418. When the image evaluation device 40 accepts the end of the automatic imaging mode, it ends the processing shown in FIG. 14 .

As described above, the image evaluation device 40 of the third embodiment includes the imaging guide section 51 that guides the position or orientation of the imaging device 31 that images the worn portion of the working machine 6 . A captured image 461 to be evaluated in the third embodiment is a live view image 501 acquired by the live view function of the imaging device 31 . When the image determination unit 48 determines that the live view image 501 cannot be used as the measurement image, the imaging guidance unit 51 guides the position or orientation of the imaging device 31 so that the live view image 501 can be used as the measurement image. Print a message.

As a result, the image evaluation device 40 of the third embodiment can allow a user who is about to capture an image of a worn portion of the work machine 6 to easily capture an image suitable for measuring the amount of wear. Therefore, the image evaluation device 40 of the third embodiment can further simplify and improve the efficiency of the work of measuring the amount of wear. Also, during photography, there are cases where object recognition is not possible due to the nature of the object being photographed, such as when the object is covered with mud. This is an obstacle to accurate measurement of the amount of wear. By capturing the image of the part to be imaged in real time with the live view image, it is possible to acquire a more accurate image that is suitable for measuring the amount of wear.

In the third embodiment, the sprocket 63 is the part of the working machine 6 that is worn. Of course, it can also be applied in some cases. Further, the image evaluation apparatus 40 of the third embodiment includes a live-view image acquiring unit 50 instead of the captured image storage unit 46 and an image capturing guiding unit 51 instead of the image extracting unit 49 . However, the image evaluation device 40 may include both the captured image storage section 46 and the live view image acquisition section 50 , and may include both the image extraction section 49 and the imaging guide section 51 .

[Other embodiments]
The wear amount measurement system 1 measures the wear amount using the measurement image extracted by the image evaluation device 40 . The amount of wear may be measured by the communication terminal 3 or by the server device 2 . The wear amount measurement system 1 can specify the level of the measured wear amount, create a wear amount indicator image including a display corresponding to the specified level, and output it to the display device 32 of the communication terminal 3. .

The level of the amount of wear is, for example, a first level at which the amount of wear requiring replacement of the worn portion is reached, and a second level at which the amount of wear requiring replacement of the worn portion after a predetermined amount of work is likely to be reached. and a third level where the amount of wear that requires replacement of the worn portion is not reached even after performing a predetermined amount of work.

The wear amount indicator image is created for each part by changing at least one of its shape, size, color and pattern according to the wear amount level. For example, if the wear amount level is the first level, the wear amount indicator image is red, if the wear amount level is the second level, the wear amount indicator image is yellow, and if the wear amount level is the third level. For example, the wear amount indicator image may be created using green. Note that the levels and colors are not limited to three stages or three colors, and can be set arbitrarily. Alternatively, for example, if the level of the amount of wear is the first level, the length of the indicator image of the amount of wear is the longest, and if the level of the amount of wear is the second level, the length of the indicator image of the amount of wear is medium. and the length of the wear amount indicator image may be the shortest if the wear amount level is the third level. That is, the indicator image may be created such that the length of the indicator image increases as the degree of replacement recommendation for the worn site increases.

Furthermore, the wear amount measurement system 1 can predict future changes in wear amount from the current operating conditions of the work machine 6 . Then, the wear amount measurement system 1 can create an image in which the predicted transition of the wear amount is superimposed on the indicator image, and output the image to the display device 32 . Alternatively, the wear amount measurement system 1 can create an animation in which the length of the indicator image changes according to the predicted transition of the wear amount, and output the animation to the display device 32 .

Furthermore, the wear amount measurement system 1 can collect the wear amounts of the worn portions of the plurality of work machines 6 in the server device 2 and specify the work machines 6 having similar wear amounts. Then, the wear amount measurement system 1 can transmit information on a plurality of work machines 6 with similar wear amounts to the user's communication terminal 3 so that the users can share the information.

With such a configuration, the wear amount measurement system 1 can notify the user of information such as the level of wear amount, part replacement timing, predicted transition of wear amount, and the like in a form that is intuitively easy to grasp.

Although the embodiments of the present invention have been described in detail above, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the spirit of the invention described in the claims. It can be performed. The present invention adds features of one embodiment to features of other embodiments, replaces features of one embodiment with others, or deletes portions of features of one embodiment. can be

REFERENCE SIGNS LIST 1 wear amount measurement system 2 server device 3 communication terminal 31 imaging device 32 display device 33 input device 40 image evaluation device 401 arithmetic processing device 402 storage device 41 ... teaching data storage unit, 411 ... learning image, 412 ... feature point data, 42 ... learning unit, 43 ... feature point extraction unit, 44 ... reference value calculation unit, 441 ... central radius calculation unit, 442 ... ratio calculation unit, 443...Width ratio calculation unit 45...Reference value storage unit 46...Captured image storage unit 461...Captured image 47...Evaluation value calculation unit 471...Reference value average calculation unit 472...Center radius calculation unit 473... Difference length calculation unit 474 Difference ratio calculation unit 475 Width ratio calculation unit 476 Width ratio difference calculation unit 48 Image determination unit 49 Image extraction unit 50 Live view image acquisition unit 501 Live View image 51 Imaging guide unit 6 Work machine 61 Lower running body 62 Idler 63 Sprocket 631 Tooth 632 Bottom 633 Front 634 Back 635 Side 64 Crawler 65 Upper revolving body 651 Swing device 652 Cab 653 Machine room 66 Front work machine 67 Boom 671 Boom cylinder 68 Arm 681 Arm cylinder 69 Bucket , 69a... Tip part 691... Bucket cylinder 692... Adapter 693... Tooth 694... Front surface 695... Side surface 696... Boundary part

Claims (3)

An image evaluation device for evaluating whether a captured image of a specific portion of a working machine is suitable as a measurement image used to measure the wear amount of the specific portion,
A learning image including the contour of the specific part and a plurality of feature points for specifying the contour of the specific part included in the learning image are machine-learned as teacher data, and the captured image is a feature point extraction unit that extracts feature points that specify the contour of the specific part included;
a reference value storage unit that stores a reference value that is a reference for evaluating the captured image, which is calculated from the feature points in the learning image captured from a direction suitable for measuring the amount of wear;
Evaluating whether the captured image is suitable for the measurement image based on the feature points in the captured image extracted by the feature point extraction unit and the reference value stored in the reference value storage unit. an evaluation value calculation unit that calculates an evaluation value;
an image determination unit that determines whether the captured image can be used as the measurement image according to the evaluation value calculated by the evaluation value calculation unit;
An image evaluation device comprising: Further comprising an image extraction unit for extracting the measurement image,
The captured image is a moving image or a plurality of still images captured in advance,
3. The image extracting section extracts the measurement image from among the plurality of frame images forming the moving image or the plurality of still images according to the determination result of the image determining section. 1. The image evaluation apparatus according to 1. further comprising an imaging guide unit that guides the position or orientation of an imaging device that images the specific part,
The captured image is a live view image acquired by the imaging device,
When the image determination unit determines that the live view image cannot be used as the measurement image, the imaging guidance unit adjusts the position or orientation of the imaging device so that the live view image can be used as the measurement image. 2. The image evaluation apparatus according to claim 1, wherein a message for guidance is output.

Images