JP7101400B2 - Diagnostic equipment, diagnostic methods, and programs - Google Patents

Description

The present invention relates to diagnostic devices, diagnostic methods, and programs.

A technique for detecting a crack generated on the surface of a concrete structure from an image of the concrete structure is disclosed (for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2015-102382

For example, when the captured image is divided by a mesh and the presence or absence of cracks is determined for each divided area using AI (artificial intelligence), if there are cracks in the corners of the mesh-separated areas. , There was a possibility that the presence or absence of the crack could not be properly determined.

The present invention has been made in view of the above circumstances, and one of the objects of the present invention is to provide a diagnostic device, a diagnostic method, and a program capable of accurately diagnosing the presence or absence of cracks generated on the surface of an object.

The present invention has been made to solve the above problems, and one aspect of the present invention is to set a mesh that divides at least a part of an image area of a photographed image of an object into a plurality of areas. At the same time, the set mesh position is set by the area setting unit and the area setting unit that moves the set mesh position to each of the first direction and the second direction orthogonal to the first direction by a distance narrower than the mesh interval. The presence or absence of cracks generated in the object is diagnosed from the captured image for each of a plurality of regions divided by the first mesh, and the second mesh and the second mesh after moving in the first direction are used. A diagnostic unit for diagnosing the presence or absence of the crack from the captured image for each of a plurality of regions divided by the third mesh after moving in the direction of The diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the mesh, the diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the second mesh, and the plurality divided by the third mesh. It is a diagnostic apparatus that diagnoses the presence or absence of cracks and identifies the position of the cracks based on the diagnosis result of the presence or absence of the cracks in each region .

Another aspect of the present invention is a diagnostic method in a diagnostic apparatus, wherein the region setting unit divides at least a part of an image region of a captured image of an object into a plurality of regions . A step of setting a mesh and a step of a diagnosis unit diagnosing the presence or absence of cracks generated in the object from the captured image for each of a plurality of regions divided by the first mesh set by the region setting unit. The area setting unit sets the position of the first mesh at a distance narrower than the distance between the first mesh and the second direction orthogonal to the first direction. For each of a plurality of regions divided by the step of moving in and the second mesh after moving in the first direction and the third mesh after moving in the second direction. The step of diagnosing the presence or absence of the crack from the captured image, the diagnosis unit is divided by the diagnosis result of the presence or absence of the crack for each of a plurality of regions divided by the first mesh, and the second mesh. Based on the diagnosis result of the presence or absence of the crack in each of the plurality of regions and the diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the third mesh, the diagnosis of the presence or absence of the crack and the diagnosis of the crack It is a diagnostic method having a step of specifying a position.

In addition, another aspect of the present invention includes a step of setting a first mesh that divides at least a part of an image region of a captured image of an object into a plurality of regions on a computer, and the above-described setting. A step of diagnosing the presence or absence of cracks generated in the object from the captured image for each of a plurality of regions divided by the first mesh, and the set position of the first mesh as the first direction. A step of moving the first mesh to each of the second directions orthogonal to the first direction at a distance narrower than the interval of the first mesh, and a second mesh and the second mesh after moving in the first direction. A step of diagnosing the presence or absence of the crack from the captured image for each of the plurality of regions divided by the third mesh after moving in the direction, and each of the plurality of regions divided by the first mesh. Diagnosis result of the presence or absence of the crack, diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the second mesh, and diagnosis of the presence or absence of the crack in each of the plurality of regions divided by the third mesh. Based on the result, it is a program for executing the step of diagnosing the presence or absence of the crack and identifying the position of the crack.

According to the present invention, cracks generated on the surface of an object can be accurately diagnosed.

The block diagram which shows an example of the diagnostic apparatus 10 which concerns on embodiment. The figure which shows the setting example of the mesh which concerns on embodiment. The figure explaining the change of the position of a crack when the mesh is moved laterally. The figure which shows the 1st example of the example which improved the diagnostic accuracy after moving a mesh in a lateral direction. The figure which shows the 2nd example of the example which improved the diagnostic accuracy after moving a mesh in a lateral direction. The figure which shows the 3rd example of the example which improved the diagnostic accuracy after moving a mesh in a lateral direction. The figure which shows the 4th example of the example which improved the diagnostic accuracy after moving a mesh in a lateral direction. The figure which shows the 5th example of the example which improved the diagnostic accuracy after moving a mesh in a lateral direction. The figure explaining the change of the position of a crack when the mesh is moved in the vertical direction. The figure which shows the 1st example of the example which improved the diagnostic accuracy after moving a mesh in a vertical direction. The figure which shows the 2nd example of the example which improved the diagnostic accuracy after moving a mesh in a vertical direction. The figure which shows the 3rd example of the example which improved the diagnostic accuracy after moving a mesh in a vertical direction. The figure which shows the 4th example of the example which improved the diagnostic accuracy after moving a mesh in a vertical direction. The figure which shows the 5th example of the example which improved the diagnostic accuracy after moving a mesh in a vertical direction. The flowchart which shows an example of the diagnostic process which concerns on embodiment. The figure which shows the 1st example of the provision example of the diagnosis result which concerns on embodiment. The figure which shows the 2nd example of the provision example of the diagnosis result which concerns on embodiment. The figure which shows the 3rd example of the provision example of the diagnosis result which concerns on embodiment. The figure explaining the method of generating the diagnosis result which combined three kinds of meshes. The figure which shows the example of providing the diagnosis result when three kinds of meshes are combined.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, the outline of the present embodiment will be described.
In recent years, as an alternative means of close-up visual inspection for inspecting cracks and the like generated on bridges and the like at actual sites, verification of image diagnosis using an ultra-high resolution camera has been carried out. For example, by using an image of 100 million pixels taken by an ultra-high resolution camera, an inspection environment almost equivalent to a close-up visual inspection performed by a human is constructed. In particular, for places where it is difficult for people to approach and visual inspection in close proximity, using a UAV (Unmanned Aerial Vehicle) equipped with a camera makes it easier to obtain a photographed image to be inspected, so the photographed image is used. The diagnosis was valid. In addition, since there is a shortage of human resources who have experience in inspection (inspectors), there is a method of performing image diagnosis using AI (artificial intelligence) on captured images. In this case, for example, a method of dividing the captured image with a mesh and diagnosing the presence or absence of cracks in each divided region using AI (artificial intelligence) can be considered. However, if there are cracks in the corners of the mesh-divided area, it may not be possible to properly determine the presence or absence of cracks. Therefore, the diagnostic apparatus according to the present embodiment is configured to improve the diagnostic accuracy by moving the mesh when diagnosing the presence or absence of cracks generated in the object.

(Configuration of diagnostic equipment)
Hereinafter, the configuration of the diagnostic apparatus according to the present embodiment will be described in detail. Here, "crack" is a general term for cracks, scratches, dents, etc. that occur on the surface of an object. The object is an object to be inspected, and is, for example, a concrete structure such as a bridge or a building. The object is not limited to the concrete structure, and may be any object as long as it causes cracks.

FIG. 1 is a block diagram showing an example of the diagnostic apparatus 10 according to the present embodiment. The diagnostic device 10 includes a communication unit 110, an input unit 120, a display unit 130, a storage unit 150, and a control unit 170.

The communication unit 110 may have hardware (for example, an antenna and a transmitter / receiver) for connecting to the Internet using a cellular network, a Wi-Fi network, or the like. Further, the communication unit 110 may be a connector connected to an external storage medium, or may be a connector connected to an external device such as an image pickup device via a cable.

The input unit 120 includes a part or all of various keys, buttons, dial switches, a mouse, and the like. Further, the input unit 120 may be, for example, a touch panel formed integrally with the display unit 130.

The display unit 130 includes, for example, a display such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence).

The storage unit 150 is, for example, a flash memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), an HDD (Hard Disk Drive), or the like. For example, the storage unit 150 stores information such as captured image data 151, teaching data 152, and learned model 153.

The photographed image data 151 is data of a photographed image of an object to be inspected (for example, a concrete structure such as a bridge).

The teaching data 152 is teaching data for generating a trained model of AI used when diagnosing the presence or absence of cracks from a captured image. For example, image data including cracks is stored as teaching data 152. The image data including cracks is, for example, image data obtained by photographing an arbitrary object (for example, a concrete structure such as a pier) in which a crack is actually generated. In addition, image data that does not include cracks may be further stored as teaching data 152. These image data may be image data in which the presence or absence of cracks has been identified by a person (for example, an inspector) in the past, or image data in which the presence or absence of cracks has been diagnosed by the diagnostic apparatus 10. It may be image data in which the presence or absence of cracks has been diagnosed by another diagnostic device. In addition, image data including an object (grass or the like) that may be included in the captured image may be further stored as teaching data 152. In addition, by including image data including an object (grass, etc.) that may be included in the captured image in the teaching data, it is possible to prevent a misdiagnosis from being mistaken for a crack. For example, a plurality of image data including cracks, image data not including cracks, and image data including grass and the like are stored as teaching data 152.

The trained model 153 stores a trained model generated by the learning unit 172 described below based on the teaching data 152 described above.

The control unit 170 includes, for example, an acquisition unit 171, a learning unit 172, and an area setting unit 173 as functional configurations realized by executing a program (software) by a hardware processor such as a CPU (Central Processing Unit). , A diagnostic unit 174 and a result providing unit 175.

The acquisition unit 171 acquires a photographed image via the communication unit 110 and stores the photographed image data 151 in the storage unit 150. The photographed image is a photographed image of an object to be inspected (for example, a concrete structure such as a bridge). For example, it is a photographed image of an object photographed by using a UAV, or a photographed image of an object photographed by a person.

The learning unit 172 generates an AI trained model 153 for diagnosing the presence or absence of cracks based on the teaching data 152. For example, the learning unit 172 performs machine learning using any or all of a plurality of image data including cracks, image data without cracks, image data including grass, etc. as teaching data, and determines the presence or absence of cracks. A trained model 153 for determination is generated as a training result.

The area setting unit 173 sets a mesh that divides at least a part of the image area of the photographed image in which the object is photographed into a plurality of areas. FIG. 2 is a diagram showing an example of setting a mesh according to the present embodiment. In the illustrated example, the object to be inspected is the surface of a pier having a concrete structure, and a mesh M is set to divide the captured image of the object into a rectangular area of 6 × 19 in width. The presence or absence of cracks is diagnosed for each region divided with respect to the range of the mesh M. The number of divisions, the shape of the divided area, the setting range of the mesh, etc. are examples, and are not limited to the illustrated examples.

Further, the area setting unit 173 moves the position of the set mesh. For example, the area setting unit 173 moves in the horizontal direction or the vertical direction at a distance narrower than the mesh spacing. The horizontal direction is a direction (horizontal direction in FIG. 2) parallel to the horizontal line (longitudinal direction line) of the mesh M having a length of 6 × a width of 19 shown in FIG. The vertical direction is a direction orthogonal to the horizontal direction, and is a direction parallel to the vertical line (vertical direction in FIG. 2) of the mesh M having a length of 6 × a width of 19 shown in FIG. ..

For example, the area setting unit 173 may move the set position of the mesh in the lateral direction (left or right) at a distance narrower than the mesh spacing. Further, the area setting unit 173 may move the set position of the mesh in the vertical direction (up or down) at a distance narrower than the distance between the meshes. Further, the area setting unit 173 may move the set position of the mesh in the horizontal direction (left or right direction) and the vertical direction (up or down direction) at a distance narrower than the mesh interval. Here, the moving distance is, for example, a distance corresponding to half of the mesh spacing.

The diagnosis unit 174 diagnoses the presence or absence of cracks from the captured image in which the region setting unit 173 sets the mesh, using the trained model 153 of AI. First, the diagnosis unit 174 diagnoses the presence or absence of cracks from the captured image for each of a plurality of regions divided by the mesh set by the region setting unit 173. Further, the diagnosis unit 174 diagnoses the presence or absence of cracks from the captured image for each of the plurality of regions divided by the mesh after movement. For example, the diagnostic unit 174 makes a diagnosis for each region divided by the initially set mesh, and then further half the mesh interval in the horizontal direction (left or right direction) or the vertical direction (up or down direction). Diagnosis is performed for each area divided by the mesh after movement after moving a distance corresponding to. As a result, the cracks existing in the corners of the initially set mesh-divided area move toward the center in the moved mesh-divided area, so the presence or absence of cracks can be accurately diagnosed. become able to.

The result providing unit 175 provides the user with the diagnosis result or the like by the diagnosis unit 174 by using the display unit 130. For example, the result providing unit 175 causes the display unit 130 to display information based on the probability of including cracks for each of a plurality of regions divided by the mesh of the captured image. The information based on the probability including the crack may be a value of the probability, or may be a symbol, a color, a light / darkness, etc. corresponding to the probability.

(Example of improved diagnostic accuracy)
Next, with reference to FIGS. 3 to 14, an actual example in which the diagnostic accuracy is actually improved by moving the mesh will be described. First, with reference to FIGS. 3 to 8, an example in which the diagnostic accuracy is improved when the mesh is moved in the lateral (left-right) direction will be described. FIG. 3 is a diagram illustrating a change in the position of a crack when the mesh is moved in the lateral direction. Here, an example is shown when the initially set mesh is moved laterally by a distance corresponding to half of the mesh spacing. The mesh M1a shows an image in one of the regions divided by the initially set mesh. The mesh M1b shows an image in the region when the mesh M1a is moved laterally by half the distance between the meshes. The crack CR1 existing at the right end in the mesh M1a moves relatively toward the center in the moved mesh M1b. This improves the diagnostic accuracy of cracks. The arrow shown in the figure indicates that the mesh is moved to the right by half the distance between the meshes, but even when the mesh is moved to the left, the area to the right of the mesh M1a is at the same position as the area of the mesh M1b. Since it moves to, the diagnostic accuracy is improved as well.

FIGS. 4 to 8 show an example in which the diagnostic accuracy is improved after the movement than before the movement when the mesh shown in FIG. 3 is moved in the lateral (left-right) direction. When the mesh is moved in the horizontal (left-right) direction, it is effective in improving the diagnostic accuracy mainly for long cracks in the vertical direction. Here, the diagnosis unit 174 shows the "probability of including cracks" as a diagnosis result using the trained model 153 of AI. The probability of including cracks is an index of the certainty that the image in each region divided by the mesh contains cracks, and the minimum value is 0% and the maximum value is 100%.

In the example shown in FIG. 4, the CR2 is cracked toward the left end in the initially set mesh M2a, but the CR2 is cracked near the center in the mesh M2b after the movement by moving the mesh in the lateral direction. Is moving. According to the diagnosis result for the mesh M2a set first, the probability that the image in the mesh M2a contains a crack was diagnosed as 61%. On the other hand, in the diagnosis result for the mesh M2b after the movement, the probability that the image in the mesh M2b contains a crack was diagnosed as 99.7%. That is, the diagnosis accuracy after moving the mesh was improved by 38.7 points.

In the example shown in FIG. 5, the CR3 is cracked toward the right end in the initially set mesh M3a, but by moving the mesh in the lateral direction, the CR3 is cracked near the center in the mesh M3b after the movement. Is moving. According to the diagnosis result for the mesh M3a set first, the probability that the image in the mesh M3a contains a crack was diagnosed as 2.9%. On the other hand, in the diagnosis result for the mesh M3b after the movement, the probability that the image in the mesh M3b contains a crack was diagnosed as 99.5%. That is, the diagnosis result after moving the mesh showed that the 96.6 point diagnosis accuracy was improved.

In the example shown in FIG. 6, the CR4 is cracked toward the right end in the initially set mesh M4a, but the CR4 is cracked near the center in the mesh M4b after the movement by moving the mesh in the lateral direction. Is moving. According to the diagnosis result for the mesh M4a set first, the probability that the image in the mesh M4a contains a crack was diagnosed as 11.2%. On the other hand, in the diagnosis result for the mesh M4b after the movement, the probability that the image in the mesh M4b contains a crack was diagnosed as 99.9%. That is, 88.7 points of diagnosis accuracy was improved in the diagnosis result after the mesh was moved.

In the example shown in FIG. 7, the CR5 is cracked toward the right end in the initially set mesh M5a, but the CR5 is cracked near the center in the mesh M5b after the movement by moving the mesh in the lateral direction. Is moving. According to the diagnosis result for the mesh M5a set first, the probability that the image in the mesh M5a contains a crack was diagnosed as 5.8%. On the other hand, in the diagnosis result for the mesh M5b after the movement, the probability that the image in the mesh M5b contains a crack was diagnosed as 99.1%. That is, the diagnosis result after moving the mesh showed that the 93.3 point diagnosis accuracy was improved.

In the example shown in FIG. 8, the CR6 is cracked toward the right end in the initially set mesh M6a, but the CR6 is cracked near the center in the mesh M6b after the movement by moving the mesh in the lateral direction. Is moving. According to the diagnosis result for the mesh M6a initially set, the probability that the image in the mesh M6a contains a crack is 17.1%. On the other hand, in the diagnosis result for the mesh M6b after the movement, it was diagnosed that the probability that the image in the mesh M6b contained a crack was 56.5%. That is, the diagnostic result after moving the mesh showed that the diagnostic accuracy was improved by 39.4 points.

Next, with reference to FIGS. 9 to 14, an example in which the diagnostic accuracy is improved when the mesh is moved in the vertical (vertical) direction will be described. FIG. 9 is a diagram illustrating a change in the position of a crack when the mesh is moved in the vertical direction. Here, an example is shown when the initially set mesh is moved in the vertical direction by a distance corresponding to half of the mesh spacing. The mesh M7a shows an image in one of the regions divided by the initially set mesh. The mesh M7b shows an image in the region when the mesh M7a is moved in the vertical direction by a distance of half the distance between the meshes. The crack CR7 existing at the lower end in the mesh M7a moves relatively toward the center in the vertical direction in the mesh M7b after the movement. This improves the diagnostic accuracy of cracks. The arrow shown in the figure indicates that the mesh is moved downward by half the distance between the meshes, but even when the mesh is moved upward, the area below the mesh M7a is at the same position as the area of the mesh M7b. Since it moves to, the same diagnosis result is obtained.

10 to 14 show an example in which the diagnostic accuracy is improved after the movement of the mesh shown in FIG. 9 after the movement than before the movement when the mesh is moved in the vertical (vertical) direction. When the mesh is moved in the vertical (vertical) direction, it is effective in improving the diagnostic accuracy mainly for long cracks in the horizontal direction. In the example shown in FIG. 10, the CR8 is cracked toward the upper end in the initially set mesh M8a, but by moving the mesh in the vertical direction, the CR8 is cracked near the center in the mesh M8b after the movement. Is moving. According to the diagnosis result for the mesh M8a set first, the probability that the image in the mesh M8a contains a crack was diagnosed as 94.6%. On the other hand, in the diagnosis result for the mesh M8b after the movement, the probability that the image in the mesh M8b contains a crack was diagnosed as 97.6%. That is, in the diagnosis result after moving the mesh, the 3-point diagnosis accuracy was improved.

In the example shown in FIG. 11, the CR9 is cracked toward the lower end in the initially set mesh M9a, but by moving the mesh in the vertical direction, the CR9 is cracked near the center in the mesh M9b after the movement. Is moving. According to the diagnosis result for the mesh M9a set first, the probability that the image in the mesh M9a contains a crack is 6.1%. On the other hand, in the diagnosis result for the mesh M9b after the movement, it was diagnosed that the probability that the image in the mesh M9b contained a crack was 57.2%. That is, the diagnosis accuracy after moving the mesh was improved by 51.1 points.

In the example shown in FIG. 12, the CR10 is cracked toward the lower end in the initially set mesh M10a, but by moving the mesh in the vertical direction, the CR10 is cracked near the center in the mesh M10b after the movement. Is moving. According to the diagnosis result for the mesh M10a set first, the probability that the image in the mesh M10a contains a crack was diagnosed as 51%. On the other hand, in the diagnosis result for the mesh M10b after the movement, the probability that the image in the mesh M10b contains a crack was diagnosed as 99.7%. That is, the diagnostic result after moving the mesh improved the diagnostic accuracy by 48.7 points.

In the example shown in FIG. 13, the CR11 is cracked toward the upper end (and the right end) in the initially set mesh M11a, but by moving the mesh in the vertical direction, the mesh M11b after movement is in the vertical direction. The CR11 is relatively moving near the center (and at the right end). According to the diagnosis result for the mesh M11a set first, the probability that the image in the mesh M11a contains a crack was diagnosed as 42.7%. On the other hand, in the diagnosis result for the mesh M11b after the movement, it was diagnosed that the probability that the image in the mesh M11b contained a crack was 99.8%. That is, the diagnostic result after moving the mesh showed that the diagnostic accuracy was improved by 57.1 points.

In the example shown in FIG. 14, the CR12 is cracked toward the lower end in the initially set mesh M12a, but by moving the mesh in the vertical direction, the CR12 is cracked near the center in the mesh M6b after the movement. Is moving. According to the diagnosis result for the mesh M12a set first, the probability that the image in the mesh M12a contains a crack is 68.3%. On the other hand, in the diagnosis result for the mesh M12b after the movement, it was diagnosed that the probability that the image in the mesh M12b contained a crack was 94.3%. That is, the diagnosis accuracy after moving the mesh was improved by 26 points.

(Operation of diagnostic processing)
Next, the operation of the diagnostic process in which the diagnostic device 10 according to the present embodiment diagnoses the cracks generated on the surface of the object will be described.
FIG. 15 is a flowchart showing an example of the diagnostic process according to the present embodiment.

First, the acquisition unit 171 acquires a photographed image of the object via the communication unit 110 and stores the photographed image data 151 in the storage unit 150 (step S101).

Next, the area setting unit 173 sets a mesh in at least a part of the image area of the captured image of the captured image data 151 (step S103).

Then, the diagnosis unit 174 diagnoses the presence or absence of cracks from the captured image for each of a plurality of regions divided by the mesh set by the region setting unit 173 (step S105).

Next, the area setting unit 173 moves the set position of the mesh in the horizontal direction (left or right). Specifically, the area setting unit 173 moves the set mesh position in the lateral direction (left or right) by a distance corresponding to half of the mesh interval (step S107).

Then, the diagnosis unit 174 diagnoses the presence or absence of cracks from the captured image for each of the plurality of areas divided by the moved mesh moved by the area setting unit 173 (step S109).

Subsequently, the area setting unit 173 moves the set position of the mesh in the vertical direction (up or down). Specifically, the area setting unit 173 moves the set mesh position in the vertical direction (up or down) by a distance corresponding to half of the mesh interval (step S111).

Then, the diagnosis unit 174 diagnoses the presence or absence of cracks from the captured image for each of the plurality of regions divided by the moved mesh moved by the region setting unit 173 (step S113).

The result providing unit 175 provides the user with the diagnosis result or the like by the diagnosis unit 174 by using the display unit 130. For example, the result providing unit 175 causes the display unit 130 to display information based on the probability of including cracks for each of a plurality of regions divided by the mesh of the captured image (step S115).

(Example of providing diagnostic results)
16 to 18 are diagrams showing an example of providing a diagnosis result according to the present embodiment. FIG. 16 shows the diagnosis result by the mesh set first. Further, FIG. 17 shows the diagnosis result by the moved mesh in which the initially set mesh is moved by a distance corresponding to half of the mesh interval in the lateral direction. Further, FIG. 18 shows the diagnosis result by the moved mesh in which the initially set mesh is moved by a distance corresponding to half of the mesh interval in the vertical direction. In each of FIGS. 16 to 18, as the diagnosis result for each region divided by the mesh, there are three types of regions having a probability of including cracks of less than 30%, regions of 30 to 60%, and regions of 60% or more. It is divided into and displayed with symbols. Areas with a probability of containing cracks less than 30% are diagnosed as having a low probability of cracking. The region where the probability of including cracks is 30 to 60% is a diagnosis of "medium crack occurrence probability". Areas with a probability of including cracks of 60% or more are diagnosed as having a high probability of cracking.

As mentioned above, if there are cracks in the corners of the mesh-divided area, the cracks are not always properly diagnosed. Especially when many images of the teaching data are images with cracks passing near the center of the image, the cracks existing in the corners are difficult to be properly diagnosed. For example, it is conceivable to improve the recognition accuracy by including an image having cracks in the corners of the image in the teaching data, but it is difficult to recognize the cracks existing in the corners in the first place, and it is also difficult. It is not easy to prepare and learn the optimum teaching data. On the other hand, by moving the mesh in the horizontal or vertical direction, the cracks existing in the corners of the area divided by the mesh can be moved relatively to the center, so that the diagnostic accuracy can be easily improved. can.

For example, when the mesh is moved laterally as shown in FIG. 17, cracks on the left side of the pier (vertical cracks) that were not accurately diagnosed by the initially set mesh shown in FIG. 16 are properly diagnosed. rice field. Further, by moving the mesh in the vertical direction as shown in FIG. 18, the crack in the center of the pier (crack in the horizontal direction), which could not be accurately diagnosed with the initially set mesh shown in FIG. 16, can be accurately diagnosed. I was diagnosed.

Further, the diagnosis result of the initially set mesh shown in FIG. 16, the diagnosis result of the mesh moved in the horizontal direction shown in FIG. 17, and the diagnosis result of the mesh moved in the vertical direction shown in FIG. 18 The diagnostic result may be generated by combining the three types of diagnostic results. For example, the crack occurrence probability of the region where the three meshes overlap may be the average of the crack occurrence probabilities of each of the three overlapping meshes. Details will be described with reference to FIG.

FIG. 19 is a diagram illustrating a method of generating a diagnostic result by combining the diagnostic results of three types of meshes. Here, in order to facilitate the explanation, a mesh of 4 vertical × 4 horizontal is extracted and shown. The mesh MA shown in (A) is the first set mesh. The mesh MB shown in (B) is obtained by moving the mesh MA laterally by half the distance between the meshes. The mesh MC shown in (C) is obtained by moving the mesh MA in the vertical direction by a distance of half of the mesh spacing. (D) is a diagram in which mesh MA, mesh MB, and mesh MC are superimposed. In (D), the region MD1 is a region where the region MA1 of the mesh MA, the region MB1 of the mesh MB, and the region MC1 of the mesh MC overlap. The diagnosis result of the overlapping region MD1 is defined as the crack occurrence probability obtained by averaging the crack occurrence probability of the mesh MA region MA1, the crack occurrence probability of the mesh MB region MB1, and the crack occurrence probability of the mesh MC region MC1. ..

FIG. 20 is a diagram showing an example of providing a diagnostic result when the diagnostic results of three types of meshes are combined. In this way, by averaging the diagnostic results of the three types of meshes, vertical cracks that could not be accurately diagnosed with the diagnostic results of the initially set mesh are moved in the horizontal direction. It is complemented by the diagnosis result, and the crack in the horizontal direction can be complemented by the diagnosis result in the mesh moved in the vertical direction, and the diagnosis accuracy can be improved. Further, by combining the diagnosis results of the three types of meshes, it is possible to perform the diagnosis for each of the finely divided regions, so that the position of the crack can be specified in more detail. As shown in FIG. 20, the diagnostic result when the diagnostic results of the three types of meshes are combined has a better accuracy of crack identification than the diagnostic result of the initially set mesh shown in FIG. Yes, more detailed.

As described above, the diagnostic apparatus 10 according to the present embodiment sets a mesh that divides at least a part of the image area of the captured image of the object into a plurality of areas, and is divided by the set mesh. Diagnose the presence or absence of cracks in the object from the captured image for each of a plurality of areas. Further, the diagnostic apparatus 10 moves the position of the set mesh and diagnoses the presence or absence of the crack from the captured image for each of a plurality of regions divided by the moved mesh.

As a result, even if the diagnostic device 10 cannot make an accurate diagnosis due to the presence of cracks in the corners of the region divided by the initially set mesh, the diagnostic device 10 moves the mesh to perform the diagnosis again. Cracks generated on the surface of an object can be diagnosed with high accuracy, and the diagnostic accuracy can be improved.

For example, the diagnostic apparatus 10 moves the set position of the mesh in the horizontal direction or the vertical direction at a distance narrower than the mesh spacing.

As a result, the diagnostic apparatus 10 can relatively move the position of the crack in the mesh-divided area when the crack is present in the corner of the initially set mesh-divided area. .. Therefore, the position of the crack that existed in the corner before the movement of the mesh moves relatively toward the center after the movement of the mesh, so that the crack generated on the surface of the object can be diagnosed accurately.

More specifically, the distance to move the mesh may be a distance corresponding to half the distance between the meshes.

As a result, when the diagnostic apparatus 10 has a crack in the corner of the initially set mesh-divided area, the position of the crack is relatively relative to the position near the center in the mesh-divided area. Since it can be moved, the accuracy of crack diagnosis can be improved.

The distance for moving the mesh is not limited to a distance corresponding to half of the mesh spacing, and may be any distance within a range narrower than the mesh spacing. Also, if the distance to move the mesh is shorter than half the distance between the meshes, the number of movements may be increased. For example, if the distance to move the mesh is half the distance between the meshes, the mesh is moved once, and if the distance to move the mesh is a quarter of the distance between the meshes, the mesh is moved to 2. It may be moved once.

Further, the diagnostic apparatus 10 moves the set mesh position in each of the first direction (for example, the horizontal direction) and the second direction (for example, the vertical direction) orthogonal to the first direction. May be good. Then, the diagnostic apparatus 10 has a captured image for each of the plurality of regions divided by the initially set mesh, and each of the plurality of regions divided by the moved mesh that has been moved in the first direction (for example, the lateral direction). Diagnose the presence or absence of cracks in the object by using each of the captured image of You may.

As a result, the diagnostic apparatus 10 moves the mesh in two directions (for example, the horizontal direction and the vertical direction) that are orthogonal to each other to make a diagnosis, so that the left and right corners within the region divided by the initially set mesh. Since both the cracks existing in the sill and the cracks existing in the upper and lower corners can be relatively moved to the vicinity of the center for diagnosis, the diagnostic accuracy of the cracks can be improved. Further, by combining the diagnosis results of the three types of meshes, it is possible to perform the diagnosis for each of the finely divided regions, so that the position of the crack can be specified in more detail.

Further, the diagnostic device 10 learns a plurality of images including cracks as teaching data at least, and based on the learned model of AI based on the learning result, it is generated from the captured image to the object for each region divided by the mesh. Diagnose the presence or absence of cracks.

As a result, the diagnostic device 10 can accurately diagnose the presence or absence of cracks by using AI even if there is no experienced person or skilled person who can visually identify the cracks. Therefore, it is expected that there will be a shortage of experienced and skilled personnel as the population declines as the number of aging piers increases in the future. Inspection is possible, and it can contribute to the maintenance of piers.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention. It is possible to do.

In the above-described embodiment, an example of moving the mesh in the horizontal direction and the vertical direction has been described. For example, the horizontal direction corresponds to the horizontal direction with respect to the object, and the vertical direction corresponds to the vertical direction with respect to the object. However, the relationship between the above directions is not limited to the horizontal direction and the vertical direction because it changes depending on the inclination of the camera when the captured image is taken and the method of setting the mesh.

Further, in the above-described embodiment, an example of the process of moving the mesh in the horizontal direction and the vertical direction for diagnosis has been described, but the process may be performed by moving the mesh in either direction for diagnosis. Further, the process may be performed by moving the mesh in three or more directions to perform diagnosis.

For example, in the above-described embodiment, in the case of a mesh divided into a rectangular area of 6 vertical × 19 horizontal, an example of moving the mesh in the horizontal direction or the vertical direction according to the shape of the mesh has been described, but the present invention is limited to this. Instead of the one, the mesh may be moved in the horizontal or vertical direction, or in addition, in the diagonal direction or the like. Further, the shape of the mesh (the shape of the divided region and the shape of the entire mesh) is not limited to a rectangle, and may be a square, a triangle, or a polygonal shape of a pentagon or more, or a curve. It may have a shape including. Further, the shape of the region divided by the mesh and the shape of the entire mesh may be different.

In addition, a part or all of the control unit 170 in the above-described embodiment may be realized by a computer. In that case, a program for realizing a part or all of the functions of the control unit 170 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by the computer system and executed. It may be realized by. The "computer system" referred to here is a computer system built in the diagnostic apparatus 10, and includes hardware such as an OS and peripheral devices. Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage device such as a hard disk built in a computer system. Furthermore, a "computer-readable recording medium" is a medium that dynamically holds a program for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In that case, a program that holds a program for a certain period of time, such as a volatile memory inside a computer system that serves as a server or a client, may be included. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

Further, a part or all of the control unit 170 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Further, for example, a part or all of the control unit 170 may be integrated into a processor. Further, the method of making an integrated circuit is not limited to the LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, when an integrated circuit technology that replaces an LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

10 Diagnostic device, 110 Communication unit, 120 Input unit, 130 Display unit, 150 Storage unit, 151 Captured image data, 152 Teaching data, 153 Learned model, 170 Control unit, 171 Acquisition unit, 172 Learning unit, 173 Area setting unit 174 Diagnosis Department, 175 Result Providing Department

Claims (7)

A second mesh that divides at least a part of the image region of the captured image of the object into a plurality of regions is set, and the position of the set mesh is orthogonal to the first direction and the first direction. The area setting part that moves in each direction with a distance narrower than the mesh spacing,
The presence or absence of cracks generated in the object is diagnosed from the captured image for each of a plurality of regions divided by the first mesh set by the region setting unit, and the second after moving in the first direction. A diagnostic unit that diagnoses the presence or absence of cracks from the captured image for each of a plurality of regions divided by each of the mesh and the third mesh after moving in the second direction.
With
The diagnostic unit
The diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the first mesh, the diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the second mesh, and the third mesh. Based on the diagnosis result of the presence or absence of the crack for each of the divided plurality of regions, the presence or absence of the crack is diagnosed and the position of the crack is specified.
Diagnostic device. The diagnostic unit
The first region of the plurality of regions divided by the first mesh, the second region after the movement of the first region before the movement among the plurality of regions divided by the second mesh, and the second region after the movement. The presence or absence of the crack in the region overlapping the third region after the movement of the first region before the movement among the plurality of regions divided by the third mesh is determined by the presence or absence of the first region, the second region, and the region. Diagnosis is made based on the diagnosis result of the presence or absence of the crack in each of the third regions.
The diagnostic device according to claim 1. The first direction is a horizontal direction or a vertical direction.
The diagnostic device according to claim 1 or 2. The distance that the area setting unit moves the mesh is a distance corresponding to half of the interval of the mesh.
The diagnostic device according to any one of claims 1 to 3. The diagnostic unit
A learning unit for learning at least a plurality of images including the cracks as teaching data is provided.
Based on the learned model based on the learning result by the learning unit, the presence or absence of the crack is diagnosed from the captured image for each region divided by the mesh.
The diagnostic device according to any one of claims 1 to 4. It is a diagnostic method in a diagnostic device.
A step in which the area setting unit sets a first mesh that divides at least a part of the image area of the captured image in which the object is photographed into a plurality of areas.
A step of diagnosing the presence or absence of cracks generated in the object from the captured image for each of a plurality of regions divided by the first mesh set by the region setting unit.
The area setting unit moves the set position of the first mesh to each of the first direction and the second direction orthogonal to the first direction at a distance narrower than the interval of the first mesh. Steps to make
The diagnostic unit cracks the captured image for each of a plurality of regions divided by the second mesh after moving in the first direction and the third mesh after moving in the second direction. Steps to diagnose the presence or absence of
The diagnostic unit uses the diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the first mesh, the diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the second mesh, and the above. Based on the diagnosis result of the presence or absence of the crack for each of a plurality of regions divided by the third mesh, the step of diagnosing the presence or absence of the crack and specifying the position of the crack,
Diagnostic method with. On the computer
A step of setting a first mesh that divides at least a part of the image area of the captured image in which the object is photographed into a plurality of areas, and
A step of diagnosing the presence or absence of cracks generated in the object from the captured image for each of a plurality of regions divided by the set first mesh.
A step of moving the set position of the first mesh to each of the first direction and the second direction orthogonal to the first direction at a distance narrower than the interval of the first mesh.
The presence or absence of the crack is diagnosed from the captured image for each of a plurality of regions divided by the second mesh after moving in the first direction and the third mesh after moving in the second direction. Steps to do and
The diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the first mesh, the diagnosis result of the presence or absence of the crack in each of the plurality of regions divided by the second mesh, and the third mesh. Based on the diagnosis result of the presence or absence of the crack for each of the divided plurality of regions, the step of diagnosing the presence or absence of the crack and identifying the position of the crack, and
A program to execute.

Images