WO2019159425A1

WO2019159425A1 - Evaluation system, evaluation device, evaluation method, evaluation program, and recording medium

Info

Publication number: WO2019159425A1
Application number: PCT/JP2018/037252
Authority: WO
Inventors: 可也中野
Original assignee: 新東工業株式会社
Priority date: 2018-02-16
Filing date: 2018-10-04
Publication date: 2019-08-22
Also published as: JP2019144013A; TW201935320A; JP6881347B2

Abstract

This evaluation system is for performing evaluation for a degree of rusting of a to-be-evaluated object by using a captured image of the to-be-evaluated object. The evaluation system is provided with: an image acquisition unit that acquires a captured image; a correction unit that corrects the captured image to generate an evaluation image; an evaluation unit that performs evaluation on the basis of the evaluation image; and an output unit that outputs an evaluation result by the evaluation unit. The correction unit extracts, from the captured image, an evaluation region that represents an image of a range having a predetermined area on a surface of the to-be-evaluated object, and generates the evaluation image on the basis of the evaluation region.

Description

Evaluation system, evaluation apparatus, evaluation method, evaluation program, and recording medium

The present disclosure relates to an evaluation system, an evaluation apparatus, an evaluation method, an evaluation program, and a recording medium.

An inspection apparatus that performs inspection using an image of an inspection object is known. For example, Patent Document 1 discloses an inspection apparatus that inputs an image obtained by imaging an inspection object to a neural network and determines the presence and type of a defect.

It is also known to evaluate the degree of rust generated in steel materials. For example, Patent Document 2 discloses a measuring apparatus that irradiates light to a steel material and measures a scale from reflected light intensity at a specific wavelength.

JP 2012-26982 A JP-A-8-254502

In order to evaluate the degree of rust generated on the surface of an object such as steel, it is conceivable to use an image of the object as described in Patent Document 1. In this case, the imaging condition of the captured image may be different for each captured image. For example, the image may be taken from a position away from the object, or the image may be taken near the object. An arbitrary magnification is set and the image is taken. However, the degree of rust can be evaluated by the extent of rust spreading in an actual object. For this reason, when the captured image of the object is used as it is, there is a possibility that sufficient evaluation accuracy cannot be obtained.

In this technical field, it is desired to improve the evaluation accuracy regarding the degree of rust.

The evaluation system according to one aspect of the present disclosure is a system that performs an evaluation on the degree of rust of an evaluation object using a captured image of the evaluation object. The evaluation system includes an image acquisition unit that acquires a captured image, a correction unit that generates an evaluation image by correcting the captured image, an evaluation unit that performs evaluation based on the evaluation image, and an evaluation result by the evaluation unit. An output unit for outputting. The correction unit extracts an evaluation area that is an image having a predetermined area on the surface of the evaluation object from the captured image, and generates an evaluation image based on the evaluation area.

An evaluation apparatus according to another aspect of the present disclosure is an apparatus that performs an evaluation on the degree of rust of an evaluation object using a captured image of the evaluation object. The evaluation device includes an image acquisition unit that acquires a captured image, a correction unit that generates an evaluation image by correcting the captured image, an evaluation unit that performs evaluation based on the evaluation image, and an evaluation result by the evaluation unit. An output unit for outputting. The correction unit extracts an evaluation area that is an image having a predetermined area on the surface of the evaluation object from the captured image, and generates an evaluation image based on the evaluation area.

The evaluation method according to still another aspect of the present disclosure is a method for performing an evaluation on the degree of rust of the evaluation object using a captured image of the evaluation object. The evaluation method outputs an evaluation result in a step of acquiring a captured image, a step of generating an evaluation image by correcting the captured image, a step of performing an evaluation based on the evaluation image, and a step of performing the evaluation Steps. In the step of generating the evaluation image, an evaluation region that is an image in a range having a predetermined area on the surface of the evaluation object is extracted from the captured image, and the evaluation image is generated based on the evaluation region.

An evaluation program according to still another aspect of the present disclosure is based on a step of acquiring a captured image of an evaluation object in a computer, a step of generating an evaluation image by correcting the captured image, and the evaluation image. A program for executing a step of performing an evaluation on the degree of rust of an evaluation object and a step of outputting an evaluation result in the step of performing the evaluation. In the step of generating the evaluation image, an evaluation region that is an image in a range having a predetermined area on the surface of the evaluation object is extracted from the captured image, and the evaluation image is generated based on the evaluation region.

A recording medium according to still another aspect of the present disclosure is based on a step of acquiring a captured image of an evaluation object in a computer, a step of generating an evaluation image by correcting the captured image, and an evaluation image. A computer-readable recording medium recording an evaluation program for executing the step of performing an evaluation on the degree of rust of the evaluation object and the step of outputting an evaluation result in the step of performing the evaluation. In the step of generating the evaluation image, an evaluation region that is an image in a range having a predetermined area on the surface of the evaluation object is extracted from the captured image, and the evaluation image is generated based on the evaluation region.

In these evaluation systems, evaluation apparatuses, evaluation methods, evaluation programs, and recording media, an evaluation area is extracted from a captured image of an evaluation object, and an evaluation image is generated based on the evaluation area. Then, evaluation regarding the degree of rust is performed based on the evaluation image, and an evaluation result is output. The evaluation area is an image of a range having a predetermined area on the surface of the evaluation object. Therefore, regardless of the conditions under which the captured image was captured (distance between the evaluation target and the imaging device, magnification, etc.), the range having the same area of the surface of the evaluation target is targeted. An assessment is made of the degree of rust that has occurred. As a result, it is possible to improve the evaluation accuracy regarding the degree of rust.

The captured image may include a marker area that is an image of a marker attached to the evaluation object. The correction unit may extract the evaluation area based on the marker area. In this case, the range in the captured image is determined based on the size of the marker region, and the determined range can be extracted as the evaluation region. For this reason, an evaluation area can be extracted without using other information. As a result, the evaluation system can be reduced in size.

The marker may have a shape that can identify the orientation of the marker. In this case, the orientation in the captured image can be identified by the marker area.

The marker may have an omnidirectional shape. In this case, since the marker shape is simple, the creation of the marker can be facilitated.

The marker may have an opening. The opening area of the opening may be equal to the above area. The correction unit may extract an image of the surface of the evaluation object exposed through the opening as an evaluation region. In this case, extraction of the evaluation area can be simplified.

The marker may be surrounded by a frame. A gap may be provided between the marker and the frame, and the color of the gap may be different from the color of the outer edge portion of the marker. In this case, a gap having a color different from the color of the outer edge portion of the marker exists between the frame and the marker. For this reason, even if the color around the marker area is similar to the color of the outer edge portion of the marker area, the outer edge of the marker area can be detected, so that the detection accuracy of the marker area is improved. As a result, it is possible to further improve the evaluation accuracy regarding the degree of rust.

The frame may have a shape in which the frame line is interrupted. In this case, in edge detection or the like, it is possible to reduce the possibility that a region surrounded by a frame is detected as a marker region, so that the detection accuracy of the marker region is improved. As a result, it is possible to further improve the evaluation accuracy regarding the degree of rust.

The evaluation system may further include a distance acquisition unit that acquires the distance between the imaging device that captured the captured image and the evaluation object. The correction unit may extract the evaluation region based on the distance. Since a range having a predetermined area on the surface of the evaluation object can be specified based on the distance between the imaging device and the evaluation object, the evaluation region can be extracted more accurately.

The correction unit may correct the color of the evaluation area based on the color of the reference area included in the captured image. The reference area may be an image of a reference body with a specific color. Even for the same evaluation object, the color tone of the captured image may change depending on the color tone of the light source used for photographing. Moreover, even for the same evaluation object, the brightness of the captured image may vary depending on the amount of light irradiation. In the above configuration, when the color of the reference region is different from the specific color, it is considered that the color in the captured image is affected by light. Therefore, for example, the influence of light can be reduced by correcting the color of the evaluation region so that the color of the reference region becomes a specific color (for example, the original color). Thereby, it becomes possible to further improve the evaluation accuracy regarding the degree of rust.

The correction unit may remove specular reflection from the evaluation area. When the evaluation object is irradiated with strong light, specular reflection may occur. When the evaluation object is imaged in this state, the captured image may be overexposed. Color information is lost in areas where whiteout occurs. For this reason, the color information can be restored by removing the specular reflection (whiteout). Thereby, it becomes possible to further improve the evaluation accuracy regarding the degree of rust.

The evaluation unit may perform evaluation using a neural network. By learning the neural network, it is possible to further improve the evaluation accuracy regarding the degree of rust.

The correction unit may adjust the size of the evaluation image to the size of the reference image used for learning of the neural network by expanding or reducing the evaluation region. In this case, the evaluation by the neural network can be appropriately performed.

The evaluation system may further include an extraction unit that extracts a candidate area where rust is generated from the captured image. The correcting unit may generate the evaluation image by correcting the candidate area. In this case, since it is not necessary to evaluate the entire surface of the evaluation object, it is possible to improve the evaluation efficiency related to the degree of rust.

Evaluation may include evaluation of rust. In this case, it becomes possible to improve the evaluation accuracy of the rust degree.

Evaluation may include evaluation of the degree of rust removal. In this case, it becomes possible to improve the evaluation accuracy of the degree of rust removal.

According to each aspect and each embodiment of the present disclosure, the evaluation accuracy regarding the degree of rust can be improved.

FIG. 1 is a configuration diagram schematically showing an evaluation system including an evaluation apparatus according to the first embodiment. FIG. 2 is a hardware configuration diagram of the user terminal shown in FIG. FIG. 3 is a hardware configuration diagram of the evaluation apparatus shown in FIG. FIG. 4 is a sequence diagram showing an evaluation method performed by the evaluation system shown in FIG. FIG. 5 is a flowchart showing in detail the correction process shown in FIG. 6A to 6F are diagrams showing examples of markers. FIG. 7 is a diagram for explaining distortion correction. FIG. 8 is a diagram for explaining extraction of an evaluation area using a marker. (A) of FIG. 9 is a figure for demonstrating the calculation method of the actual magnitude | size of an evaluation target object. (B) of FIG. 9 is a figure which shows the relationship between a focal distance and the focal distance of 35 mm conversion. 10A and 10B are diagrams for explaining the color correction. FIG. 11 is a diagram illustrating an example of a neural network. FIG. 12 is a diagram illustrating an example of the evaluation result. FIG. 13 is a diagram illustrating a display example of the evaluation result. FIG. 14 is a diagram illustrating an example of the evaluation result of the degree of rust removal. (A) and (b) of FIG. 15 is a figure which shows the example of correction of an evaluation result. FIG. 16 is a configuration diagram schematically showing an evaluation system including an evaluation apparatus according to the second embodiment. FIG. 17 is a sequence diagram showing an evaluation method performed by the evaluation system shown in FIG. FIG. 18 is a configuration diagram schematically showing an evaluation system including an evaluation apparatus according to the third embodiment. FIG. 19 is a flowchart showing an evaluation method performed by the evaluation system shown in FIG. FIG. 20 is a configuration diagram schematically illustrating an evaluation system according to a modification. (A) of FIG. 21 is a figure for demonstrating the detection of a candidate area | region. FIG. 21B is a diagram for explaining extraction of candidate regions. (A) to (d) of FIG. 22 are diagrams showing modifications of the marker. FIG. 23 is a diagram for describing a modification of the evaluation region extraction method. FIG. 24 is a diagram for explaining a modification of the evaluation region extraction method.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the description of the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted.

(First embodiment)
FIG. 1 is a configuration diagram schematically showing an evaluation system including an evaluation apparatus according to the first embodiment. An evaluation system 1 shown in FIG. 1 is a system that performs an evaluation on the degree of rust of an evaluation object. The evaluation regarding the degree of rust includes the degree of rust and the degree of rust removal. That is, the evaluation system 1 evaluates the degree of rust and the degree of rust removal of the evaluation object. Examples of the evaluation object include steel materials.

The rust degree is an index indicating the degree of rust generated on the surface of the evaluation object before the rust removal treatment. A thin black oxide film (mill scale) is formed on the surface of the steel material as it is heat-treated at a high temperature and cooled. The mill scale corresponds to black rust before red rust occurs in the steel material. When the mill scale peels off, red rust is generated in the steel base material, and further the red rust progresses, the base material is gradually eroded, and pitting corrosion that corrodes while forming holes is generated in the base material. The degree of rust includes mill scale, red rust, and the degree of pitting (progression). The degree of rust removal is an index indicating the degree of finishing of the rust removal treatment. In other words, the degree of rust removal is an index indicating the degree of deposits such as rust remaining on the surface of the evaluation object after the rust removal treatment. Examples of the rust removal treatment include blast treatment, water jet treatment, and flame cleaning. As the rust removal treatment, an electric tool, a file, a brush, a spatula, or the like may be used.

The evaluation system 1 includes one or a plurality of user terminals 10 and an evaluation device 20. The user terminal 10 and the evaluation device 20 are connected to be communicable with each other via a network NW. The network NW may be configured by either wired or wireless. Examples of the network NW include the Internet, a mobile communication network, and a WAN (Wide Area Network).

The user terminal 10 is a terminal device used by a user. The user terminal 10 captures the evaluation object to generate a captured image of the evaluation object, and transmits the captured image to the evaluation device 20. The user terminal 10 receives the evaluation result from the evaluation device 20 and outputs the evaluation result to the user. The user terminal 10 may be applied to a portable terminal that incorporates an imaging device, or may be applied to a device that can communicate with the imaging device. In the present embodiment, the user terminal 10 will be described using a portable terminal incorporating an imaging device. Examples of portable terminals include smartphones, tablet terminals, and notebook PCs (Personal Computers).

FIG. 2 is a hardware configuration diagram of the user terminal shown in FIG. As shown in FIG. 2, the user terminal 10 physically includes one or more processors 101, a main storage device 102, an auxiliary storage device 103, a communication device 104, an input device 105, an output device 106, and an imaging device 107. , And a computer including hardware such as the measurement device 108. As the processor 101, a processor having a high processing speed is used. Examples of the processor 101 include a GPU (Graphics Processing Unit) and a CPU (Central Processing Unit). The main storage device 102 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). Examples of the auxiliary storage device 103 include a semiconductor memory and a hard disk device.

The communication device 104 is a device that transmits and receives data to and from other devices via the network NW. An example of the communication device 104 is a network card. Encryption may be used for data transmission / reception via the network NW. That is, the communication device 104 may encrypt the data and transmit the encrypted data to another device. The communication device 104 may receive the encrypted data from another device and decrypt the encrypted data. For encryption, a common key cryptosystem such as Triple DES (Data Encryption Standard) and Rijndael, or a public key cryptosystem such as RSA and ElGamal may be used.

The input device 105 is a device used when the user operates the user terminal 10. Examples of the input device 105 include a touch panel, a keyboard, and a mouse. The output device 106 is a device that outputs various types of information to the user of the user terminal 10. Examples of the output device 106 include a display, a speaker, and a vibrator.

The imaging device 107 is a device for imaging (imaging). The imaging device 107 is, for example, a camera module. Specifically, the imaging apparatus 107 includes a plurality of optical system components such as a lens Cl and an imaging element Cs (see FIG. 9A), a plurality of control system circuits that drive and control the components, and an imaging element. And a circuit unit of a signal processing system that converts an electrical signal representing a captured image generated by Cs into an image signal that is a digital signal. The measuring device 108 is a device that measures a distance. Examples of the measuring device 108 include a millimeter wave radar, an ultrasonic sensor, a LIDAR (Light Detection and Ranging), a motion stereo, and a stereo camera.

Each function illustrated in FIG. 1 of the user terminal 10 is performed by causing each hardware device such as the main storage device 102 to read one or more predetermined computer programs under the control of one or more processors 101. This is realized by operating the hardware and reading and writing data in the main storage device 102 and the auxiliary storage device 103.

Functionally, the user terminal 10 includes an image acquisition unit 11, a distance acquisition unit 12, a correction unit 13, a transmission unit 14, a reception unit 15, an output unit 16, and a correction information acquisition unit 17. I have.

The image acquisition unit 11 is a part for acquiring a captured image including an evaluation object. The image acquisition unit 11 is realized by the imaging device 107, for example. The captured image may be a still image or a moving image. The captured image is acquired as, for example, image data indicating the pixel value of each pixel (pixel), but is expressed as a captured image for convenience of explanation. When the user terminal 10 does not include the imaging device 107, the image acquisition unit 11 receives, for example, a captured image captured by another device (for example, a terminal having a camera function) from the other device. To get. For example, when the image acquisition unit 11 receives a captured image from another device via the network NW, a portion that performs a captured image reception process (such as the communication device 104 in FIG. 2) functions as the image acquisition unit 11. The image acquisition unit 11 outputs the captured image to the correction unit 13.

The distance acquisition unit 12 is a part for acquiring the distance D1 between the imaging device 107 (specifically, the lens Cl) that generated the captured image and the evaluation object. The distance acquisition unit 12 is realized by the measurement device 108, for example. When the imaging device 107 is a depth camera, the distance D1 is obtained together with the captured image, so the distance acquisition unit 12 is realized by the imaging device 107. When the user terminal 10 does not have a measurement function, the distance acquisition unit 12 receives the distance D1 measured by another device (for example, a terminal having a measurement function) from the other device, for example, thereby calculating the distance D1. get. For example, when the distance acquisition unit 12 receives the distance D1 from another device via the network NW, the part that performs the reception process of the distance D1 (such as the communication device 104 in FIG. 2) functions as the distance acquisition unit 12. Alternatively, the user may measure the distance D1 using a scale or a measure, and input the distance D1 to the user terminal 10. In this case, a portion that receives an input of the distance D1 (such as the input device 105 in FIG. 2) functions as the distance acquisition unit 12. The distance acquisition unit 12 outputs the distance D1 to the correction unit 13.

The correction unit 13 is a part for generating an evaluation image by correcting the captured image. The correction unit 13 extracts an evaluation area from the captured image, and generates an evaluation image based on the evaluation area. The evaluation region is an image (image region) in a range having a predetermined area on the surface of the evaluation object. For example, the correction unit 13 performs size correction, distortion correction, color correction, specular reflection removal, noise removal, and blur correction on the captured image. Details of each correction process will be described later. The correction unit 13 outputs the evaluation image to the transmission unit 14.

The transmission unit 14 is a part for transmitting the evaluation image to the evaluation device 20. The transmission unit 14 transmits the evaluation image to the evaluation device 20 via the network NW. The transmission unit 14 further transmits the correction information acquired by the correction information acquisition unit 17 to the evaluation device 20. The transmission unit 14 is realized by the communication device 104, for example. The receiving unit 15 is a part for receiving an evaluation result from the evaluation device 20. The receiving unit 15 receives the evaluation result from the evaluation device 20 via the network NW. The receiving unit 15 is realized by the communication device 104, for example.

The output unit 16 is a part for outputting the evaluation result. The output unit 16 is realized by the output device 106, for example. When outputting an evaluation result in an output device such as a display included in another device, the output unit 16 transmits the evaluation result to the other device via the network NW, for example. In this case, a part (such as the communication device 104 in FIG. 2) that performs the evaluation result transmission process functions as the output unit 16.

The correction information acquisition unit 17 is a part for acquiring correction information of the evaluation result. For example, the user may correct the evaluation result using the input device 105 after confirming the evaluation result output by the output unit 16. At this time, the correction information acquisition unit 17 acquires the corrected evaluation result as correction information. The correction information acquisition unit 17 outputs the correction information to the transmission unit 14.

The evaluation apparatus 20 is an apparatus that performs an evaluation on the degree of rust of the evaluation object using a captured image (evaluation image) of the evaluation object. The evaluation device 20 is configured by an information processing device (server device) such as a computer, for example.

FIG. 3 is a hardware configuration diagram of the evaluation apparatus shown in FIG. As illustrated in FIG. 3, the evaluation device 20 may be physically configured as a computer including hardware such as one or more processors 201, a main storage device 202, an auxiliary storage device 203, and a communication device 204. . As the processor 201, a processor having a high processing speed is used. Examples of the processor 201 include a GPU and a CPU. The main storage device 202 includes a RAM, a ROM, and the like. Examples of the auxiliary storage device 203 include a semiconductor memory and a hard disk device.

The communication device 204 is a device that transmits and receives data to and from other devices via the network NW. An example of the communication device 204 is a network card. Encryption may be used for data transmission / reception via the network NW. In other words, the communication device 204 may encrypt the data and transmit the encrypted data to another device. The communication device 204 may receive the encrypted data from another device and decrypt the encrypted data. For encryption, a common key cryptosystem such as Triple DES and Rijndael or a public key cryptosystem such as RSA and ElGamal may be used.

Note that the communication device 204 may perform user authentication for determining whether the user of the user terminal 10 is a regular user or a non-regular user. In this case, the evaluation device 20 does not have to evaluate the degree of rust when the user is a regular user, and does not have to evaluate the degree of rust when the user is a non-regular user. For user authentication, for example, a user ID (identifier) and a password registered in advance are used. A one-time pad (one-time password) may be used for user authentication.

Each function shown in FIG. 1 of the evaluation device 20 is performed by causing each hardware item such as the main storage device 202 to read one or more predetermined computer programs under the control of one or more processors 201. This is realized by operating the hardware and reading and writing data in the main storage device 202 and the auxiliary storage device 203.

Functionally, the evaluation device 20 includes a reception unit 21, an evaluation unit 22, and a transmission unit 23.

The receiving unit 21 is a part for receiving an evaluation image from the user terminal 10. The receiving unit 21 receives an evaluation image from the user terminal 10 via the network NW. The receiving unit 21 further receives correction information from the user terminal 10. The receiving unit 21 is realized by the communication device 204, for example. The receiving unit 21 outputs the evaluation image and the correction information to the evaluation unit 22.

The evaluation unit 22 is a part for performing an evaluation on the degree of rust of the evaluation object based on the evaluation image. The evaluation unit 22 performs an evaluation on the degree of rust of the evaluation object using a neural network. The neural network may be a convolutional neural network (CNN) or a recurrent neural network (RNN). The evaluation unit 22 outputs the evaluation result to the transmission unit 23.

The transmission unit 23 is a part for transmitting the evaluation result to the user terminal 10. The transmission unit 23 transmits the evaluation result to the user terminal 10 via the network NW. The transmission unit 23 is realized by the communication device 204, for example. Since the transmission unit 23 outputs (transmits) the evaluation result to the user terminal 10, it can be regarded as an output unit.

Next, an evaluation method performed by the evaluation system 1 will be described with reference to FIGS. 4 to 15B. FIG. 4 is a sequence diagram showing an evaluation method performed by the evaluation system shown in FIG. FIG. 5 is a flowchart showing in detail the correction process shown in FIG. 6A to 6F are diagrams showing examples of markers. FIG. 7 is a diagram for explaining distortion correction. FIG. 8 is a diagram for explaining extraction of an evaluation area using a marker. (A) of FIG. 9 is a figure for demonstrating the calculation method of the actual magnitude | size of an evaluation target object. (B) of FIG. 9 is a figure which shows the relationship between a focal distance and the focal distance of 35 mm conversion. 10A and 10B are diagrams for explaining the color correction. FIG. 11 is a diagram illustrating an example of a neural network. FIG. 12 is a diagram illustrating an example of the evaluation result. FIG. 13 is a diagram illustrating a display example of the evaluation result. FIG. 14 is a diagram illustrating an example of the evaluation result of the degree of rust removal. (A) and (b) of FIG. 15 is a figure which shows the example of correction of an evaluation result.

The series of processes of the evaluation method illustrated in FIG. 4 is started, for example, when the user of the user terminal 10 photographs the evaluation object using the imaging device 107. First, the image acquisition unit 11 acquires a captured image of the evaluation object (step S01). For example, the image acquisition unit 11 acquires an image of the evaluation object generated by the imaging device 107 as a captured image. Then, the image acquisition unit 11 outputs the acquired captured image to the correction unit 13.

It should be noted that the marker MK may be attached to the evaluation object before acquiring the captured image of the evaluation object. The marker MK is used for correcting a captured image in image processing described later. The marker MK has a shape that can specify the direction of the marker MK. The marker MK is asymmetric in at least one of the vertical direction and the width direction. Specifically, as shown in FIGS. 6A to 6F, the marker MK includes a white area Rw and a black area Rb. In order to facilitate image processing to be described later, the marker MK has a square edge F1. The edge F1 is an edge of the region Rb. As shown in FIGS. 6B to 6F, the marker MK may be surrounded by a frame F2, and a gap Rgap may be provided between the frame F2 and the region Rb.

The marker MK is drawn on the sheet-like member. For example, the user of the user terminal 10 directly attaches the sheet-like member including the marker MK to the evaluation object. The user may attach a sheet-like member including the marker MK to the evaluation object using a UAV (Unmanned Aero Vehicle) or an expansion / contraction rod.

Note that the marker MK only needs to be composed of two or more regions with different colors. For example, the color assigned to the region Rw may not be white, but may be gray or the like. The color assigned to the region Rb may not be black, but may be a color having saturation. In the present embodiment, a marker MK shown in FIG.

Subsequently, the correction unit 13 corrects the captured image (step S02). As shown in FIG. 5, in the correction process in step S02, first, the correction unit 13 performs distortion correction in order to correct distortion of the captured image (step S21). The captured image may be distorted as compared to an image obtained by photographing the evaluation object from the front. For example, when the imaging device 107 is a depth camera, the distance between the imaging device 107 and each position of the evaluation object is obtained. In this case, the correction unit 13 performs distortion correction by converting the captured image into an image obtained by photographing the evaluation object from the front based on the distance between the imaging device 107 and each position of the evaluation object. . When the evaluation object is a structure having an aspect such as a pipe, the correction unit 13 may further perform an aspect correction as a distortion correction.

The correction unit 13 may perform distortion correction using the marker MK. The captured image of the evaluation object to which the marker MK is attached includes a marker region Rm that is an image (image region) of the marker MK. In this case, the correction unit 13 first extracts the marker region Rm from the captured image. The correcting unit 13 extracts the marker region Rm by performing object detection processing or edge detection processing on the captured image, for example. When the marker MK has a simple shape, edge detection processing may be used because edge detection processing has higher detection accuracy and processing speed than object detection processing.

Then, the correction unit 13 confirms whether or not the extracted marker region Rm is an image of the marker MK. For example, the correction unit 13 performs binarization processing on the marker region Rm after performing histogram averaging processing on the marker region Rm. Then, the correction unit 13 compares the binarized marker region Rm and the marker MK, and determines that the marker region Rm is an image of the marker MK when the two match. Thereby, the vertex coordinates of the marker MK in the captured image are acquired. The correction unit 13 determines that the marker region Rm is not an image of the marker MK when the two do not match, and extracts the marker region Rm again.

Then, the correction unit 13 calculates the direction of the marker MK in the captured image using the marker region Rm. Since the marker MK is asymmetric in at least one of the vertical direction and the width direction, the direction of the marker MK in the captured image can be calculated. Then, as illustrated in FIG. 7, the correction unit 13 performs projective transformation on the captured image so that the original shape of the marker MK is restored from the vertex coordinates and orientation of the marker MK in the captured image, thereby evaluating the correction. The captured image is converted into an image obtained by photographing the object from the front. Specifically, the correcting unit 13 sets the vertex Pm1 as the origin, the direction from the vertex Pm1 to the vertex Pm2 as the X1 axis direction, and the direction from the vertex Pm1 toward the vertex Pm4 as the Y1 axis direction. Then, the correction unit 13 restores the shape of the marker MK by converting the X1-Y1 coordinate system to the XY orthogonal coordinate system. Thereby, distortion correction is performed.

Subsequently, the correction unit 13 extracts the evaluation area Re from the captured image (step S22). Here, evaluation of a rust degree and a rust removal degree needs to be performed with respect to the range (evaluation range) which occupies the predetermined area among the surfaces of an evaluation target object. For this reason, the correction unit 13 determines the evaluation range in the captured image and extracts it as the evaluation region Re. That is, the evaluation area Re is an image area corresponding to the evaluation range in the captured image. Depending on the distance D1, the size (size) of the evaluation region Re in the captured image differs.

The correction unit 13 extracts the evaluation area Re from the captured image based on the marker area Rm. For example, it is assumed that the evaluation range is a size obtained by extending the length of each side of the marker MK by a predetermined magnification. In this case, as shown in FIG. 8, the correction unit 13 calculates the length of each side of the marker region Rm from the coordinates of the vertices Pm1 to Pm4 of the marker region Rm. And the correction | amendment part 13 specifies the magnitude | size of the evaluation range in a captured image by extending each edge | side of the marker area | region Rm by predetermined magnification (here 5 times). The correcting unit 13 extracts an area having the specified evaluation range size as an evaluation area Re from the captured image. The evaluation region Re may or may not include the marker region Rm.

The correcting unit 13 may extract the evaluation region Re from the captured image based on the distance D1 without using the marker MK. In this case, the distance acquisition unit 12 acquires the distance D1. Then, as illustrated in FIG. 9A, the correction unit 13 uses the formula (1), based on the distance D1, the focal length f1, and the width w1 of the imaging element Cs included in the imaging device 107. The actual width W is calculated. The width W is the horizontal length of the evaluation object imaged by the imaging device 107. When the entire evaluation object is not included in the captured image, the width W is the horizontal length of the evaluation object in the imaging range. The focal length f1 is a distance from the image sensor Cs to the lens Cl. The width w1 of the imaging element Cs of the imaging device 107 is the horizontal size (length) of the imaging element Cs and is determined in advance.

The correction unit 13 acquires the actual focal length f1 and width w1 from EXIF (Exchangeable Image File Format) information attached to the captured image. When the focal length f1 and the width w1 are not obtained from the EXIF information, the correction unit 13 acquires the name of the imaging device 107, and searches the database using the name of the imaging device 107, whereby the focal length f1 and the width are obtained. Get w1. The user may input the focal length f1 and the width w1 into the user terminal 10 using the input device 105.

In some cases, a width w2 in terms of 35 mm is obtained instead of the width w1 of the image sensor Cs. In this case, the correction unit 13 calculates the width w1 based on the focal length f1, the focal length f2 converted to 35 mm, and the width w2. As shown in FIG. 9B, the imaging field angle θ is equal to a field angle converted to 35 mm. For this reason, the following relational expression (2) is established. When formula (2) is arranged, formula (3) is obtained. The correcting unit 13 calculates the width w1 using Expression (3).

The correcting unit 13 may directly calculate the width W from the focal length f2 and the width w2 using Expression (4).

Note that the correction unit 13 acquires the focal length f2 from the EXIF information. When the focal length f2 is not obtained from the EXIF information, the correction unit 13 acquires the name of the imaging device 107 and searches the database using the name of the imaging device 107, thereby acquiring the focal length f2. The user may input the focal length f2 to the user terminal 10 using the input device 105.

The correction unit 13 calculates the width wp per pixel by dividing the width W by the number of pixels Npw in the width direction (horizontal direction) of the captured image, as shown in Expression (5). The correction unit 13 acquires the pixel number Npw from the EXIF information. The correction unit 13 may acquire the pixel number Npw by counting the pixels in the width direction in the captured image.

The correction unit 13 calculates the height H of the imaged portion in the same manner as the width W. The height H is the length in the vertical direction of the evaluation object imaged by the imaging device 107. When the entire evaluation object is not included in the captured image, the height H is the vertical length of the evaluation object in the imaging range. The correction unit 13 calculates the height hp per pixel by dividing the height H by the number of pixels Nph in the height direction (vertical direction) of the captured image, as shown in Expression (6). The correction unit 13 acquires the pixel number Nph from the EXIF information. The correction unit 13 may acquire the number of pixels Nph by counting the pixels in the height direction in the captured image.

The correction unit 13 determines the evaluation area Re in the captured image using the width wp and the height hp per pixel, and extracts the evaluation area Re. For example, the correction unit 13 calculates the number of pixels that the evaluation range in the captured image has in the width direction and the height direction. The correction unit 13 divides the width of the evaluation range by the width wp, and sets the division result as the number of pixels in the width direction. Similarly, the correction unit 13 divides the height of the evaluation range by the height hp, and sets the division result as the number of pixels in the height direction. Then, the correction unit 13 extracts an area having the number of pixels in the width direction and the number of pixels in the height direction as an evaluation area Re from the captured image.

Subsequently, the correction unit 13 corrects the size of the evaluation area Re (step S23). The size of the evaluation region Re can change according to the distance D1. For this reason, in the size correction, the correction unit 13 performs an expansion / contraction process on the evaluation region Re in order to match the predetermined evaluation size. The evaluation size is the size of a reference image (teacher data) used for learning of the neural network NN. In the expansion / contraction process, first, the correction unit 13 compares the size of the evaluation region Re with the evaluation size, and determines which of the enlargement process and the reduction process is performed.

The correction unit 13 performs enlargement processing when the size of the evaluation region Re is smaller than the evaluation size, and performs reduction processing when the size of the evaluation region Re is larger than the evaluation size. That is, the correction unit 13 enlarges or reduces the evaluation area Re to adjust the size of the evaluation image to the evaluation size. For the enlargement process, for example, a bilinear interpolation method is used. For the reduction processing, for example, an average pixel method is used. Other enlargement / reduction algorithms may be used for the enlargement process and the reduction process, but it is desirable that the image state be maintained even when the extension / reduction process is performed.

Subsequently, the correction unit 13 performs color correction of the evaluation area Re (step S24). Even for the same evaluation object, the brightness of the image may change depending on the shooting environment. Further, when the color of the light source used for photographing is different, the color of the image may be different. Color correction is performed to reduce the influence of the shooting environment. The correcting unit 13 corrects the color of the evaluation area Re based on the color of the reference area included in the captured image. The reference area is an image (image area) of a reference body with a specific color.

As shown in FIG. 10A, the region Rw of the marker MK can be used as the reference body. In this case, the color of the region Rw of the marker MK is measured in advance with a color meter or the like, and a reference value indicating the measured color is stored in a memory (not shown). As a value indicating a color, an RGB value, an HSV value, or the like is used. As illustrated in FIG. 10B, the correction unit 13 acquires the color value of the region Rw in the marker region Rm included in the captured image (evaluation region Re), and compares the acquired value with a reference value. Then, color correction is performed so that these differences are small (for example, zero). For color correction, gamma correction or the like is used. As color correction, a difference may be added to each pixel value (offset processing).

The marker MK may not be used as the reference body. In this case, even if color correction of the evaluation region Re is performed in the same manner as in the case of using the marker MK, a sample (for example, a gray board) whose color is measured in advance is used as a reference body and photographed together with the evaluation object. Good. The correction unit 13 may perform color correction based on the gray hypothesis.

Subsequently, the correction unit 13 removes the specular reflection from the evaluation region Re (step S25). Specular reflection may be caused when the evaluation object has a metallic luster. Depending on the state of the coating film of the evaluation object, specular reflection may be caused. In the image, the part that caused the specular reflection usually appears as strong white. That is, the portion where the specular reflection occurs causes whiteout in the image. After the color correction, the specular reflection portion can be detected as a white portion, so the correction unit 13 removes the specular reflection using the color corrected image (evaluation region Re).

Therefore, the correction unit 13 specifies the specular reflection part based on the pixel value of each pixel included in the evaluation region Re. For example, the correction unit 13 determines that the pixel is a part of the specular reflection portion when all of the RGB pixel values are larger than a predetermined threshold value. The correction unit 13 converts the pixel value into HSV, and specifies the specular reflection part by performing the same threshold processing on brightness (V) or both brightness (V) and saturation (S). May be.

Then, the correction unit 13 removes the specular reflection from the specular reflection portion and restores the original image information (pixel value). The correction unit 13 automatically interpolates (restores) the image information of the specular reflection portion with the image information in the vicinity of the specular reflection portion by, for example, a method using Navier-Stokes or a fast marching method of Alexander Telea. The correction unit 13 may restore the image information of the specular reflection part by learning various rust images in advance by machine learning. For machine learning, for example, GAN (Generative Adversarial Network) is used. Note that the correction unit 13 may restore the image information for a region in which the outer edge of the specular reflection portion is expanded (that is, a region that includes the specular reflection portion and is larger than the specular reflection portion).

Subsequently, the correction unit 13 removes noise from the evaluation region Re (step S26). The correction unit 13 removes noise from the evaluation region Re using, for example, a denoising filter (denoising function) such as a Gaussian filter and a low-pass filter.

Subsequently, the correction unit 13 performs blur correction on the evaluation area Re (step S27). When a user performs shooting using the user terminal 10, blurring such as camera shake may occur. The correction unit 13 performs image blur correction using, for example, a Wiener filter and a blind deconvolution algorithm.

The correction process in FIG. 5 is an example, and the correction process performed by the correction unit 13 is not limited to this. Some or all of steps S21 and S23 to S27 may be omitted. Steps S21 to S27 may be performed in any order. As described above, when the specular reflection removal is performed after the color correction, the specular reflection portion appears as strong white in the image, so that the specular reflection portion specifying accuracy is improved.

Further, as illustrated in FIG. 7, the correction unit 13 considers that the marker region Rm is configured by a plurality of blocks arranged in a grid, and uses the coordinates of the four vertices of the marker region Rm (marker MK). Thus, the vertex coordinates of each block may be obtained. Thereby, the correction | amendment part 13 can divide | segment the marker area | region Rm into a some block, and can handle it. For example, the correction unit 13 determines whether the marker region Rm is an image of the marker MK using each block. The correction unit 13 may use any block as a reference area used for color correction. Further, the correction unit 13 may calculate the degree of distortion of the captured image from the coordinates of each block and calibrate the imaging apparatus 107.

Subsequently, the correction unit 13 outputs the captured image corrected by the correction process in step S02 to the transmission unit 14 as an evaluation image, and the transmission unit 14 transmits the evaluation image to the evaluation device 20 via the network NW. (Step S03). At this time, the transmission unit 14 transmits the evaluation image to the evaluation device 20 together with a terminal ID that can uniquely identify the user terminal 10. For example, an IP (Internet Protocol) address may be used as the terminal ID. Then, the reception unit 21 receives the evaluation image transmitted from the user terminal 10 and outputs the evaluation image to the evaluation unit 22. The correction unit 13 may not output the evaluation image to the transmission unit 14 when the evaluation image is not clear. Further, as described above, the transmission unit 14 may encrypt the evaluation image and transmit the encrypted evaluation image to the evaluation device 20. In this case, the receiving unit 21 receives the encrypted evaluation image from the user terminal 10, decrypts the encrypted evaluation image, and outputs the evaluation image to the evaluation unit 22.

Subsequently, the evaluation unit 22 performs an evaluation on the degree of rust of the evaluation object based on the evaluation image (step S04). In this example, the evaluation unit 22 uses the neural network NN shown in FIG. 11 to evaluate the degree of rust and the degree of rust of the evaluation object. When the evaluation unit 22 receives the evaluation image, the evaluation unit 22 assigns an image ID that can uniquely identify the evaluation image to the evaluation image.

Neural network NN inputs an image for evaluation and outputs the matching rate of each category. As the category, rust grade, rust grade, and combinations thereof can be used. As the grade of rust, for example, grades A to D defined by JIS (Japanese Industrial Standards) Z0313 are used. In the order of grades A, B, C, and D, the degree of rust generated on the evaluation object increases. As grades of the degree of rust removal, for example, grades St2, St3, Sa1, Sa2, Sa2.5, and Sa3 defined in JIS Z0313 are used. In JIS Z0313, the grade corresponding to the state where the rust removal treatment is not performed is not defined, but in the following description, it is assumed as grade RAW for convenience. Other grades may be used as the rust grade and the rust removal grade. For example, ISO (International Organization for Standardization), SIS (Swedish Standard), SSPC (Steel Structures Painting Council), SPSS (Standard for the Preparation of Steel Surface to Painting), and NACE (the National Foundations) Grades defined in standards such as “Corrosion Engineers” may be used. As the grade of the degree of rust removal, a grade defined by standards such as ISO, SSPC, SPSS, and NACE may be used.

As shown in FIG. 12, in this embodiment, a combination of a rust grade and a rust grade is used as a category. The precision indicates the probability that the degree of rust of the evaluation object belongs to the category (grade). The higher the matching rate, the higher the possibility that the degree of rust of the evaluation object belongs to the category (grade). Note that Grade A + Grade St2, Grade A + Grade St3, Grade A + Grade Sa1, and Grade A + Grade Sa2 do not exist in any of the standards, but here, for the sake of convenience, these combinations will be described. These combinations may be set by the user or may be omitted.

The evaluation unit 22 may separate the evaluation image into one or a plurality of channels and use the image information (pixel value) of each channel as an input to the neural network NN. For example, the evaluation unit 22 separates the evaluation image into each component of the color space. When the RGB color space is used as the color space, the evaluation unit 22 separates the evaluation image into an R component pixel value, a G component pixel value, and a B component pixel value. When the HSV color space is used as the color space, the evaluation unit 22 separates the evaluation image into an H component pixel value, an S component pixel value, and a V component pixel value. The evaluation unit 22 may convert the evaluation image into a gray scale and use the converted image as an input to the neural network NN.

As shown in FIG. 11, the neural network NN includes an input layer L1, an intermediate layer L2, and an output layer L3. The input layer L1 is located at the entrance of the neural network NN, and M input values x _i (i is an integer from 1 to M) are input to the input layer L1. The input layer L1 includes a plurality of neurons 41. The neurons 41 are provided corresponding to the input values x _i , and the number of neurons 41 is equal to the total number M of the input values x _i . That is, the number of neurons 41 is equal to the total number of pixels included in each channel of the evaluation image. The i-th neuron 41 outputs the input value x _i to each neuron 421 of the first intermediate layer L21 of the intermediate layer L2. The input layer L1 includes a node 41b. The node 41b outputs a bias value b _j (j is an integer from 1 to M1) to each neuron 421.

The intermediate layer L2 is located between the input layer L1 and the output layer L3. The intermediate layer L2 is also called a hidden layer because it is hidden from the outside of the neural network NN. The intermediate layer L2 includes one or more layers. In the example shown in FIG. 11, the intermediate layer L2 includes a first intermediate layer L21 and a second intermediate layer L22. The first intermediate layer L21 has M1 neurons 421. At this time, as shown in the equation (7), the j-th neuron 421 adds a bias value b _j to the sum of values obtained by weighting the input values x _i by the weighting coefficient w _ij to obtain a calculated value. z _j is obtained. When the neural network NN is a convolutional neural network, the neuron 421 sequentially performs, for example, convolution, calculation using an activation function, and pooling. In this case, for example, a ReLU function is used as the activation function.

Then, the j-th neuron 421 outputs the calculated value z _j to each neuron 422 of the second intermediate layer L22. The first intermediate layer L21 includes a node 421b. The node 421b outputs a bias value to each neuron 422. Thereafter, each neuron performs the same calculation as that of the neuron 421 and outputs the calculated value to each neuron in the subsequent layer. The final-stage neuron (here, the neuron 422) of the intermediate layer L2 outputs the calculated value to each neuron 43 of the output layer L3.

The output layer L3 is located at the exit of the neural network NN and outputs an output value y _k (k is an integer from 1 to N). The output value y _k is assigned to each category, and is a value corresponding to the matching rate of that category. The output layer L3 includes a plurality of neurons 43. The neurons 43 are provided corresponding to the output values y _k , and the number of neurons 43 is equal to the total number N of output values y _k . That is, the number of neurons 43 is equal to the number of categories indicating the degree of rust. Each neuron 43 performs a calculation similar to that of the neuron 421 and calculates an activation function using the calculation result as an argument to obtain an output value y _k . Examples of the activation function include a softmax function, a ReLU function, a hyperbolic function, a sigmoid function, an identity function, and a step function. In the present embodiment, a softmax function is used. For this reason, each output value y _k is normalized so that the sum of the N output values y _k is 1. That is, the precision (%) is obtained by multiplying the output value y _k by 100.

Subsequently, for example, the evaluation unit 22 outputs the N output values y _k together with the image ID of the evaluation image to the transmission unit 23 as the evaluation result of the evaluation image. An array of N output values y _k is determined in advance, and each output value y _k is associated with one of the N categories. The evaluation unit 22 may use the largest output value among the N output values y _k as an evaluation result together with the category name or index (corresponding to “number” shown in FIG. 12) corresponding to the output value. . Here, an array of output values corresponding to the precision shown in FIG. 12 is output to the transmission unit 23 as an evaluation result. In this case, the user terminal 10 can determine how to output to the user.

And the transmission part 23 transmits an evaluation result to the user terminal 10 via the network NW (step S05). At this time, the transmission unit 23 identifies the destination user terminal 10 based on the terminal ID transmitted from the user terminal 10 together with the evaluation image, and transmits the evaluation result to the user terminal 10. Then, the reception unit 15 receives the evaluation result transmitted from the evaluation device 20 and outputs the evaluation result to the output unit 16. Note that the transmitting unit 23 may encrypt the evaluation result and transmit the encrypted evaluation result to the user terminal 10 as described above. In this case, the receiving unit 15 receives the encrypted evaluation result from the evaluation device 20, decrypts the encrypted evaluation result, and outputs the evaluation result to the output unit 16.

Subsequently, the output unit 16 generates output information for notifying the user of the evaluation result, and outputs the evaluation result to the user based on the output information (step S06).

In the display example of FIG. 13, a scatter diagram is used. The vertical axis of the scatter diagram shows the grade before rust removal (grade of rust), and the horizontal axis of the scatter plot shows the grade after rust removal (grade of rust removal). The color bar indicates the color corresponding to the matching rate. In the scatter diagram, the conformity rate of the category is indicated by a point plotted at the intersection of the grade before the rust removal treatment and the grade after the rust removal treatment. The color and size of the point are set according to the matching rate of the category corresponding to the point. The larger the precision of the category corresponding to the point, the larger the size of the point. The color of the point is set to a color corresponding to the matching rate of the category corresponding to the point.

Point P1 is located at the intersection of grade B and grade RAW and has a color corresponding to 15%. That is, the point P1 indicates that the grade of the grade before the rust removal treatment is grade B, and the conformity rate of the category where the grade after the rust removal treatment is grade RAW (a state where the rust removal treatment is not performed) is 15% Show. Similarly, the point P2 indicates that the grade of the grade before the rust removal treatment is grade B and the conformity rate of the category where the grade after the rust removal treatment is grade Sa1 is 5%. Similarly, the point P3 indicates that the grade of the grade before the rust removal treatment is grade D and the conformity rate of the category where the grade after the rust removal treatment is grade Sa1 is 85%.

In this display example, when the grade after the rust removal treatment of the category having the highest precision is grade RAW, the grade before the rust removal treatment of that category is used as the rust degree. On the other hand, when the grade after the rust removal treatment of the category having the highest matching rate is other than the grade RAW, the grade after the rust removal treatment of that category is used as the degree of rust removal.

In the display example of FIG. 13, even if the grade after the rust removal treatment is the same, it is classified into different categories depending on the grade before the rust removal treatment. However, when evaluating the degree of rust removal of the evaluation object, it is important how much rust has been removed. For this reason, the output part 16 may display the conformity rate for every grade after a rust removal process, integrating a precision rate for every grade after a rust removal process. For example, as shown in FIG. 14, the grade RAW compliance rate is the grade A + grade RAW grade, grade B + grade RAW grade, grade C + grade RAW grade, and grade D + grade RAW grade. Since it is the sum, it is 15% (= 0 + 15 + 0 + 0). The compliance rates of grades St2, St3, Sa2, Sa2.5, and Sa3 are all 0%, and the compliance rate of grade Sa1 is 85% (= 0 + 5 + 0 + 80). The output unit 16 may quantify the evaluation result by multiplying the matching rate for each grade after the rust removal processing by the classification coefficient of each grade and adding the multiplication results.

The output unit 16 may notify the user whether the derusting process is acceptable or not, using the evaluation result.

Similarly, when evaluating the degree of rust of the object to be evaluated, the output unit 16 may integrate the conformity ratio for each grade before the rust removal treatment and display the conformity ratio for each grade before the rust removal treatment. Good.

The output unit 16 may display the evaluation result as text. For example, the output unit 16 displays the grade name of the category having the highest relevance rate and the relevance rate in text. For example, the output unit 16 displays “result: [before rust removal treatment] grade D → [after rust removal treatment] grade Sa1 (80%)”. The output unit 16 may display the grade names of all categories and their matching rates in text.

The output unit 16 may output the evaluation result by voice or may output the evaluation result by vibration. The form of output by the output unit 16 may be set by the user.

Subsequently, the correction information acquisition unit 17 determines whether or not an evaluation result correction operation has been performed by the user. For example, after confirming the evaluation result output by the output unit 16, the user operates the input device 105 to display a screen for correcting the evaluation result.

For example, as shown in FIG. 15A, the user operates the input device 105 and designates a category on the scatter diagram using the pointer MP. That is, the user determines the category by visually inspecting the evaluation object, and the point Pu is plotted at the intersection of the grade before the rust removal and the grade after the rust removal corresponding to the category determined by the user. Is done.

As shown in FIG. 15B, the output unit 16 may display a category grade name and a radio button. In this case, when the user operates the input device 105 and checks the radio button using the pointer MP, the category determined by the user is selected. An object such as a drop-down list or a slider may be used for the user to select a category.

When the correction information acquisition unit 17 determines that the correction operation has not been performed, a series of processes of the evaluation method by the evaluation system 1 ends. On the other hand, when the correction information acquisition unit 17 determines that the correction operation has been performed by the input device 105, the correction information acquisition unit 17 acquires information indicating the corrected category as correction information together with the image ID of the evaluation image on which the correction operation has been performed. (Step S07).

Then, the correction information acquisition unit 17 outputs the correction information to the transmission unit 14, and the transmission unit 14 transmits the correction information to the evaluation device 20 via the network NW (Step S08). Then, the receiving unit 21 receives the correction information transmitted from the user terminal 10 and outputs the correction information to the evaluation unit 22. As described above, the transmission unit 14 may encrypt the correction information and transmit the encrypted correction information to the evaluation device 20. In this case, the reception unit 21 receives the encrypted correction information from the user terminal 10, decrypts the encrypted correction information, and outputs the correction information to the evaluation unit 22.

Subsequently, the evaluation unit 22 performs learning based on the correction information (step S09). Specifically, the evaluation unit 22 uses a set of the corrected category and the evaluation image as teacher data. The evaluation unit 22 may learn the neural network NN by any of online learning, mini-batch learning, and batch learning. Online learning is a method in which learning is performed using new teacher data each time new teacher data is acquired. Mini-batch learning is a method in which a certain amount of teacher data is taken as one unit and learning is performed using one unit of teacher data. Batch learning is a method of performing learning using all teacher data. For the learning, an algorithm such as back propagation is used. Note that learning of the neural network NN means updating the weighting coefficient and bias value used in the neural network NN to more optimal values.

As described above, a series of processes of the evaluation method by the evaluation system 1 is completed.

In addition, each function part in the user terminal 10 and the evaluation apparatus 20 is implement | achieved when the program module for implement | achieving each function is performed in the computer which comprises the user terminal 10 and the evaluation apparatus 20. FIG. The evaluation program including these program modules is provided by a computer-readable recording medium such as a ROM or a semiconductor memory. The evaluation program may be provided as a data signal via a network.

In the evaluation system 1, the evaluation device 20, the evaluation method, the evaluation program, and the recording medium described above, the evaluation area Re is extracted from the captured image of the evaluation object, and an evaluation image is generated based on the evaluation area Re. Then, evaluation regarding the degree of rust is performed based on the evaluation image, and an evaluation result is output. The evaluation region Re is an image of a range having a predetermined area on the surface of the evaluation object. For this reason, the range having the same area on the surface of the evaluation object is used as a target regardless of the conditions (distance between the evaluation object and the imaging device 107, magnification, etc.) where the captured image is captured. An assessment is made as to the degree of rust occurring within. As a result, it is possible to improve the evaluation accuracy regarding the degree of rust. Specifically, the rust degree and the degree of rust removal are evaluated based on the evaluation image. For this reason, it becomes possible to improve the evaluation precision of a rust degree and a rust removal degree. As described above, a general-purpose machine can be used as the imaging device 107, and the distance D1 between the imaging device 107 and the evaluation object can be freely changed.

The marker MK has a shape that can specify the direction of the marker MK. Therefore, a range having a predetermined area of the surface of the evaluation object in the captured image is determined based on the orientation and size of the marker region Rm included in the captured image, and the determined range is set as the evaluation region Re. Can be extracted. Thereby, the evaluation region Re can be extracted without using other information. Therefore, the evaluation system 1 can be reduced in size.

Based on the distance D1 between the imaging device 107 that generated the captured image and the evaluation object, a range having a predetermined area on the surface of the evaluation object can be specified, so that the evaluation region Re can be extracted more accurately. It becomes possible to do.

Even for the same evaluation object, the color tone of the captured image may change depending on the color tone of the light source used for shooting. Moreover, even for the same evaluation object, the brightness of the captured image may vary depending on the amount of light irradiation. For this reason, the color of the evaluation region Re is corrected based on the color of the reference region (for example, the region Rw in the marker region Rm) included in the captured image. When the color of the region Rw in the marker region Rm is different from the color (white) of the region Rw in the marker MK, it is considered that the color in the captured image is affected by light. Therefore, the color of the evaluation region Re is corrected so that the color of the region Rw in the marker region Rm becomes the color of the region Rw in the marker MK. Thereby, the influence of light can be reduced. As a result, it becomes possible to further improve the evaluation accuracy of the degree of rust and the degree of rust removal.

When the evaluation object is irradiated with strong light, specular reflection may occur, and when the evaluation object is imaged in this state, whiteout may occur in the captured image. Color information is lost in areas where whiteout occurs. For this reason, the color information can be restored by removing the specular reflection (out-of-white) from the evaluation region Re. Thereby, it becomes possible to further improve the evaluation accuracy of the degree of rust and the degree of rust removal.

Rust and rust removal are evaluated using the neural network NN. The rust pattern before the rust removal treatment and the rust pattern after the rust removal treatment are indefinite. For this reason, in general object detection, it is difficult to specify the position and state of an irregular object. Pattern recognition is not suitable for recognizing a myriad of patterns. On the other hand, by learning the neural network NN, it is possible to evaluate the rust degree and the degree of rust removal, and it is possible to further improve the evaluation accuracy of the rust degree and the degree of rust removal.

By enlarging or reducing the evaluation area Re, the size of the evaluation image is adjusted to the size of the reference image (teacher data) used for learning of the neural network NN. For this reason, evaluation by the neural network NN can be performed appropriately.

(Second Embodiment)
FIG. 16 is a configuration diagram schematically showing an evaluation system including an evaluation apparatus according to the second embodiment. An evaluation system 1A shown in FIG. 16 is mainly different from the evaluation system 1 in that a user terminal 10A is provided instead of the user terminal 10 and an evaluation device 20A is provided instead of the evaluation device 20.

The user terminal 10A is mainly different from the user terminal 10 in that the correction unit 13 is not provided and the captured image and the distance D1 are transmitted to the evaluation device 20A instead of the evaluation image. In the user terminal 10 A, the image acquisition unit 11 outputs the captured image to the transmission unit 14. The distance acquisition unit 12 outputs the distance D1 to the transmission unit 14. The transmission unit 14 transmits the captured image and the distance D1 to the evaluation device 20A.

The evaluation device 20A is mainly different from the evaluation device 20 in that the captured image and the distance D1 are received from the user terminal 10A instead of the evaluation image, and the correction unit 24 is further provided. The receiving unit 21 receives the captured image and the distance D1 from the user terminal 10A, and outputs the captured image and the distance D1 to the correction unit 24. In addition, since the receiving part 21 has acquired the captured image from the user terminal 10A, it can be regarded as an image acquisition part. The correction unit 24 has the same function as the correction unit 13. That is, the correction unit 24 extracts an evaluation area from the captured image, and generates an evaluation image based on the evaluation area. Then, the correction unit 24 outputs the evaluation image to the evaluation unit 22.

Next, an evaluation method performed by the evaluation system 1A will be described with reference to FIG. FIG. 17 is a sequence diagram showing an evaluation method performed by the evaluation system shown in FIG. First, the image acquisition unit 11 acquires a captured image of the evaluation object (step S31). For example, as in step S01, the image acquisition unit 11 acquires an image of the evaluation target generated by the imaging device 107 as a captured image.

Then, the image acquisition unit 11 outputs the acquired captured image to the transmission unit 14, and the transmission unit 14 transmits the captured image to the evaluation device 20A via the network NW (step S32). At this time, the transmission unit 14 transmits the captured image to the evaluation apparatus 20A together with a terminal ID that can uniquely identify the user terminal 10A. Then, the reception unit 21 receives the captured image transmitted from the user terminal 10 A and outputs the captured image to the correction unit 24. As described above, the transmission unit 14 may encrypt the captured image and transmit the encrypted captured image to the evaluation device 20A. In this case, the reception unit 21 receives the encrypted captured image from the user terminal 10 A, decrypts the encrypted captured image, and outputs the captured image to the correction unit 24.

Subsequently, the correction unit 24 corrects the captured image (step S33). Since the process of step S33 is the same as the process of step S02, the detailed description thereof is omitted. The correction unit 24 outputs the captured image corrected by the correction process in step S33 to the evaluation unit 22 as an evaluation image. Thereafter, the processing from step S34 to step S39 is the same as the processing from step S04 to step S09, and therefore detailed description thereof is omitted. As described above, a series of processes of the evaluation method by the evaluation system 1A is completed.

Note that each functional unit in the user terminal 10A and the evaluation device 20A is realized by executing a program module for realizing each function in a computer constituting the user terminal 10A and the evaluation device 20A. The evaluation program including these program modules is provided by a computer-readable recording medium such as a ROM or a semiconductor memory. The evaluation program may be provided as a data signal via a network.

Also in the evaluation system 1A, the evaluation device 20A, the evaluation method, the evaluation program, and the recording medium according to the second embodiment, the evaluation system 1, the evaluation device 20, the evaluation method, the evaluation program, and the recording medium according to the first embodiment Similar effects are produced. Further, in the evaluation system 1A, the evaluation device 20A, the evaluation method, the evaluation program, and the recording medium according to the second embodiment, the user terminal 10A does not have the correction unit 13, and therefore the processing load on the user terminal 10A can be reduced. it can. In the evaluation system 1A, the evaluation device 20A, the evaluation method, the evaluation program, and the recording medium according to the second embodiment, the captured image and the distance D1 are accumulated in the evaluation device 20A. This makes it easy to learn the neural network NN more effectively.

(Third embodiment)
FIG. 18 is a configuration diagram schematically showing an evaluation system including an evaluation apparatus according to the third embodiment. The evaluation system 1B shown in FIG. 18 is mainly different from the evaluation system 1 in that the user terminal 10B is provided instead of the user terminal 10 and the evaluation device 20 is not provided. The user terminal 10B is mainly different from the user terminal 10 in that the evaluation unit 18 is further provided, and the transmission unit 14 and the reception unit 15 are not provided. In this case, the user terminal 10B is also a stand-alone evaluation device.

In the user terminal 10B, the correction unit 13 outputs an evaluation image to the evaluation unit 18. The correction information acquisition unit 17 outputs the correction information to the evaluation unit 18. The evaluation unit 18 has the same function as the evaluation unit 22. That is, the evaluation unit 18 performs an evaluation on the degree of rust of the evaluation object based on the evaluation image. Then, the evaluation unit 18 outputs the evaluation result to the output unit 16.

Next, an evaluation method performed by the evaluation system 1B (user terminal 10B) will be described with reference to FIG. FIG. 19 is a flowchart showing an evaluation method performed by the evaluation system shown in FIG.

First, the image acquisition unit 11 acquires a captured image of the evaluation object as in step S01 (step S41). Then, the image acquisition unit 11 outputs the captured image to the correction unit 13. Subsequently, the correction unit 13 corrects the captured image (step S42). Since the process of step S42 is the same as the process of step S02, the detailed description thereof is omitted. Then, the correction unit 13 outputs the captured image corrected by the correction process in step S42 to the evaluation unit 18 as an evaluation image.

Subsequently, the evaluation unit 18 performs an evaluation on the degree of rust of the evaluation object based on the evaluation image (step S43). Since the process of step S43 is the same as the process of step S04, its detailed description is omitted. Then, the evaluation unit 18 outputs the evaluation result to the output unit 16. Subsequently, the output unit 16 generates output information for notifying the user of the evaluation result, and outputs the evaluation result to the user based on the output information (step S44). Since the process of step S44 is the same as the process of step S06, its detailed description is omitted.

Subsequently, the correction information acquisition unit 17 determines whether or not an evaluation result correction operation has been performed by the user (step S45). When the correction information acquisition unit 17 determines that the correction operation has not been performed (step S45: NO), the series of processes of the evaluation method by the evaluation system 1B ends. On the other hand, when it is determined that the correction operation has been performed (step S45: YES), the correction information acquisition unit 17 displays the information indicating the corrected category together with the image ID of the evaluation image on which the correction operation has been performed. Get as. Then, the correction information acquisition unit 17 outputs the correction information to the evaluation unit 18.

Subsequently, the evaluation unit 18 performs learning based on the correction information (step S46). Since the process of step S46 is the same as the process of step S09, detailed description thereof is omitted. As described above, a series of processes of the evaluation method by the evaluation system 1B is completed.

Note that each functional unit in the user terminal 10B is realized by executing a program module for realizing each function in a computer constituting the user terminal 10B. The evaluation program including these program modules is provided by a computer-readable recording medium such as a ROM or a semiconductor memory. The evaluation program may be provided as a data signal via a network.

Also in the evaluation system 1B, the user terminal 10B, the evaluation method, the evaluation program, and the recording medium according to the third embodiment, the evaluation system 1, the evaluation device 20, the evaluation method, the evaluation program, and the recording medium according to the first embodiment Similar effects are produced. Further, in the evaluation system 1B, the user terminal 10B, the evaluation method, the evaluation program, and the recording medium according to the third embodiment, it is not necessary to transmit / receive data via the network NW, and therefore, accompanying the communication via the network NW. There is no time lag, and the response speed can be improved. Further, it becomes possible to reduce traffic and communication charges of the network NW.

Note that the evaluation system, the evaluation device, the evaluation method, the evaluation program, and the recording medium according to the present disclosure are not limited to the above embodiment.

For example, when the

correction units

13 and 24 do not use the distance D1 in the correction process, the

user terminals

10, 10A, and 10B may not include the distance acquisition unit 12.

Further, when the evaluation result is not corrected by the user, the

user terminals

10, 10 A, and 10 B may not include the correction information acquisition unit 17.

Also, in the neural network NN, batch normalization may be performed. Batch normalization is a process of converting the output value of each layer so that the variance is constant. In this case, since it is not necessary to use a bias value, nodes (node 41b, node 421b, etc.) that output the bias value can be omitted.

Further, the

evaluation units

18 and 22 may perform an evaluation on the degree of rust based on the evaluation image using a method other than the neural network.

Further, the output unit 16 may output the evaluation result to a memory (storage device) (not shown) and store the evaluation result in the memory. For example, the output unit 16 creates management data in which an evaluation result is associated with a management number that can uniquely identify an evaluation result, a date on which the evaluation is performed, and the evaluation result, and stores the management data.

Evaluation system

1, 1A, 1B may be used as a diagnostic tool. Here, a modified example in which the evaluation system 1 is used as a diagnostic tool will be described. FIG. 20 is a configuration diagram schematically illustrating an evaluation system according to a modification. In a steel material being coated, for example, rust is generated under the coating film due to moisture passing through the coating film, whereby the coating film is lifted and the coating film is peeled off. The coating film peeling is a state in which the coating film (coating film) applied to the base (metal substrate) is peeled off in a scaly manner. When the coating film is peeled off, the base is exposed, and rust is more likely to occur. Steel materials such as weathering steel can be used without painting. In such an unpainted steel material, since there is no coating film, coating film peeling due to rust does not occur, but rust may be partially generated in the steel material. For example, when the evaluation object is a steel bridge, it is inefficient to evaluate rust on the entire surface of the evaluation object.

For this reason, the evaluation system 1 of the modified example shown in FIG. 20 is a region in which rust occurs (candidate region Rc) from a captured image obtained by imaging the entire evaluation object or part of the evaluation object. Is extracted and evaluated. In the evaluation system 1 of the present modification, the evaluation related to the degree of rust includes the degree of rust distribution in the evaluation object.

The modified evaluation system 1 further includes an extraction unit 19. The extraction unit 19 is a part for extracting the candidate region Rc from the captured image G. As shown in FIG. 21A, the extraction unit 19 screens the captured image G, and detects a region including coating film peeling (rust) as a candidate region Rc. The extraction unit 19 detects the candidate region Rc, for example, by performing object detection processing. For the object detection processing, an image recognition technique such as a method using a cascade classifier, a support vector machine (SVM), and a neural network is used. For example, the extraction unit 19 detects the candidate region Rc by distinguishing between dirt and coating film peeling (rust). That is, the extraction unit 19 does not detect a region with dirt as the candidate region Rc.

21B, the extraction unit 19 extracts a region Rex including the candidate region Rc, and outputs the region Rex to the correction unit 13. If the candidate area Rc is larger than the predetermined size, the extraction unit 19 divides the candidate area Rc into a plurality of areas, and outputs a plurality of areas Rex including the divided areas to the correction unit 13. The correction unit 13 generates the evaluation image by performing the above-described correction on the region Rex.

As the category used in the evaluation unit 22, for example, a combination of a grade indicating the magnitude of coating film peeling and a grade indicating the degree of coating film peeling can be used. For example, “overall”, “spot”, and “pinpoint” defined in the SSPC standard are used as the grade of the magnitude of coating film peeling. As the grade indicating the degree of coating film peeling, for example, “no abnormality” and “starting peeling” are used.

Also in the evaluation system 1 of the above modification, the same effect as the evaluation system 1 according to the first embodiment described above is exhibited. In the evaluation system 1 of the modified example, a candidate area Rc in which rust is generated is extracted from the captured image G, and an evaluation regarding the degree of rust is performed using the candidate area Rc. For this reason, since it is not necessary to evaluate the whole surface of an evaluation target object, it becomes possible to improve the evaluation efficiency regarding the degree of rust.

The shape of the marker MK is not limited to a square. The shape of the marker MK may be a rectangle.

In the above embodiment, the marker MK has a shape that can specify the orientation of the marker MK, but the shape of the marker MK is not limited to a shape having directivity. The shape of the marker MK may be an omnidirectional shape. For example, as illustrated in FIG. 22A, the shape of the region Rb may be a square, and the shape of the region Rw may be a square that is slightly smaller than the region Rb. The center point of the region Rb and the center point of the region Rw may be overlapped, and each side of the region Rb and each side of the region Rw may be arranged in parallel to each other.

22B, the marker MK may have an opening Hm. The opening Hm is a through hole that penetrates the sheet-like member on which the marker MK is drawn. The opening area of the opening Hm is equal to the area of the evaluation range. When the marker MK is attached to the evaluation object, the surface of the evaluation object is exposed through the opening Hm. Since the opening area of the opening Hm is equal to the area of the evaluation range, the

correction units

13 and 24 extract an image of the surface of the evaluation object exposed through the opening Hm as the evaluation region Re. Thereby, the extraction of the evaluation area Re can be simplified.

Note that the opening area of the opening Hm may be larger than the area of the evaluation range. In this case, the

correction units

13 and 24 may extract a region exposed through the opening Hm from the captured image as preprocessing for extracting the evaluation region Re. And the correction |

amendment parts

13 and 24 may extract the evaluation area | region Re from the extracted area | region.

As described above, a range having a predetermined area of the surface of the evaluation object in the captured image is determined based on the size of the marker region Rm included in the captured image regardless of the shape of the marker MK, and the evaluation region Can be extracted as Re. Thereby, the evaluation region Re can be extracted without using other information. Therefore, the

evaluation systems

1, 1A, 1B can be reduced in size.

As shown in FIG. 22A, when the marker MK has an omnidirectional shape, the marker MK has a simple shape, so that the creation of the marker MK can be facilitated. Further, since the orientation of the marker MK is not important, the user can easily photograph the evaluation object.

When the marker MK not surrounded by the frame F2 is used, the boundary between the marker region Rm and the region to be evaluated may become unclear due to light reflection or the like. In such a case, the edge may not be detected by the edge detection process. In object detection, if the determination threshold is too small, false detections increase, and if the determination threshold is excessively large, detection omissions increase. In addition, the direction (angle) of the marker region Rm cannot be obtained by object detection itself. Furthermore, when the edge enhancement process is performed after the marker area Rm is extracted by the object detection process, and the edge detection process is further performed, the detection accuracy is improved, but the color of the outer edge portion of the marker area Rm and the periphery of the marker area Rm In the case where the color of the color hardly changes, detection omission may occur.

On the other hand, in the markers MK shown in FIGS. 6B to 6F and FIGS. 22C and 22D, the marker MK is surrounded by the frame F2, and the frame F2 and the region Rb are between. A gap Rgap is provided. The gap Rgap surrounds the region Rb along the edge F1. The color of the gap Rgap is different from the color of the outer edge portion (that is, the region Rb) of the marker MK. Therefore, even if the color around the marker area Rm (outside the frame F2) is similar to the color of the outer edge portion (area Rb) of the marker area Rm, the outer edge (edge F1) of the marker area Rm is clear. Therefore, the outer edge of the marker region Rm can be detected. For example, when the edge enhancement process is performed after the marker area Rm is extracted by the object detection process and the edge detection process is further performed, the vertices (vertices Pm1 to Pm4) of the area Rb can be detected more reliably. Therefore, the marker region Rm can be extracted with high speed and high accuracy. As a result, it is possible to further improve the evaluation accuracy regarding the degree of rust. Note that the distance between the frame F2 and the region Rb (the width of the gap Rgap) may be, for example, one tenth or more of one side of the marker MK in order to secure the gap Rgap. The distance between the frame F2 and the region Rb (the width of the gap Rgap) may be, for example, less than half of one side of the marker MK in consideration of ease of use of the marker MK.

Further, as shown in FIGS. 22C and 22D, the frame F2 may not be a frame that completely surrounds the marker MK. That is, the missing part Fgap may be provided in the frame F2. For example, the frame F2 is not limited to a solid line and may be a broken line. In this case, the frame F2 has a shape in which the frame line of the frame F2 is interrupted. When the missing part Fgap is provided in the frame F2, it is possible to reduce the possibility that the region surrounded by the frame F2 is detected as the marker region Rm by edge detection processing or the like. Therefore, the detection accuracy of the marker region Rm Will improve. That is, since the possibility that the vertex of the frame F2 is detected can be reduced, the vertex of the marker region Rm (region Rb) can be more reliably detected. As a result, it is possible to further improve the evaluation accuracy regarding the degree of rust.

As illustrated in FIG. 23, when the

correction units

13 and 24 extract the evaluation region Re from the captured image G, the

correction units

13 and 24 randomly determine the evaluation region Re from the captured image G, and extract the determined evaluation region Re. Also good. In this case, first, the

correction units

13 and 24 obtain the maximum value of coordinates that can be taken by the reference point Pr of the evaluation region Re. The reference point Pr is one of the four vertices of the evaluation area Re, and here is the vertex closest to the origin of the XY coordinates among the four vertices of the evaluation area Re. For example, if the length of one side of the evaluation region Re is 100 pixels, the maximum value _{y Crop_max} maximum value _{x Crop_max} and Y coordinates of the X-coordinate of the reference point Pr, is expressed by the following equation (8). Note that the vertex Pg1 of the captured image G is located at the origin (0, 0), the vertex Pg2 is located at (X _g , 0), the vertex Pg3 is located at (X _g , Y _g ), and the vertex Pg4 is ( 0, Y _g ).

The

correction units

13 and 24 randomly determine the coordinates (x _crop , y _crop ) of the reference point of the evaluation region Re using Expression (9). The function random (minimum value, maximum value) is a function that returns an arbitrary value included in the range from the minimum value to the maximum value.

The

correction units

13 and 24 may determine the coordinates of the reference point of the evaluation area Re again when the determined evaluation area Re and the marker area Rm overlap.

As illustrated in FIG. 24, the

correction units

13 and 24 may specify the extraction direction for the marker region Rm and extract the evaluation region Re from the captured image G. In this case, first, the

correction units

13 and 24 calculate the coordinates (x _cg , y _cg ) of the center position Cg of the captured image G and the coordinates (x _cm , y _cm ) of the center position Cm of the marker region Rm. To do. And the correction |

amendment parts

13 and 24 calculate the vector V which goes to center position Cg from center position Cm, as shown by Formula (10).

The

correction units

13 and 24 determine the position of the evaluation region Re in the direction indicated by the vector V from the marker region Rm. For example, the

correction units

13 and 24 determine the position of the evaluation region Re so that the reference point Pr of the evaluation region Re is located in the direction indicated by the vector V from the center position Cm. Here, the reference point Pr is the vertex closest to the marker region Rm among the four vertices of the evaluation region Re. For example, the

correction units

13 and 24 determine the position of the evaluation region Re so as not to overlap with the marker region Rm. Specifically, the

correction units

13 and 24 include the coordinates (x _{crop_max} , y _{crop_max} ) of the reference point Pr_max farthest from the marker region Rm and the reference closest to the marker region Rm among the coordinates that the reference point Pr can take. The coordinates (x _{crop_min} , y _{crop_min} ) of the point Pr_min are calculated. Then, the correcting

units

13 and 24 determine the position of the evaluation region Re so that the reference point Pr is positioned on the line segment between these two points.

DESCRIPTION OF

SYMBOLS

1,1A, 1B ...

Evaluation system

10, 10A, 10B ... User terminal, 11 ... Image acquisition part, 13, 24 ... Correction | amendment part, 16 ... Output part, 17 ... Correction information acquisition part, 18, 22 ... Evaluation part, DESCRIPTION OF SYMBOLS 19 ...

Extraction part

20, 20A ... Evaluation apparatus, 21 ... Reception part (image acquisition part), 23 ... Transmission part (output part), D1 ... Distance, G ... Captured image, MK ... Marker, NN ... Neural network, Rc ... candidate area, Re ... evaluation area, Rm ... marker area.

Claims

An evaluation system for performing an evaluation on the degree of rust of the evaluation object using a captured image of the evaluation object,
An image acquisition unit for acquiring the captured image;
A correction unit that generates an evaluation image by correcting the captured image;
An evaluation unit that performs the evaluation based on the evaluation image;
An output unit for outputting an evaluation result by the evaluation unit;
With
The correction unit extracts, from the captured image, an evaluation region that is an image having a predetermined area on the surface of the evaluation object, and generates the evaluation image based on the evaluation region. .
The captured image includes a marker region that is an image of a marker attached to the evaluation object,
The evaluation system according to claim 1, wherein the correction unit extracts the evaluation region based on the marker region.
The evaluation system according to claim 2, wherein the marker has a shape capable of specifying the orientation of the marker.
The evaluation system according to claim 2, wherein the marker has an omnidirectional shape.
The marker has an opening,
The opening area of the opening is equal to the area,
The evaluation system according to any one of claims 2 to 4, wherein the correction unit extracts an image of the surface of the evaluation object exposed through the opening as the evaluation region.
The marker is surrounded by a frame;
A gap is provided between the marker and the frame;
The evaluation system according to any one of claims 2 to 5, wherein a color of the gap is different from a color of an outer edge portion of the marker.
The evaluation system according to claim 6, wherein the frame has a shape in which a frame line is interrupted.
A distance acquisition unit that acquires a distance between the imaging device that captured the captured image and the evaluation object;
The evaluation system according to claim 1, wherein the correction unit extracts the evaluation region based on the distance.
The correction unit corrects the color of the evaluation area based on the color of the reference area included in the captured image,
The evaluation system according to any one of claims 1 to 8, wherein the reference region is an image of a reference body with a specific color.
The evaluation system according to any one of claims 1 to 9, wherein the correction unit removes specular reflection from the evaluation region.
The evaluation system according to any one of claims 1 to 10, wherein the evaluation unit performs the evaluation using a neural network.
The evaluation system according to claim 11, wherein the correction unit adjusts the size of the evaluation image to the size of a reference image used for learning of the neural network by expanding or reducing the evaluation region.
An extraction unit that extracts a candidate area where rust is generated from the captured image;
The evaluation system according to any one of claims 1 to 12, wherein the correction unit generates the evaluation image by correcting the candidate region.
The evaluation system according to any one of claims 1 to 13, wherein the evaluation includes an evaluation of a degree of rust.
The evaluation system according to any one of claims 1 to 14, wherein the evaluation includes an evaluation of a degree of rust removal.
An evaluation apparatus that performs an evaluation on the degree of rust of the evaluation object using a captured image of the evaluation object,
An image acquisition unit for acquiring the captured image;
A correction unit that generates an evaluation image by correcting the captured image;
An evaluation unit that performs the evaluation based on the evaluation image;
An output unit for outputting an evaluation result by the evaluation unit;
With
The correction unit extracts, from the captured image, an evaluation region that is an image having a predetermined area on the surface of the evaluation object, and generates the evaluation image based on the evaluation region .
An evaluation method for evaluating the degree of rust of the evaluation object using a captured image of the evaluation object,
Obtaining the captured image;
Generating an image for evaluation by correcting the captured image;
Performing the evaluation based on the evaluation image;
Outputting an evaluation result in the step of performing the evaluation;
With
In the step of generating the evaluation image, an evaluation region that is an image having a predetermined area on the surface of the evaluation object is extracted from the captured image, and the evaluation image is extracted based on the evaluation region. The evaluation method that is generated.
On the computer,
Obtaining a captured image of the evaluation object;
Generating an image for evaluation by correcting the captured image;
Performing an evaluation on the degree of rust of the evaluation object based on the evaluation image;
Outputting an evaluation result in the step of performing the evaluation;
An evaluation program for executing
In the step of generating the evaluation image, an evaluation region that is an image having a predetermined area on the surface of the evaluation object is extracted from the captured image, and the evaluation image is extracted based on the evaluation region. Generated evaluation program.
On the computer,
Obtaining a captured image of the evaluation object;
Generating an image for evaluation by correcting the captured image;
Performing an evaluation on the degree of rust of the evaluation object based on the evaluation image;
Outputting an evaluation result in the step of performing the evaluation;
A computer-readable recording medium having recorded thereon an evaluation program for executing
In the step of generating the evaluation image, an evaluation region that is an image having a predetermined area on the surface of the evaluation object is extracted from the captured image, and the evaluation image is extracted based on the evaluation region. A recording medium to be generated.