JP6577243B2 - Surface defect evaluation apparatus, surface defect inspection system, and surface defect inspection method - Google Patents

Description

  Embodiments described herein relate generally to a surface defect evaluation apparatus, a surface defect inspection system, and a surface defect inspection method.

  Non-destructive inspection techniques such as visual inspection, ultrasonic inspection, and eddy current inspection (ECT) are used in inspection of many devices in a power plant. For example, flaw detection by ultrasonic flaw detection or eddy current flaw inspection (ECT: Eddy Current Testing) is mainly used for internal defects of stationary blades of gas turbines.

  At this time, the internal defect of the stationary blade is automatically detected and recorded by ultrasonic inspection or eddy current inspection. On the other hand, defects such as damage due to surface cracks or thinning (hereinafter referred to as surface defects) are manually recorded by visual inspection. Therefore, a great amount of repair time and cost are generated, including crack repair when a crack is generated.

  As a method for detecting a surface defect, a method by image processing and a method by shape data comparison are known. In the method based on the shape data comparison, when a measurement point group does not exist inside the surface defect, a detection defect or interruption of the surface defect occurs. In addition, in order to acquire high-resolution shape data, it is necessary to approach the object, and the range that can be measured at one time is narrowed.

  Furthermore, since the method of acquiring shape data basically uses the principle of triangulation, a blind spot may occur. For example, one side or an end of a surface defect contour may be missing, which needs to be considered. There is a problem that there is. On the other hand, in the method based on image processing, it may be difficult to recognize surface defects due to the influence of illumination, which may cause surface defect detection omission.

Japanese Patent No. 5218723 JP 2014-202534 A

  In order to perform the shape comparison between the reference shape data and the measurement result shape data as described above, first, it is necessary to perform position adjustment. In this position adjustment method, the position adjustment shift occurs due to the difference in the shape of the reference data and the measurement data due to secular deformation. For this reason, there is a possibility that a point having a different surface defect detection position or a portion that is not a surface defect is detected. Or the part which has a deformation | transformation by a thinning part produces a difference in a wide range, and the position adjustment is affected.

  Another problem is that the surface defect is buried in the thinned portion due to the difference in a wide range and cannot be detected. Furthermore, since there is a penetrating part in the crack, it may be affected by the type of the open crack.

  Therefore, an object of the present invention is to enable an automatic inspection that can increase the detection rate and measure the size of the surface defect at the level to be described in the inspection record in the surface defect inspection.

In order to achieve the above-described object, the surface defect evaluation apparatus according to the present embodiment includes a two-dimensional image of the appearance of an object to be inspected acquired by the image acquisition unit, and the shape measured by the shape measuring unit. a three-dimensional shape of the object, an input unit that accepts as input, was measured to serving as a reference standard contour data of the determination in the Ah Ri secular variations or defects in the original data for the object the 3 Surface defects existing on the surface of the object based on the acquired image data and shape data, which are information on the two-dimensional image, after adjusting the position of the shape data that is information on the two-dimensional shape And an arithmetic unit that acquires information on the information.
In addition, the surface defect evaluation apparatus according to the present embodiment includes a two-dimensional image of the appearance of an object to be inspected acquired by the image acquisition unit, and a three-dimensional image of the object measured by the shape measurement unit. An input unit that accepts an arbitrary shape as an input, reference outline data that is original data regarding the object, and position adjustment of shape data that is information about the measured three-dimensional shape are acquired and acquired. A surface defect evaluation apparatus comprising: a calculation unit that acquires information on surface defects existing on the surface of the object based on image data that is information on the two-dimensional image and the shape data, In the position adjustment, the calculation unit uses at least three points in the shape data of the object and the reference outline data, and adjusts the position by matching these points. And performing.
In addition, the surface defect evaluation apparatus according to the present embodiment includes a two-dimensional image of the appearance of an object to be inspected acquired by the image acquisition unit, and a three-dimensional image of the object measured by the shape measurement unit. An input unit that accepts an arbitrary shape as an input, reference outline data that is original data regarding the object, and position adjustment of shape data that is information about the measured three-dimensional shape are acquired and acquired. A surface defect evaluation apparatus comprising: a calculation unit that acquires information on surface defects existing on the surface of the object based on image data that is information on the two-dimensional image and the shape data, In the position adjustment, the calculation unit may include at least three regions from which the shape change amount based on the shape data of the object and the reference outer shape data is a predetermined threshold value or less. And performing position adjustment by setting the reference position.
In addition, the surface defect evaluation apparatus according to the present embodiment includes a two-dimensional image of the appearance of an object to be inspected acquired by the image acquisition unit, and a three-dimensional image of the object measured by the shape measurement unit. An input unit that accepts an arbitrary shape as an input, reference outline data that is original data regarding the object, and position adjustment of shape data that is information about the measured three-dimensional shape are acquired and acquired. A surface defect evaluation apparatus comprising: a calculation unit that acquires information on surface defects existing on the surface of the object based on image data that is information on the two-dimensional image and the shape data, In the position adjustment, the calculation unit specifies a region in which the shape of both the shape data of the target object and the reference external shape data matches by matching. To.

Further, the present embodiment is a surface defect inspection system including a surface data acquisition device and a surface defect evaluation device, the surface defect inspection system including a surface data acquisition device and a surface defect evaluation device, wherein the surface data The acquisition apparatus includes an image acquisition unit that acquires a two-dimensional image of the appearance of an object to be inspected, and a shape measurement unit that acquires a three-dimensional shape of the object, and the surface defect evaluation device, the input section for receiving as input two-dimensional image and the three-dimensional shape, reference contour as a reference for determination of the Ah Ri secular variations or defects in the original data for the object The position adjustment of the shape data which is information about the data and the measured three-dimensional shape is performed, and the acquired image data which is information about the two-dimensional image and the shape data An arithmetic unit for obtaining the information relating to the surface defects present on the surface of the object Zui characterized in that it comprises a.

In the surface defect inspection method according to the present embodiment, the input unit of the surface defect evaluation apparatus accepts a two-dimensional image of the appearance of the target object acquired by the image acquisition unit; input unit, shape as receiving step of receiving the three-dimensional shape of the object measured by the measuring unit, calculation unit, judgment of Ah Ri secular variations or defects in the original data for the object information and the position adjustment step of adjusting the position of the shape data is the reference outline data and information relating to the measured the three-dimensional shape as a reference, the calculation unit relates the acquired two-dimensional images A surface defect information acquisition step for acquiring information on surface defects existing in the object based on the image data and the shape data. To.

  According to the embodiment of the present invention, in the surface defect inspection, it is possible to perform an automatic inspection which can increase the detection rate and measure the level of the surface defect to be described in the inspection record.

It is a block diagram which shows the structure of the surface defect evaluation apparatus which concerns on embodiment. It is a block diagram which shows the structure of the calculating part of the surface defect evaluation apparatus which concerns on embodiment. It is a conceptual diagram for demonstrating the content of the calibration by the calibration part of the surface defect evaluation apparatus which concerns on embodiment. It is a conceptual diagram for demonstrating three positions at the time of calibration, (a) shows the case of the position A, (b) shows the position B, (c) shows the case of the position C. It is a conceptual diagram for demonstrating the content of the data synthesis | combination of the image data by the image data synthetic | combination part of the surface defect evaluation apparatus which concerns on embodiment, (a) is the image data of an individual rectangular area, (b) is data synthesis. It is the performed image data. It is a conceptual diagram for demonstrating the content of the data synthesis | combination of the image data by the image data synthetic | combination part of the surface defect evaluation apparatus which concerns on embodiment, (a) is the image data of an individual linear area | region, (b) is data synthesis. It is the image data which performed. It is a conceptual diagram for demonstrating the interpolation process by the interpolation part of the surface defect evaluation apparatus which concerns on embodiment. It is a block diagram which shows the structure of the image data surface defect discrimination | determination part of the surface defect evaluation apparatus which concerns on embodiment. It is a block diagram which shows the structure of the shape data surface defect discrimination | determination part of the surface defect evaluation apparatus which concerns on embodiment. It is a conceptual perspective view for demonstrating the function of the depth direction determination part of a shape data surface defect discrimination | determination part. It is a block diagram which shows the structure of the surface defect dimension estimation part of the surface defect evaluation apparatus which concerns on embodiment. It is a conceptual perspective view for demonstrating the dimension measurement process by the surface defect evaluation apparatus which concerns on embodiment. It is a flowchart which shows the whole procedure of the surface defect inspection method which concerns on embodiment. It is a flowchart which shows the basic procedure of the surface defect discrimination | determination step in the surface defect inspection method which concerns on embodiment. It is a flowchart which shows the procedure of the surface defect recognition process from the image data in the surface defect discrimination | determination step in the surface defect inspection method which concerns on embodiment. It is a flowchart which shows the procedure of the surface defect recognition process from the shape data in the surface defect discrimination | determination step in the surface defect inspection method which concerns on embodiment.

  Hereinafter, a surface defect evaluation apparatus, a surface defect inspection system, and a surface defect inspection method according to an embodiment of the present invention will be described with reference to the drawings. Here, the same or similar parts are denoted by common reference numerals, and redundant description is omitted. Here, a measurement object to be measured for inspection is hereinafter referred to as an object.

  FIG. 1 is a block diagram illustrating a configuration of the surface defect evaluation apparatus according to the embodiment. The defect measurement system 500 includes a surface data acquisition device 100 and a surface defect evaluation device 400.

  The surface data acquisition apparatus 100 includes an image acquisition unit 110 that acquires an image, a shape measurement unit 120 that acquires three-dimensional shape data, and a scan unit 130 that moves and drives the image acquisition unit 110 and the shape measurement unit 120.

  The image acquisition unit 110 includes a camera including an image sensor, a lens optical system, and an illumination optical system. The image acquisition unit 110 acquires an image of an object, and acquires image data of a linear region or image data of a non-linear two-dimensional region (hereinafter referred to as a rectangular region).

  The shape measuring unit 120 includes a laser irradiation unit 121 that irradiates an object with irradiation light such as laser light on a linear pattern, and a laser light imaging camera 122 that captures reflected light of the laser irradiated on the object. The irradiated linear pattern is measured by the principle of triangulation, and the surface irregularities are acquired as shape data. Hereinafter, image data and shape data are collectively referred to as outline data.

  The above is a case of a device using a light cutting method based on a combination of a camera that acquires shape data of a linear region and irradiation light (laser light), but two or more units that acquire shape data of a rectangular region. Stereo viewing with a camera or a laser scanner may be used.

  The scan unit 130 includes a movement drive unit 131, a rotation drive unit 132, and a fixed unit 133. Since various shapes can be considered as the object, the movement drive unit 131 moves the measurement heads of the image acquisition unit 110 and the shape measurement unit 120 in the three-axis directions in order to measure the entire surface of the object. Move and drive. The rotation drive unit 132 is configured to be able to be driven to rotate about each of the three axes.

  The fixing unit 133 includes a fixture for fixing the object in order to maintain the object, the movement driving unit 131, and the rotation driving unit 132 in a predetermined relative positional relationship. The scan unit 130 acquires, for each of the image acquisition unit 110 and the shape measurement unit 120, the movement amount, the rotation amount, and the current position from the origin of coordinates related to the surface data acquisition device 100 set at the start of measurement, for example. The acquired result is output to the input unit 410 of the surface defect evaluation apparatus 400.

  The movement drive unit 131 and the rotation drive unit 132 of the scan unit 130 also have a function of setting both the optical axis of the image data by the image acquisition unit 110 and the optical axis of the shape measurement data by the shape measurement unit 120 as the same axis.

  The surface defect evaluation apparatus 400 is a computer system having a memory 200, a CPU 300, an input unit 410, and an output unit 420. The CPU 300 includes a control unit 310 that controls the computer system and a calculation unit 320 that executes calculation for defect evaluation. The memory 200 includes an outer shape data storage unit 210 that stores shape data acquired by the surface data acquisition device 100 and an operation result storage unit 220 that stores operation results from the operation unit 320.

  The input unit 410 accepts the outline data acquired by the surface data acquisition apparatus 100 as a computer system. The output unit 420 outputs the result of the defect evaluation performed by the calculation unit 320.

In addition to the shape data acquired by the surface data acquisition device 100, the external shape data storage unit 210 includes reference external shape data externally input to the input unit 410, and the image acquisition unit 110 input from the scan unit 130 to the input unit 410. The movement amount, rotation amount, and current position of the shape measuring unit 120 are stored. The reference outer shape data is the original outer shape data relating to the object, and is data that serves as a reference for determining deformation and defects.

  FIG. 2 is a block diagram illustrating a configuration of a calculation unit of the surface defect evaluation apparatus according to the embodiment. The calculation unit 320 includes a calibration unit 321, an image data synthesis unit 322, a shape data synthesis unit 323, a position adjustment / surface recognition unit 324, an interpolation unit 325, a defect evaluation unit 330, and a result creation unit 340. The defect evaluation unit 330 includes a preprocessing unit 331, an image data surface defect determination unit 332, a shape data surface defect determination unit 333, a defect determination unit 334, and a surface defect size estimation unit 335. Hereinafter, although each element which comprises the calculating part 320 is demonstrated, each calculation result is stored in the calculation result memory | storage part 220. FIG.

  The calibration unit 321 estimates the correspondence between the shape data and the image data. Regarding the calibration unit 321, the case where the optical axis of the image acquisition unit 110 and the optical axis of the shape measurement unit 120 do not match and the case where the optical axis of the image acquisition unit 110 and the optical axis of the shape measurement unit 120 match are described below. Explained. FIG. 3 is a conceptual diagram for explaining the contents of calibration by the calibration unit of the surface defect evaluation apparatus according to the embodiment.

  The laser beam emitted from the laser irradiation unit 121 of the shape measuring unit 120 has a planar shape spread at a minute angle, and when the irradiation surface S is irradiated, the irradiated portion of the laser beam becomes linear. The straight line portions on both sides of the laser light are L1 and L2, and the central straight line is L0. Further, the intersection points of the irradiation surface S and the straight lines L1, L2, and L0 are defined as Q1, Q2, and Q0, respectively. The center line of the spread of the laser light, that is, the straight line passing through the center portion Q0 is the optical axis L0 of the shape measuring unit 120.

The position of the line segment Q1-Q2 on the irradiation surface S is determined from the angle θ at which the line segment Q1-Q2 is viewed from the laser beam photographing camera 122 arranged with a known distance d from the laser irradiation unit 121, and the distance d. The coordinate position Z _L of the object can be calculated by d / tan θ. The horizontal coordinate positions X and Y can be calculated from the information from the movement drive unit 131 and the rotation drive unit 132 and the relative positions of the original laser irradiation unit 121 and the laser light photographing camera 122, and as a result, the interval d can be calculated.

  On the other hand, the image acquisition unit 110 includes a lens 111, a full mirror 112, and a half mirror 113 between the measurement target. The light from the irradiation surface S is reflected by the half mirror 113 to change its direction, reflected by all the mirrors 112, and collected at the focal point F by the lens 111. Among these, a straight line A <b> 1 that intersects the irradiation surface S when the light vertically passing through the center of the lens 111 is traced is the optical axis A <b> 1 of the image acquisition unit 110.

  First, when the optical axis A1 of the image acquisition unit 110 and the optical axis L0 of the shape measurement unit 120 do not coincide with each other, the optical origin coordinates when the optical axis A1 of the image acquisition unit 110 and the optical axis L0 of the shape measurement unit 120 do not coincide with each other. Set. For example, the optical origin coordinates may be set based on the optical axis L0 of the shape measuring unit 120.

  The laser beam of the shape measuring unit 120 extends in a band shape with a fine spread angle, and the points Q1, Q2 at both ends of the laser beam on the image by the both side portions L1, L2 and the central portion L0 and the point Q0 at the central portion. Are obtained when the relative position with respect to the object is position A, position B, and position C. FIG. 3 shows the case of position A. As a result, the respective coordinate positions of Q1, Q0, and Q2 are obtained for the positions A, B, and C.

  FIGS. 4A and 4B are conceptual diagrams for explaining the three positions. FIG. 4A shows the case of the position A, FIG. 4B shows the case of the position B, and FIG. The intersection of the laser beam L1 on the irradiation surface S1 of the object at the position A is QA1, the intersection with L0 is QA0, and the intersection with L2 is QA2. In addition, points corresponding to these on the image are referred to as PA1, PA0, and PA2, respectively.

  Similarly, the intersection of the laser beam L1 on the irradiation surface S2 of the object at position B is QB1, the intersection with L0 is QB0, and the intersection with L2 is QB2. Also, points corresponding to these on the image are PB1, PB0, and PB2, respectively. Further, the intersection point of the laser beam L1 on the irradiation surface S3 of the object at the position C is QC1, the intersection point with L0 is QC0, and the intersection point with L2 is QC2. In addition, points corresponding to these on the image are PC1, PC0, and PC2, respectively.

  Since each light beam of the laser light travels linearly, for example, QA1, QB1, and QC1 are in the same straight line. QA2, QB2, and QC2 are also in the same straight line. The points PA1, PB1, and PC1 on the image are also collinear. The same applies to other points.

The following expression (1) is obtained by projecting the three-dimensional coordinates (X _Q , Y _Q , Z _Q ) of the laser beam and the two-dimensional coordinates on the image onto the three-dimensional space (X _P , Y _P , Z _P ). _Here, a 11 _{~a 33} is a rotational transformation component.

Here, each rotation conversion component is expressed as follows.
a ₁₁ = cosω · cosκ
a ₁₂ = −cos φ · sin κ
a ₁₃ = sinφ
a ₂₁ = cosω · sinκ + sinφ · cosκ
a ₂₂ = cosω · cosκ + sinω · sinκ
a ₂₃ = −sin ω · cos φ
a ₃₁ = sinω · sinκ−sinφ · cosκ
a ₃₂ = sinω · cosκ + cosω · sinκ
a ₃₃ = cosω · cosφ

As described above, the three-dimensional coordinates (X _Q , Y _Q , Z _Q ) of the laser light can be calculated. The two-dimensional coordinates (X _P , Y _P , Z _P ) on the image are values determined from the image data. Therefore, for each of the nine measured points, a three-dimensional coordinate (X _Q , Y _Q , Z _Q ) and a two-dimensional coordinate (X _P , Y _P , Z _P ) are given.

Using these values, the rotation transformation components a _{11 to} a ₃₃ can be obtained for the expression (1) using the least square method or the like. As a result, the coordinate position of an arbitrary point can be converted.

  Also, assuming that the point at position A is QA1, QA0, QA2 and the point at position C is QC1, QC0, QC2, the position on the image regarding an arbitrary position between position A and position C is as described above. As described above, it is obtained by interpolation using the fact that there is a linear relationship. Note that the optical origin coordinates of the image acquisition unit 110 and the shape measurement unit 120 affect the calibration accuracy. Therefore, it is preferable that the optical origin coordinates are set with higher accuracy than either the laser resolution or the image resolution. .

  Next, a case where the optical axis A1 of the image acquisition unit 110 and the optical axis L0 of the shape measurement unit 120 coincide will be described. First, the shape measurement unit 120 acquires the coordinate positions of the laser light points Q0, Q1, and Q2 on the image for the positions A, B, and C. Further, the image acquisition unit 110 acquires corresponding points P0, P1, and P2. Thereafter, the parallel movement amount, the rotation amount, and the scale factor are obtained by the least square method or the like with respect to the following equations (2) and (3).

However, X, Y, Z three dimensional coordinates, x, y two-dimensional coordinates, Xm, Ym, Zm is the three-dimensional coordinates of parallel movement _amount, a 11 _{~a 33} is rotated transform components on the image of the laser beam, c is a scale factor.

  Further, if the points P1 and P2 respectively obtained at the positions A, B and C are the point P1A and the point P2A in the case of the position A and the points P1C and P2C in the case of the position C, respectively, as described above, Using the linear relationship, the coordinate value on the image between the point P1A and the point P1C on the image can be interpolated by a linear equation. Similarly, the coordinate value on the image between the points P2A and P2C can also be interpolated by a linear equation. By this procedure, an interpolation process in the case where they do not match is performed.

  The image data composition unit 322 calculates the parallel movement amount and the rotation amount from the corresponding points based on the image data when a plurality of rectangular areas or linear areas on the surface of the object are photographed, and calculates the calculated parallel movement amount. The image is converted and combined using the amount of rotation. Hereinafter, a case where a rectangular area is imaged and a case where a linear area is imaged will be described separately.

  FIG. 5 is a conceptual diagram for explaining the contents of image data composition by the image data composition unit of the surface defect evaluation apparatus according to the embodiment, where (a) is image data of individual rectangular areas, and (b). Is image data subjected to data synthesis. FIG. 5A shows two images F1 and F2 obtained by photographing a rectangular area. As shown in FIG. 5A, in the image F1, a portion K1 indicated by circles, squares, and triangles and a portion K2 indicated by circles, squares, and triangles in the image F2 are common. As shown in FIG. 5B, the portion K1 and the portion K2 overlap each other by rotating the image F2.

  In this way, in a region where two images taken at different positions overlap each other, a matching or approximate pixel is searched by template matching, and a corresponding point between the two images is specified. Based on the coordinates of the corresponding points, the amount of translation and the amount of rotation are obtained by the following equation (4).

Here, x, 2-dimensional coordinates on the y image, x ', y' is 2-dimensional coordinates on the search destination of the _image, x m, 2-dimensional coordinates of _{y m} is the amount of parallel _movement, a 11 _{~a 22} is This is a rotation conversion component.

  In order to obtain the parallel movement amount and the rotation amount of Expression (4), it is necessary to search for at least three corresponding points in each image of FIG. 5A as indicated by circles, squares, and triangles. is there. As a result, all the coordinates of the search destination image are converted from the parallel movement amount and the rotation amount, and synthesized. The object surface can be synthesized by performing this procedure for all the continuous images.

  In addition, when a rectangular area as shown in FIG. 5A is photographed, it is possible to extract a linear area and synthesize it. On the image obtained from the parallel movement amount and the rotation amount of the calibration result. A pixel corresponding to the linear region is extracted from each image by the interpolation linear equation, and a composite image is created by arranging the pixels extracted from the next photographed image in parallel. The object surface can be synthesized by performing this procedure for all the continuous images.

  Next, a case where a linear region is photographed will be described. 6A and 6B are conceptual diagrams for explaining the contents of data composition of image data. FIG. 6A shows image data of individual linear regions, and FIG. 6B shows image data subjected to data composition. That is, (a) is each image data when the laser irradiation positions G1 to G5 to the object are sequentially shifted.

  The image data synthesis unit 322 acquires the movement amount and the rotation amount from the three-axis movement driving unit 131 and the rotation driving unit 132 of the scanning unit 130, respectively, and substitutes them in the following equations (5) and (6). Obtain the image coordinates after synthesis.

However, X, Y, Z three-dimensional coordinate of the scan unit 130, x, y two-dimensional coordinates on the _image, X _m, Y _m, 3-dimensional coordinates of the _{Z m} is the amount of parallel _movement, a 11 _{~a 33} is The rotation conversion component, c is a scale factor.

  The shape data combining unit 323 combines a plurality of shape data three-dimensionally. The shape data synthesis unit 323 is configured to be capable of either of the following two shape data synthesis methods. In addition, either may be comprised.

  The first synthesis method of shape data is as follows. First, three or more distances per measurement are acquired from the measurement by the shape measuring unit 120. As described with reference to FIG. 3, the distance can be measured and calculated using the laser irradiation unit 121 and the laser light imaging camera 122 of the shape measurement unit 120. Specifically, for example, first, the shape is measured by the shape measuring unit 120, and the distance is calculated using an arbitrary measurement position as the origin.

  Next, the scanning unit 130 is moved and rotated, and the shape data synthesis unit 323 receives the movement amount and the rotation amount from the scanning unit 130, and after the movement and rotation, according to the coordinate conversion formula shown in the following equation (7). The measurement position is calculated.

Here, each rotation conversion component is expressed as follows.
a ₁₁ = cosω · cosκ
a ₁₂ = −cos φ · sin κ
a ₁₃ = sinφ
a ₂₁ = cosω · sinκ + sinω · sinφ · cosκ
a ₂₂ = cosω · cosκ−sinω · sinφ · sinκ
a ₂₃ = −sin ω · cos φ
a ₃₁ = sinω · sinκ−cosω · sinφ · cosκ
a ₃₂ = sinω · cosκ + cosω · sinφ · sinκ
a ₃₃ = cosω · cosφ

Here, X ′, Y ′, and Z ′ are measurement data after coordinate conversion, Xm, Ym, and Zm are movement amounts of the movement driving unit 131, X ₀ , Y ₀ , and Z ₀ are reference positions of the scanning unit 130, X , Y, Z are the coordinate values of the shape data, and ω, φ, κ are the rotation angles of the rotation drive unit 132 around the X axis, Y axis, and Z axis, respectively.

  Based on the second shape measurement data, a measurement distance from a measurement position with an arbitrary point as a temporary origin is obtained from the measurement position after movement and rotation. The third distance is obtained in the same manner. As a result, distances from at least nine temporary origins are obtained. At this time, using these distances as constraint conditions, the coordinate values of the shape measurement data at each measurement position are obtained by inverse transformation of Equation (7). Thereby, it is possible to obtain the coordinate values of the shape measurement data with reference to the temporary origin at all measurement positions.

  In the second shape synthesis method, the distance with respect to the reference position is obtained with the reference position of the movement and rotation drive unit 132 determined as the first axis of the scan unit 130 as the origin. At this time, since the reference position is set as the origin, the coordinate conversion is performed on the measurement data by the equation (7). By this conversion, it is possible to synthesize the shape data of the object with respect to the position determined as the origin of the scanning unit 130.

  The position adjustment / surface recognition unit 324 uses the result of combining the entire shape or arbitrary surface of the shape measurement data for one object into one shape data by the shape combining means as described above, and using the result of combining the reference outline data and the shape. Adjust the position of the measurement data. In addition, a label that identifies the surface is added to the reference outline data in advance, and the surface is recognized by copying the label to the point where the distance between the coordinates of the shape data and the coordinates of the reference outline data is the closest during the position adjustment. I do. As described above, the position adjustment / surface recognition unit 324 has a position adjustment function and a surface recognition function.

  First, the position adjustment is performed in order to compare the shape of the object in the next defect evaluation unit 330 with respect to the shape data of the object and the reference outline data. The position adjustment / surface recognition unit 324 is configured to perform position adjustment by selecting or sequentially applying the following three functions.

  The first function of position adjustment is a method that can cope with changes in the shape of the object over time. At least three reference positions in the shape data and the reference outline data are selected, or peripheral shape feature amounts including the reference position are detected, and the reference position is determined by matching the shape data with the reference outline data.

  Here, the reference positions of the shape data and the reference outer shape data may not always match. Therefore, as correction processing, the shape data of the surrounding area including the point at the selected reference position is treated as the shape feature value, the normal vector of the point and the coordinate value of the shape part that becomes the edge are obtained, and the position of the manually selected point Correct. The shape data and the reference outline data are associated with each other using at least three corrected points. Using each of the three points associated with each other, the position is adjusted by movement and rotation using the coordinate conversion formula shown in the following formula (8).

Here, X ′, Y ′, and Z ′ are shape data after coordinate conversion, X _m , Y _m , and Z _m are movement amounts of the shape data, and X, Y, and Z are coordinate values of the shape data. Each rotational transformation component _a 11 _{~a 33} is similar to the notes of the formula (7).

  In Expression (8), for the reference position, the movement amount and the rotation amount may be obtained by, for example, the least square method so that the difference between the coordinates after conversion of the shape data and the coordinate value of the reference outer shape data is minimized.

  The second function of the position adjustment is a method of using a portion that is not affected by the secular change when the shape is deformed due to the secular change. The measurement point and surface normal vectors and the feature quantity of the shape change amount are detected by calculation processing from the shape measurement data and the reference shape data, and the area where the shape change amount is below the arbitrarily set threshold is detected. At least three reference positions are determined from the area as unaffected parts. Note that the position adjustment uses the above-described equation (8), obtains the movement amount and the rotation amount by the same method, and performs coordinate conversion.

  The third function of position adjustment specifies a region where the shapes match for both shape data and reference outline data, and determines at least three reference positions. In order to specify a region having a matching shape, a normal vector and an edge shape are obtained from the shape data. A region where the obtained normal vector and edge shape calculation data coincide with both shape data is specified by matching.

  In the same way as the first function, a point is selected from the specified area, the position adjustment is performed using the above equation (8), the movement amount and the rotation amount are obtained by the same method, and coordinate conversion is performed. I do.

  Since it is desirable that the regions having the same shape are dispersed as much as possible across the entire object, the position of the center of gravity of the specified region is obtained and the threshold is arbitrarily set for the distance between the positions of the center of gravity of the two regions. You may select by a value.

  The position adjustment / surface recognition unit 324 uses the above functions to perform position adjustment that enables comparison of shape data and reference outline data in order to specify a defect by the next defect evaluation unit 330.

  Next, the surface recognition processing function of the position adjustment / surface recognition unit 324 will be described. As described above, in the surface recognition, a label for identifying a surface is added to the reference outline data in advance, and the position between the coordinates of the shape data and the coordinates of the reference outline data is set to the closest point during position adjustment. The surface can be recognized by copying the label. Further, by obtaining a normal vector of a triangular plane composed of at least three points with respect to the shape data or the reference outline data, it is compared with the normal direction of each plane that is uniquely determined, and is the same or approximate. A method of recognizing the surface from the normal direction may be used.

  By this surface recognition processing, it is possible to distinguish and identify a surface such as a plane existing in the object or a side surface orthogonal to the plane. For example, in the case of a gas turbine stationary blade, since there are front and back surfaces for each of the stationary blade surface, the left side surface, and the right side surface, at least six surfaces are distinguished and identified.

  Further, the position adjustment / surface recognition unit 324 prevents a specific shape existing in the design of the object, for example, a boundary between the stationary blade surface and the side wall surface, a hole groove for rectification, and the like from being erroneously recognized as a defect. Has a processing function. Specifically, a template is created from reference outline data or design drawing data, and a specific shape existing in design is searched for shape data. The region obtained in this way is set in advance as a region not subject to defect evaluation.

  The interpolation unit 325 uses the result of the calibration unit 321 to calculate the parallel movement amount and the rotation amount of the image data combined by the image data combining unit 322 and the shape data combined by the shape data combining unit 323. It is used as a coordinate point on the same coordinate system, and the coordinate point between the pixel and the shape measurement point is interpolated.

  First, the corresponding three-dimensional coordinates of each pixel are matched from the above-described equation (1) that relates the point on the image derived by the calibration by the calibration unit 321 and the corresponding point of the laser beam. When the resolutions of the image data and the shape data are the same, they can be matched with the data as it is for each pixel.

FIG. 7 is a conceptual diagram for explaining the interpolation processing by the interpolation unit of the surface defect evaluation apparatus according to the embodiment. In FIG. 7, a portion displayed with a white circle Q corresponds to a pixel. Among them, a portion displayed by a black circle P indicates a portion that is also a measurement point in the shape data. For example, as shown in the case of the vertical direction in FIG. 7, the resolution of the image data is σ _{p and} the resolution of the shape data is coarse σ _m, and if both the resolutions do not match, the finer resolution, that is, In this example, with respect to the resolution σ _{p of the} image data, association is performed for the interval between pixels to be associated or the interval between shape data.

By dividing the resolution σ _{m of the} shape data by the resolution σ _p of the image data, the interval between the pixels to be handled can be obtained. For example, if the resolution σ _{p of the} image data is 0.1 mm / pixel and the resolution σ _{m of the} shape data is 0.4 mm, the correspondence can be achieved with a pixel interval of 4 pixels. Therefore, there is a shortage of shape measurement data corresponding to 2 pixels of image data that cannot be handled.

Therefore, the coordinate value of the shape measurement data to be interpolated is obtained by the weighted average interpolation method shown in the following equation (9).
Z _p = (Σw _i Z _i ) / (Σw _i ) (9)
Here, Z _p is X 3 axis after interpolation, Y, one of the coordinate values of Z (when interpolated image data, the luminance value of RGB color space), w is the weight, Z is before interpolation The coordinate value, i, is an interval number for interpolation.

  Next, the preprocessing unit 331, the image data surface defect determination unit 332, the shape data surface defect determination unit 333, the defect determination unit 334, and the surface defect size estimation unit 335 that constitute the defect evaluation unit 330 will be sequentially described.

  The preprocessing unit 331 performs preprocessing common to both image data and shape data. Specifically, based on the shape data synthesized by the shape data synthesizing unit 323 and the reference outer shape data related to the initial shape, the process of specifying the region where the shape has changed is performed.

  Here, the area that has changed from the original shape is specified by obtaining the difference between the coordinate values of the position-adjusted shape data and the reference outline data. An area where the difference is larger than a predetermined value is determined as a change area. In addition, it is good also considering the area | region where a change is clear from the design shape as the object of determination. The image data surface defect determination unit 332 and the shape data surface defect determination unit 333 perform surface defect determination on the specified region.

  FIG. 8 is a block diagram illustrating a configuration of the image data surface defect determination unit. The image data surface defect determination unit 332 calculates defect feature amounts from the image data, and identifies a set region of defect feature amounts on the image data. The image data surface defect determination unit 332 includes an image data reading unit 332a, an edge detection unit 332b, a closed region detection unit 332c, a false detection part removal unit 332d, and a defect region connection unit 332e.

  The image data reading unit 332a reads the image data that is interpolated and interpolated by the interpolation unit 325. The edge detection unit 332b extracts a small area corresponding to the arbitrarily set area size from the image data.

  The closed region detection unit 332c performs defect edge detection processing on a small region of the image data. Note that the small area extraction process is not necessarily required, and thus the process can be omitted. In the defective edge detection process, the edge intensity and density gradient are calculated by an edge detection filter (for example, Sobel filter) for four or eight adjacent luminance values based on the luminance value of the pixel of interest.

  In addition, by binarization processing using a threshold value arbitrarily set for the edge strength, it is determined whether or not it is an edge, and an edge region is detected. Further, by using the density gradient result and the edge region detection result, pixels having similar values in the density gradient direction are traced by the inner product between adjacent pixels. Using the fact that defects on the image become closed regions, the closed region (returns to the pixel of the same starting point when tracking in either direction from the pixel that is an arbitrary starting point) from the tracking result To detect.

  The false detection part removal unit 332d may include a region other than a defect in the closed region detected by the closed region detection unit 332c. Is determined from the calculation information, and the erroneously detected portion is removed. In addition, since empirical knowledge is effective for this part, the result may be displayed and an examination person may judge.

  The defective area connecting portion 332e connects the closed areas divided into two or more. As a result of processing by the edge detection unit 332b, the closed region detection unit 332c, and the erroneous detection part removal unit 332d, one defect may be recognized as a broken defect. Therefore, if the contours of the closed regions are within a predetermined pixel distance, a blank region that is not recognized as a defect is specified as a defect according to the width of the contour.

  FIG. 9 is a block diagram illustrating a configuration of the shape data surface defect determination unit. The shape data surface defect discriminating unit 333 compares the coordinate data of the position-adjusted shape data and the reference outline data, identifies the region where the shape is changing, and identifies the region surrounded by the edge of the defect. The shape data surface defect determination unit 333 includes a shape data reading unit 333a, a small region extraction unit 333b, a depth direction determination unit 333c, a data projection unit 333d, a noise removal unit 333e, an edge detection unit 333f, a closed region detection unit 333g, an error It has a detection part removing unit 333h and a defective area connecting unit 333k.

  The shape data reading unit 333a reads the shape data associated and interpolated by the interpolation unit 325. The small area extraction unit 333b extracts a small area corresponding to the arbitrarily set area size from the shape measurement data. Note that the small area extraction process is not necessarily required, and thus the process can be omitted.

  The depth direction determination unit 333c confirms and determines the direction in the depth direction of the defect from the surface identification label data whose position is adjusted by the position adjustment / surface recognition unit 324 and is surface-recognized.

FIG. 10 is a conceptual perspective view for explaining the function of the depth direction determination unit of the shape data surface defect determination unit. FIG. 10 shows an example of a stationary blade of a gas turbine. Shizutsubasamen _{S F} and Shizutsubasamen _S left side perpendicular to _{F S L} and the right side surface _{S R} is in the front and back surfaces. The internal direction of the stator blade surface is the Z axis minus direction in the crack depth direction. The left side direction of the stationary blade surface is the Y axis plus direction, the right side surface direction of the stationary blade surface is the Y axis minus direction, and the remaining direction is the X axis direction. That is, as shown in FIG. 10, the stationary blade surface is in the X and Y axis directions, and the fluid flow direction is in the minus Z axis direction.

  The data projection unit 333d projects data using the value in the Z-axis direction of the shape measurement data as a luminance value corresponding to the luminance of the image. Now, let the plane orthogonal to the calculation target axis direction (Z-axis minus direction) be the xy plane, the X coordinate of the shape measurement data is the x coordinate of the coordinate value projection data, and the Y coordinate is the y coordinate of the coordinate value projection data. . Data projection is performed using the Z coordinate as a luminance value corresponding to the luminance of the image.

  Here, the X and Y coordinates are usually driven and moved by the scanning unit 130 at equal intervals. In this case, data projection is possible with the obtained shape data as it is. If the intervals are not equal, it can be dealt with by projecting the three-dimensional coordinates onto the two-dimensional coordinates from the following equations (10) and (11).

Here, X, Y, Z is 3-dimensional coordinates of the shape data, x, y are the same as those in the two-dimensional coordinates on the projection _data, a 11 _{~a 33} is a rotating conversion component (7). C is a scale factor. The scale factor c may be calculated as 1, and when the scale of the image data is the same, a ratio between the resolution of the image data used in the interpolation unit 325 and the resolution of the shape measurement data is given. May be. Further, since the rotation components a _{11 to} a ₃₃ are basically an orthogonal coordinate system, 0 is given assuming that the angles of the three axes of each component are not rotated.

  The noise removing unit 33e detects and removes points that form uneven shapes with unique coordinate values from the coordinate value projection data. A rectangular area set to an arbitrary size is extracted from the coordinate value projection data, and a variance value between the coordinate value of the target coordinate at the center of the rectangular area and the surrounding coordinate values is calculated. If the calculated dispersion value is equal to or greater than the coordinate value around the target coordinate and a threshold value arbitrarily set, it is determined that the point is a point that forms a unique uneven shape. Only the coordinate values of the points where the individual differences between the coordinate values of the determined points and the peripheral coordinate values are within the arbitrarily set distance are aggregated and averaged to remove the points having unique uneven shapes. Further, when the variance value around the determined point is also high, it is a set of unique uneven shapes, and there is a possibility that the above-mentioned point for averaging does not exist. In this case, a point with a peculiar uneven shape is removed by either the method of increasing the rectangular region size and performing the averaging process or the method of taking the median of the coordinate values in the rectangular region.

  The edge detection unit 333f detects an edge and estimates a defect contour. Specifically, using the coordinate value projection data, an edge detection filter (for example, Sobel) is applied to the coordinate values of four neighboring locations or eight neighboring locations based on the coordinate values of the focused shape data. Edge strength distribution is created by a filter. From this edge intensity distribution, an edge intensity equal to or greater than a threshold value is recognized as a defect outline. Since the threshold value varies depending on the surface state of the object, it depends on one of a method of performing partial measurement and setting by numerical confirmation, and a method of obtaining the standard deviation of the entire edge intensity distribution as a threshold value.

  Here, depending on the surface state of the object, there may be an isolated protrusion-like shape portion, and the contour points at eight neighboring locations around the point recognized as the defect contour are counted. If the number of surrounding eight contour points is equal to or less than the arbitrarily set threshold value (usually set to 1 for isolated points), it is removed as an isolated point.

  When the set of contour points is a closed region, the closed region detection unit 333g is recognized as a defect. However, if there is a partially interrupted portion, the closed region detection unit 333g does not become a closed region and is not recognized as a defect. Add a process to connect discontinuous parts below the crack length to be recorded.

  In order to confirm whether the region detected by the closed region detection unit 333g is an erroneously detected portion, the erroneous detection portion removal unit 333h determines whether the region of the defect and the region other than the defect are the pixel area or the closed region. Is determined from calculation information such as the shape of the image, and the erroneously detected portion is removed.

  Since the defect area connecting unit 333k may recognize one defect as a broken defect, if the distance between the outlines of the closed area is within an arbitrarily set pixel distance, it is recognized as a defect according to the width of the outline. A blank area that has not been processed is recognized as a defect, and two or more closed areas are connected.

  The defect determination unit 334 determines whether the defect is a defect using the defect recognition results of both the image data surface defect determination unit 332 and the shape data surface defect determination unit 333. Here, the case where the resolution of the image data is smaller than the resolution of the shape data will be described, but the reverse case can be determined by performing the same processing.

  When the resolution of the image data is small, it is possible to detect a defect smaller than the defect recognition result of the shape data, but there is a possibility that erroneous detection is included. When a defect corresponding to a possibility of erroneous detection included in the defect recognition result of the image data does not exist in the defect recognition result of the shape measurement data, it is removed as a false detection defect. Therefore, the surface defect size estimation unit 335 obtains a logical product of the defect recognition result of the image data and the defect recognition result of the shape data. As a result, defects that do not exist on one side are removed, and defect recognition results are integrated in parallel.

  The surface defect size estimation unit 335 estimates the size of the defect. FIG. 11 is a block diagram illustrating a configuration of a surface defect size estimation unit of the surface defect evaluation apparatus. The surface defect size estimation unit 335 includes an image reference estimation unit 335a and a shape reference estimation unit 335b.

  The image reference estimation unit 335a detects the crack start and end points from the image data and estimates the size of the defect. Specifically, first, the image reference estimation unit 335a obtains a center line of a region recognized as a defect from image data by thinning or skeletonization processing. Further, the outermost rectangle of the crack region is obtained, and the contact point between the crack region and the rectangle is extended from the point corresponding to the start point or the end point to the end point detected as the center line. Thereby, the start point or the end point is determined. Here, since there are a plurality of candidates for the start point and the end point, the combination having the longest distance on the center line among all the combinations is set as the start point and the end point of the crack. By integrating the coordinate values associated with the shape measurement data of the center line between the obtained start and end points and the end points, the length of the defect is obtained.

  In addition, the width of the defect is determined by calculating the distance between the start point and the end point of the contact point of the perpendicular line and the crack outline perpendicular to the center line, and selecting an average value, maximum value, or minimum value of the defect. Calculate as width. However, if there is a branch point in the defect, there is a possibility that the width of the obtained defect is long, so it is possible to select whether or not to exclude the section before and after the arbitrarily set branch point. . Usually, this section is set based on the minimum defect width described in the inspection record.

  The shape reference estimation unit 335b estimates the size of the defect from coordinate values associated with the shape data. That is, the crack start point and end point are detected from the shape measurement data to estimate the defect size.

  FIG. 12 is a conceptual perspective view for explaining the dimension measurement process. First, the center line L0 in the crack direction of the region recognized as a defect from the shape measurement data is obtained by thinning or skeletonization processing. Further, the outermost rectangle A of the defect area is obtained, and the contact between the defect area and the rectangle A is extended from the point corresponding to the start point or the end point to the end point detected as the center line. Thereby, the start point or the end point is determined. Here, since there are a plurality of start point and end point candidates, the combination having the longest distance on the center line among all the combinations is set as the defect start point and end point.

  By integrating the coordinate values of the shape measurement data of the center line between the obtained start and end points and both end points, the length of the defect is obtained. Similarly to the case of the image reference estimation unit 335a, the width of the defect is obtained from the start point to the end point of the contact point of the perpendicular line and the crack outline perpendicular to the center line, and the average value, maximum value, and minimum value are obtained. A value arbitrarily selected from the above is obtained as the defect width. However, when there is a branch point in the defect, there is a possibility that the width of the obtained defect may be long. Therefore, it is possible to select whether or not to exclude the section before and after the arbitrarily set branch point. Usually, this section is set based on the minimum crack width described in the inspection record.

  The result creation unit 340 uses the result of the defect evaluation unit 330 to record a value exceeding a predetermined threshold for the defect length and width as an inspection record, and creates an image to be displayed. Output to 420.

  First, the result creating unit 340 records shape data for each of the shape data and the reference contour data stored in the contour data storage unit 210 in order to record each surface of the object as a two-dimensional image as an inspection record. An edge contour is detected using the coordinate value of. Furthermore, a projection image is created by projecting the contour of the detected object onto the two-dimensional space from an arbitrarily designated viewpoint on the three-dimensional space.

  In the method of detecting a shape contour, a shape change point serving as an edge is detected from a normal vector and a shape gradient amount (a differential value around the shape data) for each of the shape data of the object and the reference shape data. Using the detected shape change points, only the points where the change points are continuous are detected. In the case of discontinuity, it is detected as a continuous point if it is within a threshold set as distinguishable as a continuous point of arbitrarily set shape change points. With this function, the shape contour of the object is detected.

  Next, a projection image is created by projecting the contour of the detected object onto a two-dimensional space from an arbitrarily designated viewpoint on the three-dimensional space. The points to be arbitrarily specified are, for example, the state viewed from the upper surface or the side surface of the object, and the viewpoint can be changed as appropriate. This is a method of designating with a mouse or a keyboard which is an input auxiliary device of the arithmetic processing unit on a monitor table device which can confirm the shape data of the object from the viewpoint.

  Further, as a result of the above, a composite image is created from the shape data and the projection image created as described above for the defect determined to be recorded. Further, the defect information such as the defect length obtained by the surface defect size estimation unit 335 corresponding to each defect is drawn on the composite image. Further, defect information such as the defect length obtained by the surface defect size estimation unit 335 is centrally managed for each target object and stored in the calculation result storage unit 220 together with the shape data and the synthesized image.

  FIG. 13 is a flowchart showing an overall procedure of the surface defect inspection method according to the embodiment.

  First, the calibration unit 321 performs calibration (step S100). That is, the calibration unit 321 estimates the correspondence between shape data and image data.

  Next, external data is acquired by the surface data acquisition device 100 (step S200). That is, the image acquisition unit 110 acquires image data, and the shape measurement unit 120 acquires shape data.

  Next, data composition is performed (step S300). That is, the image data synthesis unit 322 synthesizes image data, and the shape data synthesis unit 323 synthesizes shape data. Next, the position adjustment / surface recognition unit 324 performs position adjustment of the reference outer shape data and the shape measurement data (step S400). After the position adjustment, the interpolation unit 325 associates the image data with the shape data using the result of the calibration unit 321, and interpolates the coordinate points between the pixels and the shape measurement points (step S500).

  Next, the surface defect is determined (step S600). FIG. 14 is a flowchart showing a basic procedure of the surface defect determination step S600 in the surface defect inspection method according to the embodiment.

  First, the preprocessing unit 331 of the defect evaluation unit 330 performs preprocessing common to both the image data and the shape data based on the shape data combined by the shape data combining unit 323 and the reference outline data relating to the original shape. Then, the process of specifying the region where the shape has changed is performed (step S610).

  Next, the image data surface defect determination unit 332 performs surface defect recognition processing based on the image data (step S620). FIG. 15 is a flowchart showing a procedure of surface defect recognition processing S620 from image data in the surface defect determination step S600 in the surface defect inspection method according to the embodiment.

  First, the image data reading unit 332a reads image data from the outer shape data storage unit 210 (step S621). Next, the image data surface defect determination unit 332 extracts a setting area (step S622), and the edge detection unit 332b performs edge detection processing (step S623).

  Next, the closed region detection unit 332c detects a closed region of a defect (Step S624). Thereafter, the erroneously detected part removing unit 332d performs an erroneously detected part removal process (step S625). Next, the image data surface defect determination unit 332 determines whether or not the processing for all target regions has been completed (step S626). If it is determined that it has not been completed (NO in step S626), step S622 and subsequent steps are repeated. When it is determined that the process is completed (YES in step S626), the defective area connecting unit 332e performs a defective area connecting process (step S627).

  In FIG. 14, the shape data surface defect determination unit 333 performs defect recognition processing based on the shape data (step S630). In addition, the order of step S620 and step S630 is not ask | required. You may go in parallel.

  FIG. 16 is a flowchart showing a procedure of surface defect recognition processing S630 from shape data in the surface defect determination step S600 in the surface defect inspection method according to the embodiment.

  First, the shape data reading unit 333a reads shape data from the outer shape data storage unit 210 (step S631). The small region extraction unit 333b extracts the setting region (step S632), and the depth direction determination unit 333c confirms the surface information of the setting region (step S633).

  Next, the data projection unit 333d performs data projection processing in accordance with the circumferential direction. Further, the noise removing unit 333e performs noise removal (step S634). Thereafter, the edge detection unit 333f performs edge detection (step S635), and the closed region detection unit 333g performs defect closed region detection processing (step S636). Further, the erroneously detected part removing unit 333h performs a process of removing the erroneously detected part (step S637).

  Next, the shape data surface defect determination unit 333 determines whether or not the processing for all target regions has been completed (step S638). If it is determined that the process has not been completed (NO in step S638), step S632 and subsequent steps are repeated. If it is determined that the process is completed (YES in step S638), the defective area connecting unit 333k performs a defective area connecting process (step S639).

  Next, the defect determination unit 334 uses the defect recognition results of both the image data surface defect determination unit 332 and the shape data surface defect determination unit 333 to determine whether or not it is a defect, that is, integrated determination of defect data. Processing is performed (step S640). After the above surface defect discrimination (step S600), as shown in FIG. 13, surface defect size estimation is performed (step S700). Thereafter, the result creating unit 340 creates a record based on the result of the surface defect size estimation (step S800). The output unit 420 outputs the result by display or printing.

  According to the present embodiment as described above, in the surface defect inspection, by performing arithmetic processing based on both the image data and the shape data, the detection rate is increased, and the level of the surface defect size to be described in the inspection record is set. Automatic inspection that enables measurement is possible.

[Other Embodiments]
As mentioned above, although some embodiment of this invention was described, these embodiment is shown as an example and is not intending limiting the range of invention. For example, in the embodiment, the surface defect size estimation unit 335 has shown a case where the estimation is performed based on the logical product of the defect recognition result in the image data and the defect recognition result in the shape data, but the present invention is not limited to this. For example, when a logical sum, that is, a defect recognition result in image data and either shape data or a defect recognition result is recognized as a defect, it may be determined as a defect. However, in this case, it is effective for subsequent investigation to attach information indicating that only one of the recognition results is obtained.

  Moreover, you may combine the characteristic of each embodiment. Furthermore, these embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and the equivalents thereof.

  DESCRIPTION OF SYMBOLS 100 ... Surface data acquisition apparatus, 110 ... Image acquisition part, 111 ... Lens, 112 ... All mirrors, 113 ... Half mirror, 120 ... Shape measurement part, 121 ... Laser irradiation part, 122 ... Camera for laser beam photography, 130 ... Scan , 131 ... Movement drive unit, 132 ... Rotation drive unit, 133 ... Fixing unit, 200 ... Memory, 210 ... Outer shape data storage unit, 220 ... Calculation result storage unit, 300 ... CPU, 310 ... Control unit, 320 ... Calculation unit 321 ... Calibration unit 322 ... Image data synthesis unit 323 ... Shape data synthesis unit 324 ... Position adjustment / surface recognition unit 325 ... Interpolation unit 330 ... Defect evaluation unit 331 ... Preprocessing unit 332 ... Image Data surface defect determination unit, 332a... Image data reading unit, 332b... Edge detection unit, 332c. , 332e..., Defect area connecting section, 333... Shape data surface defect determination section, 333a... Shape data reading section, 333b... Small area extraction section, 333c. , 333 f... Edge detection unit, 333 g... Closed region detection unit, 333 h... Erroneous detection part removal unit, 333 k... Defect region connection unit, 334. 335b ... shape reference estimation unit, 340 ... result creation unit, 400 ... surface defect evaluation device, 410 ... input unit, 420 ... output unit, 500 ... defect measurement system

Claims (12)

An input unit that receives, as input, a two-dimensional image of the appearance of the object to be inspected acquired by the image acquisition unit, and a three-dimensional shape of the object measured by the shape measuring unit;
And adjusting the position of the shape data is information about the three-dimensional shape that has been determined serving as a reference standard contour data of the determination in the Ah Ri secular variations or defects in the original data for the object, obtains A calculation unit that acquires information on surface defects existing on the surface of the object based on image data that is information on the two-dimensional image and the shape data;
A surface defect evaluation apparatus comprising:   The calculation unit performs position adjustment by using at least three points in the shape data of the object and the reference outline data in association with each other in the position adjustment. The surface defect evaluation apparatus described in 1.   In the position adjustment, the calculation unit adjusts the position by setting at least three reference positions from a region where a shape change amount based on the shape data of the object and the reference outer shape data is a predetermined threshold value or less. The surface defect evaluation apparatus according to claim 1, wherein:   2. The surface defect evaluation apparatus according to claim 1, wherein, in the position adjustment, the calculation unit specifies a region in which the shape of both the shape data of the target object and the reference external shape data matches with each other by matching. . An input unit that receives, as input, a two-dimensional image of the appearance of the object to be inspected acquired by the image acquisition unit, and a three-dimensional shape of the object measured by the shape measuring unit;
Image data that is information related to the two-dimensional image obtained by adjusting the position of the reference outer shape data that is original data related to the object and the shape data that is information related to the measured three-dimensional shape. And a calculation unit that acquires information on surface defects existing on the surface of the object based on the shape data,
A surface defect evaluation apparatus comprising:
In the position adjustment, the arithmetic unit uses at least three points in the shape data of the object and the reference outline data, and performs position adjustment by associating them with each other. apparatus. An input unit that receives, as input, a two-dimensional image of the appearance of the object to be inspected acquired by the image acquisition unit, and a three-dimensional shape of the object measured by the shape measuring unit;
Image data that is information related to the two-dimensional image obtained by adjusting the position of the reference outer shape data that is original data related to the object and the shape data that is information related to the measured three-dimensional shape. And a calculation unit that acquires information on surface defects existing on the surface of the object based on the shape data,
A surface defect evaluation apparatus comprising:
In the position adjustment, the calculation unit adjusts the position by setting at least three reference positions from a region where a shape change amount based on the shape data of the object and the reference outer shape data is a predetermined threshold value or less. The surface defect evaluation apparatus characterized by performing .   An input unit that receives, as input, a two-dimensional image of the appearance of the object to be inspected acquired by the image acquisition unit, and a three-dimensional shape of the object measured by the shape measuring unit;
  Image data that is information related to the two-dimensional image obtained by adjusting the position of the reference outer shape data that is original data related to the object and the shape data that is information related to the measured three-dimensional shape. And a calculation unit that acquires information on surface defects existing on the surface of the object based on the shape data,
  A surface defect evaluation apparatus comprising:
  In the position adjustment, the calculation unit specifies an area where the shapes of the shape data and the reference external shape data of the target object coincide with each other by matching.   The computing unit is
  A calibration unit for deriving a correspondence relationship between the coordinate system of the image data acquired by the image acquisition unit and the coordinate system of the shape data measured by the shape measurement unit;
  An interpolation unit that interpolates the shape data by interpolation for a portion where the image data exists but the shape data does not exist, using the correspondence relationship derived by the calibration unit;
  Based on the image data, image data surface defect determination unit for determining surface defects;
  Based on the shape data, a shape data surface defect determination unit for determining surface defects;
  A surface defect size estimation unit that identifies a surface defect location based on a determination result by the image data surface defect determination unit and a determination result by the shape data surface defect determination unit, and estimates a size of the surface defect location; ,
  The surface defect evaluation apparatus according to any one of claims 1 to 7, wherein the surface defect evaluation apparatus includes:   The identification of the surface defect location is a logical product of the determination result of the surface defect location by the image data surface defect determination portion and the determination result of the surface defect location by the shape data surface defect determination portion, or 9. The surface defect evaluation apparatus according to claim 1, wherein the surface defect evaluation apparatus is based on any one of logical sums.   A surface defect inspection system comprising a surface data acquisition device and a surface defect evaluation device,
  The surface data acquisition device includes:
  An image acquisition unit for acquiring a two-dimensional image of the appearance of an object to be inspected;
  A shape measuring unit for obtaining a three-dimensional shape of the object;
  Comprising
  The surface defect evaluation apparatus is
  An input unit that accepts the two-dimensional image and the three-dimensional shape as inputs;
  This is the original data related to the object and is obtained by adjusting the position of the reference external shape data which is the reference for the judgment of the deformation and defects over time and the shape data which is information about the measured three-dimensional shape. A calculation unit that acquires information on surface defects existing on the surface of the object based on image data that is information on the two-dimensional image and the shape data;
  A surface defect inspection system comprising:   An image receiving step in which the input unit of the surface defect evaluation apparatus receives a two-dimensional image of the appearance of the target object acquired by the image acquisition unit;
  A shape receiving step in which the input unit receives a three-dimensional shape of the object measured by the shape measuring unit;
  The calculation unit adjusts the position of the original shape data that is the original data regarding the target object and the reference shape data that is a criterion for the determination of deformation and defects over time and the information about the measured three-dimensional shape. A position adjustment step to be performed;
  A surface defect information acquisition step in which the arithmetic unit acquires information about a surface defect existing in the object based on the image data and the shape data which are information about the acquired two-dimensional image;
  A surface defect inspection method characterized by comprising:   The surface defect information acquisition step includes:
  A calibration step for deriving a correspondence relationship between the coordinate system of the image data acquired by the image acquisition unit and the coordinate system of the shape data measured by the shape measurement unit;
  An interpolation step for interpolating the shape data by interpolation for a portion where the image data exists and the shape data does not exist, using the correspondence relationship derived by the calibration unit;
  An image data surface defect determining unit, based on the image data, an image data surface defect determining step for determining a surface defect;
  A shape data surface defect determination unit, based on the shape data, a shape data surface defect determination step for determining a surface defect;
  The surface defect size estimation unit identifies a surface defect location based on the discrimination result by the image data surface defect discrimination unit and the discrimination result by the shape data surface defect discrimination unit, and estimates the size of the surface defect location A surface defect size estimation step,
  The surface defect inspection method according to claim 11, comprising:

Images