KR20220095100A - Image processing method for object with smooth surface on optical detection and detection system thereof - Google Patents

Abstract

The present invention provides an image processing method for optical measurement of an object to be measured having a smooth surface, and a measurement system thereof. According to the image processing method, images of different light irradiation directions of an object to be measured are captured and synthesized based on a photometric stereoscopic method, and an image data pre-processing step is performed after the image is obtained and before the image is synthesized, to perform non-linear adjustment of a gray scale value of each pixel, such that, by settings, gray scale change ratios of a plurality of pixels whose gray scale value before adjustment is less than a gray scale threshold value are relative to grays scale change ratios of other pixels whose gray scale value before adjustment is not less than the gray scale threshold value. Accordingly, the measurement system based on the photometric stereoscopic method provided by the present invention may be applied to measure an object to be measured having a smooth surface after corresponding image processing and is not limited by uniform parallel irradiation light.

Description

Optical measurement image processing method of a measurement object having a smooth surface and a measurement system therefor

The present invention relates to an optical measurement image processing method and a measurement system therefor, and more particularly, to an optical measurement image processing method and a measurement system for a measurement object having a smooth surface.

Photometric Stereo Method (PSM) is a calculation method that reconstructs object surface information derived from optical projection image formation. Multiple images of a measurement object are acquired from the same shooting angle through a single camera, and the measurement object is illuminated with light. After being irradiated one after another by irradiated light with different irradiation directions, the images are superimposed by an arithmetic method to generate a composite image.

Traditionally, the photometric stereoscopic method uses a perfect diffusion model of optics to calculate gradients on the surface of the measurement object, then acquires a three-dimensional model through vector field integration, and then calculates the light intensity distribution on the surface of the measurement object. acquire

Photometric stereopsis uses the light intensity value as the basis of the calculation of the entire model, and it is widely is being applied.

However, since the photometric stereoscopic method has to find a solution using the perfect diffusion model of optics, its application is limited by the condition that it is a measurement object with a rough surface and a uniform, parallel light source must be present together. Therefore, measurement cannot be carried out in the presence of a measurement object with a smooth surface or easily reflective properties, and a light source that is not aimed.

One of the objects of the present invention is to apply the photometric stereoscopic method to the measurement of a measurement object having a smooth surface and not be limited by the conditions of uniform parallel irradiation light.

Another object of the present invention is to make it possible to easily measure even a minute defect protruding or concave of a measurement object made of a metal material.

Another object of the present invention is to improve the accuracy of measurement of defects in a measurement object having a smooth surface while using a relatively small measurement system arrangement space.

In order to achieve the above and other objects, the present invention provides an optical measurement image processing method of a measurement object having a smooth surface. This method includes the steps of sequentially acquiring each original image corresponding to each light irradiation direction based on each different light irradiation direction for the measurement object - the number of acquisitions of the corresponding original image is at least three; Non-linear adjustment is performed on the gray scale value before adjustment of each pixel in each original image to generate a corresponding gray scale value after adjustment. When the gray scale limit value is smaller than the invert limit value, the gray scale value before adjustment is gray Before adjusting the gray scale change ratio of a plurality of pixels smaller than the scale limit value, the gray scale value is made larger than the gray scale change ratio of the remaining pixels that are not smaller than the gray scale limit value. When the gray scale limit value is greater than the invert limit value, an image data pre-processing step of adjusting the gray scale change ratio of a plurality of pixels whose gray scale value before adjustment is smaller than the gray scale limit value to be smaller than the gray scale change ratio of the remaining pixels whose gray scale value is not smaller than the gray scale limit value; and a synthesis step of synthesizing each original image processed through the image data pre-processing step based on the photometric stereoscopic method into a composite image used for subsequent defect measurement.

In one embodiment of the present invention, an image data post-processing step may be further included after the synthesizing step. In this step, all gradient values smaller than the first gradient limit value or greater than the second gradient limit value among the corresponding gradient distribution data are adjusted to 0 based on the gray scale distribution data of the synthesized image, and gray scale values of 0 to 255 are adjusted to 0. is distributed between the first gradient limit value and the second gradient limit value to generate an analysis target image having a new gray scale distribution used for subsequent defect measurement.

In one embodiment of the present invention, the first gradient limit value taken as an absolute value may be equal to the numerical value of the second gradient limit value.

In one embodiment of the present invention, the first gradient limit value may be -0.5, and the second gradient limit value may be 0.5.

In one embodiment of the present invention, the gray scale value of the new gray scale distribution to which the gradient value 0 corresponds may be 128.

In one embodiment of the present invention, the surface roughness Ra value of the area to be measured of the measurement object is less than or equal to 1.6 (μm).

In order to achieve the above object and other objects, the present invention also provides a measurement system using the image processing method. The system includes a stand for placing the measurement object; an image acquisition device disposed on the stand; a light source module installed to surround the image acquisition device; and a control host connected to the image acquisition device and the light source module. The light source module includes a plurality of light source devices, and the control host executes the image processing method.

In one embodiment of the present invention, the control host may be used to implement a defect determination program. Here, a pixel whose gray scale value of the analysis target image is greater than the protrusion defect threshold is defined as a protrusion defect, and a pixel whose gray scale value of the analysis target image is smaller than the concave defect threshold is defined as a concave defect.

In one embodiment of the present invention, a narrow angle between the light irradiation direction of each light source device and the optical axis of the image acquisition device may be 0 to 30 degrees.

Accordingly, in the embodiment presented by the present invention, the nonlinear transformation pre-processing step is performed on the input image data, and the computing model operated based on the photometric stereoscopic method can accurately process the image data from the measurement object having a smooth surface. and improve the problem of uneven brightness in the image, so that it can be applied to the measurement object that easily reflects. In addition, if a limited post-processing of normalization is performed on the synthesized image, the problem of inconsistent image data sections can be improved, so that even minute defects can be accurately measured.

Therefore, the measurement system based on the photometric stereoscopic method proposed by the present invention can be applied to the measurement of a measurement object having a smooth surface after the corresponding image processing, and is not limited to the use of uniform parallel irradiation light.

1 is a flowchart of an image processing method according to an embodiment of the present invention.
2 is a flowchart of an image processing method according to another embodiment of the present invention.
3 is a schematic diagram of a measurement system in one embodiment of the present invention;
Fig. 4 is a schematic view of the measurement system of Fig. 3 in a top down view;

In order to fully understand the object, features and effects of the present invention, the present invention will be described in detail as follows in conjunction with the following specific examples and drawings.

In this manual, a step, part, structure, device, module, system, part, or area is described as 'a' or 'a'. This is only for convenience of description and to provide a clear meaning to the scope of the present invention. Accordingly, except where expressly provided otherwise, these descriptions are to be understood as including one or at least one, and the singular also includes the plural at the same time.

As used herein, the term 'comprising, having, having,' or other similar terms is not limited to the members enumerated in this manual and is not explicitly enumerated, but is intended to refer to steps, parts, structures, devices, modules, systems, parts, or regions in common. Other elements unique to phosphorus may also be included.

As used in this manual, terms similar to ordinal numbers, such as 'first' or 'second', are used to distinguish or refer to the same or similar steps, programs, or computing materials, and inevitably, the timing of such steps, programs, or computing materials. It does not imply order. It should be understood that, in some situations or arrangements, the terms ordinal numbers are used interchangeably without affecting the practice of the present invention.

In the embodiment presented by the present invention, the image acquisition is performed twice or more for the measurement object, and the light irradiation conditions at the time of image acquisition are different, so that a plurality of original images that can be computed based on the photometric stereoscopic method are acquired. . However, since the photometric stereoscopic method is relatively applied to a measurement object with a surface similar to the Lambert surface, it is difficult to directly apply the acquired original image to the photometric stereoscopic computing model if the surface of the measurement object is smooth and reflection easily occurs. do. Therefore, in order to take advantage of the photometric stereoscopic method, in the embodiment presented by the present invention, preprocessing is performed on the acquired original image so that the gray scale of the original image can be applied to the photometric stereoscopic computing model, and the photometric stereoscopic method After synthesizing the image by operating the method, the image to be analyzed can be generated by normalizing the image data based on the demand for measurement of defects in the measurement object with a smooth surface. Through this, the consistency of the image data section is enhanced, and furthermore, the defect area can be easily identified in the analysis target image, thereby increasing the accuracy of back-end defect measurement.

The degree of change in the surface contour of the object to be measured can be expressed through the degree of surface roughness (measured according to ISO 13565). For example, it can be expressed as a mathematical average roughness Ra value. When the Ra value is less than or equal to 1.6 (μm), the degree of diffusion is low, so it is difficult to directly apply it to the computing model of the photometric stereoscopic method. The preprocessing presented in this embodiment can be used to solve this problem.

See FIG. 1 . This is a flowchart of an image processing method according to an embodiment of the present invention. First, proceed to step S100 to acquire each original image in different light irradiation directions, then proceed to step S200, which is the image data preprocessing (non-linear adjustment) step, and then proceed to step S300, which is the image synthesis step based on the photometric stereoscopic method. In the process, each original image is synthesized into a composite image so that it can be used for subsequent defect measurement.

In step S100, irradiation images from different light irradiation directions are acquired with respect to the measurement target surface of the measurement object. For example, by sequentially irradiating the surface to be measured using irradiated lights in three different irradiating directions, each original image corresponding to each light irradiating direction may be acquired. A relatively preferred implementation is to capture (capture) three or more corresponding original images as described above using irradiated lights from three or more different irradiation directions.

The non-linear adjustment in step S200 is to process the gray scale value of the image. That is, each non-linear adjustment is performed on the gray scale value G1 before adjustment of each pixel in each original image to generate the gray scale value G2 after the adjustment corresponding thereto. Here, the ratio G2/G1 of the gray scale values before and after the adjustment is the gray scale change ratio.

The non-linear adjustment of step S200 is disparity distribution in which one gray scale limit value is divided into two zone values. The numerical value of the gray scale value G1 before adjustment corresponds to the gray scale representation of one pixel. The gray scale values before and after adjustment of one pixel have the ratio G2/G1. The gray scale limit value is a numerical position for classifying the gray scale value G1 before adjustment, and the selection of the gray scale limit value G1 before adjustment is a condition for dividing the gray scale change rate into two zones. For example, the gray scale limit value (gray scale value G1 before adjustment) of a measurement object with a smooth surface can be selected between 50 and 60, and can be set between 170 and 180 in other situations. The gray scale limit value may be defined as an average gray scale value of the defective point and its surroundings based on the degree of unacceptable defective point. For example, when the unacceptable degree of the defect point and the average gray scale value around it are set to 40, the gray scale limit value may be set to a position greater than 40 (eg, 50). Also, for example, when the unacceptable degree of defect point and the average gray scale value around it are set to 180, the gray scale limit value may be set to a position smaller than 180 (eg, 170 ).

In the implementation manner of disparity distribution of values in two zones, a numerical zone smaller than the gray scale value G1 before adjustment of the gray scale limit value is called a first zone, and a numerical zone larger than the gray scale value G1 before adjustment of the gray scale limit value is called the second zone. When the gray scale threshold value is smaller than the invert threshold value, the non-linear adjustment refers to an adjustment rule that makes both the gray scale change ratio G2/G1 of the first zone larger than the gray scale change ratio G2/G1 of the second zone. Conversely, when the gray scale limit value is greater than the invert limit value, the non-linear adjustment means an adjustment rule that makes both the gray scale change ratio G2/G1 of the first zone smaller than the gray scale change ratio G2/G1 of the second zone do. That is, based on this adjustment rule, a measurement object having a smooth surface can be applied to the computing model of the photometric stereoscopic method. The original image adjusted in step S200 may be synthesized by a subsequent synthesizing step. The invert threshold value can define the boundary positions of the two opposite nonlinear adjustment methods described above, and is used to determine which adjustment rule of the nonlinear adjustment method the gray scale threshold value should correspond to. This invert limit value can be set to any value between 70 and 80.

In step S300, a synthesized image is generated based on the computing model of the photometric stereoscopic method. In this step, an overall surface normal vector is calculated for each input original image, and an orthogonal vector perpendicular to the surface normal vector is calculated based on this, and a depth image is obtained through integration of the vector field. This depth image is a synthesized image that can be used for subsequent defect measurement after synthesizing. The computational model of the photometric stereoscopic method is a well-known computational processing technique, and the light intensity value is based on the calculation of the entire model, and the details of the computation are omitted here.

Next, refer to FIG. 2 . This is a flowchart of an image processing method in another embodiment of the present invention. Compared with the embodiment of FIG. 1 , the image data post-processing step S400 is further included in FIG. 2 . In step S400, the image data is normalized to generate an analysis target image, and through this, the consistency of the image data section is enhanced to increase the accuracy of subsequent defect judgment.

Step S400 may include a step S410 of converting the gray scale value distribution into a gradient value distribution, a step S420 of adjusting the gradient value, and a step S430 of converting the gradient value distribution back to the gray scale value distribution.

In step S410, a gradient value is converted for gray scale distribution data corresponding to each pixel of the composite image. The gradient value refers to the degree of change in gray scale values of adjacent pixels. Therefore, in this step, the degree of change in the gray scale value is calculated for each sequentially arranged pixel based on the order direction of the row or column of the image data. That is, the gradient value and the degree of change of the immediately preceding sequence are calculated, and the gradient distribution data in the sequence direction is acquired. The values of the gradient distribution data are between -1 and 1.

Step S420 means selecting a gradient value having a gradient value smaller than the first gradient limit value or greater than the second gradient limit value from the gradient distribution data, and setting and adjusting all of these selected gradient values to 0. This step is a qualified normalization process. If we add a qualifying treatment to the gradient distribution data, we can make the defects more visible. The first gradient limit value and the second gradient limit value may be two limit values corresponding to the same value after taking an absolute value. For example, in the measurement target area in which the arithmetic mean roughness Ra value of the surface of the measurement object is less than or equal to 1.6 (μm), the first gradient limit value may be selected as -0.5, and the second gradient limit value may be 0.5 It can be selected as , and furthermore, it can be applied to the area to be measured where the arithmetic mean roughness Ra value is less than or equal to 0.8 (μm).

In step S430, gray scale values of 0 to 255 are distributed to the adjusted gradient distribution data to be converted into gray scale distribution data. This step corresponds to the reverse processing of step S410.

Gray scale values of 0 to 255 are distributed between the first gradient limit value and the second gradient limit value to generate an analysis target image having a new gray scale distribution and used for subsequent defect measurement. Since all of the gradient values in the range other than between the first gradient limit value and the second gradient limit value are 0, step S430 distributes the gray scale values of 0 to 255 between the first gradient limit value and the second gradient limit value to create a new It corresponds to generating an analysis target image having a gray scale distribution. A new gray scale value corresponding to a gradient value of 0 may be set to 128 as a transition basis.

Reference is now made to FIGS. 3 and 4 together. 3 is a schematic diagram of a measurement system in one embodiment of the present invention, and FIG. 4 is a schematic diagram of the measurement system of FIG. 3 from a top-down perspective, between the light source device 31 and the light source device 31 and the image acquisition device ( 20) can show the relative position between The measurement system includes a stand 10 , an image acquisition device 20 , a light source module 30 , and a control host 40 . The stand 10 is used to place the measurement object 50 . The image acquisition device 20 is disposed on the stand 10 . The light source module 30 is provided with a plurality of light source devices 31 installed to surround the image acquisition device 20 . The control host 40 is connected to the image acquisition device 20 and the light source module 30 to control the image acquisition device 20 and the light source module 30 and perform the image processing method described above.

In situations where the mechanical conditions are very stringent, the space available for placing the measuring system can be very limited. In particular, the image acquisition device 20 and the light source module 30 must be arranged compactly and located on the stand 10 . Even in order to further reduce the occupied space, the image acquisition device 20 must be located directly above the stand 10 and the light source module 30 must be directly adjacent to the image acquisition device 20 .

Therefore, in such a narrow space condition, it is not possible to do like a general smooth surface measurement arrangement, so that the image acquisition device 20 and the light source module 30 are each mounted on a stand ( 10) cannot be placed on opposite sides or changed relative positions. Accordingly, if the related image data is adjusted by using the well-known photometric stereoscopic method and the image processing method of the present invention together, this technical problem can be solved, and a small protrusion or concave defect of a metal material to be measured can be easily detected. , it also improves the accuracy of measurement of defects in objects with smooth surfaces.

The control host 40 may further execute a defect determination program. When the gray scale value of a pixel on the analysis target image is greater than the protrusion defect threshold value, such a pixel may be defined as a protrusion defect. In addition, when the gray scale value of a pixel on the analysis target image is smaller than the concave defect limit value, such a pixel may be defined as a concave defect. After the image is adjusted, the position and situation of the defect can be determined on the relatively smooth surface of the object to be measured based on the expression of the gray scale value of the pixel and setting the defect limit value. In a general situation, the protrusion defect limit value of the relatively smooth surface of the measurement object may be set to any value between 155 and 160, and the concave defect limit value may be set to any value between 95 and 100. In other situations, by using the gray scale representation of each pixel after image adjustment, the concave defect limit value and the protrusion defect limit value can be set according to the degree of concavity and protrusion to be detected (the condition is stricter or looser). In this way, even if the angle θ between the average light irradiation direction of each light source device 31 and the optical axis of the image acquisition device 20 is within 30 degrees or even between 0 and 10 degrees, according to the image processing method proposed by the present invention, Relevant imaging data can be adjusted and areas of concave or protruding defects can be clearly identified.

Summarizing the above, the measurement system based on the photometric stereoscopic method proposed by the present invention can be applied to measure the surface defect of a measurement object having a smooth surface after the corresponding image processing, and it is possible to avoid the limitation of uniform parallel irradiation light. This can solve the problem that the application target of the photometric stereoscopic method must have a rough surface or a surface that is not easily reflected.

10: stand
20: image acquisition device
30: light source module
31: light source device
40: control host
50: measurement object
S100~S400: Step
S410~S430: Step
θ: narrow angle

Claims (10)

In the optical measurement image processing method of a measurement object having a smooth surface,
sequentially acquiring each original image of the measurement object corresponding to each light irradiation direction based on each different light irradiation direction for the measurement object - the number of the original images is at least 3 -;
Non-linear adjustment is performed on the gray scale value before adjustment of each pixel in each original image to generate a corresponding gray scale value after adjustment, and when the gray scale limit value is smaller than the invert limit value, the gray scale value before adjustment is All of the gray scale change ratios of the plurality of pixels smaller than the gray scale limit value are greater than the gray scale change ratios of the remaining pixels whose gray scale values are not smaller than the gray scale limit value before the adjustment, and the gray scale limit value is the invert When it is greater than the threshold value, the gray scale change ratio of the plurality of pixels whose gray scale value before the adjustment is smaller than the gray scale threshold value is the gray scale change ratio of the remaining pixels whose gray scale value is not smaller than the gray scale threshold value before the adjustment image data pre-processing step of making them all smaller; and
An optical measurement image processing method of a measurement object having a smooth surface, comprising a synthesis step of synthesizing each original image processed through the image data preprocessing step based on the photometric stereoscopic method into a composite image used for subsequent defect measurement . The method according to claim 1,
After the synthesizing step, an image data post-processing step is further included, and in the image data post-processing step, based on the gray scale distribution data of the synthesized image, it is smaller than a first gradient threshold value or a second gradient distribution data of the corresponding gradient distribution data. A new gray scale distribution used for subsequent defect measurement by adjusting all gradient values greater than the gradient limit value to 0, and distributing gray scale values of 0 to 255 between the first gradient limit value and the second gradient limit value An image processing method for generating an analysis target image having 3. The method according to claim 2,
The first gradient threshold value obtained by taking an absolute value is the same as the numerical value of the second gradient threshold value. 3. The method according to claim 2,
The first gradient threshold value is -0.5 and the second gradient threshold value is 0.5. 5. The method according to claim 4,
An image processing method in which the gray scale value of the new gray scale distribution to which the gradient value 0 corresponds is 128. 6. The method of claim 1 to 5,
An image processing method wherein the surface roughness Ra value of the measurement target area of the measurement object is less than or equal to 1.6 (μm). a stand for placing the measurement object;
an image acquisition device disposed on the stand;
a light source module including a plurality of light source devices installed to surround the image acquisition device; and
Using the image processing method according to any one of claims 1 to 6, which is connected to the image acquisition device and the light source module, and includes a control host that performs the image processing method according to any one of claims 1 to 6 measuring system. 8. The method of claim 7,
An angle between the light irradiation direction of each of the light source devices and the optical axis of the image acquisition device is 0 to 30 degrees. a stand for placing the measurement object;
an image acquisition device disposed on the stand;
a light source module including a plurality of light source devices installed to surround the image acquisition device; and
and a control host connected to the image acquisition device and the light source module and configured to perform the image processing method according to claim 5 , wherein the control host executes a defect determination program so that the gray scale value of the analysis target image protrudes. A measurement system using the image processing method according to claim 5 , wherein a pixel larger than the value is defined as a protrusion defect, and a pixel having a gray scale value smaller than a concave defect threshold value of the analysis target image is defined as a concave defect. 10. The method of claim 9,
An angle between the light irradiation direction of each of the light source devices and the optical axis of the image acquisition device is 0 to 30 degrees.

Images