KR20210117456A - Foreign material detecting device and method for camera module using contrast - Google Patents

Abstract

Disclosed are a device and a method for detecting a foreign material utilizing contrast. According to one embodiment of the present invention, the device for detecting a foreign material is to detect a foreign material of a camera module. The device includes: an image acquisition unit for acquiring and averaging input images photographed using the camera module; a contrast image generation unit for generating a contrast image by performing a contrast stretching operation on the average image; an image processing unit for emphasizing foreign materials by applying a predetermined image processing algorithm to the contrast image; and an output unit for outputting the image passed through the image processing unit through a display.

Description

Foreign material detecting device and method for camera module using contrast

The present invention relates to an apparatus and method for detecting a foreign material in a camera module, and more particularly, to an apparatus and method for detecting a foreign material in a camera module using contrast.

When inspecting a foreign material for a camera module, conventionally, a method of visually checking was used. However, in this case, the cause of the defect could not be grasped, thereby lowering the productivity efficiency, and thus there was a problem regarding the destruction of the product.

As systemization progresses, various methods of detecting foreign substances are introduced, and the detection of foreign substances is carried out, and the productivity can be increased by identifying the problems and then cleaning and reassembling the camera module.

However, the problem of detection power with respect to projection and weak foreign body standards is constantly being raised. Projectability is a problem in which light transmits even in the presence of foreign objects and looks like a normal result. It refers to a phenomenon in which a foreign material is dispersed and becomes invisible when a penetrating foreign material is attached or as the size of the resolution increases. Weakness is one of the criteria for concentration (one of stiffness/weakness) when contrast is performed, and indicates a case where the difference in concentration is small.

As the resolution increases in the camera module and the size of the lens increases, the detection standard for the expansion of the foreign material is maintained even though the detection standard for the expansion of the foreign material is changed. There is a problem.

In the case of the existing method, the cause of the defective product is identified according to the characteristics by examining the luminance/color data on the pixel value of the screen using the white light data to determine whether a foreign object is attached, but if a foreign material is attached to the sensing part, it is grayed out. It is difficult to judge the characteristic of having a close illuminance value according to the concentration of foreign substances. Also, in the case of a foreign material defect, the possibility of leakage of the mass production process due to the failure of the foreign material inspection cannot be excluded depending on the error in the data accumulation and conversion range of the defect basis. should be given, but it is not easy to apply to the mass production process. In addition, with respect to the algorithm for highlighting high and low luminance, in the case of the camera module, it is difficult to operate due to the severe deviation of high and low luminance as a reference for data at the same location as a phenomenon in which a lot of data is concentrated depending on the curved surface of the lens and darkens. have.

Korean Patent Application Laid-Open No. 10-2005-0117424 (published on Dec. 14, 2005) - Apparatus and method for detecting foreign substances in a digital camera module

The present invention uses contrast to specify criteria for foreign material defects and to derive reliable results for visual judgment comparison analysis between the image judgment criteria for the output results and the evaluated criteria of the camera module using the contrast An object of the present invention is to provide an apparatus and method for detecting a foreign material.

An object of the present invention is to provide an apparatus and method for detecting a foreign material in a camera module using a contrast that improves the improvement effect by quickly identifying a problem by an administrator by outputting definitive result data for a cause of a decrease in yield due to a foreign material.

Objects other than the present invention will be easily understood through the following description.

According to an aspect of the present invention, there is provided a foreign material detection apparatus for detecting a foreign material in a camera module, comprising: an image obtaining unit for obtaining and averaging an input image photographed using the camera module; a contrast image generator configured to generate a contrast image by performing a contrast stretching operation on the average image; an image processing unit for emphasizing foreign objects by applying a predetermined image processing algorithm to the contrast image; and an output unit for outputting the image passed through the image processing unit through a display.

The contrast image generating unit may generate the contrast image by extending the contrast by changing the effective range between the minimum value and the maximum value among the brightness values of the average image to the maximum range between the absolute minimum value and the absolute maximum value.

The contrast image generator may perform a contrast stretching operation by using a gradient of a deviation amount between the minimum value and the maximum value.

The changed brightness value P of each pixel of the contrast image may be calculated as follows.

이고, 가중치

이며, 기울기

이다. Here, Y is the pixel value of the RGB channel of the corresponding pixel of the average image.

and weight

and the slope

am.

The image processing unit may include a binarization processing module that divides the contrast image into a preset number of fragment images, analyzes the shape of a range of brightness values of pixels for each fragment image, and performs individualized binarization processing for each fragment image. have.

The binarization processing module obtains a histogram of the brightness value for each fragment image, and sets a threshold value for the fragment image based on the minimum value among points where the increase and decrease of the frequency in the histogram change, and is lower than the threshold value. The frequency can be excluded.

The image processing unit may further include a noise removal module for removing noise from the contrast image that has been binarized through a morphological erosion operation.

According to another aspect of the present invention, there is provided a method for detecting a foreign material in a camera module performed in a foreign material detection apparatus, the method comprising: acquiring and averaging input images photographed using the camera module; generating a contrast image by performing a contrast stretching operation on the average image; emphasizing foreign objects by applying a predetermined image processing algorithm to the contrast image; and outputting the image passed through the image processing unit through a display.

In the step of generating the contrast image, the contrast image can be created by changing the effective range between the minimum value and the maximum value among the brightness values of the average image to the maximum range between the absolute minimum value and the absolute maximum value to extend the contrast. have.

In the generating of the contrast image, a contrast stretching operation may be performed using a gradient of a deviation amount between the minimum value and the maximum value.

The step of embossing the foreign material may include: dividing the contrast image into a preset number of fragment images; and performing individualized binarization for each of the fragment images by analyzing the range shape of the brightness values of pixels for each fragment image.

The step of reducing the foreign object may further include removing noise from the contrast image that has been binarized through a morphological erosion operation.

Other aspects, features and advantages other than those described above will become apparent from the following drawings, claims, and detailed description of the invention.

According to an embodiment of the present invention, the effect of deriving a reliable result for the visual judgment comparison analysis of the image judgment standard for the output result and the evaluated standard is the effect of specifying the criteria for the foreign material defect by using the contrast have.

In addition, by outputting the definitive result data for the cause of the decrease in yield due to foreign substances, the problem of the manager is quickly identified and the improvement effect is increased.

1 is a block diagram of a foreign object detection apparatus using contrast according to an embodiment of the present invention;
2 is a flowchart of a foreign material detection method using contrast according to an embodiment of the present invention;
3 is a view showing images (input image, average image, contrast image, output image) according to the foreign material detection process;
4 is a view showing image segmentation and histogram for the binarization process;
5 is a view for explaining a binarization process in the first histogram;
6 is a diagram illustrating a morphology calculation process.

Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.

Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

The terms used herein are used only to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In this specification, terms such as "comprises" or "have" are intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, but one or more other features It should be understood that this does not preclude the existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

In addition, the components of the embodiment described with reference to each drawing are not limitedly applied only to the embodiment, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention, and also Even if the description is omitted, it is natural that a plurality of embodiments may be re-implemented as a single integrated embodiment.

In addition, in the description with reference to the accompanying drawings, the same components regardless of the reference numerals are given the same or related reference numerals, and the overlapping description thereof will be omitted. In describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In the drawings attached to this specification, colors are given to help distinguish the components. However, even the same component may have different colors in the perspective view and the cross-sectional view. In addition, the same or similar color may be assigned to different components.

In addition, terms such as “…unit”, “…unit”, “…module”, “…group”, etc. described in the specification mean a unit that processes at least one function or operation, which is hardware or software or hardware and software. It can be implemented by combining

1 is a block diagram of a foreign material detection apparatus using contrast according to an embodiment of the present invention, FIG. 2 is a flowchart of a foreign material detection method using contrast according to an embodiment of the present invention, and FIG. 3 is foreign material detection It is a view showing images (input image, average image, contrast image, output image) according to the process, Fig. 4 is a view showing image segmentation and histogram for the binarization process, and Fig. 5 is the binarization process in the first histogram It is a diagram for explaining a process, and FIG. 6 is a diagram illustrating a morphology calculation process.

The foreign object detection apparatus and method using contrast according to an embodiment of the present invention is visually output to the camera module by making the foreign object stand out relatively through predetermined image processing for an image acquired from a camera module to be inspected. It is characterized in that foreign matter is not omitted, and a more explicit and reliable evaluation standard for foreign matter defect, which is ambiguous for the manager, can be provided.

Referring to FIG. 1 , the apparatus 100 for detecting a foreign object using contrast according to the present embodiment includes an image acquisition unit 110 , a contrast image generation unit 120 , an image processing unit 130 , and an output unit 140 . can do.

The image acquisition unit 110 acquires an input image photographed using a camera module to be inspected (step S200). For this, an image obtained by photographing a white swab (or a light source) and/or a black swab may be acquired as an input image by using the camera module. In the case of an input image obtained by photographing a white interleaf, a black foreign material may be easily detected, and in the case of an input image obtained by photographing a black interleaf, a white foreign material may be easily detected.

In this embodiment, it is intended to provide a reliable evaluation standard for the camera module to check the foreign material. Since the same environment is required as a condition to provide reliable evaluation criteria, an average image obtained by averaging a preset number of image frames included in the input image acquired by the image acquisition unit 110 may be generated (step S210).

When a specific viewpoint is photographed with the camera module, a plurality of image frames may be simultaneously acquired before and after that, and the input image may consist of a predetermined number of image frames that are continuous or have a predetermined interval. The number of image frames for generating the average image may be variably set as needed.

The average image may be generated by averaging pixel values (eg, RGB values) at each coordinate for a plurality of image frames. That is, the average pixel value obtained by averaging the pixel values p_1(x1, y1) of the (x1, y1) coordinates of the first image frame to the pixel values p_n(x1, y1) of the (x1, y1) coordinates of the nth image frame It can be set as the pixel value (p_avg(x1, y1)) of the (x1, y1) coordinates of the average image.

The contrast image generator 120 generates a contrast image by performing a contrast stretching operation on the average image (step S220).

In contrast stretching, the effective range between the minimum value and the maximum value among the brightness values of the average image is changed to the maximum range between the absolute minimum value (eg 0) and the absolute maximum value (eg 255) in which the image can express the contrast by changing the brightness value. This is the first tuning process that extends To this end, a slope for the amount of deviation between the maximum value and the minimum value may be used.

Assuming that the minimum value among the brightness values of all pixels belonging to the average image is Min and the maximum value is Max, the slope a and the weight W for the amount of deviation between the maximum value and the minimum value are obtained as in Equations 1 and 2 below.

The changed brightness value P of each pixel of the contrast image may be calculated as in Equations 3 and 4 below.

Y means the pixel value (pixel data) of the RGB channel of the corresponding pixel of the average image.

P is the tuned pixel brightness value, and can be changed to 255 when the calculated P is greater than 255, and to 0 when the calculated P is less than 0.

By tuning the brightness value of each pixel by the contrast image generator 120 , a contrast image having a greater contrast than the average image may be generated.

The image processing unit 130 may apply a predetermined image processing algorithm to the contrast image to make the foreign material stand out more explicitly.

The image processing unit 130 may include a binarization processing module 131 .

The binarization processing module 131 may divide the contrast image into a preset number of fragment images (step S230). Referring to FIG. 3 , an example in which the contrast image 300 is divided into six slice images ROI_1 to ROI_6 is illustrated. Each slice image ROI_n may have the same size and shape.

The binarization processing module 131 analyzes the characteristics of each segmented fragment image (ROI_n), that is, the shape of the range of brightness values of pixels, and sets an individualized binarization standard for each fragment image (ROI_n) to perform binarization processing. have.

As an example, the binarization processing module 131 performs a histogram conditional calculation on each segmented fragment image ROI_n (step S240).

Referring to FIG. 4 , histograms H_1 to H_6 of the pixel brightness values for each slice image ROI_n may be obtained. In each histogram (H_n), the X-axis represents the brightness value of a pixel, and the Y-axis represents the frequency of the brightness value.

Set the threshold value for the corresponding fragment image based on the minimum value among the points where the frequency changes between rising and falling in the histogram. This threshold is a criterion for binarization of sub data in the contrast image.

5 shows a histogram of the first slice image ROI_1.

In the histogram, assuming that the value with the most frequency is M_1 and the second most frequent value is M_2, the value having the minimum frequency among points where the rise and fall between M_1 and M_2 changes can be set as Mid. M_1 <= Mid && Mid <= M_2 or M_2 <= Mid && Mid <= M_1.

With Mid as a threshold value, exclusion processing (deletion processing) can be performed for the frequency with a value lower than Mid in the corresponding fragment image. In FIG. 5 , pixel values (pixel values smaller than M_3 or larger than M_4) having a frequency smaller than the frequency amount F_Mid of Mid may be determined as noise and the data may be ignored.

In addition, the brightness value of pixels belonging to the left side of Mid in the histogram can be changed to 0 (zero).

Mid may be set differently according to the brightness distribution of the fragment image ROI_n, and more sophisticated binarization processing may be possible for each fragment image.

Also, the image processing unit 130 may further include a noise removing module 133 .

The noise removal module 133 may remove noise from the binarized contrast image through a morphology operation (step S250).

In the present embodiment, an erosion operation of a 3 x 3 array may be applied as a morphology operation.

Referring to FIG. 6 , in the case of the first pixel P1 , a first input image range of a 3×3 array centered on the first pixel P1 is set. And apply a 3 x 3 erosion mask with 1 set in the center of the ten (+) shape. Since one or more pixel values corresponding to 1 in the 3 x 3 erosion mask in the first input image range do not match, the output pixel value of the first pixel P1 having a value of 1 is changed to 0.

In the case of the second pixel P2 , a range of a second input image of a 3×3 array centered on the second pixel P2 is set. Then apply the same 3 x 3 erosion mask. Since all pixel values corresponding to 1 in the 3 x 3 erosion mask in the first input image range are the same, the output pixel value of the second pixel P2 having a value of 1 is maintained as it is.

Through this morphological operation, noise (especially, noise around foreign objects) can be removed from the binarized contrast image.

The output unit 140 outputs the contrast image on which the binarization process and noise removal have been completed through the display (step S260). In the image output through the display, only the portion corresponding to the foreign material is highlighted and displayed, so that the administrator can easily check the existence and characteristics (size, location) of the foreign material.

According to this embodiment, it is possible to provide a result that can specify the selection criterion when the criterion for defective foreign matter is ambiguous, and provides a reliable result for the visual judgment comparison analysis of the image judgment criterion and the evaluation criterion for the output result. can give

In addition, by outputting the definitive result data for the cause of the decrease in yield due to foreign substances, the problem of the manager can be quickly identified and the improvement effect of the production process can be increased.

In addition, it is possible to obtain fast results even with respect to the change of the detection standard for the expansion of the foreign material according to the increase in resolution and the increase in the size of the lens angle of view.

The above-described foreign material detection method may also be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described foreign object detection method may be executed by an application basically installed in the terminal (which may include a program included in a platform or operating system basically mounted in the terminal), and a user may use an application store server, an application, or a corresponding service. It may be executed by an application (ie, a program) installed directly on the master terminal through an application providing server such as a web server related to the . In this sense, the above-described foreign material detection method may be implemented as an application (ie, a program) installed by default in a terminal or directly installed by a user, and may be recorded in a computer-readable recording medium such as a terminal.

Although the above has been described with reference to the embodiments of the present invention, those of ordinary skill in the art can variously modify and change the present invention within the scope without departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

100: foreign material detection device 110: image acquisition unit
120: contrast image generating unit 130: image processing unit
131: binarization processing module 133: noise removal module
140: output unit

Claims (12)

A foreign material detection device for detecting a foreign material in a camera module, comprising:
an image acquisition unit for acquiring and averaging input images photographed using the camera module;
a contrast image generator configured to generate a contrast image by performing a contrast stretching operation on the average image;
an image processing unit for emphasizing foreign objects by applying a predetermined image processing algorithm to the contrast image; and
and an output unit for outputting the image passed through the image processing unit through a display.
According to claim 1,
The contrast image generating unit expands the contrast by changing the effective range between the minimum and maximum values among the brightness values of the average image to the maximum range between the absolute minimum and absolute maximum values to generate the contrast image. foreign object detection device.
3. The method of claim 2,
The contrast image generator is configured to perform a contrast stretching operation by using a gradient of a deviation amount between the minimum value and the maximum value.
4. The method of claim 3,
The changed brightness value P of each pixel of the contrast image is a foreign material detection device, characterized in that it is calculated as follows;

Here, Y is the pixel value of the RGB channel of the corresponding pixel of the average image.

ego,
weight

is,
inclination

Lim.
According to claim 1,
The image processing unit includes a binarization processing module that divides the contrast image into a preset number of fragment images, analyzes the shape of a range of brightness values of pixels for each fragment image, and performs individualized binarization for each fragment image. detection device.
6. The method of claim 5,
The binarization processing module obtains a histogram of the brightness value for each fragment image, and sets a threshold value for the fragment image based on the minimum value among points where the increase and decrease of the frequency in the histogram change, and is lower than the threshold value. A foreign material detection device characterized in that the frequency amount is excluded.
6. The method of claim 5,
The image processing unit further comprises a noise removal module for removing noise from the contrast image that has been binarized through a morphological erosion operation.
A method of detecting a foreign material in a camera module performed in a foreign material detection apparatus, the method comprising:
acquiring and averaging input images photographed using the camera module;
generating a contrast image by performing a contrast stretching operation on the average image;
emphasizing foreign objects by applying a predetermined image processing algorithm to the contrast image; and
and outputting the image passed through the image processing unit through a display.
9. The method of claim 8,
In the step of generating the contrast image, the effective range between the minimum value and the maximum value among the brightness values of the average image is changed to the maximum range between the absolute minimum value and the absolute maximum value to expand the contrast to generate the contrast image A foreign material detection method characterized in that.
10. The method of claim 9,
The generating of the contrast image comprises performing a contrast stretching operation by using a gradient of a deviation amount between the minimum value and the maximum value.
9. The method of claim 8,
The foreign material relief step is,
dividing the contrast image into a preset number of fragment images;
and performing individualized binarization for each of the fragment images by analyzing the range shape of the brightness values of pixels for each fragment image.
12. The method of claim 11,
The step of reducing the foreign material further includes removing noise from the contrast image that has been binarized through a morphological erosion operation.

Images