JP2004294202A - Defect detection method and device of screen - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect accurately defects such as a point defect, a stripe defect, a line defect, a stain or an irregularity defect, respectively, to detect not only a bright defect but also a dark defect by the defect detection, and to shorten a processing time when detecting the defects. <P>SOLUTION: An image which is an inspection object is imaged by a CCD camera 6, and the difference from a background image 14 produced beforehand is taken from the image taken by imaging, and an inspection image 15 is produced. Six detection processings for detecting each of the point defect, the stain defect, the stripe defect, the irregularity defect, the line defect and a pixel irregularity defect from the inspection image 15 are performed, and brightness statistical data based on the brightness value of each pixel in the image 17 after each defect detection processing are calculated respectively. A threshold of the brightness value is set based on the brightness statistical data, and a defect candidate is extracted from the brightness statistical data and the threshold. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
INDUSTRIAL APPLICABILITY The present invention automatically and accurately detects defects such as point defects, streak defects, line defects, and spot / mura defects in an inspection process in the production of a display device such as a liquid crystal light valve or a projector which is an application product thereof. The present invention relates to a screen defect detection method and device.
[0002]
[Prior art]
In a conventional screen defect inspection apparatus, it is common that the defect inspection is performed by a single image processing represented by a binarization process. It was impossible. For example, defects appearing on a screen of a liquid crystal display device or the like include point defects, line defects, surface defects (also called spot / uneven defects), and the like. Small defects such as low-contrast defects such as spots and unevenness, point defects, and line defects connected in the pixel scanning direction cannot be detected by simple binarization processing.
Therefore, in detecting a defect appearing on a screen of a liquid crystal display device or the like, a method of separating and extracting each defect while preventing erroneous detection of a false defect by using three-level thresholds for point, line, and surface defects has been proposed. (For example, see Patent Document 1).
In addition, a defect inspection apparatus has been proposed in which various defects such as minute defects and unevenness and streaks are detected by a single inspection apparatus, the type of the defect is reported, and the defect is comprehensively determined (for example, , Patent Document 2).
[0003]
[Patent Document 1]
JP-A-9-288037 (page 1, FIG. 1)
[Patent Document 2]
JP-A-8-145907 (page 1, FIG. 1)
[0004]
[Problems to be solved by the invention]
In the above-described conventional method of separating and extracting point, line, and plane defects by using three-level threshold values, the detection power can be detected for a variety of defect sizes or a defect having a very low contrast. Was insufficient.
In addition, in a defect inspection apparatus that detects various defects with a single inspection apparatus, reports the type of the defect, and comprehensively determines the defect, the amount of change in density between pixels is basically small or large. The defect is determined by the method, which means that the strength of the defect edge is obtained, and the defect size is not taken into consideration, so that the difference from the human judgment in the visual inspection by a person becomes large. was there.
[0005]
The present invention has been made in order to solve such problems, and it is possible to accurately detect defects such as point defects, streak defects, line defects, and spot and unevenness defects. It is an object of the present invention to provide a screen defect detection method and apparatus that can detect not only defects but also dark defects and can shorten the processing time for detecting defects.
[0006]
[Means for Solving the Problems]
The screen defect detection method according to the present invention is a step of capturing an image of an inspection target, and, from an image captured by the imaging, a step of creating an inspection image by taking a difference between a background image created in advance, Six detection processing steps for detecting point defects, spot defects, streak defects, unevenness defects, line defects, and pixel unevenness defects from the inspection image, and luminance statistics based on the luminance value of each pixel in the image after each detection processing The method includes the steps of calculating data, and setting a threshold of a luminance value based on the luminance statistical data, and extracting a defect candidate from the luminance statistical data and the threshold.
[0007]
With this configuration, it is possible to perform a more advanced inspection for each defect mode of point defects, spot defects, streak defects, uneven defects, line defects, and pixel uneven defects.
Furthermore, since a defect candidate is extracted and primary determined from the luminance statistical data and the threshold value of the entire image for each defect mode, the number of detection processes can be reduced, and the processing time can be reduced.
[0008]
Further, in the screen defect detection method according to the present invention, various types of defects in each mode can be detected by setting a plurality of lighting states of an image to be inspected and capturing the set image.
Further, it is preferable that the image of the inspection object is imaged at an optimal exposure time, and the data of the captured image is 12-bit or more than 4096 gradations.
By doing so, even a slight change in luminance can be captured with high accuracy, and by using high-resolution image data, the accuracy of the luminance statistical data can be increased, thereby further improving the defect detection accuracy. Becomes possible. Further, the accuracy of the evaluation value of the defect candidate described later is also improved.
[0009]
Further, in the screen defect detection method according to the present invention, in order to detect both a bright defect and a dark defect of a point defect, a spot defect, a streak defect, an unevenness defect, and a line defect, a defect candidate is extracted. Is determined for each of the light defect and the dark defect.
The threshold is determined using the average value and the standard deviation of the luminance statistical data.
Further, the method is characterized in that the method includes a step of obtaining characteristic values of the defect candidates for the extracted defect candidates, and calculating an evaluation value based on the characteristic values and the statistical data. Therefore, defect candidates can be objectively and quantitatively evaluated, and accurate evaluation can be performed. In addition, since an evaluation result for each defect mode is obtained, tuning for each defect mode (consistency with a visual inspection) can be performed, and an inspection closer to the judgment performed visually can be performed.
[0010]
In addition, the evaluation value of the defect candidate is calculated for each of the bright defect and the dark defect as in the case of the threshold value.
Further, the evaluation value of the defect candidate is calculated using the average value and the standard deviation of the luminance statistical data, and the maximum luminance and the minimum luminance of the defect candidate.
Therefore, defects can be objectively evaluated for each product.
Further, by classifying the defective product and the non-defective product according to the magnitude of the evaluation value of the defect candidate, the defect can be ranked and the product can be graded. These statistical data can be used for quality control.
[0011]
Further, in the screen defect detection method according to the present invention, the detection processing step of detecting a point defect includes applying a top hat filter or a well filter to the inspection image to emphasize a bright point defect and a dark point defect. The defect and the dark point defect are emphasized, and at the same time, the approximate center value of the gradation displaying the image is added as an offset value. Defects can be detected, and processing time for detecting point defects can be shortened.
[0012]
Further, in the screen defect detection method according to the present invention, in the detection processing step of detecting a stain defect, a flattening process is performed on the inspection image, and a reduced image of a plurality of stages is created from the flattened image. Are subjected to a filtering process using a top hat filter or a well filter for emphasizing a stain defect, and simultaneously adding a substantially center value of a gradation displaying an image as an offset value at the same time as emphasizing the stain defect. Therefore, the contrast of the stain defect is enhanced, and the stain defect can be detected with high accuracy regardless of the size of the defect or the level of the contrast.
[0013]
Further, in the screen defect detection method according to the present invention, in the detection processing step of detecting a streak defect, a flattening process is performed on the inspection image, a plurality of reduced images are created from the flattened image, and each reduced image is formed. A line detection filter in which the emphasis angle is changed in four stages to correspond to stripes in various directions is used to enhance the stripe defects, and to enhance the stripe defects. Since the median value is added as the offset value, the streak defect can be detected with high accuracy regardless of the size of the defect for the white streak or the black streak appearing in different directions on the screen.
[0014]
Further, in the screen defect detection method according to the present invention, the detection processing step of detecting a non-uniform defect includes performing a flattening process of an image in a plurality of stages on an inspection image or a reduced image obtained by reducing the inspection image. And a second flattened image is created, and between the inspection image or a reduced image of the inspection image and the first flattened image, and between the first flattened image and the second flattened image. In this case, the difference processing is performed, and at the same time, the approximate center value of the gradation displaying the image is added as the offset value. The defect remains in the first detected image, and the mura defect having a relatively small size remains in the second detected image, so that large and small mura defects can be accurately detected.
[0015]
Further, in the screen defect detection method according to the present invention, in the detection processing step of detecting a line defect, a flattening process is performed on the inspection image, and an edge detection filter is applied to the flattened image to detect a line defect. Emphasis and line defects are emphasized, and at the same time, the approximate center value of the gradation displaying the image is added as an offset value, so that white line or black line defects appearing in different directions on the screen can be detected with high accuracy. can do.
[0016]
Further, in the screen defect detection method according to the present invention, the detection processing step of detecting a pixel unevenness defect performs an expansion process on each pixel of the inspection image, and applies a Sobel filter or a Laplacian filter to the expanded image. Is applied to emphasize the luminance difference between adjacent pixels, the component of the pixel unevenness defect existing in the inspection image becomes clear, and the pixel unevenness can be qualitatively detected.
[0017]
Furthermore, a screen defect detection apparatus according to the present invention uses the screen defect detection method according to any one of claims 1 to 15.
More specifically, the inspection apparatus includes an imaging unit and an inspection apparatus main body that performs image processing. The inspection apparatus main body is generally configured by a computer. By incorporating an inspection program for performing each of the above-described processes into this computer, various types of defects can be automatically inspected.
[0018]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 is a block diagram showing a configuration of a screen defect detection apparatus according to an embodiment of the present invention.
In this embodiment, for example, the screen 10 to be inspected is a projection screen of the liquid crystal light valve 2 using a TFT element by the projector 1. When performing an inspection, the image 4 is projected on the screen 3 by the projector 1. The image 4 is drawn by giving a predetermined pattern to the liquid crystal panel 2 by the pattern generator 5. For example, an image 4 is picked up by a CCD camera 6 as an image pickup means, and the image signal is converted from an analog signal to a digital signal by an A / D converter (not shown) and is taken into a computer 7 which is a main body of the inspection apparatus. At this time, the image data is represented by, for example, 12-bit data with black being “0” and white being “4095” for each pixel by the A / D converter with a luminance value of 4096 gradations.
[0019]
Further, the computer 7 processes the image data of the image 4 taken into the image memory by a method described later to thereby detect various defects such as point defects, streak defects, line defects, and spot / uneven defects for each light / dark defect. To detect. In defect detection, a reduced image with a plurality of reduced sizes is created according to the type of defect, a filter process for defect emphasis is performed, and statistical processing of luminance information in the detected image is performed. A threshold for extracting a defect candidate is determined based on the data, the defect candidate is extracted, and an evaluation value for quantitatively evaluating the extracted defect candidate is calculated. These inspection results are displayed on the display device 8.
[0020]
FIG. 2 is a flowchart showing a process from inputting an image to a process of displaying a result after detecting various defects, and FIG. 3 is a diagram showing a background image difference process. The detection processing of various defects is automatically performed according to the inspection program incorporated in the computer 7 or the image processing apparatus.
The processing procedure will be described according to the flowchart of FIG.
[0021]
First, an image projected on the screen 3 is photographed by the CCD camera 6, an image of the photographed data is taken into the computer device 7, and image input is performed (step S1).
Next, a display area extraction process is performed to extract only an image portion, which is a display area of the inspected portion, from the captured image captured by the computer device 7 (step S2).
This extraction screen is a pattern matching process for the coordinates of the four corners of the image of the part to be inspected (for four small areas of several tens of pixels × several tens of pixels near the four corners of the image data, four corner reference images prepared in advance and The pattern can be extracted by performing pattern matching processing and specifying the coordinates of the four corners).
[0022]
Subsequently, a background image difference process for removing a defect-like luminance change caused by something other than the liquid crystal light valve 2 such as an illumination or a lens from an extracted screen which is an image extracted only from the screen portion of the inspection target is performed ( Step S3).
In the background image difference processing, the background image 14 shown in FIG. 3B is subtracted from the extraction screen 13 which is an image obtained by extracting only the screen portion of the inspection target image data of the inspection target image data shown in FIG. The background difference image 15 shown in FIG. 3C is created by adding a half value of 4096 gradations as an offset value so that the luminance change does not become negative. Is the image of the difference between the corresponding pixels in the two images. The background image 14 is obtained by imaging a plurality of samples having as few defects as possible, creating an averaged image thereof, and extracting only the screen portion of the inspection target from the image.
The above steps S1 to S3 are referred to as pre-processing, and thereafter, the process enters individual defect detection processing for detecting various defects.
[0023]
That is, first, a point defect detection process for detecting a point defect from the background difference image is performed (step S4), then, a stain detection process for detecting a stain defect from the background difference image is performed (step S5), and a stripe is detected from the background difference image. A streak detection process for detecting a defect is performed (step S6), an unevenness detection process for detecting an unevenness defect from the background difference image is performed (step S7), and a line detection process for detecting a line defect from the background difference image is performed (step S7). (Step S8) Finally, pixel unevenness detection processing for detecting a pixel unevenness defect from the background difference image is performed (Step S9), and statistical processing of luminance information in the detected image in which the defect is emphasized is performed. A threshold value for extracting defect candidates is determined based on the extracted defect candidates, and an evaluation value for quantitatively evaluating the extracted defect candidates is calculated. It is by detecting processing of all of the input image is finished (step S10), and displays on the display device 8 the results of all of the detection process (step S11), and inspection of various defects is finished.
[0024]
Next, individual defect detection processing will be described.
(1) The point defect detection processing will be described based on the flowchart of FIG.
In the background difference image, since it is difficult to detect a minute level of a bright spot defect and a dark spot defect, a point defect emphasizing process is performed using a spatial filter which is a method of image processing (step S21).
In this embodiment, a point defect enhancement process is performed using a top hat filter (7 × 7 Tophat filter) as a spatial filter.
The 7 × 7 Tophat filter emphasizes a point-like luminance change that is isolated from the surroundings, and weights the value of the pixel of interest so that the difference from the value of the surrounding pixels is more emphasized. This is a filter that performs convolution operation. FIG. 5 shows an example of a numerical configuration of the 7 × 7 Tophat filter.
[0025]
Here, enhancement of point defects will be described.
FIG. 6 schematically shows an image having a light defect and a dark defect, which are point defects of a light point and a dark point. As a result of enhancing the image of FIG. 6 by the 7 × 7 Tophat filter, a bright point defect having a small luminance difference from the background as shown in FIG. Is a valid image. On the other hand, a dark defect having a small difference in luminance from the background is emphasized as a negative value, and therefore, the image is such that the dark defect cannot be detected. FIG. 7A shows an image subjected to the Tophat filter processing, and FIG. 7B shows an image obtained by cutting out the image shown in FIG. 7A using a predetermined threshold value for extracting a bright region. It is an image detected by highlighting a bright defect of a point.
As described above, in the image subjected to the Tophat filter, a bright point indicating a bright point defect appears as a gray level having a positive value, whereas a dark point indicating a dark point defect appears as a gray level having a negative value. In the image processing format, image data usually takes only a positive value, so that the component of the dark point becomes 0 as it is, and it cannot be detected because there is no dark point data in the processed image data.
[0026]
Therefore, when the screen is represented by 1296-bit 4096 gradations, a half value of 2048 is added as an offset value at the same time as the filtering process so that a dark point can be detected from the same image. (Step S22). When the image format is an 8-bit gray scale, the image has 256 gradations, and a half value of 128 is added as an offset value at the same time as performing the filter processing.
As a result, the data of the dark point also appears as a tone having a positive value, so that it is possible to detect the point defect (the bright defect and the dark defect) of the bright point and the dark point by one filtering process. The image in FIG. 8A is an image obtained by adding an offset value to the image resulting from the filtering process.
[0027]
Thereafter, a bright / dark defect candidate extraction process is performed to detect a bright defect and a dark defect from the image subjected to the above process, and the presence or absence of the defect candidate which is the check 1 is determined (step S23).
This bright / dark defect candidate extraction process cuts out two types of thresholds, a threshold for extracting a region with a high luminance and a threshold for extracting a region with a low luminance, and synthesizes the cut out images. (B) to obtain an image in which a bright defect and a dark defect are detected.
That is, in the point defect extraction image, a pixel having a luminance value larger than the bright defect extraction threshold can be determined to be a bright defect, and a pixel having a luminance value smaller than the dark defect extraction threshold can be determined as a dark defect. , And a defect candidate having a point defect of a bright defect and a dark defect is extracted.
[0028]
If there is no pixel having a luminance value larger than the threshold value for extracting a bright defect candidate or a pixel having a luminance value smaller than the threshold value for extracting a dark defect candidate, there are no defect candidates, and at this stage, the product is regarded as a non-defective product. It is determined and the inspection is terminated.
Here, the threshold for extracting a region with a high luminance and the threshold for extracting a region with a low luminance are obtained by calculating the average value and the standard deviation of the luminance values obtained from the entire screen of the image in which the point defect is emphasized. Based on the following formula, it is calculated.
Threshold for bright region extraction = average (average value) + a1 * σ (standard deviation)
Threshold for dark area extraction = average (average value)-a2 * σ (standard deviation)
In the above formula, a1 and a2 are certain fixed constants, which are appropriately determined according to the degree of the point defect.
[0029]
Further, the evaluation value which is the check 2 is calculated (step S24).
In the second (that is, final) check, evaluation values are calculated for the defect candidates extracted in the check 1 in order to quantify the degree of the defect. The evaluation value is calculated from the following equation using the luminance statistical data (average value, standard deviation σ, maximum value Lmax, minimum value Lmin) obtained from the image.
Bright defects:
Ev = (Lmax−Lave) / σ
Dark defect:
Ev = (Lave−Lmin) / σ
Where Lmax = maximum luminance of defect candidate
Lmin = minimum luminance of defect candidate
Lave = average luminance of the whole image
σ = standard deviation of the luminance of the whole image
By calculating evaluation values using these formulas, point defects of bright defects and dark defects extracted as defect candidates can be quantitatively evaluated with objective data, and the presence or absence of bright defects and dark defects can be evaluated. In addition to making a determination, a rank can be determined based on the degree of defect. Therefore, the detection accuracy of the point defect is high.
[0030]
Finally, the defect rank is classified based on the evaluation value obtained for the defect candidate that is the check 2 (step S25).
For this evaluation value, the threshold value for classifying the defect rank can be set in several steps for each of the defective rank and the non-defective rank of the product. This makes it possible to rank (define) point defects in good products and to grade products.
[0031]
According to the point defect detection processing of this embodiment, the computer 7 emphasizes a point defect having a small luminance difference from the background by the Tophat filter on the pre-processed background image difference processing image. Processing is in progress.
In such a 7 × 7 Topat filter, a point-like luminance change that is isolated with respect to the surroundings is emphasized, so that a filter that is isolated with respect to the surroundings such as a point defect of the liquid crystal light valve 1 is emphasized. Since moiré fringes that are not isolated from the surroundings are not emphasized, only the point defect portion of the liquid crystal light valve 1 is emphasized even in an image in which moiré fringes occur.
Further, at the same time as emphasizing the point defect, the median value of the gradation of the image is added to the image as an offset value, and a threshold value for extracting a bright region for the image to which the offset value has been added and a dark value for the luminance value Since an image capable of detecting a point defect candidate of a bright defect and a dark defect is obtained by binarizing with two kinds of thresholds for extracting a region, a bright defect of the liquid crystal light valve 1 can be obtained by one filtering process. And a dark defect can be detected, and the processing time for detecting the defect can be shortened.
[0032]
(2) The spot defect detection processing will be described based on the flowchart of FIG.
First, the background difference image is subjected to a flattening process for excluding a defect other than a spot defect having a relatively small size and a different contrast (step S31).
The background difference image (inspection image) has a large size and a gradual change in luminance over the entire display area, that is, a gradual brightness such as a bright or dark place over a wide range that is not so problematic to humans. And a nonuniformity defect having a size larger than a spot defect may be included. In order to exclude these from detection targets, a flattening process is performed on the background difference image, that is, the inspection image.
By this flattening process, a large change in brightness such as illuminance unevenness and an uneven defect having a large size are removed, and only stain defects having a relatively small size and stain defects having different contrasts remain in FIG. The flattened image 16 as shown in FIG.
Note that the flattening process is a process that applies a smoothing filter or a process that flattens by a morphological process or the like.
[0033]
Next, image size reduction processing for reducing the flattened image to various sizes is performed (steps S32 to S36).
Although the size of the spot defect is small, there are various sizes among them, and the contrast of the spot defect is also various such as high and low.
The top-hat filter, which will be described later, can enhance only a spot defect having a predetermined size. Is performed.
[0034]
Here, （(600 × 500 pixels), ４ (300 × 250 pixels), ８ (150 × 125 pixels), 1/16 ( Five reduced images reduced to five stages of 75 × 62 pixels) and 1/32 (38 × 31 pixels) are created.
In the method of reducing the image size, as shown in FIG. 11, the average value of the luminance data for the four pixels of the original image is reduced to a half size by allocating it to one pixel of the new image. By repeating this method, the image size can be reduced by half.
By gradually reducing the image size of the flattened image 16 in this manner, some of the reduced images have a size that can be enhanced by the top hat filter.
[0035]
Further, a process of enhancing the contrast of the stain defect using a spatial filter is performed on each reduced image (steps S37 to S41).
The contrast of the spot defect is enhanced using a Tophat filter as a spatial filter for each reduced image. The top-hat filter is a filter for enhancing the contrast, and is composed of, for example, 7 × 7 pixels as shown in FIG. 5, and performs a convolution operation on the target pixel with this filter configuration value to obtain a background in the pixel. If there is a bright spot (here, a white spot defect) having a size of about 3 × 3 pixels as compared with, this is emphasized.
[0036]
However, the top hat filter is effective for detecting a light defect (white spot) because the result of enhancing a bright point has a positive value, but the result of enhancing a dark point (here, a black spot defect) is a minus value. Since the image format of image processing usually takes only a positive value, the result is replaced with 0, and there is a difficulty in detecting a dark defect (black spot). Therefore, by performing the convolution operation in the top hat filter process and adding the value of 2048 as the offset value, the emphasized portion of the scotoma which is originally a negative value is made positive, and as a result, only the top hat filter process is performed. Enables bright and dark point enhancement. Of course, the spatial filter is not limited to the top-hat filter, and a well (Well) filter in which the result of emphasizing the scotoma has a positive value (the sign is opposite to that of the top-hat filter) (see FIG. 12). May be used. Further, the filter size, weighting, and the like of these spatial filters are not limited to those shown in the drawings.
[0037]
FIG. 10B schematically shows the detected image 17 when the above-mentioned top hat filter processing is performed on the reduced image (however, the image size is illustrated with the same size for easy understanding). . Actually it is a reduced size). Incidentally, reference numeral 20 in FIGS. 10A and 10B indicates a spot defect.
By simultaneously performing the top hat filter processing and the offset processing as described above, the result of emphasizing white spots and black spots can be obtained at one time, and the detection time of the spot defect 20 can be reduced.
It should be noted that in the case of a spot defect having a size in which no defect is emphasized in FIG. 10B, the defect is emphasized in the same manner in another reduced image which has been reduced in several stages to a size that can be emphasized. Will be done.
[0038]
Further, a statistical calculation for calculating statistical data based on the luminance value of each pixel in the detected image is performed (steps S42 to S46).
In the calculation of the statistical data, the average value Lave, the standard deviation σ, the maximum value Lmax, and the minimum value Lmin of the luminance value of the entire image are obtained. From these four pieces of luminance statistical data, a threshold value for extracting defect candidates for white spots and black spots is determined as follows, for example.
White spot threshold: Lave + a1 × σ
Black stain threshold: Lave-a2 × σ
Here, a1 and a2 are certain fixed constants.
Therefore, the threshold value for extracting the white spot and black spot defect candidates can be automatically determined based on the luminance statistical data by statistically calculating the luminance data in the detected image 17. Therefore, the threshold value is not artificial, trial and error, but objective and relative.
[0039]
Thereafter, it is determined whether there is a defect candidate which is the check 1 (step S47).
A first check is performed based on the threshold value to determine whether there is a candidate for a white spot defect or a black spot defect in the detection image 17 shown in FIG. That is, if the maximum value Lmax of the luminance value obtained by the calculation of the statistical data exceeds the white stain threshold, it is determined that there is a white stain defect candidate in the detected image 17, and the minimum luminance value Lmin is set to the black stain threshold. If it is below, it is determined that there is a black spot defect candidate in the detected image 17. If there are no defect candidates, the product is determined to be good with respect to the stain at this stage, and the stain defect inspection ends.
[0040]
Further, a blob process for calculating the characteristic value of the extracted defect candidate is performed (steps S48 to S52).
When it is determined in the above check 1 that there is a defect candidate in the detected image 17, the defect candidate is extracted using a threshold, and blob processing is performed to calculate the characteristic value of the defect candidate. The blob is a “cluster” having a value in a specific range existing in the image, and here is a defect candidate that is an area equal to or more than a white stain threshold or an area equal to or less than a black stain threshold.
Therefore, a defect candidate is extracted by a binarization process using the white stain threshold and the black stain threshold, and the extracted region is subjected to a blob process, which is an image processing method, to calculate the characteristic value of the defect candidate. I do.
Here, as the characteristic values of the defect candidate, the barycentric position X and Y coordinates of the area, the maximum luminance value (Lmax (n)) in the area for a white spot defect, and the area value for a black spot defect The minimum value (Lmin (n)) of the luminance is calculated.
[0041]
Further, the evaluation value as check 2 is calculated (step S53).
In the second (ie, final) check, evaluation values are calculated for the defect candidates extracted as the defect candidates in order to quantify the degree of the defect. The evaluation value is the luminance statistical data (average value Lave (i), standard deviation σ (i), i is the screen number of the reduced image) obtained for each of the reduced images (described above) and the characteristics obtained by the blob processing. Using the values (the maximum value Lmax (n), the minimum value Lmin (n), and n is a blob number), the evaluation value is calculated by the following equation.
White stain:
Ev (n) = k (i) × (Lmax (n) −Lave (i)) / σ (i)
Black stain:
Ev (n) = k (i) × (Lave (i) −Lmin (n)) / σ (i)
Where k (i) = reduced screen coefficient i = screen number
Lmax (n) = maximum luminance of defect candidate n = Blob number
Lmin (n) = minimum luminance of defect candidate
Lave = average luminance of the whole image
σ = standard deviation of the luminance of the whole image
By calculating the evaluation value using these formulas, the white spot defect and the black spot defect extracted as defect candidates can be quantitatively evaluated by objective data together with the coordinate position and the number (Blob number). In addition to determining the presence or absence of a white spot defect and a black spot defect, a rank can be determined based on the degree of the defect. Therefore, the detection accuracy of the stain defect is high.
[0042]
Finally, the defect rank is classified based on the evaluation value obtained for the defect candidate that is the check 2 (step S54).
For this evaluation value, the threshold value for classifying the defect rank can be set in several steps for each of the defective rank and the non-defective rank of the product. As a result, it is possible to assign a rank (grade) of the spot defect to a non-defective product and to grade the product.
[0043]
According to the stain defect detection processing of this embodiment, a stain defect existing on the screen of a display device such as a liquid crystal panel is automatically detected with high accuracy regardless of the size of the defect and the contrast level. And it is possible to individually and quantitatively evaluate the stain defect.
In addition, since spot defects are evaluated based on luminance statistical data, collecting and analyzing the quality data of products and parts can be used for quality control, and a method aimed at further improving quality is developed. It is also possible to build.
[0044]
(3) The streak defect detection processing will be described with reference to the flowchart of FIG.
First, a flattening process similar to the case of the above-described stain detection is performed on the background difference image (step S61).
Next, an image size reduction process similar to the above-described stain detection process is performed on the flattened image (step S62).
[0045]
Thereafter, a line detection filter process is performed on each of the five reduced images (step S63).
In this line detection filter processing, it is difficult to detect minute-level white / black streak / labis-judge defects as it is. Therefore, three lines of one reduced image are emphasized so as to emphasize only streak defects in the image. Are made, and one type of line detection filter is applied to each image, and defect enhancement processing is performed by applying a total of four types of line detection filters.
[0046]
The four types of line detection filters include a horizontal line detection filter that performs horizontal line enhancement processing, a vertical line detection filter that performs vertical line enhancement processing, and an oblique line detection filter that performs + 45 ° line enhancement processing. , An oblique line detection filter that performs a line emphasis process of −45 °.
Accordingly, three copies of each of the five reduced images shown in FIG. 16A are created, and one type of line detection filter processing is performed on each image, for a total of four types of line detection filter processing. As shown in FIG. 16B, four images obtained by performing horizontal line detection processing, vertical line detection processing, + 45 ° line detection processing, and −45 ° line detection processing from each reduced image are obtained, for a total of 20 images. An image in which the streak defect is enhanced is obtained.
[0047]
All of these four types of line detection filters detect a vertical line, a horizontal line, and + 45 ° as a detection target for a small area of several pixels × several pixels (7 × 7 pixels in FIG. 15) including the periphery of the pixel of interest. In order to detect whether there is a diagonal line or a component of a −45 ° diagonal line, when the component exists, it is emphasized by a convolution operation based on the relationship between the luminance value of the pixel of interest and the surrounding pixels. Is a filter in which the weighting is performed.
15A is a horizontal line detection filter that emphasizes horizontal lines, FIG. 15B is a vertical line detection filter that emphasizes vertical lines, and FIG. 15C is a diagonal line that emphasizes + 45 ° lines. FIG. 15D shows an example of another oblique line detection filter that emphasizes a line of −45 °.
[0048]
In the image to which the four types of line detection filters are applied as shown in FIG. 15, white stripes appear as gradations of positive values, while black stripes appear as gradations of negative values. In the image processing format, the image data usually takes only a positive value, so the black streak component becomes 0 as it is, and the processed image data cannot be detected because there is no black streak data. In this manner, the graph in which the offset value is not added as shown in FIG. 17A shows that white streaks appear but black streaks do not appear.
[0049]
Therefore, when the screen is represented by 4096 gradations of 12 bits so that a black streak can be detected from the same image, a process of adding a half value of 2048 as an offset value simultaneously with the filtering process is performed. As a result, the data of the black streak also appears as a gradation having a positive value, so that the white streak and the black streak can be detected by one filtering process. The graph to which the offset value is added as shown in FIG. 17B indicates that both white stripes and black stripes appear. When the image format is an 8-bit gray scale, 256 gradations are obtained, and a half value of 128 is added as an offset value simultaneously with the filtering process.
[0050]
Next, statistical data calculation processing is performed on the image in which the 20 streak defects obtained by performing the four types of line detection filter processing are emphasized, based on the luminance values of the pixels (step S64).
In this statistical data calculation process, the luminance value of each pixel in the entire region of an image in which 20 streak defects are obtained by performing four types of line detection filter processes on a reduced image of five stages is obtained. Is obtained, and the average value, standard deviation, maximum value, and minimum value of the entire screen are obtained.
[0051]
That is, for each of the five reduced images subjected to the horizontal line detection processing, the luminance value of each pixel in the entire area is obtained, and the average value, standard deviation, maximum value, and minimum value of the luminance value of the entire screen are obtained. In addition, for each of the five reduced images subjected to the vertical line detection processing, the luminance value of each pixel in the entire area is obtained, and the average value, standard deviation, maximum value, and minimum value of the entire screen are obtained. Further, with respect to each of the five reduced images subjected to the + 45 ° oblique line detection processing, the luminance value of each pixel in the entire area is obtained, and the average value, standard deviation, maximum value, and minimum value of the entire screen are obtained. Furthermore, for each of the five reduced images subjected to the -45 ° oblique line detection processing, the luminance value of each pixel in the entire area is obtained, and the average value, standard deviation, maximum value, and minimum value of the entire screen are obtained. .
[0052]
After that, a defect candidate presence / absence determination process of check 1 is performed (step S65).
In the defect candidate presence / absence determination processing, the average value, standard deviation, and maximum value obtained from the 20 images subjected to the horizontal line detection processing, the vertical line detection processing, the + 45 ° diagonal line detection processing, and the −45 ° diagonal line detection processing are obtained. Based on the value and the minimum value, the horizontal line threshold, the vertical line threshold, the + 45 ° oblique line threshold, and the −45 ° oblique line threshold of each image are calculated by the following equation.
Horizontal line threshold = average (average value of horizontal line detection processing image) ± a₁* Σ (standard deviation of horizontal line detection processed image)
Vertical line threshold = average (average value of vertical line detection processed image) ± a₂* Σ (standard deviation of vertical line detection processed image)
+ 45 ° oblique line threshold = average (average value of + 45 ° oblique line detection processed image) ± a₃* Σ (standard deviation of + 45 ° oblique line detection processed image)
−45 ° oblique line threshold = average (average value of −45 ° oblique line detection processed image) ± a₄* Σ (standard deviation of -45 ° oblique line detection processed image)
a₁  , A₂  , A₃  , A₄  Is a fixed constant.
Note that two values are calculated as thresholds as a calculation result of one formula. The threshold obtained by + is a threshold for white line detection, and a line detection processing image that is equal to or more than this threshold is a white line. The threshold value detected as a defect candidate and obtained with-becomes a threshold value for black streak detection, and a line streak image that is equal to or less than the threshold value is detected as a black streak defect candidate.
[0053]
However, before that, it is checked whether there is a streak defect candidate in the line detection processing image, that is, the maximum value and the minimum value of the luminance value of the entire screen obtained from each image subjected to the horizontal line detection processing, the vertical line detection processing is performed. The maximum value and the minimum value of the luminance value of the entire screen acquired from each image, the maximum value and the minimum value of the luminance value of the entire screen acquired from each image subjected to the + 45 ° oblique line detection processing, and the −45 ° oblique line detection processing is performed. It is determined from the maximum and minimum values of the brightness of the entire screen obtained from each of the images and the calculated threshold value whether or not a candidate for a streak defect exists in each detection processing screen (step S65).
[0054]
For each image subjected to the horizontal line detection processing, if the maximum value of the luminance value of the image exceeds the horizontal line threshold (+ calculation), the image is regarded as a defect candidate having a white stripe, and the next particle analysis processing (blob processing) is performed. Proceed to).
If the minimum value of the luminance value of the image does not exceed the horizontal line threshold (−calculation), the image is processed as a black streak image and the process proceeds to the next blob processing. When the maximum value of the luminance value does not exceed the horizontal line threshold value (+ calculation) and the minimum value of the luminance value exceeds the horizontal line threshold value (−calculation), it is determined that there is no white line or black line and that it is a good product. The next blob processing is not performed.
[0055]
Also, for each image subjected to the vertical line detection processing, when the maximum value of the luminance value of the image exceeds the vertical line threshold (+ calculation), the image is regarded as a defect candidate having a white stripe, and the next blob processing is performed. Proceed to.
If the minimum value of the luminance value of the image does not exceed the vertical line threshold value (-calculation), the image is processed with the black streak and the process proceeds to the next blob processing. When the maximum value of the luminance value does not exceed the horizontal line threshold value (+ calculation) and the minimum value of the luminance value exceeds the horizontal line threshold value (−calculation), it is determined that there is no white line or black line and that it is a good product. The next blob processing is not performed.
[0056]
Also, for each image subjected to the + 45 ° oblique line detection processing, if the maximum value of the luminance value of the image exceeds the + 45 ° oblique line threshold (+ calculation), it is determined as a defect candidate having a white stripe. Proceed to the next blob processing.
If the minimum value of the luminance value of the image does not exceed the + 45 ° oblique line threshold value (−calculation), the process proceeds to the next blob process as an image having black stripes. If the maximum value of the luminance value does not exceed the + 45 ° oblique line threshold (+ calculation) and the minimum value of the luminance value exceeds the + 45 ° oblique line threshold (−calculation), white stripes and black stripes The next blob processing is not performed as a non-defective product.
[0057]
Furthermore, for each of the images subjected to the -45 ° oblique line detection processing, if the maximum value of the luminance value of the image exceeds the -45 ° oblique line threshold (+ calculation), the defect candidate having a white stripe is detected. To the next blob processing.
If the minimum value of the luminance value of the image does not exceed the -45 ° oblique line threshold value (−calculation), the image is processed with the black streak and the process proceeds to the next blob processing. If the maximum value of the luminance value does not exceed the −45 ° oblique line threshold (+ calculation) and the minimum value of the luminance value exceeds the −45 ° oblique line threshold (−calculation), white streaks, The next blob treatment is not performed as a non-defective product having no black stripes.
[0058]
Next, a blob process is performed on the image determined to have the defect candidate to obtain the maximum luminance, the minimum luminance, and the area of the defect candidate (step S67).
In this blob processing, a defect candidate is extracted only for an image for which it has been determined that there is a white or black streak defect candidate, and its evaluation value is obtained. The maximum luminance, the minimum luminance, and the area of the white stripe or black stripe defect candidate in the detection processing image, the + 45 ° oblique line detection processing image, and the −45 ° oblique line detection processing image are obtained.
In order to perform the blob processing, the image must be binarized. For this purpose, a horizontal line threshold, a vertical line threshold, a + 45 ° oblique line threshold, and a −45 ° oblique line threshold are used. FIG. 18 shows an image binarized for performing the blob processing.
[0059]
For example, if it is determined that an image subjected to the horizontal line, vertical line, + 45 ° oblique line, and −45 ° oblique line detection processing includes a white streak defect candidate, the horizontal line threshold, the vertical line threshold, and the +45 calculated with + The oblique line threshold value and the −45 ° oblique line threshold value are used, respectively, and a portion beyond that is binarized as a white stripe defect candidate. In the binarized image, a plurality of defect candidates exist as clusters, and the area of all of them is determined, and the image before binarization is applied within the range of the binarized cluster region. From the maximum luminance value.
If it is determined that there is a black streak defect candidate in the image subjected to the horizontal line detection processing, the horizontal line threshold, the vertical line threshold, the + 45 ° oblique line threshold, and the −45 ° oblique line threshold calculated using − are used. Are binarized as black streak defect candidates. As in the case of the white stripe, the binarized image has a plurality of defect candidates in clusters. The area of all of them is determined, and within the range of the binarized cluster, the area is determined. The minimum luminance value is obtained from the image before the value conversion.
[0060]
Further, an evaluation value for calculating a defect evaluation value is calculated for a defect candidate of a white stripe or a black stripe (step S68).
The evaluation value processing is performed for a white line or black line defect candidate in a horizontal line detection processed image, a vertical line detection processed image, a + 45 ° diagonal line detection processed image, and a −45 ° diagonal line detection processed image obtained by the blob processing. The calculation of an evaluation value for calculating a defect evaluation value based on the information of the maximum luminance, the minimum luminance, and the area, and the average value and the standard deviation of the luminance values of the entire screen.
The defect evaluation value (Ev) of a white streak or a black streak in a defect candidate is obtained by calculation according to the following equation.
White line
Ev (i, n) = K (i) * S (n) * (Lmax (n) -Lave (i)) / σ (i)
Black stripe
Ev (i, n) = K (i) * S (n) * (Lave (i) -Lmin (n)) / σ (i)
K (i) = reduced screen coefficient, i = screen number, n = blob number,
Lmax (n) = maximum luminance of defect candidate, Lmin (n) = minimum luminance of defect candidate,
S (n) = Area of defect candidate, Lave (i) = Average luminance of entire screen,
σ (i) = standard deviation of luminance of the entire screen
Then, the maximum value of all the evaluation values is obtained, and the rank of the white streak or the black streak defect of the panel to be inspected can be determined based on this value. Therefore, the detection accuracy of the streak defect is high.
[0061]
Finally, the defect rank is classified based on the evaluation value obtained for the defect candidate that is the check 2 (step S69).
For this evaluation value, the threshold value for classifying the defect rank can be set in several steps for each of the defective rank and the non-defective rank of the product. As a result, it is possible to assign a rank (grade) of the spot defect to a non-defective product and to grade the product.
[0062]
According to the streak defect detection processing of this embodiment, the computer 7 performs a flattening processing on the pre-processed background image difference processing image to remove the influence of unevenness over a relatively wide range of the background difference image. Further, in order to emphasize and detect defects of horizontal lines, vertical lines, and ± 45 ° oblique lines with respect to the reduced images of the five stages, respectively, the size of the flattened image is reduced by five stages. Line detection filter processing was performed by four types of line detection filters to enhance the streak defect, and at the same time, the median value of the gradation of the image was added as an offset value, and 20 streak defects were enhanced. Obtain the luminance value of each pixel in each whole area of the image, perform statistical data calculation processing to find the average value, standard deviation, maximum value and minimum value of the luminance value of the entire screen, and perform statistical data calculation processing. Set the horizontal, vertical and diagonal line thresholds for judging white lines from the average value and standard deviation obtained, and set the horizontal, vertical and diagonal line thresholds for judging black lines, and set the maximum value of the screen brightness value The horizontal line, vertical line, and diagonal line that determine these white lines are subjected to a primary determination as to whether or not the screen has a white line defect based on whether or not the threshold value is exceeded. The horizontal line, the vertical line, and the diagonal line to be determined are subjected to the primary determination as to whether or not there is a black streak defect on the screen based on whether or not the threshold value is exceeded. Further, it is possible to easily determine in a short time whether the product is a non-defective product having no stripes of various sizes.
[0063]
Further, the image of the defect candidate determined to have a defect is subjected to blob processing for obtaining the maximum luminance, the minimum luminance, and the area of the white streak or the black streak of the defect candidate as preprocessing of the evaluation value calculation. Evaluation for calculating a defect evaluation value by a predetermined formula based on information on the maximum luminance, minimum luminance, and area of the white or black streak of the defect candidate obtained by the blob processing, and the average value and standard deviation of the luminance value of the entire screen. Since the value calculation process is performed and the rank of the white streak or the black streak defect is determined by the maximum value of the evaluation values obtained by the evaluation value calculation process, regardless of the size of the defect with respect to the white streak or the black streak, Streak defects can be detected with high accuracy, and since the blob processing is performed only on images with defect candidates, the calculation time is short, and defect ranking is performed in a short time. Has become can be.
[0064]
(4) The unevenness defect detection processing will be described with reference to the flowchart in FIG.
First, an image size reduction process similar to that in the case of the stain detection is performed on the background difference image (step S71).
In the image size reduction process, since the background difference image (inspection image) 15 has defects of various sizes, the unevenness to be detected (corresponding to the size of the defect) is to be detected. The inspection image 15 is reduced to an appropriate image size in order to remove defects (stains and streak defects) smaller than the defect size. For example, the inspection image 15 is reduced to a quarter image size, that is, 300 × 250 pixels. Note that the reduction ratio is not limited to 1/4.
[0065]
By reducing the image size of the inspection image 15 in this way, there is also an advantage that the processing time required for defect detection can be reduced.
In FIG. 20, all the images are shown in the same size, but the images (a) to (d) are actually reduced to 1/4.
Here, reduction of the image size has been described, but of course, it is not always necessary to reduce the image size. For example, even if the filter size is adjusted by using a smoothing filter process, defects smaller than the uneven defect size to be detected are removed. Good.
[0066]
Next, image duplication processing for creating a duplicate (copy) image of the reduced image is performed (step S72).
This image duplication process requires at least three images as described later when performing a process for separating the uneven defect from the other background portion (the portion having no defect). (Inspection image) This is for creating a duplicate (copy) image of a １ ／ reduced image of 15. The copied image can also be used for detecting spots, spots, and line defects other than the unevenness defect.
[0067]
Further, a flattening process is performed on the 縮小 reduced image of the inspection image 15 or its copy image in two stages (steps S73 and S74).
This flattening process is performed as a pre-process for separating an uneven defect from other background portions (portions without defects). Here, a two-stage flattening process is performed.
In the first flattening 1 process (step S73), a first flattened image 16 as shown in FIG. 20A is created based on a 縮小 reduced image of the inspection image 15 or a copy image thereof. Is done.
[0068]
In the first-stage flattening process, a process that applies a morphological process or the like is performed such that a relatively large luminance change is flattened while a relatively small luminance change is left.
In the second flattening 2 process (step S74), a second flattened image 17 that has been further flattened is created from the first flattened image 16 (see FIG. 20B).
In the second-stage flattening process, a process is performed on the first flattened image 16 from which a large luminance change has been removed so as to flatten a smaller luminance change by a process using a smoothing filter or the like.
[0069]
Thereafter, the difference processing is performed in two stages (steps S75 and S76).
In the difference processing in step S75, the first flattened image 16 (the image (a) in FIG. 20) is subtracted from the 1/4 reduced image of the inspection image 15 or its copy image. By this processing, a first detection image 18 as shown in FIG. In the first detection image 18, the small size uneven defect is removed, and only the relatively large size uneven defect 20a remains.
Next, in the difference processing in step S76, the second flattened image 17 (the image (b) in FIG. 20) is subtracted from the first flattened image 16 (the image (a) in FIG. 20). Then, a second detection image 19 as shown in FIG. 20D is obtained. In the second detected image 19, only the small-sized uneven defect 20b remains.
[0070]
Further, a statistical calculation based on the luminance value of each pixel in the unevenness defect detection image is performed (Steps S77 to S78).
In this statistical calculation, the statistical calculation of the luminance data over the entire screen is performed using the luminance data of each pixel in the unevenness defect detection images 18 and 19.
In the statistical calculation, the average value Lave, the standard deviation σ, the maximum value Lmax, and the minimum value Lmin of the luminance data are obtained. From these four pieces of luminance statistical data, a threshold for determining the presence / absence of a defect candidate of white unevenness and black unevenness is determined as follows, for example.
White spot threshold: Lave + a1 × σ
Black unevenness threshold: Lave-a2 × σ
Here, a1 and a2 are certain fixed constants.
Therefore, it is possible to automatically determine a threshold value for extracting a defect candidate of white unevenness and black unevenness based on the brightness statistical data by statistically calculating the brightness data in the unevenness defect detection images 18 and 19. it can. Therefore, the threshold value is not artificial, trial and error, but objective and relative.
[0071]
Thereafter, it is determined whether there is a defect candidate which is the check 1 (step S79).
In the determination of the presence or absence of the defect candidate, a first check is performed based on the threshold value to determine whether there are white and black side defect candidates in the uneven defect detection images 18 and 19.
First, it is checked whether Lmax exceeds the white unevenness threshold value, and if it exceeds, it is determined that there is a white unevenness defect candidate in the image. Next, it is checked whether or not Lmin is equal to or less than the black unevenness threshold value. If there are no defect candidates, it is determined at this stage that there is no unevenness defect, and the inspection ends.
[0072]
Next, a blob process for obtaining various characteristic values in the extracted defect candidate area is performed (steps S80 and S81).
When it is determined in the above check 1 that there is a defect candidate, the defect candidate is extracted using a threshold. At this time, if it is determined in the check 1 that there is a white unevenness defect candidate, a candidate having a white unevenness threshold or more is extracted as a defect candidate. Those that are equal to or less than the unevenness threshold are extracted as defect candidates. If both are determined, both are extracted.
[0073]
A blob process is performed on the extracted defect candidate, and the area (S (n)) of the extracted defect candidate and the maximum value Lmax (n) of the luminance value if the defect is a white uneven defect, or the maximum value Lmax (n) if the defect is a black uneven defect The minimum value Lmin (n) of the luminance value is obtained. The characteristic value in the defect candidate area obtained here is used in calculation of an evaluation value described below.
[0074]
Further, an evaluation value is calculated for the mura defect candidate extracted as the defect candidate and subjected to the blob processing (step S82).
The luminance statistical data (average value, standard deviation σ) and the characteristic values (area S (n), maximum value Lmax (n) obtained by the blob processing are obtained for the unevenness defect candidates extracted and subjected to the blob processing as the defect candidates. ), And the minimum value Lmin (n)) is used to calculate an evaluation value according to the following equation.
White spots:
Ev (n) = k (i) * S (n) * (Lmax (n) -Lave (i)) / σ (i)
Black spot:
Ev (n) = k (i) * S (n) * (Lave (i) -Lmin (n)) / σ (i)
Where k (i) = reduced screen coefficient
i = screen number
n = Blob number
Lmax (n) = maximum luminance of defect candidate
Lmin (n) = minimum luminance of defect candidate
S (n) = area of defect candidate
Lave (i) = average luminance of the entire image
σ (i) = standard deviation of luminance of the whole image
With these formulas, the white unevenness defects and black unevenness defects extracted as the defect candidates can be quantitatively evaluated by objective data together with their size (area) and number (Blob number). Therefore, the detection accuracy of the unevenness defect is high.
[0075]
Finally, the defect rank is classified based on the evaluation value obtained for the defect candidate that is the check 2 (that is, the final check) (step S83).
For this evaluation value, the threshold value for classifying the defect rank can be set in several steps for each of the defective rank and the non-defective rank of the product. As a result, it is possible to assign a rank (grade) of the spot defect to a non-defective product and to grade the product.
[0076]
According to the mura defect detection processing of this embodiment, mura defects existing on the screen of a display device such as a liquid crystal panel can be automatically detected with high accuracy regardless of the size of the defect. Can be quantitatively evaluated individually.
In addition, since unevenness defects are evaluated based on luminance statistical data, collecting and analyzing the quality data of products and parts can be useful for quality control, and a method aimed at further improving quality is developed. It is also possible to build.
[0077]
(5) The line defect detection processing will be described based on the flowchart of FIG.
First, a flattening process similar to the case of the above-described stain detection is performed on the background difference image (step S91).
Next, an edge detection filter process is performed on the flattened image to create a horizontal / vertical line detection image in which horizontal and vertical edges are enhanced (step S92).
In the edge detection filter processing, two copies of the image subjected to the flattening processing are created, and the horizontal edge detection processing is performed on one duplicated image by the horizontal edge detection filter to emphasize the horizontal edges in FIG. As shown in FIG. 23B, a horizontal line detection image as shown in FIG. 23A is created, and another copy image is subjected to vertical edge detection processing by a vertical edge detection filter to emphasize vertical edges. This is for creating a vertical line detection image.
[0078]
In this horizontal / vertical edge detection filter processing, it is difficult to detect minute levels of white / black line defects in an image that has been flattened. Is what you do.
These horizontal / vertical edge detection filters include a horizontal edge detection filter that performs a horizontal edge detection process and a vertical edge detection filter that performs a vertical edge detection process.
Both of these horizontal and vertical edge detection filters detect whether or not there is an edge component in a small area of several pixels x several pixels including the pixel of interest. This filter weights each pixel so that it is emphasized by performing a convolution operation based on the relationship between the luminance value of the pixel to be processed and the surrounding pixels.
FIG. 22A shows an example of a horizontal edge detection filter for detecting a horizontal edge, and FIG. 22B shows an example of a vertical edge detection filter for detecting a vertical edge.
[0079]
The images to which the horizontal / vertical edge detection filters of FIGS. 22A and 22B are applied are on both sides of the line (depending on the edge state), one side has a positive gradation, and the other side has a negative gradation. Appear as the value of the gray scale. In the image processing format, image data usually takes only a positive value. Therefore, a negative component becomes 0 as it is, and is not processed.
In order to detect both edges from the same image, when the screen is represented by 12-bit 4096 gradations, a process of adding half the value of 2048 as an offset value simultaneously with the filtering process is performed. As a result, a negative component also appears as a positive gradation, so that both edge components can be detected by one filtering process. When the image format is an 8-bit gray scale, the image has 256 gradations, and a half value of 128 is added to the result of the filter processing as an offset value.
[0080]
Further, the horizontal / vertical line detection image is divided, and the luminance profile of each pixel in each divided region is integrated in the horizontal and vertical directions to obtain a division profile process (step S93).
This division profile processing is a horizontal profile processing of dividing the horizontal line detection image subjected to the horizontal edge detection processing into four in the vertical direction, and integrating the luminance value of each pixel in each divided area in the horizontal direction to obtain an integrated value. And vertical profile processing in which the vertical line detection image subjected to the vertical edge detection processing is divided into four parts in the horizontal direction, and the luminance value of each pixel in each divided area is integrated in the vertical direction to obtain an integrated value.
[0081]
Thereafter, a statistical calculation for calculating the average value, the standard deviation, the maximum value, and the minimum value of the luminance values of the entire divided area screen from the integrated value in each divided area is performed (step S94).
This statistical calculation calculates the average value, the standard deviation, the maximum value and the minimum value of the brightness values of the entire divided region screen from the integrated value of each row in each divided region subjected to the horizontal profile processing, and the respective values in each divided region subjected to the vertical profile processing. A statistical data calculation process is performed to calculate an average value, a standard deviation, a maximum value, and a minimum value of the brightness values of the entire divided area screen from the integrated values of the columns. FIG. 24A shows horizontal profile data obtained by performing the horizontal profile processing, and FIG. 24B shows vertical profile data obtained by performing the vertical profile processing.
[0082]
As described above, the horizontal line detection image is divided into four parts in the vertical direction during the horizontal profile processing, and the vertical line detection image is divided into four parts in the vertical direction during the vertical profile processing. In a detected image or a vertical line detected image, there is a possibility that a thin line defect may be buried due to a change in brightness, and a short line defect may cause the components of the line defect to be diluted because other portions are also integrated. However, a small divided image is less susceptible to fluctuations in brightness, so it is possible to detect thin line defects, and the division reduces the amount of data other than line defects. This is because even a short line defect whose components are not so thin can be detected.
[0083]
Next, a defect candidate extraction process as check 1 is performed (step S95).
This defect candidate extraction process is performed by calculating the average value, standard deviation, maximum value, and minimum value of the integrated value of each row of each divided region subjected to the horizontal profile processing, and the average of the integrated value of each column of each divided region subjected to the vertical profile processing. The horizontal line threshold and the vertical line threshold are calculated by the following equations based on the values, the standard deviation, the maximum value, and the minimum value, respectively, and those below the threshold are extracted as line defect candidates. A in each formula₁, A₂Is a fixed constant.
Horizontal line threshold value (horizontal line) = average (average value) ± a₁* Σ (standard deviation)
Vertical line threshold (vertical line) = average (average value) ± a₂* Σ (standard deviation)
Note that two values are calculated as threshold values as a result of calculation of one formula. The threshold value obtained with + is a threshold value of an edge emphasized to the plus side by the filtering process, and this threshold value is included in the image subjected to the divided profile processing. Those having a threshold value or more are detected as line defect candidates. Further, the threshold value obtained by-becomes the threshold value of the edge emphasized to the minus side by the filtering process, and a line defect candidate that is equal to or smaller than the threshold value is extracted from the images subjected to the division profile processing.
[0084]
That is, for each divided region image subjected to the horizontal profile processing, if the maximum value of the integrated value of the image exceeds the horizontal line threshold value (+ calculation), the next evaluation value is determined as an image having a horizontal line defect candidate. Proceed to processing.
If the minimum value of the integrated value of the image does not exceed the horizontal line threshold value (−calculation), the image is determined to be an image having a horizontal line defect candidate, and the process proceeds to the next evaluation value processing.
Therefore, when the maximum value of the integrated value of the image does not exceed the horizontal line threshold (+ calculation) and the minimum value of the integrated value exceeds the horizontal line threshold (−calculation), there is no horizontal line defect. A non-defective product is assumed (step S96), and the evaluation value processing of the evaluation value calculation described later is not performed.
[0085]
When the maximum value of the integrated value of each divided region image that has been subjected to the vertical profile processing exceeds the vertical line threshold (+ calculation), the next evaluation value is determined as an image having a vertical defect candidate. Proceed to processing.
If the minimum value of the integrated value of the image does not exceed the vertical line threshold value (-calculation), the process proceeds to the next evaluation value process as an image having a vertical line defect candidate.
Therefore, when the maximum value of the integrated value of the image does not exceed the vertical line threshold value (+ calculation) and the minimum value of the integrated value exceeds the vertical line threshold value (−calculation), the vertical line defect is detected. It is determined that there is no defective product (step S96), and the evaluation value processing of the evaluation value calculation described later is not performed.
[0086]
Further, for the defect candidate, an evaluation value obtained by calculating a line evaluation value H and a line evaluation value B by a predetermined formula based on the average value, standard deviation, maximum value and minimum value obtained from each divided area image. Is calculated (step S97).
The calculation of the evaluation value is based on the fact that the maximum value of the integrated value of each divided region image subjected to the horizontal profile processing is extracted as being equal to or more than the horizontal line threshold (+ calculation), and the minimum value of the integrated value of the image is The horizontal line defect candidate extracted as not exceeding the horizontal line threshold value (−calculation) and the maximum value of the integrated value of the image for each of the divided region images subjected to the vertical profile processing are the vertical line threshold value (+ calculation). The above extracted items and the vertical line defect candidates extracted as the minimum value of the integrated value of the image not exceeding the vertical line threshold value (−calculation) were obtained from each divided region image. The line evaluation value H and the line evaluation value B are calculated by the following equations based on the average value, the standard deviation, the maximum value, and the minimum value, respectively.
Line evaluation value H = (maximum value−average value) / standard deviation
Line evaluation value B = (average value−minimum value) / standard deviation
Furthermore, the line evaluation value H and the line evaluation value B are set as threshold values, which are line defect candidates but have no problem as non-defective products.
[0087]
Therefore, when the maximum value in each divided region image subjected to the horizontal profile processing is extracted as being equal to or larger than the horizontal line threshold, when the line evaluation value H> threshold or the line evaluation value B> threshold, the level having a horizontal line defect is determined. , And other than the candidates for horizontal line defects, but no problem as a non-defective product.
If the maximum value in each of the divided region images subjected to the vertical profile processing is extracted as being equal to or greater than the vertical line threshold value, if there is a line evaluation value H> threshold or line evaluation value B> threshold, there is a vertical line defect. It is evaluated as a level, and the other is a candidate for a vertical line defect, but is evaluated as a level having no problem as a non-defective product.
[0088]
Finally, the defect rank is classified based on the evaluation value obtained for the defect candidate that is the check 2 (that is, the final check) (step S98).
For this evaluation value, the threshold value for classifying the defect rank can be set in several steps for each of the defective rank and the non-defective rank of the product. As a result, it is possible to assign a rank (grade) of the spot defect to a non-defective product and to grade the product.
[0089]
According to the line defect detection processing of this embodiment, the computer 7 performs the flattening processing for removing the influence of unevenness over a relatively wide range of the background difference image subjected to the background image difference processing.
In addition, horizontal and vertical edge detection processing was performed by horizontal and vertical edge detection filters to emphasize and detect horizontal and vertical line defects in the flattened image, respectively, and line defects were emphasized. Horizontal profile processing that creates a horizontal line detection image and a vertical line detection image, divides the horizontal line detection image into four in the vertical direction, and integrates the luminance value of each pixel in each divided region in the horizontal direction to obtain an integrated value. And vertical profile processing for dividing the vertical line detection image into four parts in the horizontal direction, integrating the luminance values of the pixels in each divided area in the vertical direction to obtain an integrated value, and performing horizontal profile processing. From the integrated value of each row in each divided area, the average value, standard deviation, maximum value and minimum value of the luminance value of the integrated value of the divided area and the integrated value of each column in each divided area subjected to the vertical profile processing are calculated. Average value of the integrated value of the region, the standard deviation, the statistical data calculating for calculating the maximum value and the minimum value performed.
[0090]
Then, from the average value and the standard deviation obtained by the statistical data calculation, a horizontal line threshold value for determining the presence or absence of a horizontal line defect candidate and a vertical line threshold value for determining the presence or absence of a vertical line defect candidate are set. If the maximum value of the integrated value of each of the divided areas subjected to the profile processing exceeds the horizontal line and vertical line thresholds for determining line defects, there is no line defect on the screen. The primary determination as to whether or not there is a line defect on the screen based on whether or not the horizontal line and vertical line thresholds for determining one line defect candidate have been exceeded is performed. Therefore, there is no white line or black line for each divided area image. It is possible to easily determine in a short time whether it is a non-defective product or a defect candidate having a line defect of a white line or a black line.
[0091]
Further, for the image of the defect candidate determined to have a line defect, the line evaluation value H and the line evaluation value are calculated by a predetermined formula based on the average value, standard deviation, maximum value, and minimum value of the integrated value of the divided area. The evaluation value calculation processing for calculating the value B is performed, and whether the line evaluation value H and the line evaluation value B obtained by the evaluation value calculation processing exceed the threshold value of the level of a preset line defect candidate but no problem Since the defect rank of the line defect candidate is determined based on the reason, thin line defects and short line defects can be detected with high accuracy with respect to the line defects, and the evaluation value is calculated only for the defect candidate. The calculation time is short, and the defect can be ranked in a short time.
[0092]
(5) Pixel unevenness defect detection processing will be described with reference to the flowchart in FIG.
First, expansion processing is performed on the background difference image (step S101).
This dilation processing is performed, for example, when a liquid crystal light valve includes a black matrix as a light-shielding portion such as a driving element, in order to reduce the influence of the black matrix.
In this expansion processing, for example, a maximum filter process is performed in which a region including eight neighboring pixels centered on a certain pixel of interest, that is, a maximum value in a 3 × 3 pixel region centered on the pixel of interest is replaced with the pixel of interest. Things. By this processing, the white region expands and the black matrix portion, which is a minute black region, is reduced, so that the components of the black matrix can be reduced. This expansion processing is not limited to the maximum filter, and a morphology processing or the like for expanding a white area may be used.
[0093]
Next, a Sobel filter process is performed on the expanded image (step S102).
In the Sobel filter processing, the image subjected to the expansion processing is detected by using a Sobel filter so as to detect edges and contours of the image and emphasize the image.
The Sobel filter is a type of filter that emphasizes edges and outline components of an image (here, a defect), and expresses the edge (edge) or outline of an image by expressing a difference (inclination) between adjacent pixels as a luminance value. Emphasize. For example, when a Sobel filter is applied to a 3 × 3 pixel area as shown in FIG. 26, the luminance value L of the central E pixel_RIs calculated by the following formula:
L_R= (X²+ Y²)^1/2
Where X = (C + 2F + I)-(A + 2D + G)
Y = (A + 2B + C)-(G + 2H + I)
Using this Sobel filter, the filter is applied to the lower right corner while shifting one pixel at a time from the upper left corner of the image to the right and downward, and the luminance value of each pixel is calculated. An example is shown in FIG. FIG. 7A shows the luminance value of each pixel before processing, and FIG. 7B shows the calculation result of the luminance value after Sobel filter processing.
As is apparent from the result, after the filtering process, the difference in the luminance value between the adjacent pixels is detected and emphasized, so that the edge and the outline of the image are clearly displayed.
Other filters for such image (defect) enhancement processing include a Laplacian filter or the like, and this filter may be used.
[0094]
Further, an area division process is performed after the Sobel filter process (step S103).
In the area division processing, the inspection image 15 is divided horizontally and vertically into a plurality of areas after the Sobel filter processing. Here, as shown in FIG. 28, the area is divided into 16 4 × 4 areas 16. The purpose of dividing the image into a plurality of areas is to statistically process the brightness value of each of the divided areas 16 described below to specify the location of the pixel unevenness defect and to quantify the pixel unevenness defect. is there.
It should be noted that as the number of area divisions of an image increases, the pixel unevenness defect in a local region can be detected, and the detection accuracy increases. However, in this embodiment, since the actual pixel unevenness defect appears in such a manner as to cover an area about one-fourth of the display area, it is considered that such a division is sufficient. × 4 16 divisions are selected.
[0095]
Thereafter, the statistical calculation in each divided area is performed (step S104).
The statistical calculation in each divided area is to perform statistical calculation of the luminance value of each pixel in the divided area for each divided area 16. Then, the standard deviation σ of the luminance value is obtained for each divided area 16.
For example, FIG. 29 is a graph three-dimensionally representing the standard deviation σ of the luminance value for each divided area. The graph of FIG. 29 is displayed on the display device 8.
[0096]
Further, the maximum standard deviation corresponding to the calculation of the evaluation value is extracted (step S105).
The extraction of the maximum standard deviation is a step of obtaining a maximum value (maximum standard deviation) σmax from among the standard deviations σ of the luminance values for each divided area obtained as described above. The maximum standard deviation σmax indicates an area where pixel unevenness defects occur most intensively.
Therefore, in the present embodiment, the maximum standard deviation σmax is used as an evaluation value when detecting a pixel unevenness defect.
Finally, the defect rank is classified based on the evaluation value obtained for the defect candidate that is the check 2 (that is, the final check) (step S106).
In detecting a pixel unevenness defect, a predetermined threshold value is set, and if the maximum standard deviation σmax is equal to or smaller than the threshold value, it is determined that there is no pixel unevenness defect. Further, the threshold value can be set in several steps for each good product rank. As a result, it is possible to assign a rank (grade) in a non-defective product of the pixel unevenness defect and to classify the product.
[0097]
According to the pixel unevenness defect detection processing of this embodiment, it is possible to quantitatively automatically detect a pixel unevenness defect present on the screen of a display device such as a liquid crystal panel.
In addition, since pixel unevenness defects are evaluated using the maximum standard deviation, the quality data of products and parts can be aggregated and analyzed, which can be used for quality control. It is also possible to build.
[0098]
As described above, according to this embodiment, when detecting defects in various types of defects, the defect modes are changed to point defect detection processing, spot detection processing, streak detection processing, unevenness detection processing, line detection processing, and pixel unevenness detection processing. Classify as such, apply the processing method for each mode, calculate the luminance statistical data based on the luminance value of each pixel in the image for each mode, and set the luminance value threshold based on the luminance statistical data Since the defect candidate is extracted from the luminance statistical data and the threshold value, the primary decision is made based on the statistical data of the entire image for each defect mode, so that the number of processed images can be reduced and the processing time can be reduced. Can be.
Further, a plurality of lighting states of the liquid crystal light valve are set, and an image is captured. That is, for example, by not only capturing a specific image for inspection, but also changing the lighting state and capturing an all-white image, an all-black image, and an image in an intermediate state between white and black, respectively, as an inspection target. It is also possible to detect various types of defects that occur only in a certain gradation in each mode.
[0099]
Furthermore, an optimal exposure time is set for each image, for example, a short exposure time is set for a bright image, and a long exposure time is set for a dark image. Accuracy can be captured.
Furthermore, for the extracted defect candidate, the evaluation value is calculated based on the luminance statistical data, and the defect is evaluated. Therefore, the state of the defect can be quantitatively evaluated for each defect mode. High-precision inspection has become possible.
Further, since the evaluation result for each defect mode is obtained in this manner, detection sensitivity tuning (consistency with a visual inspection) for each defect mode becomes possible, and an inspection closer to the judgment made visually can be performed. Was.
[0100]
Furthermore, by setting a threshold value for each defect processing evaluation value in several stages for each defective product and good product rank, it is possible to assign a rank (grade) within the non-defective product and grade the product. .
FIG. 30 is a graph showing the classification of the defect rank of the evaluation value for each mode for the defect candidates of various defects. By displaying this graph on the display device 8, it is possible to understand at a glance the rank (grade) of the defect in the non-defective product. In this graph, the larger the evaluation value is, the worse the defect is, and the smaller the evaluation value is, the better the defect is. Also, the evaluation value for judging a defective product and a good product for each defect is slightly different.
[0101]
As shown in the graph of FIG. 30, the calculated evaluation value is used not only for defect determination, but also according to the magnitude of the value, the products are classified into several groups, and the defects are ranked. And products have been graded. For example, when a liquid crystal panel is used as a light valve of a projector for each of R (red), G, G ′ (green), and B (blue) colors, the relative luminous efficiency is green (wavelength λ = around 555 nm). ), The product with the lowest evaluation value ranks the product of G, the product with the next highest rank is the product of G ', the product with the higher rank is the product of R, and the product with the next rank is B. Products, and those with higher evaluation values, are classified as defective.
In each defect in the graph, an asterisk (*) indicates a specific evaluation value of a defect in one captured image.
Therefore, by looking at the location where the * mark is present in the graph, it is possible to know the degree and defect rank of each defect in the captured image.
[0102]
The present invention is not limited to a liquid crystal panel using the above-described TFT element, and displays such as a liquid crystal panel, a plasma display, an organic EL display, and a DMD (direct mirror device) using other diode elements. It can be used for inspection of body parts and display devices and products using them, and it is needless to say that the use of such components is not excluded from the scope of the present invention.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration of a screen defect detection device according to an embodiment.
FIG. 2 is a flowchart showing a process from detection of various defects to display of results.
FIG. 3 is a diagram showing a background image difference process.
FIG. 4 is a flowchart illustrating a point defect detection process.
FIG. 5 is a diagram illustrating an example of a 7 × 7 Tophat filter.
FIG. 6 is an image having a bright defect and a dark defect.
FIG. 7 is an image subjected to Tophat filter processing.
FIG. 8 is an image subjected to light / dark defect extraction processing.
FIG. 9 is a flowchart showing a stain detection process.
FIG. 10 is a schematic diagram of a flattened image and a reduced stain defect detection image.
FIG. 11 is an explanatory diagram of a method for reducing an image size.
FIG. 12 is a diagram showing an example of a well filter.
FIG. 13 is a flowchart illustrating a streak detection process.
FIG. 14 is a graph showing a luminance value cut out from each image of FIG. 3;
FIG. 15 is a diagram showing examples of various line detection filters.
FIG. 16 is a diagram showing an image subjected to a line detection filter process.
FIG. 17 is a graph showing luminance values of various images subjected to the line detection filter processing.
FIG. 18 is a diagram showing an image binarized for performing a blob process.
FIG. 19 is a flowchart illustrating unevenness detection processing.
FIG. 20 is a schematic diagram of a flattened image and a non-uniform defect detection image.
FIG. 21 is a flowchart illustrating a line detection process.
FIG. 22 is a diagram showing examples of various edge detection filters.
FIG. 23 is a diagram showing a line defect detection image that has been subjected to the edge detection filter processing.
FIG. 24 is a view showing profile data that has been subjected to split profile processing.
FIG. 25 is a flowchart illustrating pixel unevenness detection processing.
FIG. 26 is a diagram showing a pixel positional relationship in a calculation formula of the Sobel filter.
FIG. 27 is a diagram showing examples of numerical values before and after processing of a Sobel filter.
FIG. 28 is a diagram showing an example of image area division.
FIG. 29 is a graph showing a standard deviation of luminance values for each divided area.
FIG. 30 is a graph showing defect rank classification of defect candidates for various defects.
[Explanation of symbols]
1 projector, 2 liquid crystal panels, 3 screens, 4 images, 5 pattern generators, 6 CCD cameras, 7 computers, 8 display devices, 10 inspection target screens.

Claims (16)

Capturing an image of the inspection target;
From the image captured by the imaging, a step of taking a difference from a background image created in advance to create an inspection image,
Six detection processing steps for detecting each of a point defect, a spot defect, a streak defect, a mura defect, a line defect, and a pixel mura defect from the inspection image;
Calculating luminance statistical data based on the luminance value of each pixel in the image after each detection process,
Setting a threshold value of the luminance value based on the luminance statistical data, and extracting a defect candidate from the luminance statistical data and the threshold value;
A screen defect detection method, comprising: 2. The screen defect detection method according to claim 1, wherein a plurality of lighting states of the image to be inspected are set, and the set images are imaged. 3. The screen defect detection method according to claim 1, wherein an image of the inspection target is captured at an optimum exposure time, and data of the captured image is 1296 bits or more of 4096 gradations. . The threshold value for extracting the defect candidates of the point defect, the stain defect, the streak defect, the unevenness defect, and the line defect is determined for each of the bright defect and the dark defect. Screen defect detection method described. The threshold value for extracting the defect candidates of the point defect, the stain defect, the streak defect, the unevenness defect, and the line defect is determined using an average value and a standard deviation of the luminance statistical data. Screen defect detection method described. For the extracted defect candidate of the point defect, the spot defect, the streak defect, the streak defect, the unevenness defect, and the line defect, a characteristic value of the defect candidate is obtained, and an evaluation value is calculated based on the characteristic value of the defect candidate and the statistical data. The screen defect detection method according to claim 1, further comprising a step. 7. The screen defect detection method according to claim 1, wherein the evaluation value of the defect candidate is calculated for each of a light defect and a dark defect. 8. The method according to claim 6, wherein the evaluation value of the defect candidate is calculated using an average value and a standard deviation of the luminance statistical data, and a maximum luminance and a minimum luminance of the defect candidate. 9. Screen defect detection method. 9. The screen defect detection method according to claim 6, wherein the good product rank is classified according to the magnitude of the evaluation value of the defect candidate. In the detection processing step of detecting the point defect, a bright spot defect and a dark spot defect are emphasized by applying a top hat filter or a well filter to the inspection image, and the bright spot defect and the dark spot defect are simultaneously emphasized. 4. The screen defect detection method according to claim 1, wherein a substantially center value of the displayed gradation is added as an offset value. In the detection processing step of detecting the stain defect, a flattening process is performed on the inspection image, a reduced image of a plurality of stages is created from the flattened image, and a top image is formed on each of the reduced images to emphasize the stain defect. The filter processing by a hat filter or a well filter is performed to emphasize a stain, and at the same time, an approximate center value of a gradation displaying an image is added as an offset value. Screen defect detection method according to 1. In the detection processing step of detecting the streak defect, a flattening process is performed on the inspection image, a plurality of reduced images are created from the flattened image, and each reduced image corresponds to a streak in various directions. In this way, the line detection filters whose emphasis angles are changed in four stages are respectively applied to enhance the streak defect, and the streak defect is emphasized, and at the same time, the approximate center value of the gradation displaying the image is added as an offset value. The screen defect detection method according to claim 1, wherein: In the defect detection step of detecting the uneven defect, the inspection image or a reduced image obtained by reducing the inspection image is subjected to a plurality of levels of image flattening processing to generate first and second flattened images, A difference process is performed between an image between an inspection image or a reduced image of the inspection image and the first flattened image and between an image between the first flattened image and the second flattened image. 4. The screen defect detection method according to claim 1, wherein a substantially center value of gradations displaying an image is added as an offset value. The detection processing step of detecting the line defect performs a flattening process on the inspection image, and emphasizes the line defect on the flattened image by applying an edge detection filter that emphasizes horizontal and vertical edges, respectively. 4. The screen defect detection method according to claim 1, wherein a line defect is emphasized and, at the same time, a substantially central value of a gray scale displaying an image is added as an offset value. The detection processing step of detecting the pixel unevenness defect performs an expansion process on each pixel of the inspection image, and applies a Sobel filter or a Laplacian filter to the expanded image to emphasize a luminance difference between adjacent pixels. 4. The method of detecting defects on a screen according to claim 1, wherein: A screen defect detection apparatus using the screen defect detection method according to claim 1.

Images