TW202217659A - Image identification device and image identification method - Google Patents

Abstract

An image identification device and an image identification method for identifying a defect on a liquid crystal display module (LCM) are provided, wherein a surface of the LCM is covered with a covering film. The image identification method includes: providing at least one side light source to illuminate the LCM; obtaining at least one black image of the LCM in a non-displayed state of the LCM under the at least one side light source turning on, and obtaining a marked area from the at least one black image when at least one defect in the at least one black image; obtaining a plurality of images of the LCM in a displayed state in a darkroom; and identifying at least one defect on the LCM according to the plurality of images and the marked area.

Description

Image recognition device and image recognition method

The present invention relates to an electronic device and an identification method using the electronic device, and more particularly, to an image identification device and an image identification method suitable for identifying defects on a liquid crystal display module.

With the increasing demand for different types of display panels such as televisions, electronic signboards, notebook computers, mobile phones or automotive displays, display panels have gradually developed into various sizes, resolutions and specifications. Among many display technologies, liquid crystal display module (LCM) is the most mature and widely used display panel technology. The liquid crystal display module does not emit light through its liquid crystal display panel, but projects the light source through the rear backlight module to provide brightness. The backlight module is mainly composed of optical films, light guide plates, diffusers, reflectors, light-emitting diode (LED) light strips, backplanes and plastic frames. Although the assembly of the liquid crystal display module is usually carried out in a clean room, it is still impossible to completely avoid the interference of foreign objects such as dust or hair during the packaging process. In addition, the electronic components themselves tend to absorb small dust, and the components themselves have defects or defects caused by the assembly process, which cause defects in the final assembled liquid crystal display module.

In response to these defects, most LCD module factories rely on human visual inspection to catch the defects and defects of semi-finished panels. However, there are problems such as inconsistent professional quality of personnel and human negligence in human inspection. On the other hand, in order to avoid external damage during the assembly stage of the LCM, a transparent plastic protective film is attached to prevent the display panel of the liquid crystal display module from being scratched during moving. Some foreign objects such as scratches, air bubbles, glue residue, oil stains or dust on the protective film will still be imaged on the display panel when the backlight module is turned on. Therefore, these foreign objects may cause the display panel to be judged as defective during automated optical inspection (AOI) or human visual inspection. In particular, AOI can only obtain images of semi-finished panels with a single viewing angle, so foreign matter on the protective film will make AOI more difficult.

AOI technology uses machine vision as an inspection technology to replace human inspection. AOI technology is usually contactless. In a fixed environment, AOI technology can acquire and analyze images with high-speed and high-precision optical imaging technology. In order to automate the detection process and improve the accuracy rate, in the past, when developing AOI technology, technicians paid more attention to the adjustment of parameters. The technician must have a certain understanding of the algorithm to know how to adjust the parameters. Furthermore, when the inspection items are replaced, technicians often need to spend a lot of time readjusting the parameters of the AOI. In the era of product diversification, LCM detection faces the following challenges: (1) There are nearly 100 types of defects, and display panels of different grades and models have different definitions for defects. (2) There are LCMs of various sizes, the appearance size, pixel resolution, and the distance between the LCM and the AOI image capture device. The relationship between these three affects the versatility of the AOI technology. (3) Long detection time and unstable detection capability. In order to meet the above-mentioned challenges, it is necessary to propose a technology that can quickly and accurately perform image recognition.

The invention provides an image identification device and an image identification method, which can accurately identify whether a liquid crystal display module covered with a protective film has defects on its surface.

An image recognition device of the present invention is suitable for identifying defects on a liquid crystal display module, wherein the surface of the liquid crystal display module is covered with a protective film, and the image recognition device includes an image capture device, at least one light-emitting element, a transceiver, and a storage medium and processor. The at least one light-emitting element is used for providing at least one side light source to illuminate the liquid crystal display module. The transceiver is communicatively connected to the liquid crystal display module, wherein the processor configures the liquid crystal display module to display through the transceiver. The storage medium stores multiple modules. The processor is coupled to the storage medium, the transceiver, the image capture device and the at least one light-emitting element, and accesses and executes a plurality of modules, wherein the plurality of modules include an image processing module and a defect identification module, wherein the image processing module The group obtains at least one black screen image of the liquid crystal display module in the non-display state under at least one side light source by using an image capturing device, and when the at least one black screen image has defects, obtains the marked area from the at least one black screen image, and A plurality of images of the liquid crystal display module in the display state in the darkroom are obtained by an image capture device, wherein the defect identification module identifies at least one defect on the liquid crystal display module according to the plurality of images and the marked area.

In an embodiment of the present invention, the above-mentioned image processing module filters quasi-periodic noise of a plurality of images to generate a plurality of filtered images, wherein the defect identification module extracts a plurality of candidate defects from the plurality of filtered images and filtering at least one candidate defect area according to the marked area to generate a defect area.

In an embodiment of the present invention, the above-mentioned defect identification module inputs the defect area of each of the plurality of images into at least one convolutional neural network to identify at least one defect on the liquid crystal display module.

In an embodiment of the present invention, the above-mentioned at least one convolutional neural network includes a first convolutional neural network and a second convolutional neural network, wherein the defect identification module is based on the first convolutional neural network and the second convolutional neural network. The voting results of the two convolutional neural networks are used to identify at least one defect on the liquid crystal display module.

In an embodiment of the present invention, the above-mentioned modules further include a MURA defect detection module. The MURA defect detection module divides each of the plurality of filtered images into a plurality of regions, calculates the background image grayscale value of each of the plurality of regions, and determines each of the plurality of regions according to the background image grayscale value. Whether one has a MURA defect.

In an embodiment of the present invention, the above-mentioned multiple images include white screen images, and the multiple modules further include a luminance detection module. The luminance detection module identifies the luminance defect among the at least one defect according to the white screen image and the distance between the image capture device and the liquid crystal display module.

In an embodiment of the present invention, the above-mentioned multiple images include special pattern images, and the multiple modules further include a line defect detection module. The line defect detection module is based on a speckle detection technology to identify a line defect in at least one defect according to a special pattern image.

In an embodiment of the present invention, the above-mentioned special pattern image makes the pixel of the liquid crystal display module different in color from the surrounding pixels adjacent to the pixel.

In an embodiment of the present invention, the above-mentioned multiple images include outer frame images, and the multiple modules further include a frame defect detection module. The frame defect detection module is based on the image binarization technology to identify the frame defect in the at least one defect according to the outer frame image.

An image recognition method of the present invention is suitable for identifying defects on a liquid crystal display module, wherein the surface of the liquid crystal display module is covered with a protective film, and the image recognition method includes: providing at least one side light source to illuminate the liquid crystal display module; obtaining At least one black screen image of the liquid crystal display module in the non-display state under at least one side light source, when the at least one black screen image has defects, the marked area is obtained from the at least one black screen image; the liquid crystal display module in the display state is obtained in the a plurality of images in the darkroom; and at least one defect on the liquid crystal display module is identified according to the plurality of images and the marked area.

In an embodiment of the present invention, the above-mentioned step of identifying at least one defect on the liquid crystal display module according to the plurality of images and the marked area includes: filtering quasi-periodic noise of the plurality of images to generate a plurality of filtered images ; extracting a plurality of candidate defect regions from a plurality of filtered images; and filtering at least one candidate defect region according to the marked region to generate a defect region.

In an embodiment of the present invention, the above-mentioned step of identifying at least one defect on the liquid crystal display module according to the plurality of images and the marked area further includes: inputting the defect area of each of the plurality of images into at least one A convolutional neural network is used to identify at least one defect on the liquid crystal display module.

In one embodiment of the present invention, the above-mentioned at least one convolutional neural network includes a first convolutional neural network and a second convolutional neural network, wherein the defect region of each of the plurality of images is input to at least one The step of identifying at least one defect on the liquid crystal display module by a convolutional neural network includes: identifying at least one defect on the liquid crystal display module according to the voting results of the first convolutional neural network and the second convolutional neural network flaw.

In an embodiment of the present invention, the above-mentioned step of identifying at least one defect on the liquid crystal display module according to the plurality of images and the marked regions includes: dividing each of the plurality of filtered images into a plurality of regions; calculating a background image grayscale value of each of the plurality of regions; and determining whether each of the plurality of regions has a MURA defect according to the background image grayscale value.

In an embodiment of the present invention, the plurality of images include white screen images obtained by an image capture device, wherein the step of identifying at least one defect on the liquid crystal display module according to the plurality of images and the marked area includes: The luminance defect among the at least one defect is identified according to the white screen image and the distance between the image capturing device and the liquid crystal display module.

In an embodiment of the present invention, the plurality of images include special pattern images, wherein the step of identifying at least one defect on the liquid crystal display module according to the plurality of images and the marked area includes: based on the speckle detection technology, according to the special pattern The pattern image is used to identify line defects in at least one defect.

In an embodiment of the present invention, the above-mentioned special pattern image makes the pixel of the liquid crystal display module different in color from the surrounding pixels adjacent to the pixel.

In an embodiment of the present invention, the plurality of images include frame images, and the step of identifying at least one defect on the liquid crystal display module according to the plurality of images and the marked area includes: based on the image binarization technology to A frame defect in the at least one defect is identified according to the outer frame image.

Based on the above, the present invention can obtain the black screen image of the liquid crystal display module in the non-display state through the light-emitting element, and compare the black screen image with the image of the liquid crystal display module in the display state on the protective film. Accurately determine whether the liquid crystal display module is defective when there may be foreign objects. In addition, the present invention can also identify the image of the liquid crystal display module, so as to determine whether the liquid crystal display module has defects such as MURA defects, luminance defects, line defects or frame defects. In this way, it can be automatically and accurately judged whether the liquid crystal display module is a defective product.

1 illustrates a schematic diagram of an image recognition device 100 according to an embodiment of the present invention, wherein the image recognition device 100 is suitable for identifying defects on a liquid crystal display module (eg, the liquid crystal display module 200 shown in FIG. 2 ). Defects include, for example, scratches, air bubbles, glue residue, oil or dust.

The image recognition device 100 may include a processor 110 , a storage medium 120 , a transceiver 130 , an image capture device 140 and a first light-emitting element 150 . In one embodiment, the image recognition device 100 may further include a second light-emitting element 160 . However, the number of the light-emitting elements of the image recognition device 100 can be configured by the user according to requirements, and the present invention is not limited thereto.

The processor 110 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose micro control unit (micro control unit, MCU), microprocessor (microprocessor), digital signal processing digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processor (graphics processing unit, GPU), image signal processor (image signal processor, ISP) ), image processing unit (IPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (field programmable gate array) , FPGA) or other similar elements or a combination of the above. The processor 110 may be coupled to the storage medium 120 , the transceiver 130 , the image capture device 140 , the first light-emitting element 150 and the second light-emitting element 160 , and access and execute a plurality of modules and modules stored in the storage medium 120 . Various applications.

The storage medium 120 is, for example, any type of fixed or removable random access memory (random access memory, RAM), read-only memory (ROM), and flash memory (flash memory). , a hard disk drive (HDD), a solid state drive (SSD), or similar components or a combination of the above components for storing a plurality of modules or various application programs executable by the processor 110 . In this embodiment, the storage medium 120 can store a luminance detection module 121, a line defect detection module 122, a frame defect detection module 123, a cloud spot (MURA) defect detection module 124, and a defect identification module The functions of the group 125 and the image processing module 126 and other modules will be described later.

The transceiver 130 transmits and receives signals in a wireless or wired manner. Transceiver 130 may also perform operations such as low noise amplification, impedance matching, frequency mixing, up or down frequency conversion, filtering, amplification, and the like. The transceiver 130 can be communicatively connected to the liquid crystal display module 200 . The processor 110 can control the liquid crystal display module 200 through the transceiver 130 to display (or turn off the liquid crystal display module 200).

The image capturing device 140 is, for example, a video camera or a camera. The processor 110 can obtain the image of the liquid crystal display module 200 in the displayed state or the non-displayed state through the image capturing device 140 .

When the surface of the liquid crystal display module 200 is covered with a protective film, the current image recognition technology may misjudge the foreign matter (or defect) on the protective film as a defect of the liquid crystal display module 200 . Foreign objects on the protective film (for example: scratches, dirt, glue residue or dust) often form irregular shapes. In order to identify foreign objects on the protective film, the processor 110 can adjust the liquid crystal display module 200 to a non-display state (not turned on) through the transceiver 130, and control the first element 150 and/or the second light-emitting element 160 to provide to illuminate the side light source of the liquid crystal display module 200 .

可大於第二發光元件160照射在液晶顯示模組200的入射光的入射角

。由第一發光元件150及第二發光元件160所提供的入射光可使保護膜300上的異物更加明顯。當第一發光元件150的光束以及第二發光元件160的光束分別照射液晶顯示模組200時，處理器110可借由收發器130將液晶顯示模組200調整為未顯示狀態。接著，影像處理模組126可借由影像擷取裝置140來取得未顯示狀態下的液晶顯示模組200在側光源下的至少一黑屏影像。當液晶顯示模組200形成黑屏影像，影像處理模組126判斷黑屏影像中是否有瑕疵存在，也就是在黑屏影像中是否有不均勻的亮度區域存在。 FIG. 2 shows the use of light-emitting elements (ie, the first light-emitting element 150 and the second light-emitting element 160 ) and the image capturing device to capture images on the protective film 300 on the surface 210 of the liquid crystal display module 200 according to an embodiment of the present invention Schematic diagram of a foreign body. As shown in FIG. 2 , the first light-emitting element 150 and the second light-emitting element 160 are side light sources, which can be arranged on the side of the liquid crystal display module 200 and are arranged at different heights (taking the liquid crystal display module 200 as a reference) plane) to provide incident light of the liquid crystal display module 200 at different angles. For example, the incident angle of the incident light illuminated by the first light-emitting element 150 on the liquid crystal display module 200

The incident angle of the incident light that the second light-emitting element 160 illuminates on the liquid crystal display module 200 can be greater than

. The incident light provided by the first light emitting element 150 and the second light emitting element 160 can make the foreign matter on the protective film 300 more obvious. When the light beam of the first light emitting element 150 and the light beam of the second light emitting element 160 respectively illuminate the liquid crystal display module 200 , the processor 110 can adjust the liquid crystal display module 200 to a non-display state through the transceiver 130 . Next, the image processing module 126 can obtain at least one black screen image of the liquid crystal display module 200 in the non-display state under the side light source through the image capturing device 140 . When the liquid crystal display module 200 forms a black screen image, the image processing module 126 determines whether there is a defect in the black screen image, that is, whether there is an uneven brightness area in the black screen image.

3A shows a schematic diagram of a black screen image 20 of the liquid crystal display module 200 in a non-display state according to an embodiment of the present invention, and FIG. 3B shows a schematic diagram of an image 24 of the liquid crystal display module 200 in a display state according to the prior art Schematic. In the black screen image 20 , the areas pointed by the arrows are foreign objects or defects on the protective film 300 , not foreign objects or defects on the liquid crystal display module 200 . That is, under the illumination of the side light source, the black screen image 20 of the liquid crystal display module 200 in the non-display state can make foreign objects or defects on the protective film 300 more obvious; There is no light in the area), the image 24 of the liquid crystal display module 200 in the display state (the backlight is turned on), in the image 24, no foreign objects or defects of the liquid crystal display module 200 are identified, but sometimes the protective film still appears The possibility of misjudgment of the liquid crystal display module 200 is caused by foreign objects or defects on the 300 . In other words, the present invention clearly identifies foreign objects or defects on the protective film 300 in the black screen image 20 of the liquid crystal display module 200 in the non-display state under the illumination of the side light source.

After capturing the black screen image 20 of the liquid crystal display module 200 , the image processing module 126 can identify the black screen image 20 by using an image recognition technology such as an image binarization technology, so as to find the protective film Foreign objects on the 300. Next, the image processing module 126 can define the area where the foreign object is located as the marking area 21, as shown in FIG. 3A or FIG. 3B .

On the other hand, the processor 110 can control the liquid crystal display module 200 through the transceiver 130 to perform display (turn on the backlight of the liquid crystal display module 200) when the first light-emitting element 150 and the second light-emitting element 160 are turned off. . The image processing module 126 can obtain a plurality of images of the liquid crystal display module 200 in the display state under the ambient light source through the image capture device 140 , wherein the plurality of images include, for example, images with the backlight turned on and a black screen displayed ( For example, the backlight module of the liquid crystal display module 200 is turned on to emit light, and the liquid crystal module of the liquid crystal display module 200 does not allow the backlight to pass through, wherein the liquid crystal module includes a liquid crystal panel and a polarizer), a white screen image (for example: the liquid crystal display module 200 The liquid crystal element of the liquid crystal display module 200 allows the backlight to pass through) and presents a grayscale image (for example, the liquid crystal element of the liquid crystal display module 200 allows part of the backlight to pass through, for example, through 30% of the backlight). The image processing module 126 can identify the defects of the liquid crystal display module 200 according to the plurality of images. FIG. 4A is a schematic diagram illustrating an image 22 of a liquid crystal display module 200 having flaws 40 according to an embodiment of the present invention. As shown in FIG. 4A, the grid and moiré in the image 22 make the defect 40 difficult to identify. The defects 40 are, for example, scratches on the liquid crystal display module 200 or foreign objects in the liquid crystal display module 200 .

At present, electronic products using liquid crystal display modules are very diverse, so the sizes of the liquid crystal display modules are also very diverse. When using the image capture device 140 to inspect the liquid crystal display module 200 , the size of the liquid crystal display module 200 , the resolution of the pixels of the liquid crystal display module 200 , and the distance between the image capture device 140 and the liquid crystal display module 200 are three The relative relationship between them will affect the frequency of the grid and the moiré displayed in the image 22 of the liquid crystal display module 200 obtained by the image capturing device 140 . Therefore, the image processing module 126 cannot filter the grid and moiré of an image by the preset filtering function. Therefore, the following content proposes how to filter images with grids and moiré. 5A and 5B illustrate schematic diagrams of images 10 and 12 with grids and moirés, wherein arrow 11 indicates the direction of the moiré of image 10, and arrows 13 and 14 indicate the moiré of image 12, according to an embodiment of the present invention. The direction of the column ripple.

In order to eliminate the grid and moire, the image processing module 126 can filter the plurality of images obtained by the image capture device 140 by filtering quasi-periodic noise, thereby generating a plurality of filtered images image. Specifically, the image processing module 126 can detect the peak value in the frequency domain of the obtained image, and regard the peak value with the lowest frequency and the peak value of its frequency multiplication as the lattice frequency constituting the moiré. The image processing module 126 can reduce the energy of the moiré by eliminating the energy of the lattice frequency in the frequency domain, thereby filtering the moiré, as shown in FIG. 6 . 6 is a schematic diagram illustrating filtered images 31 , 32 and 33 of the liquid crystal display module 200 according to an embodiment of the present invention, wherein the filtered image 31 is generated by filtering an image when the liquid crystal display module 200 displays a black screen , The filtered image 32 is generated by the image 22 after filtering the white screen image of the LCD module 200 , and the filtered image 33 is generated by filtering the grayscale image of the LCD module 200 . The image processing module 126 can perform quasiperiodic noise according to the method disclosed in " Sur, Frédéric, and Michel Grediac. "Automated removal of quasiperiodic noise using frequency domain statistics." Journal of Electronic Imaging 24.1 (2015): 013003 " filter.

After obtaining the plurality of filtered images 31 , 32 and 33 , the defect identification module 125 can identify the defects on the liquid crystal display module according to the marked area 21 and the plurality of filtered images 31 , 32 and 33 , but does not include protection Foreign objects or defects on the membrane 300. Specifically, the defect identification module 125 can obtain at least one defect candidate area or multiple defect candidate areas from the plurality of filtered images 31 , 32 and 33 , and perform the at least one defect candidate area on the at least one defect candidate area according to the marked area 21 . Filtering to generate defective areas of the liquid crystal display module 200 . Taking the image 22 and the filtered image 32 as examples, FIG. 4A also includes candidate defect regions 50 of the liquid crystal display module 200 . FIG. 4B is a schematic diagram illustrating defects 40 in the filtered image 32 of the liquid crystal display module 200 according to an embodiment of the present invention. Referring to FIGS. 4A and 4B , after the image processing module 126 filters the quasi-periodic noise of the image 22 to generate the filtered image 32 , the imperfections 40 that were otherwise indistinguishable in the image 22 will be clearly presented in the filtered image 32 . Accordingly, the defect identification module 125 can be based on machine vision or dynamic thresholding (refer to "Kang, Wenxiong, Yang, Qing-Qiang, and Liang, Run-Peng, "The Comparative Research on Image Segmentation Algorithms." First International Workshop on Education Technology and Computer Science, 2009" ) and other technologies to identify the defects 40 in the filtered image 32 , and extract candidate defect regions 50 corresponding to the defects 40 from the image 22 .

After obtaining the candidate defect region 50 , the defect identification module 125 may respond that the marked region 21 is not the same as the candidate defect region 50 , and determine that the marked region 21 corresponds to a defect or foreign matter on the protective film 300 , and the candidate defect region 50 corresponds to defects in the liquid crystal display module 200 . The defect identification module 125 may define the candidate defect area 50 as a defect area in response to judging that the candidate defect area 50 corresponds to the defect 40 of the liquid crystal display module 200 .

After obtaining the defect area of each of the images, the defect identification module 125 can identify the defect 40 of the liquid crystal display module 200 through a neural network. FIG. 7 is a schematic diagram illustrating the identification of defects 40 on the liquid crystal display module 200 by means of convolutional neural networks 71 and 72 according to an embodiment of the present invention. The defect identification module 125 may input the defect region of each of the image 22 , such as a black frame image, a grayscale image, and a white screen image, to the convolutional neural networks 71 and 72 . Taking the candidate defect area 50 of the image 22 in FIG. 4A as an example, the defect identification module 125 can input the image of the candidate defect area 50 and at least one black screen image corresponding to the position of the candidate defect area 50 to the convolutional neural networks 71 and 72 . After the convolutional neural networks 71 and 72 respectively generate and judge whether the candidate defect region 50 contains a plurality of identification results, the defect identification module 125 can generate and judge whether the candidate defect region 50 exists in the candidate defect region 50 in a voting mode according to the identification results. Defect 40 judgment result. It is worth noting that the number of convolutional neural networks used by the defect identification module 125 can be adjusted according to usage requirements, and the present invention is not limited thereto. Compared to inputting the entire image 22 to the convolutional neural networks 71 and 72, inputting only the image of the candidate defect region 50 to the convolutional neural networks 71 and 72 will significantly reduce the amount of time required by the convolutional neural networks 71 and 72. The amount of computation consumed.

Since the backlight module of the liquid crystal display module 200 uses components such as a light guide plate, a diffuser plate, or a reflective plate, the light emitted by the side LED light bars is diffused to the entire display panel. In other embodiments, the backlight module of the liquid crystal display module 200 is a direct type backlight module. Therefore, defects in the materials of the above-mentioned elements may cause the backlight to become non-uniform. In addition, the brightness detection module 121 can be used to identify whether the backlight of the liquid crystal display module 200 maintains good brightness quality and uniformity. Specifically, the brightness detection module 121 can identify the brightness of the liquid crystal display module 200 according to the image 22 (white screen image) of the liquid crystal display module 200 and the distance between the image capture device 140 and the liquid crystal display module 200 flaw. After obtaining the white screen image 22 and the distance, the brightness detection module 121 can obtain a preset corresponding to the distance in a specific area of the white screen image 22 from the mapping relationship between the brightness and the distance pre-stored in the storage medium 120 Brightness. The specific area can be a preset area, such as the middle area or the edge area of the image. If the preset luminance matches the luminance of a specific area of the white screen image 22, the luminance detection module 121 can determine that the specific area of the white screen image 22 does not have luminance defects. In contrast, if the preset luminance does not match the luminance of a specific area of the white screen image 22, the luminance detection module 121 can determine that the specific area of the white screen image 22 has a luminance defect. The luminance detection module 121 can perform luminance defect detection according to the calibration parameters of the image capture device 140 and luminance measurement based on the content disclosed in the US application US6982744B2 "Multi-point calibration method for imaging light and color measurement device". Identify.

The line defect detection module 122 is used for identifying line defects of the liquid crystal display module 200 . Specifically, the image processing module 126 can obtain a plurality of images of the liquid crystal display module 200 in the darkroom in the display state through the image capturing device 140 , and the plurality of images can include special pattern images. The special pattern image can make a pixel of the liquid crystal display module 200 different in color from surrounding pixels adjacent to the pixel. After obtaining the special pattern image, the line defect detection module 122 can identify the line defect of the liquid crystal display module 200 according to the special pattern image based on the blob detection technology. The line defects of liquid crystal are usually vertical lines or horizontal lines. We change the Laplacian of Gaussian function commonly used in blob detection to detect vertical lines or horizontal lines by filtering.

The frame defect detection module 123 can be used to identify the frame defect (WAKU defect) of the liquid crystal display module 200 . Specifically, the image processing module 126 can obtain a plurality of images of the liquid crystal display module 200 in the display state under the ambient light source through the image capture device 140 , wherein the plurality of images may include the outer surface of the liquid crystal display module 200 . Frame the image. After acquiring the outer frame image, the frame defect detection module 123 can identify the frame defect of the liquid crystal display module 200 according to the outer frame image based on the image binarization technology.

The MURA defect detection module 124 can be used to identify MURA defects of the liquid crystal display module 200 . Specifically, MURA defect detection module 124 may segment each of filtered images 31, 32, and 33 into multiple regions. Taking the filtered image 32 as an example, FIG. 8 illustrates a schematic diagram of dividing the filtered image 32 into a plurality of regions 61 and 62 according to an embodiment of the present invention. After the filtered image 32 is segmented to generate the regions 61 and 62, the MURA defect detection module 124 can calculate the background image grayscale values of the regions 61 and 62, respectively. Next, the MURA defect detection module 124 can perform automatic threshold segmentation on the grayscale value of the background image in the area 61 to determine whether there is a MURA defect in the area 61, and can perform automatic threshold segmentation on the grayscale value of the background image in the area 62 to obtain It is determined whether there is a MURA defect in the area 62 . The MURA defect detection module 124 can be based on " Fan, Shu-Kai S., and Yu-Chiang Chuang. "Automatic detection of MURA defect in TFT-LCD based on regression diagnostics." Pattern recognition letters 31.15 (2010): 2397- 2404 " to automatically threshold the grayscale values of the background image.

9 shows a flowchart of an image recognition method according to an embodiment of the present invention, wherein the image recognition method can be used to recognize whether a liquid crystal display module covered with a protective film has defects, and the image recognition method can be used as follows The image recognition apparatus 100 shown in FIG. 1 is implemented. In step S901, multiple images of the liquid crystal display module are obtained, wherein the multiple images include a black screen image, an image with a backlight and displaying a black screen, a grayscale image, a white screen image, a special pattern image, and an outer frame image. In step S902, the brightness defect of the liquid crystal display module is identified according to the white screen image. In step S903, the line defects of the liquid crystal display module are identified according to the special pattern image. In step S904, the frame defects of the liquid crystal display module are identified according to the outer frame image. In step S905, the plurality of images are filtered to generate filtered images. In step S906, the MURA defect of the liquid crystal display module is identified according to the filtered image. In step S907-1, the liquid crystal display module 200 forms a black screen image, and the image processing module preliminarily determines whether there is a defect in the black screen image, that is, whether there is an uneven brightness area in the black screen image. The black screen image is obtained by using an image capturing device to obtain an image of the liquid crystal display module in a non-display state under at least one light source. If not, go to step S910. If yes, go to step S907. In step S907 , according to the black screen image, the area where the foreign matter on the protective film is located is defined as the marked area, and the candidate defect area corresponding to the defect is extracted from the filtered image. In step S908, the candidate defect areas are filtered according to the marked areas to generate defect areas. In step S909, the defect area is input to a plurality of one or more convolutional neural networks to identify the defects of the liquid crystal display module. In step S910, it is determined whether the identification liquid crystal display module has any defects. If the liquid crystal display module does not have any defects, go to step S911. If the liquid crystal display module has at least one defect, step S912 is entered. In step S911, the liquid crystal display module is determined as a good product. In step S912, the liquid crystal display module is determined as a defective product.

To sum up, the present invention can obtain the black screen image of the liquid crystal display module in the non-display state through the light-emitting element, and protect the protection by comparing the black screen image with the image of the liquid crystal display module in the display state Accurately determine whether the liquid crystal display module has defects when there may be foreign objects on the film. In addition, the present invention can also judge whether the liquid crystal display module has MURA defects according to the grayscale value of the background image. Furthermore, the present invention can also judge whether the liquid crystal display module has a brightness defect by the distance between the image capturing device and the liquid crystal display module. On the other hand, the present invention can utilize a liquid crystal display module to display a special pattern image that can assist the processor to identify line defects. The present invention can also judge whether the liquid crystal display module has frame defects based on the image binarization technology. However, the above are only preferred embodiments of the present invention, and should not limit the scope of the present invention, that is, any simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the contents of the description of the invention, All still fall within the scope of the patent of the present invention. In addition, any embodiment or claimable scope of the present invention is not required to achieve all of the objects or advantages or features disclosed in the present invention. In addition, terms such as "first" and "second" mentioned in this specification or the scope of the patent application are only used to name the names of elements or to distinguish different embodiments or ranges, rather than to limit the upper or lower limit of the number of elements .

10, 12: Image with grid and moiré 100: Image recognition device 11, 13, 14: Arrows 110: Processor 120: Storage Media 121: Brightness detection module 122: Line defect detection module 123: Frame defect detection module 124:MURA defect detection module 125: Defect recognition module 126: Image processing module 130: Transceiver 140: Image capture device 150: The first light-emitting element 160: The second light-emitting element 20: Black Screen Image 200: LCD module 21: Mark the area 22: Video 24: Video 210: Surface 300: Protective film 31, 32, 33: Filtered images 40: Flaws 50: Candidate defect area 61, 62: Area of filtered image 71, 72: Convolutional Neural Networks S901, S902, S903, S904, S905, S906, S907-1, S907, S908, S909, S910, S911, S912: Steps

FIG. 1 is a schematic diagram of an image recognition device according to an embodiment of the present invention. 2 is a schematic diagram of capturing foreign objects on the protective film on the surface of the liquid crystal display module by using the light-emitting element and the image capturing device according to an embodiment of the present invention. 3A is a schematic diagram illustrating a black screen image of a liquid crystal display module in a non-display state according to an embodiment of the present invention. FIG. 3B is a schematic diagram illustrating an image of the liquid crystal display module in a display state according to the prior art. 4A is a schematic diagram illustrating an image of a liquid crystal display module having defects according to an embodiment of the present invention. 4B is a schematic diagram illustrating defects in a filtered image of a liquid crystal display module according to an embodiment of the present invention. 5A and 5B illustrate schematic diagrams of images with grids and moirés, according to embodiments of the present invention. 6 is a schematic diagram illustrating a filtered image of a liquid crystal display module according to an embodiment of the present invention. 7 illustrates a schematic diagram of identifying defects on a liquid crystal display module by means of a convolutional neural network (CNN) according to an embodiment of the present invention. FIG. 8 illustrates a schematic diagram of dividing a filtered image into a plurality of regions according to an embodiment of the present invention. FIG. 9 is a flowchart illustrating an image recognition method according to an embodiment of the present invention.

S901, S902, S903, S904, S905, S906, S907-1, S907, S908, S909, S910, S911, S912: Steps

Claims (18)

An image recognition device suitable for recognizing defects on a liquid crystal display module, wherein the surface of the liquid crystal display module is covered with a protective film, and the image recognition device comprises: image capture device; at least one light-emitting element for providing at least one side light source to illuminate the liquid crystal display module; a transceiver, communicatively connected to the liquid crystal display module, wherein the processor controls the liquid crystal display module to display through the transceiver; storage media, storing multiple modules; and a processor, coupled to the storage medium, the transceiver, the image capture device and the at least one light-emitting element, and accesses and executes the multiple modules, wherein the multiple modules include images Processing module and defect identification module, among which The image processing module obtains at least one black screen image of the liquid crystal display module in a non-display state under the at least one side light source through the image capture device, When the at least one black screen image has defects, a marked area is obtained from the at least one black screen image, and a plurality of the liquid crystal display modules in the display state in the darkroom are obtained by the image capturing device images, which The defect identification module identifies at least one defect on the liquid crystal display module according to the plurality of images and the marked area. The image recognition device of claim 1, wherein the image processing module filters quasi-periodic noise of the plurality of images to generate a plurality of filtered images, wherein the defect recognition module extracts the data from the plurality of At least one candidate defect area is extracted from the filtered image, and the at least one candidate defect area is filtered according to the marked area to generate a defect area. The image recognition device of claim 2, wherein the defect identification module inputs the defect region of each of the plurality of images to at least one convolutional neural network to identify the liquid crystal display module the at least one defect above. The image recognition device of claim 3, wherein the at least one convolutional neural network comprises a first convolutional neural network and a second convolutional neural network, wherein the defect recognition module is based on the first The voting results of the convolutional neural network and the second convolutional neural network are used to identify the at least one defect on the liquid crystal display module. The image recognition device according to claim 2, wherein the plurality of modules further comprise: The cloud spot defect detection module divides each of the plurality of filtered images into a plurality of regions, calculates a background image grayscale value of each of the plurality of regions, and calculates the grayscale value of the background image according to the background image. The order value determines whether the each of the plurality of regions has cloud speckle defects. The image recognition device according to claim 1, wherein the plurality of images comprise white screen images, and the plurality of modules further comprise: The luminance detection module identifies the luminance defect among the at least one defect according to the white screen image and the distance between the image capturing device and the liquid crystal display module. The image recognition device according to claim 1, wherein the plurality of images comprise special pattern images, and the plurality of modules further comprise: The line defect detection module is based on the speckle detection technology to identify the line defect in the at least one defect according to the special pattern image. The image recognition device according to claim 7, wherein the special pattern image makes pixels of the liquid crystal display module different in color from surrounding pixels adjacent to the pixels. The image recognition device according to claim 1, wherein the plurality of images comprise frame images, and the plurality of modules further comprise: The frame defect detection module is based on an image binarization technology to identify a frame defect in the at least one defect according to the outer frame image. An image recognition method, suitable for identifying defects on a liquid crystal display module, wherein the surface of the liquid crystal display module is covered with a protective film, and the image recognition method includes: providing at least one side light source to illuminate the liquid crystal display module; acquiring at least one black screen image of the liquid crystal display module in the non-display state under the at least one side light source, When the at least one black screen image has defects, obtaining a marked area from the at least one black screen image; obtaining a plurality of images of the liquid crystal display module in a display state under an ambient light source; and At least one defect on the liquid crystal display module is identified according to the plurality of images and the marked area. The image recognition method according to claim 10, wherein the step of recognizing the at least one defect on the liquid crystal display module according to the plurality of images and the marked area comprises: filtering quasi-periodic noise of the plurality of images to generate a plurality of filtered images; extracting at least one candidate defect region from the plurality of filtered images; and The at least one candidate defect area is filtered according to the marked area to generate a defect area. The image recognition method according to claim 11, wherein the step of recognizing the at least one defect on the liquid crystal display module according to the plurality of images and the marked area further comprises: The defect regions of each of the plurality of images are input to at least one convolutional neural network to identify the at least one defect on the liquid crystal display module. The image recognition method of claim 12, wherein the at least one convolutional neural network comprises a first convolutional neural network and a second convolutional neural network, wherein the each of the plurality of images is The step of inputting the defect area of the user to the at least one convolutional neural network to identify the at least one defect on the liquid crystal display module includes: The at least one defect on the liquid crystal display module is identified according to the voting results of the first convolutional neural network and the second convolutional neural network. The image recognition method according to claim 11, wherein the step of recognizing the at least one defect on the liquid crystal display module according to the plurality of images and the marked area comprises: segmenting each of the plurality of filtered images into a plurality of regions; calculating a background image grayscale value for each of the plurality of regions; and Whether the each of the plurality of regions has cloud speckle defects is determined according to the background image grayscale value. The image recognition method according to claim 10, wherein the plurality of images comprise white screen images obtained by an image capture device, wherein the liquid crystal display module is identified according to the plurality of images and the marked area The at least one flawed step above includes: A luminance defect in the at least one defect is identified according to the white screen image and the distance between the image capturing device and the liquid crystal display module. The image recognition method according to claim 10, wherein the plurality of images comprise special pattern images, wherein the at least one defect on the liquid crystal display module is identified according to the plurality of images and the marked area The steps include: A line defect in the at least one defect is identified according to the special pattern image based on a speckle detection technique. The image recognition method according to claim 16, wherein the special pattern image makes pixels of the liquid crystal display module different in color from surrounding pixels adjacent to the pixels. The image recognition method according to claim 10, wherein the plurality of images comprise frame images, wherein the at least one defect on the liquid crystal display module is identified according to the plurality of images and the marked area The steps include: A frame defect in the at least one defect is identified according to the outer frame image based on an image binarization technique.

Images