TW201003038A - Method and apparatus for multiplexed image acquisition and processing - Google Patents

Abstract

A method and an apparatus for inspecting articles entails illuminating an article (50) from two or more directions with light sources (52, 56) matched to the spectral response of a Bayer filter (12) attached to an image sensor (10). The image data are separated by color and derivative images formed, then processed to detect defects.

Description

201003038 VI. Description of the Invention: [Technical Field to Which the Invention Is Applicable] The present invention is directed to the manufacture of articles using machine vision (10) achinev. In particular, when the object is moved, the image is captured by a plurality of images of the item captured at the same time. The camera uses different colors of light to illuminate in different directions—the color of the item is 2 Eight Setoes and Uses - A first-hand mirror is used to separate images from multiple images. [Prior Art] The goods produced in the Noon House are in the process of manufacturing, such as pq you _»4 么一&& Typical inspections The dust inspection system uses a two-dimensional gray-scale camera to determine the plague on the item based on the difference in grayscale reflectivity or two-dimensional shape from L (Russian).纟 Some situations where the grayscale difference pin cannot be measured, the basin uses a color camera to detect the fatigue that can be distinguished by spectral reflectance differences. In the case of any difference in grayscale or color images, it is impossible to recognize the situation of 瑕疵. The three-dimensional vision system can be used to speculate on the fatigue of the three-dimensional type difference. A two-dimensional vision system typically consists of a camera, optical instruments, and lighting equipment. Camera technology is generally limited to commercially available sensors, so it typically customizes optical instruments and illumination to provide the correct magnification, fiieid of view, and acquisition speed (acquisition). Speed). An example of a machine vision system made with off-the-shelf image sensors and custom optical instruments and lighting equipment is manufactured by Electro Scientific Industries, Inc. (the assignee of this patent application) 201003038 (IV) BulletTM Yen Yen ID reading Device. The system consists of a video sensor, optical instrument and lighting device designed to view the identification marks on semiconductor wafers, or other items. Although it is designed to capture images of fine marks formed on a mirror-like surface, the system is unable to ingest small flaws, which is the subject of the present invention. This is because there is no difference between the camera to be discovered and the surrounding environment 2 camera. These flaws may only be visually small depressions on the surface of the object and therefore cannot be captured in conventional two-dimensional images. One of the possible ways to take a picture of a sputum is to illuminate the item from at least two different directions with light projected by a grazing angle and take its image. Light must be illuminated from at least two directions, as not all: it can be seen from a single-ray angle. Dealing with all of these images is likely to identify the plague, but a separate frame phase is required for each-growth, making this process unacceptable. It is possible to flatten the ground wire, but for each direction of the light will need - the sensor, with the filter so that each light can only be seen by the camera of the field. This will cause the process to be unacceptable for the problem of shooting subtle fatigue - the possible way is to use three: and the second: the visual system produces the corresponding three-dimensional shape of the item ^ shirt image >矣+White reflects the intensity of the light wavelength from the item. According to the household, the technology used, = maintenance test, ,, _ - Wei Wei can be classified into many ethnic groups. The first type of two-dimensional system utilizes an image of a clue within a two-dimensional image. One of the examples is to obtain two or more two-dimensional images; stereoscopic image reconstruction, occupation, f > detection of the characteristics of the mother-image, try to attribute the characteristics of each image 1 to - to resolve this correspondence The problem. Inconsistent 4 201003038 The height is derived from the difference in the degree of ambiguity between images. This is the same as the case of the above-mentioned two-dimensional image. This means that the time to obtain multiple images and the equipment to be taken in parallel, 'to be solved, because 1I γ I π π here

The harness has no features to extrapolate the height. D is also a v-type. The dimensional imaging requires a projection line, usually a lightning ray or other shape is projected onto the object and the image is formed up and down. The displacement table of these rays does not have a two-dimensional contour. This method also requires multiple captures, so this application is too slow to accept. Other types of three-dimensional imaging depend on the weighted focus quality (confocal imaging) in the series of image steps or the capture of a special grid projected image (moire imaging). These two methods require multiple image captures to create a three-dimensional image, which is an unacceptable solution to the problem to be solved. In addition, such methods typically require the item to remain stationary during the scanning process. This means that the item must be designated, stopped, and placed in a stable position for scanning, which will slow down the manufacturing process. A consistent example of this method is described in Oleksandr et al., "Technology for Phase Measurement and Surface Reconstruction by Use 〇f Colored Structured Light" (Applied 〇ptics ¥ 〇1 41,

Issue 29, ΡΡ 6104-61 17 (2002)), project multiple color patterns to resolve the projected light and attempt to improve the accuracy of the three-dimensional measurement. The discussion uses the structural light to determine the surface profile of the windshield, and uses the differential equation to extract two-dimensional information from the structured light shirt. The commonality of many of these methods is that it is necessary to obtain more than one image to measure the σ u initial '- the top of the 以 u. Capture-Shift lies in • In order to capture multiple images with a single camera, the problem of multiple images of a silly knife is because the component is assumed to move through 4 and it requires multiple exposures. The image must be stopped at the position, = phase = the part is capturing multiple throughputs, = moving the camera while the part is stopped will reduce the system and it is quite difficult and expensive. Another... The solution is to make the camera, but it is more expensive and requires special: alignment between the optical instrument and the sensor. "Fal II, which requires an image capture mechanism, has the ability to capture images of the two-dimensional view that are easily obtained by the method. In addition, this mechanism must not use multiple cameras or other expensive extra equipment, and can One of the objects of the present invention is to provide an image capture system in the form of an image capture system that operates in less than a single frame time to minimize the need for the article to be forced to stop when the image is taken. The method and device have improved detection capability that is not apparently visible in the two-dimensional inspection device. Another object of the present invention is to perform a capture action without requiring the article to remain stationary during image capture. The above and other objects of the present invention are achieved by the following detailed description of a method and apparatus. By illuminating/objects with two or more different colors of light and using an installed Bayer filter, a light mirror (Bayer filter) One of the structures of the image sensor captures the data, thereby capturing two or more images of the item in a single image frame 201003038

image. The Bayer filter is a color filter that is mounted on an image sensor consisting of red, green, and blue color filters. A Bayer filter is attached to the image sensor and captures image data. When the image data is captured into the controller (either within the camera itself or an external controller), the pixels (P1Xei) are sorted according to their corresponding filter colors, in other words, all red pixels are placed in one Image, all blue pixels are placed in a different image'. And all the green pixels are placed in another different image. In this manner, a monochrome image capture sensor can be used to simulate a color image system with one of the single sensors' with a slightly smaller spatial resolution. Processed in a state of being aligned with each other such that features within an image will be the same as in other images. By matching the wavelength of the illumination to the Bayer filter, it is possible to derive three separate images from a single image taken. For example, if an item is illuminated in two different colors from two or more of the three colors, it can process the derived image to show inconsistencies between the images, thereby representing the subtle differences in the three-dimensional form that represent fatigue. In addition, the flash can be used as a flash or it can be used to gate the image sensor to paste the tribute in a very short period of time, so that not only can three aligned images be obtained at the same time, and the shirt images can be Obtained without stopping the item. In the embodiment of the Twilight 2, it illuminates an item in two different directions in a Bayer directional mirror. It illuminates in a diffuse manner by matching the third color in the Bayer mirror: pair = image data and dividing it into three separate different images: three colors of the mirror. The first two images are subtracted from each other to 7 201003038 to highlight the difference between them. @ @ appears in one image and does not appear in another image: it: the sign is likely to be 瑕疵. The resulting image is then processed using traditional: machine vision techniques to identify and classify defects. The diffused image is also processed using conventional machine vision techniques to determine if there is other fatigue such as a shape that deviates from normal values, such 瑕 = cannot be detected by directional illumination. In one embodiment of the invention, the three illumination sources are illuminated for a very short period of time by flash or flash projection. This allows the sensor to capture a moving portion of the image data without blurring due to movement. The sensor is set to integrate illumination, and the illumination source is set to flash when the sensor is integrating light. After the flash is fired, the sensor is directed to read out the image data generated by the flash, and the image data generated in the sensor is read out to the controller, wherein the image data is divided into three corresponding to the three color filters. image. " In the case where the movement of the item is extremely fast and passes through the field of view of the sensor, it is impractical to attempt to freeze the movement of the item only by flashing. In this embodiment, 'the sinusoidal curve speed curve' is implemented by the transport mechanism of the moving article so that its motion is slowed as it passes through the field of view of the sensor. In this embodiment, a controller is coupled to the delivery mechanism during operation to control the speed at which the item moves. By speeding up and slowing down the conveying mechanism according to a sinusoidal speed curve, the point of the lowest speed coincides with the time point of the illumination source flashing', then the magnitude of the item speed is reduced enough to eliminate the occurrence of blurring and at the same time slow down due to the conveying mechanism. Production reduction is minimized. 8 201003038 [Embodiment] The present invention is directed to capturing image data to support automatic viewing of electronic components. One example of this type of component is a wafer capacitor constructed from alternating layers of metal conductors and ceramic ceraniic dielectrics. These components create a struggle that is difficult or impossible for traditional 2D or 3D systems. In addition, these components are manufactured at extremely high speeds. Mass production of these devices can be performed at speeds of hundreds of thousands per hour. Finally, these components are assembled into circuits in almost all commercially available electronic devices; therefore, they are preferably tested prior to assembly into the circuit. The present invention can reduce the image capture period to a single image exposure time (2 milliseconds) and use a faster moving curve of 18 milliseconds. This 2 〇 millisecond period will allow it to reach production speeds of 180,000 units per hour. Wide-spectrum image sensors with Bayer filters are often used to obtain color image information. Figure 加 Addition - Bayer filter 12 - Schematic diagram of semiconductor image sensing $ ίο. This m mirror is made up of three different colors.

The light mirror is composed of red 14, 繂耷1 6 |, n A , color 16 and blue 18. Bayer filter 2 is attached to sensor 1 每一 from each color of the Bayer filter 12 and both are opposite to the image sensor 1 〇 image 7L or pixel, one of which The standard is not 20. In this way, each of the 20 elements is integrated with only one color of light, and the resulting image data is self-inducted by a controller (not shown) / 'J. . 1 〇 When reading, this 箄 m 枓 枓 can be classified into three separate images. The image represents the pixels that illuminate only the color. In general, this is used to create full-color images using early-color, wide-band sensors, but at 9 201003038

In the embodiment of the present invention, after this mode, m y L is used to form independent images, and each image corresponds to an independent illumination trace..., β #'. It can make the three images coincide with each other in size, bit depth, and alignment square μ π for further processing. Figure 2 is a graph 30, complex as -, from, t is a graph showing the quantum efficiency 32 of the three types of color filters in a typical Bayer filter versus wavelength 34. . Quantum efficiency means that a photon of a particular wavelength is captured and converted by the sensor into a percentage of charge' which is equivalent to the spectral response. The three color peaks correspond to blue 36, green 38, and red 4 滤 filters. As shown, the blue filter peak is located at approximately 45 nanometers (nm), the green filter peak is at approximately 55 nanometers and the red filter peak is at approximately 625 nanometers. It should be noted that all of the three filters can transmit energy in the infrared (IR) region of the spectrum (approximately 700 nm and above). Most of this type of sensor also includes an IR filter to block additional energy from reaching the sensor. Examples of devices having this type of image sensor include a mobile camera and most low-priced video cameras. The example image sensing system attached to Bayer's Mirror is part of the ICX445 EXview HAD CCD sold by Tokyo Sony Corporation. One embodiment of the present invention uses a technique applied to color image sensing to capture three separate individual images in a frame by a Bayer filter. By illuminating the component from at least two different directions and multiplexing the component, the emphasis is on a single image to reduce image capture to exposure to a single sensor. Each illumination source uses a unique color or wavelength that matches the color of the Bayer filter. In one embodiment of the present invention, the following colors are used to enter the image of the 2010 20103838 line: -470 nm blue - for north side image illumination - 525 nm green - for diffuse image illumination (third image) -636nm Red - for East Side Image Illumination Example LEDs available here include: 470 nm-HLMP-CB30-M0000, Avago Technologies, San Jose, CA; 525 nm-LTST-C190TGKT, Lite-on Semiconductor r, CorP〇ration, Taipei, Taiwan; and SML-LX0402IC-TR, Lumex, Palatine, IL. In a typical Bayer filter, '5% of the pixels are green, 25% are blue and 25% are red. This is because these sensors are designed to produce a color image from a single color sensor that reflects the sensitivity of the human eye to green light to the sensitivity of blue or red light. It should also be noted that this represents only a specific type of color filter. For example, other color filters that can be used to separate colors in a monochrome pupil mirror use color bars aligned with the C pixel rows of the sensor. Other filters can use slightly different colors. Any of these other filters can be used in embodiments of the invention without departing from the basic concepts of the invention. Figures 3a and 3b show an embodiment of the invention. Figure 3a shows a perspective top view of an embodiment and Figure 3b shows a side view of the same embodiment showing an article 50' carried by a transport mechanism 51 through a camera 6 and by a first illumination source 52 from a first The direction projects a first color collimated light 54, and the first illumination source 56 projects a second color collimated light 58. In addition, a third source, source 62 illuminates the article 5 with a third color of diffused light 64. An internal plus 11 201003038 A sensor with a Bayer lens (not shown) is used to capture the image data of the illuminated item 50. Figure 4a shows image data taken from an item 7 that is illuminated from a first direction 74 by a first illumination source 72 of a first-color light. The image data 70 obtained by this illumination mechanism displays a 瑕疵 76 which is displayed as a lighter color region. Figure 4b shows image data 78 taken from an item that is used by a second illumination source 80 of a second color light from a second direction 82... / Thoughts In this case, the above-mentioned exhaustion is invisible in the image data 78. Figure 4C shows synthetic image data 84 obtained by subtracting image data 70 from image data 78. In the synthetic image data 84, the above-mentioned madness % is clearly visible. From the two different directions of the image 7 (), 78 combined with the purpose of the image data = the information appearing in the hidden two images and strengthen the information that only appears in the image. The idea is that the feature to be detected can only be seen in one direction. The features that can be seen in one direction are usually not classified as 瑕苑d# :: Attention' subtraction is only combined with images to strengthen the 瑕疵 bond test: Other operations can be used, including other arithmetic operations, ^ operations such as the smallest/maximum operation, the orderly statistical operation, or the operation of the operation, and the α~ Standard surgery was used to identify the snoring within the synthetic image. Awareness

In the case of squeakiness appearing in two sides A 矣..., in the case of the direction image, it uses the diffuse of the item, and the ϊν is measured by the debt. Figure #姑_ Shows an item using the diffused light above. One of the grayscale images captured. 9 The figure shows that a 92 is circled. This special image data can be processed separately in combination with one or - _ _ 视觉 视觉 视觉 视觉 视觉 , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , The device uses a three-image multiplex processing to use a single sensing material in time. By limiting the time of the acquisition to the single picture:: box: obtaining the information in the middle, and then restarting the unit τ 8 to stop the part, and using the sensing U skin to minimize. In addition, the light is integrated in time to allow it to freeze at a very short time without stopping the part. It can also be used in Qiu, s. The yak is monitored by the sensor and the source is flashed in the form of flash to make ', ,, Ming, ,, good ^ 丨iT < linkage. The part can be moved at its normal speed and passed through the phase: a shutter or a flash light to make a clear picture: in the case of even if the electronic moon/original image is still containing the result of motion, then The movement of the component, and the program is set to a sinusoidal curve: when the component is monitored by the sensor, the image is accelerated when it is decelerated.

The details of the foregoing embodiments may be modified in many ways without departing from the basic principles of the invention, which will be apparent to those skilled in the art. The scope of the invention is therefore intended to be defined by the scope of the following claims. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a schematic view showing a case where an RGB Bayer filter and a light mirror structure are attached to an image sensor. Figure 2 is a graph showing the spectral response of a typical RGB Bayer filter structure. Figures 3a and 3b show a view of a Bayer filter directional image capture system 13 201003038. Figures 4a, 4b and 4c show directional illumination image processing. Figure 5 shows a diffuse illumination image processing. [Main component symbol description] 14

Claims (1)

201003038 VII. Patent application scope: The improved method of cracking the image data of Bayer filter, which is changed to 1. Using the first and second illumination sources, sensors, and controllers to capture the image of the object, including: Having the Bayer filter spectrally matched - the light of the first color illuminates the article with the first illumination source from a first direction; to spectrally match the Bayer filter to a second color of light a second direction illuminating the item with the second illumination source; And capturing, by the image sensor, the image data from the item illuminated by the first illumination and the second illumination to the controller; and separating, by the controller, the image data to include corresponding to the first illumination a first color image and a second color image containing one of the data corresponding to the second illumination; and using the controller to form between the first color image and the second color image to form An image is derived to obtain image data of the item. 2. The method of claim 2, wherein the first direction and the second direction are 180 degrees opposite each other. 3. The method of claim 2, wherein the first color of the Bayer filter is about 55 nanometers. 4. The method of claim 1, wherein the second color of the Bayer filter is about 635. 5. The method of claim 2, wherein the calculation is subtracted between the first color image and the second color image. 15 201003038 and the method of claim 2, wherein the first illumination - the first illumination is printed in the form of a flash. ^ t kinds of use force ° installed image of the optical sensor 11 _ take the item ζ / / improved system, including the addition of Bayer's light mirror image sensing = connected to the image sense when operating One of the controllers, the improvement system now contains: 夯浐',,, and the source of the moon's function to match the Bayer filter to the spectrum: the light of the color - the first direction illuminates the item ; : The second illumination source's function to use the Bayer 遽 first, the brother-second color of light to illuminate the item from a second direction; : The controller acts as a self-installing Bayer illuminator The image sensor 4 μ is like a bedding material, and the image data is separated into a corresponding image to the image illuminated by the first illumination source. The first color image (4) should be sent to the second e-month source. a second color image of the image data of the illumination; and - the controller is further configured to obtain an image of the article by calculating a derived image from the first color image and the color image - 8 The system of claim 7, wherein the first party and the second direction are This reverse of 180 degrees. The system of claim 7, wherein the first color of the Bayer filter brother is about 550 nm. The system of claim 7, wherein the second color of the Bayer lens is about 635 nm. I1. The system described in claim 7 The method of subtracting the calculation between the first color shirt image and the second color image. 16 201003038 12. The system of claim 7, wherein the first illumination and the second illumination are in the form of a flashlight Hit out. Eight, schema: (such as the next page) 17