GB2440951A

GB2440951A - Edge detection for checking component position on a circuit board

Info

Publication number: GB2440951A
Application number: GB0616167A
Authority: GB
Inventors: Richard Evans; James Mahon; Gareth Bradshaw; Iain Lennox
Original assignee: MV Res Ltd
Current assignee: MV Res Ltd
Priority date: 2006-08-15
Filing date: 2006-08-15
Publication date: 2008-02-20
Also published as: GB0616167D0

Abstract

An automated optical inspection (AOI) system is used to determine the position of a surface mount technology (SMT) component on a circuit board. Colour images, e.g. RGB, are captured of the component on the circuit board and an image processor analyses regions of interest (ROI), namely an ROI which is expected to include only pixels of the board, and an ROI which is expected to include only pixels of the component. The system performs training on pixels representing the component and the board, and uses this training to detect an edge of the component. This may be done by providing a 3D histogram for the component colour and a separate 3D histogram for the board colour. The edge detection can be used to check the correct positioning of components on the board even when there is low contrast between the component and the board.

Description

"Inspection of Components in SMT"

Introduction

The invention relates to inspection of discrete items such as electronic components on a substrate such as a circuit board.

Quality control on SMT (surface mount technology) lines can be performed by an automated optical inspection machine (AOl). An important function of SMT post reflow quality control is the ability to measure the positional placement error of components on the circuit board. AOl inspection of printed circuit boards works by using a lighting arrangement to illuminate a board and then analysing the resulting images.

Existing AOl machines locate the component by finding edges in i single image plane. While this is simple and fast, it suffers from the disadvantage that clutter on the board such as screen print can provide a stronger edge than that of the component and thus cause mislocations. It is possible to reduce the impact of clutter by taking multiple images (e.g. RGB planes from a colour image) and combining them to produce a single image using a formula that is either fixed or programmed by a user, such as a Hue transform. For some situations, for example where the component and background are both dark, the performance of such a transform can degrade suddenly due to subtle changes in appearance of board and component due to manufacturjng process variability.

The invention is therefore directed towards providing an improved method and system for AOl inspection of discrete items on substrates such as SMT components on a board.

Summary of the Invention

According to the invention, there is provided a method of machine vision inspection of a discrete item on a substrate with low item/substrate contrast to detect edges of the item, the method being carried out by a machine vision inspection system comprising a camera for capturing images of a scene and an image processor, the method comprising the steps of: capturing an image of a scene including the item and surrounding substrate; generating substrate training data for pixels expected to correspond to the substrate; generating item training data for pixels expected to correspond to the item; for at least some pixels of the captured image, analysing the pixel according to the training data to determine if it represents the substrate or the item, and determining an edge of the item according to the analysis.

In one embodiment, a set of plurality of colour images of the scene is captured and processed.

In one embodiment, the training data compriss a histogram.

In one embodiment, a set of plurality of colour images of the scene is captured and processed; and the histogram is multi-dimensional, with a dimension for each image plane of the colour images.

In another embodiment, the training data is generated for each captured image or set of images of a scene.

In one embodiment, the expected locations of the item and substrate are indicated by pre-set regions of interest.

In a further embodiment, there is user interactivity for selecting a region of interest.

In one embodiment, each histogram is generated by incrementing counters according to pixel values of the images.

In one embodiment, each pixel grey level is quantised before incrementing a relevant counter.

In one embodiment, during the analysis step the edge of the item is determined by transforming image pixels to provide a transformed image having improved contrast between the item and the substrate.

In another embodiment, the transformation applies a transformed value to each of the item and the substrate pixels in the transformed image, said transformed value being based on the likelihood that the pixel represents the item or the substrate In one embodiment, the training data comprises a histogram for the substrate and a histogram for the item, and the transforming step comprises determining a pixel value for the transformed image according to comparison between the pixel value and a counter value from each histogram.

In one embodiment, the method is repeated a plurality of times for a single image or set of images with progression of expected locations of the item and substrate.

In a further embodiment, the said repetition takes place if it is expected that the item is offset to a large extent.

In one embodiment, the method comprises the further steps of storing historic training data and subsequently using said historic training data for scenes of a similar type.

In one embodiment, the item is an SMT diode and the substrate is a circuit board.

In one embodiment, the item is a flip-clip and the substrate is a circuit board.

In one embodiment, the item is a ball grid array and the substrate is a circuit board.

In another aspect, the invention provides a machine vision inspection system comprising a camera for capturing images of a scene and an image processor, wherein, the camera captures an image of a scene including an item and a surrounding substrate, and wherein the image processor: generates substrate training data for pixels expected to correspond to the substrate; generates item training data for pixels expected to correspond to the item; for at least some pixels of the captured image, analyses the pixel according to the training data to determine if it represents the substrate or the item, and determines an edge of the item according to the analysis.

In another aspect, the invention provides a computer readable medium comprising software code for implementing image processor steps of any method defined above when executing on a digital processor.

Detailed DescrjDtjon of the Invention The invention will be more clearly understood from the following description of some embodiments thereof, given by way of example only with reference to the accompanying drawings in which:-Fig. 1 is a flow diagram illustrating an AOl inspection method of the invention; Fig. 2 shows an original image of the method on the left, and a transformed image of the method on the right; and Figs. 3 to 11 are a sequence of screen shots including images to illustrate implementations of the method in different use case scenarios.

An AOl system operates in a method 1 of Fig. 1 to accurately determine position of an SMT component on a board.

Colour images 2, in this embodiment RGB, are captured of the scene and an image processor of the system retrieves from storage pre-set regions of interest (ROl) 3, namely: an ROl which is expected to include only pixels of the board, and an ROI which is expected to include only pixels of the component.

Pixels within the two ROIs are extracted (4) to provide extracted pixels 5.

The system performs training 6 to provide a 3D histogram 7 for the component colour and a separate 3D histogram 8 for the board colour. In each 3D histogram 7 and 8 there is a dimension for each colour of the captured colour images 2.

In a step 9 the pixels 5 are transformed into a transformed image which has excellent component/substrate contrast. Fig. 2 shows a sample source image on the left and a corresponding transformed image on the right.

S Tuini

The training step 6 uses N dimensional histograms (or Look-up Tables (LUT)) with each colour plane corresponding to a different dimension. A given colour combination e.g. combination of particular RGB values maps to a particular bin in the histogram.

Fig. I shows the use of three colour planes but even better results can be obtained using four colour planes by mixing not just images obtained using lighting at different wavelengths but also different lighting angles.

In more detail, the training step 6 operates as follows: The input is a list of extracted pixels 5 with a pixel intensity for each image plane for each pixel location within the image. The histogram is built up by the system as follows: If training the board histogram 8, it updates count values in the board histogram 8.

If training the body histogram 7, it updates count values in the body histogram 7.

For each pixel location in the region to be trained, the pixel intensities in each image plane (red, green and blue) are combined to decide which counter is to be incremented.

For a 16x16x16 histogram (with 4096 counters) the first step performed by the image processor is to quantise each grey level into a number between 0 and 15.

For example, say there are the intensities red = 128, green = 23, blue 34 -.7- (where grey levels are in the range 0 -255) it divides by 16 to get the coordinates of the counter [8,1,2]. To convert these 3D coordinates to the number of the counter it gets 8*256 + * 16+ 2*1 = 2066. So it then adds I to the value in counter number 2066.

Transformation 9 Once the histograms 7 and 8 have been generated, the image processor then in step 9 performs a transformation by analysing each pixel of the colour images 2 according to the histograms. The entire region surrounding the component is transformed. For each pixel location in the image the two corresponding counts from the board and body histograms are retrieved and combined to form a pixel intensity. For example, if the body count is very much higher than the board count then the pixel will have an intensity of 255. Each pixel within the search region of the component is used to produce a transformed image that indicates which pixels are part of the component body and which are part of the background (board, silkscreen etc.). The pair of histograms 7 and 8 are queried to produce a count that indicates the likelihood that the given pixel is part of the class represented by that histogram (i.e. component or not).

These two counts are then combined to produce a confidence that that point is part of the body. For example if the pixel scores a very high count for being part of the body and a very low count for being part of the board then the transformed image pixel will be very bright (grey scale of 255) indicating a high confidence in the pixel being part of the body.

For each pixel position to be transformed the image processor calculates the number of the counter as for the training 6 from the set of grey level intensities. Then it reads the value from the counter in the board histogram (a) as well as the body histogram (b). The corresponding pixel value in the transformed image is then set to 256*b/(b+a) or 255 whichever is less. If both a and b are zero it sets the pixel value to zero.

The effect of the transformation can be seen in Fig. 2 where the image on the left is an original 3-plane coJour image of a component on a circuit board, and the image on the right is the corresponding transformed image. It will be apparent that the original image has low contrast against the background, whereas the transformed image has significantly improved contrast.

Examole I The component is a small diode having a low background contrast. The following steps are implemented as seen from the user's perspective.

* Set the diode size.

* Multiply the body sizes x2 to get appropriate search area sizes, but ensure the search area does not enclose any neighbouring parts of similar colour to the device being programmed.

* Go to a location pane, shown in Fig. 3.

* Set Location mode to "Low contrast box filter" and press inspect, as shown in Fig. 3 * The system generates a transformed image, shown in Fig. 4. This can be made visible by setting a "Display Transform" option to "On" * The contrast may be improved for small parts (<3000 jim) by setting "Colour contrast" to "Small flat parts", as shown in Fig. 5. If the top surface of the part is not planar then the "Standard" option may give better results. The "Small flat parts" option consumes Sl2KBytes of memory for each device for which it is used, while the "Standard" mode consumes 32Kbytes. The "Small flat parts" setting switches on the use of a 4 dimensional histogram using 4 colour planes.

* More accurate skew measurements can be obtained for small parts like this by using a "Rotate 90" option. The user sets this parameter to "Both", and then presses "Inspect", as shown in Fig. 6.

In implementation of the method I there is an automatic switch to "Historic" mode which gives improved location accuracy for heavily of.fet components and uses a blue location box, as shown in Fig. 7. This mode utilises training from previous inspections with the same device type on the current board. The training is initialised for each device type by its first inspection after a board program is loaded. If in}Iistoric" mode the colour of the part or board changes significantly for some reason, then the method is automatically carried out a second time in order to learn the colour change.

This could, for example occur due to heavily textured parts or significant tilt variations.

If components are heavily offset, this can result in the component body falling within the region interest which is expected to contain board, and vice versa. This may cause learning of inappropriate colours and reduces the contrast of the transformed image, giving degraded location accuracy or even complete mislocation.

The "historic" mode gets around this problem as follows: * Before the first component is inspected the histograms contain no information (all counts are zero). During inspection the histograms are trained from the current image.

* In historic mode the histogram counts are saved for future use instead of being discarded.

* For all subsequent inspections when the component is inspected, the transform is carried out using the previously learned histogram counts.

* After the component is correctly located, the regions of interest are aligned with the actual component location and the histogram counts are supplemented with new learning from the current component. To prevent the counts increasing without limit, before the training is supplemented the existing count values are reduced by 10%. Thus the histograms contain a rolling average of the most recent component inspections.

Example 2

In this example the component is a small flip clip. Flip chips can have extremely low body/board contrast and can also have a heavy textured appearance depending on surface finish. Small flip chips (e.g. l000x 1000 microns) are among the most difficult devices to locate.

The following are the steps, as seen from a user's perspective: * Set the body sizes.

* Go to the location pane * Set Location mode to "Low contrast box filter" and press inspect, as shown in Fig. 8.

* The part is programmed as before but slightly better skew results can be obtained on square parts with an edge % of 15%, as shown in Fig. 9.

Example 3

In this example the component is a ball grid array (BGA). The steps are as follows: * Set the body sizes.

* On a large device the search area is less critical than for a small part. However it is larger to ensure the search area does not enclose neighbouring parts of similar colour to the device being programmed.

* Go to the location pane * Set Location mode to "Low contrast box filter", and press inspect as shown in Fig. 10.

* On a large part "Color Contrast" being set to "Standard" should give results that are most robust.

* For tall parts with tight placement tolerances it may be necessary to enable "Parallax Correction". The new transform tends to highlight the top surface (and not the sides) of the part more so than with previously available colour transforms. On tall components near the edge of the field of view this can result in offset errors due to parallax. To enable, set "Parallax Correction" to -11 - "On". This will make a new Body height parameter appear. The height of the part off the board in microns is entered and "Inspect" is pressed. The location results are adjusted depending on the position in view of the component.

In summary, the method identifies the statistical pattern of intensities from multiple images of the scene from learning regions. These correspond to the body of the component and also to the board on which the component is placed. It then analyses these patterns to infer for each pixel in the scene the probability that the pixel is part of the component. This probability is converted to a pixel intensity within a transformed image which then shows a bright component against a dark background.

The techniques used to cope with even more heavily offset components are very advantageous. They maintain training data from inspections of the corresponding component on previously inspected boards. The counts in these histograms form a rolling average of counts obtained. This is achieved by reducing all histogram counts (e.g by 10%) prior to the training of each component. This data is self refreshing so that it can cope with changes in board or component colours, but will allow a reasonably accurate location of even a very heavily offset component. A second pass using the previously described standard technique can then be performed to obtain a more accurate location result.

It will be appreciated that the invention provides for improved component edge location where conventional techniques do not apply. Examples of such situations are where the component is low contrast against the background either due to similarity in colour or low lighting levels.

The invention is not limited to the embodiments described but may be varied in construction and detail. For example, the items may be components of different types than described.

Claims

Claims I. A method of machine vision inspection of a discrete item on a

substrate with low item/substrate contrast to detect edges of the item, the method being carried out by a machine vision inspection system comprising a camera for capturing images of a scene and an image processor, the method comprising the steps of: capturing an image of a scene including the item and surrounding substrate; generating substrate training data for pixels expected to correspond to the substrate; generating item training data for pixels expected to correspond to the item; for at least some pixels of the captured image, analysing the pixel according to the training data to determine if it represents the substrate or the item, and determining an edge of the item according to the analysis.

2. A method as claimed in claim 1, wherein a set of plurality of colour images of the scene is captured and processed.

3. A method as claimed in claims I or 2, wherein the training data comprises a histogram.

4. A method as claimed in claim 3, wherein a set of plurality of colour images of the scene is captured and processed; and wherein the histogram is multi-dimensional, with a dimension for each image plane of the colour images.

5. A method as claimed in any preceding claim, wherein the training data is generated for each captured image or set of images of a scene.

6. A method as claimed in any preceding claim, wherein the expected locations of the item and substrate are indicated by pre-set regions of interest.

7. A method as claimed in claim 6, wherein there is user interactivity for selecting a region of interest.

8. A method as claimed in any of claims 3 to 7, wherein each histogram is generated by incrementing counters according to pixel values of the images.

9. A method as claimed in claim 8 wherein each pixel grey level is quantised before incrementing a relevant counter.

10. A method as claimed in any preceding claim, wherein during the analysis step the edge of the item is determined by transforming image pixels to provide a transformed image having improved contrast between the item and the substrate.

11. A method as claimed in claim 10, wherein the transformation applies a transformed value to each of the item and the substrate pixels in the transformed image, said transformed value being based on the likelihood that the pixel represents the item or the substrate.

12. A method as claimed in claims 10 or Ii, wherein the training data comprises a histogram for the substrate and a histogram for the item, and the transforming step comprises determining a pixel value for the transformed image according to comparison between the pixel value and a counter value from each histogram. -14-

13. A method as claimed in any preceding claim, wherein the method is repeated a plurality of times for a single image or set of images with progression of expected locations of the item and substrate.

14. A method as claimed in claim 13. wherein the said repetition takes place if it is expected that the item is offset to a large extent.

15. A method as claimed in any preceding claim, comprising the further steps of storing historic training data and subsequently using said historic training data for scenes of a similar type.

16. A method as claimed in any preceding claim, wherein the item is an SMT diode and the substrate is a circuit board.

17. A method as claimed in any of claims I to 15, wherein the item is a flip-clip and the substrate is a circuit board.

18. A method as claimed in any of claims I to 15, wherein the item is a ball grid array and the substrate is a circuit board.

19. A machine vision inspection system comprising a camera for capturing images of a scene and an image processor, wherein, the camera captures an image of a scene including an item and a surrounding substrate, and wherein the image processor: generates substrate training data for pixels expected to correspond to the substrate; generates item training data for pixels expected to correspond to the item; for at least some pixels of the captured image, analyses the pixel according to the training data to determine if it represents the substrate or the item, and determines an edge of the item according to the analysis.

20. A system as claimed in claim 19, wherein the camera captures a set of plurality of colour images of the scene and the image processor processes said set of images.

21. A system as claimed in claims 19 or 20, wherein the training data comprises a histogram.

22. A system as claimed in claim 21, wherein the histogram is multi-dimensional, with a dimension for each image plane of the colour images.

23. A system as claimed in any of claims 19 to 22, wherein the image processor generates training data for each captured image or set of images of a scene.

24. A system as claimed in any of claims 19 to 23, wherein the expected locations of the item and substrate are indicated by pre-set regions of interest, and the system comprises a database storing said regions of interest.

25. A computer readable medium comprising software code for implementing image processor steps of a method of any of claims I to 19 when executing on a digital processor.