WO2000038494A2

WO2000038494A2 - Automatic inspection system with stereovision

Info

Publication number: WO2000038494A2
Application number: PCT/US1999/030206
Authority: WO
Inventors: Christopher B. Jackson
Original assignee: Cyberoptics Corporation
Priority date: 1998-12-19
Filing date: 1999-12-17
Publication date: 2000-06-29
Also published as: CA2321096A1; IL137778A0; AU3342500A; WO2000038494A3; JP2003522347A; GB9828109D0; WO2000038494A8; KR20010040998A; EP1057390A2

Abstract

In one embodiment of the present invention, two banks (22, 24) of fixed array cameras provide tiled stereo images (I1, I2, I3) and electronics assemble the tiled stereo images into a mosaic image. The mosaic image includes height information derived from the two sets stereo images, and is also compensated as a function of reference points in the image to more closely conform to the actual board (10). Preferably, a SAM model is used to locate the reference points. Reconstructed SAM models corresponding to components on the board are applied within a search area where a certain type of component is expected to be found. The best-fit location of the component within the search area, as well as the skew angle and the probability are computed. The best-fit location is the location where the Manhabolis distance and the residual error are optimized. Once the best-fit location is computed, it is compared to tolerances to identify improperly placed components, the absence of components or the like.

Description

AUTOMATIC INSPECTION SYSTEM WITH STEREOVISION

FIELD OF INVENTION This invention relates to inspection equipment and to methods of inspection for articles of manufacture, with particular emphasis on the area of assembling components onto a printed circuit board, as in electronic assembly.

BACKGROUND OF THE INVENTION The invention relates to inspection of articles of manufacture such as printed circuit boards. This is a main and extremely important application of the invention and will be mainly used for explaining the invention, but as will be indicated hereinafter the invention has wider application than the inspection of printed circuit boards. In the manufacture of printed circuit boards a rigid sheet of a synthetic composition is used as a substrate, and onto this substrate are provided copper conductors in a pattern established by a conventional photoresist and etching process.

By silk screening, a solder paste is applied to the copper conductors at locations where electrical components such as capacitors and resistors are to be applied, and where processor chips and the like, which usually have multiple terminals to be connected to the conductors, are to be applied. The components and terminals stick to the paste, and then the board and the applied components and chips are passed into an oven where the solder paste forms a secure solder connection with the component and chip terminals. These boards have two sides and the process is repeated in respect of the second side. These boards can be extremely complicated and small, and any one board may have a vast number of components and consequently a vast number of electrical connections. Printed circuit boards are now produced in large quantities, and as they are expensive and are used in expensive equipment, it is important that they be produced accurately, with minimum wastage. Unfortunately, because of the manufacturing methods available, wastage (because of rejects), is still higher than in other industries. Typical faults on printed circuit boards comprise inaccuracy of placement of components on the board, which might mean that the components are not correctly electrically connected in the board, and or electrical connections are not made, or that there is insufficient paste deposits leading to poor connections or too much paste leading to short circuits, and so on. The industry experiences so much difficulty in providing reliable production that extensive inspection is necessary to check the accuracy of manufacture of these boards. The current inspection equipment is either too expensive, or too slow, or too inaccurate, or suffers from combinations of these disadvantages. There is therefore a huge demand for accurate inspection equipment, and if such equipment is faster, and/or less expensive than existing equipment, so much the better.

As to the equipment that is available and the methods used for inspection of printed circuit boards, the most basic method is a manual one. Inspecting operators simply used their vision to detect the faults indicated above. To assist, the operator has a mask which he or she puts over the board and examines the board, section by section, to check that the components are correctly positioned and are electrically connected, and there are no electrical short cuts, Manual inspection, in fact, can be extremely accurate, but the trouble with it is that operators tend to be accurate at the start of a shift, but as time goes by, their accuracy drops. Also, it is a very slow method, and is labor intensive, which is expensive.

Another method is to scan the board with a telecentric camera, that is to say a camera with telecentric lenses, which views in parallel beam imaging, to photograph a section of the board, and use software to analyze the image. The camera and board are relatively movable so that any required section of the board can be viewed. In one case, the camera is mounted on a stationary gantry, and the board can be moved past the camera. In another case, the board is stationary and the camera is movable.

In using a telecentric camera, problems due to parallax can be overcome, but there is a disadvantage with this arrangement in that the camera can only view the image in two directions, which is not a problem as long as the board is exactly flat, but although normally these boards are nominally flat, exact flatness is difficult, if not impossible to achieve for various reasons. Steps are taken to keep the board flat to the degree necessary to make the method accurate, but this is not easy, and in fact the board is sometimes clamped in an effort to keep it sufficiently flat to make the results acceptable. It should be borne in mind at this time that the dimensions of the degree of out of flatness (i.e. warpage) of the board which are involved are measured in microns, but even warpage of the board to this minute degree can have a serious effect on the accuracy of the inspection.

Previous automated methods have also suffered from the inability of gray scale models to accurately match a component under test. Such models were unable to compensate for variations in shape, colour, lighting and the like, which resulted in numerous identifications of properly placed components as having been improperly placed (i.e. "false calls").

Another disadvantage of the above method is that the equipment is expensive and involves an expensive, precision X-Y gantry. This method also tends to be slow, and requires precision mechanical registration of each section of the board, which is viewed by the camera. In the present invention we have provided inspection equipment which gives greater accuracy than the known methods, provides a low "false call" rate and does not require the use of expensive equipment.

SUMMARY OF THE INVENTION

In one embodiment of the present invention there is provided inspection equipment comprising means creating at least two viewing beams, which view the surface or object to be inspected. The beams are being arranged to view the surface or object at different angles, to view one common virtual or actual reference point. By this means, the distance of that point from a reference point can be computed. Also, by this means a photo mosaic of a large area can be created, which has particular advantage as applied to the viewing of a printed circuit board.

The said viewing beams are preferably conical in nature, such as are produced by conventional CCTV cameras. By this means the cost of the equipment can be kept to a minimum. By viewing the surface at two different angles and providing a reference point, a distance or height coordinate, as well as x and y coordinates can be obtained, which in the case of a printed circuit board, gives an indication the profile of the board, which although nominally flat, in fact rarely is because for example, during the heating in the oven, as a result of thermal expansion, the board usually warps slightly, and becomes other than perfectly flat, even although the warping may not be visible to the naked eye. This warpage may also be caused because the board is of such low mechanical strength that it will sag, or simply as a result of manufacturing tolerances. If the imaging is repeated and sufficient reference points are provided or are present, then by sequential imaging a complete picture of the board surface profile can be built up. As the board profile affects the inspection accuracy (hence the reason for the sophisticated clamping in the prior art), if the board profile is known, the equipment can take it into account, for example in the software processing, to provide a more accurate inspection result.

In the equipment of the invention this profiling is fed into the processing devices of the equipment. The improved results are obtained without the use of expensive devices to clamp the board, or an expensive gantry (which also clamps the board), or the use of telecentric cameras, which are more expensive than the cameras which can be used in the invention.

The beams may be provided by pairs of low grade CCTV cameras of pixel resolution in the order of 760 x 575, arranged at the appropriate angle, or they may be provided from a single higher-grade camera of pixel resolution in the order of 1024 x 1024 and a beam splitting optical arrangement. In the latter case, a single camera takes the two overlapping images, which enhances registration. It so happens that printed circuit boards have what are termed "fiducial" points which are reference points on the board, and also apertures called "vias" through which electrical connections can be made through the board (after the inspection). The fiducial points, vias and the like can be used as the reference points for building the overall profile shape of the board. When this profile is taken into account the positional accuracy of the components and chips on the board is enhanced during the inspection process.

In one form of the invention, the board to be inspected is carried on a pair of conveyor belts of simple construction, past a bank of cameras arranged in pairs above the conveyor belts to image the board at different angles. Typically the cameras view the board along different axes arranged at an angle on the order of 3 degrees to each other. The conveyor belts convey the board past the cameras in step-by-step fashion. At each step, each camera takes an image of part of the board, and the images from the cameras of respective pairs overlap and register the same reference point or points, so that the images in the electronic scanning of same can be "stitched" together or placed in exact registration. The same can be done with the images of side-by-side camera pairs. The result is that the complete image of the board is built up, and that is compared to a model image of what the board should look like, and as long as the components are positioned, compared with the model, within limits the inspection will show an acceptable product. The particular advantage of the invention is that in addition to taking account, for example, of linear deviation of the reference points as a result of linear distortion of the board compared to the model, the equipment also, by the optical technique, takes into account the height deviation of the points on the surface of the board, providing enhanced results. For example, the conventional method using telecentric cameras might not detect a height distortion, and in consequence reject the board, whereas the present invention, in taking into account the detection of a height distortion, may well pass the board. The present invention preferably employs a statistical model of a component or of a reference point, called a SAM model. The SAM model incorporates variability in colour, shape, lighting and the like of the component and its immediate vicinity. The first application of SAM models is preferably done in the stitching process, where SAM models of reference points aid in precisely locating the reference points to accurately stitch the mosaic image together. The second application of SAM models is preferably during inspection of a printed circuit board, where a search area for a specific component is defined and a SAM model corresponding to the component which is expected to be found within the search area is applied to points within the search area. At each point within the search area, the SAM model is reconstructed to take into account the specific variations of that portion of the search area, and the reconstructed SAM model applied to each of the points within the search area. At each point within the search area, a measure of fit is computed, and the point at which the measure of fit is optimized is used as the best-fit point representative of the actual location of the component on the board.

It can be seen therefore that the invention has enormous potential in the particular field of printed circuit board inspection. It is to be mentioned however, that the invention has other uses in and outside of the printed circuit board industry. One example in the printed circuit board industry is that the inventive equipment can be used for calculating the volume of solder or solder paste on a printed circuit board, by detecting height of the solder or solder paste pads on the board surface, or pre or post solder application inspection.

The invention is of particular novelty in the stereovision inspection of surfaces which are nominally flat, but in practice deviate from complete flatness.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiment of the present invention will now be described by way of example, with reference to the accompanying drawings, wherein:

Fig. 1 A shows a section of printed circuit board, which is to be inspected by the equipment according to the invention; Fig. IB is an enlarged view showing how vertical distortion of the board also leads to lateral displacement of component;

Fig. 2 is a diagrammatic side elevation of equipment according to a first embodiment of the invention;

Fig. 3 is an enlarged side view showing the optical system of the embodiment of the invention shown in Fig. 2;

Fig. 4 is an enlarged perspective view showing the optical effect which applies when the printed circuit board is distorted; Fig. 5 shows the spacing of the images of a reference point as seen by the two cameras in Fig 4;

Fig. 6 is a view similar to Fig. 2 showing an alternative embodiment of the present invention; Fig. 7 is a flow chart of the method of the invention;

Fig. 8 is a schematic representation of a SAM model; and

Fig. 9 is an overall block diagram of the system of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED

EMBODIMENT Referring to Fig.l, a printed circuit board to be inspected is indicated by reference numeral 10, and it is shown in this example as having thereon a processing chip 12, components 14 and printed circuit conductor wires 16. It is to be pointed out that base components may be extremely small, and very tightly packed on the board. It is usual to have such items attached to both sides of the board. The objective of the present invention is to provide an inspection means for the board whereby the correct positioning of the various items on the board can be checked. This is done by scanning by means of closed circuit television cameras as will be explained.

Referring momentarily to Figs. 2 and 3, the equipment for performing the scanning is shown diagramatically in these figures, and comprises a pair of conveyor belts 18 and 20 which are spaced by a distance to enable the board 10 to be supported therebetween. The spacing between the conveyors 18 and 20 can be adjusted to accommodate boards of different sizes.

The inspection cameras are located vertically above the board 10, and they are arranged in pairs such as are indicated by reference numerals 22 and 24 in Fig. 2. There is a bank of camera pairs A, B, C and D and so on arranged in a direction transverse to the direction, indicated by arrow 26 in which the board is transported by the conveyors 18 and 20. The conveyors are arranged to operate in a stepping fashion so that the board 10 steps past the fields of view of the cameras so as to be photographed progressively in strips which lie in the direction of arrow 26, and are arranged in parallel and side by side in a direction at right angles to direction 26 and indicated by arrow 28 in Fig. 3. Thereby, the cameras are arranged to photograph all of the board, and the photograph of the board can be reconstructed on a display screen 30 of electronic computing equipment 32 to which the outputs of the cameras are directed.

Pre-loaded into the computing equipment 32, is a model of the printed circuit board so that the computing equipment can compare what is viewed by the cameras, and the model details, to indicate whether or not the board is of satisfactory manufactured quality or has to be rejected. A comparison will be mainly to ensure that the items on the board are correctly and exactly positioned, but the comparison can also check items such excess solder or shortage of solder, which faults respectively could mean short-circuiting or imperfect electrical connection. The process is detailed below.

Reverting to Fig. 1 A, the rectangular and overlapping areas II and 12 respectively represent the images as seen by the cameras 22 and 24 at the first step in the inspection process. Fig. 2 illustrates that these images are generated by divergent beams 36 and 38 of which the beams axes lie at an angle X to one another. Such angle may be in the order of 3 degrees, but the net effect is that the cameras 22 and 24 look at the board in a stereovision manner and by arranging degrees that the images II and 12 overlap, accurate re-creation of a mosaic image of the board on the screen 30 can be achieved by "stitching" the images II and 12 when they are processed electronically, but retaining the stereo nature of the information in the mosaic image. This is done by pre-programming the computing equipment 32 with information concerning reference points such as the vias 40 and 42 which exist on the board 10 and relating their position to the fiducials, such as 44, which are also on the board 10. If it is considered that the images II and 12 overlap lengthwise of the board, the images from the side-by-side cameras also overlap sidewise as shown by image 13, and be stitched in this direction also, so that a complete mosaic picture of the board 10 can be built up by the electronics 32 and a comparison with the model input 34 can be made accurately. Any arrangement of cameras in lengthwise and sidewise directions may be adopted to provide the stitching of image facility.

By stereoscopically viewing the board using two cameras 22 and 24 in each pair, compensation can be made for any outer flatness (i.e. warpage) of board 10. Such outer flatness or distortion arises for the reasons given herein, and by way of explanation, Fig. 4 shows the board 10 in its actual distorted shape, whereas reference 10A indicates the optimum flat configuration of the board 1 (which rarely exists in practice).

If one now examines the beam paths 36 and 38, and considers, for example, the via reference point 40 on the board 10, it would be seen that in actual fact the point 40 is displaced downwards from where the computing system would expect such point to be located (in the plane 10A).

In Fig. 5, the outputs from the cameras 22 and 24 therefore show two images 40A and 40B as being displaced one relative to the other in that the cameras 22 and 24 would be looking for the images 40A and 40B to be in the plane 10 if the board 10 were at the correct distance from the cameras.

This spacing, S, between the images 40 A and 40B is calculated which in turn gives an indication of the extent of displacement of the actual via 40, and so a height compensation factor can be provided. If this height compensation factor is not included in the comparison, then spurious results can be provided and this is demonstrated by reference to Fig. IB. Fig. IB shows an enlarged elevation at position 50 where the board ideally would be expected to be, and a component is shown at 52. The component has a width 54 and the inspection electronics would be looking for component 52 to be in the position shown and to exhibit the width 54. However, if the board is distorted as shown at 56, the component 52 will in fact not only be deflected downwards, but will also be displaced laterally by distance D, and if the electronics does not compensate the board profile as a result of the distortion, what the electronics will see in looking at position 4 will only be part of the component 52 and it may conclude that component 4 is therefore "out of position". When the height or warpage compensation and profile shape, however, is taken into account, the electronics will calculate that there has been distortion of the board downwards and lateral movement of the component 52 and therefore will not reject component 52 but rather will accept it in position 58. A tile-by-tile, piecewise linear fit is preferred, but other methods are acceptable for use with the present invention.

The stereoscopic inspection of the board; therefore, provides improved performance of the equipment, without requiring expensive devices as are employed in the known methods.

In the embodiment described, pairs of relatively inexpensive and relatively poor resolution CCTV cameras are used, and there is no need to make any attempt to mechanically flatten the board during inspection. A typical camera resolution is 760 x 575 pixels. In another embodiment again there is no need to mechanically flatten the board, but a single camera can be used in place of each pair, the single camera being of a higher resolution quality but arranging to have its beam split to provide the two stereo images at each step. Such an arrangement is illustrated in Fig. 6. The method of operation is otherwise similar to what has already been described.

In Fig. 6, the high resolution camera 60 has a viewing beam 62 which impinges upon a beam splitter prism 64 which splits the beam into two identical but oppositely directed beams 66 and 68. These stereo beams 66 and 68, respectively, impinge upon mirrors 70 and 72, resulting in the provision of incident stereo beams 74 and 76 which view the board 10 optically in the identical manner as do the beams 36 and 38. The advantage of this arrangement is that both beams 74 and 76 are generated by the same camera, and the registration of the stereo image tiles and the processing of the information are slightly similar. It has been mentioned that the images are stitched together by viewing reference points on the board. These reference points are real, but the system can also be made to work with virtual reference points provided, for example, by spots of light, and Fig. 6 shows one possible arrangement wherein pencil reference beams 78 and 80 travel through the same optical system as the camera beam but are set to impinge upon a common spot 82 to form a reference point. If the board 10 is distorted or warped as described in relation to Fig. 4, the viewing of that reference spot will produce two images in a manner similar to that shown in Fig. 5. The present invention provides equipment and method enabling the accurate high speed inspection of surfaces and objects, such as printed circuit boards, without the need for adopting expensive gantry XY devices, or telecentric cameras or expensive mounting device for clamping the board flat.

The invention of course has wider application as indicated herein, and in one example stereo viewing can be used for viewing other spots to provide an indication of volume of the solder in that spot. Indeed, the concept of viewing image regions II and 12 and relating these to reference points such as 40 and 42 followed by the stitching of the images to provide an accurate representation constitutes a novel aspect, even if the viewing beams are arranged in parallel as long as they diverge and overlap.

The present invention is also able to be practiced with electromagnetic radiation of varying wavelengths. A x-ray source would replace the cameras and appropriate x-ray receivers would be employed to record an image of the article which is being viewed. Additional hardware in the x-ray embodiment would perform the same functions as disclosed herein. In fact, it is apparent that even a line scan image of the article, where an image of the article is built up line by line while the article and a linear detector moves relative to each other, is contemplated under the method and apparatus of the present invention. In a line scan embodiment of the present invention, a series of collected outputs from a linear detector would be necessary to provide a single image of the article, and another series of collected outputs from the linear detector would be necessary to construct the mosaic image. The following discussion refers to Fig. 7 and provides a more detailed understanding of how the statistical appearance modeling process is used with the mosaic image in the preferred embodiment of the present invention. Precise and accurate measurement of component position relies on establishing the shape of the surface (substrate) upon which the component is mounted which is required to accurately take account of the path length distance between points on a curved surface and to overcome the errors arising from the use on non- telecentric optics in the imaging device in the preferred embodiment.

At box 302, the tiled images are acquired by the camera pairs. At box 304, the positions of reference points visible in both images of the stereo pairs, are established. The discrepancy in location between the measured positions of reference points in each of the stereo images, and measurements describing the positions of all the cameras in the system obtained during a calibration process, allows the system to establish the distance between the reference points and the cameras, which imaged the reference points. This distance is used to establish the height of the reference point above a reference plane, which is established during system calibration. Due to limitations in processing power and the need for the system to work within the cycle time of the manufacturing process producing the articles, it is necessary to restrict the number of points at which this height measurement is made, and so a sparsely populated height map of the entire surface of the substrate of the article is created. In order to produce a complete description of the three- dimensional shape of the substrate it is necessary to interpolate between the points in the height map. Different mathematical models identified below can be imposed upon the height map data.

1. "Sag" where the substrate is assumed to have drooped between the rails of the conveyor system. The height of the surface of the substrate is defined by: z(x, y) = ax² + bx + c so that the surface of the substrate is exclusively quadratic with respect to x. The parameters {a, b, c} are determined from n measured reference points (X_J, yj, Zj) (1 < i < n) in a least squares fashion, which fits this model to the height map.

2. "Thickness/tilt" where the substrate is thick enough to be stiff and therefore relatively flat and the effect to compensate for is due only to the thickness and tilt of a plane, given by: ax + by + cz + d = 0 The parameters {a, b, c, d} are, again, determined from the measured reference points (XJ, y„ Zj) (1 < i < n) in a least squares fashion, which fits this model to the height map. 3. "Warp" where the substrate is considered to be warped in many directions simultaneously. Here, the (XJ, y_t, z,) (1 < i < n) measurements of the reference points are used to construct an interpolating surface known as a "thin plate spline". The thin plate spline uses a model of pliable material which is "bent" to match exactly the heights Zj at the point (XJ, y;) and does so such that the amount of conceptual energy required to bend the plate is minimized.

The result is a continuous surface, z=f(x,y), which interpolates the measured reference points, (i.e. Zj=f(xj, yj)) is fitted to the height map. The reference points must appear in the row overlaps defined by the image acquisition process, or the row overlaps must be set from the positions of available reference points. The location of the reference points can be determined in a number of ways but employing a SAM model of the reference points is preferred. One way is to use an example image, where a user defines the coordinates of a suitable candidate reference points. A second way is to use design information for the article (e.g. CAD, Gerber) which defines the position of suitable reference points. A third way is to analyze the image of the article using alternative image processing and analysis algorithms (e.g. Hough Transform) to determine the positions of objects of particular characteristic shapes. It is advantageous that a 'golden' article is not required to define the position of the reference points because it is sometimes not possible to obtain a perfect article. However, should design information be available for the article, greater precision and accuracy can gained by comparing the actual positions of the reference points with their expected positions.

The stereo overlapping tiles are stitched together to form a mosaic image where it is necessary to move the article or the imaging device array, in order to capture image data for larger articles. An image registration process takes place simultaneously to the height mapping process described above. In this way, a mosaic image of many rows of stereo pairs of image tiles is built up to form a height and movement corrected mosaic image. The entire process by which the mosaic image is produced is called "stitching". Having established a rigorous mathematical image of the topology of the substrate, and having accounted for the errors introduced into the image acquisition process by virtue of imprecise mechanics, accurate measurements of location of certain features (e.g., components) can now be made.

Components mounted on the substrate of the article are located using statistical appearance models, as shown in box 308. The use of statistical appearance modeling (SAM) results in a very reliable assessment of component presence and a very accurate and repeatable assessment of component position. The objective is to measure component position with respect to the coordinate system of the article, typically based on the commonly found fiducials marks on the printed circuit board. The inspection procedure involves first detecting and measuring the position of these fiducials and then producing a coordinate system from their measured positions (box 306). The detection and position measurement of the fiducials is achieved using SAM, although other image analysis methods are within the scope of the present invention. The substrate suffers from errors resulting from the positioning of the article under the imaging device array and the normal tolerances associated with the manufacturing process (these errors result in distortions in the printed/etched pattern on the substrate). Having now established a compensated coordinate system of the article in the mosaic image at box 306, all components which are to be inspected are detected and their positions measured at boxes 308-312. For each kind of component there exists a corresponding SAM model and a list of acceptance criteria that must be satisfied in order for an inspection for a component to be deemed to be passed. The criteria include, but are not limited to, tolerance in x, y, angle of skew and a measure of how likely the component in question is described by its associated SAM model, expressed as a probability of fit.

The measurement of the position of the fiducials and the components must take account of their height in relation to the reference plane, so that their position on the substrate is accurately established. In order for a SAM model to be used to detect and measure the position of a component, the model corresponding to the component or feature is applied in the vicinity of where the component or feature is to be expected at a range of angles in the mosaic image, as indicated in box 308. The location process, at box 310, evaluates the correspondence between the SAM model and the feature at all points within a defined search area and establishes the best-fit points (one from each stereovision set of information in the mosaic image) at which the SAM best describes the data in the search area. This process returns a best-fit x,y coordinate, an angle of skew and the probability that the SAM model has properly described the component. The discrepancy between the x, y coordinates of the best-fit points allows the distance of the surface of the component from the imaging devices to be computed, which as in the stitching process allows a height measurement to be computed with respect to the reference plane. This height measurement is then used to compute a height compensated coordinate of the projection of the position of the surface of the component onto the substrate at box 312. In box 314, the corrected x, y, skew angle and probability measures are then tested against the acceptance criteria for this style of component, and the inspection for this component passes if these measures are within the acceptance criteria specified. However, if the inspection indicates that the component is outside of the acceptance criteria, then the board can be either discarded, scheduled for re-work or appropriate warning given to an operator (box 316).

In another embodiment of the invention, the compensated location for a component can be computed from the best-fit location obtained from one of the stereo images in which the component appears, in combination with a height estimate of the component obtained from CAD and other component design information.

An overall block diagram of the system is shown in Fig. 8. A printed circuit board 402 rests on a conveyor belt 400. Conveyor belt 400 is actuated by motor and motor drive 404, which operates under instructions from a computer 406. Computer 406 is multi-processor computer of standard design and includes circuits for acquiring and digitizing images from two banks of video cameras 408, 410 and a man machine interface 412 consisting of keyboard, mouse and screen. Computer 406 controls the movement of the board 402 with a precision of +/- 0.5 mm, in such a way to position the board for acquisition of the images. Computer 406 also directs acquisition of partial images of the board 402 from banks 408,410. An illuminator 414 provides lighting for image acquisition. The SAM model 500 shown in Fig. 9 allows proper account to be taken of legitimate variability in shape, color, lighting, surface patterns and the like. This results in a reliable determination as to whether the feature is present, absent, of the correct type or an accurate assessment as to where it is placed on the article. SAM model 500 describes the value of every point of intensity (pixel) in an image of an object, it also describes how each point of intensity can vary in value with respect to all other points of intensity. There are two distinct stages in the use of SAM. The first involves the construction of a SAM model by analysis of images of a variety of features of the same type, as shown at 502 - 508, where it is clear that any number of examples may be used in creating a SAM model. The second involves using the SAM model to detect and locate the feature it describes, in an image where such a feature is detected, but which may or may not be present, as detailed above with respect to fiducial marks and to components.

A SAM model is constructed by collecting together example images of the feature of interest, pre-processing the image data from each example and turning each into a one-dimensional vector of pixel values. Given all the vectors, one for each example, a mean vector x_mean is produced representing the average appearance of the feature, as given in Equation 1 :

Xme X -l

N ^

where X_j is an nxl vector of pixel values, n is the number of pixels in each vector and N is the number of example images.

Each example can now be represented as a zero centered vector, x'j expressing its variance from the mean, called a mean adjusted vector, given by Equation 2:

X j ⁼ — Xmean A compact description of how each example varies from the mean can now be generated by constructing a co-variance matrix, S, using all the mean adjusted vectors from the previous stage, as stated in Equation 3:

where x' is an nxl matrix and x',^τ is an lxn matrix, so that the product of the two matrices is an nxn symmetric matrix.

The eigensystem of the co- variance matrix, S, is given by Equation 4;

Sp_k = P_k l ≤ k ≤ n where λ_k is the k eigen value of the co-variance matrix, p are the orthogonal eigenvectors and where orthonormality is defined by P T

The eigensystem yields an orthogonal system of eigenvectors which represent the particular ways, called modes of variation, in which the pixels of images of the feature vary in shape, colour, lighting, surface patterns and the like, the eigenvalues representing the magnitude of each of the modes of variation. Only the most significant modes of variation are retained, so as to reduce the affects of random noise, which exists in the image data of each example. Only the more significant modes of variation, which explain typically 95% of the variability in the examples, are used, as shown in Equation 5:

P = [P ~P_n] where P is an nxm matrix, P^τ is an mxn matrix, and PP^T=1. A SAM model 500 has now been constructed which allows the reconstruction of any example, seen or unseen, whose appearance (pixel intensity or gray values) lies within the bounds dictated by the magnitudes of the modes of variation. Next, the SAM model is used to reconstruct itself to more closely conform to the component under test as detailed at box 308 in Fig. 7. A newly reconstructed model is computed at a range of angles for each point in the vicinity of the component under test (i.e. the search area). The best-fit reconstructed model occurs at the best-fit location. Equation 6 shows the general form of reconstruction:

Solving Equation 6 for b, the parameter vector, yields Equation 7: b = P^J( ^x -x mean s

Once the b vector values are computed, an overall quality of fit measure of how well the reconstructed SAM model fits the component under test is computed, as a function of the Manhanobolis distance and residual error. In particular, the Mahanobilis distance provides a normalized measure of how well the SAM model describes the candidate and is given by Equation 8:

J mαnhαl / , « 7=1 ^Λj where m is the number of example images used to construct the SAM model. The other measure of quality of fit evaluates the. residual error between the component under test and the reconstructed SAM model, as given by Equation 9: n 2 resici _ι 3 -t ^Λr.j where n is the number of pixels in the model, rj is the error of the j^th pixel between the relevant search area pixels and the reconstructed approximation of the SAM model, and λ_{r j} is the variance of η over the examples stored previously. The overall "quality of fit" is derived in Equation 10:

J fit J mαnhob J resid

The value f _f,_t is transformed into a "probability of fit" representative of the probability that the model describes the component under test by assuming that the f _f,_t values from a population of examples follow a chi- squared distribution. The probability of fit value, P_{f ,} is preferably computed using the incomplete gamma function, given by:

Pf= G ( ffit )

The use of SAM models allows a very reliable decision to be made as to whether or not a feature is present in an image, as the reconstruction can smoothly interpolate between all the examples used in constructing the model to produce an appearance of the feature which, although not seen before as an example, lias been varied from the mean. The newly generated appearance of the feature, however, is consistent with the variability captured during the model construction phase. Also, the best-fit location returned from the location process is very accurate and repeatable because the variability in the feature is properly described by the model.

When making a comparison between a feature and a corresponding SAM model there will be two sources of error; one will be caused by the difference between the feature and the model, the other will be caused by random noise resulting from the image acquisition process and other factors. The location process and the SAM model which the system uses remove the first source of error leaving only the second source of error, which being at a much lower level, results in improvements in measurement performance and detection reliability over systems which do not employ this method.

Although the present invention has been described with reference to preferred embodiments, workers skilled in the art will recognize that changes can be made in form and detail without departing from the spirit and scope of the invention. For instance, insubstantial changes in the logical flow of the steps of the present invention which realize the substantially the same function will be recognized as within the scope of the present invention. Additionally, the optics discussed for use in the present invention should not be used to limit the invention, and it will be recognized that other schemes from which a height map of a surface of interest may be computed, and the location of features of interest on the surface compensated by the height are within the scope of the present invention. Image analysis methods other than the ones presented in this disclosure are also within the scope of the present invention, as long as they are able to be reconstructed to more accurately comport with an image under test. Finally, the present invention is not limited to use in the area of electronics assembly inspection machines, but may be used in other inspection and manufacturing systems, which must accurately identify the presence or absence of a certain item on a surface with variations in its planarity.

Claims

WHAT IS CLAIMED IS:

1. A method of assessing a quality of an article, the method comprising: collecting two stereo images of the article; applying a statistical model corresponding to a feature on the article to the two stereo images to provide a pair of best-fit locations; computing a height of the feature with respect to a reference as a function of the pair of best fit locations to form a compensated location; and comparing the compensated location to an acceptance criterion on an expected location of the feature.

2. The method of claim 1 , where the two stereo images are collected by two fixed array cameras, each camera positioned to view the article from an angle.

3. The method of claim 1, where the two stereo images are collected by two banks of cameras, each bank of cameras positioned to view a portion of the article from a different angle.

4. The method of claims 1 or 2, where the angle is substantially three degrees.

5. The method according to claim 4, where each bank of cameras provides a set of partial images of the article, and each set of partial images are stitched together to form a mosaic image.

6. The method of claim 5 where the mosaic includes at least one fiducial and the method further comprises the step of computing a corrected coordinate system for the article from a location of the fiducial.

7. The method according to claim 1, where the step of computing a height includes computing a height map of the article.

8. The method of claim 6 where the height map is sparsely populated.

9. The method of claim 2 where the height map is mathematically computed according to an equation selected from the group of equations called sag, thickness/fit and warp.

10. The method of claim 1 where the statistical model is a SAM model.

11. The method of claim 1 where the SAM model models the variability in appearance of the feature.

12. The method of claim 10 where the model is representative of a feature on the article, and the step of applying the statistical model includes the step of translating and rotating the model in the vicinity of an expected location of the feature.

13. The method of claim 10 where application of the SAM model results in an improved repeatability and accuracy of measurement.

14. The method of claim 1 where the article is a printed circuit board.

15. The method of claim 1 where the tolerance comprises one of a measure of position, skew angle and a probability of fit.

16. The method of claim 1 where the step of computing a height includes an estimate of the height at an expected location.

17. The method of claim 1 where the reference is a reference plane.

18. The method of claim 17 where the reference plane is substantially coincident with a plane of the article.

19. The method of assessing a quality of an article, the article having a reference flatness, the method comprising: collecting two stereo images of the article and constructing a height map of a plurality of features on the article; computing a height of a feature with respect to the reference flatness; compensating a location of the feature with the height to provide a compensated location; and comparing the compensated location to an ideal tolerance.

20. A system for assessing a quality of an article, the system comprising: two cameras and associated electronics providing a pair of stereo images, each camera positioned to view a portion of the article from an angle; a processor computing a height of a feature on the article, the height a function of the pair of images, the processor compensating a location of the feature with the height to provide a measure of the quality of the article.

21. The system of claim 20, where the system further compromises at least one additional pair of cameras and associated electronics, the pair positioned so as to view another portion of the article and to provide an additional pair of stereo images, where the processor stitches the pair and additional pairs of stereo images together to form a mosaic image.

22. The system according to claim 20, where the mosaic image covers an area substantially corresponding to the article.

23. The system according to claim 20, further compromising a conveyor system for moving the article into a viewing area, and the cameras are positioned so as to view the article within the viewing area.

24. The system according to claim 20 further comprising a conveyor system for moving the article, where the conveyor system moves the article in a controlled fashion while the pair of images are acquired.

25. The system according to claim 20, where the system further comprises electronics for correcting a coordinate system corresponding to each of the pair of images.

26. The system according to claim 20, where a plurality of pairs of stereo images are stitched together into a mosaic, the mosaic further compromising reference points.

27. The system according to claim 26, where the reference points are selected from the group of reference points comprising an image of a light spot, fiducial marks and vias.

28. A system for inspecting articles of manufacture, the system comprising: a camera for acquiring an image of the article; a processor for processing the image, the processor comprising: means for detecting fiducial marks to provide a registered image; means for applying a SAM model representative of a feature on the article to the registered image to locate a best-fit coordinate of the feature; means for compensating the best-fit coordinate for a deviation of the feature from a reference flatness measure to provide a compensated image; and means for comparing the compensated image of the feature with acceptance criteria corresponding to the expected location of the feature.

29. The system of claim 28 where the camera is a single fixed-array camera.

30. The system of claim 28 where the camera is a line scan camera.

31. The system of claim 28 further comprising an additional camera for acquiring a second image of the article, where the processor creates a stereovision image from the image and the second image, and the processor operates on the stereovision image.

32. A process for inspecting articles of manufacture, the system comprising: acquiring an image of the article with a camera; processing the image, such step including: detecting fiducial marks to provide a registered image; applying a SAM model representative of a feature on the article to the registered image to locate a best-fit coordinate of the feature; compensating the best-fit coordinate for a deviation of the feature from a reference flatness measure to provide a compensated image; and comparing the compensated image with acceptance criteria corresponding to the expected location of the feature.

33. The system of claim 32 where the camera is a single fixed-array camera.

34. The system of claim 32 where the camera is a line scan camera.

35. The system of claim 32 further comprising an additional camera for acquiring a second image of the article, where the step of processing includes creating a stereovision image from the image and the second image, and the processor operates on the stereovision image.