JP2003506801A - Object inspection apparatus and method - Google Patents

Abstract

The present invention discloses an object inspection system and method, the method comprising: a reference image for a representative object, wherein the first boundary is at least partially vectorized in the image. A reference image including a representation of the second boundary, obtaining an image of the test object including a representation of the second boundary in the image, and combining the representation of the second boundary with the at least partially vectorized first image. Identifying the defect by comparing it with a representation of the boundary. A method and system for image processing in a software environment, wherein the method provides a combination of a user-defined region and a defect trigger received from a hardware processor to a software-based image processor, each of which is dynamically selected. Also disclosed are methods and systems that include inspecting each of the regions surrounding the user-defined region and the hardware defect trigger with the provided inspection algorithm.

Description

Detailed Description of the Invention

[0001] [Field of the Invention]   The present invention relates to an apparatus and method for image processing.

BACKGROUND OF THE INVENTION Devices and methods for analyzing images, particularly devices and methods for image processing and image analysis, useful for inspecting patterned objects are known in the art. is there.

The following documents describe image processing methods that may be useful in understanding the present invention. C. Gonzalez and P.G. Wintz, “Digital Image Processing (Digi
tal Image Processing) ", Addison Wesle
y, Reading, MA, 1987; John C. et al. Russ, “Image Processing Handbook (The Image P
Processing Handbook), CRC Publishing, 1994. The following documents describe edge detection methods that may be useful in understanding the present invention. D. Marr and E. Children, “Theory of Edge Detection (Theor
y of Edge Detection, The Royal Society of London Bulletin (Proc
needs of the Royal Society of London
on); Chapron, "new color edge detector to be used for color image segmentation", the 11th APR International Conference on Pattern Recognition ⁽¹¹ th APR Intern
national Conference on Pattern Recogn
edition). The following references describe color image segmentation methods that may be useful in understanding the present invention. Philip Pujas and Marie-Jose Aldon, "
Robust Color Image Segm
entation) ", 7th advanced robotics International Conference ⁽⁷ th Inter
national Conference on Advanced Robo
tics), San Filiu de Guixols, Spain, 1995.
22 Sept; Leila Sararenko, Maria Petrou, and Jo
Sef Kittler, “Automatic Watershed Segmentation o Randomly Textured Color Image
f Randomy Textured Color Images, IE
EE.

Lapidott US Pat. No. 4,758,888 describes a method and means for inspecting products moving across a manufacturing line, including online defect inspection without any progression or product interruption on the manufacturing line. Have been described.

US Pat. No. 5,586,058 and US Pat. No. 5,619,429 (A
loni et al.) examine binary-level object representations, inspect gray-level object representations, and preferably re-examine grayscale object representations to eliminate false alarms, and An apparatus and method for inspecting patterned objects and detecting defects, including classifying defects, is described.

US Pat. No. 5,774,572 (Caspi) associates a filter function with a second derivative of a Gaussian function to superimpose a two-dimensional digital grayscale image of an object and has signed values. An automated visual inspection system is described that functions to form a two-dimensional superimposed image. Localization of edges within the object is achieved by finding zero-crossings between oppositely signed adjacent values.

US Pat. No. 5,774,573 (Caspi et al.) Uses a convolution of a two-dimensional digital grayscale image of an object with a filter function associated with a second derivative of the Gaussian function to sign it. A visual inspection system for forming a two-dimensional superimposed image having a value of is described. The superimposition of Caspi or the like can be performed by using the difference between two Gaussian values, positive and negative.

[0008] PCT application IL98 / 00393 describes color-based inspection of printed circuit boards, including color-based identification of certain types of conditions, such as color-based conductor oxidation. .

[0009] PCT application IL98 / 00477 allows morphological operations such as dilation, erosion and scale measurements on non-binary pixels to be performed, preferably more efficiently than with prior methods. Describes a method for analytically representing an image.

Applicant's unpublished Israeli patent application No. 125929 describes an article inspection system and method, including an improved image acquisition system. The following publications describe methods that may be useful in image analysis.

Dorin Comaniciu and Peter Meer, “Distr
ibution Free Decomposition of Multiv
Associate Date ”, SPR'98 Request Recommendation (Invited Sub)
mission, Rutgers University Department of Electrical and Computer Engineering (Piscatawa)
y, NJ08855, USA).

The disclosures of the above-referenced publications and all publications mentioned herein, as well as the publications cited in this specification, are hereby incorporated by reference.

SUMMARY OF THE INVENTION The present invention seeks to provide improved apparatus and methods for image processing.

The present invention is described in Applicant's Israeli patent application IL131092, entitled “Optical Inspection System,” filed July 25, 1999, the disclosure of which is incorporated herein by reference. It is useful when used in combination with the existing optical inspection system.

Inputs to the image processing system presented and described herein typically include at least one of the following items: a. Optionally, a snapshot of a target portion or area that includes some of a user predetermined area, an area surrounding a suspicious defect, and an area surrounding a predetermined type of feature. b. Binary cell data (sometimes referred to herein as "cell data") that is data that identifies contour elements corresponding to boundaries between regions in a pattern. c. Color cell data, that is, data that identifies the contour elements that correspond to the boundaries between each area in the pattern and identifies some type of area on either side of the cell. d. Morphological feature inspection trigger. e. Color defect inspection trigger. f. A binary defect inspection trigger (sometimes referred to herein as a "hardware defect") that is a cell on the skeleton that is located on a nick, protrusion, and morphological skeleton. f. The area of the target inspection trigger that the user identifies.

Preferably, these inputs are the applicant's Israeli patent application IL13109
2 is provided by an image processing unit substantially as described in 2.

Accordingly, provided by a preferred embodiment of the present invention is a method of inspecting an object, the method comprising a reference image of a representative object within the range of the image. Generating a reference image including a representation of a first boundary that is at least partially vectorized with, obtaining an image of an object under examination that includes a representation of a second boundary within the image, And comparing the representation of the second boundary with the representation of the at least partially vectorized first boundary, thereby identifying defects.

Further in accordance with a preferred embodiment of the present invention the step of comparing comprises a user-selected variable of an acceptable distance between corresponding portions of boundaries in the first representation and the second representation. Adopt a threshold.

Further provided by another preferred embodiment of the present invention is a system for image processing, a boundary identification device having the capability of generating a representation of the boundaries of known elements in an image, A hardware defect candidate identification device having a function of identifying a defect candidate in the image by hardware, and an output from the hardware defect candidate identification device by software, and at least one false alarm in the output. A software defect candidate inspector that uses the boundary representation to identify the. The identification of the at least one false warning is preferably provided by comparing the boundary with a reference.

Further, according to a preferred embodiment of the present invention, the boundary identification device comprises a hardware boundary identification device having a function of generating a representation of a boundary of a known element in the image by hardware.

Furthermore, according to a preferred embodiment of the present invention, the system further comprises a software defect candidate identification device having a function of identifying additional defect candidates in the image by software.

Further, according to a preferred embodiment of the present invention, the software defect candidate inspector receives, by software, a second output from the software defect candidate identification device, and at least one of the second outputs is included in the second output. The boundary representation is used to identify false alarms.

Furthermore, according to a preferred embodiment of the present invention, the hardware defect candidate identification device employs the representation of the boundary to identify at least some defect candidates.

Furthermore, according to a preferred embodiment of the present invention, the software defect candidate identification device employs the representation of the boundary to identify at least some defect candidates.

Further provided by another preferred embodiment of the present invention is a system for image processing, the learning sub having the function of defining first and second regions in an object under examination. A system, wherein the first region comprises at least one known critical object element and the second region does not comprise such a known critical object element; Using a first procedure based on prior knowledge of known critical object elements, said first
Defect detector having a function of inspecting the second region and inspecting the second region using a second procedure different from the first procedure.

Further, according to a preferred embodiment of the present invention, the second procedure includes a hardware inspection of a second area having a function of identifying a defect candidate in the second area, and a second area. Only if at least one defect candidate is found in
A continuous software inspection of said second area is provided, which has the function of analyzing at least one defect candidate found in said second area and of identifying false alarms therein.

Further provided in accordance with another preferred embodiment of the present invention is a method for inspecting an object, wherein in a first inspecting step, the location of at least one defect candidate is identified, And, for each defect candidate, determining a candidate region for inspection of at least size and shape of the contour region at the location is based at least in part on the output of the identifying step, and Inspecting each of the candidate regions to identify the defects in an inspecting step.

Furthermore, according to a preferred embodiment of the present invention, the first step is implemented in hardware.

Furthermore, according to a preferred embodiment of the present invention, the second step is implemented in software.

Furthermore, according to a preferred embodiment of the present invention, the step of determining comprises determining the size of the candidate region at a known location.

Furthermore, according to a preferred embodiment of the present invention, the second step comprises performing different inspections depending on the criteria of the candidate area or the defect candidate.

Further, according to a preferred embodiment of the present invention, the second step includes performing different inspections according to characteristics of at least the one defect candidate identified in the identifying step.

Furthermore, according to a preferred embodiment of the present invention, the second step comprises performing different inspections depending on the functionality of the part of the object in which the defect candidate is present.

Furthermore, according to a preferred embodiment of the present invention, the second step includes performing different inspections depending on the degree of criticality of functionality of the portion of the object in which the defect candidate exists.

Further provided by another preferred embodiment of the present invention is a modular image processing system with user customizable image analysis functions for use with a scanner, the system comprising: An image processing engine for receiving at least one image data stream to be analyzed from the image processing engine, and a function for receiving a sequence of at least one user-customized image analysis function definition executed by the image processing engine on the image data. An engine configurator having, wherein the image processing engine depends on a current definition arriving at the engine configurator from among the sequence of definitions,
The ability to analyze at least one channel of image data according to their respective definitions in the sequence of said definitions sent to said engine configurator, comprising performing different image analysis functions which differ from each other only by parameters. Have.

Further provided by another preferred embodiment of the present invention is a method for automatically optically inspecting an object, the first step comprising: Of target areas defined by the user and including at least one target area automatically defined by optically inspecting the article, and each Providing an image processor with an image of a region surrounding the region of interest, and in a second step automatically determining the image of each of the regions surrounding the region of interest to determine the presence of defects in the article. Processing.

Furthermore, according to a preferred embodiment of the present invention, each image of the area surrounding the target area is smaller than the image of the target object.

Further, according to a preferred embodiment of the present invention, the target area automatically determined by optically inspecting the article is a candidate for a defect in a pattern formed on the object. Including.

Further, according to a preferred embodiment of the present invention, the target area automatically determined by optically inspecting the article has a predetermined shape formed in a pattern on the object. Inclusive characteristics are included.

Also according to a preferred embodiment of the present invention, the providing step comprises providing the image processor with parsed information relating to boundaries of the region of interest. Preferably, the providing step comprises providing a color image of the region of interest to the image processor.

Furthermore, according to a preferred embodiment of the present invention, the step of automatically processing comprises automatically defining an object in an object area defined by a user by optically inspecting the article. Applying at least one image processing method different from the image processing method applied to the region.

Further in accordance with a preferred embodiment of the present invention said providing step comprises identifying a type of defect in the area of interest, said automatic processing step being suitable for the type of defect in the area of interest. Applying the image processing method described above.

Further in accordance with a preferred embodiment of the present invention said providing step comprises identifying a type of morphological feature in said region of interest, said step of automatically processing said in said region of interest. Applying an image processing method suitable for the type of morphological feature.

Also according to a preferred embodiment of the present invention, at least one user defined area of interest is defined prior to optically inspecting said object in a software defining step, wherein said article The at least one region of interest automatically defined by optically inspecting the image is performed in a hardware inspection step, and automatic processing of each image of the region of interest is performed in a software image processing step. To be done.

Further provided by another preferred embodiment of the present invention is a method for inspecting a ball grid array substrate, the method comprising at least one feature in a reference image of the ball grid array substrate. Generating at least one model for storing the model in memory, obtaining an image of the ball grid array substrate, and determining if features within a predetermined area fit the model. To determine, including inspecting a predetermined area of the image of the ball grid array substrate.

[0046]   Furthermore, according to a preferred embodiment of the present invention, the features include circles.

Furthermore, according to a preferred embodiment of the present invention, the model of the circle includes a center point at a predetermined position and a radius within a predefined tolerance.

Furthermore, according to a preferred embodiment of the present invention, the parameters of the model are at least partially adjustable in an offline mode prior to inspection.

Also in accordance with a preferred embodiment of the present invention the features include bonding pads.

The invention will be understood by reading the detailed description with reference to the drawings described below.

Detailed Description of the Preferred Embodiments The following specification sets forth extensively BG for the purpose of illustrating the invention.
Although reference is made to A (ball grid array substrate), it will be readily understood that the present invention is applicable to the inspection of any patterned object. Therefore, the term BGA used below is not limited to ball grid array substrates only, but rather is considered to be exemplary, including printed circuit boards, laminated printed circuit boards, lead frames, flat panel displays, hybrids. Of any suitable form, such as chip package substrates, self-tapping substrates, and any other suitable patterned objects including various eroded and engraved metal substrates that may be used, for example, in medical implants. It is considered to include patterned objects.

In the drawings, squares represent functional units and circular blocks represent data processed by the functional units, unless otherwise specified.

Reference is now made to FIG. 1, which is a schematic block diagram of an image processing system constructed and operative in accordance with a preferred embodiment of the present invention.

In this embodiment, the scanner 2 preferably optically scans the object 4 to produce an RGB (red, green and blue) two-dimensional image representing the object 4,
The B image information is transferred to the image processor 5. Preferably, a target spatial area discriminator 6 is provided, the image processor 5, the target spatial area discriminator 6, and the line width and positioning processor 3 being provided with additional inspection triggers and others for the SIP unit 7 located downstream. The data of is output. The SIP unit 7 is inspected by an additional inspection function not provided by the image processor 5, preferably by each different type of inspection trigger, the area associated with the trigger is inspected by an inspection routine most suitable for inspecting the triggered area. As described above, it performs a test function that dynamically adapts to the test trigger input.

The scanner 2 uses “Illumination for Inspecting”.
An image illumination and acquisition system such as that described in Applicant's copending application filed May 5, 2000 under the name Surfaces of Objects (illumination for inspecting the surface of an object). Preferably. The scanner 2 preferably has the function of acquiring RGB optical images,
Scanners enclose other types of images, such as monochrome gray level images, HIS (Hue, Saturation and Intensity) type images, or printed circuit boards that define certain electrical properties associated with electrical circuits on the printed circuit board. It is readily appreciated that it may function to acquire an image that is associated with non-optical properties of the object, such as an electric field. The present invention is
It is understood that it may be associated with image processing of any suitable optical or non-optical image.

According to a preferred embodiment of the present invention, the image processor 5 preferably receives the image information data, decomposes the image information data into basic image data and triggers based on the detection of suspicious defects and certain features. Are provided for further inspection in the downstream inspection unit. The image processor 5 is preferably a multi-module processing unit comprising one or more of the modules described below.

(I) Binary cell identification device 8 modules. The binary cell is a contour element that represents an edge boundary line between adjacent areas of the object 4 in which a large reflection intensity gradient exists between the dark area and the bright area in the monochrome image. Binary cells are referred to as "cells" or "binary cells" in the following description. The binary cell identification device 8 includes the scanner 2
Receives and processes the image data from and outputs a report of the binary cells present in the image of object 4.

(Ii) Color cell identification device 10. The color cell represents an edge contour between adjacent regions or between different color groups in the pattern on the surface of the object 4. A color population is a region of substantially uniform color, preferably associated with a substance. Color cells are generated for a color group selected from a number of predetermined color groups to identify which color group is on each side of the edge. The color cell identification device 10 preferably processes the image data received from the scanner 2 to identify each color group in the image data and to identify the known color group within the object. And outputs a report of the color cells existing in the image of the object 4 that defines the regions of the respective substances and the boundaries between the regions.

(Iii) Morphological feature identification device 11. Morphological features are predetermined features that can be identified in a pattern. The morphological feature identification device 11 preferably has one or more information or data of image data from the scanner 2, binary cell information from the binary cell identification device 8 and color cell information from the color cell identification device 10. It has the function of receiving input from. The morphological feature identification device 11 processes these inputs and outputs a morphological feature report that triggers further inspection in the downstream inspection unit.
Preferably, at least some morphological characteristics are associated with the substance within the object 4. In the context of BGA testing, morphological features include, for example, contact pads, open ends, shorts, junctions, islands, and other similar features that an operator may select.

(Iv) Defect identification device 13. Defects are suspicious defects of a given type that can be identified in a pattern. The defect identifying device 13 includes a scanner 2 and a binary cell identifying device 8.
And a function of receiving input of one or more image data from the color cell identification device 10. The defect identification device 13 processes these inputs and outputs a report of suspected defects that triggers additional inspections in downstream inspection equipment. Preferably,
Suspicious defects reported by the defect identification device 13 are defects that are identified based on a set of predetermined design rules and are identified without comparing the input data with a reference. In the context of BGA inspection, suspicious defects include, for example, irregularities that exceed a predetermined parameter for smoothness in the contours defining the conductor, low-contrast color surface defects such as oxidation, residual chemicals, and high-contrast surface defects such as scratches. Can be mentioned.

(V) Optional snap generator 14. If a snap generator 14 is present, it preferably has the ability to generate a snapshot image of a selected area within the image in response to the feature and defect triggers received from the morphological feature identifier 11 and the defect identifier 13. Have. The snap image is the portion of the image received from the scanner 2 that is preferably generated as a “target window” surrounding each inspection trigger received from the morphological feature identifier 11 and the defect identifier 13. Further, the snap generator 14 can generate a snap image for the target spatial region according to a trigger received from the target spatial region identification device 6, which will be described in more detail below.

Preferably, the shape and dimensions of the snap images output by snap generator 14 are dynamically configured in response to a trigger by a particular type of region of interest. The snap generator 14 preferably has a function of outputting image data that is easily converted into a moving image in downstream processing. A preferred hardware implementation of an image processor suitable for use as the image processor 5 is described in Applicant's co-pending patent application IL131092. It will be readily understood that the above description is centered around the preferred image processing functions present in image processor 5. However, the functions of the image processor 5 may actually be classified into several modules, for example, binary processing, color processing and morphological processing, which issue functional reports classified as described above.

The reports can be categorized into various reporting channels depending on the hardware architecture. Therefore, in the preferred embodiment of the present invention, the image processor 5 produces the reports provided to the SIP 7 on two channels.
In this case, each reporting channel contains a number of reports. Preferably, the first channel, referred to as the "snap channel", reports defect data relating to color surface defects by the defect identification device 13 and the snap generator 14.
And a report containing a snap image of the target area according to. The second channel, called the "cell" channel, reports the binary cells by the binary cell identification device 8 and the color cells by the color cell identification device 10, the morphological features by the morphological feature identification device 11, and the defect identification. And a report called a defect report that reports binary defects such as nicks and protrusions by the device 13.

According to a preferred embodiment of the invention, an object space area identification device 6 is provided for defining a continuously occurring object space area within the inspected object. The spatial region of interest is the region present in the object 4 in which a particular type of feature is predicted to be present, and a predetermined inspection routine tailored to that particular type of feature is typically performed.

Preferably, the target spatial area generator 6 operates in an offline “learn” mode, which will be described in more detail below, prior to defect detection. In the preferred embodiment, the target spatial area identification device 6 includes two modules, a user-defined target area module 28 and a computer-generated target area module 9.

Preferably, in learning mode, the general user interface described in more detail with respect to FIGS.
Via the interface, which is part of 8, the user defines a schematic area to be inspected, which is located on the object 4 to be inspected in one lot of objects. The user preferably also specifies the type of structure expected to be located within the region of interest. Thus, in the context of BGA inspection, a user can define a general area including a bond pad, target, chip area or ball.

A computer processor within the computer-generated target area module 9, which may include a portion of the image analyzer 24 of SIP 7 in an alternative embodiment, includes a target within a user-defined target area module 28 in a user-defined spatial area. "Learning" the position of the structure defined by the user as the structure. The computer-generated target area module 9 generates a report that identifies a local area that closely surrounds each structure in the space defined by the user. Thus, in the context of a BGA test, when a user defines a general area that includes a bond pad, the computer-generated target area module "learns" the structure and position of each bond pad within the user-defined area. , The computer generated target area module for each location where the position of the bonding pad is supposed to be specified,
Define a local area of interest. The local object space area defined around each structure in the user defined area is preferably SIP7 (for SIP) for subsequent use during inspection.
Stored in the memory associated with (not shown).

According to a preferred embodiment of the present invention, the SIP 7 is provided downstream of the image processor and has the function of receiving image data from the image processor 5 and executing inspection routines not provided by the image processor 5. . For example, SI
P7 has in part the function of providing a comparative mode inspection that compares the analyzed image data from the object 4 being inspected with reference data for that object. Further, the SIP, in part, according to a predetermined rule relating to the spatial region according to the stored information from the target spatial region identification device 6, an inspection routine, a type of feature detected through image processing by the image processor 5, and It has the function of dynamically adapting to the type of defects detected through image processing by the image processor 5.

Preferably, SIP7 is a software-based image processor unit that executes customized software image processing routines on a dedicated workstation, such as a microcomputer workstation operating in a LINUX ^™ environment, the (i) line. The width and positioning processor module 3 and (i
i) task packer and target window generator module 12, and (ii
i) includes an analyzer module 15.

According to a preferred embodiment of the present invention, the line width and positioning processor 3 has two
The binary cell information from the value cell identification device 8 and the morphological feature data from the morphological feature identification device 11 are received, and the input information is processed in the following modules.
That is, (i) the conversion calculator 20 performs positioning conversion to position the image of the object 4 to be inspected with respect to the reference image, and the excess / defective calculator 22 determines the presence or absence of features. From the positioning generator module 18, which has the function of determining whether all expected features and only those expected features are present in the inspected object 4, and (ii) the conversion calculator 20. Receives the positioning conversion data, calculates the line width for the conductor, confirms that the line width at each specific position is within the allowable range for that position, and the space between the conductors is a predetermined general value. A line width calculator 16 that has the function of confirming that it is smaller.

A preferred positioning generator module 18 and its respective conversion calculator module 20 and excess / missing calculator module 22.
Will be described in more detail below.

According to a preferred embodiment of the present invention, the SIP7 target task packer / target window generator unit 12 preferably comprises the following modules:
Inputs from each of the value cell discriminator 8, the color cell discriminator 10, the morphological feature discriminator 11, the defect discriminator 13, the conversion calculator 20, the surplus defect calculator 22 and the line width calculator 16 are received. Further, the task packer 12 receives the target space area defined by the target space area identifying device 6 in the learning mode and then stored in the memory.

The task packer 12 has a function of generating a target window surrounding various inspection triggers. The inspection trigger is preferably a predetermined type of feature received from the morphological feature identification device 11, a predetermined type of defect received from the defect identification device 13, and a predetermined target spatial region defined by the target spatial region identification device 6. , And the predetermined line width defects and surplus / defective defects received from the line width and positioning processor 3 and stored in memory. In addition, each inspection trigger has a snap generator 1
A snapshot of the area surrounding the trigger from 4 may be provided. The shape and dimensions of the snap image depend on the trigger type and / or the
Alternatively, it is preferably generated dynamically according to the position. Also, the shape and dimensions of the snap image may be generally standard for all inspection triggers.

For each test trigger, the task packer 12 preferably identifies the appropriate test task that is dynamically selected to perform the particular test task in response to the particular type of test trigger received, and Generate a package containing the appropriate image information data needed to perform the selected task. For example, in the situation of BGA inspection,
The package for the bonding area defined as the spatial area of interest preferably includes an identification of the appropriate bonding area inspection task, along with a complete collection of binary cells, color cells and morphological feature data for that spatial area. . The package for areas identified as having color surface defects preferably includes an identification of the color surface defect inspection task and a snap image of the affected area, but no cell information. The package for areas identified as having nick / projection defects preferably includes identification of a suitable nick / projection inspection method, binary cells of the affected area, and further includes a snap image. Is also good.

The task packer 12 typically transmits to the analyzer unit 15 a dynamic processing package containing image and method information, represented by the large arrows seen in FIG. The dynamic information is preferably transmitted only to the analyzer 15 for a particular defined image area based on the defect, feature and area of interest information received by the task packer 12. This "pre-processing" significantly reduces the amount of image data analyzed by the analyzer unit 15, thereby reducing the computational load and ultimately reducing the inspection time cost related to the analysis of the object. become.

According to the preferred embodiment of the present invention, the analyzer unit 15 preferably comprises two units, an image analyzer module 24 and a post-processor module 26.

The image analyzer module 24 preferably has the ability to receive the dynamic processing package and analyze the information contained therein by employing the method specified by the task packer 12. Typically, the inspection method identified by the task packer 12 and performed by the image analyzer 24 is an inspection task-based comparison that analyzes the area within the image surrounding each inspection trigger and compares it to a reference. The comparison is preferably based on a comparison of the cells in the image of the object 4 to be inspected with the cells in the reference image for the area associated with the area triggered as having suspicious defects.

Post processor 26 preferably has the capability to implement the defect filter of FIG. 37, described below. Preferably, it has the ability to analyze the snap image from scratches with respect to areas that the image analyzer 24 considers to contain defects of a given type, which makes it more likely that a suspicious defect is a false positive.

SIP 7 functions substantially as described herein, and preferably provides an output report 30 of object defects and features based on the subject window provided by the subject window generator. .

Reference is now made to FIG. 2, which is a schematic functional block diagram of a BGA (ball grid array) substrate inspection system constructed and operative in accordance with a preferred embodiment of the present invention. The BGA inspection system of FIG. 2 is similar to the preferred embodiment described with respect to FIG. 1, but the BGA inspection system seen in FIG. 2 is part of scanner 34 (corresponding to scanner 2 of FIG. 1). It differs significantly in that it shows inputs from three sensors or cameras and does not include the snap generator 14. BGA
It is understood that the reference of the inspection system for inspection of is for the purpose of illustration and any suitable object can be inspected.

As can be seen, the inspection system of FIG. 2 is generated by a user workstation 32, typically a scanner 34 that produces an RGB or other digital image of a given panel 36 to be inspected, and a scanner 34. It comprises a HIP (Hardware Image Processing) unit 38, which typically includes dedicated hardware for processing images. HIP
Is preferably implemented in dedicated hardware, but it is understood that HIP can also be implemented in a DSP environment or custom software running on a general purpose computer workstation. As explained, (the image processor 5 of FIG.
HIP unit 38 (corresponding to the following) typically produces the following outputs detailed below. a. Binary cell b. Morphological features c. Color cell d. Color defect e. A snapshot of the RGB image around the selected defect

Downstream of the HIP unit 38 is a SIP (software image processing) unit 40 that receives the above five inputs from the HIP 38 unit and performs the image processing operations described in detail below. SIP is preferably implemented in software, but it is understood that it may also be implemented in other at least partially programmable environments such as DSPs. The output of the SIP can typically be displayed to the user on the screen of the user's workstation 32 (defect report 30 of FIG. 1).
Multiple images and defect reports).

The system of FIG. 2 typically operates in at least three scanning scenarios or models detailed below: a learning mode, a design rules mode, and an inspection mode. The "learning" mode typically comprises two sub-scenarios, namely a "learning mix" and a "learning reflex", which are preferably performed in an offline examination preparation step before processing a series of panels. In the learning mode, information about the panel to be inspected is entered into the system with respect to design rules and / or inspection modes, ie, “learned” to serve as a reference for later processing.

As can be seen, two fundamental channels of input data (snap data input channel from 50 and cell data input channel from 70) are typically received by the SIP 40. Each of the two data input channels is preferably two sub-channels, namely (i) a snap report sub-channel referred to herein as "report snap", and a color defect report referred to herein as "report color defect". “Sub-channel” and (ii) a cell reporting sub-channel referred to herein as “report cell” and a defect reporting sub-channel referred to herein as “report defect”. As seen in FIG. 2, these input data channels are provided by HIP 38, a preferred embodiment of which is described in Applicant's co-pending Israel Patent Application No. 131092, cited above.

Preferably, the reporting defective sub-channels are demultiplexed (at unit 120) to produce four raw data arrays detailed below, namely morphological features (f), hard, as seen in FIG. A wear defect (hd), a binary cell (c) and a color cell (cc) are acquired. In general, the channel task converter and splitter units 80 and 120 have the capability of demultiplexing the raw data to screen the input data for use by the SIP 40. The demultiplexed data is referred to as “SIP raw data” in this specification. SIP raw data typically includes an array of some information. Each array is typically referred to herein as "
"ds_array" and is stored in the data structure described in detail below.

In FIG. 2 and subsequent figures, the following abbreviations are used to indicate demultiplexed data sources in a compact form. C_0, C_1, C_2: Represents a "data structure array cell" type data source associated with a binary cell. Each number represents a data source from the three cameras in the preferred embodiment from which the data source was obtained (0, 1 and 2, respectively).
Associated with one of. HD_0, HD_1, HD_2: A “data structure array defect” type data source associated with a hardware defect, such as a nick protrusion defect, which is a bump that is located by HIP in a binary cell. Each number represents a data source from the three cameras (0, 1 respectively) in the preferred embodiment from which the data source was obtained.
And one of 2). F_0, F_1, F_2: Represents a "Data Structure Array Feature" type data source associated with a morphological feature. Each number represents a data source from the three cameras in the preferred embodiment (0, 1 and 2 respectively) from which the data source was obtained.
) Associated with one of. CC_0, CC_1, CC_2: Represents a "data structure array color cell" type data source associated with a color cell. Each number represents a data source, the three cameras in the preferred embodiment (0 for each) from which the data source was obtained.
1 and 2). CD_0, CD_1, CD_2: Represents a "data structure array color defect" type data source associated with color defects such as oxidation or color surface defects caused by residual or residual chemicals remaining on the surface of, for example, BGA. Each number represents a data source, the source of the data source, 3 in the preferred embodiment.
It is associated with one of the two cameras (0, 1 and 2 respectively). S_0, S_1, S_2: SDD (Small Defect Detector) data sources that are small high-contrast surface defects such as scratches and pinholes. Each number represents a data source, the source of the data source, 3 in the preferred embodiment.
It is associated with one of the two cameras (0, 1 and 2 respectively). Qname: represents the name of a window queue type data source queue, which is a queue of windows created in response to an inspection trigger awaiting image processing by SIP 40 (corresponding to analyzer 24 in FIG. 1). WD: Represents a line width defect type data source that is a defect in line width within conductors and in spaces between conductors in BGA. PIM Mask: Represents a PIM service type data source. The PIM is a point in the map, and the PIM service determines the position in the image to be inspected to ensure the relationship between the point to be inspected and the application of appropriate inspection rules and methods according to its specific position. A collection of PIMs used to define. PIM masks are zones that are drawn by the user, or zones that are created during the study phase of the study described below and are associated with a particular study execution sequence.

As shown in FIG. 2, one of the inputs to SIP 40 is typically user-defined configuration data 180 that specifies one of a SIP scenario, eg, learning mode, design rule mode or inspection mode scenario. Is.

Execution Scenarios for BGA Inspection by SIP SIP is a versatile and flexible image processing unit that can easily meet the inspection needs of various objects and can be operated in various inspection scenario situations. Is. For purposes of explanation, with respect to testing the BGA panel 36, various testing scenarios in which SIP can operate will be described.

The preferred scanning scenarios described for BGA inspection are illustrated here as run diagrams and are not intended to be limited to BGA inspection, but are suitable objects that can be inspected using SIP. Is representative of. Each of the preferred inspection scenarios or modes has the ability to perform different inspection functions, and generally includes the following scenarios. i. A design rule scenario in which a panel is inspected with reference to a set of general design rules.
Inspection in the design rule scenario is to quickly identify hardware defects (nicks and protrusions), as well as minimum line width violations related to conductor width in BGAs and minimum space violations related to minimum space between conductors in BGAs. It is intended. Execution of the design rule scenario is described in more detail below with reference to FIGS. 3 and 4. ii. A learning scenario in which you understand the design content and structure of a panel to generate references for use when inspecting a series of homogeneous panels. The learning scenario grasps the general attributes of the panel of the reference panel, such as the actual line width of each part of the conductor, and also the location of the specific target area within the panel, as described schematically with reference to FIG. Including that. Execution of the learning scenario will be described in more detail below with reference to FIGS. iii. An inspection scenario in which a panel is inspected and defects are identified with reference to a first, etc. panel having known desired attributes. The inspection scenario is a scenario typically used to inspect several lots of BGA for manufacturing and processing defects. Execution of the inspection scenario will be described in more detail below with reference to FIGS.

It will be appreciated that each of the SIP run diagrams is connected to at least one of the data input channels 50 and 70 of FIG. The input of channel data from channels 50 and 70 is dynamic and corresponds to the specific needs of each inspection scenario. The data in each of the channels 50 and 70 may preferably be provided in a file format for offline inspection or in TCP / IP (Transport Control Protocol / Internet Protocol) format. Typically, configuration variables are provided before running the inspection scenario to determine whether the data from the input channels 50 and 70 is from a file channel or a TCP / IP channel.

Scan scenarios are generally described herein in the context of SIP configuration files that define a set of inspection functions performed in the context of each scenario. Here, the name of the configuration file including the execution diagram is defined as far as applicable. next,
The following sets of preferred scanning scenarios are described in more detail.

Next, with reference to FIG. 3, a preferred SIP scenario named “design rule” (with a configuration file represented by dr.config) will be described.

This scanning scenario is intended to quickly identify hardware defects, as well as minimum line width and space violations, either from a file or online at the discretion of the user prior to execution of the scenario. Can be executed. This scenario is typically performed on the panel being considered for use during the learning scenario described in more detail below.

Design rule scenarios usually require a minimum set-up time, and the image quality of the inspected panel and the quality of the inspected panel are well tolerated, from which it is ready for use in the inspection scenario. It gives an initial rough indication of whether a particular panel can be used as a first-class panel that can constitute a comparative reference. The scenario is preferably all predetermined hardware defects (nicks and protrusions, cells on the skeleton, etc.), all color surface defects, all minimum line width violations in the conductor areas within the BGA panel, and within the BGA panel. It identifies all minimal space violations between conductors and prints out everything like defects.

The aforementioned defects are preferably identified without being associated with any external reference. If the BGA panel exhibits most of the above defects, it is easily understood that it is not suitable for use as a reference. However, even if a BGA panel passes all of the above tests, it may not be used as a suitable reference, so the panel was inspected to ensure that all features were present and according to design specifications. It is preferable to ensure that the features are correctly placed on the panel.

As seen in FIG. 3, the user defined design rule configuration data is entered into the SIP 40. The data typically includes the minimum required distance between the conductors and the minimum line width thickness of the conductors. Input image data for panel 36 is received from snap channel 50 and cell channel 70 and demultiplexed to provide the following image information data:
That is, color defect data, binary cell data, and hardware defect (nick and protrusion) data for each camera are used.

Suspicious defects in image data input for each of the cameras 0 to 2 are identified by the defect identification device 210, an appropriate inspection method is identified in accordance with each suspicious defect, and image data of the suspicious defect region is identified. It is incorporated into the containing window and the data methods are collectively operated by the data structure storage window 220 which has the ability to incorporate them into the data queue to be processed. The defect report generator 230 (analyzer 15 in FIG. 1) inspects the image data in the window according to the specified method. If the actual defect is found, the defect report 240
To report the defect.

Reference is made to FIG. 4, which is a sub-execution chart showing the flow of image data in and between the defect identification device 210 and the data structure storage window 220 in FIG. As can be seen, the defect detector 250, which has the ability to detect defects associated with very narrow conductors or wires, and very narrow spaces between conductors or wires, has two channels from channel 70.
Value cells are provided. "Very Narrow" Defect Output is Width Defect Data Report (WD
Data structure) 252 and report 252 is provided to merge unit 254. In addition to report 252, the merge unit receives input of color surface defects from channel 50, hardware defects from channel 70, and input data from the GUI for offline determination of defect analysis mask 256. The merge unit 254 merges the reports, filters the reports found in the masked area 258, and outputs a consolidated data report 260 in the form of defective data queues awaiting downstream processing. The mask area 258 is generated via the user interface.

Once a BGA panel has been tested in a design rule scenario and determined to be suitable to serve as a reference that can be compared to the panel being tested, the structure of the particular panel is learned and stored in memory. It will be necessary to execute the stored learning scenarios of the examination. The learned information about that panel is then used in the inspection scenario described below with reference to FIGS. 13-20 as a reference by which the presence of defects can be determined by comparison with the inspection panel.

[0100]   Next, a learning scenario will be described with reference to FIGS. 5 to 12.

Reference is made to FIG. 5 showing a schematic functional configuration diagram of a learning mode selected by the user. User-determined learning configuration data 180 is preferably entered via a generic user interface 170 (FIG. 2). The panel 36 is scanned and image data is generated. The following image data from each camera, namely color cell data 130,
134 and 138, binary cell data 140, 144 and 148, hardware defect (nick and protrusion) data 150, 154 and 158, and morphological feature data 160, 164 and 168 are used.

Learning is typically performed on the image data coming from each different camera by applying the three run diagrams in this order. learn. mixed. A "learning mix" whose details are contained in a configuration file called config will be described with reference to FIGS. learn. ref. A “learning reference” in which detailed components are contained in a configuration file called config will be described with reference to FIGS. 11 and 12. At the end of the learning scenario, scenario-specific output data is received and stored for use in the inspection scenario.

Each of the scanning scenarios typically receives raw data from an offline file channel. The raw data is extracted from a DMA (Direct Memory Access) type data input channel using a grabber program run by the application. The program reads the information from the DMA channel and writes the raw data to the file. The learning scenario then reads the data from those files. Next, a preferred embodiment of each learning execution chart (or scenario) will be described in detail.

Reference is made to FIG. 6 which shows a run chart 300 for a learning mixed scenario. The configuration of the learning mixed scenario is preferably stored in the configuration file.

In the learning mixed scenario, a window is established around each of the features that characterize the mask zone. The learning mixed scenario run diagram typically uses each of the user-defined windows, such as those defined via the GUI in an offline process before executing the learned mixed scenario.
Therefore, the input file to the PIM defect filter 310 becomes the user confirmation window. The PIM defect filter 310 outputs the defect reference and the defect window reference window 330 to the defect handler 320, which has the function of generating the target window in the form of SIP, which will later be used in the inspection scenario.

The user-defined window that can be defined in the learning mixed scenario for BGA type testing includes one or more of the following types of zones. -Target-A region with an indeterminate shape feature that the user wants to ensure that a particular shape is maintained within the BGA being inspected. -Ball-A region characterized by several circular pads on a BGA. -Connecting area-Area characterized by various bonding pads or power lines on the BGA. -Voids-areas that need to be separated from any morphological feature or cell. -Chip area-The area on the BGA where the chip is located.

The type of function performed for each window type is defined in the definition file.

The execution diagram of FIG. 6 receives the following demultiplexed image data. a. Color cells from each of cameras 0 to 2 (reference numbers 130, 134 and 138). b. 2 from each of cameras 0 to 2 (reference numbers 140, 144 and 148)
Value cell. c. Hardware defects from each of cameras 0 to 2 (reference numbers 150, 154 and 158). d. Morphological features from each of cameras 0 to 2 (reference numbers 160, 164 and 168).

The demultiplexed data is provided to the execution unit 340, which is associated with each camera and defines the following execution sequence.

Reference is made to FIG. 7, which is a preferred sub-execution diagram showing the execution sequence executed in each of the execution units 340. The execution chart of FIG.
, Processing raw data from one camera. The task packer / cell packer task 342 is defined in the general user interface of the workstation 32 (FIG. 2) and packs the raw image information data into the examination window according to a user confirmation window 344 sent to the task packer 342 as an event service 345. . Each inspection window contains raw image information data and a user confirmation window 34
Identify the assay function according to the type of 4. The window is the task packer 34
2 is pushed into the queue of window 346 and picked from that queue by assay manager 348. The assay manager pulls a window from the queue, performs the function associated with this window, and pushes it into one of the output queues according to its forward destination. Certification Manager 348
If is able to complete the verification on the window, the window is sent to the unified reference queue 350 and then added to the SIP defect handler 320 (FIG. 6).

If the verification manager 348 is unable to complete the verification function based solely on the raw image information received for a single camera, the window is unresolved queue 360.
Be pushed into. This can occur, for example, if the user-defined window is positioned so that it is not completely imaged by a single sensor.

Returning now to FIG. 6, the multi-slice manager 370 has the ability to merge each of the unresolved windows into the appropriate windows and generate a merged test to perform the inspection task. Push the outstanding queues 360 from each execution unit 340 in the proper order.

The multi-slice manager allocates information according to the following sequence to provide one or more functions detailed in FIG.

Unify all the basic image data for the corresponding unresolved windows from the neighboring slices acquired by the neighboring image sensor into a unified window and apply its appropriate function to the unified window according to its type. .

If the input image data includes data about the ball, that data is provided to the ball reference queue 380. In the proper order, the ball measurement unit 382 receives the data, measures the balls in the area, and outputs a data file SIP ball measurement data 384 showing apparent ball measurements.

If the input image data includes data about the connection area, the data is provided to the connection area reference queue 390. In the proper order, the mask extractor 392 receives the data, such as bond pads and power lines in the context of BGA testing.
A mask surrounding each inspection area in the window is extracted from the data and a data file called a polygonal reference mask and a stable feature reference 394 are output.

If the input data for the slice does not contain balls and does not require mask extraction, then that information is provided to the unified queue 398 along with any ball measurements and extraction masks. The unified queue is then transferred to the defect handler 32 for additional processing.
Provide to 0.

The functions and sub-execution charts of the qualification manager 348 (FIG. 7) relate to the raw data associated with a single camera, as well as the functions performed by the multi-slice manager 370 for combinations of slices received from multiple camera inputs. It is easily understood that they are substantially similar.

The functions performed in each inspection window in the learning scenario are defined in the file. The tables in FIGS. 8-9 list the window types and their corresponding verification functions (or a list of functions when the window attaches to a composite function) and a brief description of the function is presented. There is. The table describes the effect of the function on the window and return states, including the destination string that determines which output queue the window is sent to.

Both unified queues 350 and 398 proceed to the SIP Defect Handler task which writes the defects and creates a top down reference. A top-down reference is a reference that functions as a target spatial region during an inspection scenario, as described with reference to FIG. The function of attaching to each top-down window is performed at each spatial location associated with the location-based top-down window. The following functions can be performed before the top-down reference is created. a. The strip ball leaves only the circular data structure in the top-down window and establishes a region of interest around each circular data structure. b. Strip bonding returns a subwindow indication that directs the task to write a different reference window for each subwindow associated with each bond pad feature in the window. c. Strip targets do nothing in the top-down window. d. nop. Do nothing

In the learning phase, the parts of the panel known beforehand to be of particular interest from an inspection point of view, such as the ball area, pad area and target area in BGA inspection applications, are described below. Identify so that it can serve as a trigger for additional testing in the described testing scenario.

When a window is constructed by the task packer 342, the data stored inside each window is necessary to model that part of the window and inspect the window contents with a strict inspection method. Any data needed to perform each of the inspection functions. For example, in the BGA inspection, for the ball area, a list of each ball in the area including the center point and the radius is stored. For the target area, compressed polygons that delineate the boundaries of the target are typically stored.

Reference is now made to FIG. 11, which is a preferred run chart for the learning reference scan scenario. In the learning reference scenario, general reference data is grasped for the entire panel without being associated with any specific target spatial area. The execution chart of FIG. 11 is
Functions in the BGA test to establish reference data for the following attributes of BGA. a. The apparent width of conductors on the BGA and linewidth data, which is the space between conductors. b. In addition to the morphological features in the image of the reference object, positioning data that is an identification of features that are determined to be stable in the learning scenario. c. Cell data that is data about the position of a cell in the reference image. d. Feature data, which is data about the type and location of features in the reference image.

The execution diagram of FIG. 11 uses the following input data. a. Demultiplexed raw image data for morphological features. b. Demultiplexed raw image data for the cell. c. The polygon reference data created in the learning mixed scanning scenario described with reference to FIG. d. Disable mask data defining, for example, zones that act as masks for which line width calculations are not performed in BGA inspection, “target” zones, “ball” zones and “mask” zones. e. A stable feature reference that defines the region in which the feature is stable, ie, the feature is considered to have a stable position.

The run diagram of FIG. 11 produces the following output for the learning reference scenario. a. A rectangle around the area where the line width can be calculated. That information is stored in the learned line width discriminated window file. b. In BGA inspection, a line width reference that stores information about the width of conductors and the space between conductors. c. Reference feature for positioning in the reference feature file. d. A morphological feature file that contains raw references to cells and features within the cells, and reference data that is each indexed and that associates the feature information with the camera from which it was obtained.

As can be seen in the run diagram of FIG. 11, the demultiplexed raw image data for the binary cells from each camera 140, 144 and 148, and the morphological features from each camera 160, 164 and 168 is SIP. It is provided to the composite task unit 400. Each compound task unit 400 is associated with a camera. The execution chart for the composite task unit 400 will be described in detail with reference to FIG.

In addition, demultiplexed raw data information about morphological features, preferably information about each camera even if only information from one of the cameras is available, is provided to the task learning positioning unit 420, The unit is panel 3
6 receives stable feature data 422, which is a feature having a known stable position. Stable feature data is created in the learning mixed scenario.

The learning positioning unit 420 keeps track of the types and locations of stable features on the panel 36 (FIG. 2), identifies surplus and missing features, and serves as a reference for positioning the inspection panel relative to the reference panel. It has a function of generating a reference file 424 (FILEREF_FEATURES) used for an inspection scenario.

In addition, demultiplexed raw data information about the cells, preferably information about each camera even if only information from one of the cameras is available, is provided to the task learning linewidth unit 430, The unit receives a mask of polygon data disable mask data 432 generated in the learning mixed scenario. The polygon data and the disable mask data 432 are the PIM.
"MIM" in the map, which is a reference that associates a line width with a location in the map, a point in the map, and an area on the panel 36 to be examined in the learning reference scenario.

The learning line width unit 430 grasps the line width and associates the line width with the position on the panel 36 (FIG. 2) so that the calculated predetermined line width measurement value falls within the predetermined allowable measurement value range. Reference file 4 used in the inspection scenario as a reference to determine if there is
34 (FILELEarn_LW_DISQAULIFIED_WINDOWS
(Registered trademark) WIDTH REFERENCE FILE).

Reference is now made to FIG. 12, which is a sub-execution diagram for tasks executed in the composite task unit 400. The sub-execution diagram relates to the tasks performed with respect to the raw image data information for each camera.

A sub-execution diagram is a Geometric Reference Task Unit that operates on demultiplexed raw image data input for cells and morphological features to compute geometric references that identify cells and morphological features. Includes 450, associates them with locations in the reference image, and outputs the reference file. The first file is the cell type and location reference 452 and the second file is the morphological feature type and location 454.

Next, FIG. 13 to FIG. 20 showing an execution diagram of the inspection scenario and explaining the inspection test applied thereto will be referred to.

The following terms are used in the following description and are useful in understanding FIGS. 13-20. In the execution chart of FIG. 13, the following data sources are generated. R_1: Positioning transformation between the image slice associated with the reference central camera and the online image. E_1, IM_1: Surplus match and false match for the morphological features respectively acquired by the central camera. A surplus match is a morphological feature that exists in online images but not in reference images. Mismatches are morphological features that exist in the online image but whose coordinates are comparable to different morphological features in the reference image. T_XX (where XX = 0, 1, 2): Transformation from the coordinate system of the XX camera to the reference aligned coordinate system, derived from R_1. T_a2a: Transformation from reference aligned coordinate system to online aligned coordinate system, derived from R_1.

Reference is now made to FIG. 13, which is an execution diagram for locating and identifying surplus and missing feature tasks. The run diagram of FIG. 13 is executed by the positioning generator 18 of FIG. 1 through an inspection scenario.

The run chart shows color cells 130, 134 and 138, binary cells 140, 144 and 148, a hardware defect 150, from each camera of the scanner 34.
Raw image data is acquired including 154 and 158, morphological features 160, 164 and 168, and color defect 90 (FIG. 2; only data from camera 0 is shown for simplicity).

In the illustrated embodiment, for simplicity, the positioning task unit 510 has the function of locating features of the camera 1, which is the central camera. The positioning task unit 510 receives the feature 164 along with the reference feature 520 from memory,
It has the ability to generate the output of a positioning transformation 530 for camera 1, a surplus morphological feature report 532 for camera 1, and an incorrect match report 534 for camera 1.

The calculated transforms are transformed by the transform splitter unit 540 into three camera-to-reference alignment transforms per camera, namely T_0, T_1 and T_2, and between the reference data and the online data from the panel being examined. There is a conversion between two aligned coordinate systems, T_A2A. The data sources, raw image and computational data, surplus and missing features, and mismatches for the positioning transformation are all inputs to the inspection unit 550, and the run chart for it is shown in FIG.
4 is shown in detail.

Reference is made to FIG. 14, which is an execution chart for the inspection unit 550. The inspection unit 550 receives the intra-slice inputs for each camera as described with respect to the run chart shown in FIG. Inspection task unit 550 includes a first compound task inspection task unit 560 that operates on external image slices provided by cameras 0 and 2, and a second inspection task unit 600 that operates on a central image slice provided by camera 1.
With.

Next, FIG. 15 which is an execution chart showing the functions of the external compound task inspection unit 560
Refer to. Each external compound task checking unit 560 corresponds to the target task packer / window generator in FIG. 1, and has a task packer unit 570 having a function of receiving the input described with reference to FIG. 14 and a top-down reference.
Equipped with. The top-down reference includes, for example, the target spatial region generated by the target spatial region identifying device through the learning scenario as described. Each inspection trigger, eg top dow window, or hardware defect or morphological feature,
And for triggers that are not in the top-down window that require additional inspection, the task packer 570 identifies the inspection method and generates an inspection window that contains the image data needed to perform that inspection method. Have the function to The inspection window is pushed into the inspection window queue 572. The verification manager 580 has a function of collecting the inspection window from the inspection window queue 572 and applying the test method to the image data in the window. It is easily understood that the assay manager works on data for a single camera. Therefore, features, areas and top-down windows that extend outside the camera boundary or cover an area where two or more cameras overlap are unresolvable and unresolved queue 5
Whereas the fully resolved defects are pushed into the defect queue 584 while they are pushed into 82.

In parallel with the functionality of the task packer 570 and the qualification manager 580, two additional functional qualification units, a cell watchdog unit 590 and a lateral defect watchdog unit 595, function within the compound task inspection unit 560. The cell watchdog unit 590 has the function of receiving a binary cell and translating the input with a PIM cell reference defining an area where the cell does not exist. The cell watchdog unit 590 monitors the cells to ensure that there are no cells in the area designated by the PIM cell reference as being non-existent. If a cell is found in the area designated as having no cells, the defect report is pushed into the defect queue 584.

The lateral defect watchdog 595 functions as a filter for hardware defects and color defects. The PIM main defines the mark area. If the color defect or the hardware defect is located within the area displayed by the PIM main, the defect report is not issued. If the color defect or hardware defect is located outside the marked area, the defect report is pushed into the defect queue 584.

The hardware defect data source is typically a lateral defect watchdog 595.
And providing task packer 570, as well as input to switch P. Typically, the data from the hardware defect data source is sideways so that the defect report is only received after the hardware defect has been investigated via additional inspection functions provided by the task packer 570 and the verification manager 580. Switch P is turned off so as not to reach the defective watchdog 595. When switch P is on, it issues a double report useful for controlling and calibrating the effectiveness of the assay manager 580 in filtering out true and false defects.

Reference is now made to FIG. 16, which is an execution diagram showing the functions of the internal compound task inspection unit 600 provided with respect to image data from the central camera, camera 1. The operation of the internal composite task inspection unit 600 is performed by the external composite task inspection unit 56.
Similar to the operation of 0, but it has the ability to process the additional information coming about the linewidth defects shown with respect to FIG.

A linewidth inspection task unit 610 corresponding to the linewidth calculator of FIG. 1 is provided to calculate the linewidth in the online image of the panel under inspection 36 (FIG. 2).
. The line width inspection task unit 610 has the function of receiving line width reference data 612 and, in the described embodiment, the function of receiving positioning data and binary cell data for the central camera, ie the designated camera 1. Have.
The line width inspection task unit 610 applies the input positioning data to the binary cell and compares the result with a reference. If a line width defect is detected, the line width defect report 61
Issue 4.

The task packer unit 570 corresponds to the target task packer / window generator 12 of FIG. 1 and has a function of receiving the input described with reference to FIG. 14 and the top-down reference. In addition to the inputs received by the task packer unit 570 shown in FIG. 16, additional inputs are received: a line width defect report 614, a surplus defect report E_1 for the central camera, and an unmatched defect defect report IM_1 for the central camera. It is noticeable to do.

Not present in each inspection trigger, eg, top-down window, hardware defect or morphological feature or line width defect, or surplus / miss or mismatch, as well as additional inspection required For triggers, the task packer 570 has the ability to identify the inspection method and generate an inspection window containing the image data used to perform the method. The inspection window is pushed into the inspection window queue 572. Certification Manager 580
Has a function of collecting an inspection window from the inspection window queue 572 and applying the inspection method to the image data in the window. Certification Manager 580
Is easily understood to work on data for a single camera.
Thus, features, defects and top-down windows that extend outside the camera boundary or cover an area where two or more cameras overlap are considered non-decomposable and are pushed into the outstanding queue 582, whereas complete Defects resolved into 5 are defect queue 5
It is pushed into 84.

In parallel with the operation of task packer 570 and qualification manager 580, a cell watchdog and defect watchdog unit 620 is provided. The watchdog unit 620 operates as a filter for line width defects, color defects, hardware defects, redundant / missing defects and mismatch defects. The PIM main defines the mask area and conversion information T_1 about the central camera is provided for positioning. If line width defects, color defects, hardware defects, excess / missing defects or mismatched defects are located outside the mask area, the defect report will be displayed in defect queue 5.
It is pushed into 84.

It is noted that hardware defect data source, surplus / miss data source E_1, mismatched data source IM_1 and line width data source 614 provide inputs to watchdog 620 and task packer 570, respectively, and include switch P. It Normally, switch P is turned off so that data from the data source does not arrive at watchdog 620. When switched off, defect reports will only be received after checking for hardware defects via the additional inspection features provided by task packer 570 and qualification manager 580. If switch P is on, the qualification manager 580 that filters out true and false defects.
Double reports are issued that are useful for controlling and testing the effects of

It is noted that the inspection window generated by task packer 570 can be any one of the following two windows: (1) A predetermined (top-down) inspection window taken from the input reference file. (2) Defects (hardware, surplus / miss, line width), as well as windows around feature triggers that are not in the top-down window.

In a preferred embodiment of the present invention, a top-down window is provided for the features for which a rigorous model is created. For example, BGA inspection has produced rigorous models for balls, bond pads, targets and voids. Preferably, the model is dynamic, cannot fit exactly to a given model, and it is possible to identify unevenly distributed shapes in various forms according to general inspection routines.

Preferably, an inspection window is provided for features having an unknown shape, or a shape not predetermined according to the model. For that window, it is preferable to perform the inspection by comparing the general method of directly comparing the shape in the online image of the panel to be inspected with the shape of the corresponding feature present in the corresponding area of the reference. . Preferably, a comparison mode is employed that utilizes edge contour comparison, as described below.

By optimizing the inspection function as a whole by combining the region-dependent local modeling function and the comparison of the reference function, it is possible to determine whether objects with certain characteristics and those without certain characteristics are It is easily understood that it enables the detection of defects.

Define and store in a file the function to be performed for each top-down inspection window type. The tables in FIGS. 17-19 list each window type and its corresponding test function (or list of functions when the window attaches to a composite function). The tables in FIGS. 17-19 also describe the operation of the function in the window and return states, including the destination string that determines which output queue the window is sent to.

Referring again to the run chart shown in FIG. 14, pushes the outstanding queues 582 from all three cameras into the task multi-slice manager 630. This task collects all unresolved windows with the same identification tag, consolidates the inspection windows from the connection slices, performs their function, and pushes defects into the unified defect queue 640. The table of FIG. 20 describes the possible functions performed by the multi-slice manager 630 throughout the inspection scenario.

The defect handler task unit 650 preferably registers the alignment transformation data for positioning between the reference image and the online image of the panel under inspection 36 (FIG. 2), as well as the unified defect report from the unified defect queue 640. It has the function of receiving. The defect handler task 650 outputs the defect list 660 extracted from the windows in the unified defect queue 640. Further, when instructed by the configuration file, the defect task handler 650 may output a defect window file with its associated data, eg, cells, color cells, morphological features, defects depending on the type of window. . The defect window file is useful, for example, for debugging the operation of a defect handler.

Windowing of the Inspection Environment In SIP-based image processing, the “interest of interest” event is a rectangular area of interest that, from an image processing perspective, occupies a relatively small portion of the surface area of the object under inspection. Usually occurs within the (ROI) or window. Thus, SIP inspection includes at least some inspection routines that are preferably limited to the area of interest. This avoids the need to perform complex inspection methods on the magnified image to identify inspectable properties that are believed to lie within the region of interest surrounding the identifiable inspection trigger.

In order to speed up the calculation and allow more complex image understanding, preferably a three part image processing step is employed.

In the first stage of the process, a predetermined target spatial region, eg a top-down window, is defined as an additional inspection trigger and stored in the offline memory. The top-down window defines the reference image information for the target area contained in the top-down window, and the image processing that should be applied to analyze the target area at the same spatial position as the top-down window in the online image of the inspection object. Identifies a sequence of functions or image processing functions.

In the second step of the process, fast “quick and dirty” preferably hardware-based (but not limited to) image processing is used to identify additional inspection triggers and Generate decomposed basic image data for the entire online image. The decomposed basic image data is sent to SIP for SIP image processing and analysis.

In the third stage, image processing of three parts is preferably executed. First, a computational task is extensively applied to the decomposed basic image data to compute various computational information about the image, such as line widths, surplus features, missing features, improperly matched features, and positioning transformations. Secondly, a target window is defined around the predetermined features and defect candidates output from the additional inspection triggers, for example the second and third parts of the third stage. Third, an additional examination window is received for the top-down window created in the first stage and an additional examination window is generated for additional examination triggers located outside the top-down window. It

Basic image data from an online image of the object to be inspected for the area of interest, identification of the appropriate image processing function or sequence of image processing functions required to inspect the area of interest, and image processing for image processing Dynamically pack additional windows, including top-down windows and windows defined around additional test triggers, to contain any reference data needed by the function. The image processing function is preferably selected according to the type of the region of interest in order to provide the optimum image inspection according to the type of region or the inspection trigger in the inspection window.

With the above preferred structure, the intelligent (and resource demanding) image processing calculations can only be applied to a specified inspection window on a particular area of interest, rather than to the entire object.

A “window check trigger” that triggers an additional check is the center point of the ROI or window. As mentioned, the following items can be listed as the window inspection trigger. a. Defects, eg defects detected by hardware (eg nick / protrusion report) or defects detected by software (eg surplus / miss or line width report). b. Features such as morphological events (islands and open ends in electrical circuits). c. A top-down trigger, eg a predetermined position defined by the user. For example, a top-down trigger that the user draws with a square ROI around all balls in the ball area, or a pre-specified position as a result of executing a learning scenario as described above.

Typically, a dimensioning specification of the inspection window created around the trigger is attached to each type of window trigger. In addition, the window trigger preferably contains a definition of the type of calculations performed on the data that is supposed to reside within the window and the data inside the window. The definition is typically
Specified in a special configuration file.

Once the basic image data has been packed into the window, the inspection window is ready for processing. Typical processing functions operate on the inspection window and use the data inside the window. The processing function may sometimes generate additional data attached to the window. Some processing functions on the window are described below.

A particular feature of the preferred embodiment of the present invention is that different functions can be applied locally to regions of interest inside different windows. Thus, for example, for a region of interest associated with a top-down window, intelligent and rigid feature modeling functionality can be applied to the region of interest without indiscriminately applying it to the entire image. Feature modeling in the top-down window is defined geometrically as a feature with a known shape that can be easily represented and inspected by a suitable method, such as a two-dimensional circle that can be represented by a center point and a radius. It is especially suitable for the ball pad used.

For regions of interest associated with other non-top-down reference window inspection triggers, preferably comparison-based image processing functions can be applied. For example, comparison-based image processing functions are preferably suitable for inspecting features and defect candidates that are not stable and that are characterized by shapes and dimensions that cannot be easily modeled in advance.

According to a preferred comparison method, the binary cells are combined and transformed into vector lines containing directional components in the learning phase. The directional component represents an edge, or a contour between the morphological feature and the substrate in the reference image, and is stored in memory for use in inspecting an online image of the inspected object. Regarding generation of a cell and a vectorized line component representing a set of cells, FIGS.
In more detail below with reference to d), the cell-to-vector comparison function will be described in more detail below with reference to FIGS. 34 (a) to 36 (b).

The following preferred steps are useful in understanding the windowing behavior. Step I: The user typically defines a window in the reference object around a spatial region containing at least one unit of a given model shape. In an inspection object such as panel 36 (FIG. 2), it is assumed that at least one shape defined by the model is located in the reference object in a spatial region corresponding to the region defined by the user. It is preferable to select the appropriate model shape from the library of modeled geometric shapes stored offline. For example,
A circular "ball" 882 (Fig. 31), typically as found on a BGA, can be defined by a center point 884 and a radius.

Step II: Define the attributes of the reference area. For example, the user may
With respect to the position, size, and other aspects of No. 2, it is possible to set parameters regarding the allowable deviation in the image under inspection. Therefore, the center point 884 can be defined as one that should be located within the range of the allowable distance from the predetermined coordinates, and the radii at various positions on the circumference are defined as those that should be within the predetermined range of the allowable radius. be able to. Other parameters such as lines or edges, shapes, or other parameters such as whether the image element is accepted within the region surrounding the modeled shape may also be defined. Steps I and II are preferably performed in the learning phase described above.

Step III: An image of the inspection object is acquired in the inspection stage. Preferably, the image is acquired using the apparatus according to the preferred embodiments of the invention described herein, and also one such as a cell, a color cell, morphological feature information and other information about the image. It is preprocessed to generate basic image data in the above format.

Step IV: In the online image of the object to be inspected, an additional inspection area is cut off. Each of the additional inspection areas preferably corresponds spatially to one of the reference areas defined in step II, or a defect or feature defined during setup when the user requires additional inspection. Corresponding to the spatial region surrounding some form of predetermined test trigger.

[0174] Preferably, each of the additional inspection regions includes a stored shape model and basic image data to identify one or more inspection functions performed on the window and may be required by the image processing functions. Incorporated in an additional inspection window that specifies any user-defined parameters. For example, if the additional inspection area includes a ball 882, the window preferably includes all cells for the area in the additional inspection window, any morphological features found therein, line width information, and the like. In addition, the inspection window typically includes inspection features that operate on the ball, as well as tolerances from the desired position to the center, and radius tolerances that tie to different line segments of the circumference. Specify user-defined parameters for.

Thus, when an inspection window is generated for an area surrounding a defect candidate, it is especially a function of identifying that defect candidate, which is different from the function used to identify the modeled shape. Features are included in an additional inspection window.

Step V: Image processing functions packed with additional inspection windows are performed on the basic data in the windows and evaluated according to user defined parameters.

Preferably, if the additional inspection window comprises a modeled shape, the basic image data contained in the additional inspection window are processed to generate geometric objects representing the features. . It is understood that the basic image data is derived from the online image of the inspected object at the spatial position associated with the reference window. For example, the feature inspected in an additional inspection window is the ball 882, and then all cells in the inspection window can be combined to produce a line representing the circumference of the ball. Once the geometry is generated, the geometry in the window is compared to the model in the reference window.

It is possible to create a library of many possible shapes, but
It is understood that it is not feasible to predict shapes that represent defects that are especially random in nature.

Therefore, in the preferred embodiment of the present invention, the geometric shape in the object to be inspected at a specific position in the image and the shape assumed to exist at that position by reference are referred without referring to the mathematical model. The ability to analyze the shape of a geometric feature is provided by a direct comparison with and. Direct comparisons are performed, for example, by the cell-to-vector comparison function described below, or by generating and comparing geometric representations by bitmap comparisons as is well known in the art.

Step VI: A report is created to show the presence, absence and location of features and attributes in the inspection and whether any of those features are defective.

Preferred General Data Structure in SIP Reference is now made to FIG. 21, which is a diagram of a preferred data structure of input data imported from the image processor 5 (FIG. 1) that can be used in the context of BGA inspection. Figure 1
The raw data, which serves as the input for, typically comprises a stream of data words arranged as shown in FIG. Typically, the SIP channel task represents the input raw data as a data structure array element type where the element is defined as one of binary cells, color cells, snap images, morphological features, defects or color defects. Convert to. The structure of the data structure array and the structure of the type of each element in the data structure array are detailed below.

In a preferred embodiment of the present invention, raw data is received in a data structure as described in FIG. 21 and may be included in a data structure array, for example by the data converter and splitter unit 120 shown in FIG. Split and converted into possible data structures.

The header for a typical report element describing one row y of a scanned image as received from one camera in the scanner 34 (FIG. 1) is shown in FIG. There is. As shown, the header typically specifies the type of report element, such as the type of event that occurred in a column y, the (cnt) number, and the camera or slice of interest.

The preferred typing scheme for identifying the type of report element is the following 3-bit scheme. 0--report without type. 1 --- Report of color defects. 2--Report of defects. 3--Report of cells related to edges. 4--Snap Report.

“Slice” means, for example, the camera number of the three cameras used to image the panel. Multiple cameras and slices are discussed later in this specification.

One preferred SIP data structure array is a “data structure array element”.
Is a data source called. This data structure treats the data as bundled in lines ordered by increasing Y coordinate. Each line contains zero or more data items of a particular type or element (4 bytes for all SIP raw data items). The range of accessible lines includes all lines from the smallest line in an image to the largest line in that image. An empty data source is identified if the following conditions are met: Minimum line> Maximum line

When the data source is created, the minimum line value is set to 0 and the maximum line value is set to -1. The data source specifies the maximum number of lines in the scan, which is preferably greater than or equal to the coordinates of the maximum line produced by the scanner.

Therefore, each column in the scanned image corresponds to the column specified in the data structure. At any given moment, the number of columns between the smallest and largest lines in the data structure corresponds to the scanned image column ready for processing.

Each element in the data structure array typically has only one producer and at least one consumer. It is possible to add a line to the data source only by the producer and only if the y coordinate in the added line is larger than the largest line in the data source. When adding a line, all lines between the largest line and the new line (not including the new line) are added as empty lines, each containing zero data items. If the new line is not empty, then the new line is added with all its elements and the maximum line is updated to be equal to the total number of new lines.
The producer of the data source needs to ensure that at the end of the scan, the largest line of the data source equals the largest line in the scan. It is understood that the largest line can be empty.

Preferred Data and Queue Management in SIP Generally, SIP includes multiple tasks managed from an advanced image processor, such as image processor 5 of FIG. 1, and by a SIP manager that handles the flow of data between tasks. . The SIP manager handles the synchronization between reading data from the external raw data channel and performing calculations. The manager typically does not care about the specific type of data in the system or the type of calculations performed by the various computational tasks.

SIP managers typically manage the flow of data and computations between tasks, manage the input of data from external channels and the output of data to external channels, and provide a uniform error and logic mechanism. To provide statistics on timing and memory consumption.

The basic building blocks of SIP include data sources, tasks, run diagrams and configuration files, all of which are described next. 1. Data Source All data used by more than one computing unit is typically in the form of a data source. Each data source preferably has a producer and a consumer (task or channel, see description below). Producers create data sources and consumers consume data from those data sources. Each consumer is responsible for notifying the data source of the data it no longer needs. Data that is not needed by any consumer can be safely removed from the data source, freeing memory space and resources for additional input data. It is S to remove data to make space
This is the task of the IP manager.

2. Task Preferably, a task is a computing unit capable of performing some computation on at least one input data source. Push the results of the calculations into at least one output data source. Individual tasks are preferably not used for data processing (eg internal interference overflow or data and computation synchronization) but for some data. Data processing issues are typically handled by the SIP manager.

The channel task has a role of connecting the SIP as a process executed on the computer to the outside, for example, operates on the input data to demultiplex the input data, and puts the data in a format usable by the SIP. Play a role of converting. Each data channel is typically attached to a physical channel (DMA, TCP / IP or file channel). The input channel transforms the input data into a data source. Output channels are attached to the data source, and they write the data source to the specified channel (as soon as the data source is updated). The use of file channels allows the scanning to be easily simulated without actually needing a connection to the DMA or TCP / IP channels. The input data can simply be injected from a file containing data in the same format as the data coming from the DMA or TCP / IP channel.

Some tasks, particularly inspection tasks, call functions that perform image processing calculations on the data.

3. Execution Diagram A given execution diagram (also referred to herein as a "scanning scenario") is defined by the set of data sources and tasks participating in the scenario.

Next, refer to FIG. It is noted that the scanning scenario can be generally represented by the chart shown in FIG.

In the scenario of FIG. 23, SIP is connected to two input channels and one output channel. One DMA channel 600 for generating the data source 2,
And there is one TCP / IP channel 610 that produces data source 1. Using three calculation units (tasks), namely, a data source 2 and a data source 1 are used to generate a data source 3, a data source 1 is used, a data source 1 is not generated, and a data source 3 is used. Then, there is a task 3 for generating the data source 4. The data source 4 is fed to the TCP / IP channel (channel 3) and written into it as soon as it is updated by task 3.

4. Configuration File The run diagram and all parameters, tasks and data sources used by the channel are typically specified in a configuration file. This allows many different scanning scenarios to be run without the need to recompile or relink.

The inclusion of scanning scenarios in the configuration file makes SIP relatively easy to extend and modify. From a software development perspective, one of the most time consuming aspects of SIP is typically data processing, as well as synchronization between data acquisition and computation, but they are often associated with many AOIs (automatic optics). Inspection) applications can be very similar or identical. Preferably, the SIP is configured so that tasks are easily added and data sources that are examples of built-in template data sources are easily added. Thus, in order to add a new data source or task channel, a suitable class of data source or task is derived from the already existing corresponding base class of data source and task. It typically converts as much of the work as possible to the base class (due to the extensive use of template patterns).

Reference is now made to FIG. 24, which is an execution chart for the two steps. A given run diagram can typically be processed by more than one SIP. To do this, the run diagram can be divided into several run diagrams, each running for a different SIP. A data source created by a task in one SIP can be used by a task in another SIP. As shown in FIG. 23, the data source is written to the TCP / IP channel by the first SIP and read from TCP / IP by the second SIP. Minimal crosstalk gives a short calculation time between two SIP processes.

Basic Formats of Basic Image Data-Cells and Events In the following sections, for basic image data, for the preferred embodiment of the present invention employed for inspection of BGA, for basic image data, the data is preferably represented in a data structure within SIP. The method will be described.

Binary Cell Representation The following definitions use the following definitions of cells (contour elements). That is, a cell is a unidirectional straight line defined within a pixel. The position and orientation of a cell within a pixel is calculated as a function of the gradient difference in reflected intensity when aligned with neighboring pixels. The collection of cells has the function of defining a boundary between two populations in a pattern that is typically formed by a binary image (ie an image containing only two populations) and with sub-pixel accuracy. . If each cell in the image is oriented in one direction, make sure that the "white" area in the binary image, which contains only white and black areas, is always to the left of the oriented or directed cell. , The orientation of each cell is defined.

US Pat. Nos. 5,774,572 and 5,774, incorporated herein by reference.
, 573 (Caspi et al.), DoGs (Gaussian difference).
Compute the cell as a function of the zero-crossings of. Other suitable edge detection techniques include "Digital Image Processing" (C.
Gonzalez and P.G. Wintz), Addison Wesley, R
Eeding, MA, 1987 ".

In the present invention, cells are preferably calculated for pattern edges in the red band of the RGB image. The “white” region is a region where the value of DoGs is negative.

Next, refer to FIG. 25A showing a pixel 710 having a cell 720 inside.
The pixel 710 has, for example, one corner (X, Y) and (X + 1, Y) located at the origin.
Pixels in the image of the object 4 to be examined (FIG. 1) with diagonal corners located at +1). The cell 720 is located within the pixel 710. Pixel 710 is "white"
It spans region 730 and "black" region 740. As shown in FIG.
0 is vectorized such that the white area 730 is to the left of cell 720.

A preferred 24-bit data structure for representing cells as elements in the cell data structure array used for SIP includes the following fields: x (12 bits) is the X coordinate of the cell. Note that as long as each line in the image is grouped into a data array, the y coordinate of the cell is specified from the line in the cell data array that contains the cell. Direction (1 bit) and Edge Code- (3 bits) define the orientation and direction of the cell. Last or "L"-(4 bits) defines the location of the last endpoint of the cell on the edge of the pixel. The final point is the point on the edge of the pixel where the cells intersect. First or "F"-(4 bits) defines the position of the first end point of the cell on the edge of the pixel.

26 (a) -26, which are generally self-explanatory, showing possible cell orientations, each corresponding to an edge code value from 0 to 5, with orientation value 0.
Reference is made to (f). Therefore, the edge code value 0 is assigned to the cell of FIG. 26 (a), the edge code value 1 is assigned to the cell of FIG. 26 (b), the edge code value 2 is assigned to the cell of FIG. 26 (c), The edge code value 3 is assigned to the cell of FIG. 26 (d), the edge code value 4 is assigned to the cell of FIG. 26 (e), and the edge code value 5 is assigned to the cell of FIG. 26 (f).

FIG. 26 (g) and FIG. 26 (h) showing the edge code value of 6 with the direction value set to 0.
Refer to. A saddle occurs when there are two cells in a pixel as shown in FIG. In the case of a saddle, individual cell information is expressed in a format similar to that shown in FIG. However, an edge code value of 6 results in an edge code of 6 through the next step of combining the cells to calculate the contour representing the edge.
It differs from the edge code of 5 in that it allows the reconstruction of the saddle based on the evaluation of neighboring pixels. A code value is typically assigned to represent each of the states of FIGS. 26 (a) -26 (f) and 26 (h), with an additional "illegal" code value indicating a skeleton event. Saved for.

27 (a) to 27 (f), in which the cell directions that may correspond to edge code values of 0 to 5 are shown respectively, and the direction value is 1, are generally understood.
). 27 (g) and 27 (h) show a saddle state that exists when the direction value is 1, and the whole is different except that the relationship between the "white" region and the "black" region is preserved. 26 (g) and 26 (h).

Once the edge code and direction code are defined, the orientation of the cell with respect to the edge of the pixel is known and the first and last points of the cell can also be defined. The first point is the point represented by “F” in FIGS. 26A to 27H, while the last point is the point represented by “L”.

The first and last fields of the cell type preferably represent the positions of the first and last points in the circular coordinate system. For this coordinate system, see FIG.
You can understand it by looking at (i).

[0213] Preferably, each edge of the cell is divided into units of 1/16 and represented by 4 bits.
Therefore, in the example of FIG. 27 (i), the value of “F” is 14 and the value of “L” is zero.

Vector Display of Feature Edges Reference is made to FIGS. 25 (b) to 25 (d) showing the relationship between features and analytical and vectorized representations as cells. FIG. 25B shows a substrate background 760.
FIG. 7A illustrates first and second features 750 of FIG. FIG. 25C is a diagram showing a map including an aggregate of pixels 770 in which cells 780 are seen. The cell represents an edge subpixel intersection between the feature 750 and the substrate 760. Features are usually shown as "black" regions because they have a relatively high reflection coefficient and represent a negative DoG. Substrates usually have a relatively low reflection coefficient and exhibit a positive DoG, so
Shown as the "white" area.

Reference is now made to FIG. 25 (d), which shows a display of edges of the feature 750 (FIG. 25 (b)), where the edges of the feature 750 are represented by edge vectors 790. In accordance with a preferred embodiment of the present invention, the location and orientation of cells within individual pixels, eg, cell 780, can be determined through the learning reference scenario described above to determine the target object 4 (FIG. 1).
Calculate and record the image of. A collection of cells in substantially the same direction are combined into a connected component, transformed into an edge vector 790, which is stored and used as a reference during the inspection scenario described above.

Color Cell Display A color cell is an extension of a binary cell. Color cells differ from binary cells in that they identify and establish a boundary between two different regions in the image, each region being one of several regions having a substantially uniform color. The color cell indicates the position of the border with sub-pixel accuracy, and also indicates which region is located on each side of the cell. The data structure of a color cell is generally understood in light of the above discussion of binary cells.

Each color cell typically contains the following fields: x (12 bits) is the X coordinate of the cell. Note that as long as each line in the image is grouped into a data array, the y coordinate of the cell is identified from the line in the cell data array that contains the cell. Direction (1 bit) and Edge Code- (3 bits) define the orientation and direction of the cell. Last or "L"-(2 bits) defines the position of the last endpoint of the cell on the edge of the pixel. The final point is the point on the edge of the pixel where the cells intersect. First or "F"-(2 bits) defines the position of the first end point of the cell on the edge of the pixel. Material Edge- (4 bits) defines a uniform color or group of materials present on each side of the cell.

The preferred 24-bit data structure sacrifices resolution (compared to binary cells) to identify color populations. However, this is a design choice and higher resolution can be obtained by employing a data structure with more bits. Furthermore, another meaning is assigned to edge code = 7. In this case, this edge code 7 represents a color connection point. A color connection point is a pixel that separates two color cells with material edges having different values.

Morphological Feature Display Each morphological feature is preferably represented in the following fields. x (12 bits) is the X coordinate of the morphological feature. Note that as long as each line in the image is grouped into a data array, the y-coordinates of the morphological features are identified from the lines in the feature type data array in which the feature is included. Data- (20 bits) typically represents one of the following types of morphological features: Open end Island Three-way connection point Four-way connection point Five-way connection point Blob connection point Pinhole “inside” corner “outside” corner

Hardware Defect Indication Each defect is preferably represented by the following fields. x (12 bits) is the X coordinate of the defect feature. Note that as long as the elements in each line in the image are grouped into a data array, the Y coordinate of the defect is identified from the line in the defect type data array in which the defect is identified. Data- (20 bits) typically represents one of the following types of defects: Micro-cell defects Strong nicks Strong protrusions General micro-cell defects Cells on the skeleton-cells located on the morphological skeleton, typically due to too thin lines. Match Defects Extra Features-Online inspection scenario with no features matching the reference (
Features in the image of the object detected in eg "inspect"). Mismatch-A feature that is detected in an online inspection mode that matches features in the reference, but for which the reference feature is different. For example, the online feature is "connection point"
However, the reference feature may be an “open end”. Defect-a feature in the reference image that does not have a matching feature in the image of the object acquired in the online inspection scenario.

Color Defect Display Each color defect is preferably represented by the following fields. x (12 bits) is the X coordinate of the color defect. Note that as long as the elements at each line in the image are grouped into a data array, the Y coordinate of the color defect is identified from the line in the color defect type data array in which the color defect is identified. Color defect data (20) bits. In the preferred embodiment of the present invention, color defects are filtered by a watchdog task, such as watchdog 620 of FIG. 16, and sent directly to the defect report.

Width Defect Report The width defect report is output from the line width inspection process. The report takes the form of a data source containing line width defects detected by the process. A collection of width defects is provided in the width defect data structure array. The single width defect data structure includes the following fields. x (12 bits) is the X coordinate of the width defect. Note that as long as the elements in each line in the image are grouped into a data array, the Y coordinate of the width defect is identified from the line in the width defect type data array in which the width defect is identified. The type (1 bit) designates the type of width measurement as one of "spatial skeleton" which is a spatial measurement and "line skeleton" which is a line width measurement. Direction- (4 bits) specifies the measurement direction, preferably from one of the following possible directions: Unknown direction, east direction, south direction, southeast direction, southwest direction. Width (15 bits) specifies the width measurement of the width defect.

Other Main Data Formats in SIP The following sections describe various other main data formats used in SIP.

Snap Image Display Each snap image is preferably represented by the following fields. x (8 bits) is the X coordinate of the snap image. Note that as long as the elements at each line in the image are grouped into a data array, the Y coordinate of the snap image is identified from the line in the snap data array from which the snap image is identified. R- (8 bits) which is the intensity of the red RGB band. G- (8 bits) which is the intensity of the green RGB band. B- (8 bits), which is the intensity of the blue RGB band.

Line Envelope A line envelope is an envelope structure that surrounds a line, such as an edge contour, and is generally adopted in the cell-to-vector comparison function described below with reference to FIGS. 34 (a) to 36 (b). It

Reference is now made to FIG. 28, which illustrates a line envelope construct 800 referred to herein as a “directed straight line envelope”. The envelope of a line is a rectangle constructed around the line. Line envelope 800 has the same orientation as line 810. The vertical distance from the line 810 to both sides of the rectangle is represented by the parameter of the display width 820, and the distance from each end of the line 810 to the end of the envelope is represented by the parameter of the tongue width 830. Another parameter used to define the envelope is the angle tolerance parameter, which is a parameter in the cell-to-vector comparison function that ensures that the cell orientation is close to the line orientation.

Reference is now made to FIG. 29, which shows a line envelope for polygon 850. Polygon 850 is simply a collection of lines 810. As can be seen, the polygon envelope 860 is merely a collection of all envelopes for each line segment 810 forming the polygon 850.

PIM PIM (Points of Map) is a collection of data structures and processes that is useful for solving points in Map Queri. A map query point is a query that identifies where a particular point is located in an image and is useful in ensuring that a task or function is performed at the appropriate point or region in the image. PIMs are preferably used by SIP filtering and watchdog tasks.

MIM MIM is preferably a PIM-based data structure. It adds the additional functionality of attaching different data items to different areas. These data items may represent thresholds for calculations, for example. The use of MIM allows different events, such as features and defects, to be treated separately as a function of position in addition to event type.

[0230] For example, a calculation step of measuring a region of a group of surface defects. If the defect is inside any given critical region, then a small value in that region is sufficient to assert that the location is defective. However, if the defect is outside of a given critical region, the same value will pass through the MIM filter so that the region is not declared as defective. As an additional example, assume that a nick has been designated as an additional check trigger. Nicks are characterized by having a height and width that exceed a predetermined threshold. For SIP inspection, MIM can be used to dynamically adjust height and width thresholds as a function of the position of an additional inspection trigger for the nick in the pattern of the inspected object. It is easily understood that the threshold can be entered by the user via a common user interface.

Top-Down References Top-down references are used to define top-down (TD) triggers. The top-down trigger holds the image and type information about the target area around the point, which is learned during the learning scenario, so that the target area can be inspected by a predetermined method using the inspection function depending on the type of the top-down trigger. Instruct SIP. For example, a top-down trigger for a bond pad would instruct the SIP to perform a series of inspection functions for the area of interest defined in the top-down report surrounding the bond pad.

The Top Down Report preferably contains the following fields: X sub-pixel (4 bits): Defines the sub-pixel movement of the top-down report in the X direction within the range of pixels for which the top-down report is defined. Y sub-pixel (4 bits): Defines the sub-pixel movement of the top-down report in the Y direction within the range of pixels for which the top-down report is defined. Index (24 bits): Defines the index of the top-down report within the array of all top-down windows for the object to be inspected.

Next, refer to FIG. FIG. 31 shows the outline of a selection element that may be present in a typical top-down window, and a circle “” showing a line segment 880, a center point 884.
FIG. 8 is a simplified illustration of a shape representing a “ball” 882 and a rectangular pad 886 showing an open end 888. Each of these shapes is shown in Figures 25 (a) to 25 (d).
) Is derived from the cell described with reference to FIG.

Once the raw data has been packed into the window, it is ready for processing. Preferably, the processing functions manipulate on-window data located within the window, as described below.

SIP Defect Report The SIP Defect Report identifies the structure of a single SIP defect. The SIP defect report includes the following items. Coordinates (X, Y) of the position of the defect. A descriptive string. Defect counting type.

Unified Event A Unified Event is an event that is guided from multiple cameras and packed into a Unified Window. Each unified event preferably identifies the (X, Y) position of the pixel for which the report is defined and specifies an event type selected from one of the following event types. Top-Down Report Features Defects Width Defects The data associated with each unified event depends on the type of event and includes data as described above for each event.

Affine Transform Affine Transform performs positioning of cameras between cameras and information about affine transform used to perform positioning between an online image of an inspection target object and its reference image in the inspection mode. It is a data source that contains. The affine 2D conversion is as described above.

Window Queue The window queue is a queue of windows arranged in order and waiting for the task to perform certain calculations. Windows enqueue the primary data source used to move windows between tasks. The window is
It may be pushed into a queue by one task, or after a task has finished working on a window, that window will be pushed to another queue until another task that attempts to work with that window pops it. Placed inside.

Positioning and Multiple Camera Considerations According to a preferred embodiment of the present invention, the scanner 2 in the embodiment of FIG. 1 and 34 in the embodiment of FIG. An image of the object 4 (FIG. 1) or panel 36 (FIG. 2) is acquired. To use multiple cameras, the input of each camera must be transformed by the corresponding alignment transformation. The alignment transform is an affine transform calculated such that each point in the panel viewed by two or more cameras is mapped by the alignment transform to the same point in the alignment coordinate system. The coordinate system is preferably set so that the units in this aligned coordinate system are approximately equal to 15 microns. The points in this coordinate system are (Xa, Ya
).

Scanning with a large number of cameras is clearly relevant when designing the defect-finding process. For example, a preliminary step in the inspection of large objects that are not wholly visible with a single camera operates on fragments of the object that are wholly identified with different cameras. Unresolved fragments are then unified and inspected. Finally, each piece from the camera's coordinate system in which the task is performed is transformed into a unified aligned coordinate system and the partial images are stitched together into one large object represented by the aligned coordinate system.

According to a preferred embodiment of the invention, eg the number of cameras, the width of each camera (preferably expressed as the number of diodes or pixels), the overlap between adjacent cameras,
And a camera model is provided which specifies the geometry of the optical head including the alignment transformations required to transform the points on a given camera into a camera independent coordinate system. The unified coordinate system is called the aligned coordinate system.

Preferably, two camera models are stored. “Camera model”, which is camera model alignment information regarding a specific inspection device at the time of inspection. “Camera model reference”, which is the camera model of the inspection system when the learning scenario is executed. The camera model reference may or may not be the same as the camera model at the time of inspection. The use of double references allows, for example, the learning and inspection system to be performed on separate inspection systems.

In the preferred inspection scenario of the present invention, additional alignment transformations are used to locate the reference image and the online [camera / alignment] image respectively when comparing the unified image of the object under inspection with the reference. To do. A preferred method for performing registration between images is US Pat. No. 5,495,53, incorporated herein by reference.
5 (Harel et al.), Which is generally adapted to identify clusters of features, and to perform dynamic positioning between clusters rather than by positioning between individual features. To be done.

Next, for example, the panel 36 (FIG. 2) is prepared for inspection according to a preferred embodiment of the present invention.
3) is a schematic diagram showing a collection of similar frames 870 that can be assembled on top of FIG.
Refer to 0. Each frame is the same. Preferably, the parameters and attributes are learned for only a single frame 870, and the global positioning coordinates may be changed so that each frame is examined and compared to the reference separately.

Tasks The following sections describe the main tasks performed by SIP.
Tasks are represented as functional blocks in the execution diagrams of FIGS.

[0246]   I / O task   The I / O task connects the SIP process to the outside.

Input Channel Task The Input Channel Task is connected to a FILE or TCP / IP channel.
The input channel task reads the input raw data and transforms it into a data source. In the preferred embodiment of FIG. 2, the input channel tasks are associated with data converters and splitters 80 and 120.

Output Channel Task The Output Channel Task writes a data source to a specified channel, namely a FILE or TCP / IP channel. For example, in the run diagram of FIG. 3, the output channel is associated with the defect report generator 230.

Composite Task A composite task is a task that manages a collection of tasks and data sources.
When you execute a compound task, it determines the order in which the tasks are executed. The selected task is the highest priority task of all the tasks that can be invoked when a request is made to execute the task.

Line Width Measurement Reference is made to FIGS. 32 (a) and 32 (b) which show a preferred method for measuring line width. The line width of features such as bond pads is preferably measured with respect to the morphological framework 900. One line is defined by the two lines 920 and 910 formed by the collection of cells located on each side of the skeleton 900. The line width is defined as the length of the line perpendicular to the skeleton 900 and tangent to the two opposing lines 910 and 920. FIG. 32 (a) is a diagram showing linewidth measurements for a framework with substantially parallel edges. FIG. 32 (b) is a diagram showing a width measurement for a frame whose edges are not parallel.

The line width measurement task is executed in each of the design rule scenario, learning scenario and inspection scenario described above.

The line width measurement preferably outputs statistics on line width and line spacing for all positions on the object being inspected and issues an error report if the minimum line width is not adhered to. . Line width is usually related to line and position, but the minimum spacing violation applies generally.

Linewidth Learning A linewidth reference comprises a group of rectangles, each encompassing a region of the image within a range where all linewidth measurements have approximately the same value. In areas where line width changes are large, it is typically not possible to create a suitable linewidth reference. The line width learning task preferably has the function of identifying areas where it is practically impossible to form a line width reference due to the large degree of change in the line width. These areas are included in a rectangular area identified as an area whose line width cannot be grasped.

Line Width Inspection The line width inspection measures the width of lines and the spacing between lines in the image of the object under inspection. The calculated width is compared to the line width and spacing for the corresponding position in the reference image. Positioning transformations are used to associate respective positions in the image under inspection and the reference image. A line width defect is reported as a mismatch between the calculated line width in the online image of the object and the corresponding tolerance range for the line width specified in the line width reference for a given position.

Positioning The Positioning task transforms between the online coordinate system and the reference coordinate system for the object image under inspection. If several cameras are used, the transformations are preferably provided per camera, but for ease of calculation only one camera, for example only the central camera, is used to perform the positioning operation. It is understood that it may be used.

Reference is made to FIG. 33, which is a diagram showing an example of conversion from an online coordinate system to a reference coordinate system and vice versa. The left coordinate system 950 is an online coordinate system, and the right coordinate system 960 is a reference coordinate system. The figure shows a fixed transformation involving only rotation and movement. However, it is understood that the positioning transform need not be a fixed transform, but may generally include any suitable transform.

Positioning Learning The Positioning Learning task creates a reference to a feature that serves as a reference for positioning. Preferably, the (X, Y) coordinates for each morphological feature and the respective feature type are stored. A particular type of feature is defined as a "stable" feature, which means that the position within the object to be inspected is stable, preferably within a small error, and positioning uses stable features. It is preferably carried out.

Positioning Check The Positioning Check receives an input array data structure as input and outputs a transformed data structure for affine transformation. Further, the data structures of the surplus defect, the defective defect and the improper match are output.

Using the comparative approach with the reference, where the reference is made through the pre-inspection learning scenario as what is considered a “perfect” object, the object being inspected is at least partially inspected. preferable. The complete object is preferably obtained by scanning a first-class substrate specifically selected as free of defects or by learning from CAM data. Throughout the inspection scenario, the online image from the scanned object is compared to the full reference to determine if an error on the online panel typically makes it significantly different to the reference panel.

[0260] Typically, it is not possible to directly compare the reference image with the online panel because the position of the online panel on the scanner and the position where the reference image was generated are slightly different. Therefore, the coordinates of a given feature on a first class substrate are typically different from the coordinates of the same feature on an online panel.

The positioning task uses the features to calculate a positioning transformation that maps points from the online coordinate system to the reference coordinate system.

The preferred method used to test BGA is briefly described below. First, the input features are centralized into a cluster of generalized features, thereby trying to overcome the inherent instability of morphological features on the BGA. The same feature (in terms of circles) may result, for example, in one scan on one island and on another scan in two very close aperture ends. The two open ends can be defined as a single generalized feature, and a matching rule can be defined that justifies matching the generalized feature to the island feature. Generalized features and match rules are typically specified in a special configuration file called a matcher table. The configuration file may also define a centralized distance for consolidating different morphological features into a single generalized feature.

The positioning module preferably also has a function of identifying reference features that are not recognized as “missing” in the online feature group and online features that are not recognized as “surplus” in the reference feature group. The positioning task also preferably identifies mismatches. Mismatches occur when a feature is detected in an online coordinate system that is matched to a reference feature (ie, after transforming a reference feature using a positioning transform, that feature is close to an online feature with a completely different type). I understand). A typical example of a false match is a match between a junction and an island.

[0264]   Surplus / miss and mismatched features are reported as surplus / miss defects.

Positioning Transform Split In one preferred embodiment of the present invention, the positioning task only works for camera features that are located within the field of view of the central camera. The transform split task takes a positioning transform and splits it into three positioning transforms, one for each camera. In other embodiments that obtain positioning data for each of the cameras, this task is not required and the positioning task can work on aligned features.

Snap Data Packing In the Snap Data Packing task, input image data is processed to construct a window containing all image data for a desired area. The windows are calculated by pooling them together with the input color defects so that each snap window contains snap data around a particular color defect cluster.
After the snap data has been packed into a window that preferably also contains color defect data, that window is pushed into the output queue.

Construction of PPM File for Viewing Snap Data In the preferred embodiment of the present invention, a snap image is constructed for the selected defect. The input data is processed to produce a set of output files containing RGB images of nearby snap areas.

Watchdog The watchdog compares the data in the input data source with the specified PIM.
If it is recognized that the specified data of a specific type exists inside the prohibited area described in the PIM, the violation data is converted into a SIP defect. SIP defects are stored in a SIP defect data structure, they are embedded in a window, and the window is pushed into the window's output queue along with all its defects. One or more of the following features and defects: color defects, hardware defects, line width defects and morphological features are preferably monitored by a watchdog.

Task Packer Packer packs features, cells and defects into a specified window (rectangular area) forming an area of interest for additional downstream processing. Windows are preferably specified for each top-down event, as well as for each width defect, redundant / missing defect and hardware defect.

A top-down window, preferably forming part of a top-down reference through a learning scenario, is constructed, preferably before cells and features are packed inside the top-down window (using positioning transformations) The top-down window is transformed into an online coordinate system.

The following scheme is typically used to minimize the number of windows created and to include all the information of interest (top down window and windows around the defect area). First, the packed data for all top-down windows adds each packed window to the packed window's queue. Consider all defect and line width defect reports. 1. Take the first defect that is not inside the already created window (ordering is done by increasing the y coordinate and then the x coordinate). 2. Create a window around the selected defect, pack all data inside this window and add it to the queue of packed windows. 3. If there are no more defects inside the already created window, go to step 1.

At the end of the packing process, all data (cells, color cells, defects, features, width defects) are packed into one or more windows and the windows are output to the window queue. It is noted that the event data items in the overlap region can be stacked such that they are packed onto each of the overlapping windows.

Raw Data Reference Configuration This task takes input raw data and creates a raw data reference suitable for fast access in a test scenario.

Filters This task is attached to the PIM for filtering. It is assumed that the PIM is defined in the reference alignment coordinates. The task receives input data for color cells, features, width defects, defects and affine transformations and considers each element of the input data source. First, the input affine transformation is used to transform the elements into the reference (alignment) coordinate system. If no input transform is specified, the alignment transform is used to transform the elements from the camera into the aligned coordinate system.

Next, consider each element from the input data. This element is transformed into a reference-aligned coordinate system and checked for if the element is inside the PIM's specified filter zone. The configuration file determines whether to filter this element. If you do not want to filter that element, push it into the output data source. Whether an element is filtered inside a specified zone or if it is outside a specified zone is typically controlled by the value of a configuration variable.

Extract Information from Window This task has the function of extracting a window from the input data queue and extracting data from that window as needed. When finished, push that window into the output queue.

Generation of Zone and Stable Feature References This task is typically declared as a “stable” feature due to (a) various types of PIM regions and (b) previous functionality operating on the window. Write auxiliary data from the input window for the data including all features.

Ball Measurement This task writes auxiliary data from the input window. The data typically includes a file containing information about all the circles found in the window, and two other files containing statistics about the circle distribution within the panel.

Assay Manager This task preferably pulls a window from the input window queue and uses the basic image information data contained in that window to perform the function associated with that window. Based on the output of the function, decide what to do below. Table 1. Behavior of the qualification manager for each return state from the function

[0280]

[Table 1]

Preferably, if the function is not attached to the window, the return status is OK. Multi-Slice Manager The multi-slice manager preferably has the ability to receive windows containing outstanding data from different cameras. Preferably, the window is located in the aligned coordinate system. The multi-slice manager unifies all windows having the same window identification number and stores the unified window in memory. A unified window is created that includes all windows and then the function attached to the unified window is executed. Based on the output of the inspection function, determine what to do below. Table 2. Behavior of the multi-slice manager for each return state of the function

[0282]

[Table 2]

It is typically understood that if the function is not attached to the window, the return status is OK. Defect Writer The Defect Writer preferably accepts four filenames from the configuration file. (A) Output window file name: File containing the window. A value of "none" means that the file name is not a valid file name and this file is not normally created. (B) Defect window file name: File containing a defect window. "None"
A value of means that this is not a valid filename and this file is not normally created. (C) Defect file reference coordinates: A file containing defects in reference alignment coordinates. A value of "none" means that this is not a valid filename and this file is not normally created. (D) Defect file online coordinates: a file containing defects in online aligned coordinates. A value of "none" means that this is not a valid filename and this file is not normally created.

The defect writer pulls a window from the input queue and preferably proceeds to the following steps. 1. If the window contains a defect: If it is desired to generate a file or output file with a defective window, store the window for later processing. Alternatively, all data except defects is removed from the window and the window is stored for later processing. 2. If the window does not contain defects: If you want to generate an output window file, either store the window for later processing or delete the window.

At the end of scanning, the following processing is performed on all the windows already stored. First, it accumulates all defects from all windows. In the accumulation step, two defect vectors are created. 1. A first vector containing defects in reference-aligned coordinates, preferably present for each window, uses the self-to-reference affine transformation to transfer the contents of the window to the reference-aligned coordinate system, so that defects inside each window are Convert to system. The defect is added to the vector only if there are no defects inside the mask area defined by the PIM specified in the input PIM data source. If PIM is not defined, add all defects to the vector. 2. A vector containing the same defects in online aligned coordinates. This vector is generated from the first vector by applying the positioning transform specified as input to the task. Note that the match transform is used when the input affine transform is not defined.

If a valid output file containing reference coordinates is defined, write the first vector to that file. If a valid output file containing online coordinates is defined, write a second vector to that file.

Each defect file contains a set of lines, one for each defect. Each line typically has the following form: XY-defect type-frame index where X and Y are coordinates of the defect (reference alignment coordinate system or online alignment coordinate system). The defect type is a list of defects. The frame index is an index of a frame including a defect.

Next, if there is a valid defect window file for the named file, write all defective windows to that file. This is especially useful for visualizing defect windows.

Finally, in the case of a valid output window for the name file, the function attached to that window is first executed, and then the return state of the function is considered to make a judgment as to how to proceed with the processing. . Table 3. Operation for each return state of the function

[0290]

[Table 3]

It is generally understood that if the function is not attached to the window, the return status is OK. Align Cell Render The Align Cell Render task considers each input cell. If a cell does not exist inside the crop area specified in the configuration file, it is converted from the camera to an aligned coordinate system and written to a special graphical format file that makes the file suitable for display.

This task is useful for checking alignment transformations in inspection systems. Preferably, the task is performed for all cameras and the output files are displayed in sequence. The preferred alignment transform is a transform that transforms cells from two overlapping cameras such that cells from the first camera are exactly located above cells from the second camera in the region where the two cameras overlap. Is.

Function A function is an encapsulation of an inspection method used for image information contained in the inspection window. The functions are activated by a window-oriented task, preferably the assay manager task or the multi-slice manager task described above, preferably in a predetermined order for examining the image data content of the window. The active task applies a function to the data in the window and executes the function with the specified window as an argument. The function performs its computational task on the window and returns one of the three return codes described below. Based on the return code, the calling task determines what to do next.

Each function preferably receives from the configuration file all the parameters needed for the calculation and extracts from the window any image data information needed for the calculation.
Preferably, the functions are modular and concatenated when more complex calculations are required on a given window.

The function preferably returns one of three possible values. "OK" indicating that the function was successfully calculated. An "error" indicating that the function failed to calculate (eg due to a bug or other serious problem). "Forward", which indicates that the function succeeded in its computation, but required additional computation to complete the computation task. Completion of the task is accomplished by forwarding the window to the forward destination and assigning the window an additional function for that window. The active task typically knows how to forward the window to its forward destination and how to activate the forward function. A "forward subwindow" that indicates that the function succeeded in its computation, but that additional computation is required to complete the computation task. Completion of the task is accomplished by forwarding all subwindows within the window to the forward destination and assigning each subwindow an additional function for that subwindow. Active tasks typically know how to send subwindows to their forward destinations and how to activate forward functions.

[0296]   Inactivity (knop) function   This function does nothing and preferably returns an OK value.

Composite Function This function preferably implements the set of functions provided in the configuration file. The functions are executed in the order specified in the configuration file.

Transfer Function This function gets the name of the forward function and forward destination for additional processing from the configuration file. Transfer functions typically preferably accept flags that specify special exceptions, including the following exceptions: An "exception of transfer only when inside the camera", which has the ability to ensure that the window is transferred only if it is completely inside the camera boundary. An "exception for transfer only if a defect is found", which has the function of ensuring that the window is transferred only if the window contains a defect. An exception to the "subwindow transfer" which has a true / false flag and the ability to return the state so that the forward subwindow state is returned if the value of the flag is true. Otherwise, the forward state is returned.

Windows are preferably not transferred if the following conditions are met: 1. No defect exists in the window, provided that the window is transferred only if there is a defect in the window. 2. A window is not completely inside the camera boundary, provided that the window is transferred only if the window is completely inside the camera boundary.

Convert Window Function to Reference Coordinates The convert window function to reference coordinates removes non-convertible data items from the window and converts the window coordinates from online image coordinates to the reference coordinate system.

Remove Non-Convertible Data From Window The remove non-convertible data from window feature preferably removes non-convertible image data, eg cells, present in the window.

Watchdog Function The Watchdog function scans every input cell in the window. Report any cell found in the rectangle enclosing the reference window as a defect.
This function is typically used for regions where it is desired to ensure that no features are present. For example, in the context of BGA testing, the watchdog function may be applied to the area defined in the top-down window as being absolutely devoid of any features.

Defect Filtering Functions Inside Windows Defect filtering functions inside windows are attached to PIM data sources. This function receives a PIM that has been matched with aligned coordinates. This function, when executed, transforms the input defect into a reference alignment coordinate system and checks the PIM for alignment defects. Mask zone specified by PIM (or multiple mask zones)
If a defect is found in it, it is filtered out or pushed into the output list of defects.

Connection Component Function The connection component function creates a connection component from cells (including both binary cells and color cells) that are inside the window. The resulting connection component is placed in a window, preferably for further processing by the vectorizer function described below.

Vectorizer Function The vectorizer function acquires connection components and approximates each connection component with a straight line so that the deviation between the connection component and the linear approximation does not exceed the specified tolerance specified from the configuration file. .

Ball (Circle) Modeling and Inspection Function The ball modeling function, preferably as used in the context of BGA inspection, is outlined below. Ball modeling is representative of a type of modeling suitable for use in inspecting online images of the object under inspection in a top-down window situation.

Ball Modeling with Connected Components Ball modeling deals with connected components found inside the inspection window. The function has the function of creating a reference model in the learning phase and of determining the characteristics of the ball in the online image of the object under inspection in the inspection phase. If the connection component does not already exist, the function creates it from the cell. The inspection window is assumed to contain only the ball. Therefore, if the connection component in the window is closed, it is assumed to be a single ball, whereas if the connection component is open, it is assumed to be part of the ball. This function analyzes the connected component, for example, by testing it against a center point and then measuring the radius of the connected component's contour to determine if it is within a given tolerance, and To see if it's a ball. Information about the circle of a single connected component is held in a data structure representing a generalized circle.

Creating a Ball Window for Reference Creating a ball window for reference removes all objects other than the ball from the window.

Reference Ball to Online Ball Comparison The reference ball to online ball comparison function compares the ball in the reference image with the ball in the online image for the object being inspected. The position and radius of the online ball are preferably compared to the reference image for that ball. A ball in the online image is considered to match a ball in the reference image if the online ball has a similar radius with a specified tolerance and is within a specified distance from the reference. Any excess balls and missing circles are recorded as defects.

Cell-to-Vector Comparison The cell-to-vector comparison function compares the reference vector (polygon) obtained from the reference window with the cells present in the online image of the object under examination 4 (FIG. 1), Based on the comparison, it is determined whether the estimated defect is a true defect or an error detection. Cell-to-vector comparisons are particularly suitable for inspecting for hardware defects, redundant / missing defects, mismatched defects and morphological features. In addition, it is used to test certain top-down windows, such as bond pads.

Next, a reference vector or polyline defining the reference feature 1020.
Reference is made to Fig. 34 (a), which is a diagram showing an inspection window 1000 in which 1010 is seen. Each polyline 1010 is associated with an envelope 1030.

Further, FIG. 34 is a diagram showing an inspection window 1002 in which a group 1040 of online cells calculated from the online image of the inspection object 4 (FIG. 1) is superimposed on the polyline 1010 of FIG. 34 (a). Reference is made to (b). Cell aggregate 104
It is easy to see that 0 is aligned with the reference polyline 1010. Since the cell aggregate 1040 is the putative protrusion 1050, it is identified as a defect candidate by the defect identification device 13, and the cell pair vector function shows that the putative protrusion is a defect candidate or a true defect. Applied to verify if there is.

As can be seen, the putative protrusion 1050 extends outside the area defined by the envelope 1030. Thus, according to the preferred embodiment of the present invention, the cell pair vector outputs a defect flag for the inspection window 1002.

Reference is further made to FIG. 34 (c), which is a diagram showing an inspection window 1004 in which an online cell aggregate 1060 is calculated from an online image of the inspection object 4 (FIG. 1). This object is different from the inspection object 4 (FIG. 1) from which the cell 1040 (FIG. 34 (b)) is calculated. It will be readily appreciated that the collection of cells 1060 is properly positioned using the reference polyline 1010. The cell aggregate 1060 is identified as a defect candidate by the defect identification device 13, for example, because of the putative protrusion 1070. The cell pair vector function is applied to verify whether the putative protrusion is simply a defect candidate or a true defect.

As can be seen, despite the putative protrusions 1070, the collection of cells 1060 are all within the area defined by the envelope 1030. Thus, according to a preferred embodiment of the present invention, cell 1060 has envelope 1
The cell-to-vector function does not output a defect for cell 1060 because it is not outside 030.

However, in FIG. 34 (c), a second cell assembly 1080 completely outside the envelope 1030 is also seen. Each cell of the second aggregate 1080 is marked as defective. It is readily understood that cells located outside the envelope, such as cell 1080, can be filtered by the defect handler function and reported as defective or filtered out of scope.

The cell-to-vector function preferably proceeds according to the following sequence. Check the inspection window, eg, inspection window 1000 for the presence of cells, and ensure that the number of cells in the window does not exceed the maximum number of cells allowed in the inspection window. Check. The number of cells is preferably calculated by counting the midpoint of each cell. It is understood that the parameter is a variable and can be set or changed depending on the position of the window within the image.

Check the inspection window 1000 for the presence of the reference polyline 1010. If the inspection window 1000 does not include the polyline 1010, the defect is returned.

Preferably, the polyline 1010 and the cell aggregates 1030, 1040 and 1060 are finely aligned. Reference is now made to FIGS. 35-36 (b), which presents and describes a preferred method of micro registration.

Micro-registration is preferably carried out by the following procedure. Step 1100: Check each cell 1110 located in the inspection window 1120 and enclose one or more envelopes 1130 surrounding the reference polyline 1240.
Identify those cells 1110 that are inside the and tag them appropriately. Step 1200: Check each of the cells 1110 inside the envelope 1130 for orientation. Each cell that has the same orientation as the polyline 1240 within the angular tolerances presented by the configuration file based on inspection requirements is represented as a "matching" cell 1210. Step 1300: Least-squares calculation is performed on the distance between each matching cell 1210 and its neighboring polylines to obtain a collection 121 of cells in the online image.
Move 0 to polygon 1140 until the value is minimal. When the value is minimized, fine positioning of the image is performed as shown in FIG. 36 (b).

Following the micropositioning, cell-to-vector comparison is performed as follows. Envelope 1030 (Fig. 34 (a)) is constructed with a width around the reference polyline, which is also equal to the value of the configuration variable. The cell 1040 is then compared to the envelope 1030. If no cell 1040 is present in any envelope, report a defect. Preferably, each envelope contains all cells 1040 that reside inside envelope 1030.
Record the number and length of. If no cell is recorded in a given envelope in the inspection window 1000, a defect indicating that the affected envelope is empty is reported for the midpoint of that envelope.

Creating Reference Windows from Connected Regions Preferably, in BGA checking, a function is created that creates a reference to a connected region. The reference creation function for the connection area analyzes the connection component and
It has the function of locating the connection components that are part of the functional entity called the "power line". User-defined configuration variables adapted to the power line of a specific structure are used in the decision making process. When a power line is detected, calculations are made for the three zones under test. -Critical Zones that preferably exhibit tightly defined attributes such as location and material. -Non-critical Zones with a high degree of breadth in terms of required attributes and locations. -A zone whose location is likely to change without being considered as an unstable defect. The unstable zone is checked only if required by the configuration variables.

The function that creates the bond pad reference subwindow is preferably BGA.
It also functions in inspection and is preferably employed to detect if the inspection window covers any bond pads. The function analyzes all the connection components in the window and identifies those connection components that are part of the bond pad. Preferably, the critical, non-critical and unstable zones are identified.

In the preferred embodiment for BGA inspection, in order to isolate the solder mask area from all other materials, (color cell The contours of the uniform color population (identified by

Cell-to-Vector Error Classification Generally, all cell-to-vector errors are considered defective. However, sometimes the cell-pair vector may indicate the presence of "surface defects". Surface defects are extraneous connecting components that were present in the reference data.

Referring again to FIG. 34 (c). Reference polyline 1010 is assumed to be a bonding pad. According to a preferred embodiment, a collection of cells, such as collection 1250, corresponds to the polyline 101 corresponding to the desired bond pad in the reference image.
If it is detected to have a polarity opposite to that of 0, then a "pinhole" is indicated. Polarity is the orientation of the cell or polygon, respectively.

Post-Processing Reference is now made to FIG. 37, which is a flow chart in accordance with a preferred embodiment showing the operation incorporating an image post-processing system that may function as a task in the image post-processor 26 (FIG. 1). The image post-processor task shown in FIG. 37 acts as a defect filter that outputs defects when micro-defects are located by their quantity rather than their quality, and a defect report needs to be issued. For example, in the context of BGA inspection, individual pinholes in the metallization on the bonding pad are acceptable, but if the amount of pinholes exceeds a threshold, then the BGA is considered a defect.

In the present embodiment, the defect in the image of the object is designated as a regular defect or a micro defect. Regular defects in the image that correspond to regular defects in the object are usually unacceptable, whereas small amounts of microdefects that correspond to microdefects in the object are usually acceptable.
In the above example, pinhole defects can be considered as micro defects and defect defects such as defective bonding pads can be considered as normal defects.

Step 3305: The user defines the criteria used to define the defects in the image. A screen display similar to that of FIGS. 38-46 allows the user to specify defect criteria.

Step 3310: If any number of microdefects of any type can be tolerated, the user specifies, for each type of microdefect, the maximum allowable number (x) of each type of microdefects. . The number (x) can be one defect or more than one defect. In short, if the number of microdefects of any particular type is x + 1, it is unacceptable.

Step 3315: Inspect the image, preferably using one or more image processing methods presented and described with reference to FIGS. 1 to 36 (b) of the present invention.

Step 3320: Determine whether a defect is recognized according to the decision tree.

[0333]   Step 3325: If no defects are found, consider the image OK.

Step 3330: Further decision trees are continued to classify the defects as regular defects or minor defects.

[0335]   Step 3335: If a regular defect exists, issue a defect report.

[0336]   Step 3340: Count micro defects of each type.

Step 3345: If the number (n) of micro defects of a certain type is smaller than the number of permissible micro defects of a predetermined type, no defect report is issued. However, if the number of microdefects of that type exceeds the permissible number of microdefects of that type, ie, n> x, then a defect report is returned at step 3335.

Users of image processing systems including the microdefect post-processor described with reference to FIG. 37 typically specify parameters and criteria for inspection of the image,
It is easily understood that the same or different parameters can be adopted for the inspection of the user-determined target area and the inspection of the defect by the single processor.

It is easily understood that the above description of the post-treatment unit is not intended to limit the above-mentioned method, and that other post-treatment methods can be adopted. Suitable post-processing methods can be filtered, for example, by a function that further analyzes the area of interest surrounding the candidate color defect, a defect resulting from oxidation and photoresist residues, and additional image processing on the selection of the strict image. Functions may be included to identify and classify other suitable defect candidates.

General User Interface (GUI) The image processing system according to the invention is particularly flexible and by way of example not intended to be limiting, the identification of various inspection parameters including general parameters, calibration, repetition, Ip, color and light. And it is easily understood that it enables control.

The inspection criteria are also preferably specified by the user. The parameters may include, by way of example and not limitation, the top down window type, location and content for balls, targets, bond pads and voids, power lines, and the like. Additional parameters such as minimum line width can also be given. According to the invention, preferably various parameters are associated with a particular position of the inspection object.

Reference is now made to FIGS. 38-46, which are examples of screen displays generated through user determination of criteria for inspection of images by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 38 shows a screen display that allows the user to specify general inspection criteria.

FIG. 39 is a diagram showing a screen display that allows a user to specify bonding inspection criteria.

FIG. 40 shows a screen display that allows the user to specify target inspection criteria.

FIG. 41 is a diagram showing an example of a screen display that allows a user to specify criteria for inspection of a power line.

42 and 43 are examples of screen displays that allow the user to specify criteria for ball inspection.

44-46 are examples of screen displays that allow a user to specify advanced criteria and parameters for inspection of an image of an object.

According to a preferred embodiment of the present invention, a two-part general user interface is provided for defining a region of interest of an object to be examined, such as a BGA. A portion of the user interface provides a graphical interface that provides a graphical representation of the object to be inspected, the user using a pointing device, typically a mouse, on the graphical representation of a spatial object area for inspection. Specify the type. In the second part, parameters are provided for each of the spatial regions of interest specified.

A part of the disclosure content of this patent document includes content subject to copyright protection. The copyright owner has no objection to the reproduction of patent documents or patent disclosures in what appears in the Patent Office's patent files or records, but in all other respects Protect your rights.

It will be appreciated that the software components of the present invention may be implemented in ROM (Read Only Memory) format, if desired. The software components may generally be implemented in hardware and / or DSP units using conventional techniques as desired.

It will be appreciated that, for clarity, various features of the invention, which are described in the context of individual embodiments, can be combined and provided in a single embodiment.
On the contrary, it is understood that the various features of the invention, which are, for the sake of simplicity, described in the context of a single embodiment, may be provided separately or in any suitable combination.

Those skilled in the art will appreciate that the present invention is not limited to that specifically shown and described herein. Rather, the scope of the present invention is defined only by the appended claims.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an image processing system constructed and operative in accordance with one preferred embodiment of the present invention.

FIG. 2 is a schematic functional configuration diagram of an image processing system configured and functioning according to another preferred embodiment of the present invention.

FIG. 3 is a schematic functional configuration diagram of the device in FIG. 2 when the user selection scenario is the “design rule” mode.

[Figure 4]   FIG. 4 is a schematic functional configuration diagram of a preferred embodiment of the unit 210 of FIG. 3.

FIG. 5 is a schematic functional configuration diagram of the device of FIG. 2 when the user selection scenario is the “learning” mode.

FIG. 6 is a schematic functional block diagram of a preferred embodiment of individual units of the “learning mixed” unit of FIG.

7 is a schematic functional block diagram of a preferred embodiment of each of the three blocks in FIG.

FIG. 8 is a diagram forming a table for explaining a preferable test function provided for each window type in the course of a learning scenario in combination with FIG. 9.

FIG. 9 is a diagram forming a table for explaining a preferable test function provided for each window type in the course of a learning scenario in combination with FIG. 8.

FIG. 10 is a table of preferred functions typically performed by the task multi-slice manager of FIG. 6 during a learning scenario.

FIG. 11 is an execution diagram of a scanning scenario described herein as “lern.ref.config”.

FIG. 12 is an execution diagram of a scanning scenario described herein as “lern.ref.eg.config”.

FIG. 13 is an execution diagram of a scanning scenario described herein as “inspect.mixed.config”.

FIG. 14 is “inspect.mixed.egmain.config” in the present specification.
3 is an execution chart of a scanning scenario described as ".

FIG. 15 is “inspect.mixed.egside.config” in the present specification.
3 is an execution chart of a scanning scenario described as ".

FIG. 16 is an execution diagram of a scanning scenario described herein as “inspect.mixed.eg.config”.

FIG. 17 is a diagram forming, together with FIGS. 18 and 19, a table for explaining a preferable test function provided for each window type in the course of an inspection scenario.

FIG. 18 is a diagram forming a table for explaining a preferable test function provided for each window type in the course of an inspection scenario, in combination with FIG. 17 and FIG. 19;

FIG. 19 is a diagram forming, together with FIG. 17 and FIG. 18, a table for explaining a preferable test function provided for each window type in the course of an inspection scenario.

20 is a table of preferred functions typically performed by the task multi-slice manager of FIG. 14 during a test scenario.

FIG. 21 is a diagram of a preferred data structure for a report element that can be received from the hardware image analysis system of Israeli patent copending application 131092.

FIG. 22   FIG. 22 shows a preferred data structure for the header of FIG. 21.

23 is a diagram showing an example of a schematic execution chart for the image processing operating system SIP of FIG. 1. FIG.

FIG. 24 illustrates another example of a schematic execution diagram that differs from the diagram of FIG. 23 in that it is configured for execution by a computer farm that includes at least two concurrent parallel processors.

FIG. 25 (a) is a diagram showing the concept of a pixel including one cell, and FIG.
(D) is a figure which shows the concept of the relationship between the analysis display by a feature and a cell, and the vectorized line component.

FIG. 26 shows an edge code preferably used to define the type of edge or edges contained in an individual record.

27A to 27H show edge code values 0 to 9 and dir =, respectively.
FIG. 27 is a diagram showing a cell direction for No. 1, and FIG. 27 (i) is a diagram showing a cell defined by parameters of edge code = 0, first = 14, and last = 0.

FIG. 28   It is a figure which shows the envelope data structure which surrounds a cell.

FIG. 29 is a diagram showing two envelopes each including two continuous line segments.

FIG. 30 is a schematic diagram showing an assembly of identical frames assembled on an inspection panel according to a preferred embodiment of the present invention.

FIG. 31   It is a figure which shows the example of the data which can exist in a window.

FIG. 32 shows preferred forms of linewidth measurements for parallel and unbalanced lines, respectively.

FIG. 33   FIG. 6 is a diagram showing a conversion between an online and a reference coordinate system.

34 (a) is a diagram showing a vectorized polygon reference associated with an edge contour, and FIG. 34 (b) is a diagram showing a complete alignment between online and reference data. FIG. 34 (c) is a diagram showing an online cell superimposed on the polygon reference of FIG.
34A shows an online cell different from the online cell of FIG. 34B overlaid on the polygonal reference of FIG. 34A by perfect alignment between the online and reference data.

FIG. 35 is a schematic flow chart illustrating a preferred micro registration method according to a preferred embodiment of the present invention.

36 (a) is a diagram showing an online cell overlaid on a polygonal reference due to incomplete alignment between the online data and the reference data, and FIG. 36 (b) is a diagram of the present invention. It is a figure which shows the structure of FIG.36 (a) after execution of the alignment method by a preferable embodiment.

FIG. 37 is a flow chart illustrating the functionality of an image processing system constructed and operative in accordance with another preferred embodiment of the present invention.

FIG. 38 illustrates a screen display with a general user interface used to define parameters for inspection of an image by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 39 illustrates a general user interface screen display configured to define parameters for inspection of an image by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 40 illustrates a screen display with a general user interface used to define parameters for inspection of images by a functional image processing system constructed and in accordance with a preferred embodiment of the present invention.

FIG. 41 illustrates a general user interface screen display used to define parameters for inspection of an image by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 42 illustrates a general user interface screen display configured to define parameters for inspection of images by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 43 shows a screen display with a general user interface used to define parameters for inspection of an image by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 44 illustrates a general user interface screen display used to define parameters for inspection of an image by a functional image processing system constructed and in accordance with a preferred embodiment of the present invention.

FIG. 45 illustrates a general user interface screen display used to define parameters for inspection of an image by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

FIG. 46 shows a screen display with a general user interface used to define parameters for inspection of images by a functional image processing system constructed in accordance with a preferred embodiment of the present invention.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) G06T 7/00 300 G06T 7/00 300B (81) Designated country EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OA (BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, GM, KE, LS, MW, MZ, SD, SL, SZ, TZ, UG, ZW), EA (AM, AZ, BY, KG, KZ) , MD, RU, TJ, TM), AE, AG, AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, BZ, CA, CH, CN, CR, CU, CZ, DE, DK, DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP , KR, KZ, LC, LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SL, TJ, TM, TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW (72) Inventor Klingbel Eyar, Rishon Lezion, Israel 75254 Haarmoni Street 5 (72) Inventor Mayo Mail Rehovot 76410 Benzion Street, Israel 26/1 (72) Inventor Lippa Shmuel Ramat Gan 52281, Rimalt Street / 10 (72) Inventor Smilanski Zev Israel Mayshall 76850 41 F-term (reference) 2F065 AA49 CC25 FF04 JJ03 JJ19 JJ26 QQ08 QQ25 QQ31 2G051 AA65 AB02 AC04 BC05 ED04 5B057 AA03 CA01 CA12 CC03 CE0 DB06 DC16 DB01 DB03 DB02 DB16 5 AA02 BA03 DA02 FA06 FA66 FA69 GA51 HA08 JA16 JA18 KA04

Claims (33)

[Claims] 1. A method of inspecting an object, said reference image for a representative object comprising a representation of a first boundary at least partially vectorized within said image. Generating a reference image, obtaining an image of the object under examination, which includes a representation of a second boundary within the image, and the representation of the second boundary and the at least partly vectorization Comparing the rendered first boundary representation and thereby identifying the defect. 2. The method of claim 1, wherein the comparing step employs a user-selected variable threshold for an acceptable distance between corresponding portions of boundaries in the first representation and the second representation. The method described. 3. A system for image processing, a boundary identification device having a function of generating a representation of boundaries of known elements in an image, and a function of identifying a defect candidate in the image by hardware. And a software defect candidate using the boundary representation to identify at least one false alarm in the output by the software. And an inspector. 4. The system according to claim 3, wherein the boundary discriminator comprises a hardware boundary discriminator capable of generating, by hardware, a representation of a boundary of known elements in the image. 5. The system of claim 3, further comprising a software defect candidate identification device having the ability to identify additional defect candidates in the image by software. 6. The software defect candidate inspector receives, by software, a second output from a software defect candidate identification device and identifies the boundary for identifying at least one false alarm in the second output. The system of claim 5, using a representation. 7. The system of claim 3, wherein the hardware defect candidate identification device employs the representation of the boundary to identify at least some defect candidates. 8. The system of claim 5, wherein the software defect candidate identification device employs the representation of the boundary to identify at least some defect candidates. 9. A system for image processing, the learning subsystem having the function of defining first and second regions in an object under examination, each first region being at least one. A learning subsystem that includes two known critical object elements and the second region does not include such known critical object elements; and a second subsystem based on prior knowledge of the known critical object elements. A defect detector having a function of inspecting the first area using a first procedure and inspecting the second area using a second procedure different from the first procedure. 10. The second procedure is a hardware inspection of a second area having a function of identifying a defect candidate in the second area, and at least one defect candidate is recognized in the second area. Solely comprising a software inspection of the second area of the continuation, having the function of analyzing at least one defect candidate found in the second area and identifying false alarms therein. 9. The system according to item 9. 11. A method for inspecting an object, wherein in a first inspecting step, the position of at least one defect candidate is identified and, for each defect candidate, a contour area at the position is determined. The determination of candidate regions for at least size and shape inspection is based at least in part on the output of the identifying step, and in a second inspection step, each of the candidate regions to identify the defect. Inspecting the area. 12. The method of claim 11, wherein the first step is performed in hardware.
The method described in. 13. The method of claim 11, wherein the second step is implemented in software.
The method described in. 14. The method of claim 11, wherein the determining step comprises determining a size of the candidate region at a known location. 15. The method of claim 11, wherein the second step comprises performing different inspections depending on criteria of the candidate area or the defect candidate. 16. The second step includes performing different inspections depending on characteristics of at least the one defect candidate identified in the identifying step.
The method according to claim 11. 17. The method of claim 11, wherein the second step comprises performing different inspections depending on the functionality of the portion of the object in which the defect candidate resides. 18. The method of claim 11, wherein the second step includes performing different inspections depending on the degree of criticality of functionality of the portion of the object in which the defect candidates are present. 19. A modular image processing system with a user customizable image analysis function for use with a scanner, the image processing engine receiving at least one image data stream to be analyzed from the scanner; At least one executed on said image data by a processing engine
An engine configurator having a function of receiving a sequence of user-customized image analysis function definitions, the image processing engine being dependent on a current definition arriving at the engine configurator from the sequence of definitions. And analyzing at least one channel of image data according to each definition in the sequence of definitions sent to the engine configurator, which includes performing different image analysis functions that differ from each other only by parameters. Have a function. 20. A method for automatically optically inspecting an object, the method comprising: a plurality of object areas for image processing, the object-defined at least one object area, and an article. Defining an area that includes at least one area of interest automatically defined by optical inspection, and providing an image of an area surrounding each area of interest to an image processor; Automatically processing the image of each of the regions surrounding the region of interest to determine its presence. 21. The method of claim 20, wherein each image of the area surrounding the area of interest is smaller than the image of the object. 22. The target area automatically defined by optically inspecting the article includes defect candidates in a pattern formed on the object.
21. The method of claim 20. 23. The object area automatically defined by optically inspecting the article includes predetermined morphological features formed in a pattern on the object. the method of. 24. The method of claim 20, wherein the providing step comprises providing a color image of the region of interest to the image processor. 25. An image processing method, wherein the step of automatically processing is applied to a target area defined by a user, the target area automatically defined by optically inspecting the article. 21. The method of claim 20, comprising applying at least one different image processing method. 26. The step of providing includes identifying a type of defect in the region of interest, and the step of automatically processing includes applying an image processing method suitable for the type of defect in the region of interest. 23. The method of claim 22. 27. The step of providing includes identifying a type of morphological feature in the region of interest, and the step of automatically processing the image suitable for the type of morphological feature in the region of interest. 24. The method of claim 23, comprising applying a processing method. 28. At least one region of interest defined by a user is
The at least one region of interest defined prior to optically inspecting the object in a software defining step and automatically defined by optically inspecting the article is defined in the hardware inspecting step. 21. The method of claim 20, wherein the method is performed in and the automatic processing of each image of the region of interest is performed in a software image processing step. 29. A method for inspecting a ball grid array substrate, the method comprising generating at least one model for at least one feature in a reference image of the ball grid array substrate and storing the model in memory. Acquiring an image of the ball grid array substrate, and determining a predetermined feature for the image of the ball grid array substrate to determine if features within a predetermined region fit the model. Inspecting the exposed area. 30. The method of claim 29, wherein the features include circles. 31. The method of claim 30, wherein the model of the circle includes a center point at a predetermined location and a radius within a predefined tolerance. 32. The method of claim 31, wherein the model parameters are at least partially adjustable in an off-line mode prior to inspection. 33. The method of claim 29, wherein the features include bond pads.