WO2006131883A1 - Visual inspection system and process for electronic modules - Google Patents

Visual inspection system and process for electronic modules Download PDF

Info

Publication number
WO2006131883A1
WO2006131883A1 PCT/IB2006/051802 IB2006051802W WO2006131883A1 WO 2006131883 A1 WO2006131883 A1 WO 2006131883A1 IB 2006051802 W IB2006051802 W IB 2006051802W WO 2006131883 A1 WO2006131883 A1 WO 2006131883A1
Authority
WO
WIPO (PCT)
Prior art keywords
strip
modules
defects
lighting
subsystem
Prior art date
Application number
PCT/IB2006/051802
Other languages
French (fr)
Inventor
Frédéric Ros
Joseph Chardon
Maria De Brito
Original Assignee
Axalto Sa
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Axalto Sa filed Critical Axalto Sa
Publication of WO2006131883A1 publication Critical patent/WO2006131883A1/en

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/89—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles
    • G01N21/8914—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles characterised by the material examined
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/89—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles
    • G01N21/8901—Optical details; Scanning details
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01R—MEASURING ELECTRIC VARIABLES; MEASURING MAGNETIC VARIABLES
    • G01R31/00—Arrangements for testing electric properties; Arrangements for locating electric faults; Arrangements for electrical testing characterised by what is being tested not provided for elsewhere
    • G01R31/28—Testing of electronic circuits, e.g. by signal tracer
    • G01R31/302—Contactless testing
    • G01R31/308—Contactless testing using non-ionising electromagnetic radiation, e.g. optical radiation
    • G01R31/311—Contactless testing using non-ionising electromagnetic radiation, e.g. optical radiation of integrated circuits

Definitions

  • This invention concerns a visual inspection system and process for electronic modules.
  • the invention is in the field of smart card manufacturing. It targets the manufacturing steps related to handling microelectronic strips before the electronic modules are transferred to the card and embedded. More specifically it targets the automatic visual inspection of electronic modules both on their contact face and on their chip face .
  • the problematics involved in the visual inspection of electronic modules are known and are common to all embedders . Inserting non-functional modules into card bodies results in unnecessary operations, heavy financial losses and also in complaints from dissatisfied customers. Furthermore, embedding functional modules damaged by the presence of visual defects such as dents, fingerprints, scratches or micro-deformations, for example, can result in complaints from some customers who find the presence of these defects unacceptable. Now, customer complaints directly affect the card manufacturer' s image and can also lead to financial penalties.
  • This manual inspection takes a long time to get a correct diagnostic.
  • An operator processes on average 2,000 modules an hour.
  • a manual inspection comprises a subjective component, with the result that the quality of visual inspections is not always very reliable.
  • visual inspection outputs vary from one operator to another, so that the quality level of the modules is not consistent.
  • a same operator does not check the modules consistently; his judgment varies particularly based on the type of film, but also based on the length of processing, for fatigue and habit modify his behavior.
  • manual visual inspection does not guarantee a consistent level of quality for the modules to be embedded, which can therefore result in complaints from dissatisfied customers.
  • Inspection machines for the contact face of modules have already been designed. To do this, the machine compares the image of a module with the image of a standard, defect-free module and deduces whether or not there is a defect. This machine is incapable of characterizing the defects; it is limited to a hierarchical approach and can only discriminate between "good” or "bad.” However, this approach is not sufficient, for it does not take into account the different quality requirements of different customers.
  • the industrial vision systems currently on the market are too general to provide a satisfactory response to this need. Indeed, due to their generality, they cannot incorporate knowledge of the product. Furthermore, the complexity of the diagnostic is such that an assembly of standard image processing techniques is insufficient. While the interest of a module inspection machine is real for embedders, the problematics related to automatic diagnostics are complex in practice. Indeed, the detection of defects by humans is difficult to translate into an automatic system. Furthermore, the automatic system must have great flexibility, but still be easy to use in order to adapt to the wide variety of existing products, such as the geometry or the texture of the strips, as well as to the quality requirements of different customers. Finally, the system must allow rapid processing of the modules to ensure good profitability and to be integratable in a production chain.
  • the technical problem addressed by this invention consists in proposing an automatic visual inspection system for electronic modules arranged on a microelectronic strip that would allow rapid, independent or simultaneous inspection of the contact face and the chip face of electronic modules, deliver reliable diagnostics resulting in a consistent level of quality, characterize defects effectively and that would be flexible to adapt to the different types of strips and to the different quality level requirements demanded by customers .
  • the system comprises a first subsystem dedicated to inspecting the contact face of the modules.
  • This subsystem comprises:
  • each camera being able to view the contact face of each module arranged on a first row of modules, respectively on a second row of modules, of the microelectronic strip,
  • the system also comprises a second subsystem dedicated the inspection of the chip face of the modules.
  • This second subsystem comprises : > at least two cameras, each camera being able to view the chip face of each module positioned on a first row of modules, respectively on a second row of modules, of the microelectronic strip, ⁇ a lighting bank for the microelectronic strip, and > calculation means able to deliver, based on the images captured by the cameras, a diagnostic concerning the presence or absence of defects and a characterization of the defects.
  • the system also comprises a strip support common to both subsystems .
  • the system is adaptable since it is capable of handling any type of microelectronic strip. This system also offers a much faster consistent module processing rate than humans can offer, since it is around 40,000 modules per hour. The system also guarantees a consistent level of quality and adjusts to the level of quality required by a customer and therefore eliminates under- quality or over-quality production. Finally, given all the advantages mentioned, processing costs are reduced considerably compared to manual inspection.
  • the invention also concerns an automatic visual inspection process for electronic modules arranged on a microelectronic strip.
  • This method is particularly noteworthy in that it includes the following steps:
  • the process according to the invention has great flexibility with respect to apprehending the level of quality required for a given product.
  • the reasoning leading to the diagnosis is modeled after the reasoning of operators .
  • figure IB a schematic view of the respective position of the cameras of the system of figure IA with respect to a microelectronic strip
  • figure 2A a schematic view of means for lighting the reference indices used in the system of figure IA
  • FIG. 2B a schematic view of means for lighting the faces of the modules, used in the system in figure IA, - figure 3, a simplified diagram of the internal layout of the calculation means and of the processing interfaces making up the core of the system of figure IA,
  • FIG. 4 a flow chart schematizing the main steps of a visual inspection process according to the invention, - figure 5, a view of the inspection zone of a contact face broken down into basic zones,
  • the invention proposes an automatic inspection system and process for the contact face and the chip face of electronic modules . Thanks to the preliminary parametering of this system, it is possible to inspect either of the faces independently or to inspect both of them simultaneously.
  • the inspection of the contact face consists in detecting the presence of defects such as dents, scratches or traces of contamination, for example, and in characterizing these defects.
  • the inspection of the chip face consists in determining whether the protective resin has turned up or has shifted and, if this is the case, in making sure that the authorized angle of rotation or the authorized displacement distance have not been exceeded. It also consists in making sure that the resin has not extended beyond an allowed overflow area or that a minimum of stress deviators have been filled by the resin. Finally, it consists in displaying the presence of defects that have appeared in the protective resin, such as dents or bubbles, for example, and in characterizing these defects. The appearance of the protective resin is not consistent because the strips do not have the same texture or the same brightness.
  • the automatic visual inspection system for electronic modules as schematized in figure IA includes two independent subsystems Sl and S2.
  • a first subsystem Sl is dedicated to the inspection of the contact face of the modules, and the other S2 is dedicated to the inspection of the chip face of the modules.
  • Each of the two subsystems respectively comprises: two cameras 11, 12 and 13, 14 able to display respectively the face of a module arranged in one of the two rows of modules making up the microelectronic chip.
  • Each subsystem Sl, S2 also comprises a lighting bank 15 and 16 for the microelectronic strip and calculation means capable of delivering, based on the images captured by the cameras, a diagnostic concerning whether or not there are defects and a characterization of the defects detected.
  • calculation means consist, for each subsystem, of a computer 40 and 45, for example, as shown in figure 3.
  • Each calculation device 40 and 45 of each subsystem Sl, S2 is connected to two cameras 11, 12, and 13, 14.
  • the system also comprises a strip support 17 common to both subsystems Sl, S2.
  • the microelectronic strip travels step by step in the direction indicated by the arrow F in figure IA.
  • the two cameras 11, 12, and 13, 14 of each subsystem Sl, S2 are anchored in fixed position on a same common support 18, and 19. Furthermore, they are preferably mounted in a pipeline architecture so that the images captured by the two cameras 11, 12, and 13, 14, can be processed simultaneously.
  • the two cameras are standard low-resolution cameras, because they make it possible to obtain a sufficient signal, that is, to capture an image containing enough details to reveal the presence of defects .
  • Figure IB is a diagram of a microelectronic strip 20 and of the position of the cameras 11, 12 and 13, 14 with respect to the electronic modules 21 to be inspected.
  • the strip comprises along its length, two rows 22, 23 of electronic modules represented by rectangles in figure IB.
  • the strip 20 also comprises through-holes forming reference indexes 24 situated along its longitudinal edges. The presence of these reference 24 is very important for ensuring proper alignment of the strip and for allowing good definition of the inspection areas by the cameras.
  • the lighting bank 15 and 16, used in each subsystem Sl, S2 makes it possible both to align the microelectronic strip and to allow effective visual inspection.
  • the lighting bank includes a first mean 25, respectively 27, of backlighting dedicated to lighting the reference indexes 24, and a second mean 26, respectively 28, of lighting dedicated to revealing visual defects on the contact face or the chip face of a module.
  • a control system for the two lighting devices also makes it possible to determine the lighting context of a new strip and to adjust the lighting of the indexes and of the strip with great flexibility, while taking into consideration the light integration time of the cameras .
  • FIG. 2A is a schematic representation of the means 25 and 27 of backlighting dedicated to the reference indexes.
  • This means comprises a diffusion plate 30 to diffuse the light.
  • This diffusion plate is made of plexiglas, for example .
  • a filter plate 31 is equipped on either side with opaque elements 32 and through-holes 33.
  • the filter plate 31 filters the light on the surfaces of the modules 21 to be inspected and allows the light to pass through the reference indexes 24 of the strip 20, thanks to the through-holes 33.
  • a lighting plate 34 made of electroluminescent diodes, for example, makes it possible to light the diffusion plate 30, the filter plate 31 and the strip uniformly. This means of backlighting thus allows optimal lighting of the indexes suited to all strips and does not disturb the appearance of the modules to be inspected.
  • the means 26 and 28 of lighting dedicated to the visual defects of the modules include a mean 35 for the axial diffusion of the light that makes it possible to light area of the two cameras 11, 12 and 13, 14 in a pipeline configuration.
  • a semi-reflective mirror 36 sends the light beams toward the cameras 11, 12 and toward the modules 21 to be inspected.
  • the first mean 25, 27 of back-lighting the indexes lights a face of the strip opposite the face lit by the second mean 26, 28 of lighting dedicated to visual defects .
  • Figure 3 is a simplified schematic representation of the internal layout of the system, and, more particularly, the calculation means and their connections with the cameras and with a central management unit.
  • the two cameras 11, 12 of the first subsystem Sl communicate with a first means of calculation 40.
  • the two cameras 13, 14 of the second subsystem S2 communicate with a second mean of calculation 45.
  • These means of calculation 40, 45 use software that use algorithms dedicated to detecting visual defects.
  • Each mean of calculation 40 and 45 is connected to a dedicated management device 51 and 52 comprising man/machine interfaces as well as the database necessary for parametering and then detecting the presence of visual defects on the two faces of the modules analyzed and for characterizing these defects.
  • Each calculation mean 40, 45 and its associated management means 51, 52 may, for example, consist of a computer.
  • the first mean of calculation 40 for example, collects the data from the second mean of calculation 45, then transmits all the data resulting from the calculations to a central management unit 50.
  • This management unit 50 establishes a single link with the strip handling machine into which the system is integrated, to synchronize the advancement of the strip and the delivery of the different diagnostics. With each advancement, each camera captures the image of a different module on the strip and makes it possible to get an intermediate result thanks to the calculation mean 40, 45 and the dedicated management mean 51, 52.
  • the role of the central management unit 50 is therefore to concatenate the information from each calculation unit 40, 45 to make them intelligible for the strip handling machine, particularly by managing the pipeline architecture of the cameras.
  • the module is either good, bad or very bad. A succession of very bad modules results in a tape connector, while a bad module results in the invalidation of the module.
  • the automatic visual inspection system just described allows the use of an automatic visual inspection process for electronic modules whose principal steps are schemtaized on the flow chart in figure 4.
  • the process includes two phases: a preliminary parametering phase 100, and a diagnostic phase 200. These two phases are implemented by software stored on at least one recording medium like a CD-Rom or a hard disk, or even a transmittable medium like an electrical, optical or radio signal.
  • the software is used by the central management unit 50, by the management means 51,52 and by the calculation means 40, 45 to produce a final diagnostic .
  • the steps described in reference to figure 4 correspond to the steps performed following the display, by a single camera 11 of a first subsystem Sl, of modules positioned in the same row 22 of a microelectronic strip 20.
  • the overall detection of defects is primarily linked to the notion of visibility and, to a lesser extent, to the notion of size or shape.
  • the local detection of defects that is, detection over portions of the surface of a module, combines in balanced fashion the notion of size and shape and the notion of visibility.
  • a very thin, long scratch will be detected immediately due to its geometric shape and to a lesser extent to its size and its visibility, since these two factors are not predominant in this case.
  • defect detection is more qualitative than quantitative in the sense that an operator never characterizes a defect by measurements, but rather by general impressions such as "small,” “fine,” “unmissable,” etc .
  • the characterization of defects therefore involves defect modeling based on a digitization of the reasoning of operators that privileges the qualitative interpretation of the defect over the quantitative interpretation of the defect. Contrary to "unambiguous” and “measurable” defects found, for example, on the surfaces of laser disks or CD-Roms because the surface is very shiny, uniform and identical from one product to another, the detection and characterization of defects on the modules are more difficult. Indeed, a same "geometric" defect is not perceived in the same way from one strip to another .
  • a typology consisting of three categories of local defects is defined: a first category, called scratch, concerns slender and more or less "visible" defects; a second category, called mark, concerns more compact defects but that are less visible; and finally, a third category called contamination concerns defects that are also compact but even less visible than in the mark category.
  • a typology consisting of two categories is also suited to an overall defect detection approach.
  • the mark and contamination categories are combined.
  • a defect has been identified locally by its geometry, it may not be identified as sufficiently serious with respect to its visibility and will therefore not be identified.
  • it may be part of an overall detection of a defective module.
  • a first step 110 of the parametering phase 100 consists in modeling the geometry of the microelectronic strip under inspection.
  • the geometry of a strip corresponds to the manufacturer' s data with all the associated basic dimensions. Modeling the geometry of the strip depends on the preliminary masking and breakdown of the inspection zone, corresponding to the surface of a module, into basic zones. These two elements have the indexes 24 of the strip as common reference.
  • the notion of a mask makes it possible to consider a module as a unit and to inhibit diagnostication of the masked areas such as, for example, the inter-track areas, etc. This masking process simplifies the parametering phase. Indeed, without this automatic masking process, the user must himself select as many areas of interest as parts not containing any inter- track areas. This type of manual selection is slow, presents risks of errors, and is not repeatable.
  • a mask is produced once per strip geometry and based on the same reference as the inspection zone, that is, based on the index holes 24, which means that the mask parametered in this fashion can be transferred to the images of the modules being inspected displayed by the cameras.
  • the breakdown into basic zones is very useful and relevant: it allows a macroscopic representation of the module and greater processing efficiency and speed. It also makes it possible to inhibit the processing of certain basic zones if they are not important for a given customer.
  • Figure 5 illustrates this step. It represents the image of a contact face of a module divided into basic shaded zones each in reference to the indexes 24 of the strip, and whose inter-track areas 29, in black, that are not to be inspected, are masked.
  • the masks are essentially positioned over the stress deviators, that is, around the zone protected by the resin, and the inspection area, situated inside these stress deviators is divided into basic zones.
  • strips by a same manufacturer may have the same geometry but a different visual appearance, that is, a different appearance due to the manufacturing process or the material making up the strip.
  • Humans have the ability to disregard uncharacteristic elements of the module due to their ability to select the pertinent information to be analyzed.
  • the geometry of the strips namely the basic dimensions of the strips and the more or less complex positions and shapes of the inter-tracks of the modules vary greatly from module to module.
  • the creation of a new strip must, however, remain simple and quick, since there is no interest in inspecting intertracks.
  • there may be several textures ranging from very dull to very shiny with different materials, such as palladium or gold, for example, thus requiring an ability to adapt to the light cast onto the module.
  • the following step 111 therefore consists in modeling the appearance of the strip.
  • the lighting bank 15, respectively 16 is also controlled in order to obtain an optimum optical configuration, that is, a configuration that avoids image saturation while allowing good visibility of the indexes 24.
  • the parameters that translate the geometry of the strips are stored in a first recording medium such as a database, and the parameters connected to the appearance of the strips are stored in a second recording medium, such as a database.
  • a first recording medium such as a database
  • a second recording medium such as a database
  • a quality level is defined taking into account the customer's requirements (step 112).
  • a quality level corresponds to the different quality criteria that define, for a given production, what a good or bad module is. It is based on the appearance and geometric properties of the strips .
  • the parameters translating the quality level required are themselves also preferably stored in a third database that is managed directly by the operator using the visual inspection machine, that is, by an operator interface associated with the management means 51 and 52, of the subsystem Sl and S2.
  • the customer' s requirements are specified, for example, through the intermediary of the system user by means of a Weg 6 interface as represented in figure 6, for defining a defect through a scratch.
  • This interface allows the user to specify, in gray, all the configurations that correspond to a defect and in black, the configurations that correspond to an accepted level of quality.
  • the notion of "low” and "high” criticality expresses how visibility must be taken into consideration with geometric attributes defined in the administrator interface .
  • a first step 210 of this diagnostic phase 200 consists in checking and potentially correcting the alignment of the microelectronic strip.
  • the automatic placement of the strips in a step by step mode is never perfectly repeatable, especially at a high rate of speed, and therefore necessitates an alignment procedure that requires precision and speed.
  • This alignment procedure must be effective regardless of the strips and their appearance. It effectively makes it possible to superimpose the attributes of the reference image perfectly over those of the image analyzed.
  • This step is based on detecting the position of the reference indexes lit by the dedicated back-lighting means 25, 27. Then, based on the detected position of the reference indexes of the strip to be analyzed, the system defines in relative fashion the inspection zones Z corresponding respectively to the surface of a module to be inspected (step 211) .
  • each inspection area Z to be inspected is broken down into basic areas ZeI to Zemax (step 212), and a mask is applied to the areas that are not to be inspected, such as the intertracks, for example.
  • step 213 For each basic zone Ze (step 213) of a same inspection zone Z, different steps are performed to detect the presence of local defects by analyzing in particular the difference in contrast and the difference in texture.
  • a local defect is characterized by very simplistic three-parameter laws for scratches and two-parameter laws for marks and contamination.
  • the definition accessible to the system user based on the parameters defined by him in the sexual interface represented in figure 6, is as follows: "If the defect is of average length, covers a small area and has critical visibility, then the module is bad.” Similarly, for marks and contamination, the accessible definition is: "If the defect has a large surface area and slight visibility, then the module is bad.”
  • each basic zone Ze is therefore analyzed through a contrast filter (step 214) then through a texture filter (step 215) .
  • each basic zone is identified through a contrast filter.
  • a contrast differential makes it possible to translate the notion of average visibility.
  • the visual inspection system thus detects, in each basic zone Ze analyzed, whether there is a difference in contrast related to the presence of a defect resulting in a brightness differential.
  • Large defects such as a dent on the module or a deep scratch, for example, can be very easily characterized by the notion of contrast.
  • small defects or certain scratches are difficult to detect through the notion of contrast, since the variability of the contrast that these defects create is easy to confuse with the natural variability of the contrast between two good modules. This highlights an essential limitation of the standard systems on the market based on even a sophisticated segmentation of the distribution of brightness.
  • a statistical study of a representative panel of strips shows that a local zone measuring from 1 to 2 mm 2 provides both a stable gray level distribution from one module to another, and is sufficiently small to reveal small defects. This study also shows that the distribution is related to a substantially Gaussian distribution. Considering the mean and the standard deviation as characteristic elements makes it possible to construct a simple approach to the notion of visibility.
  • the "distance" concerning the standard deviations corresponds to a Euclidian distance even though the distance of the means is a Euclidian distance offset by a shift revealing the natural dispersion of the means for a good product, this being derived from a preliminary statistical calculation.
  • Each basic zone is therefore assigned a good or bad status depending on the distribution of gray values.
  • Bad status is associated with an overall level of severity and a quantitative estimate of the defective surface of the basic zone Ze. This differentiation in contrast reveals the presence of mark or contamination defects. However, it is still not sufficient to characterize scratches affecting the texture aspect. It is, however, the combination of these notions that expresses the notion of visibility .
  • a subsequent step 215 therefore consists in analyzing each basic zone Ze, that has been assigned a good status by the first contrast filter (step 214), with a texture filter.
  • the texture filter reveals defects not seen by the contrast filter, generally defects such as "small scratches” or “small dents” that are not very obvious despite their presence. This step is not routine. Indeed, if a contrast defect has been detected, it is not necessary to add unnecessary processing time, and we then go directly to the next step 216.
  • the visual inspection system thus applies a texture filter to detect any difference in gradient in each basic zone analyzed.
  • Weber's laws express the eye's sensitivity to differences in gray values according to value ranges.
  • the texture of a module tends to be continuous and repeatable, which allows characterization using dedicated image processing tools like the coocurrence matrices that are very well known in the image processing literature.
  • a cooccurrence matrix averaging two directions of perpendicular images with adaptively selected classes CO- Cn, characterizes a good module and makes it possible to reveal, thanks to two levels of visibility indicators, the presence of defects that could not be detected on the basis of contrast.
  • Figure 8 illustrates this type of cooccurrence matrix.
  • CO to C8 are automatically calculated gray value classes.
  • Each box Cij represents the standardized occurrence number between a class Ci and the class Cj, according to two orthogonal directions and a distance of 1 pixel. If, for example, there are 200 times a Ci level pixel positioned at a distance of one Cj level pixel, then the class Cj will be equal to 200.
  • the representative texture of a defect-free module is mostly marked by the boxes formed by C3, C4, C5, which are revealed by dotted lines. The occurrence level in the other boxes reveals the presence of defects and also allows a characterization of the criticality.
  • Each basic zone Ze of an inspection zone Z is thus analyzed through contrast and texture filters until all the basic zones Ze have been analyzed, according to steps 214 to 217.
  • a related filling algorithm applied to the adjacent basic zones then makes it possible to clusterize the related zones whose status has been declared poor (step 218) .
  • This consolidation work for related defective zones is very fast and makes it possible to define a macroscopic view of the defects that corresponds more to human interpretability .
  • Each group of related defects belonging to adjacent basic zones Ze is then characterized (steps 219 and 220) .
  • a first decision level allows the characterization of local related defects (steps 219)
  • a second decision level allows the characterization of the related defects at a global level (step 220) .
  • the characterization process tries to imitate intuitive human behavior. The eye is very sensitive to a specific defect like a small dent, that is clearly visible, but overlooks it when it is less visible.
  • the first type of decision (219) at the local level, makes it possible to determine a defect and also to characterize it, for example, a scratch or a dent.
  • the eye is generally sensitive to a "contamination" of defects without being able to make a clear distinction concerning the characteristics of the defect.
  • the operator thus deduces from this that the module is "dirty.”
  • the second type of decision (220) on the global level, makes it possible to determine this type of contamination. The result of these two decision levels leads to a diagnostic concerning the good or bad status of the module
  • each group of related defects is analyzed respectively by three decision systems for marks, contamination and scratches.
  • the geometry and the degree of visibility of a group of related defects then makes it possible to designate a good / bad status for the module and to determine whether or not it belongs to one of the three characteristic categories.
  • the dimensions, that is, the length and the surface area, of the related defects are added together.
  • the system Based on the estimated dimensions of the related defects, that is, based on the length, the surface area, the visibility, etc., the system compares them with the customer' s quality requirements saved previously when the system was parametered (step 221) . If the related group corresponds to a defect deemed unacceptable by the customer, the module containing this defect is then diagnosed as bad, even though locally the diagnostic was good.
  • the system also provides data concerning the qualification of defects, such as a local, global defect, a scratch, a mark, contamination, etc., which makes it possible to establish a global PARETO by production lot
  • step 222 The PARETOS thus created during the inspection of the modules of a production lot make it possible, among other things, to improve certain parameters connected to a given production run.
  • An end of strip is characterized by the presence of a white or transparent plastic film instead of modules.
  • a double criterion involving the texture and the brightness of the inspection surface area guarantees the detection of the presence or absence of a strip.
  • Diagnostic steps 212 to 222 are thus repeated throughout the step-by-step run P of the microelectronic strip until the end of the strip is detected.
  • the system that has just been described also provides means for easily parametering the system. Using an image bank of good modules stored in the machine, it is possible to extract statistical indices tied to the natural variability between modules and to their texture.
  • the modes of embodiment that have just been described are merely examples for the sake of illustration and the invention is not limited to them. Indeed, numerous variants of the modes of embodiment described above can be envisioned while remaining with the framework of the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Textile Engineering (AREA)
  • Immunology (AREA)
  • Chemical & Material Sciences (AREA)
  • Computer Hardware Design (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Electromagnetism (AREA)
  • Toxicology (AREA)
  • General Engineering & Computer Science (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)

Abstract

The invention concerns a visual inspection system and process for electronic modules. More specifically it concerns the automatic visual inspection of electronic modules both on their contact face and on their chip face, before they are transferred to a card. The inspection of the contact face and of the chip face may be performed simultaneously or independently. The system comprises a first subsystem (Sl) dedicated to inspecting the contact face of the modules and a second, independent subsystem (S2), dedicated to inspecting the chip face of the modules. Each subsystem (Sl ; S2) comprises at least two cameras (11, 12 ; 13, 14), each camera being able to display the face dedicated to it of each module (21) positioned in a row (22) of modules of the microelectronic strip (20); a lighting bank (15 ; 16) for the microelectronic strip (20), and calculation means (40 ; 45) capable of delivering, based on the images captured by the cameras (11, 12 ; 13, 14), a diagnostic concerning whether or not there are defects present and a characterization of the defects .

Description

Visual inspection system and process for electronic modules
This invention concerns a visual inspection system and process for electronic modules.
The invention is in the field of smart card manufacturing. It targets the manufacturing steps related to handling microelectronic strips before the electronic modules are transferred to the card and embedded. More specifically it targets the automatic visual inspection of electronic modules both on their contact face and on their chip face . The problematics involved in the visual inspection of electronic modules are known and are common to all embedders . Inserting non-functional modules into card bodies results in unnecessary operations, heavy financial losses and also in complaints from dissatisfied customers. Furthermore, embedding functional modules damaged by the presence of visual defects such as dents, fingerprints, scratches or micro-deformations, for example, can result in complaints from some customers who find the presence of these defects unacceptable. Now, customer complaints directly affect the card manufacturer' s image and can also lead to financial penalties.
Even though the electrical functionality check is performed entirely by dedicated test machines, the visual inspection of the modules is currently a completely manual process.
This manual inspection takes a long time to get a correct diagnostic. An operator processes on average 2,000 modules an hour. Furthermore, a manual inspection comprises a subjective component, with the result that the quality of visual inspections is not always very reliable. In particular, visual inspection outputs vary from one operator to another, so that the quality level of the modules is not consistent. Likewise, a same operator does not check the modules consistently; his judgment varies particularly based on the type of film, but also based on the length of processing, for fatigue and habit modify his behavior. As a result, manual visual inspection does not guarantee a consistent level of quality for the modules to be embedded, which can therefore result in complaints from dissatisfied customers.
Inspection machines for the contact face of modules have already been designed. To do this, the machine compares the image of a module with the image of a standard, defect-free module and deduces whether or not there is a defect. This machine is incapable of characterizing the defects; it is limited to a hierarchical approach and can only discriminate between "good" or "bad." However, this approach is not sufficient, for it does not take into account the different quality requirements of different customers.
The industrial vision systems currently on the market are too general to provide a satisfactory response to this need. Indeed, due to their generality, they cannot incorporate knowledge of the product. Furthermore, the complexity of the diagnostic is such that an assembly of standard image processing techniques is insufficient. While the interest of a module inspection machine is real for embedders, the problematics related to automatic diagnostics are complex in practice. Indeed, the detection of defects by humans is difficult to translate into an automatic system. Furthermore, the automatic system must have great flexibility, but still be easy to use in order to adapt to the wide variety of existing products, such as the geometry or the texture of the strips, as well as to the quality requirements of different customers. Finally, the system must allow rapid processing of the modules to ensure good profitability and to be integratable in a production chain.
Furthermore, while there are numerous image processing algorithms available to solve problems connected with texture and shape recognition problems, they are too unwieldy for real-time processing in non- specialized configurations. One of the problems to overcome is therefore offering very effective algorithms that can be used at very high processing rates on the order of at least 40,000 modules / hour, in basic environments and thus at a very restricted cost.
So, the technical problem addressed by this invention consists in proposing an automatic visual inspection system for electronic modules arranged on a microelectronic strip that would allow rapid, independent or simultaneous inspection of the contact face and the chip face of electronic modules, deliver reliable diagnostics resulting in a consistent level of quality, characterize defects effectively and that would be flexible to adapt to the different types of strips and to the different quality level requirements demanded by customers .
The technical problem raised is solved, according to this invention, by the fact that the system comprises a first subsystem dedicated to inspecting the contact face of the modules. This subsystem comprises:
> at least two cameras, each camera being able to view the contact face of each module arranged on a first row of modules, respectively on a second row of modules, of the microelectronic strip,
^ a lighting bank for the microelectronic strip, and
> calculation means able to deliver, based on the images captured by the cameras, a diagnostic concerning the presence or absence of defects and a characterization of the defects. According to another advantageous feature, the system also comprises a second subsystem dedicated the inspection of the chip face of the modules. This second subsystem comprises : > at least two cameras, each camera being able to view the chip face of each module positioned on a first row of modules, respectively on a second row of modules, of the microelectronic strip, ^ a lighting bank for the microelectronic strip, and > calculation means able to deliver, based on the images captured by the cameras, a diagnostic concerning the presence or absence of defects and a characterization of the defects.
According to yet another advantageous feature, the system also comprises a strip support common to both subsystems .
The system is adaptable since it is capable of handling any type of microelectronic strip. This system also offers a much faster consistent module processing rate than humans can offer, since it is around 40,000 modules per hour. The system also guarantees a consistent level of quality and adjusts to the level of quality required by a customer and therefore eliminates under- quality or over-quality production. Finally, given all the advantages mentioned, processing costs are reduced considerably compared to manual inspection.
Correlatively, the invention also concerns an automatic visual inspection process for electronic modules arranged on a microelectronic strip. This method is particularly noteworthy in that it includes the following steps:
- parametering the strip to be analyzed based on its geometry, its appearance and a required quality level, - defining inspection zones with respect to reference indices, each inspection zone corresponding respectively to the surface of a module to be analyzed, and - for each inspection zone:
- detecting the presence of a defect,
- estimating the dimensions of the defect detected,
- characterizing the defect detected and - issuing a diagnostic concerning the status of the module analyzed.
The process according to the invention has great flexibility with respect to apprehending the level of quality required for a given product. The reasoning leading to the diagnosis is modeled after the reasoning of operators .
Other specific aspects and advantages of the invention will emerge from the following non-limiting description given for the purposes of illustration in reference to the appended figures, which represent:
- figure IA, a sectional schematic view of an inspection system according to the invention,
- figure IB, a schematic view of the respective position of the cameras of the system of figure IA with respect to a microelectronic strip,
- figure 2A, a schematic view of means for lighting the reference indices used in the system of figure IA,
- figure 2B, a schematic view of means for lighting the faces of the modules, used in the system in figure IA, - figure 3, a simplified diagram of the internal layout of the calculation means and of the processing interfaces making up the core of the system of figure IA,
- figure 4, a flow chart schematizing the main steps of a visual inspection process according to the invention, - figure 5, a view of the inspection zone of a contact face broken down into basic zones,
- figure 6, a diagram of a discursive interface directly usable by an operator and qualitatively translating the numerical values of the length and size of scratches,
- figure 7, a possible example of a visibility matrix connected to the notion of contrast,
- figure 8, a possible example of a co-occurrence matrix linked to the notion of texture,
- figure 9, a possible example of an administrator interface translating the notions of size and length of scratches defined in the operator interface of figure 6 into numerical values. The invention proposes an automatic inspection system and process for the contact face and the chip face of electronic modules . Thanks to the preliminary parametering of this system, it is possible to inspect either of the faces independently or to inspect both of them simultaneously.
The inspection of the contact face consists in detecting the presence of defects such as dents, scratches or traces of contamination, for example, and in characterizing these defects. The inspection of the chip face consists in determining whether the protective resin has turned up or has shifted and, if this is the case, in making sure that the authorized angle of rotation or the authorized displacement distance have not been exceeded. It also consists in making sure that the resin has not extended beyond an allowed overflow area or that a minimum of stress deviators have been filled by the resin. Finally, it consists in displaying the presence of defects that have appeared in the protective resin, such as dents or bubbles, for example, and in characterizing these defects. The appearance of the protective resin is not consistent because the strips do not have the same texture or the same brightness. Only the introduction of knowledge concerning the product's geometry and the use of heuristics linked to the texture of the resin make it possible to provide a consistent diagnostic for all the configurations. The case of an FCI T2 strip configuration sold by FCI is very significant. This strip is, in fact, very dark, and the distribution of brightness values partially overlaps the resin distribution in the most standard case.
The following description deals essentially with the inspection of the contact face. However, we must not forget that the system and the process described are also suited to inspecting the chip face.
The automatic visual inspection system for electronic modules as schematized in figure IA includes two independent subsystems Sl and S2. A first subsystem Sl is dedicated to the inspection of the contact face of the modules, and the other S2 is dedicated to the inspection of the chip face of the modules. Each of the two subsystems respectively comprises: two cameras 11, 12 and 13, 14 able to display respectively the face of a module arranged in one of the two rows of modules making up the microelectronic chip. Each subsystem Sl, S2 also comprises a lighting bank 15 and 16 for the microelectronic strip and calculation means capable of delivering, based on the images captured by the cameras, a diagnostic concerning whether or not there are defects and a characterization of the defects detected. These calculation means consist, for each subsystem, of a computer 40 and 45, for example, as shown in figure 3. Each calculation device 40 and 45 of each subsystem Sl, S2 is connected to two cameras 11, 12, and 13, 14. The system also comprises a strip support 17 common to both subsystems Sl, S2. The microelectronic strip travels step by step in the direction indicated by the arrow F in figure IA.
Preferably, the two cameras 11, 12, and 13, 14 of each subsystem Sl, S2 are anchored in fixed position on a same common support 18, and 19. Furthermore, they are preferably mounted in a pipeline architecture so that the images captured by the two cameras 11, 12, and 13, 14, can be processed simultaneously. The two cameras are standard low-resolution cameras, because they make it possible to obtain a sufficient signal, that is, to capture an image containing enough details to reveal the presence of defects .
Figure IB is a diagram of a microelectronic strip 20 and of the position of the cameras 11, 12 and 13, 14 with respect to the electronic modules 21 to be inspected. The strip comprises along its length, two rows 22, 23 of electronic modules represented by rectangles in figure IB. The strip 20 also comprises through-holes forming reference indexes 24 situated along its longitudinal edges. The presence of these reference 24 is very important for ensuring proper alignment of the strip and for allowing good definition of the inspection areas by the cameras. Two cameras 11, 12 of the first subsystem Sl, positioned below the strip, for example, inspect the contact face of the modules, while two other cameras 13, 14 of the second subsystem S2 positioned above the strip 20 for example, inspect the chip face of the modules. All the cameras are staggered with respect to one another not only for reasons of mechanical integration in a strip handling machine, but also so that the visual field of each of them is not disturbed by the environment of the others .
The lighting bank 15 and 16, used in each subsystem Sl, S2 makes it possible both to align the microelectronic strip and to allow effective visual inspection. To achieve this, the lighting bank includes a first mean 25, respectively 27, of backlighting dedicated to lighting the reference indexes 24, and a second mean 26, respectively 28, of lighting dedicated to revealing visual defects on the contact face or the chip face of a module. A control system for the two lighting devices, not shown in figure IA, also makes it possible to determine the lighting context of a new strip and to adjust the lighting of the indexes and of the strip with great flexibility, while taking into consideration the light integration time of the cameras .
Figure 2A is a schematic representation of the means 25 and 27 of backlighting dedicated to the reference indexes. This means comprises a diffusion plate 30 to diffuse the light. This diffusion plate is made of plexiglas, for exemple . A filter plate 31 is equipped on either side with opaque elements 32 and through-holes 33. Thus, the filter plate 31 filters the light on the surfaces of the modules 21 to be inspected and allows the light to pass through the reference indexes 24 of the strip 20, thanks to the through-holes 33. A lighting plate 34 made of electroluminescent diodes, for example, makes it possible to light the diffusion plate 30, the filter plate 31 and the strip uniformly. This means of backlighting thus allows optimal lighting of the indexes suited to all strips and does not disturb the appearance of the modules to be inspected.
The means 26 and 28 of lighting dedicated to the visual defects of the modules, as schematically represented in figure 2B, include a mean 35 for the axial diffusion of the light that makes it possible to light area of the two cameras 11, 12 and 13, 14 in a pipeline configuration. A semi-reflective mirror 36 sends the light beams toward the cameras 11, 12 and toward the modules 21 to be inspected. Thus, the first mean 25, 27 of back-lighting the indexes lights a face of the strip opposite the face lit by the second mean 26, 28 of lighting dedicated to visual defects . Figure 3 is a simplified schematic representation of the internal layout of the system, and, more particularly, the calculation means and their connections with the cameras and with a central management unit. The two cameras 11, 12 of the first subsystem Sl communicate with a first means of calculation 40. The two cameras 13, 14 of the second subsystem S2 communicate with a second mean of calculation 45. These means of calculation 40, 45 use software that use algorithms dedicated to detecting visual defects. Each mean of calculation 40 and 45 is connected to a dedicated management device 51 and 52 comprising man/machine interfaces as well as the database necessary for parametering and then detecting the presence of visual defects on the two faces of the modules analyzed and for characterizing these defects. Each calculation mean 40, 45 and its associated management means 51, 52 may, for example, consist of a computer. The first mean of calculation 40, for example, collects the data from the second mean of calculation 45, then transmits all the data resulting from the calculations to a central management unit 50. This management unit 50 establishes a single link with the strip handling machine into which the system is integrated, to synchronize the advancement of the strip and the delivery of the different diagnostics. With each advancement, each camera captures the image of a different module on the strip and makes it possible to get an intermediate result thanks to the calculation mean 40, 45 and the dedicated management mean 51, 52.
The role of the central management unit 50 is therefore to concatenate the information from each calculation unit 40, 45 to make them intelligible for the strip handling machine, particularly by managing the pipeline architecture of the cameras. As for the diagnostics supplied to the machine, the module is either good, bad or very bad. A succession of very bad modules results in a tape connector, while a bad module results in the invalidation of the module.
The automatic visual inspection system just described allows the use of an automatic visual inspection process for electronic modules whose principal steps are schemtaized on the flow chart in figure 4.
The process includes two phases: a preliminary parametering phase 100, and a diagnostic phase 200. These two phases are implemented by software stored on at least one recording medium like a CD-Rom or a hard disk, or even a transmittable medium like an electrical, optical or radio signal. The software is used by the central management unit 50, by the management means 51,52 and by the calculation means 40, 45 to produce a final diagnostic . To simplify the explanations, the steps described in reference to figure 4 correspond to the steps performed following the display, by a single camera 11 of a first subsystem Sl, of modules positioned in the same row 22 of a microelectronic strip 20. We must not forget that a second camera 12 of the same subsystem Sl displays the modules of the second row 23 of the strip 20, and that because the two cameras 11, 12 are mounted according to a pipeline architecture, the images displayed are processed simultaneously by the calculation mean 40 and its dedicated management mean 51.
The overall detection of defects, that is, detection over the entire surface of a module, is primarily linked to the notion of visibility and, to a lesser extent, to the notion of size or shape. On the other hand, the local detection of defects, that is, detection over portions of the surface of a module, combines in balanced fashion the notion of size and shape and the notion of visibility. Thus, for example, a very thin, long scratch will be detected immediately due to its geometric shape and to a lesser extent to its size and its visibility, since these two factors are not predominant in this case.
For an operator, defect detection is more qualitative than quantitative in the sense that an operator never characterizes a defect by measurements, but rather by general impressions such as "small," "fine," "unmissable," etc .
The characterization of defects therefore involves defect modeling based on a digitization of the reasoning of operators that privileges the qualitative interpretation of the defect over the quantitative interpretation of the defect. Contrary to "unambiguous" and "measurable" defects found, for example, on the surfaces of laser disks or CD-Roms because the surface is very shiny, uniform and identical from one product to another, the detection and characterization of defects on the modules are more difficult. Indeed, a same "geometric" defect is not perceived in the same way from one strip to another .
Thus a typology consisting of three categories of local defects is defined: a first category, called scratch, concerns slender and more or less "visible" defects; a second category, called mark, concerns more compact defects but that are less visible; and finally, a third category called contamination concerns defects that are also compact but even less visible than in the mark category.
A typology consisting of two categories is also suited to an overall defect detection approach. In this case, the mark and contamination categories are combined. Thus, if a defect has been identified locally by its geometry, it may not be identified as sufficiently serious with respect to its visibility and will therefore not be identified. However, if it is combined with other local defects, it may be part of an overall detection of a defective module.
A first step 110 of the parametering phase 100 consists in modeling the geometry of the microelectronic strip under inspection.
The geometry of a strip corresponds to the manufacturer' s data with all the associated basic dimensions. Modeling the geometry of the strip depends on the preliminary masking and breakdown of the inspection zone, corresponding to the surface of a module, into basic zones. These two elements have the indexes 24 of the strip as common reference. The notion of a mask makes it possible to consider a module as a unit and to inhibit diagnostication of the masked areas such as, for example, the inter-track areas, etc. This masking process simplifies the parametering phase. Indeed, without this automatic masking process, the user must himself select as many areas of interest as parts not containing any inter- track areas. This type of manual selection is slow, presents risks of errors, and is not repeatable. Thanks to the automatic masking step, a mask is produced once per strip geometry and based on the same reference as the inspection zone, that is, based on the index holes 24, which means that the mask parametered in this fashion can be transferred to the images of the modules being inspected displayed by the cameras. The breakdown into basic zones is very useful and relevant: it allows a macroscopic representation of the module and greater processing efficiency and speed. It also makes it possible to inhibit the processing of certain basic zones if they are not important for a given customer. Figure 5 illustrates this step. It represents the image of a contact face of a module divided into basic shaded zones each in reference to the indexes 24 of the strip, and whose inter-track areas 29, in black, that are not to be inspected, are masked. Concerning the chip face, the masks are essentially positioned over the stress deviators, that is, around the zone protected by the resin, and the inspection area, situated inside these stress deviators is divided into basic zones. Furthermore, strips by a same manufacturer may have the same geometry but a different visual appearance, that is, a different appearance due to the manufacturing process or the material making up the strip. Humans have the ability to disregard uncharacteristic elements of the module due to their ability to select the pertinent information to be analyzed. For example, the geometry of the strips, namely the basic dimensions of the strips and the more or less complex positions and shapes of the inter-tracks of the modules vary greatly from module to module. The creation of a new strip must, however, remain simple and quick, since there is no interest in inspecting intertracks. For a same strip geometry, there may be several textures ranging from very dull to very shiny with different materials, such as palladium or gold, for example, thus requiring an ability to adapt to the light cast onto the module.
The following step 111 therefore consists in modeling the appearance of the strip. The lighting bank 15, respectively 16, is also controlled in order to obtain an optimum optical configuration, that is, a configuration that avoids image saturation while allowing good visibility of the indexes 24.
The parameters that translate the geometry of the strips are stored in a first recording medium such as a database, and the parameters connected to the appearance of the strips are stored in a second recording medium, such as a database. These two databases are managed by the administrator of the visual inspection machine, that is, by an administrator interface provided in the management means 51 and 52, associated with the calculation means 40 and 45 of a subsystem Sl and S2, so that the user of the visual inspection system does not have direct access to it.
Then, a quality level is defined taking into account the customer's requirements (step 112). A quality level corresponds to the different quality criteria that define, for a given production, what a good or bad module is. It is based on the appearance and geometric properties of the strips . The parameters translating the quality level required are themselves also preferably stored in a third database that is managed directly by the operator using the visual inspection machine, that is, by an operator interface associated with the management means 51 and 52, of the subsystem Sl and S2.
The customer' s requirements are specified, for example, through the intermediary of the system user by means of a discursive interface as represented in figure 6, for defining a defect through a scratch. This interface allows the user to specify, in gray, all the configurations that correspond to a defect and in black, the configurations that correspond to an accepted level of quality. The notion of "low" and "high" criticality expresses how visibility must be taken into consideration with geometric attributes defined in the administrator interface .
Thus, a module image combined with all of the parameters designating the level of quality required make it possible to establish the structure used for the diagnostic. Once the preliminary parametering steps have been completed, the diagnostic phase 200 per se for a strip to be analyzed can begin.
A first step 210 of this diagnostic phase 200 consists in checking and potentially correcting the alignment of the microelectronic strip. The automatic placement of the strips in a step by step mode is never perfectly repeatable, especially at a high rate of speed, and therefore necessitates an alignment procedure that requires precision and speed. This alignment procedure must be effective regardless of the strips and their appearance. It effectively makes it possible to superimpose the attributes of the reference image perfectly over those of the image analyzed. This step is based on detecting the position of the reference indexes lit by the dedicated back-lighting means 25, 27. Then, based on the detected position of the reference indexes of the strip to be analyzed, the system defines in relative fashion the inspection zones Z corresponding respectively to the surface of a module to be inspected (step 211) .
For each of the inspection zones Z thus defined, the steps leading to the diagnostic are initiated. Thus, each inspection area Z to be inspected is broken down into basic areas ZeI to Zemax (step 212), and a mask is applied to the areas that are not to be inspected, such as the intertracks, for example.
For each basic zone Ze (step 213) of a same inspection zone Z, different steps are performed to detect the presence of local defects by analyzing in particular the difference in contrast and the difference in texture.
To approximate human expertise, a local defect is characterized by very simplistic three-parameter laws for scratches and two-parameter laws for marks and contamination. Thus, for a scratch, for example, the definition accessible to the system user based on the parameters defined by him in the discursive interface represented in figure 6, is as follows: "If the defect is of average length, covers a small area and has critical visibility, then the module is bad." Similarly, for marks and contamination, the accessible definition is: "If the defect has a large surface area and slight visibility, then the module is bad." To determine the visibility and the size of local defects, each basic zone Ze is therefore analyzed through a contrast filter (step 214) then through a texture filter (step 215) .
In step 214, each basic zone is identified through a contrast filter. A contrast differential makes it possible to translate the notion of average visibility. The visual inspection system thus detects, in each basic zone Ze analyzed, whether there is a difference in contrast related to the presence of a defect resulting in a brightness differential. Large defects, such as a dent on the module or a deep scratch, for example, can be very easily characterized by the notion of contrast. On the other hand, small defects or certain scratches are difficult to detect through the notion of contrast, since the variability of the contrast that these defects create is easy to confuse with the natural variability of the contrast between two good modules. This highlights an essential limitation of the standard systems on the market based on even a sophisticated segmentation of the distribution of brightness.
A statistical study of a representative panel of strips shows that a local zone measuring from 1 to 2 mm2 provides both a stable gray level distribution from one module to another, and is sufficiently small to reveal small defects. This study also shows that the distribution is related to a substantially Gaussian distribution. Considering the mean and the standard deviation as characteristic elements makes it possible to construct a simple approach to the notion of visibility.
In the same way that we can describe what a scratch is using the rational that approximates human expertise, it is possible to describe the notion of visibility through contrast by the following type of discursive law: if the "distance" between the local mean of the module tested and of the reference module (s) is "great" and if the "distance" between the local standard deviation of the module tested and of the reference module (s) is "small," then the local visibility attains a criticality level of 3. By pursuing this reasoning, a single reference matrix between the distances between the mean and standard deviation, illustrated in figure 7, can be developed to characterize the definition of visibility in this two- dimensional attribute space. The "distance" concerning the standard deviations corresponds to a Euclidian distance even though the distance of the means is a Euclidian distance offset by a shift revealing the natural dispersion of the means for a good product, this being derived from a preliminary statistical calculation.
By assigning "discursive / numerical" correspondences, it is therefore possible to weight the severity associated with visibility and therefore to accommodate a customer requirement. Classes of correspondences ranging from very severe criticality to tolerable are established by defect and may be used as nominals for subsequent adjustments.
Each basic zone is therefore assigned a good or bad status depending on the distribution of gray values. Bad status is associated with an overall level of severity and a quantitative estimate of the defective surface of the basic zone Ze. This differentiation in contrast reveals the presence of mark or contamination defects. However, it is still not sufficient to characterize scratches affecting the texture aspect. It is, however, the combination of these notions that expresses the notion of visibility .
A subsequent step 215 therefore consists in analyzing each basic zone Ze, that has been assigned a good status by the first contrast filter (step 214), with a texture filter. The texture filter reveals defects not seen by the contrast filter, generally defects such as "small scratches" or "small dents" that are not very obvious despite their presence. This step is not routine. Indeed, if a contrast defect has been detected, it is not necessary to add unnecessary processing time, and we then go directly to the next step 216.
The visual inspection system thus applies a texture filter to detect any difference in gradient in each basic zone analyzed. Weber's laws express the eye's sensitivity to differences in gray values according to value ranges. The texture of a module tends to be continuous and repeatable, which allows characterization using dedicated image processing tools like the coocurrence matrices that are very well known in the image processing literature. A cooccurrence matrix averaging two directions of perpendicular images with adaptively selected classes CO- Cn, characterizes a good module and makes it possible to reveal, thanks to two levels of visibility indicators, the presence of defects that could not be detected on the basis of contrast. Figure 8 illustrates this type of cooccurrence matrix. CO to C8 are automatically calculated gray value classes. Each box Cij represents the standardized occurrence number between a class Ci and the class Cj, according to two orthogonal directions and a distance of 1 pixel. If, for example, there are 200 times a Ci level pixel positioned at a distance of one Cj level pixel, then the class Cj will be equal to 200. The representative texture of a defect-free module is mostly marked by the boxes formed by C3, C4, C5, which are revealed by dotted lines. The occurrence level in the other boxes reveals the presence of defects and also allows a characterization of the criticality. Each basic zone Ze of an inspection zone Z is thus analyzed through contrast and texture filters until all the basic zones Ze have been analyzed, according to steps 214 to 217.
A related filling algorithm applied to the adjacent basic zones then makes it possible to clusterize the related zones whose status has been declared poor (step 218) . This consolidation work for related defective zones is very fast and makes it possible to define a macroscopic view of the defects that corresponds more to human interpretability .
Each group of related defects belonging to adjacent basic zones Ze, is then characterized (steps 219 and 220) . In fact, a first decision level allows the characterization of local related defects (steps 219) , while a second decision level allows the characterization of the related defects at a global level (step 220) . The characterization process tries to imitate intuitive human behavior. The eye is very sensitive to a specific defect like a small dent, that is clearly visible, but overlooks it when it is less visible. As a result, the first type of decision (219) , at the local level, makes it possible to determine a defect and also to characterize it, for example, a scratch or a dent. If there are a number of small, less visible dents, the eye is generally sensitive to a "contamination" of defects without being able to make a clear distinction concerning the characteristics of the defect. The operator thus deduces from this that the module is "dirty." In this case, the second type of decision (220), on the global level, makes it possible to determine this type of contamination. The result of these two decision levels leads to a diagnostic concerning the good or bad status of the module
(step 221) . To do this, each group of related defects is analyzed respectively by three decision systems for marks, contamination and scratches. The geometry and the degree of visibility of a group of related defects then makes it possible to designate a good / bad status for the module and to determine whether or not it belongs to one of the three characteristic categories. To accomplish this, the dimensions, that is, the length and the surface area, of the related defects are added together.
The numerical translation of the measurable notions of length and size for scratches is performed by an administrator interface that is not accessible to the machine user, as illustrated in figure 9. This intuitive step is based on operator knowledge, and the space reserved for the different characteristics detectable by an operator is divided into "miniscule," "small," "medium, " and "large" partitions based on the length L or the surface area S of the defects detected. The ambiguity of the operators' analysis in designating boundary values is taken into account by the use of blurred trapezoidal partitions .
Based on the estimated dimensions of the related defects, that is, based on the length, the surface area, the visibility, etc., the system compares them with the customer' s quality requirements saved previously when the system was parametered (step 221) . If the related group corresponds to a defect deemed unacceptable by the customer, the module containing this defect is then diagnosed as bad, even though locally the diagnostic was good.
The system also provides data concerning the qualification of defects, such as a local, global defect, a scratch, a mark, contamination, etc., which makes it possible to establish a global PARETO by production lot
(step 222) . The PARETOS thus created during the inspection of the modules of a production lot make it possible, among other things, to improve certain parameters connected to a given production run.
An end of strip diagnostic is added to each analysis
(step 223) . An end of strip is characterized by the presence of a white or transparent plastic film instead of modules. A double criterion involving the texture and the brightness of the inspection surface area guarantees the detection of the presence or absence of a strip.
Diagnostic steps 212 to 222 are thus repeated throughout the step-by-step run P of the microelectronic strip until the end of the strip is detected. The system that has just been described also provides means for easily parametering the system. Using an image bank of good modules stored in the machine, it is possible to extract statistical indices tied to the natural variability between modules and to their texture. The modes of embodiment that have just been described are merely examples for the sake of illustration and the invention is not limited to them. Indeed, numerous variants of the modes of embodiment described above can be envisioned while remaining with the framework of the invention.

Claims

1. Automatic visual inspection system for electronic modules (21), of a microelectronic strip (20), characterized in that it comprises a first subsystem (Sl) dedicated to inspecting the contact face of the modules (21) , this first subsystem (Sl) comprising:
> at least two cameras (11, 12), each camera being able to view the contact face of each module (21) positioned in a first row (22) of modules, respectively in a second row (23) of modules, of the microelectronic strip (20), ^ a lighting bank (15) for the microelectronic strip (20),
^ calculation means (40) able to deliver, from the images captured by the cameras (11, 12), whether or not there are defects present and to characterize the defects.
2. System as claimed in claim 1, characterized in that it also comprises a second subsystem (S2) dedicated to inspecting the chip face of the modules (21) , and in that this second subsystem comprises:
> at least two cameras (13, 14), each camera being able to view the chip face of each module (21) positioned in a first row (22) of modules, respectively in a second row (23) of modules, of the microelectronic strip (20),
> a lighting bank (16) for the microelectronic strip (20) , > calculation means (45) capable of deliver, from the images captured by the cameras (13, 14), whether or not there are defects present and to characterize the defects.
3. System as claimed in claim 1 or 2, characterized in that it also comprises a strip support (17) common to the two subsystems (Sl, S2) .
4. System as claimed in claim 1 or 2, characterized in that the cameras (11, 12 ; 13, 14) of each subsystem (Sl ; S2) are mounted in a pipeline architecture .
5. System as claimed in any of claims 1 to 4, characterized in that the two cameras (11, 12 ; 13, 14) of each subsystem(Sl ; S2) are kept stationary on a common support (18 ; 19) .
6. System as claimed in any of claims 1 to 5, in which the lighting bank (15 ; 16) comprises: a first mean (25 ; 27) of back-lighting dedicated to lighting the reference indexes (24) of the strip (20), - a second mean (26 ; 28) of lighting dedicated to revealing visual defects, and a mean for controlling the two lighting means.
7. System as claimed in claim 6, in which the mean (25 ; 27) of back-lighting dedicated to the reference indexes comprises: a diffusion plate (30) intended to diffuse the light, a filter plate (31) comprising opaque elements (32) positioned opposite the modules (21) to be inspected on the strip, and through-holes (33) positioned opposite the reference indexes (24) of the strip (20) , a lighting plate (34) intended to light said plates (30, 31) and the strip (20) .
8. System as claimed in claim 6, characterized in that the second mean (26 ; 28) of lighting dedicated to the visual defects comprises a mean (35) allowing for axial diffusion of the light.
9. System as claimed in claim 1 and/or 2, characterized in that it also comprises a central management unit (50) in communication with the calculation means (40 ; 45) of the subsystem(s)
(Sl ; S2) .
10. Automatic visual inspection process for electronic modules (21) positioned on a microelectronic strip (20), said method comprising the following steps : parametering (100) the strip to be analyzed based on its geometry (110), its appearance (111) and a required quality level (112), - defining (211) inspection zones (Z) with respect to reference indexes, each zone corresponding respectively to the surface area of a module to be analyzed, and for each inspection zone (Z) : > detecting the presence of a defect,
^ estimating the dimensions of the defect detected(218) ,
> characterizing the defect detected (219, 220),
> issuing a diagnostic concerning the status of the module analyzed (221) .
11. Process as claimed in claim 10, according to which for each inspection zone (Z), a preliminary step (212) consists in breaking the inspection zone down into basic zones (Ze) and in masking basic zones (29) that are not to be analyzed.
12. Process as claimed in claim 10 or 11, according to which the estimate of the dimensions of a defect consists in adding the dimensions of the defects related to at least two adjacent basic zones (218) .
13. Process as claimed in claims 10 to 12, according to which the step for characterizing a defect consists in assigning it a qualification based on its dimensions and based on its degree of visibility .
14. Process as claimed in any of claims 10 to 13, according to which the contact and chip faces of the modules are inspected simultaneously.
15. Process as claimed in any of claims 10 to 13, according to which the contact and chip faces of the modules are inspected independently.
PCT/IB2006/051802 2005-06-06 2006-06-06 Visual inspection system and process for electronic modules WO2006131883A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP05291212A EP1731899A1 (en) 2005-06-06 2005-06-06 System and apparatus for optical control of electronic modules
EP05291212.8 2005-06-06

Publications (1)

Publication Number Publication Date
WO2006131883A1 true WO2006131883A1 (en) 2006-12-14

Family

ID=35134519

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2006/051802 WO2006131883A1 (en) 2005-06-06 2006-06-06 Visual inspection system and process for electronic modules

Country Status (2)

Country Link
EP (1) EP1731899A1 (en)
WO (1) WO2006131883A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018133014A1 (en) * 2017-01-19 2018-07-26 深圳市汇顶科技股份有限公司 Test device for strip chips

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4040745A (en) * 1975-06-30 1977-08-09 International Business Machines Corporation Inspection tool
US4969199A (en) * 1986-02-28 1990-11-06 Kabushiki Kaisha Toshiba Apparatus for inspecting the molded case of an IC device
JPH11251362A (en) * 1997-11-14 1999-09-17 Samsung Electronics Co Ltd Visual inspecting apparatus for tape carrier package
US6700603B1 (en) * 1998-07-28 2004-03-02 Matsushita Electric Works, Ltd. Inspection system for inspecting discrete wiring patterns formed on a continuous substrate sheet of a flexible material
JP2004153170A (en) * 2002-10-31 2004-05-27 Mitsui Mining & Smelting Co Ltd Method of inspecting film carrier tape for mounting electronic part and device of inspecting the same therefor

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4040745A (en) * 1975-06-30 1977-08-09 International Business Machines Corporation Inspection tool
US4969199A (en) * 1986-02-28 1990-11-06 Kabushiki Kaisha Toshiba Apparatus for inspecting the molded case of an IC device
JPH11251362A (en) * 1997-11-14 1999-09-17 Samsung Electronics Co Ltd Visual inspecting apparatus for tape carrier package
US6700603B1 (en) * 1998-07-28 2004-03-02 Matsushita Electric Works, Ltd. Inspection system for inspecting discrete wiring patterns formed on a continuous substrate sheet of a flexible material
JP2004153170A (en) * 2002-10-31 2004-05-27 Mitsui Mining & Smelting Co Ltd Method of inspecting film carrier tape for mounting electronic part and device of inspecting the same therefor

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
PATENT ABSTRACTS OF JAPAN vol. 1999, no. 14 22 December 1999 (1999-12-22) *
PATENT ABSTRACTS OF JAPAN vol. 2003, no. 12 5 December 2003 (2003-12-05) *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018133014A1 (en) * 2017-01-19 2018-07-26 深圳市汇顶科技股份有限公司 Test device for strip chips

Also Published As

Publication number Publication date
EP1731899A1 (en) 2006-12-13

Similar Documents

Publication Publication Date Title
US6614520B1 (en) Method for inspecting a reticle
EP2388702B1 (en) Automated damage assessment, report, and disposition
WO2019014937A1 (en) Method and apparatus for detecting defect of laser welding protection lens on automobile production line
KR101294218B1 (en) System and method for evaluating quality of polarized film
JP2002100660A (en) Defect detecting method, defect observing method and defect detecting apparatus
CN104303264A (en) Method, computer system and apparatus for recipe generation for automated inspection semiconductor devices
US7808624B2 (en) Inspecting end surfaces of fiber optic connectors
JP2001326263A (en) Method of assessing structural defect on wafer surface
US11650126B2 (en) Systems and methods for automatic visual inspection of defects in ophthalmic lenses
Kopardekar et al. Manual, Hybrid and Automated Inspection Literature andCurrent Research
CN115825103B (en) End detection system for angular port optical fiber array based on image technology
CN108951075A (en) It is a kind of to screen out method and with fault cut-parts positioning system with fault cut-parts
CN110428764A (en) Display panel testing method
MORAWSKI et al. The optimum speed of visual inspection using a random search strategy
CN104408542B (en) Quality models and graphical defect input method and system
WO2013001594A1 (en) Method for management of inspection of electronic substrate, apparatus for same, and apparatus for visual inspection of electronic substrate
WO2006131883A1 (en) Visual inspection system and process for electronic modules
US20230294173A1 (en) Method and Apparatus for Additive Manufacturing of a Workpiece using Statistical Learning Model
JP4944824B2 (en) Inspection result display device
JPH08240536A (en) Method and equipment for checking burr of lead of integratedcircuit
JP7043645B1 (en) Board inspection method
JP6027717B2 (en) Visual inspection apparatus provided with gaze position specifying means
Weckenmann et al. Model and simulation of fringe projection measurements as part of an assistance system for multi-component fringe projection sensors
US7855088B2 (en) Method for manufacturing integrated circuits by guardbanding die regions
CN114092382A (en) Product flaw marking device and method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

WWW Wipo information: withdrawn in national office

Country of ref document: DE

122 Ep: pct application non-entry in european phase

Ref document number: 06756069

Country of ref document: EP

Kind code of ref document: A1