WO2020223594A2 - Inspection automatisée d'objets en points denses à 360 degrés - Google Patents

Inspection automatisée d'objets en points denses à 360 degrés Download PDF

Info

Publication number
WO2020223594A2
WO2020223594A2 PCT/US2020/030940 US2020030940W WO2020223594A2 WO 2020223594 A2 WO2020223594 A2 WO 2020223594A2 US 2020030940 W US2020030940 W US 2020030940W WO 2020223594 A2 WO2020223594 A2 WO 2020223594A2
Authority
WO
WIPO (PCT)
Prior art keywords
point cloud
laser
point
inspection
objects
Prior art date
Application number
PCT/US2020/030940
Other languages
English (en)
Other versions
WO2020223594A3 (fr
Inventor
Bruce A. Link
Robert W. Johnson
Jose Zvietcovich ZEGARRA
Peter O. Stubler
Mark Barton
Frank R. Brockler
Michael S. Axman
Madirakshi Das
Gabor J. Ferencz, Jr.
Original Assignee
Kodak Alaris, Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/401,890 external-priority patent/US11042146B2/en
Application filed by Kodak Alaris, Inc filed Critical Kodak Alaris, Inc
Priority to EP20727498.6A priority Critical patent/EP3963414A2/fr
Priority to CN202080048597.0A priority patent/CN114041168A/zh
Publication of WO2020223594A2 publication Critical patent/WO2020223594A2/fr
Publication of WO2020223594A3 publication Critical patent/WO2020223594A3/fr

Links

Classifications

    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/42—Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine
    • G05B19/4202—Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine preparation of the programme medium using a drawing, a model
    • G05B19/4207—Recording and playback systems, i.e. in which the programme is recorded from a cycle of operations, e.g. the cycle of operations being manually controlled, after which this record is played back on the same machine preparation of the programme medium using a drawing, a model in which a model is traced or scanned and corresponding data recorded
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/245—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures using a plurality of fixed, simultaneously operating transducers
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2504—Calibration devices
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2518—Projection by scanning of the object
    • G01B11/2522—Projection by scanning of the object the position of the object changing and being recorded
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/84—Systems specially adapted for particular applications
    • G01N21/88—Investigating the presence of flaws or contamination
    • G01N21/93—Detection standards; Calibrating baseline adjustment, drift correction
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
    • G05B19/41875—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by quality surveillance of production
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/001—Industrial image inspection using an image reference approach
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1679—Programme controls characterised by the tasks executed
    • B25J9/1692—Calibration of manipulator
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/18—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
    • G05B19/401—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by control arrangements for measuring, e.g. calibration and initialisation, measuring workpiece for machining purposes
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/18—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
    • G05B19/406—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by monitoring or safety
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/18—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form
    • G05B19/4093—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by part programming, e.g. entry of geometrical information as taken from a technical drawing, combining this with machining and material information to obtain control information, named part programme, for the NC machine
    • G05B19/40937—Numerical control [NC], i.e. automatically operating machines, in particular machine tools, e.g. in a manufacturing environment, so as to execute positioning, movement or co-ordinated operations by means of programme data in numerical form characterised by part programming, e.g. entry of geometrical information as taken from a technical drawing, combining this with machining and material information to obtain control information, named part programme, for the NC machine concerning programming of machining or material parameters, pocket machining
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B19/00—Programme-control systems
    • G05B19/02—Programme-control systems electric
    • G05B19/418—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM]
    • G05B19/4184—Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control [DNC], flexible manufacturing systems [FMS], integrated manufacturing systems [IMS] or computer integrated manufacturing [CIM] characterised by fault tolerance, reliability of production system
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/37—Measurements
    • G05B2219/37008—Calibration of measuring system, probe, sensor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10141—Special mode during image acquisition
    • G06T2207/10152—Varying illumination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/80—Management or planning

Definitions

  • This invention relates to automated inspection of physical objects, and in particular to an inspection system and method implementing non-contact, three-dimensional scanning of physical objects.
  • CMMs coordinate measuring machines
  • laser trackers and three-dimensional scanning measurement systems
  • CMMs three-dimensional scanning measurement systems
  • these inspection systems require a long and complicated set-up and further require particular positioning of the parts or products to be inspected within the inspection system before inspection can begin.
  • the parts are required to be positioned within a jig to hold the part properly.
  • a significant amount of time and cost is invested in simply setting up the products or parts and the inspection system, and the set-up process needs to be completed separately for each different part to be inspected.
  • CMMs also capture measurements at specific points, thus producing a sparse sampling of measurements of the object.
  • multiple specific points may need to be measured.
  • Increasing the number of points increases the complexity of measurement, adding to time and cost.
  • Automated manufacturing techniques such as molding, machining, and assembling used to make components and parts requires that the manufactured parts conform to specifications. Manually inspecting each part is laborious and time consuming and if not done in a timely fashion would result in the manufacture of non-conforming components, requiring rework and waste of time and materials. Automated inspection as part of the manufacturing process can identify non-conforming parts and track trends that indicate and predict that an out of specification condition will happen at some point in the future due to a mold, tool beginning to wear, or a supplier provided non-conformant components.
  • the use of non-contact profilometers to acquire a 3D image of a manufactured component to compare the acquired image to the component specification to verify component conformance.
  • the present invention is directed to an in-line system and method for quality inspection of manufactured parts and products, including first article inspection, critical to function parameters (CTF) and defect detection.
  • the inspection determines if the material or item is in the proper quantity and condition, and if it conforms to the applicable or specified requirements, either of a particular user or an industry standard.
  • the inspection is generally divided into three categories: 1) receiving inspection, 2) in-process inspection, and 3) final inspection.
  • the system and method perform non-contact quality inspection by scanning the three-dimensional (3-D) surface geometry of an object being inspected and comparing information from the scanning process to 3-D models of the object.
  • the objects may be solid, opaque, translucent, or transparent, and may have specular or diffusive surface features.
  • Objects may include a combination of these features, and still be inspected with the system and method of the present invention. Moreover, an object may be inspected at several different stages of its fabrication, such that multiple in-process inspections are performed on each stage. Each stage may require its own inspection process, as described herein.
  • the system and method may include an optical scanning system and a laser module, including a laser profilometer, for performing precise in-line inspection of objects.
  • the light beam of the laser profilometer may have a wavelength in the violet or ultraviolet range, although other wavelengths may also be used.
  • objects, such as parts or products, are being inspected, they may be placed, at predetermined position and in a predetermined orientation, on a transport system, such as a transport belt, that moves them past the optical scanning system and laser profilometer.
  • the system and method may inspect multiple objects placed on the transport at the same time, with the transport moving the objects past the inspection system components in an in-line process.
  • the inspected objects could be the same or different part or products, with the optical scanning system recognizing each object as it passes.
  • the optical scanner system may identify contamination such as dust, oil, or other foreign objects present on the object, and may also detect and measure objects for accuracy of the expected color, texture or finish of the objects.
  • a computer processor in the quality inspection system receives the determined identity of the objects being inspection from the optical scanning system, and loads a data file corresponding to the identified object or product being inspected.
  • the object to be inspected can be predetermined, and a predetermined data file corresponding to that object can be loaded.
  • the data file may be a CAD model, which is then converted into a uniform point cloud representing the object.
  • the laser system outputs a regularly spaced three-dimensional (3- D) coordinate point cloud representing the object or objects.
  • the system and method compares this obtained 3-D coordinate point cloud representing the objects with the uniform point cloud retrieved from the computer processor.
  • Differences between the two are determined, and are used to determine the extent of variations between the object being inspected and the expected product (based on the stored data file).
  • the system and method may then identify parts or sections of the object being inspected as potentially being defective, and may alert a system operator when defective objects are identified.
  • the system may be used to inspect parts manufactured from a mold.
  • the mold would have the desired dimensions of the final part, and it is expected that parts produced by the mold would have these desired dimensions within a certain tolerance level. Determining defect data in real time from injected molded parts can be used as a predictive measuring method to determine when significant wear has occurred and the mold needs to be replaced before defects becomes objectionable.
  • the system may be used as part of a geometric dimensioning and tolerancing quality inspection process.
  • Dimensioning specifications define the nominal, as-modeled or as-intended geometry.
  • Tolerancing specifications define the allowable variation for the form and possibly the size of individual features, and the allowable variation in orientation and location between features.
  • Figure 1A is a high-level diagram showing the components of the quality inspection system.
  • Figure IB is an illustration of the components of the laser system in the quality inspection system.
  • FIG. 2 is an illustration of an overview functional diagram of the inspection system modules.
  • Figure 3 is an illustration of the parameter-set up recommendation system.
  • Figure 4 is an illustration of a laser (4009-4010) and sensor (4011-4012) showing ray casts and volume in focus (4001-4008)
  • Figure 5 illustrates a block diagram of the ray casting technique used to find the capture set-up parameters that produce an adequate scan of the object.
  • Figure 6 is a simulation of an object being scanned by a two laser-camera system.
  • Figure 7 depicts a calibration target containing fiducial marker holes on steps.
  • Figure 8 illustrates an example scan already segmented into planar segments.
  • Figure 9 illustrates a histogram of distances of points to the defined reference plane.
  • Figure 10 illustrates a histogram of distances from points to a poorly oriented defined reference plane.
  • Figure 11 illustrates an example of an image of the plane in which the presence of a point is indicated by one value and the absence of a point is indicated by another color.
  • Figures 12A and 12B are illustrations of the product inspection system in use, showing motion of the transport and laser module during inspection.
  • Figure 13 is a block diagram of the components of the quality inspection system.
  • Figure 14 is an illustration of the scanning paths of the laser system when moving only perpendicular to the transport path
  • Figure 15 is an illustration of the scanning paths of the laser system when the laser system moves both perpendicular to and in the same direction as the transport path.
  • Figure 16 depicts the details of the processing of converting stored CAD models of an object into a point cloud for comparison to the point cloud from the laser system.
  • Figure 17 depicts the details of the processing of image data from the optical acquisition unit to identify the objects being inspected.
  • Figure 18A depicts the processing to merge data from the laser system with CAD model information to determine differences between actual object dimensions and expected object dimensions.
  • Figure 18B depicts a system flow diagram for library generation of an object based on its reference mesh model.
  • Figure 18C depicts a system flow diagram for registering and matching a target point cloud to a reference mesh model.
  • Figure 19 depicts the processing of the tolerance determinations made by the system and method.
  • Figure 20 depicts an example of a transport system that may be used in the inspection system to move objects being inspected past the optical acquisition unit and the laser scanning system.
  • Figure 21 depicts the processing of determining whether an object being inspected is defective.
  • Figure 22 depicts a sample user interface that may be displayed to the operator.
  • Figure 23 depicts the process for monitoring performance of the scanning system, correcting spatial displacements, and notifying a user when the degree of spatial displacement exceeds a threshold.
  • the present invention is directed to a system and method for high precision quality control inspection of objects.
  • the system is configured to perform geometric inspection of manufactured parts for quality control and inspection (including first article inspection, critical to function parameters (CTF), and defect detection).
  • inspection determines if the material or item is in the proper quantity and condition, and if it conforms to the applicable or specified requirements, either of a particular user or an industry standard.
  • the inspection is generally divided into three categories: 1) receiving inspection, 2) in-process inspection, and 3) final inspection.
  • Receiving inspection includes first inspecting an object received to determine if it initially meets applicable requirements and standards. In-process inspection is performed when an object is manufactured in several stages, and an inspection can be performed after all, or certain, of the
  • Final inspection may be performed on completed objects that are not expected to undergo further fabrication or alterations.
  • the system performs scanning of the three-dimensional (3-D) surface geometry of an object such as manufactured components, parts, and products to detect defects.
  • the system performs automated, in-line or offline quality inspection by scanning objects on a movable platform, or platen, with both an optical acquisition unit and a scanning system, and making comparisons between the data from the resulting object scans and predetermined data files accessible by a system processor, the data files corresponding to the objects being inspected.
  • the scanning system may include depth measurement devices whose measurements can be used to capture 3-D object geometry during the scanning process.
  • the depth measurement device may be, for example, a laser profilometer system, confocal system, structured light system or interferometer, configured to scan objects on the movable platform.
  • sensors may be included in the inspection system.
  • the system may be configured such that an operator or automated program may select which sensors to use, with sensors that will not be used being retracted or repositioned so that they do not interfere with the inspection.
  • a sensor mount may be included, allowing one or more sensors to be interchangeably mounted to the system for use in the inspection process. Such a sensor mount allows an operator to swap out imagers not needed for particular inspection.
  • the system may use different sensors, or combinations of sensors, in imaging different parts, types of objects, particular features of the same object, or different stages of the fabrication of an object. Thus, the system may have new set-up parameters that are implemented for each new sensor or combination of sensors used.
  • the system can produce an inspection report by comparing the captured dimensions of the actual object with the geometric dimensioning and tolerancing (GD&T) specifications provided for the object.
  • Dimensioning specifications define the nominal, as- modeled or as-intended geometry.
  • Tolerancing specifications define the allowable variation for the form and the size of individual features of the object being inspected, and further define the allowable variation in orientation and location between features and what is expected.
  • the system includes a moving transport, an optical system to image objects at the beginning of the transport path, and a one or more depth measurement systems such as a laser profilometer, confocal system, structured light system or interferometer, to record 3-D object geometry located after the optical system on the transport path.
  • the laser profilometer or other depth capture modality may capture 3- D object geometry in the form of an unstructured point cloud representing the object.
  • the system also includes a computer processor for computation and storage of data, including stored object data information such as CAD data that shows the expected dimensions of the objects being inspected.
  • An operator control panel and display may be provided to allow an operator to view CAD images from the storage database, view scanning system captured 3-D images of the objects being inspected, and view heat map and histogram information detailing differences between the CAD images from the stored database as compared to the actual imaged objects.
  • the operator control panel may further allow the operator to modify parameters of the inspection system, control operation of or pause the inspection system, and review defect information regarding the inspected objects.
  • the quality inspection system 100 includes an optical acquisition unit 110 at one end of a transport 150, which moves objects 160 to be inspected through the inspection system.
  • the transport 150 may be, for example, a conveyer belt or system of rollers, and moves the objects being inspected along the transport path in direction A.
  • the optical acquisition unit 110 may be an area or linear array optical system, including one or more image capturing units.
  • the optical acquisition unit 110 may be connected to a processing device configured to analyze captured images of the objects on the transport 150 in order to identify the object, as well as its position and orientation on the transport.
  • a laser module 200 is included along the transport path, after the optical acquisition unit 110, and is connected to an operator control panel 120.
  • the laser module includes a laser driver 210, and laser sensor 250.
  • the laser may include a mounting assembly 260, connecting it to a traversal mechanism 270, such as a stationary or moveable leadscrew.
  • the traversal mechanism 270 may be configured to allow the laser module to move in three dimensions with respect to the object being inspected.
  • the laser module emits a laser beam 140, which is reflected from the object being inspected 160 and then sensed by the laser sensor 250.
  • the laser module 200 may be, but is not limited to, a laser profilometer with an illumination light beam having a wavelength in the violet or ultraviolet range, used to scan the object and output a three-dimensional (3-D) unstructured point cloud.
  • the laser profilometer may use, but is not limited to, laser triangulation or time of flight displacement sensing to measure depth.
  • the system may also include an operator control panel 120, allowing a system operator to perform inspection of objects, modify inspection parameters, review object inspection results, and respond to defect or alert notifications.
  • the operator control panel may include a user input 125, such as a keyboard, mouse, or touchscreen, and may further include a display 128.
  • An alert notification 130 may be included in the quality inspection system, and may be a visual or audible alert.
  • the user loads the inspection system 100 with the appropriate CAD models and tolerance data for the object or objects to be inspected. Objects that exceed a predetermined tolerance are identified as defective.
  • the user can view the defective three-dimensional (3-D) coordinate point cloud variations from the CAD model through a heat-map displayed on the user display 128 or define actions that should occur for the defective object.
  • the user display may be local to the inspection system, or may be a display communicating with a computing device of the inspection station over a network connection.
  • Figure IB is an illustration of the components of the laser module 200, which includes a laser driver 210 that outputs a laser beam 140, in the form of a laser cone 230, that projects a line 240, representing the field of view, across the object being scanned.
  • a laser driver 210 that outputs a laser beam 140, in the form of a laser cone 230, that projects a line 240, representing the field of view, across the object being scanned.
  • the length of the line 240 formed is related projection angle Q 280 of the laser beam 140 and the distance between the laser driver 210 and the object being inspected.
  • the laser beam 140 is reflected from the object being inspected back to a laser sensor 250 on the laser module 200.
  • the laser driver 210 could output a laser beam where a laser beam 140 is a point that hits the object and is reflected off the object.
  • the laser module 200 may be a laser profilometer.
  • a laser profilometer with Blue- violet laser uses 405-nm wavelength may be used due to its shorter wavelength.
  • a laser module with an illumination light beam with a wavelength in the violet or ultraviolet range is superior to standard sensors that use, for example, red laser diode on metals and organic materials.
  • the shorter wavelength is less susceptible to objects that are self-illuminated due to thermal characteristics.
  • the violet or ultraviolet wavelength minimizes the spread of laser light on the surface of the object 160 being inspected and does not penetrate the object thereby producing more precise measuring results allowing the detection and measurement of a broad range of objects made from plastics, metals, glass, and other material regardless of the features surface finishes.
  • FIG. 2 provides an overview functional diagram of the inspection system modules, including a Point Cloud Data Capture sub-system 2000, a Part View Processing sub-system 2020, and a Part Inspection sub-system 2030.
  • the Point Cloud Data Capture sub-system 2000 works by acquiring the system set-up parameters 2002 and CAD pre processing data 2006 from the library of objects and associated data 2010. Once acquired, the system captures the scan data 2001 and then normalizes the scan data 2003.
  • the normalized scan data 2003 includes calibration data 2004. After the scan data is normalized, the data is aligned and stitched 2005. If more scan data is needed, more scan data can be captured 2001. Once the scan data is aligned and stitched, it is generated into a 3D Point Cloud File 2007. All information associated with the expected objects to be inspected is stored in a library of objects database 2010. This information includes the CAD model, GD&T information related to the object, system set-up parameters for scanning the object, and other data computed during pre-processing.
  • the following operations are carried out on the CAD models as a part of pre processing, to prepare all the necessary data for point cloud comparisons when the captured point cloud is available.
  • the pre-processing may be carried out by software containing instructions executable by a computer, or may be included as part of an embedded chip or as a cloud-computing resource.
  • a user specifies the CAD file and accepts (or modifies) certain default settings, if necessary. These computations are only done once for each CAD model, and it is not necessary to repeat the procedure for each captured point cloud.
  • the following operations on the CAD models may be performed as part of the pre-processing:
  • the CAD model stereolithography (STL) file is loaded.
  • the STL file conveys the object geometry in terms of a list of triangles in 3D. Each triangle specifies 3 vertices, and an outward facing normal vector.
  • the inspection system described also provides GUI tools to an administrator who is performing the pre-processing set up.
  • the administrator sets up the GD&T information during the pre-processing, and the set-up associated with the object is then saved in the library of objects database 2010.
  • the GUI tools allow the administrator to:
  • the administrator also selects the number and type of imagers needed based on the number of surfaces to be imaged, and specific measurement requirements of the object as determined from the GD&T information. For example, measurements needed inside boreholes may require a confocal imager, measurement of small displacements and surface irregularities may require the use of an interferometer, and correctness of color measurements may need a calibrated vision camera.
  • the inspection system may provide multiple imaging options that can be selected, where the imagers not selected can be swapped out or retracted.
  • the system set-up that includes the imagers selected is stored in the library entry associated with the object in library 2010.
  • the inspection process is split into multiple parts with each part specifying the imager(s) to use, and the measurements needed.
  • the library entry for the object then will have multiple steps corresponding to each part of the inspection process.
  • Setting up the system includes at least setting up the configuration of imagers, sensors, and imaging angles for the inspection system, platen or conveyor speed, and object placement instructions for the particular object that will be inspected.
  • the system is initially calibrated using a specialized calibration target, such as an object with known dimensions and properties.
  • the specialized calibration target may be an ideal version of the type of object desired to be inspected.
  • the system is set up for the specific object being inspected using system set-up parameters 2002 determined to be appropriate for that object.
  • the set-up parameters 2002 include the positions and orientations of the imagers, such as laser modules, a confocal system, a structured light system or interferometer, or combinations thereof, and the placement of the object on the platen or conveyor.
  • Initial set up can be performed while the system is offline (that is, not in use in inspecting objects), but can also be performed online and refined if the system detects a new part or object.
  • Repositioning of imaging devices may be automatic, depending on the object being inspected, so when a new object is identified by the system, the system performs a new set up procedure and the position of the imaging devices is automatically updated for the new object.
  • Multiple placements of the object may be included to cover all the views needed for measurements. For example, where an object to be inspected has multiple different faces, a particular placement of the object and imagers for each face may be included.
  • the set-up parameters 2002 may include each of these three different positions and orientations of the object, along with the position and orientation for the imagers for each object rest position.
  • depth measurement data is recorded by traversing the object with the imager using motion of the imager, the platen or conveyor carrying the object, or both. That is, the object may be left in a stationary position and the imagers may move relative to the object, capturing scans as they move.
  • the data from the scanning elements, or imaging devices are processed to produce a point cloud for each traversal, and the point clouds for each transversal are then merged in a stitching process to form an overall point cloud that corresponds to the scanning element. Later, data from all scanning elements will be merged and analyzed, to combine the data for a particular object obtained by each scanning element during the process.
  • the object may be advanced along a movable platen conveyor, such that it passes under the imagers.
  • the imagers may then be stationary, or may move relative to the platen or conveyor during the scans.
  • the data from the scanning traversals of the object is aligned using the calibration data derived from scanning the specialized object. Any drift in the calibration over time is tracked, and the system can produce a notification when a re-calibration is required.
  • the calibration process is described in more detail herein.
  • the data from the aligned scanning traversals is merged to produce a point cloud. Multiple point clouds may be generated from different placements of the object to cover all surfaces that need to be measured, as indicated by the system set-up parameters. For example, if the system set-up parameters indicate that three different scans, each having a different object and/or imager position and orientation, are required, then these three scans are conducted and three point clouds are generated.
  • the function of the Part View Processing sub-system 2020 is to align and merge the point clouds produced during point cloud capture to create a single point cloud covering a 360-degree view of the object being inspected.
  • the system performs floor subtraction 2021 on the obtained point clouds to remove points that correspond to the floor (conveyor, platen, or other platform) that the object rests upon while being inspected.
  • a 3D part view file 2022 for the object is retrieved and imported into the system 2023 for comparison to the point cloud data.
  • the point cloud data is aligned and merged 2024, with reference to the 3D part view files, in order to obtain a 3D part file of the scanned object 2025.
  • the Part Inspection sub- system 2030 analyzes the obtained 3D part files to determine if the object being inspected has defects. To do this, the 3D part files, each representing a single point cloud for individual objects inspected, obtained from the Part View Processing sub-system 2020 are imported 2033 into the part inspection module 2030, where they are compared with CAD models 2031 for the particular objects being inspected. The comparison process involves CAD-Part Registration 2035, that can be adjusted per a user’s preferences 2038.
  • Deviations between the obtained 3D part files and the expected data based on the CAD models 2037 are used to generate an inspection report 2039, indicating to an operator whether the object being inspected falls within acceptable limits, or whether it is determined to be defective 2040.
  • the report generated 2039 is customizable with user preferences 2038.
  • the Part Inspection sub-system 2030 starts with receiving a part CAD model 2031 that is then imported 2032 into the Part Inspection sub-system 2030. If a 3D part file is not imported 2033, the sub-system will identify CTFs 2034 and then provide the inspection parameters 2036. Then, deviations will be calculated 2037 between the part CAD model and CTFs, which will generate a report 2039, indicating to an operator whether the object being inspected falls within acceptable limits, or whether it is determined to be defective 2040.
  • the report generated 2039 is customizable with user preferences 2038.
  • the system may first be set up using a parameter-recommendation system to ensure that the objects are placed in the appropriate positions and orientations, and that the system is set to properly capture the necessary data for the particular objects.
  • An inspection set-up typically has many parameters that need to be set before a scan can be performed. Many of these parameters may depend on the particular object being scanned. For example, the size of the object determines the distance each traversal of the scan needs to cover, and the number of traversals needed. The configuration and type of imagers needed based on the number of surfaces to be imaged, and specific measurement requirements.
  • the angles and positions of the imagers are influenced by the areas that need to be scanned based on the geometric dimensioning and tolerance (GD&T) measurements needed.
  • a scan set-up parameter recommendation module part of Point Cloud Capture 2000 shown in Figure 2, guides set up for the particular objects being scanned.
  • the scan set-up parameter recommendation module includes software that models the imaging set-up of the system to compute the capture set-up parameters needed for adequate coverage of the object being inspected. To do this, CAD models and the GD&T measurements needed for the particular object being inspected are provided to a system processor that executes a software simulation of the inspection system that computes appropriate parameters to be used for the particular object.
  • the computed parameters are stored in a library of object inspection information stored in computer memory.
  • inspection applications involve in-process inspections, where the object being manufactured is tested at intermediate points during its manufacturing process.
  • These objects may have multiple CAD models that represent the expected state of the object at the intermediate points.
  • the system set-up requirements are determined separately for each of these intermediate CAD models, and a separate entry is maintained in the library for each.
  • Figure 3 shows the use of the scan set-up parameter recommendation system, which is implemented as a software module that includes a modeling simulator.
  • CAD model and GD&T information for a particular object is retrieved 301 by the module.
  • This information is used in the execution of a scan set-up parameter recommendation system software simulation process 302, which results in a determination of the set-up parameters for the system for that particular object.
  • the simulation software includes a complete specification of the system, including the number, type, and positioning of the sensors and capture devices, their characteristics such as region in focus, adjustments supported in angle and height, motion characteristics of the sensors and capture devices, and the transport, platen or conveyor speed and direction parameters.
  • the determined set-up parameters are saved in a system library 303, in an entry that relates the set-up to the object.
  • the simulation process is executed for each of the objects expected to be scanned by the system, and the computed set-up parameters are stored in the library. This process is completed offline before the inspection system is ready for scans and before scans are being performed by the system.
  • the system may be considered to be offline when pre-processing steps are being carried out, such as parameter computations to store in a library. Scans may be performed only after the offline stage is complete, the system set up, and deployed for use. For a given object or inspection process, the system can have multiple different stored set up parameters.
  • the system may select from the stored set-up parameters depending on the requirements or preferences of the user, and may be based on, for example, the type of object being inspected, the type of inspection being performed.
  • the operator may select which set-up parameters to use, or the system may automatically determine which set-up parameters to use based on identification of an object to be inspected by the optical acquisition system.
  • the inspection system is ready for inspection of objects that correspond to entries in the scan set-up library. If new types of objects are to be inspected, an administrator with access to the recommendation system software can complete a new simulation so that set-up parameters of the new object are stored in the library.
  • process 310 shown in Figure 3 is followed. First, CAD model and GD&T information for a particular object is retrieved 311. Then, the system reads the scan-sup instructions 312 for that object from the library before each scan and the system is set up accordingly. After the system is set-up according to the retrieved parameters and instructions, the system perform scans according to these set up parameters and instructions 313. This ensures that the resulting scan will have the data necessary for the inspection of the object.
  • the part may be set-up in accordance to the system set-up parameters using an automated system such as a robotic arm, or may be done manually by an operator following provided guidelines.
  • the guidelines may be provided by displaying graphics and figures in the GUI, as well as by projecting visual guide markers on the platen or conveyor.
  • projections When projections are used, they may be formed by using the laser module or other light source of the depth measurement system that is then used to obtain depth measurement captures during scanning inspections of the object. Projections can take the form of a scaled outline of the part and textual and/or graphical instructions and indicators to guide the operator in proper part placement and orientation.
  • the optical acquisition unit is included in the inspection system, this optical acquisition unit can capture images of the object to be inspected.
  • the captured images of the object can then be analyzed by the system to determine if the object has been placed in the correct position and orientation for inspection before it is moved past the scanner in the inspection process.
  • the camera may capture images of the object and the illuminated outline areas of where it is expected to be. These images may be analyzed, and if the object is positioned within the illuminated areas, the system may determine that the object is ready for inspection.
  • the scanning process may be interrupted and an alert issued for the object to be repositioned. This procedure may repeat until it is determined that the object is in the proper position and orientation. Moreover, if the captured images from the camera indicate an object having a different finish or texture than what was expected has been placed in the system for inspection, the system may again issue an alert and allow for new inspection set-up parameters to be loaded before inspection begins.
  • the scan set-up parameter recommendation system includes a simulator that computes the resulting point cloud that would be obtained for a given configuration of the laser and optical receiver, object and the motion of the laser imaging module as well as the platen or conveyor carrying the object.
  • the computation uses ray casting to determine the points on the object surface that would be visible to the imaging module at any given position.
  • Ray-casting is a process where the expected path of a ray from its origin source to a reception source is projected, implemented as a computer graphics algorithm that determines the intersection between a ray and a surface. For example, in a laser system, the ray casting projects the path of laser light from the laser to a receiver.
  • Visible points are points where a ray from the source is incident on the point on the surface of the object, and the reflected ray from the point is received at the receiver without being blocked (typically by a part of the object itself). If an obstructing surface is in the path, the ray casting will show that the emitted light would not be received at the receiver.
  • the visible points are computed over the entire traversal routes to generate a point cloud that would be detected. For example, a typical laser profilometer set-up shown in Figure 4, where 4009- 4010 indicates the laser, and 4011-4012 indicates the optical receiver that detects the reflected laser light.
  • the laser cone covers the area 4001-4002-4003-4004 on the platen, but only the volume included in 4001-4002-4003-4004-4005-4006-4007-4008 is in focus and structures in this volume are included in the visible points. Points closer than the
  • the laser 4011-4012 and optical receiver 4011-4012 are in fixed positions relative to each other, but the whole laser-camera assembly can move, traversing the platen. The depth can also be adjusted so that plane 4001-4002-4003-4004 is on the platen surface. By performing this ray casting and computation of visible points, the simulator can then determine the se-up parameters.
  • the simulator is used to find the capture set-up parameters that would produce an adequate scan of the object as shown in Figure 5.
  • input data 501 including the CAD model, rest positions, GD&T information, and imagers needed for the desired object for the simulation is loaded into the program from the library of objects 2010. If the object has multi-part inspection process, each such part is simulated separately.
  • the system simulates the selected imager(s) as an emitter and a receiver which can be modeled as a point or as lines based on the type of imager.
  • the program estimates initial capture parameters 502 from the size and extent of the CAD model.
  • a range and step-size for each capture parameter is also determined based on the system set-up. For example, the right laser can be set at a range of -25 to +25 degree angle about the Y-axis, with a step size of 5 degree.
  • a simulated scan of object 503 is computed by the simulator program using ray casting. At specified intervals during the simulated traversal of the laser, the intersection of the laser line with the object is computed by ray-casting. If there are no intersections, the object is not in the laser’s path at that position. If there are intersections, the first intersection with the object is on the surface the laser is incident on.
  • a ray is constructed from the laser-object intersection to the receiver, and the number of intersections with the object is computed for this new line. If there are zero intersections, it means that the camera can see the intersection, and this point is added to the detected point cloud. If there are one or more intersections, then it means that though the laser is incident on the object surface, the camera is unable to see the surface because it is obstructed by other parts of the object (i.e. this is in the shadow cast by the object structure itself). These computations are repeated after moving the laser- camera set-up by one step interval. Smaller intervals produce a denser estimated point cloud, but will also be slower because of the computational cost of computing more intersections.
  • the simulator evaluates the estimated point cloud, and computes a score 504 using a weighting function 505 for the surfaces of the CAD model indicating the importance of that section of the object.
  • weighting function 505 is computed from the GD&T information as follows.
  • the CTF areas of the object which are areas that are of particular relevance as to whether an inspected object would be acceptable or not, are obtained from the GD&T information of the object.
  • a cell-level weighting function is defined as a set of weights between 0 and 1 for each CAD model cell. For example, a cell containing a feature that is critical to function may be weighted 1.0, whereas an area of the object that has a large tolerance for error may be weighted 0.1.
  • the areas of the object that need not be measured can be weighted 0.0.
  • a default weight of 0.5 can be assigned if no specific CTF areas are available for the cell.
  • Cells in the original CAD model may also be resampled to a finer resolution (creating more cells) when finer control of the weighting function is desired in cells containing CTF areas.
  • the sampling resolution of the surface mesh (which is the area covered by each cell) may also be adjusted to be finer (using more cells) based on the tolerances specified in the GD&T information.
  • the estimated point cloud is scored based on the density of points, the weighting function, and any other factors considered to be important for processing steps downstream, such as the overlap of coverage between the left and right imagers.
  • the density of points around each point can be computed as the number of points within an area of a given radius from the point. While being most accurate, this method of determining the density of points can be computationally slow, given that there are typically hundreds of thousands of points in the point cloud. While having less accuracy, a quicker determination of approximate density may be obtained by computing the number of points within each cell of the CAD model, divided by the area of the cell.
  • a CAD model is often described in a format where the 3D object surface is modeled as a collection of cells, and the cells are typically triangular. This approximate density can be computed with very little additional computation if the identity of the cell where the intersection of the laser beam and the object surface occurs is maintained during the computation of intersections using ray-casting.
  • the point density computed for each cell of the CAD model is multiplied by the weight for the cell from the weighting function to obtain a cell-level score.
  • the normalized (scaled between 0 and 1) weighted sum of the point densities over all cells is used as the point cloud score for comparing between point clouds.
  • a determination is then made to determine if the score is adequate 506.
  • the score is considered to be adequate if it exceeds a pre-defined threshold. For finer discrimination, each cell score may be required to meet a cell-level threshold; or a single threshold can be specified for the overall point cloud for coarser scoring. Additional factors may also be used to determine if a score is adequate.
  • an overlap score can be computed from the number of points in each triangular cell of the CAD model of the object from view A and view B.
  • the overlap score for the cell is then computed as (1 - difference in count between the two views / the greater of the two counts), and an overall overlap score computed as the normalized weighted sum of the cell-level overlaps.
  • a threshold can be set for the minimum overlap needed, and point clouds with overlap scores not meeting the threshold are considered to have an inadequate score.
  • the system parameters and corresponding rest positions that achieved that score in the simulation are output 515 for storage in the library 2010 as the recommended system set- up parameters for that particular object. They can then be accessed by the scanning system prior to inspection of physical samples of the modeled object, to ensure that the captured scans can be used to adequately inspect the object.
  • Parameter combinations are generated by incrementing capture parameters to step through their range by a step (equal to their step-size).
  • Capture parameters include the start and end of the imager scan path, start and end position of platen or conveyor, the height and angle(s) of the imaging system, the rest position of the object and number of traversals. Parameters may be incremented one at a time, or some may be grouped to
  • a new simulation is launched 503 with the updated capture parameters 508. The process is repeated with the new parameter combination till an adequate score is obtained.
  • the parameter settings for the best scoring scan are saved 509 along with the rest position used.
  • the cell-level weighting function is re-computed 510 so that the cells where the threshold is already met in the previously stored best scan are reassigned to 0.0, and cells where the threshold was not met are reassigned to 1.0.
  • a different rest position is selected for simulation, and the process is repeated using the new cell-level weighting function. Determinations are made as to whether additional rest positions remain to be considered 511. At each rest position available, as determined in 511 , the additional cell weights that meet the threshold are reassigned to 0.0 before selecting the next rest position 513 until an adequate score is achieved.
  • the overall scan parameter recommendation includes the capture parameters for each rest position that was used in achieving the final adequate score.
  • Figure 6 shows a two-laser profilometer system modeled by the simulator.
  • the laser emitter is modeled as a line shown in dark blue and the receiver is also modeled as a line shown in light blue.
  • the set-up simulation includes a right laser 6001, a right receiver 6002, a left laser 6003, and a left receiver 6004.
  • the object 6000 for simulated inspection is rendered from a CAD model.
  • the light emitted from the lasers 6001 and 6003 would be reflected back to the receivers 6002, 6004. Areas of focus 6005 can be imaged by the system, whereas areas falling outside the areas of focus would not be imaged.
  • simulated traversals of the object with the two laser profilometers are modeled, with each traversal comprising of multiple steps.
  • the intersection of the laser with the object is computed at each step of the traversal, and it is determined if the receivers 6002, 6004 can obtain a measurement for each point on the surfaces of the object the laser is incident on. If the number of intersections with the object of the ray-cast from a receiver to the laser intersection points is zero, then the point is visible. That is, the path of the emitted light from the laser to receiver is not blocked by another part of the object, and the reflected angle of the light is such that it would be received at the receiver. These points are added to the estimated point cloud.
  • the operator retrieves the system
  • An optical acquisition unit such as a vision camera system, can be used to identify the object on the transport or platen, inspect the set-up of the imagers as well as the positioning and orientation of the object, to verify that the set-up matches the recommended set-up for that object.
  • a notification can be sent to the operator in case of detected mismatch.
  • the simulator can produce a density map visualization of the computed point cloud that indicates the point densities on the object surface.
  • a visualization of the point cloud showing the points visible to each imager, and the overlaps between any two imagers can also be presented to the user.
  • Another visualization can show points visible in one rest position, a second rest position, and the overlap between the two point clouds.
  • the point cloud(s) may be displayed as an overlay over the CAD model to visualize the coverage of the surface area of the object.
  • These visualizations can be ran with different parameters, and a side-by-side or overlapping view comparing the point cloud from one set of parameter settings with another set can be used for fine-tuning the capture set-up parameters.
  • one or more laser profilometers may be mounted on a mechanical stage, such as a rack and pinion assembly, so that the profilometer can sweep across the object being inspected.
  • the object is positioned upon a supporting surface, which can be a stationary platform, a mechanical stage with control of up to six degrees of movement, or a conveyor belt.
  • the one or more profilometers can be mounted to a robotic arm which can inspect an array or matrix of objects that can be made up of a plurality of manufactured parts or components.
  • mechanical assemblies used to move the object support surface or the one or more laser profilometers such as with a rack and pinion mechanical assembly drift over time due to wear, repeatedly reaching the end of travel, and or changing directions.
  • Rotational components of the transformation are less sensitive to the mechanical hysteresis of the system than the translational components.
  • Translational components can be averaged but not rotational components.
  • the calibration target may include a plurality of surfaces at different depth levels from the laser profilometers, with the plurality of surface including fiducial elements in predetermined locations.
  • the calibration target may include a plurality of step-like surfaces 701, with the surfaces including fiducial elements in the form of marker holes 702 at predetermined locations.
  • Other types of fiducial markings, such as raised surfaces, lines, grooves, or gaps may be used.
  • the fiducial element marker holes 702 may be singular holes, or may be groups of holes arranged in different patterns.
  • the fiducial element holes 802 and surfaces 801 of the calibration target are identifiable, and their location can be determined. Detecting and locating the fiducial element holes is implemented as a calibration process.
  • the obtained point cloud of the calibration target shown in Figure 8 is segmented into separate planes, with each plane representing the one of the surfaces of different depths, and the bottoms of the holes in that surface. For example, in the calibration target shown in Figure 7, each step surface and the bottom of the holes in that step surface would form a plane. Then, the calibration process examines the planes to detect and locate the fiduciary element holes.
  • the determined positions of the holes in the point can be compared to the known dimensions of the calibration target, and the inspection system may be calibrated so that the measurements in the obtained point cloud match the known dimensions of the calibration target.
  • the calibration process determines an affine transformation matrix (size 4x4) that is used to remove rotations and shear caused by scanning with the laser module at non-orthogonal angles to the direction of motion of the platen or conveyor carrying the object.
  • the holes, or other fiduciary elements, in the calibration target are distributed in a pattern, with clusters of holes located at different known positions on the calibration target, that makes the holes identifiable based on the number of holes in the cluster and alignment with other clusters.
  • the calibration target shown in Figure 7 includes a plurality of surfaces 701, and a plurality of spaced out clusters of holes 701 arranged in patterns.
  • the calibration process uses a random sample consensus (RANSAC) algorithm to find a transformation matrix that produces the best fit between the hole locations in the obtained point cloud and the known hole center locations of the calibration target.
  • RANSAC random sample consensus
  • the calibration process requires two parts. In a first part, the calibration target must be scanned to obtained a point cloud, and the point cloud must be separated into the planes representing each surface of the object. In the second part, the fiduciary elements in each surface must be detected, and their locations determined so that the transformation matrix can then be found. There are two alternative methods to implement the first part of the fiduciary element detection and location process.
  • the obtained point cloud of the calibration target is separated into planes for each different depth surface in a process using the following steps:
  • the plane model parameters include four values: a three-dimensional unit vector normal to the surface of the plane, and the distance of the plane from the origin of the coordinate system.
  • FIG. 9 illustrates an example of such a histogram, with spikes 9002 included at the distances from the reference plane 9000 of each step of the calibration target as represented in the point cloud.
  • the population 9001 represents the number of points at the given depth.
  • This first method of determining the planar segments may also include the following, to provide a refinement of the planar segments: 13. For each planar segment, fit a plane model; specifically, calculate the normal of the segmented points.
  • the obtained point cloud of the calibration target is separated into planes for each different depth surface in a process using the following steps:
  • d. Select the perturbation with the lowest entropy.
  • e. Decrease the perturbation step size and repeat until termination conditions are met.
  • termination conditions including a maximum number of iterations, an entropy threshold, a statistical evaluation of the distances around each spike, or other analytical measurement. The final histogram is then analyzed.
  • each local maximum in the histogram create a group of points each with a distance to the defined reference plane that falls within a tolerance threshold to the local maximum.
  • Each of these groups represents a planar segment of the original cloud corresponding to a detected plane (step surface or the bottom of the holes).
  • Each of the detected planes is analyzed to detect fiducial holes.
  • the top plane is cyan, while the bottom of the holes in that plane are magenta.
  • the calibration process may search for these holes on the surface planes rather than by finding the bottom of the holes, because the bottom of the holes may be incomplete due to shadowlike occlusions created by oblique angles of detection. Points on the top surface plane are less subject to such occlusion.
  • the area surrounding the hole must have a sufficiently dense population of points to define the hole boundary.
  • the plane segment, representing the area around the holes, of the point cloud is transformed into a two-dimensional image.
  • This two-dimensional image is processed to detect and locate the holes in the image.
  • the location of each detected hole is then transformed back to the original three-dimensional space of the original point cloud, obtained by recording depth information using a recording device (such as the laser profilometer) that scanned the image by moving relative to the calibration target.
  • a recording device such as the laser profilometer
  • the steps in this process are outlined in more detail below, with the motion components (including the scanners, cameras, and platen or conveyor) expressed as the x and y directions, and depth expressed as the z direction. 1. Analyze the point cloud to determine the median (or average) space between points in the x and y directions.
  • the range of point locations in the x and y dimension correspond to pixel locations.
  • the image may be processed to mitigate noise.
  • image processing techniques can be employed to detect and locate the holes, such as template matching via correlation convolution, Haar object detection,
  • convolutional neural networks and Hough transformation of edge pixels. Due to the binary nature of the image and the anisotropic scaling of the image, the preferred embodiment employs convolutional template matching using an integral image of the plane.
  • the detected hole centers are used to compute a transformation matrix that
  • the transformation matrix is 4 by 4 for transforming homogeneous coordinates. As such it has only 12 variables since the last row is fixed to be (0, 0, 0, 1). Solving for these 12 values will require four points in three dimensions.
  • the preferred embodiment employs the RANSAC algorithm.
  • the transformation matrix is determined, it is applied to subsequent scan data.
  • Application of the transformation matrix allows for the removal of skew or shear introduced by the relative motion between the object and the scanner combined with the orientation of the Scanner-
  • Figures 12A and 12B illustrate system configurations for performing the product quality inspection on one or more objects.
  • the inspection system may include a transport 150 that moves the objects 160 being inspected in direction A.
  • the laser module 200 may be included on guides that allow it to move perpendicular to the transport direction, for example moving in directions B and C as shown in Figure 12A.
  • the laser field of view 240 it makes multiple passes over the object being inspected.
  • the laser beam 140 forms a line on the transport 150 that corresponds to the field of view 240 which is parallel to the direction of the transport 150 as shown by direction A.
  • the laser module’s field of view makes multiple scans of the object.
  • Overlapping scanning i.e. raster scanning
  • raster scanning of the object may be used, which results in reduced noise inherent in translation of both the object 160 on transport 150 and laser module 200, and which increases the sampling resolution.
  • Having multiple data per point on the point cloud allows for reduction in the total scan error by minimizing data variance.
  • the laser module 200 may be rotated 90 degrees so that the laser field of view 240 is parallel to the directions B and C. In this case the laser module 200 captures a scan of the object equal to the width of the laser field of view 240. The laser module 200 continues moving across the transport 150 in direction B or C. When the laser module 200 travels a distance equal to the laser field of view 240, the laser module 200 captures another scan.
  • the laser module 200 may be positioned on guides that allow it to move parallel to the transport 150. The laser may module may move back and forth above the object 160 being inspected, with the reflected laser cone passing repeatedly over the object.
  • the laser module 200 orientation is rotated 90 from Figure 12A such that the laser field of view 240 is perpendicular to the transport 150 direction.
  • the laser module 200 moves in direction D scanning a predefined area.
  • the laser module 200 travels in the direction C by a distance equal to the width of the laser FOV 240 before starting to another scan in the direction E.
  • the laser module continues scanning in directions D and E as the laser module 200 moves in direction C. Once the laser module 200 scans the width of the transport, it repeats the process of scanning in directions D and E as the laser module moves in direction B.
  • a known object is placed onto the transport 150 in a predetermined position and orientation.
  • the object is moved along the transport and is first imaged by the optical acquisition unit 110.
  • the captured images of the object from the optical acquisition unit 110 are used to both identify the object and to determine its orientation.
  • the predetermined, or determined, coordinate position and orientation of the object 160 on transport 150 can be sent to a processing unit connected to the laser scanning profilometer module.
  • This processing unit may control the laser profilometer so that it scans only the area of the transport where the object is located.
  • the profilometer may be configured to scan the entire width of the transport instead of focusing on a particular region where an object is determined to be positioned.
  • the profilometer scans the entire transport
  • the point cloud resulting from the scans is cropped for the region of the transport where the objects exist.
  • multiple objects may exist in the same profilometer scan, which can then be separated out during subsequent processing.
  • the laser profilometer module can locate the part and directly extract the part out from the background.
  • a system processing unit loads the CAD data corresponding to the identified object from a CAD model database, and converts it to a uniform point cloud.
  • a predetermined object may also be loaded, such that it is not necessary to first capture images from the optical acquisition unit.
  • the CAD model from the database for the predetermined object is simply loaded.
  • the point cloud from the laser profilometer will be overlaid and registered with the CAD point cloud.
  • the deviation between the CAD point cloud and the point cloud from the laser profilometer is calculated to create a heat map of the differences.
  • the operator can use the 3-D registered heat map for visualizing any variations of the object from the CAD model.
  • These visualizations of object variations are recorded over time along with object counts, mold cavity positions, material compositions, and other factors relevant to the monitored manufacturing process.
  • the visualizations are presented in a selectable chronological sequence with object illustrations containing with 3-D registered heat maps.
  • Each individual image of the objects is also registered so that an operator can observe a representation of a degradation condition represented by the heat map over a selected period.
  • This form of dynamic visualization can be useful for diagnostic purposes by providing illustrations of wear patterns and for determining in advance when the production of a given object will exceed tolerance by projecting the monitored
  • a histogram of error can be presented to the operator.
  • the system can be configured to identify and flag parts that exceed predefined tolerance as being defective.
  • the user can configure the system to produce a notification, stop the production or sort the detective part from other parts with the use of a part picker or similar means.
  • Figure 13 illustrates a block diagram which shows the schematic illustration of the quality inspection system components.
  • the inspection system 100 includes an optical acquisition unit 110.
  • the optical accusation unit includes an optical imager 400 to capture images of the objects being inspected, an image A/D converter 402 converts an analog image signal output from the optical imager 400 and generates digital image data which is then output to the pixel correction 404.
  • the image device 400 has an image capture element using CMOS (complementary metal oxide semiconductors) which are arranged in a line in the main scan direction which is perpendicular to the transport 150 as shown by arrow A.
  • the image capture device could be a CIS (contact image sensor).
  • the optical acquisition unit 110 could also include a stereo imager.
  • the pixel correction 404 corrects for any pixel or magnification abnormalities.
  • the pixel correction 404 outputs the digital image data to the image controller 440 within the system processing unit 170.
  • the system further includes a first laser module 200a, and optionally a second laser module 200b. Although two laser modules are shown, the system may include any number of laser modules. With more laser modules, the system may be able to capture profiles of larger objects, or capture more profiles of complex object during the inspection process. The use of multiple laser modules further minimizes occlusions or shadows.
  • An occlusion is a blockage of the laser beam 140 traveling from laser driver 210a, 210b to object and back to the laser sensor 250a, 250b.
  • the laser module 200a, 200b mounted within the quality inspection system 100 can be changed manually or dynamically to optimize the viewing angle, field of view, or working distance in order to minimize occlusions or shadows to control the position and motion of the laser module 200a, 200b to get the best scans.
  • Each laser module 200a, 200b includes a laser driver 210a, 210b, a laser sensor 250a, 250b, and laser processing 405a, 405b.
  • the laser processing 405a, 405b may be a microprocessor or processing circuitry, or may include a communication module to receive processing commands from an external processing device.
  • the laser processing packages and sends the unstructured point cloud representing the measurement data to the image controller 440 which stores the data in the laser coordinate memory 435.
  • the optical acquisition unit 110 sends the digital image data to the image controller 440.
  • the image processing unit 485 then processes the digital image to identify the object being scanned.
  • the object may be a predetermined object and the identification by the optical acquisition unit may not be necessary.
  • predefined setup parameters or recipes for laser module 200a, 200b may be pulled out system memory 455 to optimize the configuration of the laser module 200a, 200b.
  • the characteristics of the object related to the predefined setup parameters in the system memory 455 may be, for example, reflectivity, color, geometry, or surface finish.
  • Predefined setup parameters or recipes may be laser driver 210a, 210b strength, profiles per second, or exposure.
  • the operator configures the inspection system 100 to perform the required inspection either through the operator control panel 120 or network interface 445.
  • An operator control panel 120 includes an operator input 125, which may be, for example, a mouse, keyboard, touchscreen, or touchpad.
  • An operator display 128 and alert notification 130 may also be included in the operator control panel.
  • the alert notification 130 may be, for example, a Red, Yellow, and Green light stack, audio alarm or other visual alert mechanism.
  • the network interface 445 allows the system to communicate with external processing units and network devices.
  • the network interface may be connected to a local area network, wide area network, or to the internet through wired or wireless connections.
  • the network interface can be of any form known to those skilled in the art including, but not limited to, Ethernet, USB, Wi-Fi, cellular, or other data network interface circuit.
  • the network interface 445 also provides a means to remotely control the inspection system 100 by supplying various types of information required for inspection.
  • the network controller 490 manages the network interface 445 and directs network communications to a machine controller 430.
  • a system memory 455 may also be connected to the system processing unit 170, as well as other processing components in the inspection system.
  • the system processing unit 170, and other processing units may be, for example, one or more microprocessors, one or more coprocessors, one or more multi-core processors, one or more controllers, processing circuitry, an ASIC (application specific integrated circuit) or FPGA (field programmable gate array), or a combination thereof.
  • ASIC application specific integrated circuit
  • FPGA field programmable gate array
  • the processing units may be configured to execute instructions stored in the system memory 455 or otherwise accessible to the processing circuitry, such as over the communications network through network controller 490. These instructions, when executed by the processing circuitry, may cause the inspection system to perform one or more of the functionalities described herein.
  • the memory 455 may include volatile and/or non-volatile memory, and may be a hard disk, random access memory, read only memory, cache memory, flash memory, an optical disc, or circuitry configured to store information.
  • System memory 455 may be any non-transitory computer readable storage medium, in communication with the system processing units.
  • the memory may be configured to store information, data, applications, or program
  • the system processing unit 170 includes an image controller 440 configured to receive and analyze images from the optical acquisition unit 110.
  • the received images from the optical acquisition unit 110 are stored in the image buffer memory 475 to hold the processed images.
  • An image processing unit 485 is connected to the system processing unit 170.
  • the image processing unit 485 is configured to receive the captured images, in digital form and after pixel correction, from the optical acquisition unit and perform further processing on the images. These captured images may be stored in an image buffer memory 475, with is connected to the image processing unit 485, the system processing unit 440, or both.
  • the image buffer memory store recently captured images sent by the optical acquisition unit during an inspection process.
  • the image processing unit 485 may interact with the image controller 440 of the system processing unit 170.
  • the system further includes a CAD model database 425, which stores CAD data profiles for objects being inspected during the quality control inspection process.
  • This CAD model database 425 may include predetermined CAD profiles of objects anticipated to be inspected, and may further be an updatable database allowing a user to manually, or automatically, store additional CAD profiles in the database for new objects being inspected.
  • the CAD model database 425 may be updated by executing a process to upload CAD models stored on a local memory, or to upload CAD models over the network communication interface.
  • the image controller 440 is also configured to receive and analyze unstructured point cloud measurement data from the laser module unit 200a and 200b.
  • the received unstructured point cloud data from the laser module unit 200a, 200b are stored in the laser coordinate memory 435 to hold for further processing by the image processing unit 485.
  • a machine controller 430 is configured to control the operation of the optical acquisition unit 110 and/or the laser module units 200a, 200b.
  • the machine controller may control the timing of the image capturing, and/or use the image capture data to control the movement of the transport 150 includes transport speed, and the movement and scanning of the laser module units.
  • a timer unit 480 may be included in the system processing unit 170, as well as a part position detector 450 and a defect detector 460.
  • the part position detector may be configured to determine the position and orientation of the objects being detected, including position on the transport and orientation relative to either the direction of transport or another fixed coordinate in the system.
  • the defect detector 460 is configured to determine, based on comparisons between the laser module scans of an object being inspected and stored data files regarding expected dimensions of the object, whether the product is within certain tolerance levels. If the product is not within tolerance levels, the defect detector may issue a defect warning. The defect detector may also be configured to determine defects based on the captured images from the optical acquisition unit 110.
  • a part rejection unit 470 may be in communication with the defect detector 460 in the system processing unit 170.
  • the part rejection unit 470 may receive information from the defect detector 460, and based on a determination of whether the defects meet predetermined quality threshold requirements, issue a command to stop the inspection process, flag the inspected object as defected.
  • the part rejection unit 470 with and the timer unit 480 will track the defective object and take action to remove the defective object from the transport 150 or alert the operator of a defect condition through, for example, the alert notification 130, or the network interface 445.
  • the inspection system further includes controls for controlling the transport 150 and laser modules 200a, 200b during the inspection process.
  • Transport motor control 465 may control the speed and timing of the transport mechanism that moves the objects along the inspection transport path.
  • the transport motor control 465 may be connected to a motor driving a conveyor belt, platen or rollers forming transport 150, and may drive the motors at varying speeds to control the speed of the objects being inspected.
  • the transport motor control may also stop the transport 150 when a defective object is detected, allowing an operator time to remove the object from the inspection line or take other appropriate action.
  • Two laser motor controls may be included, with a laser X-Y motor control 410 controlling the movement of the laser module 200a, 200b in an X-Y plane, and a laser Z motor control 420 controlling the movement of the laser module units in the Z direction (or height).
  • the laser X-Y motor control 410 may move the laser module units 200a, 200b in the X direction to cause the laser to traverse the width of the transport 150 thereby causing the laser field of view 240 to make multiple scans over the object being inspected.
  • the laser Z motor control 420 may move the laser modules up and down so that the system can inspect objects of varying height. This allows the distance between the laser driver 210a, 210b and the object being inspected 160 to be maintained.
  • Figure 14 illustrates movement of the laser module 200 during a scanning process in which the object moves in direction A and the laser module moves only in directions perpendicular to direction A.
  • the laser module 200 is rotated 90 degrees such that the laser field of view 240 is parallel to direction A.
  • the field of view 240 of the laser scans a path across the transport 150.
  • the laser module alternatingly moves in directions B and C, traversing repeatedly over the object as it conducts the scans.
  • the laser module thus creates an overlapping scan pattern with a slant, with the slant being imparted because the object being inspected continues to move in direction A as the scans are performed.
  • This method of scanning creates regions of overlap between successive traversals of the object.
  • the gray region 1400 depicts a region of overlap between traversal Ci and Bi. Similar overlap regions are created between each pair of successive traversals.
  • Overlap scanning can be performed by either altering the velocity of the transport 150 or laser module 200. For complete coverage of the scanned part, the following relation must hold true:
  • Y is the length of the laser field of view 240
  • W is the distance the laser module 200 is scanning across the transport 150 as shown in Figure 14
  • V_B and V_L are the velocities of the transport 150 and laser module 200, respectively.
  • the angular relationship between laser field of view and the transport may also be considered. If the angular relationship is not zero degrees, meaning they are not parallel, one end of laser field of view will be closer to the transport than the other end. Since the object being scanned is moving in direction A as shown in Figure 12A, any point on the surface of the object will appear to increase or decrease in Z direction or height based on the angular relationship of the laser field of view with respect to the transport. As laser module traverses in directions B and C as shown in Figure 14, each laser traversal over the object will create a stair step in the scanned data in Z direction as the object travels through this laser field of view angular relationship in direction A. This angular relationship can be compensated for by performing a calibration.
  • a scan of the transport is performed without any object present. Any difference in height between the two end points of the laser field of view is known to be due to the angular relationship.
  • the angular relationship may be stored in system memory, allowing the system processing unit to subtract out any stair step in the scanned data.
  • the angular relationship direction A of the object must be zero degrees with respect to the laser field of view. In this case, any point on the surface of the object will appear to move in either directions B or C based on the angular relationship of the laser field of view with respect to the transport direction A.
  • a scan of a calibration target is used to determine the angular relationship. Any stair stepping difference along the edge between two laser module traversals is known to be due to the angular relationship.
  • the angular relationship may be stored in system memory allowing the system processing unit to subtract out any stair step in the scanned data.
  • the distance W that the laser module is scanning across the transport may be the width of the transport 150 when scanning the entire transport, or W could be a region of the transport if the position of the object on the belt is known.
  • W By setting W to correspond to only a region of the transport when the position of the object is known, higher scanning throughput at the same resolution, or the same throughput at a higher resolution, may be achieved.
  • This configuration allows the capture of higher resolution images based on the distance between motor steps and acquisition rate when traversing the laser over a larger area, and not limited to the native resolution of the profilometer itself.
  • performing the overlapping scans helps to reduce sensitivity to noise.
  • Noise is introduced by inherent measurement error and mechanical noise due to the object and laser module 200 moving in the inspection process.
  • By performing scanning so that there is an overlap of the scanning of segments of the inspected object multiple point cloud data points for the same spatial location on the object are created. This multiplicity of point cloud points for a single spatial location allows for the calculation of a mean and variance (or confidence interval) for that spatial coordinate.
  • By scanning portions of the inspected part several times and averaging the data overall noise in the system is not only reduced, but remaining noise can be accurately calculated and reported. More details regarding the procedure for maintaining the performance of the scanning system are described herein.
  • Regions of overlap may be created between consecutive traversals by only moving along one axis at a time.
  • the object may be stationary in the A direction while the scanners move in the B or C directions.
  • the overlap regions would be rectangular shaped.
  • Figure 15 illustrates a scanning process in which the laser module 200 moves in the same direction as the object 160 along the transport path 150, as well perpendicular to the transport path.
  • laser module 200 may move in direction C perpendicular to the direction of motion A of the object 160 along the transport 150, with the laser field of view 240 scanning a path across the transport 150 in a raster scanning process.
  • the laser module 200 also moves in direction A at the same speed as the object as it conducts the scan.
  • the laser module 200 When the laser module reaches the end of a scan, it then moves a distance A’ that is equal to the length of the field of view 240 of the laser. Then, the laser module 200 reverses direction and moves in direction B while at the same time moving in direction A at the same speed as the object 160. Thus, relative to the object 160, the laser module 200 makes a scan of the entire object once, as shown in Figure 15.
  • Figure 16 is flow diagram illustrating the processing performed to convert the CAD models into a format useable for interfacing with a corrected 3-D point cloud data obtained from the laser module 200 scans of an inspected object.
  • the CAD models can be dgn, dwg, dxf, or stl file formats, or other formats that allow generation of a CAD file.
  • This process may be performed in real time as the objects are being inspected, or can be performed prior to initiating a quality inspection on objects.
  • CAD models of the objects anticipated to be inspected may be retrieved and processed, with the results being stored in CAD model database 425.
  • a CAD model 1600 is loaded into the system, and mesh conversion step 1601 is performed on the CAD model, resulting in a CAD mesh 1602.
  • extract planes step 1603 is performed on the CAD mesh 1602 resulting in CAD plane meshes 1604.
  • the CAD plane meshes 1604 represent a plane or view of the object from a predefined point of view.
  • planes or views may be manually generated in advance by editing the CAD model using CAD editing software such as MeshLab and are available to the extract planes 1603 step. This may be done so that the CAD point of view matches with the predetermined object location and orientation, set by the operator placing the operator at a particular location and in a particular orientation in the inspection system.
  • the planes or views can be automatically identified based on the natural rest positions of the object in reference to the orientation of the transport. The orientation of transport may be obtained by over scanning the object to capture the transport in the scan. Once defined, these planes can automatically be generated.
  • the critical to function (CTF) parameters for the CAD plane meshes 1604 may then be extracted by the extract CTF parameter step 1605.
  • CTF critical to function
  • CTF parameters are user defined dimensions that need to be measured and verified. These parameters can be manually identified by an operator, or can be determined automatically by system processing. For example, many times a soft- or hardcopy drawing of the object is provided with tolerance and CTF data included, which can be extracted using document processing or OCR software.
  • CAD CTF parameters 1609 are generated and stored in the CAD model database 425.
  • the CAD plane meshes 1604 also undergo format conversion step 1606 to generate CAD point clouds 1607 and CAD range images 1608 which can also be stored in the CAD model database 425.
  • CAD range images 1608 represents the six faces of the object being inspected.
  • Figure 17 is a flow diagram illustrating the processing performed on image data from the optical acquisition unit 110. This processing includes identifying the objects on the transport 150 being inspected, as well as the position and orientation of the object 160 on the transport 150. As shown, the optical acquisition unit 110 performs an image capture
  • a part recognition and lookup process 1707 is then performed on each separated part 1706 from the digital image 1702.
  • a pattern matching process is performed in which the system performs a CAD model lookup 1708 to compare the objects located in the CAD model database 425 to separated parts 1706.
  • the bounding boxes 1704 may include different objects being inspected that require different CAD models to be referenced.
  • the pattern matching processing may be any commonly known pattern matching algorithm. When a match is found in the CAD model database 425, the pertinent CAD point cloud 1607 is identified for later processing.
  • the CAD range image 1608 is identified and then used in the part registration step 1709 to identify the translation & orientation matrices 1710, which are used to rotate the CAD data to match the orientation of the point clouds obtained from the laser module 200 scans of the object 160, as explained below.
  • the surface dimensions of the objects 160 in the digital images 1702 may be determined, and used as an alternative to the pattern matching used in the part recognition and lookup process 1707.
  • Figure 18A illustrates the process to merge data from the laser module scans with the information obtained from the CAD model information for the object being scanned. This process determines differences between the actual object dimensions obtained from the laser scans, and the expected object dimensions based on the CAD model tolerances.
  • raw, unstructured point cloud data 1810 is obtained from laser module scans of the object 160, as described above with respect to Figures 14 and 15.
  • line profile data sets (raw point cloud data acquired by a single laser field of view 240 line from the laser driver 210) will be offset from one another by a distance equivalent to v*At, v being the traversing velocity of the object, and At is the time interval between the a line scan of an object and the next line scan.
  • the timer unit 480 generates a trigger signal to the laser module 200 to capture a line profile.
  • Timer unit 480 also generates a trigger to machine controller 430 to capture positional data of the transport and laser module. Using the trigger signal allows the system to read positional data from, for example, an encoder, as opposed to calculating the position based on velocity.
  • a rephasing of the raw point cloud data 1811 is implemented to align and average these multiple point clouds together in a single rephased object point cloud 1812.
  • the raw point cloud data is rephased by translating all points in the point cloud by an amount equivalent to - v*A t.
  • At can be taken to be the total time elapsed from a set time, for example the time at which the machine was engaged, to the current line scans timestamp.
  • the object being inspected is then registered to the CAD model 1813 for that particular object.
  • the CAD point cloud 1607 is retrieved, as well as translation and rotational matrices 1819 to match the CAD point cloud to the rephrased object point cloud 1812.
  • the rephased object point cloud is also registered at 1814 to the CAD model, resulting in a point cloud aligned to the CAD model. That is, the obtained rephased point cloud is overlaid on the CAD model information for the particular object.
  • the differences between the CAD model point cloud 1607, which comprises structured data, and the registered point cloud 1814 are then calculation at step 1815. Subtractive matching may have been conducted to register the CAD model to the point cloud data as closely as possible.
  • This matching finds a minimum total variance between the CAD model and the point cloud to determine the optimal overlay between the two.
  • a D value is calculated representing the absolute difference between these two points’ spatial coordinates. This D value will be calculated as the magnitude of the vectors of the two points.
  • x, y, and z represent the spatial coordinates of the CAD and point cloud data.
  • This same D value is used in the CAD model registration process mentioned above, with the CAD model being registered to the point cloud data based on the orientation where the variance of D values is at its minimum.
  • a set of point cloud differences 1816 is obtained.
  • a heat map creation process 1817 is then implemented to generate a heat map 1818 showing areas of high deviation between the CAD model and the obtained point cloud from the object scans.
  • the heat map may be used by an operator to determine if the deviations between the dimensions of the actual object being inspected, and the expected dimensions of that object from the CAD model database are outside of predetermined tolerance levels set in the extract CTF parameters 1605. If the heat map 1818 shows areas outside the tolerance levels, the part may be identified as defective.
  • the heat map may be user configurable, with the user setting predetermined limits for when a deviation will be considered relevant enough to indicate on the heat map.
  • Figures 18B and 18C illustrate details of the registration process 1813 from Figure 18 A.
  • the overall registration process is divided in two stages: 1) library generation, and 2) point clouds registration.
  • Figure 18B depicts the library generation steps to create a set of characteristic vectors (or weights) for use in the next stage.
  • the system receives a reference 3D mesh model 1820 representing the whole geometry of a rigid object.
  • the various rest positions of the rigid object are calculated based on the convex hull and center of gravity of the reference mesh model 1821.
  • the rest positions of a rigid object are position where the object is stable on a horizontal plane.
  • a library of views from a number of rotated versions (e.g., 360 at 1- degree increment) of the reference mesh model is generated 1822.
  • a view corresponds to a 2D projection of the reference 3D mesh model at a given rest position and a specific angle. Note that down-sampling may be applied during the 2D projection process to reduce computational cost.
  • the library of views is then processed in the Eigen- space using a Principle Component Analysis to generate a set of characteristic vectors (or weights) for each element in the library set 1823.
  • the elements in the library set may be determined by performing Eigen decomposition on the generated set of library views 1822 corresponding to the rotated versions of the images of the rest positions, after subtracting out the mean image (which is the average of the set of views).
  • the data after subtracting the mean contains information regarding the differences between the set of views.
  • the top eigenvectors, corresponding to the highest eigenvalues, are chosen to generate a projection matrix.
  • Each image in the set of library views has its vector representation, w, in
  • the system saves both the Eigen-projected library (or projection matrix) and the characteristic vector set 1824 in a library for use in recognizing new point clouds generated in future scans. If more than one rigid object is considered in the system, the steps 1820 to 1824 can be repeated to generate the final library (or projection matrix) and characteristic vector set. These saved library and characteristic vector set will be used in the next stage for 2D and 3D point cloud registrations as described in Figure 18C.
  • Figure 18C depicts the process of 2D and 3D registration and matching of point clouds.
  • the system receives a target 3D point cloud 1830, which may be the rephrased part point cloud.
  • This target point cloud represents the geometry of a rigid object in one of its rest positions.
  • the corresponding rigid object could be one of the rigid objects considered in the previous library generation stage.
  • the saved Eigen- projected library (or projection matrix) and the characteristic vector set are loaded 1831.
  • the 2D projection of the target point cloud is calculated to generate its characteristic vector (weights) based on the Eigen-projected library generated from the previous stage.
  • down-sampling may be applied during the 2D projection process to reduce computational cost.
  • the Eigen-projected library is assumed to be generated using the same rigid object.
  • a coarse registration is performed by identifying the rest position and rotation angle of the rigid object that best matches the target point cloud. This is achieved by finding the best match or closest distance between the characteristic vector from the target point cloud and the saved characteristic vector set of the rigid object(s) with the corresponding reference mesh model(s) from the previous stage.
  • a metric such as the least mean squares error or other error metrics, can be used to compute the distance (or matching error) between the characteristic vectors.
  • the match can be also computed as an Euclidean distance between the two vectors.
  • the search for the closest match can be optimized by using a KD-Tree representation to store the vectors.
  • the object and the orientation of the closest match is output as the hypothesis for the object and orientation of the object represented by the target point cloud.
  • the translation is computed as the difference in the position of the centroids between the image from the 2D projection of the target point cloud and the closest match determined above.
  • the rigid 2D geometric transformation includes information such as translational and rotational matrices.
  • the geometric transformation obtained is applied to the target point cloud 1835.
  • the output of this step is a translated and rotated target point cloud that will be coarsely aligned with the identified reference mesh model (and the associated rigid object). This may be implemented as a 3D transformation matrix, T, using the computed translation and rotation.
  • the transformation matrix T is applied to align the target point cloud with the CAD model, producing a registration between the point cloud and the CAD model.
  • the scanning process can produce artifacts in the scan data, resulting from secondary reflections of the laser off of the surface being scanned. This typically occurs in concave portions of the surface, or in areas of the part that are close to parallel with the laser path.
  • the resultant artifacts appear as small clusters of points distinctly separated from the main point cloud. These are effectively filtered by the following artifact filtering method. At each point in the cloud, all nearby points are identified within a predetermined connectivity radius threshold. These points are then all considered connected. Points that are directly connected are logically clustered together. Secondary connections also constitute connectivity.
  • a predetermined size threshold in either point count, or geometrical dimensions
  • the fine 3D registration 1836 may use algorithms such as the Iterative Closest Point (ICP), which is a popular algorithm due to its simplicity.
  • the inputs of the algorithm are the target and reference point clouds, initial estimation of the geometric transformation (from step 1834) to align the target to the reference, and criteria for stopping the iterations.
  • the output of the algorithm is a refined geometric transformation matrix.
  • the algorithm includes:
  • a geometric transform is an affine transform consisting of one or a combination of translation, scale, shear, or rotation transformations.
  • the finer 3D registration and iterative process to determine the refined geometries transformation matrix may also be implemented using an Iterative Closest Face
  • ICF Registration Iterative Closest Face
  • ICF is an algorithm that iteratively optimizes a transformation matrix to rotate and reposition a point cloud so that it is optimally aligned with the CAD model mesh.
  • ICP a movable point cloud is aligned to a stationary reference point cloud over the course of multiple iterations.
  • a movable point cloud is aligned to a stationary mesh.
  • the resultant transformation matrix is typically constrained to only perform translation and rotation. There is no spatial distortion, and usually no scaling.
  • ICF locates the nearest point that is on a face of the CAD model mesh during each pass, instead of utilizing the nearest point in a reference point cloud. This point may occur anywhere on the continuous planar surface of any face of the mesh, within the boundaries of the face, at the edge of the face, or at one vertex of the face. It may not occur on the extended face plane outside the boundaries of the CAD model face.
  • a CAD model mesh may be very 'sparse' in vertices.
  • a simple cube model may only contain 8 points. Using just those 8 points, ICP could only relate every point in the movable point cloud to one comer of the cube. This will not produce a good fit to the model, particularly when the point cloud contains point on all surfaces of the cube.
  • a CAD mesh can be interpolated over all its faces, to produce a point cloud that has a uniform grid of coverage over all the surfaces. ICP is then performed relative to the interpolated CAD point cloud, rather than the original mesh.
  • ICP has a granularity/quantization error arising from the discrete points at which the interpolation is performed.
  • the values of a, b, and c can be determined from the cross product of the vectors AB and AC.
  • the value of d can be determined by substituting the values into the equation and solving for d using one of the vertices.
  • CAD models are typically far more compact than point cloud representations of the same object.
  • the point on the CAD model surface, that each movable point is optimized toward, is precisely located without the granularity/quantization error that maybe present in the ICP approach.
  • ICF also allows for reduced memory utilization as the algorithm is executed, because the CAD model, which is more compact than the point cloud, can be used without interpolation, as well as potentially faster computation because fewer reference faces need to be searched to find the nearest.
  • the final geometric transformation resulting from the process is then applied 1838 to the target point cloud to obtain a final alignment or registration with the reference mesh model. Finally, the difference between the transformed target point cloud and the same reference 3D mesh is measured 1839. The matching result is recorded online or passed to the next step of the inspection system 1840.
  • Figure 19 provides an illustration of how the quality inspection system checks for defects in the objects being inspected.
  • the system may automatically check tolerances to the defined CTF specifications and will notify an operator if system tolerances are being exceeded during the inspection of an object.
  • the mold will wear over time. As the mold wears, the parts coming from the mold will have dimensions altered from the expected dimensions in proportion to the amount of wear on the mold. When a mold becomes overly worn, the parts produced will no longer be within the tolerance range for their proper and intended use.
  • the point cloud differences 1816 calculated for each point in Figure 18 A, along with the CAD CTF parameters 1609 are also retrieved, and are used to determine whether each point in the registered point cloud is a CTF in step 1901. If a point is not a CTF point, then the point is ignored at 1902. However, if the point is a CTF, then an update to the running tolerance is performed at 1903. A check is then made to determine if the running tolerance exceeds a predetermined threshold at 1904. If the threshold is not exceeded as defined by the extract CTF parameters 1605, then the running tolerance is determined to be within acceptable limits at 1905. However, if the running tolerance exceeds the predetermined threshold, the system issues a notification 1906 to the operator.
  • the defective object may also automatically be removed from the transport 150 using any commonly known removal method in the industry.
  • Defects specific to manufacturing of parts by injection molding may include flow lines, burn marks, sink marks, jetting, delamination, short shot, flashing or other defects.
  • CAD CTF Parameters 1609 may also include surface defects or optical characteristics such as, but not limited to, color, texture or finish in order to identify contamination such as dust, oil, or other foreign objects present in the separated parts within the digital image captured by the image acquisition unit 110.
  • step 1901 would determine if the optical characteristics exceeds the specified optical CAD CTF Parameters.
  • Figure 20 provides an example of the inspection system 1100, using a conveyor belt as the transport 150.
  • the conveyer belt may be placed around rollers 1120 that move the conveyer belt as they turn in the direction shown by A.
  • the rollers may be driven by a transport motor control 485.
  • an optical acquisition unit 110 may be located at a first position along the transport 150, and the laser module 200, including the laser driver 210, laser sensor 250, and laser beam 140, is located further downstream on the inspection path.
  • the object 160 being inspected is moved by the conveyer belt so that it passes under the optical acquisition unit 110, if present, and the laser module 200.
  • a belt tensioner 1140 may be provided beneath the moving conveyor belt to provide a solid surface for inspection, and to remove any vibrational motion in the conveyor belt. The reduced vibration motion lessens the noise in the obtained optical images and laser scans of the inspected objects.
  • the conveyer belt may be marked to allow for automated calibration of optical acquisition unit 110 or laser module 200.
  • tick marks 1110 Beveled markings along one side or on both sides of the transport, such as tick marks 1110, are used for calibration and location identification on the transport.
  • other markings may be included to assist in identification of the objects being inspected or the position of the objects along the transport.
  • These references can include numbering, differently shaped ticks, or other identifying marks.
  • a single reference tick on either side of the transport may be used. This tick mark will be elongated with relation to the other uniform ticks for easy identification.
  • the system may perform a modulation transfer function (MTF) of the tick marks, and optical acquisition unit 110 can additionally be calibrated for sharpness, blooming, focus, and depth of field using the tick marks as a reference.
  • MTF modulation transfer function
  • tick marks Due to the uniformity of the tick marks, position, depth of field, and resolution can be calibrated for both optical acquisition unit 110 and laser module 200, along with identification of image and point cloud distortion due to non-perpendicularity of tools. From this data, distortion can be calculated and corrected for.
  • the tick marks can also control for velocity flutter by identifying any changes in velocity not recorded by the transport motor control 465. This information can be used to correct the rephasing point cloud data 1811 by adding a phase shift.
  • Figure 21 provides a flow diagram detailing an overview of the process 2100 to determine if an object being inspected is defective.
  • a CAD model of the object is uploaded and converted at 2101, resulting in a CAD model mesh point cloud.
  • the object 160 moves down the transport 150 in the inspection system 100. If present, the object is imaged by an optical acquisition unit 110.
  • the object passes under the laser acquisition modules 200 which perform laser scans.
  • the laser scans result in the capture of a 3-D point cloud of the object at 2102.
  • the 3-D point cloud of the object is aligned with the CAD model point cloud at step 2103, and a comparison between the two is then performed at 2104. Differences in points are determined, and a determination is made if the object 160 being inspected is defective at 1205.
  • the object 160 may be rejected from the transport and statistics regarding defective objects may be updated at 2106. If the object is not defective, then the statistics are updated to count another acceptable product passing the inspection process at 2107. Statistics may include environmental characteristics such as temperature and humidity as well as running tolerances against critical to function parameters. The statistical information can be related back the mold or the specific cavity based on the number formed on the object or other unique characteristics. This statistical information can be used to schedule preventative maintenance for both the machine and mold.
  • Figure 22 provides a sample user interface that may be displayed to the operator at the end of the inspection process showing the inspection results.
  • This user interface provided to the operator further includes the following functionality:
  • a user interface 2200 may be displayed to the operator on the operator display 128 or remotely through the network interface 445.
  • the user interface 2200 may update real-time as the objects 160 are inspected or the operator may review the historical data for objects inspected or statistical trends.
  • the user interface 2200 may organize the objects inspected to show only the objects exceeding or close to exceed tolerances so operator can focus on problem areas.
  • the user interface 2200 may include a display of the CAD model 2210, a point cloud 2220 from the laser model 200, as well as the histogram of error 2250 and 3D heat map of errors 2230.
  • An operator can select the CAD model 2210 to open the CAD Model database 425 to display, add, or edit the CTF parameters as used by extract CTF parameters 1605.
  • the point cloud 2220 may be the raw part point cloud 1810 or the registered point cloud 1814 as defined by the operator preferences.
  • the operator may also define if data from one or multiple planes scans are displayed in the 3D heat map of errors 2230.
  • An inspection report 2240 may also be displayed.
  • the inspection report 2240 may include the critical to function parameters with ranges and the actual measured values.
  • the results may be color coded to enable the operator to quickly identify measurements that exceed tolerances. For example, red may be used to indicate a measurement exceeded its tolerance while yellow may indicate near the tolerance.
  • the user interface 2200 may also allow the operator to edit critical to function parameter limits thereby changing what is or is not acceptable.
  • the inspection report 2240 may also include a drawing of the object with the measured dimensions for the object shown directly thereon. Overlaying the measured dimensions with the drawing displays the measured results in a format similar to the original drawing specification.
  • the user interface 2200 may also include history trends for the critical to function parameters and statistical process control information across multiple batches or shifts. Using the trends and statistical process control, the mold wear over time can be identified and tracked. The data could be used to correlate changes in incoming material or process changes such as temperature or pressure. The operator can take the appropriate action to fix or replace molds that are producing objects that exceed tolerances. In cases where the mold has multiple cavities, the objects from specific cavities may be identified and tracked across multiple batches or shifts.
  • the baseline CAD model of the mold can be loaded and a color heat-map overlay of the errors for an object or a batch of objects can be overlaid with the mold to indicate where on the mold tolerances are being exceeded. This will characterize and track the mold from its initial installation to the time the mold needs to be repaired, reconditioned, or replaced.
  • a display of the heat maps over time may be provided to the operator to show the operator the wear of the mold over time.
  • Each mold may contain one or more cavities that produce objects.
  • each cavity will be produced based on a CAD model.
  • each cavity will have unique deviations from the CAD model.
  • Each cavity can be characterized when the mold is made to produce a cavity CAD mesh unique to each cavity.
  • an operator is able to identify which cavity made the defective object. This will characterize and track each cavity from its initial installation to the time the cavity needs to be repaired. This could be displayed to show the operator the wear of each cavity over time.
  • the inspection process for an object requires several motion traversals between the object and scanning device to ensure that the entire region of interest of the object has been scanned.
  • Components of the motion system that moves both the scanning devices and the object itself, are subject to wear and performance degradation that can result in misregistration of the scanning data from each traversal with respect to the others.
  • This degradation can invalidate carefully obtained calibration information intended to enable accurate and precise registration of scanning data from multiple traversals of an object.
  • Algorithmic methods of registration exist that can compensate for some of this degradation, yet without intervention the degradation may persist or worsen. Further, analysis of this degradation can be useful in improving the compensation and prediction of system failure when adequate compensation is no longer possible.
  • the mechanical assemblies used to move the profilometer or other scanning devices, as well as the platen or conveyor that advances the object being inspected may cause noise in the measurements and misregistration.
  • the noise may change over time as the mechanical assemblies become worn, shift positions, and suffer performance degradation with use, and as connections between parts are altered from operation of the system.
  • the introduction of noise and mis registration requires continued calibration of the system to compensate for these conditions.
  • a calibration process to monitor the overlap areas between subsequent traversals that project onto a known surface, such as the top of the surface that supports the part being inspected is used. This top surface that supports the object may be referred to as a“floor” of the inspection system. If misalignment is determined in the overlap area, the system either notifies the operator that a calibration is required or automatically calibrates the system and documents the event.
  • a transport mechanism 150 such as a conveyor belt or platen may move the object along an inspection path, and the scanning device 200 may be moved relative to the object with a traversal mechanism 270, such as a leadscrew, and mounting assembly 260, as illustrated in Figure 1A.
  • the position of the scanner may be tracked by monitoring an encoder mounted on the traversal mechanism.
  • the mounting assembly constrains the attitude of the scanner to ensure it points in a consistent direction.
  • Complete scanning coverage of the region of interest of the object may be accomplished by moving the scanner back and forth traversing the object several times creating multiple traversal scans which contain regions of overlap, as illustrated in Figure 14 and the description of Figure 14 herein.
  • a key aspect of this configuration is that some of the translational elements may change velocity by altering direction or speed. Changes in direction, particularly reversals of direction, may manifest mechanical or electrical hysteresis in the position or attitude of either the object or the scanner. For example, as the scanner is moved by rotation of the leadscrew, mechanical hysteresis may exist due to tolerance limitations in the mounting assembly. This may lead to several effects when the leadscrew changes rotational direction or speed. Changes in rotational direction or speed may impart a different amount or direction of torque on the mounting assembly. Due to mechanical tolerance limits, this may introduce changes in the attitude of the scanner. Further, the mechanical hysteresis may introduce an offset in the scanner position indicated by the encoder. The unwanted changes in the attitude of the scanner and mechanical hysteresis may result in apparent spatial displacement of corresponding points within the overlap regions of the traversals.
  • points recorded during different traversals may correspond to the same physical location on the object yet have different coordinate values.
  • the difference between these points is referred to as a spatial displacement. This spatial displacement causes the points in the traversals to be misregistered.
  • FIG 23 depicts the process for monitoring performance of the scanning system, correcting spatial displacements, and notifying a user when the degree of spatial displacement exceeds a threshold, thus allowing the user to request a service procedure or recalibration so that the scanning continues to perform accurate object scans.
  • the process begins with the scanning of an object 2301, using scanners to traverse over the object two or more times such that there exists an overlap in the scanned regions.
  • the region of overlap does not necessarily need to incorporate the object of the scan and may include only the surface on which the object rests, such as the top of the conveyor or platen.
  • the regions of overlap are detected 2302 and the overlapping regions are segmented from the traversals.
  • the detection of regions of overlap may be accomplished by assuming that the positional accuracy of the one scan is sufficiently similar to that of previous scans. Detection and location of fiducial elements on the supporting surface may also aid in establishing regions of overlap.
  • the precise degree of displacement between the overlap regions is estimated and recorded 2303.
  • the displacement may be estimated using well known algorithms such as Iterative Closest Point (ICP).
  • ICP Iterative Closest Point
  • the degree of displacement is classified 2304 and logged in a memory 2305.
  • the classification may be based upon changes in velocity. For example, one classification may be defined to be when one of the mechanical assemblies used to move the profilometer or other scanning devices changes from one direction of travel to another. In this case, one class could occur when the direction of travel changes from left to right, and another class occurs when the direction of travel changes from right to left. If the classified degree of displacement exceeds a predetermined threshold 2306, a transformation matrix is created to correct the displacement and register the full traversal scans to each other thus creating a complete scan of the object 2308.
  • the classified degree of displacement in the overlap regions is compared to the last record 2307. If there is a change in the spatial displacement, a projection will be calculated of when the spatial displacement in the overlap regions will exceed a pre-determined threshold 2312, and the user will be notified 2309 or 2313. The user may optionally be notified about the degree of displacement 2309. If the degree of displacement exceeds a predetermined threshold or cannot be corrected 2310, the user may be notified of a request for a service procedure to correct the cause of the displacement or a recalibration of the system 2311.
  • the history of displacements of a class may be tracked and analyzed 2313 to predict when the degree of displacement is likely to exceed a predetermined threshold 2310.
  • the user may optionally be notified about the predicted failure 2313, thus allowing the user to take actions to recalibrate or perform a service procedure before the performance of the inspection system degrades to an unacceptable level.
  • An operator using the inspection system first loads CAD models for the objects to inspect into the inspection system.
  • the CAD models may already be included in a CAD model database, as explained above, or may be loaded by the operator into the CAD model database by the operator prior to beginning the inspection process.
  • the objects to be inspected are then placed on the transport at a known position and in a predetermined orientation, either manually by an operator or automatically, and the transport motors control drives the transport so that objects are moved into the system at a known velocity.
  • the position and orientation may be determined by imaging the object with an optical acquisition unit.
  • optical acquisition system captures at least one image of the object.
  • the at least one image is sent to a system processing unit, such as described above with respect to Figure 13.
  • the system processing unit analyzes the images and determines an outline of the surface of the object.
  • the system processing unit performs pattern matching techniques between the determined outline of the surface of the object and the CAD range images from the CAD model database. Based on the pattern matching results, a corresponding CAD model is selected from the database for comparison to the object being inspected.
  • the system processing unit further uses the captured images to determine the position and orientation of the object on the transport.
  • a corresponding CAD model for the object being inspected is selected from the database for comparison to the object.
  • the system processing unit instructs the laser module where to collect data along the transport.
  • the laser module traverses perpendicularly to the transport direction, back and forth over a predefined area as objects are moved along the transport, such that the laser module field of view passes over the object one or more times.
  • Reflected laser light is received at a laser sensor, and the received laser light is used to generate a 3-D point cloud of the scanned object.
  • This raw part point cloud of the scanned object is sent to the system processing unit, which performs corrections to the point cloud based on the velocity at which the object was moving along the transport as well as the velocity and direction of the laser module as the scans were performed. As noted earlier, the corrections to the point cloud could also be performed by using the positional data.
  • This corrected 3-D point cloud of the object is then prepared for analyzing.
  • the system processing unit retrieves the CAD point cloud for the object.
  • the CAD point cloud is rotated so that it matches the determined coordinate geometry of the object.
  • the corrected 3-D point cloud from the laser scans of the object is then interpolated to a predetermined geometric grid for comparison to the CAD model.
  • the interpolated corrected 3-D point cloud and CAD model are paired and a series of D values are calculated for, and associated with, each point in the point cloud.
  • the D values are the subtractive difference between the relative positions of the CAD model corresponding to the inspected object and corrected 3-D point cloud data of the object 160.
  • the D values may correspond to a set color, based on user preference.
  • a user may select red for D values outside a predetermined tolerance allowance, and green for values within the tolerance.
  • the predetermined tolerance is based on how much deviation between the CAD model for the object and the point cloud of the scanned object is allowed.
  • the CAD model with the color overlay will be generated and saved in a report, and a smoothing process may be applied so that the colors look uniform in the final overlay in the heat map.
  • the smoothing of D value parameters can be performed by averaging or other means, in order to obtain a smooth gradation between color-coded segments of the visual representation of the point cloud data.
  • the visual representation of the point cloud data may be presented as a set of color-coded point cloud data points, a set of color-coded point cloud data points layered atop an image of registered CAD model, or a color-coded polygon mesh created by forming polygon surfaces between the point cloud data points.
  • polygon surfaces are color-coded according to their assigned D value, being the average of D values for the points that the polygon surface connects.
  • the transport stops, or the object being inspected will be removed from the inspection line, and color overlaid heat map for that particular object may be shown on the operator control panel or saved to a failure report.
  • the system sends out an alert to notify operator.
  • the alert may be an audible or visual alert presented at the operator control panel. Alerts may be color coded to impart urgency, based on user preferences. When the alerts are audible alerts, different tones, sounds, or volume may be used to signify the urgency.
  • the alerts may also be a predefined or custom e-mail or text message sent over a network interface to predetermined recipients.
  • D values do not exceed a certain threshold as defined by CAD CTF parameters, the transport and the object are allowed to continue.
  • the color overlaid heat map for that particular object may be shown on the operator control panel or saved to an operator report.
  • the optical acquisition unit and laser module 200 inspect the plane or face of the object presented to the optical acquisition unit and/or laser module.
  • the object may have CAD CTF parameters on the plane or face not presented to the optical acquisition unit and/or laser module. That is, one surface of the object will be contacting the transport, and thus will not be visible to the optical acquisition unit and/or laser module positioned above the transport.
  • the system may include an additional laser module positioned under at least a top surface of the transport.
  • the belt may be made of a transparent material, allowing the bottom laser module to scan the surface of the object through the transparent transport.
  • both the bottom and top laser modules would scan the object being inspected, capturing laser scans of all surfaces.
  • the scans of the bottom surface are combined with the scans from the top laser module to form a combined point cloud representing all surfaces of the object.
  • This combined point cloud is then compared to the CAD modules in the database, as described above.
  • An additional optical acquisition unit may also be placed underneath the transport, to capture images of the surface in contact with the transport.
  • the transport may comprise two separate belts, with a small gap in between.
  • a laser module may be placed underneath the gap, with its field of view parallel to the length of the gap. As an object being inspected crosses over the gap from one transport belt to the other, the laser module positioned under the gap would capture scans of the bottom surface.
  • these scans of the bottom surface may be combined with the scans from the top laser modules to form a combined point cloud representing all surfaces of the object.
  • the object being inspected may be flipped or rotated to expose a hidden plane of object as it moved along the transport.
  • the object may be simply flipped or it may be automatically captured and rotated so that all planes of the object are presented to the optical acquisition unit and/or laser module for inspection.
  • the objects to be inspected may be funneled to the optical acquisition unit which may identify the object to determine if the correct plane with the CAD CTF parameters was presented to optical acquisition unit. If the correct plane of object is presented, then it is allowed to continue along the transport. If the wrong side is presented, then the object is flipped over before be being inspected by the laser module. This has the advantages of getting all the objects in the same orientation and allows for a simpler flipping mechanism and reducing the area on the transport that needs to be scanned by the laser module.
  • a golden object that is known to be within specification may be available.
  • the golden object would be scanned with the laser module to automatically generate and upload the CAD profiles for the object anticipated to be inspected, and where further updates may be perform in CAD model database.
  • the operator would place the golden object on the transport and configure the system processing unit to store the CAD plane meshes and associated CAD point clouds, CAD range images into the CAD model database for future part recognition and lookup.
  • the operator may add the required CAD CTF parameters to the CAD model database either through the operator control panel or network interface.
  • the inspection system may send an alert to the operator when it detects an object being scanned that it fails to identify in the part recognition and lookup process so that the operator could take the appropriate action.
  • the golden object with known dimensions may be inspected to verify the calibration of the inspection system.
  • the operator configures the inspection system for calibration and manually or automatically places the golden object on the transport to be scanned by the laser module.
  • the working distance, focus, magnification and similar image parameters for the laser module can be calibrated.
  • Polygonal scanners are commonly used in laser print engines, bar code scanners. Polygon scanners can be used for line-oriented scanning objects at fine resolutions.
  • a spinning polygon mirror may be used to create a laser beam that performs high-speed linear scanning across the transport.
  • the polygonal mirror may be rotated via a motor, and may be supported by a ball bearing or an air-bearing rotary spindle, to produce smooth rotation to minimize the distortions within laser beam.
  • the laser sensor could be a linear array, CMOS, or similar technology known in the art.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Automation & Control Theory (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Manufacturing & Machinery (AREA)
  • Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Investigating Materials By The Use Of Optical Means Adapted For Particular Applications (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

La présente invention concerne un système et un procédé permettant d'effectuer une inspection de qualité en temps réel d'objets. Le système et le procédé font appel à un transport pour déplacer des objets en cours d'inspection, ce qui permet d'effectuer une inspection en direct. Au moins une unité d'acquisition optique est prévue, laquelle a capturé des images optiques des objets en cours d'inspection. Les images optiques capturées sont mises en correspondance avec des modèles d'objets de CAO, et le modèle de CAO correspondant est extrait. Un laser dont un faisceau lumineux d'éclairement a une longueur d'onde dans la gamme des violets ou des ultraviolets effectue ensuite des balayages des objets, qui sont formés en nuages de points tridimensionnels. Les nuages de points sont comparés aux modèles de CAO extraits pour chaque objet, des CTF étant comparées à des informations d'entrée utilisateur ou de modèle de CAO et l'objet étant déterminé comme étant acceptable ou défectueux sur la base de l'amplitude d'écart entre le nuage de points et le modèle de CAO.
PCT/US2020/030940 2019-05-02 2020-05-01 Inspection automatisée d'objets en points denses à 360 degrés WO2020223594A2 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP20727498.6A EP3963414A2 (fr) 2019-05-02 2020-05-01 Inspection automatisée d'objets en points denses à 360 degrés
CN202080048597.0A CN114041168A (zh) 2019-05-02 2020-05-01 自动化360度密集点对象检验

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US16/401,890 US11042146B2 (en) 2017-11-17 2019-05-02 Automated 360-degree dense point object inspection
US16/401,890 2019-05-02

Publications (2)

Publication Number Publication Date
WO2020223594A2 true WO2020223594A2 (fr) 2020-11-05
WO2020223594A3 WO2020223594A3 (fr) 2020-12-17

Family

ID=70779929

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2020/030940 WO2020223594A2 (fr) 2019-05-02 2020-05-01 Inspection automatisée d'objets en points denses à 360 degrés

Country Status (3)

Country Link
EP (1) EP3963414A2 (fr)
CN (1) CN114041168A (fr)
WO (1) WO2020223594A2 (fr)

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112381908A (zh) * 2020-11-27 2021-02-19 三峡大学 一种地形扫描点云边界线提取方法
CN112446907A (zh) * 2020-11-19 2021-03-05 武汉中海庭数据技术有限公司 一种单线点云与多线点云配准的方法及装置
CN112504123A (zh) * 2020-12-07 2021-03-16 重庆顺泰铁塔制造有限公司 一种用于输电塔板件自动化检测设备和方法
CN112902868A (zh) * 2021-01-20 2021-06-04 上海云铸三维科技有限公司 一种随形轮廓扫描表面形貌的方法及装置
CN113701648A (zh) * 2021-08-13 2021-11-26 浙江四点灵机器人股份有限公司 一种基于三维线激光扫描仪的金属轮毂尺寸测量装置及方法
CN113701821A (zh) * 2021-09-01 2021-11-26 西南科技大学 一种基于局部特征的建筑铝模板识别结构及其方法
CN113701626A (zh) * 2021-08-10 2021-11-26 哈尔滨岛田大鹏工业股份有限公司 汽车纵梁3d机器视觉检测方法
CN113793296A (zh) * 2021-08-06 2021-12-14 中国科学院国家天文台 一种点云数据处理方法及装置
CN114282379A (zh) * 2021-12-28 2022-04-05 滁州金诺实业有限公司 一种铝合金铸件无模成型铸造用成品质检管控系统
EP4001834A1 (fr) * 2020-11-12 2022-05-25 Covestro Deutschland AG Procede et appareil de surveillance en ligne de la qualite de la mousse pendant le procede de production de mousse
CN114739312A (zh) * 2022-04-26 2022-07-12 黄晓明 一种手持式路面构造深度激光测定装置
CN114937124A (zh) * 2022-07-25 2022-08-23 武汉大势智慧科技有限公司 基于倾斜摄影的片状目标物的三维重建方法、装置及设备
CN115047471A (zh) * 2022-03-30 2022-09-13 北京一径科技有限公司 确定激光雷达点云分层的方法、装置、设备及存储介质
CN115423835A (zh) * 2022-11-02 2022-12-02 中汽创智科技有限公司 杆状物点云数据处理方法、装置、电子设备及存储介质
CN115544594A (zh) * 2022-09-20 2022-12-30 杭州宏深科技有限公司 通用的自动批量三维cad建模评分方法
EP4113057A1 (fr) * 2021-06-29 2023-01-04 Medit Corp. Procédé de réglage d'une région de balayage
GB2610438A (en) * 2021-09-07 2023-03-08 Adaptix Ltd A device and method of authenticating a component against reference data
EP4202749A1 (fr) * 2021-12-27 2023-06-28 Visionx Inc. Systèmes et procédés d'alignement et de comparaison d'un dessin de référence avec des images d'une pièce
CN116379953A (zh) * 2023-05-30 2023-07-04 武汉中岩科技股份有限公司 一种远距离双目三维变形测量系统的拍摄控制方法
CN116721081A (zh) * 2023-06-12 2023-09-08 南京林业大学 基于三维点云和模态转换的动车侧墙板缺陷提取方法
WO2023172231A1 (fr) * 2022-03-11 2023-09-14 Simtek Simulasyon Ve Bilisim Teknolojileri Muhendislik Danismanlik Ticaret Limited Sirketi Système de commande d'objet
CN116758006A (zh) * 2023-05-18 2023-09-15 广州广检建设工程检测中心有限公司 脚手架质量检测方法及装置
CN117183331A (zh) * 2023-09-06 2023-12-08 张家港爵启科技有限公司 一种基于数字孪生三维模型3d打印建模方法
CN117299583A (zh) * 2023-11-25 2023-12-29 东莞市欧宇精密科技有限公司 一种光学筛选机的控制方法
EP4369124A1 (fr) * 2022-11-14 2024-05-15 AM-Flow Holding B.V. Système et procédé d'inspection

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114888794B (zh) * 2022-04-24 2024-01-30 天津工程机械研究院有限公司 一种机器人人机交互运行路径标记方法及装置

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102008045746A1 (de) * 2008-09-04 2010-03-25 Lufthansa Technik Ag Verfahren zum Vermessen des Innenraums eines Flugzeugs
US20180374239A1 (en) * 2015-11-09 2018-12-27 Cognex Corporation System and method for field calibration of a vision system imaging two opposite sides of a calibration object
CN105844057B (zh) * 2016-04-15 2018-10-19 中国科学院上海技术物理研究所 基于光束和三角面片求交的激光扫描成像快速仿真方法
CN108303037B (zh) * 2018-01-31 2020-05-08 广东工业大学 一种基于点云分析的工件表面形状差异的检测方法及装置
CN108458659A (zh) * 2018-05-03 2018-08-28 华中科技大学无锡研究院 一种叶片非接触式检测装置及方法

Cited By (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4001834A1 (fr) * 2020-11-12 2022-05-25 Covestro Deutschland AG Procede et appareil de surveillance en ligne de la qualite de la mousse pendant le procede de production de mousse
CN112446907A (zh) * 2020-11-19 2021-03-05 武汉中海庭数据技术有限公司 一种单线点云与多线点云配准的方法及装置
CN112446907B (zh) * 2020-11-19 2022-09-06 武汉中海庭数据技术有限公司 一种单线点云与多线点云配准的方法及装置
CN112381908B (zh) * 2020-11-27 2022-05-17 三峡大学 一种地形扫描点云边界线提取方法
CN112381908A (zh) * 2020-11-27 2021-02-19 三峡大学 一种地形扫描点云边界线提取方法
CN112504123A (zh) * 2020-12-07 2021-03-16 重庆顺泰铁塔制造有限公司 一种用于输电塔板件自动化检测设备和方法
CN112902868A (zh) * 2021-01-20 2021-06-04 上海云铸三维科技有限公司 一种随形轮廓扫描表面形貌的方法及装置
EP4113057A1 (fr) * 2021-06-29 2023-01-04 Medit Corp. Procédé de réglage d'une région de balayage
CN113793296A (zh) * 2021-08-06 2021-12-14 中国科学院国家天文台 一种点云数据处理方法及装置
CN113701626A (zh) * 2021-08-10 2021-11-26 哈尔滨岛田大鹏工业股份有限公司 汽车纵梁3d机器视觉检测方法
CN113701626B (zh) * 2021-08-10 2023-08-04 哈尔滨岛田大鹏工业股份有限公司 汽车纵梁3d机器视觉检测方法
CN113701648B (zh) * 2021-08-13 2023-12-19 浙江四点灵机器人股份有限公司 一种基于三维线激光扫描仪的金属轮毂尺寸测量装置及方法
CN113701648A (zh) * 2021-08-13 2021-11-26 浙江四点灵机器人股份有限公司 一种基于三维线激光扫描仪的金属轮毂尺寸测量装置及方法
CN113701821B (zh) * 2021-09-01 2024-03-05 西南科技大学 一种基于局部特征的建筑铝模板识别结构及其方法
CN113701821A (zh) * 2021-09-01 2021-11-26 西南科技大学 一种基于局部特征的建筑铝模板识别结构及其方法
GB2610438A (en) * 2021-09-07 2023-03-08 Adaptix Ltd A device and method of authenticating a component against reference data
WO2023037190A1 (fr) * 2021-09-07 2023-03-16 Adaptix Ltd Dispositif et procédé d'authentification d'un composant par comparaison à des données de référence
EP4202749A1 (fr) * 2021-12-27 2023-06-28 Visionx Inc. Systèmes et procédés d'alignement et de comparaison d'un dessin de référence avec des images d'une pièce
CN114282379A (zh) * 2021-12-28 2022-04-05 滁州金诺实业有限公司 一种铝合金铸件无模成型铸造用成品质检管控系统
WO2023172231A1 (fr) * 2022-03-11 2023-09-14 Simtek Simulasyon Ve Bilisim Teknolojileri Muhendislik Danismanlik Ticaret Limited Sirketi Système de commande d'objet
CN115047471B (zh) * 2022-03-30 2023-07-04 北京一径科技有限公司 确定激光雷达点云分层的方法、装置、设备及存储介质
CN115047471A (zh) * 2022-03-30 2022-09-13 北京一径科技有限公司 确定激光雷达点云分层的方法、装置、设备及存储介质
CN114739312B (zh) * 2022-04-26 2024-04-23 黄晓明 一种手持式路面构造深度激光测定装置
CN114739312A (zh) * 2022-04-26 2022-07-12 黄晓明 一种手持式路面构造深度激光测定装置
CN114937124A (zh) * 2022-07-25 2022-08-23 武汉大势智慧科技有限公司 基于倾斜摄影的片状目标物的三维重建方法、装置及设备
CN115544594A (zh) * 2022-09-20 2022-12-30 杭州宏深科技有限公司 通用的自动批量三维cad建模评分方法
CN115423835A (zh) * 2022-11-02 2022-12-02 中汽创智科技有限公司 杆状物点云数据处理方法、装置、电子设备及存储介质
EP4369124A1 (fr) * 2022-11-14 2024-05-15 AM-Flow Holding B.V. Système et procédé d'inspection
CN116758006B (zh) * 2023-05-18 2024-02-06 广州广检建设工程检测中心有限公司 脚手架质量检测方法及装置
CN116758006A (zh) * 2023-05-18 2023-09-15 广州广检建设工程检测中心有限公司 脚手架质量检测方法及装置
CN116379953A (zh) * 2023-05-30 2023-07-04 武汉中岩科技股份有限公司 一种远距离双目三维变形测量系统的拍摄控制方法
CN116379953B (zh) * 2023-05-30 2023-09-05 武汉中岩科技股份有限公司 一种远距离双目三维变形测量系统的拍摄控制方法
CN116721081B (zh) * 2023-06-12 2024-01-26 南京林业大学 基于三维点云和模态转换的动车侧墙板缺陷提取方法
CN116721081A (zh) * 2023-06-12 2023-09-08 南京林业大学 基于三维点云和模态转换的动车侧墙板缺陷提取方法
CN117183331A (zh) * 2023-09-06 2023-12-08 张家港爵启科技有限公司 一种基于数字孪生三维模型3d打印建模方法
CN117299583A (zh) * 2023-11-25 2023-12-29 东莞市欧宇精密科技有限公司 一种光学筛选机的控制方法
CN117299583B (zh) * 2023-11-25 2024-05-03 东莞市欧宇精密科技有限公司 一种光学筛选机的控制方法

Also Published As

Publication number Publication date
EP3963414A2 (fr) 2022-03-09
CN114041168A (zh) 2022-02-11
WO2020223594A3 (fr) 2020-12-17

Similar Documents

Publication Publication Date Title
US12001191B2 (en) Automated 360-degree dense point object inspection
WO2020223594A2 (fr) Inspection automatisée d'objets en points denses à 360 degrés
US11087458B2 (en) Automated in-line object inspection
CN107578464B (zh) 一种基于线激光扫描的传送带工件三维轮廓测量方法
US8103376B2 (en) System and method for the on-machine 2-D contour measurement
US8050491B2 (en) CAD modeling system and method
US11562502B2 (en) System and method for calibrating a plurality of 3D sensors with respect to a motion conveyance
US9704232B2 (en) Stereo vision measurement system and method
CN106469448B (zh) 利用3d视觉进行自动化工业检查
CN112161619B (zh) 位姿检测方法、三维扫描路径规划方法和检测系统
CN104024793B (zh) 形状检查方法及其装置
Lee et al. A framework for laser scan planning of freeform surfaces
US20150153726A1 (en) Method for inspecting a manufacturing device
US7649545B2 (en) Inspection system and method
CN114241061A (zh) 线结构光成像的标定方法、标定系统和标定靶及使用其的测量系统
CN109000583A (zh) 使用激光位移传感器进行有效表面测量的系统和方法
Anchini et al. Metrological characterization of a vision-based measurement system for the online inspection of automotive rubber profile
Emam et al. Dimensional deviation measurement of ceramic tiles according to ISO 10545-2 using the machine vision
CN115210664A (zh) 用于控制对物品的自动检查的系统和方法
García et al. Rail surface inspection system using differential topographic images
CN105783782B (zh) 表面曲率突变光学轮廓测量方法
US20230385479A1 (en) Making a measurement relating to an object depending on a derived measurement surface
Munaro et al. Fast 2.5 D model reconstruction of assembled parts with high occlusion for completeness inspection
Marcotuli et al. A Vision-based Measurement System for Semi-Finished Cylindrical Geometries
Zolfaghari et al. On-line 3D geometric model reconstruction

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20727498

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2020727498

Country of ref document: EP

Effective date: 20211202