CN118511068A - Device for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of the visual quality of underlying objects, in particular electronic components, circuit boards and the like - Google Patents

Abstract

An apparatus (1) for acquiring three-dimensional information of an object and a surface for an artificial vision system for automatic optical inspection of the visual quality of an underlying object (100), the apparatus comprising: a first camera (10) having a vertical optical axis Z and a first flat sensor having an orthogonal axis X, Y lying in a horizontal plane, and a second camera (20) having a horizontal optical axis X ' and a second flat sensor having orthogonal axes Y ', Z ' lying in a vertical plane; a system for illuminating an object from above, the system comprising a plurality of light projectors (30 i); an optical group of first and second cameras (10, 20) comprising a dual objective optical group (55) having a first vertical optical arm (51) associated with the first camera (10) and coaxial with a vertical optical axis Z and a second horizontal optical arm (52) associated with the second camera (20) and coaxial with a horizontal optical axis X'; an optical beam splitter (50) configured to split a light beam reflected by the object (100) into a first light beam directed along the first optical arm (51) and a second light beam directed along the second optical arm (52), wherein the light projector (30 i) is angularly spaced about a perpendicular optical axis Z of the first camera (10), wherein the first and second flat sensors (110, 120) have the same shape and are arranged to acquire the object (100) with the same field of view.

Description

Device for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of the visual quality of underlying objects, in particular electronic components, circuit boards and the like

Detailed Description

The present invention relates to an apparatus for acquiring three-dimensional information of objects and surfaces for use in an artificial vision system for automated optical inspection of objects, particularly but not limited to electronic components, circuit boards and the like.

As is well known, artificial vision systems for vision quality inspection are widely used in manufacturing, semiconductor, food and pharmaceutical industries with high throughput and are based on standard image processing and artificial vision techniques such as, but not limited to, edge detection, connected component analysis, drawing analysis and projection geometry.

These methods are simple and very effective when well-defined entities (such as length, width, color, fine-grained patterns) need to be quantitatively measured; once the measurement is complete, simple tools based on pre-established rules can be used to evaluate whether the observed product meets acceptance criteria.

In particular, in the field, the expression "automated optical inspection" (AOI) generally refers to an automated vision inspection system of the quality of objects, which may consist of electronic components, such as printed circuit boards (i.e. PCBs) and surface mount technologies (i.e. SMTs), in which cameras independently scan the objects under test.

In particular, with respect to electronic assemblies, cameras enable identification of both manufacturing defects (e.g., missing components) and quality defects (e.g., assembly of components or tilted dimensions or shapes). AOIs are commonly used in production processes because they are non-contact test and inspection methods; AOI is implemented in many stages of the production process, including die inspection, solder Paste Inspection (SPI), pre-and post-die casting, and other stages.

As is known, all automatic optical inspection systems essentially require projecting light or one or more luminous wefts onto the object to be inspected and acquiring the light reflected by the object via a digital sensor; the acquired images are analyzed by a processing unit configured to determine physical and/or geometrical properties of the object to be examined based on the light acquired by the sensor.

Today, there is an increasing need to obtain coordinated measurements online in the field of automated quality vision inspection systems for electronic components.

Two-dimensional automated optical inspection techniques using gray scale image analysis or color image analysis obtained from a side camera may no longer be an effective option due to the increasing complexity of today's electronic boards (due to the use of multiple components, multiple joints, the presence of higher density components, and the use of new packaging techniques such as 01005 even 008004 size microchips).

To overcome these limitations, three-dimensional scanning techniques have been effectively combined with AOI and are now used in many applications, such as inspection of microelectronic components and solder paste deposits below 100 microns, as well as other challenging applications.

It is also well known that while these known systems are functional, they all have limitations and have some drawbacks and limitations.

In particular, some limitations stem from the nature of the measurement technique, while others more particularly relate to the measurement of electronic components (SMT and PCB) and include:

it is difficult to ensure a complete measurement of the low parts close to the high parts due to shadow effects (if the reference drawing is projected at an angle, the high sections may cast shadows, thus preventing measurement of the adjacent low sections);

It is difficult to avoid measurement errors caused by multiple reflections between the components (multiple specular reflections between lighting elements such as soldered joints, tinned cables and metal oscillators can cause distortion of the fringe pattern and errors in height measurement);

it is difficult to ensure an all-round rapid, high-precision and repeatable measurement in the micrometer (μm) field.

Another difficulty is the physical complexity and overall size of the components of the image acquisition system of the known system, which requires relatively large dimensions for a complete inspection system and ensures that the different components do not interfere.

Accordingly, there is a need for improving the structure of known artificial vision systems for inspecting visual quality.

The technical task addressed by the present invention is therefore to realise an apparatus for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of the visual quality of an artefact, which allows to eliminate the above-mentioned technical drawbacks of the prior art.

As part of this technical task, it is an object of the present invention to achieve a device for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of vision quality that is simple and efficient.

Another object of the invention is to achieve a device for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of vision quality, which device is compact in size.

Last but not least, the object of the present invention is to achieve a device for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of vision quality, which device ensures fast, high-precision and repeatable measurements.

These and other objects according to the invention are achieved by: an apparatus for acquiring three-dimensional information of an object and a surface for an artificial vision system for automatic optical inspection of visual quality of an underlying object is implemented, the apparatus comprising: a first camera having a vertical optical axis Z and a first flat sensor having an orthogonal axis X, Y lying in a horizontal plane, and a second camera having a horizontal optical axis X ' and a second flat sensor having an orthogonal axis Y ', Z ' lying in a vertical plane; a system for illuminating an object from above, the system comprising a plurality of light projectors; optics of a first camera and a second camera, the optics comprising a dual objective optical group having a first vertical optical arm associated with the first camera and coaxial with the vertical optical axis Z and a second horizontal optical arm associated with the second camera and coaxial with the horizontal optical axis X'; an optical beam splitter configured to split a light beam reflected by the object into a first light beam directed along the first optical arm and a second light beam directed along the second optical arm, wherein light projectors are angularly spaced about a vertical optical axis Z of the first camera, and wherein the first flat sensor and the second flat sensor have the same shape and are arranged to acquire the object with the same field of view.

The light projector may comprise a structured light source or an unstructured light source.

The optics group is preferably, but not necessarily, a telecentric or a double telecentric optics group. In one embodiment, one of the first camera and the second camera is preferably a high resolution camera.

In one embodiment, one of the first camera and the second camera is a camera capable of acquiring an image containing information about the polarization of light incident on the sensor.

In one embodiment, the light projector is positioned with a vertical projection axis.

In one embodiment, the light projector is positioned with an oblique projection axis.

In one embodiment, a plurality of specular reflectors are interposed between the plurality of light projectors and the station for the object for transitioning the optical path of the light from the projectors to the object.

In one embodiment, a series of monochromatic or polychromatic light emitting rings are included, coaxial with the vertical optical axis of the first camera, with decreasing diameters and decreasing distances away from the first camera.

Other features of the invention are further defined in the following claims.

Further characteristics and advantages of the invention will emerge more fully from the description of a preferred, but not exclusive, first embodiment of a device according to the invention, illustrated by way of non-limiting example in the accompanying drawings, in which:

fig. 1 shows an elevation schematic of the apparatus in a first embodiment;

fig. 2 shows a schematic plan view of the apparatus in the first embodiment;

Fig. 3 shows an elevation schematic of the apparatus in a second embodiment;

fig. 4 shows a schematic plan view of the apparatus in a second embodiment.

The following detailed description refers to the accompanying drawings, which form a part hereof.

In the drawings, like reference numerals generally identify like components unless context dictates otherwise.

The embodiments described in the detailed description and drawings are not intended to be limiting.

Other embodiments may exist and other modifications may be made without departing from the spirit and scope of the subject matter in the discussion depicted herein.

As generally described in this context and as illustrated in the accompanying figures, aspects of the present description may be arranged, substituted, combined, and designed in a wide variety of different configurations, all of which are explicitly contemplated and are part of this description.

Referring to the above figures, there is shown an apparatus for acquiring three-dimensional information of an object and a surface for an artificial vision system for automatic optical inspection of the visual quality of an underlying object, the apparatus being indicated in its entirety by reference numeral 1.

The device 1 comprises a first camera 10, typically in an upper position, having a vertical optical axis Z and a first flat sensor 110 having an orthogonal axis X, Y lying in a horizontal plane.

The device 1 further comprises a second camera 20 having a horizontal optical axis X ' and a second flat sensor 120 having orthogonal axes Y ', Z ' lying in a vertical plane.

The apparatus comprises a dual objective optical group 55 comprising one or more lenses and possibly other optical elements acting as optics for both the first camera 10 and the second camera 20.

Preferably, the optical group 55 is telecentric or double telecentric, i.e. it forms telecentric or double telecentric optics with infinite input and/or output pupils.

The optical group 55 includes an optical beam splitter 50 (typically, but not limited to, a prism) configured to split a light beam reflected by the object 100 into a first light beam directed along a first vertical optical arm 51 associated with the first camera 10 and coaxial with a vertical optical axis Z of the first camera 10 and a second light beam directed along a second horizontal optical arm 52 associated with the second camera 20 and coaxial with a horizontal optical axis X' of the second camera 20.

The optical arms 51, 52 referred to are actually optical paths, preferably linear, which may be defined by optical elements and structural elements of known type.

Advantageously, the horizontal optical arm 52 of the optical group 55 and thus the optical axis X' of the second camera 20 is positioned at a first offset angle α with respect to the axis X of the first sensor of the first camera 10.

Advantageously, the dual objective optical group 55 is able to support two cameras with identical or even different size sensors and thus allows measuring objects with different magnification factors for each camera and also two different magnification factors.

The first flat sensor 110 and the second flat sensor 120 are two-dimensional pixel sensors, such as, but not necessarily, CMOS or CCD sensors.

Advantageously, and in general, the first camera 10 is a high resolution camera, wherein the resolution is preferably at least 12 megapixels, which allows high resolution 3D data to be obtained.

Advantageously, the second camera 20 is a camera capable of acquiring images containing information about the polarization of the light incident on the sensor, typically but not necessarily a camera with an integrated polarization sensor, which allows to acquire images without reflection, or images with reflection and glare strongly reduced on reflective surfaces such as, for example and without limitation, plastics and metals.

With the image obtained from the second camera, it is possible to reconstruct the shape of those parts of the component of reflective material (mainly metal, such as welds) that cannot be effectively reconstructed with structured light projection techniques (just because they are reflective material), and thus reconstruct 3D information.

The device 1 comprises a system for illuminating an object 100 to be inspected from above, which system is typically placed in a lower position, which system comprises a plurality of light projectors 30i, typically four light projectors, which are preferably configured to emit structured light, such as DLP (digital light processing) projectors or other projectors capable of emitting structured light stripes, even more preferably configured to emit sinusoidal or binary patterns (stripe images) suitable for implementing a profilometry method called Phase Shift Profilometry (PSP).

In a first embodiment, as illustrated in fig. 1 and 2, the light projector 30i preferably has a vertical projection axis parallel to the vertical optical axis Z of the first camera 10, and the illumination system comprises a plurality of specular reflectors 31i interposed between the plurality of projectors 30i and the station for the object 100 for transforming the plurality of light paths 32i of the light from the plurality of projectors 30i to the object 100.

The individual light projectors 30i and the corresponding specular reflectors 31i are equally angularly spaced about the vertical optical axis Z of the first camera 10.

Typically, the at least one light projector 30i is placed in an angular position coinciding with the angular position of the axis X of the first sensor 110 of the first camera 10 with respect to the perpendicular optical axis Z of the first camera 10.

In a second embodiment, as illustrated in fig. 3 and 4, the light projector 30i has a projection axis inclined with respect to the vertical axis Z and arranged to converge on a station for the object 100.

Also, in the second embodiment, the individual light projectors 30i are equally angularly spaced about the vertical optical axis Z of the first camera 10.

As a non-limiting example, each light projector 30i may include at least one LED light source, at least one lens, an optical beam splitter, and a Digital Micromirror Display (DMD) device, wherein the lens may be positioned between the LED light source and the optical beam splitter, the optical beam splitter may be positioned between the lens and the Digital Micromirror Display (DMD) device, and another lens (or lens system) may be positioned between the DMD and the object to be illuminated.

Generally, in a preferred solution, each light projector 30i comprises at least three different monochromatic LED light sources, each monochromatic LED light source being associated with a corresponding lens, and an optical system interposed between the lens and the optical beam splitter is configured to collimate the light beams emitted by the three LED light sources.

The device 1 may also comprise a system for direct irradiation of the object 100. Such an indirect illumination system preferably comprises a plurality of mono-or polychromatic light emitting rings 40i, or a plurality of light emitting rings arranged along a plurality of rings, coaxial with the vertical optical axis Z of the first camera 10, having an increasing diameter and increasing distance away from the first camera 10.

Furthermore, advantageously, the axis Z' of the second sensor of the second camera 20 is oriented with respect to a vertical axis parallel to the vertical optical axis Z by a second offset angle β, which is uniquely determined by the value of the first offset angle α.

The offset of the angle beta relative to the vertical axis corrects the rotation of the projection caused by the first offset angle alpha.

In particular, β=α+90°, where α+.0 can be set.

Therefore, if the first sensor and the second sensor have the same shape, and the optical system 55 is appropriately sized such that the image transmitted to the camera 10 and the image transmitted to the camera 20 have the correct magnification, so that the field of view 100 is collimated to the sensors in an accurate manner, the object 100 can be acquired in the same field of view by the first camera 10 and the second camera 20 according to the present invention.

Suitably, the light projector 30i and the cameras (10 and 20) are operatively connected to an electronic control unit (not illustrated) which commands and controls the actuation, i.e. the activation of the light projector 30i, the cameras (10 and 20) and possibly also the system for indirect illumination of the object 100.

According to a preferred solution, the electronic control unit is configured to alternately actuate (i.e. activate) the light projectors 30i such that the object 100 to be inspected is illuminated by a single light pattern generated by a single light projector 30 i. In this way, the cameras 10, 20 acquire successive images of the same object illuminated by (structured) light from different angles-without moving either the object 100 or the projector 30 i-and these images can be combined to have a complete and accurate detection of the whole of the object.

Preferably, in embodiments where the projector 30i projects structured light, a series of light patterns are projected onto the object 100 due to alternating activation of the projector 30i itself, which light patterns can then be combined to perform contour measurements.

In a preferred embodiment, the first camera 10 and the second camera 20 are operatively connected to an electronic processing unit (not illustrated) configured to process and combine the images acquired by the two cameras 10, 20 in order to obtain a detection of a characteristic of the object.

The electronic processing unit may be comprised in or by the above-mentioned electronic control unit or may be comprised in or by an external computing device, such as a computer or the like.

The operation of the apparatus for acquiring images to inspect an underlying object according to the present invention is apparent from the description and illustration, and in particular, is substantially as described below.

The object 100 to be inspected is positioned in a station generally lower than the apparatus and is suitably illuminated by a plurality of light projectors 30i and a plurality of mono-or polychromatic light-emitting rings 40 i.

The light beam reflected by the object 100 through the dual objective optical group 55 is split into a first light beam directed along a first vertical optical arm 51 associated with the first camera 10 and coaxial with the vertical optical axis Z of the first camera 10 and a second light beam directed along a second horizontal optical arm 52 associated with the second camera 20 and coaxial with the horizontal optical axis X' of the second camera 20.

Due to the specific structure of the optical system proposed in the present invention, the images acquired by the first camera and the second camera are consistent and stackable.

In order for the inspection readings of the device 1 to be effective, immediate and repeatable, the consistent images collected by the first camera 10 and by the second camera 20 must be properly superimposed and processed by known processing systems, for example by an electronic processing unit in the manner described above.

The basic condition of a correct reading of the device is to ensure alignment of the images acquired by the first camera 10 and the second camera 20 and agreement of at least two corners of the images acquired by the first sensor and the second sensor.

In case the horizontal optical arm 52 is positioned at a first offset angle α (generally coinciding with the angular position of the at least one light projector 30 i) with respect to the axis X of the first sensor 110 of the first camera 10 due to the configuration and overall dimensions of the device 1, the second sensor 120 of the second camera 20 rotates the axis Z' at a second offset angle β with respect to a vertical axis parallel to the axis Z, which second offset angle is uniquely determined by the value of the first offset angle α.

In practice, it has been found that the device according to the invention for acquiring three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of the visual quality of an underlying object is particularly advantageous because it is simple and efficient, has compact dimensions, and because it does not present any interference between the different components.

The device thus conceived for obtaining three-dimensional information of objects and surfaces for an artificial vision system for automatic optical inspection of the visual quality of an underlying object is susceptible to numerous modifications and variants, all falling within the scope of the inventive concept as defined in the claims; furthermore, all the details may be replaced with technically equivalent elements.

For example, special means can be provided to reconfigure the image projected on the object.

In practice, the materials used, as well as the dimensions, may be any according to requirements and to the state of the art.

Claims (14)

1. An apparatus (1) for acquiring three-dimensional information of an object and a surface for an artificial vision system for automatic optical inspection of the visual quality of an underlying object (100), the apparatus comprising: a first camera (10) having a vertical optical axis (Z) and a first flat sensor (110) having an orthogonal axis (X, Y) lying in a horizontal plane, and a second camera (20) having a horizontal optical axis (X ') and a second flat sensor (120) having an orthogonal axis (Y ', Z ') lying in a vertical plane; a system for illuminating the object from above, the system comprising a plurality of light projectors (30 i); -a dual objective optical group (55) having a first vertical optical arm (51) associated with the first camera (10) and coaxial with the vertical optical axis (Z) and a second horizontal optical arm (52) associated with the second camera (20) and coaxial with the horizontal optical axis (X'); -an optical beam splitter (50) configured to split a light beam reflected by the object (100) into a first light beam directed along the first optical arm (51) and a second light beam directed along the second optical arm (52), wherein the light projector (30 i) is angularly spaced about the perpendicular optical axis (Z) of the first camera (10), and wherein the first and second flat sensors (110, 120) have the same shape and are arranged to acquire the object (100) with the same field of view.

2. The device (1) according to the preceding claim, characterized in that said optical group (55) is telecentric or double telecentric.

3. The device (1) according to any preceding claim, wherein one of the first camera (10) and the second camera (20) is a high resolution camera.

4. The device (1) according to any preceding claim, wherein one of the first camera (10) and the second camera (20) is a camera capable of acquiring an image containing information about the polarization of light incident on the sensor.

5. The device (1) according to any preceding claim, characterized in that at least one light projector (30 i) is arranged in an angular position coinciding with the angular position of the axis (X) of the first sensor (110) of the first camera (10) with respect to the perpendicular optical axis (Z) of the first camera (10).

6. The device (1) according to any preceding claim, characterized in that it comprises a plurality of mono-or polychromatic light-emitting rings (40 i) coaxial with the vertical optical axis (Z) of the first camera (10), distant from the first camera (10) with increasing diameter and increasing distance.

7. The apparatus (1) according to any preceding claim, wherein the plurality of light projectors (30 i) comprises at least four light projectors (30 i) that are evenly spaced apart angularly about the perpendicular optical axis (Z) of the first camera (10).

8. The device (1) according to any preceding claim, wherein the plurality of light projectors (30 i) have a vertical projection axis parallel to the vertical optical axis (Z) of the first camera (10).

9. The apparatus (1) according to any preceding claim, comprising a plurality of specular reflectors (31 i) interposed between the plurality of projectors (30 i) and a station for the object (100) for transforming a plurality of light paths (32 i) of light from the plurality of projectors (30 i) to the object (100).

10. The apparatus (1) according to any one of claims 1 to 7, wherein the plurality of light projectors (30 i) have oblique projection axes arranged to converge on the station for the object (100).

11. The apparatus (1) according to any preceding claim, wherein each light projector (30 i) of the plurality of light projectors (30 i) comprises at least one LED light source, at least one lens, an optical beam splitter and a Digital Micromirror Display (DMD), wherein the lens is positionable between the LED light source and the optical beam splitter is positionable between the lens and the Digital Micromirror Display (DMD) and at least one further lens is positionable between a digital micromirror device and the object to be illuminated.

12. The device (1) according to the preceding claim, wherein at least one of the plurality of light projectors (30 i) and preferably each light projector (30 i) comprises: at least three different monochromatic LED light sources, each monochromatic LED light source being associated with a corresponding lens; and an optical system interposed between the lens and the optical beam splitter and configured to collimate light beams emitted by the three LED light sources.

13. The apparatus (1) according to any preceding claim, further comprising means for reconfiguring the image projected onto the object (100).

14. The device (1) according to any preceding claim, wherein the first and second flat sensors (110, 120) are two-dimensional pixel sensors.