KR19990082673A - Automated Methodology for 3D Image Measurement Systems - Google Patents

Abstract

3D image metrology method using auto-verifiable targets by determining the external orientation of images composed of targets and / or patterns or groups of targets.

Description

Automated Methodology for 3D Image Measurement Systems

Background of the invention is a methodology for the automation of manual tasks in a 3D image metrology system to date.

For example, today robots are programmed offline, which means using a computer simulation program to train the robot. Due to technical limitations in the mechanical movement of the robot arm, there is a difference between the programmed and actual position of the robot arm. This difference is measured with a robot calibration system and calibration polynoms for different ankles are derived. By using the correction for the simulation, it is possible to increase the accuracy of the actual robot movement. Three or four CCD cameras are mounted around the robot to observe the measurement volume. Cameras have a specific relationship to the tool center point and detect auto-recognizable targets mounted thereon. These detections activate the calibration process and establish calibration for the simulation.

Targets are designed to facilitate automatic detection and verification. For example, the targets can be designed as concentric white circles on a black background. There is another concentric circle as the label region where 10 bits of target identification is coded around the concentric circle. Concentric circles have a black dot in the center to facilitate manual measurement. Black and white are the colors chosen for maximum contrast.

Thus, part of the invention, that is, an auto-recognizable target, has already been used.

Many attempts have been made to reach this purpose of automation. To date, all attempts have yielded partial results and are not suitable for widespread use.

One approach to automatically identifying each target relies on different forms and forms of targets. Targets composed of various geometric shapes such as circles, rectangles and crosses are used to distinguish 2D patterns of targets. This approach requires a relatively large target and is unacceptable for many different patterns.

Another approach used a target with a linear pattern region added around circular targets. The pattern was similar to a typical barcode. The code is not integrated with the target itself and requires a relatively wider space. These targets are called "coded" targets and were developed by several groups at the time.

Another approach with coded targets uses a bit pattern around the target. The pattern was a binary bit pattern designed independent of the rotation of the target. Two major drawbacks are common to these targets. When using CCD cameras, the distance between the target and the code is too short and it is possible, for example, to use only labels provided with numerical codes such as 1, 2, 3, typically not alphanumeric labels. Insufficient amount of space between the target and the cord degrades the measurement accuracy of the target. Groups with hundreds of targets were identified with this approach. All codes were provided to use only the label for the target, which is the same as the label encoded within the labeling area of the target.

In addition, some systems use target groups to identify one particular group and through that group identify individual targets of the group. The techniques rely on the specific geometric relationship of all targets in the group, which can be identified in the image and / or 3D space.

Another approach to identifying elements is to use groups of targets that have specific geometric relationships with each other and are identified with other targets in the image or even in 3D space. The disadvantage of the above techniques lies in the fact that one always requires several targets and that the technique cannot tolerate many different groups with reasonable economic effort and reasonable space conditions.

In addition the approach has been combined with the use of color to increase the number of identifiable targets.

Another aspect of automation is to establish consistency of unidentified targets through specific individual characteristics such as label, color, shape, and the like. Two basic techniques have been developed to overcome this problem. One uses a geometric relationship between images that can derive geometric conditions, namely epipolar line. Using this technique, it is possible to automatically establish correspondence between multiple targets. A similar approach in which the geometric relationship between image and object space is used is Multiphoto Geometrically Constrained Matching. In another approach, the 3D relationship of the points is used directly. Since the techniques require knowing in advance the outer orientation, relative orientation of the images or both, the techniques both addressed only part of the problem.

Establishing correspondence of corresponding points in the images can be performed using epipolar geometry as well as using the 3D relationship of the points.

The known solutions that can automatically identify each target are somewhat limited, with the following disadvantages:

1. Can't measure targets with sufficient precision (not for all techniques).

2. Although alphanumeric labels are required in many applications, only numeric labels are associated with targets.

3. A group of targets requires more space than a target with an associated label area.

4. In applications with multiple targets, the group of targets may not be sufficiently identified with other targets.

Auto-identifiable targets were previously envisioned as film based 3D image metrology systems. They did not address the imaging characteristics of CCD sensors or other digital imaging systems.

Consistent establishment established initial image orientation without the use of automatic identification targets. For the complete system, they did not allow to check the object automatically.

1. Not provided for a mechanism to automatically load all relevant data for a particular application.

2, not provided for high measurement precision, immunity to occlusions, and alphanumeric labels.

3. It is not provided for automatic identification of adapters that are irrelevant to their spatial location.

Overall, current methods have only addressed part of the problem so far.

Purpose of the Invention

It is an object of the present invention, in part or in whole, to provide partial and / or complete automation for dimensional measurements using image metrology. In automated production systems managed for robotic or other manufacturing systems such as machining, fitting, moving, etc., the system in which 3D image metrology is performed fully automatically is of paramount importance.

It is another object of the present invention to provide a higher degree or more complete automation in 3D image metrology and / or other applications in which the systems are used.

Summary of the Invention

The invention applies to the following dependent tasks:

Investigating targets for automatically determining the external orientation of the image.

Search for targets in several pictures to establish matches in the pictures.

Find targets in different fields of view in the electro-optic system to establish their relationship.

Identification of the target used as a reference for the definition of the coordinate system.

Identifying the measuring adapters for identifying the particular adapter and its geometric and / or other properties.

Examples for the adapters are:

Respective targets knowing certain offsets and other geometric parameters.

Two or more targets in line with other spatial locations for their reference.

Three or more targets in spatial relationship with one or more other spatial locations.

Examples are:

Adapters consisting of three or more visible targets and three or more hemispheres on the object surface. Visible targets and other positions are in a known relative spatial relationship.

Adapters consisting of three or more visible targets associated with one or more locations. The visible targets and other positions are within known relative spatial relationships. Physical examples are spindles in a DigiPen or TI ² system, which is a touchprobe of an Imetric system, and targets on the spindle holder.

LUT

The present invention will be part of the future enhancement of 3D image metrology systems.

It is also applicable to other systems. It can also be used in industrial manufacturing systems.

The present invention relates to 3D image measurement using photogrammetry.

The present invention applies to 3D image measurement, but can also be applied to other electro-optical systems. 3D image metrology applies to all areas where 3D geometric information and other information are extracted from imagery obtained through standard imaging systems or other devices such as imaging radar or sonar, etc. can do. Typical areas of application are dimensional measurements that can be performed in areas such as the arts and medicine as well as industrial processes, from design, development and production to quality control.

The present invention utilizes the following steps in the current implementation in accordance with any combination of individual principles or individual techniques.

1. Measurement accuracy: In order not to reduce the accuracy with which the central target can be placed, the design of the auto-identifiable target is carried out between the central target and the label area, ie the area with the binary or other code, used to identify the target. The distance is big enough When the central target was designed to be given for optimal measurement accuracy, i.e., to be imaged generally on a diameter of 6 pixels, the distance was set to be greater than the diameter of the central target.

2. Partial Occlusion of Label Regions: In many applications, the region portion containing a label may be occluded from other objects between the imaging system and the label region. The above problems can be avoided by combining or combining extra areas in the code, additional areas such as label perimeters, and a priori knowledge using auto-identifiable targets.

3. Labeling Freedom: In order to overcome the problem of applying specific targets on specific points that would like to be labeled differently, a translation between the label of the target and the physical label is used. The auto-identifiable target with label "11" can be used as a target with label "A". This can be done, for example, using a look up table in which a "user" label is associated with each physical label of the target. The lookup table may be user defined or may be fixed. In addition, a double look up table can be used, that is, the label provided by the label area is first translated into another label and then finally associated with the label used in further processing. For example, the label on the target includes the label "11". On the target, the name "A" may be used to identify a person, but in certain applications the target may be used like that target "X". This helps to reduce the number of automatically recognizable targets that are physically required to implement many different applications.

4. Object Measurement Verification: Here, labels of one or more automatically verifiable targets are used to identify the specific object. For example, the hood of the model "A" of the car would have auto-identifiable targets with the label "K", and the hood of the model B of the car would have the auto-identifiable target with the label "Y". One or more labels may be used. Based on the information, data associated with the particular object can be automatically selected. The object identification is used as a safety device (eg in manufacturing, when performing a quality overhaul) to load data associated with a particular object.

5. Default Project Method: Methodology in which data pertaining to a particular task (eg, coordinate system of reference positions, compensation for adapters, dimensions performed) is used in connection with the automatically identifiable targets collected in the "default project". This may be a collection of files containing related data or any other technique of accessing and storing a particular access to a database or related data.

6. Automated Bearings:

A procedure where automatically identifiable targets in the placement of the measurement system automatically calculate the external orientation and / or the relative orientation of one or more cameras / images.

During the measurement, the auto-identifiable targets are used to describe one or more cameras / images in order to account for the last change in the relationship between the fixed objects and the camera (s) and / or to perform other measurements. A procedure for automatically calculating the external bearing of a person.

Any procedure wherein three or more automatically verifiable targets are used to establish a particular coordinate system.

In all of the above procedures, three or more auto-identifiable targets must be known.

7. Automated Measurement:

In order to be able to establish a correspondence between standard targets in two or more images, it is possible to establish a coordinate system associated with auto-identifiable targets (external orientation) or any arbitrary coordinate system such as one or more cameras / images (relative orientation). A procedure used to establish the geometric relationship of the images with respect.

3D spatial coordinates in a certain 3D coordinate system are determined prior to the start of the measurement, or determine the external orientation of each image used for three or more auto-identifiable targets determined during the measurement.

Determine matches in the pictures. This can be used, for example, to calculate the relative orientation of the images.

Combination of auto-identifiable targets and standard targets. The technique determines the spatial geometry of the images, ie their external orientations or their relative orientations, through unidentified but last other targets and auto-identifiable targets already placed within the images (ie, specific within the image). The relationship between the target and its origin in the object space and / or other image thereof within that image is unknown.) And the images (ie their external orientation or their relative orientations) and / or they are images Using the geometric relationship of the images taken, calculations are made possible, allowing the establishment of a correspondence between coincident targets imaged in different images using the 3D geometry of the identifiable targets.

Targets can be retro-reflective targets, but can also have some other feature of an object that can be placed within a picture or group of pictures.

8. Scale Bar: Use one or more auto-identifiable targets to identify a particular scale bar to obtain a calibrated distance or distances (if two or more points are placed on the scale bar). The following elements are part of the description.

1) Check the specific scale bar.

2) Help to find other targets on the scale bar.

3) Use several known distances on one scale bar for improved precision and reliability.

They can be applied individually or in combination.

9. Specific adapters or objects are then identified using one or more labels. Using a single or group of adapters is used in turn.

One or more labels are used to identify a particular object. The object is:

XYZ for orientation can be individual points with or without any given properties, such as coordinates. Using the label, to calculate the external orientations, we obtain the 3D spatial coordinates of the point from the database, and calculate the spatial position of the object relative to some coordinate system through a spatial like transformation.

Each label may identify an adapter by its characteristics. These can be, for example, offset of the button target, vector distance for reduction of double vector targets or hidden point bars.

The label may be a multipoint adapter or another point having a specific geometric relationship with the targets used to find the geometric relationship between the visible targets and another point.

May be a specific scale bar.

Specific touch probes or other devices such as spindles or other manufacturing devices and other objects to be identified.

Adapters may have one or more auto-identifiable targets on any type of adapter.

Any technique of identifying a particular adapter that derives geometric parameters of the adapter,

Any technique for identifying a particular adapter that retrieves its spatial coordinates from a database or other storage mechanism,

Any technique that identifies a particular adapter controlling software that performs a particular measurement sequence or other process.

Examples:

a) The system finds a double vector target and automatically calculates the mechanical points associated with the two optical targets.

b) The system finds a specific target group and commands the robot to go and grab the pieces.

The following first defines a number of adapters and then uses the adapters to define the use of auto-identifiable targets.

Single point adapter: A "single point adapter" refers to a single optically visible target such as a re-reflective target or LED, and one or more mechanical points such as the position of a shank, the center of the sphere where the optical and mechanical targets are generally known and fixed. It consists of a device.

Typical examples are:

"Button target" consisting of one optical target mounted on a piece of machine with a shank suitable for tooling holes. The target is at the center of the shank axis, but at some constant distance from the mechanical interface.

Targets that accumulate in partial spheres, such as the Taylor Hobson sphere.

Auto-identifiable targets are used to identify each of the "single point adapters" and generally derive certain geometrical characteristics from the database. The information may be an offset of the optical target and one or more dynamic points. This allows in one measurement operation to use adapters with various offsets without requiring the user to confirm the adapter, without resorting to applying one particular adapter with a specific offset to special locations.

Auto-identifiable targets are used to find the 3D reference coordinates of an "single point adapter" such as an optical target or reference position.

A "two point adapter" is a device consisting of two or more optically visible targets and one or more mechanical or virtual points. The distinction between "two point adapters" and "multipoint adapters" means that the electron establishes a 3D relationship between the optically visible target and mechanical or virtual points without additional knowledge about the spatial location of the "two point adapter". It is in the fact that it does not allow. From a geometric point of view, the points of the "two point adapter" cannot be used to determine the 3D space-like transformation with six parameters. In general, the point (s) to be calculated using a "two point adapter" is on a 3D line with two optical targets.

Typical examples of "two point adapters" are:

"Double vector targets". The adapter typically consists of two optically visible targets. An important mechanical or imaginary point is on a space line with two optically visible targets, the (spatial) distance from one or both targets is known, or it may be at the geometric center of the two optically visible targets. have.

Identify the adapter, obtain its naming convention for its geometric parameters and / or one or more points to be calculated,

Identifying the adapter regaining the spatial coordinates of one or more reference positions of the one or more optical targets and / or adapters,

To identify adapters that control certain software functions or indicate some action by the system

One or more auto-identifiable targets are used to identify a particular "two point adapter"

A "multipoint adapter" is a device consisting of three or more optically visible targets that are geometrically related to an object and / or one or more mechanical or virtual points. A "multipoint adapter" can be used, for example, to determine the spatial position and orientation of an adapter or an object physically connected with said adapter. It may also be used to derive the spatial location of one or more points on an adapter or other object physically connected with the adapter.

Examples of "multipoint adapters" are:

V plate: device consisting of three optical targets and three hemispheres.

-Robot calibration facility: a device fixed to the robot to calibrate the robot

Another device consisting of a touch probe, such as a digit pen, and three or more optical targets connected to the CMM touch probe to determine the 3D spatial coordinates of the probe.

Tooling fixtures: Any device on a tool that locates pieces in the product manufacturing process. Adapters are installed to locate the tooling facility.

Tool holders or tools (drills, grippers, cutters and other end effectors): These can be mounted on manufacturing systems such as robots or robotic arms, CNC machines, etc., but also on portable devices. .

Identify a particular adapter to retrieve its geometric properties and / or its spatial reference coordinates,

Use of auto-identifiable target (s) to identify a specific adapter that initiates certain actions or calculations performed, such as automatically calculating invisible targets:

caution:

Targets with targets that can be automatically identified, labeled and measured with a computer program and mounted in tooling holes or other locations.

Automatically measure the target to automatically determine the orientation of one or more cameras to orient the camera.

-Defense update.

A tool with three or four targets that automatically orient the cameras.

-Can be checked automatically

1. The position and / or posture of the device attached to the space;

2. An apparatus having three or more points used to derive the location of one or more other points that are physically connected to the points in space.

A tool that automatically orients and automatically calculates approximations for object points.

The points are labeled by simply marking their labels in one image.

Coded targets to place other devices such as robots, CMMs, CNC machines and the like.

Claims (26)

A method of 3D image metrology using auto-identifiable targets, characterized in that it determines the external orientation of the images consisting of targets and / or patterns or groups of targets. The method of claim 1, wherein auto-identifiable targets in several images are found and matching points in the images are established. The method of any one of claims 1 to 2, characterized by finding auto-identifiable targets from different fields of view and establishing their relative relationship. Method according to any one of the preceding claims, characterized in that the identification of automatically identifiable targets defined as reference of the definition of the coordinate system. The 3D image measuring method according to any one of claims 1 to 4, wherein the characteristic identifies a measuring adapter for identifying a specific adapter. 6. A label area and a center area according to any one of claims 1 to 5, wherein the label area and the center area are sufficiently enlarged so as not to reduce the precision with which the center target can be placed. 3D image measurement method. 7. The method of claim 6, wherein when the center target is designed with optimal measurement accuracy, the distance is adjusted to at least the diameter of the center target. 8. The method of claim 7, wherein the distance is determined as a result of the central target, which is generally drawn 6 pixels within diameter. The method according to any one of claims 1 to 8, wherein techniques using a priori knowledge, such as redundancy in the code, additional areas such as perimeters around labels, or auto-identifiable targets are used individually or in combination, 3. A method of metrology, characterized by establishing an exemption for partial occlusion of the label area. The method according to any one of claims 1 to 9, wherein a translation is made between the label of the target and its physical label in order to establish labeling freedom to apply specific targets at specific points where they want to be labeled differently. 3D image measuring method characterized by the above-mentioned. The method of claim 1, wherein the labels of one or more automatically identifiable targets are used to identify a particular object. 12. The method of any one of claims 1 to 11, wherein data such as coordinates of reference positions belonging to a particular task, compensation for adapters, performed dimensions, etc. are collected in a default project. 13. The method according to any one of claims 1 to 12, wherein if the 3D coordinates of the three or more auto-identifiable targets are known, the auto-identifiable targets in the setup of the measurement system are the external orientation and / or relative of one or more cameras / images. 3D image measurement method, characterized by establishing an automated orientation by a procedure used to automatically calculate the orientation. The method according to claim 1, wherein the auto-identifiable targets are measured to account for the last change in the relationship between the fixed objects and the camera (s) and / or to perform other measurements. During which the auto-identifiable targets are used to establish an automated orientation by a procedure to automatically calculate the external orientation of the one or more cameras / images. 15. The method of any one of claims 1 to 14, wherein three or more automatically verifiable targets are used to establish an automated orientation by a procedure to establish a particular coordinate system. 16. The coordinate system according to any one of claims 1 to 15, or any arbitrary coordinate system such as one associated with auto-identifiable targets, in order to be able to establish a correspondence between standard targets in two or more images. A 3D image measurement method characterized in that automatic measurement is performed by a procedure for establishing a geometric relationship of images with respect to any coordinate system related to the above cameras / images. 17. The external orientation of each image according to any one of claims 1 to 16, wherein the 3D spatial coordinates in a certain 3D coordinate system are known before the start of the measurement or determined during the measurement using the three or more auto-identifiable targets. 3D image measurement method characterized in that automatic measurement is performed by determining. 18. The method according to any one of the preceding claims, wherein automated measurements are made by determining matching points in the images. 19. The method according to any one of claims 1 to 18, which establishes the spatial geometric relationship of the images via unidentifiable but already auto-identifiable targets and last other targets already disposed within the images, and the images and / or Or automated measurement by establishing the correspondence between coincident targets imaged in different images, using the geometric relationship of the 3D geometry of the identifiable targets, which can be calculated using the geometric relationship of the imaged images. 3D image measurement method characterized by the above-mentioned. 20. The method of any one of claims 1 to 19, wherein the particular scale bar is identified to obtain a calibration distance or distances. 21. A method according to any one of the preceding claims, wherein one or more automatic confirmation targets are applied on any kind of adapter. 22. A method according to any one of the preceding claims, wherein specific adapters are identified to derive the geometric parameters of the adapter. 23. A method according to any one of the preceding claims, wherein the particular adapter is identified to retrieve its spatial coordinates from a database or other storage mechanism. 24. The method according to any one of the preceding claims, wherein the particular adapter is identified which controls the software to perform a particular measurement sequence or any other process. A device using 3D image metrology auto-identifiable targets, characterized by a device for determining the external orientation of images composed of targets and / or patterns or groups of targets. 26. The device of claim 25, wherein the device consists of a combination of features claimed in any one of claims 1 to 24 or any features.