EP1190207A1

EP1190207A1 - Measurement apparatus for measuring the position and orientation of a first part to be worked, inspected or moved

Info

Publication number: EP1190207A1
Application number: EP00937085A
Authority: EP
Inventors: Richard Michael Gooch
Original assignee: BAE Systems PLC
Current assignee: BAE Systems PLC
Priority date: 1999-06-26
Filing date: 2000-06-08
Publication date: 2002-03-27
Also published as: WO2001001068A1; GB9914914D0; JP2003503684A; AU5237600A

Abstract

Apparatus for measuring the position and orientation of a first part (1) to be worked, inspected or moved, which first part (1) is to be carried by the apparatus or is separate therefrom, including an end-effector (2), at least two imaging devices (3a, 3b), each attachable to the end-effector (2) and each configurable to image a first face (4) of the first part (1), a plurality of first light sources (5), which first light sources (5) include active light sources and/or illuminable reflecting points and are positionable on said first part (1), so that when light is projected or reflected from each of the first light sources (5) when positioned on the first surface (4) of the first part (1), a distribution of light is generated at each of said at least two imaging devices (3a, 3b), each of which at least two imaging devices (3a, 3b) is operable to output signals (6a, 6b) indicative of the distribution of light received by the respective imaging device, a processor (7) for receiving and processing the output signals (6a, 6b) indicative of the distribution of light from each of said imaging devices (3a, 3b), and calibrating means (12) in operative association with the processor (7) for calibrating the output signals (6a, 6b) indicative of the distribution of light processed by the processor (7) so as to define the position and orientation of the first part (1).

Description

MEASUREMENT APPARATUS FOR MEASURING THE POSITION AND ORIENTATION OF A FIRST PART TO BE WORKED. INSPECTED OR

MOVED

This invention relates to measurement apparatus for measuring the position and orientation of a first part to be worked, inspected or moved.

In order to work on, inspect, or move a part that is either in isolation or located among multiple objects and other parts, the position and orientation thereof must be known. The selection of an appropriate measurement system to generate this information depends on many factors including environment, accuracy, proximity, time and space constraints, and conventional measurement systems can broadly be split between those which require contact with the part to be measured, and those which do not.

A variety of contact measurement devices have been developed having a touch probe device at one end to register contact with a feature or object on a part, and having a number of pin-jointed angled connections along its length. The number of connections determines the degrees of measurement, and there are typically six. Each joint is precision machined to a specified tolerance, each of which in combination provides a discrete overall system accuracy. The obvious limitation with such systems is that the touch probe is required to contact the part to be measured, limiting its range of application.

Laser trackers, which are non-contact polar devices utilising an interferometer, are commonly used to measure the angles of elevation and azimuth as well as the range of a single retro-reflective target. Typical systems require positioning of corner cubes (tooling-balls) to re-direct the incoming laser beam at various positions on the part. The most important limitation of this type of system is that it only measures position to three degrees, and thus provides no information on orientation of the part.

Traditional methods of photogrammetry, such as film photogrammetry, use retro-reflective targets on the object surface, expensive and precise geometrically stable cameras, silver-film based photographic imagery and require skilled operator assistance. Such methods can be highly accurate, 1 part in 1000 000 of the object space being achievable, but time penalties in chemically processing the photographic materials, the time necessary for either manual or semi-automated measurement and the degree of expertise necessary to achieve full measurement potential makes traditional photogrammetric methods unsuitable for the majority of manufacturing tasks. For measurements of gradual changes in positions however, such as measurements of structures such as steel bridges under loading or movements in dams and reservoir embankments, where extremely accurate data is required over long time periods, this method is well suited.

With the arrival of geometrically stable Charge Couple Device (CCD) cameras, the development of real time digital photogrammetric systems has gained significant ground, enabling six degree of freedom measurements of multiple objects with particular applications in assembly processes in the manufacturing environment. Essentially, images of illuminated high contrast targets, which are placed on the part to be worked, tested or inspected, are captured by the CCD camera and processed to provide a measure of the position of the part. The majority of configurations currently used to perform these measurements have the CCD cameras located remote from the part to be measured, and in order to achieve sufficiently precise measurements, the imaging devices coupled with the CCD cameras, such as lenses, are required to be of high optical quality. For measurements of this type, the measurement equipment is typically set up in a 'workcell' area, and the part to be worked, tested or inspected is brought into the workcell for measurements to be made. The costs involved in setting up the part, the measurement system, and of the hardware required to perform the measurements, for example the imaging system detailed above, make such a system extremely expensive.

A photogrammetry system has been developed which positions the imaging devices (may be CCD cameras) on a robot body, which brings them closer to the part to be worked by the robot than if they were mounted remotely therefrom, as described above. Nevertheless, this still constrains the part to be worked to be brought to the cell where the robot is mounted, as these are typically substantial objects.

There is thus a need for a generally improved measurement system that can be moved to the place where the part to be measured is located, can utilise cheap, standard imaging devices, and can be fitted to a wide variety of controlling devices, such as robots and machine tools, if required.

According to first aspect of the present invention there is provided apparatus for measuring the position and orientation of a first part to be worked, inspected or moved, which first part is to be carried by the apparatus or is separate therefrom, including an end-effector, at least two imaging devices, O 01/01068 _ ₄ _ PCT/GBOO/02236

each attachable to the end-effector and each configurable to image a first face of the first part, a plurality of first light sources, which first light sources include active light sources and/or illuminable reflecting points and are positionable on said first part, so that when light is projected or reflected from each of the first light sources when positioned on the first surface of the first part, a distribution of light is generated at each of said at least two imaging devices, each of which at least two imaging devices is operable to output signals indicative of the distribution of light received by the respective imaging device, a processor for receiving and processing the output signals indicative of the distribution of light from each of said imaging devices, and calibrating means in operative association with the processor for calibrating the output signals indicative of the distribution of light processed by the processor so as to define the position and orientation of the first part.

Preferably the end-effector includes means for mounting a second part thereon.

Conveniently each of the at least two imaging devices is a metrology sensor operable to create digitisable images.

Advantageously there are at least two second light sources, each associated with a respective imaging device, and the plurality of first light sources is a plurality of reflective targets, each positionable on the first part.

Preferably each of the plurality of reflective targets is fabricated from retro-reflective material. O 01/01068 c _ PCT/GBOO/02236

Conveniently the plurality of reflective targets includes at least six reflective targets, each of which is non-planar and non-linearly spaced with respect to each other.

Advantageously there are communication links between the imaging devices and the processor for transmitting the output signals indicative of the distribution of light at each of the operating positions, which links include coaxial cables and framegrabber ports.

Preferably the calibrating means includes a sensor co-ordinate frame of reference, which sensor co-ordinate frame of reference is a co-ordinate frame of the at least two imaging devices, combining means for combining the output signals from the at least two imaging devices so as to define first light positions of said first light sources in the sensor co-ordinate frame, and transformation means for locating said first light positions on the first part in the sensor coordinate frame.

Conveniently the means for mounting the second part is a drill mount.

Advantageously the second part is a drill.

Preferably there are two said imaging devices substantially equi-spaced about a central axis of the drill mount.

Conveniently the means for mounting the second part is a jig having a plurality of suction devices removably attachable thereto, which suction devices are operatively associated with pneumatic means.

Advantageously there are four said imaging devices substantially equi- spaced about a central axis of the jig. O 01/01068 _ g _ PCT/GBOO/02236

According to a further aspect of the present invention there is provided a method for measuring the position and orientation of a first part to be worked, inspected or moved, which first part is to be carried by the apparatus or is separate therefrom, including the steps of imaging on at least two imaging devices a distribution of light projected or reflected from a plurality of first light sources, which first light sources include active light sources and/or illuminable reflecting points, transmitting signals indicative of the distribution of light from each of the at least two imaging devices to a processor, processing the signals, and calibrating from the processed signal the distribution of light using calibrating means so as to determine the position and orientation of the first part.

Preferably the calibration of the distribution of light includes the steps of identifying each active light source and/or illuminable reflecting point comprising the plurality of first light sources from the output signals from the at least two imaging devices so as to define a first position thereof in a sensor co-ordinate frame, which sensor co-ordinate frame is a co-ordinate frame of the at least two imaging devices, combining each of the first positions to generate a plurality of first positions in the sensor co-ordinate frame, and transforming the plurality of first positions into locations with respect to the first part, so as to define the position and orientation of the first part in the sensor co-ordinate frame.

For a better understanding of the present invention, and to show how the same may be carried into effect, reference will now be made, by way of example, to the accompanying drawings, in which: Figure 1 is a schematic perspective representation of apparatus for measuring the position and orientation of a first part to be worked, inspected or moved according to a first embodiment of the present invention,

Figure 2 is a schematic perspective representation of apparatus for measuring the position and orientation of a first part to be worked, inspected or moved according to a second embodiment of the present invention, and

Figure 3 is a schematic block diagram of the apparatus of Figures 1 or 2 showing calibrating means.

Apparatus for measuring the position and orientation of a first part to be worked, inspected or moved, according to the present invention, as shown in Figures 1 to 3 is intended for use in situations where non-contact, accurate measurements are required, and where the working volume of the apparatus should overlap with the working volume of additional parts, for example a robot, which are used to control the position and orientation of the first part. The apparatus is thus suited for use with a variety of robots and machine tools.

Figure 1 of the accompanying drawings shows apparatus according to a first embodiment of the present invention for measuring the position and orientation of a first part 1 to be worked, inspected or moved, which first part 1 is to be carried by the apparatus or is separate therefrom, includes an end- effector 2, at least two imaging devices 3a, 3b, each attachable to the end- effector 2 and each configurable to image a first face 4 of the first part 1 , and a plurality of first light sources 5. The first light sources 5 include active light sources and/or illuminable reflecting points and are positionable on said first O 01/01068 _ o _ PCT/GBOO/02236

part 1 , so that when light is projected or reflected from each of the first light sources 5 when positioned on the first surface 4 of the first part 1 , a distribution of light is generated at each of the at least two imaging devices 3a, 3b. Each of the imaging devices 3a, 3b is operable to output signals 6a, 6b indicative of the distribution of light, and a processor 7 is provided for receiving and processing the output signals 6a, 6b, which processor is in operative association with calibrating means 12, shown in Figure 3. The calibrating means 12 calibrates the output signals 6a, 6b processed by the processor 7 so as to define the position and orientation of the first part 1.

As further shown in Figure 1 , the end effector 2 includes means 2a for mounting a second part 8 thereon, where the means 2a for mounting a second part is a drill mount 13 and the second part 8 is a drill.

The apparatus includes at least two second light sources 9a, 9b, each associated with a respective imaging device 3a, 3b. The plurality of first light sources 5 in the Figure 1 embodiment is a plurality of reflective targets, each positionable on the first part 1 , and each fabricated from retro-reflective material such that light projected by each of the second light sources 9a, 9b is reflected back therefrom in the exact direction of the incident ray. In order to derive a non-degenerate solution for the position and orientation of the first part 1 , at least six such targets, each of which is non-planar and non-linearly spaced with respect to each other, require to be placed on the first part 1. Alternatively, if the position of the targets is known relative to the first part 1 , a minimum of three is required. O 01/01068 _ g _ PCT/GBOO/02236

Each of the at least two imaging devices 3a, 3b is preferably a metrology sensor operable to create digitisable images, such that the light projected or reflected from each of the first light sources 5 is reproduced as an image of white pixels against a dark background, which white pixels define a two dimensional spatial location of the first light sources 5 on each of the imaging devices 3a, 3b. These images are communicated as output signals 6a, 6b by means of communication links 11 , which are preferably coaxial cables or twisted pairs, to the processor 7 through framegrabber ports 11a, 11b. The output signals 6a, 6b contain video data in a format such as International Radio Consultative Committee (CCIR).

As illustrated by the block diagram in Figure 3, the calibrating means 12 operates upon this data, using a sensor co-ordinate frame of reference 12a, typically determined off-line by standard stereo triangulation techniques, which sensor co-ordinate frame of reference 12a defines a single co-ordinate frame of reference corresponding to the at least two imaging devices 3a, 3b. The calibrating means 12 further includes combining means 12b for combining the output signals 6a, 6b in the form of bitmap images corresponding to the at least two imaging devices 3a, 3b so as to define first light positions 12c of the first light sources 5 in the sensor co-ordinate frame 12a, and transformation means 12d for locating the first light positions 12c on the first part 1 in the sensor coordinate frame 12a.

Figure 2 of the accompanying drawings shows apparatus according to a second embodiment of the present invention for measuring the position and orientation of a first part to be worked, inspected or moved, generally similar to O 01/01068 _ .| Q _ PCT/GBOO/02236

that of Figures 1 and 3 in which like parts have been given like reference numerals and will not be described further in detail. As shown in Figure 2, the end effector 2 includes means 2a for mounting a second part 8 thereon, where the means 2a for mounting a second part is a jig 14 having a plurality of suction devices 15 removably attachable thereto and the second part 8 is the first part 1. The means 2a may alternatively be provided by magnetic or mechanical gripping means. In this embodiment, and as shown in Figure 2, there are four imaging devices 3a, 3b, 3c, 3d, which are substantially equi-spaced around a central axis 14a of the jig 14.

The apparatus of the invention, described above, is operable to measure a position and orientation of a first part 1 to be worked, inspected or moved by implementing a method of the invention, which method is identically applicable to both embodiments. For the sake of clarity the method will be described with reference to the first embodiment, illustrated in Figure 1 , and includes the steps of imaging on at least two imaging devices 3a, 3b a distribution of light projected or reflected from a plurality of first light sources, then transmitting signals 6a, 6b indicative of the distribution of light from each of the imaging devices 3a, 3b to a processor 7. These signals 6a, 6b are analogue signals that are digitised by a framegrabber in the processor 7 and are stored in memory as bitmaps 16a, 16b for further processing.

The further processing includes identifying each of the first light sources 5 from the bitmaps 16a, 16b corresponding to the output signals 6a, 6b so as to define positions thereof in a sensor co-ordinate frame 12a, shown schematically in Figure 3. This therefore requires determination of the sensor co-ordinate O 01/01068 „ „ PCT/GBOO/02236

- 11 - frame 12a, which sensor co-ordinate frame 12a conveniently defines a single co-ordinate frame of reference, to which measurements taken by either of the imaging devices 3a, 3b can be related. This is typically performed off-line, and there are several methods known in the art to achieve this. One such method includes taking measurements of active light sources which are positioned at pre-specified locations in a known co-ordinate frame from numerous imaging positions, and mathematically optimising the measurements so as to derive a transformation describing the relationship between each of the imaging devices

3a, 3b. Once the co-ordinate frame 12a has been derived, this is used to transform subsequent measurements of active light sources positioned at unknown locations relative to the imaging devices 3a, 3b, such as is described in the present invention.

After performing the further processing, the distribution of light stored in the bitmaps 16a, 16b are calibrated so as to determine the position and orientation of the first part. The bitmaps 16a, 16b are stored in memory, contain a two dimensional array of pixel light intensity values corresponding to a sampling of the output signals 6a, 6b, and are each analysed by the processor 7 to locate, in two dimensional space, and in the sensor co-ordinate system 12a, a plurality of bright dots 17a, 17b. These bright dots 17a, 17b are assumed to correspond with first light sources 5 recorded by the imaging devices 3a, 3b.

The calibration is performed by the processor 7, which performs calculations to project rays from each of the plurality of bright dots 17a, 17b into three dimensional space, using a focal length characteristic of the respective imaging device 3a, 3b. In this way, each of the plurality of dots 17a, 17b have corresponding lines 18a, 18b projecting therefrom, and the intersection of each of these lines defines a plurality of first positions 12c in the sensor co-ordinate frame 12a.

Once the first positions 12c have been derived, these require transforming into locations 12d with respect to the first part 1 , so as to define the position and orientation of the first part 1 in the sensor co-ordinate frame 12a. This can be achieved by following one of several techniques known in the art, for example by positioning at least three of the first light sources 5 at a datum location specified in a corresponding CAD model of the first part 1. The locations 12d of the remaining first light sources 5 are then readily calibrated relative thereto.

Claims

1. Apparatus for measuring the position and orientation of a first part to be worked, inspected or moved, which first part is to be carried by the apparatus or is separate therefrom, including an end-effector, at least two imaging devices, each attachable to the end-effector and each configurable to image a first face of the first part, a plurality of first light sources, which first light sources include active light sources and/or illuminable reflecting points and are positionable on said first part, so that when light is projected or reflected from each of the first light sources when positioned on the first surface of the first part, a distribution of light is generated at each of said at least two imaging devices, each of which at least two imaging devices is operable to output signals indicative of the distribution of light received by the respective imaging device, a processor for receiving and processing the output signals indicative of the distribution of light from each of said imaging devices, and calibrating means in operative association with the processor for calibrating the output signals indicative of the distribution of light processed by the processor so as to define the position and orientation of the first part.

2. Apparatus according to claim 1 , wherein the end-effector includes means for mounting a second part thereon.

3. Apparatus according to claim 1 or claim 2, wherein each of the at least two imaging devices is a metrology sensor operable to create digitisable images.

4. Apparatus according to claim 3, including at least two second light sources, each associated with a respective imaging device, and wherein the plurality of first light sources is a plurality of reflective targets, each positionable on the first part.

5. Apparatus according to claim 4, wherein each of the plurality of reflective targets is fabricated from retro-reflective material.

6. Apparatus according to claim 5, wherein the plurality of reflective targets includes at least six reflective targets, each of which is non-planar and non-linearly spaced with respect to each other.

7. Apparatus according to claim 6, including communication links between the imaging devices and the processor for transmitting the output signals indicative of the distribution of light at each of the operating positions, which links include coaxial cables and framegrabber ports.

8. Apparatus according to claim 7, wherein the calibrating means includes a sensor co-ordinate frame of reference, which sensor co-ordinate frame of reference is a co-ordinate frame of the at least two imaging devices, combining means for combining the output signals from the at least two imaging devices so as to define first light positions of said first light sources in the sensor co-ordinate frame, and 1/01068 _ ₁ g _ PCT/GBOO/02236

transformation means for locating said first light positions on the first part in the sensor co-ordinate frame.

9. Apparatus according to claim 8, wherein the means for mounting the second part is a drill mount.

10. Apparatus according to claim 9, wherein the second part is a drill.

11. Apparatus according to claim 10, having two said imaging devices substantially equi-spaced about a central axis of the drill mount.

12. Apparatus according to claim 8, wherein the means for mounting the second part is a jig having a plurality of suction devices removably attachable thereto, which suction devices are operatively associated with pneumatic means.

13. Apparatus according to claim 12 having four said imaging devices substantially equi-spaced about a central axis of the jig.

14. Apparatus for measuring the position and orientation of a first part to be worked, inspected or moved substantially as hereinbefore described and as illustrated in Figures 1 or 2 and modified or not by Figure 3 of the accompanying drawings.

15. A method for measuring the position and orientation of a first part to be worked, inspected or moved, which first part is to be carried by the apparatus or is separate therefrom, including the steps of imaging on at least two imaging devices a distribution of light projected or reflected from a plurality of first light sources, which first light sources include active light sources and/or illuminable reflecting points, transmitting signals indicative of the distribution of light from each of the at least two imaging devices to a processor, processing the signals, and calibrating from the processed signal the distribution of light using calibrating means so as to determine the position and orientation of the first part.

16. A method according to claim 15, in which the calibration of the distribution of light includes the steps of identifying each active light source and/or illuminable reflecting point comprising the plurality of first light sources from the output signals from the at least two imaging devices so as to define a first position thereof in a sensor co-ordinate frame, which sensor co-ordinate frame is a co-ordinate frame of the at least two imaging devices, combining each of the first positions to generate a plurality of first positions in the sensor co-ordinate frame, and transforming the plurality of first positions into locations with respect to the first part, so as to define the position and orientation of the first part in the sensor co-ordinate frame.

17. A method for measuring the position and orientation of a first part to be worked, inspected or moved substantially as hereinbefore described and as illustrated in Figures 1 or 2 and modified or not by Figure 3 of the accompanying drawings.