JP2017524944A - 6 DOF triangulation scanner and camera for augmented reality - Google Patents

Abstract

The 3D coordinate measurement system includes a 6 degrees of freedom (6-DOF) unit, which has a unit reference frame, and includes a structure, a retroreflector, a triangulation scanner, and a virtual reality (AR) color camera. Including. The retro reflector, scanner, and AR camera are attached to the structure. The scanner includes a first camera configured to form a first image of pattern light projected onto an object by a projector. The first camera and the projector are configured to cooperatively determine a first 3D coordinate of a point on the object in the unit reference frame, and this determination is at least partially related to the projected pattern light. And based on the first image. The system also includes a coordinate measuring device having a device reference frame and configured to measure the attitude of the retroreflector within the device reference frame, wherein the measured attitude is a measurement of the six degrees of freedom of the retroreflector. Including.

Description

CROSS REFERENCE TO RELATED APPLICATION This application claims the benefit of US Provisional Patent Application No. 62 / 035,587 filed on August 11, 2014, which is incorporated herein by reference in its entirety. To do. This application also claims US Patent Application No. 14 / 733,130, filed June 8, 2015, which claims the benefit of US Provisional Application No. 62 / 011,151, filed June 12, 2014. And both applications are incorporated herein by reference in their entirety.

  The present disclosure relates to a six-degree-of-freedom (6-DOF) triangulation scanner and an integral camera configured to implement augmented reality (AR).

  One group of coordinate measuring devices belongs to the class of instruments that measure the three-dimensional (3D) coordinates of a point by sending a laser beam to that point. The laser beam may hit the point directly or may hit a retroreflector target that contacts the point. In either case, the instrument determines the coordinates of the point by measuring the distance to the target and two angles. The distance is measured by a distance measuring device such as an absolute distance measuring device or an interferometer. The angle is measured by an angle measuring device such as an angle encoder. A gimbal beam steering mechanism in the instrument guides the laser beam to the point of interest.

  A laser tracker is a specific type of coordinate measuring device that tracks a retroreflector target with one or more laser beams it emits. The laser tracker is therefore a “time of flight” type measuring device. Coordinate measuring devices closely related to the laser tracker are a laser scanner and a total station. A laser scanner directs one or more laser beams stepwise to a point on the object surface. It picks up the light scattered from this surface and from this light determines the distance to each point and the two angles. Total stations are most often used for surveying, but may also be used to measure the coordinates of targets that are divergently scattered or specularly reflected. Hereinafter, the term laser tracker is used in a broad sense including a laser scanner and a total station.

  Usually, a laser tracker sends a laser beam to a retro-reflector target. A common type of retroreflector target is a spherically mounted retroreflector (SMR), which includes a corner cube retroreflector embedded in a metal sphere. The corner cube retro-reflector includes three mirrors that are perpendicular to each other. The vertex that is the intersection of the three mirrors is located at the center of the sphere. Since the corner cubes are thus arranged in the sphere, the vertical distance from the apex to the surface on which the SMR is placed remains constant even when the SMR rotates. As a result, the laser tracker can measure the 3D coordinates of the object plane by tracking the position of the SMR as it moves over the surface. In other words, a laser tracker can fully characterize the 3D coordinates of a surface if it measures three degrees of freedom (one radial distance and two angles).

  One type of laser tracker includes only an interferometer (IFM) and does not have an absolute distance meter (ADM). If the object blocks the path of the laser beam from one of these trackers, the IFM loses its distance criteria. The operator then has to track the retroreflector to a known position, reset the reference distance and continue the measurement. A way to avoid this drawback is to include the ADM in the tracker. The ADM can measure distance by a point-and-shot (autofocus) method, as will be described in more detail later. Some laser trackers do not have an interferometer and contain only ADMs. Bridges et al. U.S. Pat. No. 7,352,446 (the '446 patent) describes a laser tracker that has only an ADM (without an IFM) that can accurately scan a moving target. Which is incorporated herein by reference. Prior to the '446 patent, absolute distance measuring instruments were too slow to accurately locate the moving target.

  A gimbal mechanism in the laser tracker may be used to direct the laser beam from the tracker to the SMR. Part of the light specularly reflected by the SMR is incident on the laser tracker and is transmitted to the position detector. The control system in the laser tracker uses the position of the light on the position detector to adjust the rotation angle of the mechanical axis of the laser tracker so that the laser beam is kept at the center on the SMR. In this way, the tracker can follow (track) SMR moving on the surface of the object of interest.

  An angle measuring device such as an angle encoder is attached to the mechanical shaft of the tracker. One distance measurement and two angle measurements performed by the laser tracker can fully identify the three-dimensional position of the SMR at any point on the surface of the object to be measured.

  Several laser trackers have been disclosed for measuring 6 degrees of freedom rather than the usual 3 degrees of freedom. Such six degrees of freedom include three translational degrees of freedom and three azimuth degrees of freedom as will be described in more detail later. An exemplary degree-of-freedom (6-DOF or 6DOF) laser tracker system is described in Bridges et al. U.S. Pat. No. 7,800,758 (the '758 patent), Bridges et al. U.S. Pat. No. 8,525,983 (the '983 patent), Cramer et al. U.S. Pat. No. 8,467,072 (the '072 patent), the contents of each of which are incorporated herein by reference.

  An alternative to a time-of-flight measuring device such as a laser tracker is a scanning system that measures the 3D coordinates of the object surface based on the principle of triangulation. Systems such as laser trackers that use time-of-flight rangefinders are sometimes more accurate than triangulation scanners, but non-contact triangulation scanners typically produce multiple light spots at a time. Projecting onto the object surface can be relatively fast.

  A typical triangulation scanner projects either linear light (eg, light from a laser line probe) or a two-dimensional (2D) pattern of light over a certain area (eg, structured light) onto an object surface. To do. In a triangulation scanner, a camera (eg, CCD or CMOS photoreceptor array) is coupled to a projector (eg, a laser light source) in a fixed mechanical relationship. A line or pattern of light emitted and projected from the projector is reflected by the object surface and imaged by the camera. Because the camera and projector are placed in a fixed relationship to each other, the distance and angle to the object surface can be calculated from the projected line or pattern, the captured camera image, and the baseline distance separating the projector and camera. It may be determined according to the principle of triangulation. Triangulation systems offer the advantage of quickly acquiring 3D coordinate data over a large area.

  In some systems, during the scanning process, a triangulation scanner acquires a series of 3D images, which may be aligned with respect to each other, thereby knowing the position and orientation of each 3D image with respect to the other 3D images. . If the scanner is stationary, such image alignment is not necessary. Similarly, it is not necessary to provide such image alignment if the triangulation scanner is attached to or operates in conjunction with a mechanical device that can measure the position and orientation of the triangulation scanner. Examples of such mechanical devices include laser trackers, coordinated measurement machine (CMM) and Cartesian CMM.

  On the other hand, if the scanner is handheld and therefore movable, various techniques can be used to align the images. One common technique uses features (eg, key points) in an image to match overlapping areas of adjacent image frames. This technique is effective when the object to be measured has many features relating to the field of view of the scanner. However, if the object includes a relatively large plane or curved surface, the images may not be properly aligned.

  Therefore, although the existing coordinate measuring apparatus is suitable for its intended purpose in operation with the triangulation scanner as described above, it is still in particular in improving the alignment of images acquired by the triangulation scanner apparatus. There is a need for improvement.

  Augmented reality (AR) is a relatively new kind of technology developed from virtual reality. Augmented reality fuses, superimposes, or transprojects actual real-world information or data with virtual information or data on, in, or towards it. That is, virtual information or data “extends”, complements, or supplements real, sensed, measured, captured, or imaged real-world information or data about an object or scene. Augmented reality applications include technical or industrial fields such as the manufacture and assembly and / or repair and maintenance of parts, components or equipment, and the layout and construction of facilities, buildings, or structures. A number of modern AR applications are available at http: // en. wikipedia. org / wiki / Augmented_reality.

  Actual information or data regarding a part, component, or device or area can be obtained in various ways using various devices. One type of equipment includes coordinate measuring devices such as CMMs or laser trackers, for example. The camera may also be used to capture still or video images of the actual part, component or device and / or the desired area itself, or surrounding or associated with the component, component or device. .

  The virtual information or data may be stored artificial information about a part, component, or device. The stored virtual information or data may relate to the design of a part, component or device, ranging from simple text or symbols to 3D CAD design data with relatively more complex graphics. In addition to visual information, the stored virtual information or data may also include audible, ie audio information or data. Stored virtual information or data may also be in text or in the repair or maintenance description of a part, component or device, or in the design of an office or manufacturing and / or repair facility (eg, building or facility layout) It may relate to visual information representing a part, component or device that may be used.

  The complex actual and virtual information or data of an AR system is typically digital in nature and can take a variety of forms or forms to the user, such as a desktop or laptop computer monitor, tablet, smartphone, or It may be provided in real time (i.e., when actual information is being measured or sensed) on a display screen such as associated with glasses, a hat, or a head mounted display associated with a helmet. The audio information may be transmitted through a speaker.

  Although some innovation has already been made in the area of augmented reality used in various devices, a new application of augmented reality is a hand-held 6-DOF triangulation scanner (used for laser trackers). For example, there is a need with a structured light scanner, a laser line probe.

  According to an embodiment of the present invention, a three-dimensional (3D) coordinate measurement system includes a six degree of freedom (6-DOF) unit having a unit reference frame, the unit comprising a structure, a retroreflector, a triangle. A measurement scanner and a virtual reality (AR) camera, wherein the retroreflector, the triangulation measurement scanner, and the AR camera are attached to the structure; the triangulation measurement scanner includes a first camera and a projector; Is configured to project the pattern light onto the object, the first camera is configured to form a first image of the pattern light on the object, and the projector and the first camera are within the unit reference frame Are configured to cooperatively determine a first 3D coordinate of a point on the object, the determination including, at least in part, the projected pattern light and Be based on one image, AR camera is a color camera configured to acquire a color image in the unit reference frame. The 3D coordinate measurement system also includes a coordinate measurement device having a device reference frame, wherein the device is configured to measure the attitude of the retroreflector within the device reference frame, the measured posture being the six degrees of freedom of the retroreflector. Including measurements.

  In accordance with another embodiment of the present invention, a three-dimensional (3D) measurement method includes a first beam of first light from a coordinate measurement device on a retro-reflector in a 6 degrees of freedom (6-DOF) unit. Sending and receiving a reflected first beam of reflected first light in response, the coordinate measuring device has a device reference frame, the 6-DOF unit has a unit reference frame, The 6-DOF unit includes a structure, a retroreflector, a triangulation scanner, and an AR camera. The retroreflector, the triangulation scanner, and the AR camera are attached to the structure. The AR camera is a color camera configured to acquire a color image in the unit reference frame. The method also includes determining, by the coordinate measurement device, a first attitude of the retroreflector based at least in part on the reflected first beam, the determined first attitude being an instrument reference 6 is a measurement of 6 degrees of freedom of the retro-reflector in the frame. The method projects a first pattern light from a projector onto an object, a first camera forms a first image of the reflection of the first pattern light from the object, and one or more In the processor, the second 3D coordinate of the second point on the object in the device reference frame is at least partially in the second pattern light, the second image, the determined second position of the retroreflector. And determining based on. The method also includes supplementing the first color image in the unit reference frame with the AR camera and the first 3D coordinates and the first color image in the common reference frame with one or more processors. Combining at least in part based on the determined first attitude of the retroreflector.

  Referring now to the drawings, illustrative embodiments are shown, which should not be construed as limiting with respect to the overall scope of the present disclosure, and are Are given similar reference numbers.

1 is a perspective view of a laser tracker system having a retro-reflector target according to an embodiment of the invention. FIG. 1 is a perspective view of a laser tracker system having a 6-DOF target according to an embodiment of the present invention. FIG. FIG. 2 is a block diagram illustrating elements of a laser tracker optical system and an electronic device according to an embodiment of the present invention. 4A and 4B are diagrams illustrating two prior art afocal beam expanders, including FIGS. 4A and 4B. FIG. 1 illustrates a prior art fiber optic beam injection system. FIG. FIG. 4 is a schematic diagram illustrating four types of prior art position detector assemblies. FIG. 4 is a schematic diagram illustrating four types of prior art position detector assemblies. FIG. 4 is a schematic diagram illustrating four types of prior art position detector assemblies. FIG. 4 is a schematic diagram illustrating four types of prior art position detector assemblies. 1 is a schematic diagram illustrating a position detector assembly according to an embodiment of the present invention. FIG. 1 is a schematic diagram illustrating a position detector assembly according to an embodiment of the present invention. FIG. 1 is a block diagram of electro and electro-optic elements in a prior art ADM. FIG. 1 is a schematic diagram showing optical fiber elements in a prior art optical fiber network. FIG. 1 is a schematic diagram showing optical fiber elements in a prior art optical fiber network. FIG. 1 is a schematic diagram illustrating fiber optic elements in a fiber optic network according to an embodiment of the invention. FIG. 1 is an exploded view of a prior art laser tracker. FIG. 1 is a cross-sectional view of a prior art laser tracker. FIG. 3 is a block diagram of the operation and communication elements of a laser tracker according to an embodiment of the invention. FIG. 3 is a block diagram of elements of a laser tracker that uses one wavelength, in accordance with an embodiment of the present invention. FIG. 3 is a block diagram of elements of a laser tracker that uses one wavelength, in accordance with an embodiment of the present invention. FIG. 3 is a block diagram of elements of a laser tracker with 6-DOF capability, in accordance with an embodiment of the present invention. 2 is a block diagram of elements of a laser tracker with 6-DOF capability, in accordance with an embodiment of the present invention. FIG. 2 is a block diagram of elements of a laser tracker with 6-DOF capability, in accordance with an embodiment of the present invention. FIG. It is the schematic explaining the principle of operation of the scanning measurement system based on triangulation. It is the schematic explaining the principle of operation of the scanning measurement system based on triangulation. 6 is a flowchart including steps in a method for obtaining a three-dimensional representation of a surface using an augmented reality camera attached to a 6-DOF triangulation scanner, according to an embodiment of the invention.

  One exemplary laser tracker system 5 shown in FIG. 1 includes a laser tracker 10, a retroreflector target 26, an optional auxiliary unit processor 50, and an optional auxiliary computer 60. One exemplary gimbal beam steering mechanism 12 of the laser tracker 10 includes a zenith carriage 14 that is attached to an azimuth base 16 and rotates about an azimuth axis 20. The payload 15 is attached to the zenith carriage 14 and rotates around the zenith shaft 18. The zenith axis 18 and the azimuth axis 20 are orthogonal to each other at the gimbal point 22 inside the tracker 10, typically the origin of distance measurement. The laser beam 46 effectively passes through the gimbal point 22 and is directed perpendicular to the zenith axis 18. In other words, the laser beam 46 is substantially perpendicular to the zenith axis 18 and in a plane that passes through the azimuth axis 20. The emitted laser beam 46 is directed in a desired direction by rotation of the payload 15 around the zenith axis 18 and rotation of the zenith carriage 14 around the azimuth axis 20. The zenith angle encoder is attached to the zenith machine shaft that coincides with the zenith shaft 18 within the tracker. The azimuth angle encoder is attached to the azimuth machine axis that coincides with the azimuth axis 20 inside the tracker. Zenith and azimuth angle encoders measure rotational zenith and azimuth angles with relatively high accuracy. The outgoing laser beam 46 travels to the retroreflector 26, which may be, for example, the spherical mount retroreflector (SMR) described above. By measuring the radial distance between the gimbal point 22 and the retroreflector 26, the rotational angle around the zenith axis 18, and the rotational angle around the azimuth axis 20, the position of the retroreflector 26 is determined by the polar coordinate system of the tracker. Specified in.

  The emitted laser beam 46 may include one or more laser wavelengths, as described below. For the sake of clarity and brevity, the following description assumes a steering mechanism of the type shown in FIG. However, other types of steering mechanisms can be used. For example, the laser beam can be reflected by a mirror that rotates about the azimuth and zenith axes. The technique described in this specification can be applied regardless of the type of the steering mechanism.

  A magnetic nest 17 may be included in the laser tracker to reset the laser tracker to a “home” position such as SMRs of different sizes, eg, 1.5, 7/8, and 1/2 inch SMR. A tracker-mounted retro-reflector 19 may be used to reset the tracker to a reference distance. In addition, although not visible in the diagram of FIG. 1, a tracker mounted mirror may be used with a tracker mounted retroreflector to allow self-compensation, which is described in US Pat. No. 7,327,446 ( '446 patent), the contents of which are incorporated herein by reference.

  FIG. 2 shows an exemplary laser tracker system 7, which is similar to the laser tracker system 5 of FIG. 1 except that the retroreflector target 26 is replaced with a 6-DOF probe 1000. In FIG. 1, other types of retro reflector targets may be used. For example, a cat-eye retro-reflector is sometimes used, which is a glass retro-reflector where the light is focused on a small spot of light on the reflective back surface of the glass structure.

  FIG. 3 is a block diagram illustrating optical and electrical elements in an embodiment of a laser tracker. This shows the elements in the laser tracker emitting two wavelengths of light: a first wavelength for ADM and a visible pointer and a second wavelength for tracking. The visible pointer allows the user to see the position of the laser beam spot emitted by the tracker. The two types of wavelengths are combined using a free space beam splitter. An electro-optic (EO) system 100 includes a visible light source 110, an isolator 115, an optional first fiber injection system 170, an optional interferometer (IFM) 120, a beam expander 140, A first beam splitter 145, a position detector assembly 150, a second beam splitter 155, an ADM 160, and a second fiber injection system 170 are included.

  Visible light source 110 may be a laser, a superluminescent diode, or other light emitting device. The isolator 115 may be a Faraday isolator, an attenuator, or other device that can reduce light returning to the light source by reflection. The optional IFM may be configured in various ways. As one specific example of possible implementations, the IFM includes a beam splitter 122, a retroreflector 126, quarter wave plates 124, 130, and a phase analyzer 128. The visible light source 110 may emit light into free space, which then travels through the isolator 115 and optional IFM 120 in free space. Alternatively, isolator 115 may be coupled to visible light source 110 by a fiber optic cable. In this case, the light from the isolator may be emitted into free space through the first optical fiber injection system 170, which will be described below with reference to FIG.

  Although beam expander 140 may be assembled using a variety of lens configurations, two commonly used prior art configurations are shown in FIGS. 4A and 4B. FIG. 4A shows a configuration 140A based on the use of a negative lens 141A and a positive lens 142A. The collimated light beam 220A incident on the negative lens 141A is emitted from the positive lens 142A as a larger collimated light beam 230A. FIG. 4B shows a configuration 140B based on the use of two positive lenses 141B, 142B. The collimated light beam 220B incident on the first positive lens 141B exits from the second positive lens 142B as a larger collimated light beam 230B. A small amount of the light emitted from the beam expander 140 is reflected and lost by the beam splitters 145 and 155 while leaving the tracker. The portion of the light that has passed through beam splitter 155 is combined with the light from ADM 160 to form composite light beam 188 that exits the laser tracker and travels to retro-reflector 90.

  In some embodiments, the ADM 160 includes a light source 162, ADM electronics 164, a fiber network 166, an interconnecting electrical cable 165, and interconnecting optical fibers 168, 169, 184, 186. The ADM electronics send electrical modulation and bias voltage to the light source 162, which may be a distributed feedback laser operating at a wavelength of about 1550 nm, for example. In some embodiments, the fiber network 166 may be the prior art fiber optic network 420A shown in FIG. 8A. In this embodiment, light from the light source 162 in FIG. 3 travels on an optical fiber 184 that is equivalent to the optical fiber 432 in FIG. 8A.

  The fiber network of FIG. 8A includes a first fiber coupler 430, a second fiber coupler 436, and low transmittance reflectors 435, 440. The light travels through the first fiber coupler 430, the first path through the two paths, the optical fiber 433 to the second fiber coupler 436, the second through the optical fiber 422 and the fiber length equalizer 423. Divide into routes. The fiber length equalizer 423 is connected to the fiber 168 of FIG. 3 going to the reference channel of the ADM electronics 164. The purpose of the fiber length equalizer 423 is to match the length of the optical fiber through which light in the reference channel passes with the length of the optical fiber through which light passes in the measurement channel. By matching the fiber length in this manner, ADM errors caused by changes in ambient temperature are reduced. Such an error can occur because the effective optical path length of the optical fiber is equal to the average refractive index of the optical fiber multiplied by the length of the fiber. Since the refractive index of the optical fiber depends on the temperature of the fiber, the effective optical path length of the measurement and reference channels changes with changes in the temperature of the optical fiber. If the effective optical path length of the optical fiber in the measurement channel changes with respect to the effective optical path length of the optical fiber in the reference channel, the apparent shift in the position of the retro-reflector target 90 will occur even if the retro-reflector target 90 remains stationary. Occur. To avoid this problem, two steps are taken. First, the fiber length in the reference channel is matched as much as possible with the fiber length in the measurement channel. Second, the measurement and reference fibers are wired as close as possible so that the temperature changes of the optical fibers in the two channels are substantially the same.

  The light travels through the second optical fiber coupler 436 and is split into two paths: a first path to the low reflectivity fiber terminator 440 and a second path to the optical fiber 438, from which Proceed to the third optical fiber 186. The light on the optical fiber 186 travels to the second fiber injection system 170.

  In one embodiment, a fiber injection system 170 is shown in prior art FIG. Light from optical fiber 186 in FIG. 3 travels to fiber 172 in FIG. The fiber injection system 170 includes an optical fiber 172, a ferrule 174, and a lens 176. The optical fiber 172 is attached to the ferrule 174, which is fixedly attached to the structure within the laser tracker 10. If desired, the end of the optical fiber may be polished obliquely to reduce reflection attenuation. The light 250 emerges from the fiber core, which may be a 4-12 micrometer diameter single mode optical fiber, depending on the wavelength of light used and the specific type of optical fiber. Light 250 diverges diagonally and passes through lens 176, which collimates the light. A method for transmitting and receiving optical signals through a single optical fiber in an ADM system has been described with respect to FIG. 3 of the '758 patent.

  Referring to FIG. 3, the beam splitter 155 may be a dichroic beam splitter, which transmits different wavelengths than it reflects. In one embodiment, light from ADM 160 is reflected by dichroic beam splitter 155 and combines with light from visible laser 110 that has passed through dichroic beam splitter 155. The composite light beam 188 travels from the laser tracker to the retroreflector 90 as a first beam, which returns a portion of the light as a second beam. Of the second beam, the ADM wavelength portion is reflected by the dichroic beam splitter 155 and returns to the second fiber injection system 170, which recombines the light into the optical fiber 186.

  In certain embodiments, the optical fiber 186 corresponds to the optical fiber 438 of FIG. 8A. Return light travels from the optical fiber 438 through the second fiber coupler 436 and is split into two paths. The first path leads to optical fiber 424, which in one embodiment corresponds to optical fiber 169, which leads to the measurement channel of the ADM electronic device of FIG. The second path leads to the optical fiber 433 and then to the first fiber coupler 430. The light exiting the first fiber coupler 430 is split into two paths: a first path to the optical fiber 432 and a second path to the low reflectivity terminator 435. In some embodiments, the optical fiber 432 corresponds to the optical fiber 184, which leads to the light source 162 of FIG. In most cases, the light source 162 includes an integral Faraday isolator, which minimizes the amount of light that enters the light source from the optical fiber 432. If excessive light is supplied to the laser in the opposite direction, the laser can become unstable.

Light from the fiber network 166 enters the ADM electronics 164 through optical fibers 168 and 169. One embodiment of a prior art ADM electronic device is shown in FIG. The optical fiber 168 in FIG. 3 corresponds to the optical fiber 3232 in FIG. 7, and the optical fiber 169 in FIG. 3 corresponds to the optical fiber 3230 in FIG. Referring now to FIG. 7, the ADM electronics 3300 includes a frequency reference 3302, a synthesizer 3304, a measurement detector 3306, a reference detector 3308, a measurement mixer 3310, a reference mixer 3312, and a conditioning electronics 3314. 3316, 3318, 3320, an N-dividing prescaler 3324, and an analog-to-digital converter (ADC) 3322. The frequency reference may be, for example, a quartz crystal with a thermostatic oven (OCXO), and sends a reference frequency f _REF, which may be 10 MHz, for example, to the synthesizer, which is one of two electrical signals, namely one of the frequencies f _RF Two signals of a signal and a frequency f _LO are generated. The signal f _RF goes to the light source 3102, which corresponds to the light source 162 in FIG. The two signals of frequency f _LO go to measurement mixer 3310 and reference mixer 3312. Light from optical fibers 168 and 169 of FIG. 3 appears in fibers 3232 and 3230 of FIG. 7, respectively, and enters the reference and measurement channels, respectively. The reference detector 3308 and the measurement detector 3306 convert the optical signal into an electrical signal. These signals are adjusted by electrical components 3316 and 3314, respectively, and sent to mixers 3312 and 3310, respectively. The mixer generates a frequency f _IF , which is equal to the absolute value of f _LO −f _RF . The signal f _RF may be a relatively high frequency, for example 2 GHz, while the signal f _IF may be a relatively low frequency, for example 10 kHz.

The reference frequency f _REF is sent to the prescaler 3324, which divides the frequency by an integer. For example, the frequency of 10 MHz is divided by 40 to obtain an output frequency of 250 kHz. In this example, the 10 kHz signal entering ADC 3322 is sampled at 250 kHz, resulting in 25 samples per cycle. The signal from the ADC 3322 is sent to the data processor 3400, which may be, for example, one or more digital signal processors (DSPs) in the ADM electronics 164 of FIG.

  The distance extraction method is based on the calculation of the phase of the ADC signal for the reference and measurement channels. This method is described in Bridges et al. U.S. Pat. No. 7,701,559 (the '559 patent), the contents of which are incorporated herein by reference. The calculation includes the use of equations (1)-(8) of the '559 patent. In addition to this, when the ADM first starts measuring the retro-reflector, the frequency generated by the synthesizer is changed several times (eg, three times) and the possible ADM distance is calculated each time. By comparing possible ADM distances for each of the selected frequencies, ambiguity in ADM measurements is eliminated. By combining equations (1)-(8) of the '559 patent with the synthesis method described with respect to FIG. 5 of the' 559 patent and the Kalman filter described in the '559 patent, the ADM moves the target. Can be measured. In other embodiments, other methods of obtaining absolute distance measurements may be used, for example, using pulsed time of flight rather than phase difference.

  A portion of the return light beam 190 that passes through the beam splitter 155 reaches the beam splitter 145, which sends a portion of the light to the beam expander 140 and another portion of the light to the position detector assembly 150. The light emitted from the laser tracker 10 or the EO system 100 may be considered as a first beam, and the portion of the light reflected by the retroreflector 90 or 26 may be considered as a second beam. Some of the reflected beam is sent to different functional elements of the EO system 100. For example, the first portion may be sent to a distance meter such as ADM 160 of FIG. The second portion may be sent to the position detector assembly 150. In some cases, the third portion may be sent to other functional units, such as an optional interferometer 120. In the example of FIG. 3, the first and second portions of the second beam are reflected by beam splitters 155 and 145, respectively, and then sent to the distance meter and position detector, but reflect the light. Rather, it is important to understand that it would have been possible to transmit through a rangefinder or position detector.

  Four examples of prior art position detector assemblies, 150A-150D, are shown in FIGS. FIG. 6A shows the simplest embodiment, where the position detector assembly includes a position sensor 151, which is mounted on a circuit board 152 that takes power from the electronics box 350 and returns a signal thereto. The device box 350 may represent electronic processing capabilities at any location within the laser tracker 10, auxiliary unit 50, or external computer 60. FIG. 6B includes an optical filter 154 that blocks unwanted light wavelengths from reaching the position sensor 151. Unwanted light wavelengths may also be blocked, for example, by coating the surface of beam splitter 154 or position sensor 151 with a suitable film. FIG. 6C includes a lens 153 that reduces the size of the light beam. FIG. 6D includes both an optical filter 154 and a lens 153.

  FIG. 6E shows a novel position detector assembly that includes an optical adjuster 149E. The optical adjuster includes a lens 153 and may also include an optional wavelength filter 154. In addition, this includes at least one of a diffuser plate 156 and a spatial filter 157. As mentioned above, a popular type of retro reflector is the corner cube retro reflector. One type of corner cube retroreflector consists of three mirrors, each joined at right angles to the other two mirrors. The intersection line where these three mirrors join may have a finite thickness where the light is not completely reflected back to the tracker. Lines of finite thickness diffract as they propagate so they may not look exactly the same as that in the position detector when they reach the position detector. However, the diffracted light pattern generally deviates from perfect symmetry. As a result, the light hitting the position detector 151 may increase or decrease in optical power (hot spot), for example, near the diffraction line. Because the uniformity of light from the retro-reflector varies from retro-reflector and the light distribution on the position detector can change with the rotation or tilt of the reflector, position detection is achieved by including a diffuser 156. It may be advantageous to improve the smoothness of the light striking the vessel 151. The ideal position detector should respond to the center of mass, and the ideal diffuser should broaden the spot symmetrically, so the resulting position obtained by the position detector should not be affected. There may be an argument that it is. In practice, however, the diffuser plate has been observed to improve the performance of the position detector assembly, which is probably due to the non-linearity (imperfection) of the position detector 151 and lens 153. A glass corner cube retro-reflector may also produce a non-uniform spot of light at the position detector 151. The variation in the spot of light in the position detector may be particularly noticeable due to the reflected light from the corner cube in the 6-DOF target, which is the same as that of Brown et al. U.S. Pat. No. 8,740,396 (the '396 patent) and the above-referenced Cramer et al. U.S. Pat. No. 4,467,072 (the '072 patent), the contents of both patents are hereby incorporated by reference. In some embodiments, the diffuser plate 156 is a holographic diffuser. A holographic diffuser provides a uniform light controlled over a specific diffusion angle. In other embodiments, other types of diffuser plates are used, such as ground glass or “opal” diffuser plates.

  The purpose of the spatial filter 157 of the position detector assembly 150E is to block the ghost beam from hitting the position detector 151, which may be the result of unwanted reflections at the optical surface, for example. The spatial filter includes a plate 157 having an aperture. By placing the spatial filter 157 from the lens at a distance approximately equal to the focal length of the lens, the return light 243E passes through the spatial filter when it is near its narrowest portion, ie, at the waist of the beam. For example, a beam traveling at a different angle as a result of reflection of the optical element hits the spatial filter at a position away from the aperture and is blocked from reaching the position detector 151. An example is shown in FIG. 6E, where unwanted ghost beam 244E is reflected off the surface of beam splitter 145 and travels to spatial filter 157 where it is blocked. Without a spatial filter, the ghost beam 244E passes through the position detector 151, which makes the determination of the position of the beam 243E on the position detector 151 inaccurate. Even a weak ghost beam may greatly change the position of the center of mass on the position detector 151 if the ghost beam is a relatively long distance from the main spot of light.

  Retroreflectors of the type as described herein, such as corner cubes or cat eye retroreflectors, have the property of reflecting light incident on the retroreflector in a direction parallel to the incident light. In addition, the incident and reflected rays are arranged symmetrically around the retroreflector symmetry point. For example, in an open air corner cube retroreflector, the symmetry point of the retroreflector is the vertex of the corner cube. In a glass corner cube retroreflector, the symmetry point is likewise the apex, but in this case it must be considered that the light bends at the glass-air interface. In a cat-eye retroreflector with a refractive index of 2.0, the symmetry point is the center of the sphere. In the case of a cat-eye retroreflector consisting of two glass hemispheres placed symmetrically on a common plane, the symmetry point is the point at the center of each hemispherical sphere on that plane. The point is that for the type of retro-reflector normally used in laser trackers, the light returned by the retro-reflector to the tracker is shifted to the opposite side of the apex with respect to the incident laser beam.

  This behavior of the retroreflector 90 in FIG. 3 is the basis for tracking the retroreflector by the laser tracker. The position sensor has an ideal regression point on its surface. The ideal regression point is a point where the laser beam sent to the symmetry point of the retroreflector (for example, the vertex of the corner cube retroreflector in SMR) returns. Usually, the regression point is near the center of the position sensor. When the laser beam is sent to one side of the retroreflector, it reflects back to the other side and appears off the regression point on the position sensor. By recognizing the position of the return light beam on the position sensor, the control system of the laser tracker 10 can cause the motor to move the light beam toward the point of symmetry of the retroreflector.

  As the retro-reflector moves laterally to the tracker at a constant speed, the light beam at the retro-reflector strikes the retro-reflector (after the transient signal settles) at a constant offset distance from the retro-reflector symmetry point. The laser tracker corrects for this offset distance in the retroreflector based on the scale factor obtained from the controlled measurement and the distance from the light beam on the position sensor to the ideal regression point.

  As mentioned above, the position detector serves two important functions: a function that enables tracking, and a function that corrects the measured values for the movement of the retroreflector. The position sensor in the position detector may be any type of device capable of position measurement. For example, the position sensor may be a position detection element or a photoreceptor array. The position detection element may be, for example, a lateral effect detector or a quadrant detector. The photoreceptor array may be, for example, a CMOS or CCD array.

  In some embodiments, return light that is not reflected by the beam splitter 145 is transmitted through the beam expander 140, thereby making it smaller. In other embodiments, the positions of the position detector and rangefinder are reversed so that the light reflected by the beam splitter 145 travels to the rangefinder and the light transmitted through the beam splitter travels to the position detector.

  The light passes through the optional IFM, through the isolator, and enters the visible light source 110. At this stage, the optical power should be small enough so that it does not destabilize the visible light source 110.

  In certain embodiments, light from visible light source 110 is incident through beam injection system 170 of FIG. The fiber injection system may be attached to the output of the light source 110 or the optical fiber output of the isolator 115.

  In some embodiments, the fiber network 166 of FIG. 3 is the prior art fiber network 420B of FIG. 8B. Here, the optical fibers 184, 186, 168, and 169 in FIG. 3 correspond to the optical fibers 443, 444, 424, and 422 in FIG. The fiber network of FIG. 8B is similar to the fiber network of FIG. 8A, but the fiber network of FIG. 8B has one fiber coupler instead of two fiber couplers. Although the advantage of FIG. 8B over FIG. 8A is simplicity, FIG. 8B is more likely to cause unwanted optical back reflections into optical fibers 422 and 424.

  In some embodiments, the fiber network 166 of FIG. 3 is the fiber network 420C of FIG. 8C. Here, the optical fibers 184, 186, 168, and 169 in FIG. 3 correspond to the optical fibers 447, 455, 423, and 424 in FIG. 8C. The fiber network 420C includes a first fiber coupler 445 and a second fiber coupler 451. The first fiber coupler 445 is a 2 × 2 coupler having two input ports and two output ports. This type of coupler is usually made by bringing two fiber cores close together and then stretching the fiber while heating. In this way, a desired portion of light can be split into adjacent fibers by evanescent coupling between the fibers. The second fiber coupler 451 is of a type called a circulator. It has three ports, each with the ability to transmit or receive light, but is limited to a specified direction. For example, light on optical fiber 448 enters port 453 and is transported toward port 454 as indicated by the arrow. At port 454, light may be transmitted to optical fiber 455. Similarly, light traveling on port 455 may enter port 454 and travel in the direction of the arrow to port 456, where some light may be transmitted to optical fiber 424. If only three ports are needed, the optical power loss of the circulator 451 may be less than with a 2 × 2 coupler. On the other hand, the circulator 451 may be more expensive than a 2 × 2 coupler, and polarization mode dispersion may occur, which can be problematic in some circumstances.

  FIGS. 9 and 10 show an exploded view and a cross-sectional view, respectively, of the prior art laser tracker 2100 depicted in FIGS. 2 and 3 of the aforementioned '983 patent. The azimuth assembly 2110 includes a post housing 2112, an azimuth encoder assembly 2120, lower and upper azimuth bearings 2114 </ b> A, 2114 </ b> B, an azimuth motor assembly 2125, an azimuth slip ring assembly 2130, and an azimuth circuit board 2135.

  The purpose of the azimuth encoder assembly 2120 is to accurately measure the rotation angle of the yoke 2142 relative to the column housing 2112. The azimuth encoder assembly 2120 includes an encoder disk 2121 and a read head assembly 2122. The encoder disk 2121 is attached to the shaft of the yoke housing 2142, and the read head assembly 2122 is attached to the column assembly 2110. Read head assembly 2122 includes a circuit board on which one or more read heads are secured. Laser light transmitted from the read head is reflected by fine grating lines on the encoder disk 2121. The reflected light is picked up by a detector on the encoder read head and processed to determine the rotation angle of the encoder disk relative to the fixed read head.

  The azimuth motor assembly 2125 includes an azimuth motor rotor 2126 and an azimuth motor stator 2127. The azimuth motor rotor includes a permanent magnet attached directly to the shaft of the yoke housing 2142. The azimuth motor stator 2127 includes a field winding that generates a predetermined magnetic field. This magnetic field interacts with the magnets of the azimuth motor rotor 2126 to produce the desired rotational motion. The azimuth motor stator 2127 is attached to the support frame 2112.

  The azimuth circuit board 2135 represents one or more circuit boards that provide the electrical functions required by azimuth components such as encoders and motors. The azimuth slip ring assembly 2130 includes an outer portion 2131 and an inner portion 2132. In some embodiments, the tie wire 2138 exits the auxiliary unit processor 50. The bundling wire 2138 may supply power to the tracker or send / receive signals to / from the tracker. A portion of the tie wire 2138 may be directed to a connector on the circuit board. In the example shown in FIG. 10, the wires are routed to the azimuth circuit board 2135, the encoder read head assembly 2122, and the azimuth motor assembly 2125. The other wire is routed to the inner portion 2132 of the slip ring assembly 2130. Inner portion 2132 is attached to post assembly 2110 so that it remains stationary. Outer portion 2131 is attached to yoke assembly 2140 so that it rotates relative to inner portion 2132. Slip ring assembly 2130 is designed to allow low impedance electrical contact when outer portion 2131 rotates with respect to inner portion 2132.

  The zenith assembly 2140 includes a yoke housing 2142, a zenith encoder assembly 2150, left and right zenith bearings 2144A, 2144B, a zenith motor assembly 2155, a zenith slip ring assembly 2160, and a zenith circuit board 2165.

  The purpose of the Zenith encoder assembly 2150 is to accurately measure the rotation angle of the payload frame 2172 relative to the yoke housing 2142. Zenith encoder assembly 2150 includes a Zenith encoder disk 2151 and a Zenith readhead assembly 2152. The encoder disk 2151 is attached to the payload housing 2142 and the read head assembly 2152 is attached to the yoke housing 2142. Zenith readhead assembly 2152 includes a circuit board on which one or more readheads are secured. Laser light transmitted from the read head is reflected by fine grating lines on the encoder disk 2151. The reflected light is picked up and processed by a detector on the encoder read head to determine the angle of rotation of the encoder disk relative to the fixed read head.

  Zenith motor assembly 2155 includes a Zenith motor rotor 2156 and a Zenith motor stator 2157. Zenith motor rotor 2156 includes a permanent magnet attached directly to the shaft of payload frame 2172. Zenith motor stator 2157 includes a field winding that generates a predetermined magnetic field. This magnetic field interacts with the rotor magnet to produce the desired rotational motion. Zenith motor stator 2157 is attached to yoke frame 2142.

  Zenith circuit board 2165 represents one or more circuit boards that provide the electrical functions required by Zenith components such as encoders and motors. Zenith slip ring assembly 2160 includes an outer portion 2161 and an inner portion 2162. The tie wire 2168 may exit the azimuth outer slip ring 2131 and carry power or signals. A part of the wires of the binding wire 2168 may be connected to a connector on the circuit board. In the example shown in FIG. 10, the wires are routed to Zenith circuit board 2165, Zenith motor assembly 2150, and encoder readhead assembly 2152. The other wire is routed to the inner portion 2162 of the slip ring assembly 2160. Inner portion 2162 is attached to yoke frame 2142 so that it rotates only to the azimuth angle but not to the zenith angle. Outer portion 2161 is attached to payload frame 2162 so that it rotates to both zenith and azimuth angles. Slip ring assembly 2160 is designed to allow low impedance electrical contact when outer portion 2161 rotates relative to inner portion 2162. Payload assembly 2170 includes a main optical assembly 2180 and a second optical assembly 2190.

  FIG. 11 is a block diagram illustrating a dimension measurement electronics processing system 1500, which includes a laser tracker electronics processing system 1510, a processing system for peripheral elements 1582, 1584, 1586, a computer 1590, and in this case a cloud (cloud). And other networked components 1600 shown as). The exemplary laser tracker electronics processing system 1510 includes a master processor 1520, payload function electronics 1530, azimuth encoder electronics 1540, zenith encoder electronics 1550, display, and user interface (UI) electronics 1560. , Removable storage hardware 1565, RFID (radio identification) electronic device, and antenna 1572. Payload function electronics 1530 includes a number of sub-functions including 6-DOF electronics 1531, camera electronics 1532, ADM electronics 1533, position detector (PSD) electronics 1534, and level electronics 1535. Most of the subfunctions have at least one processor unit, which may be, for example, a digital signal processor (DSP) or a field programmable gate array (FPGA). The electronics units 1530, 1540, and 1550 are separated as shown, depending on their location within the laser tracker. In some embodiments, payload function 1530 is in payload 2170 of FIGS. 9 and 10, while azimuth encoder electronics 1540 is in azimuth assembly 2110 and zenith encoder electronics 1550 is in zenith assembly 2140. It is in.

  Although many types of peripheral devices can be used, three such devices are shown here: a temperature sensor 1582, a 6-DOF probe 1584, and a personal digital assistant 1586 such as a smartphone. As will be described in more detail later with embodiments of the present invention, other types of peripheral devices not shown in FIG. 11 are triangulation scanners, particularly 6-DOF triangulation scanners, such as laser line probes (LLP) or This is an area structured light scanner. The laser tracker may communicate with peripheral devices in various ways, such as by wireless communication with antenna 1572, by a vision system such as a camera, and a collaborative target such as a 6-DOF probe 1584 or a triangulation scanner. By the distance and angle reading of the laser tracker. The peripheral device may include a processor. In general, the 6-DOF accessory may include a 6-DOF probing system, a 6-DOF scanner, a 6-DOF projector, a 6-DOF sensor, and a 6-DOF indicator. The processors of these 6-DOF devices may be used with external computers and cloud processing resources, as well as processing devices within the laser tracker. In general, when the term laser tracker processor or measurement device processor is used, this shall include available external computers and cloud support.

  In one embodiment, another communication bus leads from the master processor 1520 to each of the electronic device units 1530, 1540, 1550, 1560, 1565, and 1570. Each communication line may have three serial lines including, for example, a data line, a clock line, and a frame line. The frame line indicates whether the electronic device unit should pay attention to the clock line. If it indicates that it should be noted, the electronics unit reads the current value of the data line with each clock signal. The clock signal may correspond to, for example, a rising edge of a clock pulse. In some embodiments, information is transmitted in the form of packets on the data line. In some embodiments, each packet includes an address, a numeric value, a data message, and a checksum. The address indicates where in the electronic device unit the data message is directed. The location may correspond to, for example, a processor subroutine in the electronic device unit. The numerical value indicates the length of the data message. The data message includes data or instructions executed by the electronic device unit. The checksum is a number that is used to minimize the chance that an error will be sent on the communication line.

  In some embodiments, the master processor 1520 sends information packets on the bus 1610 to the payload functional electronics 1530, on the bus 1611 to the azimuth encoder electronics 1540, on the bus 1612 to the Zenith encoder electronics 1550, and on the bus 1613. Send to display and UI electronics 1560 to removable storage hardware 1565 over bus 1614 and to RFID and wireless electronics 1570 over bus 1616.

  In some embodiments, the master processor 1520 also sends a sync pulse to each of the electronic device units simultaneously on the sync bus 1630. The sync pulse provides a way to synchronize the numerical values collected by the measurement function of the laser tracker. For example, azimuth encoder electronics 1540 and Zenith electronics 1550 latch their encoder values as soon as they receive a sync pulse. Similarly, the payload functional electronic device 1530 latches data collected by the electronic device contained within the payload. 6-DOF, ADM, and position detectors all latch data when a sync pulse is applied. In most cases, the camera and inclinometer collect data at a slower rate than the sync pulse, but may latch data at multiples of the sync pulse period.

  Azimuth encoder electronics 1540 and Zenith encoder electronics 1550 are typically separated from each other and from payload electronics 1530 by, for example, slip rings. Therefore, in FIG. 11, bus lines 1610, 1611, and 1612 are shown as separate bus lines.

  The laser tracker electronics processing system 1510 may communicate with an external computer 1590 or may provide calculation, display, and UI functions within the laser tracker. The laser tracker communicates with computer 1590 over a communication link 1606, which may be, for example, an Ethernet line or a wireless connection. The laser tracker also communicates with other elements 1600 represented in the cloud over a communication link 1602 that may include one or more electrical cables such as Ethernet cables and one or more wireless connections. Also good. An example of element 1600 is other three-dimensional test equipment, such as an articulated arm CMM, which may be moved by a laser tracker. Communication link 1604 between computer 1590 and element 1600 may be wired (eg, Ethernet) or wireless. An operator sitting toward the remote computer 1590 may connect to the Internet represented by the cloud 1600 via an Ethernet or wireless line that is itself connected to the master processor 1520 via an Ethernet or wireless line. In this way, the user may control the operation of the remote laser tracker.

  Today's laser trackers use one visible wavelength (usually red) and one infrared wavelength for ADM. The red wavelength may be provided by a frequency stable helium neon (HeNe) laser suitable for use in an interferometer and to provide a red pointer beam. Alternatively, the red wavelength may be provided by a diode laser that functions like a pointer beam. The disadvantage of using two light sources is the additional space and cost required for additional light sources, beam splitters, isolators, and other components. Another disadvantage of using two light sources is that it is difficult to perfectly align the two light beams along the entire path they travel. This may cause various problems, for example, good performance cannot be obtained simultaneously from different subsystems operating at different wavelengths. A system that eliminates these problems by using a single light source is shown in the optoelectric system 500 of FIG. 12A.

  FIG. 12A includes a visible light source 110, an isolator 115, a fiber network 420, an ADM electronics 530, a fiber injection system 170, a beam splitter 145, and a position detector 150. The visible light source 110 may be, for example, a red or green diode laser or a vertical cavity surface emitting laser (VCSEL). The isolator may include a Faraday isolator, an attenuator, or any other device that can sufficiently reduce the amount of light fed back to the light source. The light from the isolator 115 travels into the fiber network 420, which in one embodiment is the fiber network 420A of FIG. 8A.

  FIG. 12B shows an embodiment of an opto-electrical system 400 in which only one wavelength of light is used, but the modulation is performed by electro-optic modulation of light rather than direct modulation of the light source. The The optoelectric system 400 includes a visible light source 110, an isolator 115, an electro-optic modulator 410, an ADM electronic device 475, a fiber network 420, a fiber injection system 170, a beam splitter 145, and a position detector 150. Including. The visible light source 110 may be, for example, a red or green laser diode. The laser light is sent through an isolator 115, which may be, for example, a Faraday isolator or an attenuator. The isolator 115 may be coupled with a fiber at its input and output ports. The isolator 115 sends the light to the electro-optic modulator 410, which modulates the light to a selected frequency that may be up to 10 GHz or higher as desired. Electrical signal 476 from ADM electronics 475 drives the modulation in electro-optic modulator 410. The modulated light from electro-optic modulator 410 travels to fiber network 420, which may be a fiber network 420A, 420B, 420C, or 420D as described above. Some of the light travels over the optical fiber 422 to the reference channel of the ADM electronics 475. Another part of the light exits the tracker, is reflected by the retro-reflector 90, returns to the tracker and reaches the beam splitter 145. A small amount of the light is reflected by the beam splitter and proceeds to the position detector 150, which was described above with respect to FIGS. Some of the light enters the fiber injection system 170 through the beam splitter 145, enters the optical fiber 424 through the fiber network 420, and enters the measurement channel of the ADM electronics 475. In general, the system 500 of FIG. 12A can be manufactured less expensively than the system 400 of FIG. 12B, but the electro-optic modulator 410 may be able to achieve higher modulation frequencies, which may be advantageous in some circumstances.

  FIG. 13 shows an embodiment of a locator camera system 950 and an optoelectric system 900 in which an orientation camera 910 is combined with the optoelectric function of a 3D laser tracker to measure the six degrees of freedom of the device. The optoelectric system 900 includes a visible light source 905, an isolator 910, an optional electro-optic modulator 410, an ADM electronics 715, a fiber network 420, a fiber injection system 170, a beam splitter 145, and a position detector 150. A beam splitter 922 and a direction camera 910. Light from the visible light source is emitted into the optical fiber 980 and travels through the isolator 910, which may be coupled to the input and output ports. The light may travel through an electro-optic modulator 410 that is modulated by an electrical signal 716 from the ADM electronics 715. Alternatively, the ADM electronic device 715 may modulate the visible light source 905 by transmitting an electrical signal over the cable 717. Some of the light incident on the fiber network travels through the fiber length equalizer 423 and the optical fiber 422 and enters the reference channel of the ADM electronics 715. Electrical signal 469 may optionally be applied to fiber network 420 to provide a switching signal to fiber optic switches in fiber network 420. Some of the light travels from the fiber network to the fiber injection system 170, which sends the light over the optical fiber as a light beam 982 to free space. A small amount of light is reflected by the beam splitter 145 and lost. Part of the light passes through the beam splitter 145, through the beam splitter 922, exits the laser tracker, and reaches the six degree of freedom (DOF) device 4000. The 6-DOF device 4000 may be a probe, projector, sensor, or any other type of 6-DOF device. In embodiments of the invention described in more detail below, the 6-DOF device 4000 includes a 6-DOF triangulation scanner, such as a laser line probe (LLP) or structured light area scanner, for example. The 6-DOF triangulation scanner has a retro-reflector attached to it, making it easy to measure or determine the scanner's six degrees of freedom.

  On its return path to the laser tracker, light from the 6-DOF device 4000 enters the opto-electric system 900 and reaches the beam splitter 922. Part of the light is reflected by the beam splitter 922 and enters the azimuth camera 910. The orientation camera 910 records the position of several marks provided on the retro-reflector target. From these marks, the azimuth angle (ie, three degrees of freedom) of the 6-DOF probe device is specified. The principle of the orientation camera is described later in this application and is also described in the aforementioned '758 patent. In the beam splitter 145, a portion of the light passes through the beam splitter and is sent to the optical fiber by the fiber injection system 170. Light travels to the fiber network 420. A portion of this light travels to the optical fiber 424, where it enters the measurement channel of the ADM electronic device 715.

  Locator camera system 950 includes a camera 960 and one or more light sources 970. The locator camera system is also shown with respect to the laser tracker 10 of FIG. 1, where the camera is element 52 and the light source is element 54. Camera 960 includes a lens system 962, a photoreceptor array 964, and a body 966. One application of the locator camera system 950 is to locate the retro-reflector target in the workspace. This is done by blinking the light source 970 which is picked up as a bright spot on the photoreceptor array 964 by the camera. A second use of the locator camera system 950 is to determine the general orientation of the 6-DOF device 4000 based on the observed position of the reflector spot or LED on the 6-DOF device 4000. If two or more locator camera systems 950 can be used on the laser tracker 10, the direction to each retro-reflector target in the workspace can be calculated based on triangulation principles. If one locator camera is positioned to pick up the light reflected along the optical axis of the laser tracker, the direction to each retroreflector target can be identified. If one camera is arranged offset from the optical axis of the laser tracker, the rough direction of the retroreflector target can be obtained immediately from the image on the photoreceptor array. In this case, a more accurate direction to the target can be specified by rotating the mechanical axis of the laser in a plurality of directions and observing changes in the spot position on the photoreceptor array.

  FIG. 14 shows an embodiment of a 6-DOF probe 2000 for use with a laser tracker optoelectric system 900 and a locator camera system 950. In embodiments of the present invention, the laser tracker may be any of the laser trackers disclosed and exemplified herein, such as the laser tracker 10 of FIG. 1, or other similar device not disclosed herein. It may be. The optoelectric system 900 and the locator camera system 950 have been described with reference to FIG. In other embodiments, instead of the optoelectric system 900, an optoelectric system having two or more wavelengths of light is used.

  The 6-DOF probe (or “wand”) 2000 may be handheld in embodiments of the present invention, and includes a body 2014, a retroreflector 2010, a probe telescopic assembly 2050, and an optional electrical cable. 2046, an optional battery 2044, an interface component 2012, an identification element 2049, an actuator button 2016, an antenna 2048, and an electronic device circuit board 2042. The retro reflector 2010 may be a corner cube retro reflector having a hollow core or a glass core. The retro reflector 2010 may be marked so that the orientation camera in the opto-electric system 900 can determine the three orientation degrees of freedom of the 6-DOF probe 2000 physically separated from the laser tracker. One example of such a marking is darkening the line of intersection between the three flat reflector surfaces of the retroreflector 2010, as described in the aforementioned '758 patent.

  The probe telescopic assembly 2050 includes a probe telescopic part 2052 and a probe tip 2054. The probe tip 2054 may be a “hard” contact type probe tip, which makes a 3-coordinate measurement of the object surface by physically contacting the test object and determining the 3D coordinates of the probe tip 2054. In the embodiment of FIG. 14, the probe tip 2054 is connected to the body 2014 of the retroreflector 2010, but is separated from it and positioned a distance away from it, but the 6-DOF laser tracker is separated from the laser tracker. It is known that the 3D coordinates of the probe tip 2054 at a point hidden from the line of sight of the light beam 784 sent to the 6-DOF probe 2000 can be easily determined. Therefore, the 6-DOF probe is sometimes referred to as a hidden-point probe.

  Power may be supplied on an optional electrical cable 2046 or by an optional battery 2044. The power supply supplies power to the electronic device circuit board 2042. The electronic device circuit board 2042 provides power to an antenna 2048 that may be in communication with the laser tracker or external computer and to an actuator button 2016 that allows the user to communicate with the laser tracker or external computer in a conventional manner. . The electronic device circuit board 2042 also provides power to LEDs, material temperature sensors (not shown), air temperature sensors (not shown), inertial sensors (not shown), or inclinometers (not shown). It's okay. The interface component 2012 may be, for example, a light source (such as an LED), a small retro-reflector, a reflective material region, or a fiducial mark. The interface component 2012 is used to determine the approximate orientation of the retro-reflector 2010, which is necessary for the 6-DOF angle calculation to determine the 6-DOF probe 2000 reference frame. The identification element 2049 is used to provide a parameter or serial number for the 6-DOF probe 2000 to the laser tracker. The identification element may be, for example, a barcode or an RF identification tag.

  The laser tracker may alternatively supply a light beam 784 to the retro-reflector 2011. By providing a light beam 784 to any of the plurality of retro-reflectors, a hand-held 6-DOF probe, or wand 2000, can be physically moved in various directions using the probe telescopic assembly 2050 during object exploration. Can be oriented.

  The six degrees of freedom of the probe 2000 measured by the laser tracker may be considered to include three translational degrees of freedom and three azimuth degrees of freedom. The three translational degrees of freedom may include a radial distance measurement between the laser tracker and the retroreflector, a first angle measurement, and a second angle measurement. Radial distance measurements may be made with an IFM or ADM in the laser tracker. The first angle measurement may be performed by an azimuth angle measurement device such as an azimuth angle encoder, and the second angle measurement may be performed by a zenith angle measurement device such as a zenith angle encoder. Alternatively, the first angle measuring device may be a zenith angle measuring device, and the second angle measuring device may be an azimuth angle measuring device. The radial distance, the first angle measurement, and the second angle measurement constitute three coordinates in the spatial coordinate system, which can be converted to three coordinates in a Cartesian or other coordinate system.

  The three orientational degrees of freedom of the probe 2000 may be determined using a patterned corner cube, as described in the '758 patent cited above and cited above. Alternatively, other methods for determining the three orientation degrees of freedom of the probe 2000 may be used. For example, a usable method is to image a set of points of light on a 6-DOF haptic probe and simultaneously send a light beam to a retro reflector on the 6-DOF probe to determine the distance to the retro reflector and two angles. It is to provide a 6-DOF laser tracker having a camera for judging. The point of light imaged on the camera in the laser tracker provides three orientation degrees of freedom. In another example, a usable method provides a 6-DOF laser tracker having a retro-reflector that has its vertex partially moved so that light can be transmitted through the vertex to the position detector. It is to be. The position of the transmitted light on the position detector can be used to determine the pitch and yaw of the retro reflector. In addition, the retroreflector may be mounted in a structure that includes a multi-axis inclinometer configured to measure the roll angle of the 6-DOF sensor. The combination of the measured roll angle and the measured pitch and yaw angle provides three orientation degrees of freedom. In addition, the laser beam sent to the retro-reflector may be used to determine the distance to the retro-reflector and two angles, thereby providing three translational degrees of freedom.

  Whichever method is used, the position and orientation of a 6-DOF probe (and hence probe tip 2054) in space, such as 6-DOF probe 2000, in space, with three translational degrees of freedom and three orientational degrees of freedom. Is well defined. It is important to note that this is the case for the system considered here, but it is not independent of 6 degrees of freedom, and in 6 degrees of freedom the position and orientation of the device in space. This is because there may be a system that cannot sufficiently define. The term “translation set” is an abbreviated version of a 3-DOF translation of a 6-DOF accessory (such as 6-DOF probe 2000) in the reference frame of the laser tracker. The term “orientation set” is a shortened form of the three orientation degrees of freedom of a 6-DOF accessory (eg, probe 2000) in the reference frame of the laser tracker. A “surface set” is a shortened form of a three-dimensional coordinate of a point on an object surface in a reference frame of a laser tracker measured by a probe tip 2054.

  FIG. 15 shows an embodiment of a 6-DOF scanner 2500 used with the optoelectric system 900 and the locator camera system 950. The 6-DOF scanner 2500 may also be referred to as a “target scanner”. The optoelectric system 900 and the locator camera system 950 have been described with respect to FIG. In other embodiments, an optoelectric system that uses two or more wavelengths of light is used instead of the optoelectric system 900. The 6-DOF scanner 2500 includes a body 2514, one or more retro-reflectors 2510, 2511, a scanner camera 2530, a scanner light projector 2520, an optional electrical cable 2546, an optional battery 2544, and an interface. A component 2512, an identification element 2549, an actuator button 2516, an antenna 2548, and an electronic device circuit board 2542 are included. In certain embodiments, one or more retro-reflectors 2510, 2511, scanner camera 2530, and scanner light projector 2520 are fixed with respect to body 2514. The retroreflector 2510, the optional electrical cable 2546, the optional battery 2544, the interface component 2512, the identification element 2549, the actuator button 2516, the antenna 2548, and the electronic device circuit board 2542 of FIG. Corresponds to reflector 2010, optional electrical cable 2046, optional battery 2044, interface component 2012, identification element 2049, actuator button 2016, antenna 2048, and electronic device circuit board 2042. The description for these corresponding elements is the same as that described for FIG. The scanner projector 2520 and the scanner camera 2530 are used to measure the three-dimensional coordinates of the workpiece 2528 in cooperation.

  Camera 2530 includes a camera lens system 2532 and a photoreceptor array 2534. The photoreceptor array 2534 may be, for example, a CCD or CMOS array. Scanner projector 2520 includes a projector lens system 2523 and a light source pattern 2524. The light source pattern may emit point light, linear light, or an encoded or unencoded two-dimensional (2D) structured light pattern, which will be described in more detail later. If the scanner emits linear light, the scanner may be referred to as a “laser line probe” (LLP). In contrast, if the scanner emits a 2D structured light pattern, the scanner may be referred to as a “structured light scanner”. Structured light patterns are described in, for example, various patterns such as those described in Proceedings of SPIE, a magazine article by Jason Geng published in Volume 7932, “DLP-Based Structured Light 3D Imaging Technologies and Applications”. One of them may be used, which is incorporated herein by reference.

  In general, there are two types of structured light: encoded structured light and uncoded structured light. The general form of uncoded structured light depends on a striped pattern that varies periodically along one dimension. These types of patterns are usually applied continuously to provide a rough distance to the object. Some uncoded pattern embodiments, such as sinusoidal patterns, can provide relatively high accuracy measurements. However, to enable these types of uncoded patterns, it is usually necessary to keep the scanner device and the object stationary relative to each other. If the scanner device or object is moving (with respect to the other), an encoded pattern may be used. The encoded pattern allows an image to be analyzed using one acquired image. Several encoded patterns may be placed in a particular orientation on the projector pattern (eg, perpendicular to the epipolar line on the projector plane), thereby providing a three-dimensional surface coordinate based on one image Analysis becomes easier.

  Light from the projector 2520 is reflected by the surface of the workpiece 2528, and the reflected light is received by the camera 2530. It should be understood that changes or features in the workpiece surface, such as one or more protrusions, cause distortion in the structured pattern when an image of the pattern is taken by the camera 2530. Since the pattern is formed by structured light, in some examples, the controller can determine a one-to-one correspondence between the pixels in the emitted pattern and the pixels in the imaged pattern. As a result, the coordinates of each pixel in the imaged pattern can be determined using the principle of triangulation. A set of 3D coordinates on the workpiece surface is sometimes referred to as a point cloud. For example, a point cloud can be created from the entire workpiece 2528 by moving the 6-DOF scanner 2500 on the surface with a spindle or the like.

  If the scanner light source emits point light, this point may be scanned, for example by a moving mirror, thereby generating a line or an array of lines. If the scanner light source emits linear light, this line may be scanned, for example with a moving mirror, thereby generating an array of lines. In one embodiment, the light source pattern is an LED, a laser, or a DMD (digital micromirror device) such as Texas Instruments' digital light projector (DLP), a liquid crystal device (LDC), or a liquid crystal on LCOS (liquid crystal device). Other light sources may be used, or similar devices used in transmission mode rather than reflection mode. The light source pattern may also be a slide pattern, such as a chroma-on-glass slide, which may have a single pattern or multiple patterns, and the slide moves to a predetermined position as needed. Is removed from there. Additional retro-reflectors such as retro-reflector 2511 may be added to the first retro-reflector 2510 to allow the laser tracker to track the 6-DOF scanner from various directions, thereby directing light to the 6-DOF projector 2500. Increases the flexibility in terms of the direction in which it can be projected.

  The 6-DOF scanner 2500 may be held by hand or attached to a tripod, instrument stand, electric carriage, or robot end effector, for example. The three-dimensional coordinates of the workpiece 2528 are measured by the scanner camera 2530 using the principle of triangulation. There are several measurement methods by triangulation depending on the pattern of light emitted by the scanner light source 2520 and the type of the photoreceptor array 2534. For example, when the pattern light emitted from the scanner light source 2520 is linear light or dot light scanned in the shape of a line, and the photosensitive element array 2534 is a two-dimensional array, one dimension of the two-dimensional array 2534 is This corresponds to the direction of the point 2526 on the surface of the workpiece 2528. The other dimension of the two-dimensional array 2534 corresponds to the distance of the point 2526 from the scanner light source 2520. Therefore, the three-dimensional coordinates of the 6-DOF scanner 2500 at each point 2526 along the linear light emitted from the scanner light source 2520 are known with respect to the local reference frame. The six degrees of freedom of the 6-DOF scanner is grasped by the 6DOF laser tracker using the method described in the '758 patent. With six degrees of freedom, the three-dimensional coordinates of the scanned linear light can be identified within the tracker's reference frame, which means that the laser tracker measures, for example, three points on the workpiece, It may be converted into a reference frame.

  When the 6-DOF scanner 2500 is held by hand, the line of laser light emitted by the scanner light source 2520 may be moved to “paint” the surface of the workpiece 2528, thereby Three-dimensional coordinates are obtained. It is also possible to “paint” the surface of the workpiece using a scanner light source 2520 that emits a structured light pattern. Alternatively, when using a scanner 2500 that emits a structured light pattern, more accurate measurements can be made by attaching the 6-DOF scanner to a tripod or instrument stand. The structured light pattern emitted by the scanner light source 2520 may include, for example, a fringe pattern, and the illuminance of each fringe changes sinusoidally on the surface of the workpiece 2528. In some embodiments, the sine wave is shifted by three or more phase values. The amplitude level recorded by each pixel of camera 2530 for each of three or more phase values is used to provide the position of each pixel in a sinusoidal shape. The use of this information helps to determine the three-dimensional coordinates of each point 2526. In other embodiments, the structured light may be in the form of an encoded pattern that is evaluated and evaluated in three-dimensional coordinates based on a single image frame rather than a plurality collected by the camera 2530. Can be determined. By using the encoded pattern, it may be possible to make a relatively accurate measurement while moving the 6-DOF scanner 2500 by hand at a reasonable speed.

  Projecting a structured light pattern to linear light has several advantages. In linear light projected from a handheld 6-DOF scanner 2500 such as LLP, the density of points may be high along the lines and much lower between the lines. In a structured light pattern, the space between points is usually substantially the same in each of two orthogonal directions. In addition, in some modes of operation, 3D points calculated with structured light patterns may be more accurate than other approaches. For example, a series of structured light patterns may be emitted by fixing the 6-DOF scanner 2500 in place, for example by attaching it to a stationary stand or mount, thereby capturing one pattern. It can be calculated more accurately than is possible with other methods (ie, single shot method). An example of a series of structured light patterns is one in which a pattern having a first spatial frequency is projected onto an object. In some embodiments, the projected pattern is a striped pattern in which the optical power varies sinusoidally. In some embodiments, the phase of the sinusoidally changing pattern is shifted, thereby shifting the fringes laterally. For example, the pattern may be projected at three phase angles that are shifted 120 degrees each time with respect to the previous pattern. This projection sequence provides sufficient information to determine the phase of each point of the pattern relatively accurately, regardless of background light. This can be done point by point without considering adjacent points on the object surface.

  The above procedure determines the phase of each point when the phase range between two adjacent lines extends from 0 to 360 degrees, but there may still be doubt as to which line is which. A way to identify the line is to use a sinusoidal pattern that repeats the phase sequence as described above, but with different spatial frequencies (ie, different flange pitches). In some cases, the same method needs to be repeated for three or four different fringe pitches. Methods for eliminating ambiguity using this method are well known in the art and will not be described further here.

  In order to obtain as high accuracy as possible using a continuous projection method such as the sinusoidal phase shift method described above, it may be advantageous to minimize the movement of the 6-DOF scanner. The position and orientation of the 6-DOF scanner can be determined from the 6-DOF measurement performed by the laser tracker, and it is possible to correct for the movement of the hand-held 6-DOF scanner, but the resulting noise is Will be somewhat larger than if it were kept stationary by placing it on a stationary mount, stand, or fixture.

  The scanning method represented by FIG. 15 is based on the principle of triangulation. The principle of triangulation will be described in more detail with reference to system 2560 in FIG. 15A and system 4760 in FIG. 15B. Referring first to FIG. 15A, system 2560 includes a projector 2562 and a camera 2564. Projector 2562 includes light source pattern 2570 in the light source plane and projector lens 2572. The projector lens may include several lens elements. The projector lens has a lens perspective projection center 2575 and a projector optical axis 2576. Ray 2573 travels from point 2571 on the light source pattern through the lens perspective projection center to object 2590 and strikes it at point 2574.

  Camera 2564 includes a camera lens 2582 and a photoreceptor array 2580. The camera lens 2582 has a lens perspective projection center 2585 and an optical axis 2586. Ray 2583 strikes the photoreceptor array 2580 at point 2581 from point 2574 on the object, through the camera perspective projection center 2585.

  The line segment connecting the perspective projection centers is the base line 2588 in FIG. 15A and the base line 4788 in FIG. 15B. The length of the baseline is called the baseline length (2592, 4792). The angle between the projector optical axis and the baseline is the baseline projector angle (2594, 4794). The angle between the camera optical axis (2586, 4786) and the baseline is the baseline camera angle (2596, 4796). If it is known that a point (2571, 4771) on the light source pattern corresponds to a point (2581, 4781) on the photoreceptor array, the baseline length, baseline projector angle, and baseline camera angle are used. Thus, it is possible to determine the sides of the triangle connecting points 2585, 2574, and 2575 and thus determine the surface coordinates of the points on the surface of object 2590 relative to the reference frame of measurement system 2560. To do this, a triangulation scheme as described with respect to FIG. 15 is used. The angle of the small triangular side between the projector lens 2572 and the light source pattern 2570 is between the known distance between the lens 2572 and the plane 2570 and the point where the point 2571 and the optical axis 2576 intersect the plane 2570. Identified using distance. These small angles can be added to or subtracted from the larger angles 2596 and 2594 as necessary to obtain the desired angle of the triangle. For those skilled in the art, equivalent mathematical techniques can be used to determine the length of the sides of triangle 2574-2585-2575, or other related triangles can be used to obtain the desired coordinates of the surface of object 2590. It will be clear that it is good.

  Referring first to FIG. 15B, system 4760 is similar to system 2560 of FIG. 15A, except that system 4760 does not include a lens. The system may include a projector 4762 and a camera 4764. In the embodiment shown in FIG. 15B, the projector includes a light source 4778 and a light modulator 4770. The light source 4778 may be a laser light source because such a light source is kept in focus over a long distance using the shape of FIG. 15B. Light ray 4773 from light source 4778 strikes light modulator 4770 at point 4771. Other rays from light source 4778 strike the light modulator at other locations on the modulator surface. In some embodiments, the light modulator 4770 changes the power of the emitted light, which is most often done by a certain amount of attenuation of the light power. In this case, the light modulator imparts an optical pattern to the light, referred to herein as the light source pattern, which is on the surface of the optical modulator 4770. The optical modulator 4770 may be, for example, a DLP or LCOS device. In some embodiments, modulator 4770 is transmissive rather than reflective. The light emitted from the light modulator 4770 appears to be emitted from the virtual fluoroscopic projection center 4775. Rays are emitted from the virtual fluoroscopic projection center 4775 and travel through point 4771 to point 4774 on the surface of object 4790.

  The base line is a line segment extending from the camera lens perspective projection center 4785 to the virtual light perspective projection center 4775. In general, triangulation methods include identifying the length of the sides of a triangle, eg, a triangle having vertices 4774, 4785, and 4775. One way to do this is to identify the baseline length, the angle between the baseline and the camera optical axis 4786, and the angle between the baseline and the projector reference axis 4776. In order to identify the desired angle, additional, smaller angles are identified. For example, the small angle between the camera optical axis 4786 and the light beam 4783 is the small triangular angle between the camera lens 4784 and the photoreceptor array 4780, the distance from the lens to the photoreceptor array and the camera optical axis to the pixel. It can be specified by solving based on the distance. The small triangle angle is then added to the angle between the baseline and the camera optical axis to identify the desired angle. Similarly for projectors, the angle between projector reference axis 4776 and ray 4773 is the angle of the small triangle between these two lines, the known distance between light source 4777 and the surface of the light modulator, and reference axis 4776 is It is specified by solving based on the distance of the projector pixel at 4771 from the point intersecting the surface of the light modulator 4770. By subtracting this angle from the angle between the baseline and the projector reference axis, the desired angle is obtained.

  The camera 4764 includes a camera lens 4782 and a photoreceptor array 4780. The camera lens 4782 has a camera lens perspective projection center 4785 and a camera optical axis 4786. The camera optical axis is an example of a camera reference axis. From a mathematical point of view, any axis that passes through the camera lens perspective projection center can be used equally easily in triangulation calculations, but the camera optical axis, which is the lens's symmetry axis, is routinely selected. Ray 4783 passes from the point 4774 on the object through the camera perspective projection center 4785 and strikes the photoreceptor array 4780 at point 4781. Other equivalent mathematical techniques may be used to solve the side lengths of triangles 4774-4785-4775, as will be apparent to those skilled in the art.

  The triangulation method described here is well known, but for completeness, additional technical information is provided below. Each lens system has an entrance pupil and an exit pupil. The entrance pupil is a point where light appears to appear from the viewpoint of the primary optical system. The exit pupil is the point at which light appears to appear as it travels from the lens system to the photoreceptor array. For a multi-element lens system, the entrance and exit pupils do not necessarily match, and the angle of the rays with respect to the entrance and exit pupils is not necessarily the same. However, the model considers the perspective projection center as the entrance pupil of the lens and then adjusts the distance from the lens to the light source or imaging plane so that the rays continue along a straight line and enter the light source or imaging plane By doing so, it can be simplified. In this way, a simple and widely used model is obtained as shown in FIG. 15A. It should be understood that this description provides a good first-order approximation of the light behavior, but a finer correction for lens aberrations that may slightly shift the ray with respect to the position calculated using the model of FIG. 15A. It can be performed. Baseline length, baseline projector angle, and baseline camera angle are commonly used, but it should be understood that saying these quantities are necessary is similar but slightly different The possibility of applying the formula is not excluded, and in that case the generality in the description provided herein is not impaired.

  When using a 6-DOF scanner, several types of scan patterns may be used, and it may be advantageous if different types can be combined to obtain the best performance in the shortest time. For example, in one embodiment, the fast measurement method may use a two-dimensional encoded pattern, in which case the three-dimensional coordinate data is obtained in a single shot. In the usage of the encoded pattern, different elements may be provided, for example using different letters, different shapes, different thicknesses or sizes, or different colors, which are encoded elements or encodings. Also called feature. Such a feature may be used to allow point 2571 to be matched to point 2581. Encoded features on the light source pattern 2570 may be identified on the photoreceptor array 2580.

  A scheme that can be used to simplify the matching of the encoded features is the use of epipolar lines. The epipolar line is a mathematical line, and is formed by the intersection of the epipolar surface and the light source surface 2570 or the imaging surface 2580. The epipolar plane is any plane that passes through the perspective projection center of the projector and the perspective projection center of the camera. Epipolar lines on the light source plane and the imaging plane may be parallel in some special cases, but are generally not parallel. One aspect of the epipolar line is that one epipolar line on the projector plane has a corresponding epipolar line on the imaging plane. Thus, any particular known pattern for epipolar lines in the projector plane may be immediately observed and evaluated in the imaging plane. For example, if the encoded pattern is arranged along an epipolar line in the projector plane, the spacing between the encoded elements in the imaging plane uses the numerical value read by the pixels of the photoreceptor array 2580. This information may be used to determine the three-dimensional coordinates of point 2574 on the object. It is also possible to efficiently extract the object surface coordinates by tilting the encoded pattern at a known angle with respect to the epipolar line.

  The advantage of using an encoded pattern is that 3D coordinates for points on the object surface can be quickly obtained. However, in most cases, the continuous structured light method such as the sine wave phase shift method as described above produces more accurate results. Thus, the user can advantageously choose to measure a specific object or a specific object region or feature using different projection methods depending on the desired accuracy. Such a selection can be easily made by using a programmable light source pattern.

  According to one embodiment of the invention, the 6-DOF triangulation scanner 2500 of FIG. 15 also includes an augmented reality (AR) camera 2508. The AR camera 2508 may be considered to be able to capture “full field of view” images. The AR camera 2508 includes a camera lens 2502 and a photoreceptor array 2504. The photoreceptor array 2504 may be a CCD or CMOS array, for example. Therefore, the AR camera 2508 may be digital in nature and may capture still images or video images.

  As shown in FIG. 15, the AR camera 2508 is connected to the main body 2514 of the 6-DOF triangular measurement scanner 2500 by a broken line 2506. However, this is for illustration purposes only. The AR camera 2508 may be an integral part of the scanner main body 2014. As used herein, the term “integral” means that the AR camera 2508 is permanently or temporarily attached to the scanner body 2014 and the AR camera 2508 is in a fixed spatial relationship with respect to the scanner body 2014. Means. In any method, since the six degrees of freedom of the scanner 2500 are known as described above, the six degrees of freedom (ie, “posture”) of the AR camera 2508 is known for each image taken by the AR camera 2508. Thus, the laser tracker, 6-DOF scanner 2500, and AR camera 2508 may all be installed in a common reference frame. In some embodiments, the AR camera is also a camera used in triangulation. In other words, in one embodiment, the AR camera is the camera 2530 of FIG.

  The lens 2502 (which may be a lens system including a plurality of lens elements) is the perspective projection center of the lens. It may be considered that the light beam passing through the lens 2502 reaches the photoconductor array 2504 after passing through the perspective projection center. In careful analysis, the lens 2502 may be characterized by causing lens aberrations that cause the light beam intersection position on the photoreceptor array 2504 to be slightly misaligned. However, without loss of generality, it can be said that the rays pass through the perspective projection center and that aberration correction of the image is provided within another step of image processing.

  The surface of the object to be investigated is imaged on the photoreceptor array 2504 by the lens 2502, and an image is formed on a pixel set that is a part of the photoreceptor array 2504. Light that strikes each pixel is converted from a charge to a digital signal during the integration period of the camera. An analog-to-digital converter that is either inside the photoreceptor array 2504 (in the case of a CMOS array) or outside the array (in the case of a CCD array) performs an analog to digital signal conversion. The signal for each pixel is typically provided in a binary representation between 8-12 bits. These bits 1 and 0 are carried in the parallel channel and may be converted to serial form using the serializer / deserializer function for transmission on the bus line.

  As described above, the 6-DOF triangulation scanner 2500 is handheld in the embodiment of the present invention. However, in other embodiments, the scanner 2500 may be held stationary by placing it on a stationary mount, stand, or fixture, such as a tripod. Furthermore, although the position and orientation of the 6-DOF triangulation scanner 2500 can be known from the 6-DOF measurement by the laser tracker as described above, the movement of the 6-DOF scanner 2500 held by the hand can be corrected. The resulting noise may be somewhat greater than if the scanner 2500 was kept stationary. It is also possible to attach the 6-DOF triangulation scanner 2500 to a robot or machine tool.

  In an embodiment of the present invention, a plurality of two-dimensional (2D) camera images taken by an augmented reality camera 2508 that is part of a 6-DOF triangulation scanner 2500 (which is itself used with a laser tracker) are: Coupled or “aligned” to each other in accordance with the methods described in, eg, various real-world features such as the surface of an object or any real-world scene (eg, the interior of a building, the scene of a vehicle accident, or Get a three-dimensional (3D) image representation of the crime scene. In this method, since the posture or 6 degrees of freedom of the scanner 2500 having an integrated AR camera 2508 is known for each 2D photograph or image captured by the AR camera 2508, a plurality of 2D photograph images captured by the AR camera 2508 are used. Based on the fact that they may be combined together to form a 3D image.

  As used herein, the term 3D image or composite 3D image is taken to mean a two-dimensional displayable representation of the recorded scene from various perspectives, and a displayed 2D image. Changes from viewpoint to viewpoint, and a 3D representation of the scene is recreated for the viewer. In addition, 3D coordinate values for points observed in the 3D image may be extracted.

  The method according to this embodiment will now be described with reference to the method 1600 of FIG. In step 1605, a 6-DOF triangulation scanner 2500 and a coordinate measurement device are provided. The 6-DOF scanner 2500 also includes a retro-reflector 2510 and an integral AR camera 2508 as described above with respect to FIG. The coordinate measuring device may include a laser tracker such as the laser tracker 5 described above, has a device reference frame, and is separated from the 6-DOF triangular measurement scanner 2500. The coordinate measuring device includes an orientation sensor, first and second motors, first and second angle measuring devices, a distance meter, a position detector, a control system, and a processor. The laser tracker and 6-DOF scanner with AR camera may be the same as or similar to those described and illustrated above.

  Step 1610 is a step of measuring, in the first case, two rotational angles of the retroreflector and the distance to the retroreflector and three orientation degrees of freedom of the 6-DOF triangulation scanner 2500 using the apparatus. In this step, a 2D image is also formed on the AR camera 2508. The electronic device circuit board 2542 in the 6-DOF scanner 2500 may process and / or transmit position and orientation information from the AR camera 2508. The electronic device circuit board 2542 may also receive a first digital signal representing a 2D image sent to the photoreceptor array 2504 through the camera lens 2502.

  In step 1615, the 6-DOF triangulation scanner 2500 is moved to a new position, and the apparatus measures the two angles and distance to the 6-DOF scanner 2500 and the orientation of the 6-DOF scanner 2500. This also forms a 2D image at the new location. The electronic device circuit board 2542 may process the position and orientation information from the AR camera 2508 at this second position and orientation and / or transmit it to the laser tracker. The electronics circuit board 2542 may also receive a second digital signal representing a 2D image that is sent through the camera lens 2502 to the photoreceptor array 2504.

  At step 1620, common points common to the first image and second image are identified. The term “principal point” is typically used to refer to points that are identified in an image and that can be used to interconnect or align the images. Also, these points are typically not intentionally placed by someone at that location. The step includes determining a corresponding position of a principal point on the photoreceptor array in the first and second cases. The position of the main point in the first case is called a first position, and the position of the main point in the second case is called a second position. Many techniques that can be used to determine such a main point have been developed, and a method called image processing or feature detection is generally used. A commonly used but general category for identifying principal points is called interest point detection, and the detected points are called interest points. According to normal definitions, points of interest are mathematically well-founded definitions, well-defined locations in the space, image structures rich in local information around the points of interest, and time Even with relatively stable illumination level variations. A specific example of a point of interest is a corner point, which may be, for example, a point corresponding to the intersection of three planes. Another example of signal processing that can be used is the SIFT (Scale Invariant Feature Transform), which is well known in the art and is described in Lower US Pat. No. 6,711,293. At step 1620, the processor identifies these key points common to the first and second images and obtains at least one key point (although typically a large set of key points). Other common feature detection methods for identifying key points include edge detection, blob detection, and ridge detection.

  Step 1625 is a step of determining 3D coordinates of corresponding principal points in the first and second 2D images in the first reference frame. This determination first uses triangulation to determine the 3D coordinates of the main points in the first and second images in the reference frame of the 6-DOF scanner 2500, and then uses coordinate transformation to determine the device reference. This may be done by obtaining a key point in the first reference frame, which may be a frame. These coordinate transformations are based, at least in part, on two angle measurements, a distance measurement, and an azimuth measurement provided by the device. Of course, in this calculation, the position and orientation of the 2D camera 2508 in the 6-DOF scanner 2500 is known, for example, based on measurements at the factory. Based on the measured values, a baseline having a baseline distance is between the perspective projection centers of the cameras of the first example (corresponding to the first 2D image) and the second example (corresponding to the second 2D image). May be drawn. Since the baseline distance is known from the 6-DOF measurement by the tracker, each principal point is determined in 3D space within the first reference frame. In general, the 3D coordinates of the principal point of the first image and the 3D coordinates of the corresponding principal point in the second image will not match exactly. However, 3D coordinate averaging or similar methods may be used to determine a representative 3D image in the first reference frame for the corresponding principal point in the first and second 2D images. When this method is used, the 3D coordinates are properly scaled, i.e., the 3D coordinates have the correct length units in the first reference frame. This is in contrast to a method in which two images are acquired with cameras held at two different positions, in which the position and orientation of the camera perspective projection center is not known in each case. This method does not provide enough information to properly scale (in length units) the 3D image.

  Step 1630 is a step of forming a composite 3D image based at least in part on the first and second 2D images and the 3D coordinates of principal points in the first reference frame. In most cases, the first and second 2D images share many principal points, and for each of these, 3D coordinates can be obtained. These principal points form a framework for 3D representation, on which other image elements may be interpolated between two 2D images or (in the more general case) between multiple 2D images. In addition to providing 3D coordinate information, the composite 3D image can also provide texture and color information obtained not only from the principal points, but also from the visible region between the principal points using the same interpolation method. The 3D coordinates obtained in steps 1625 and 1630 are the same for the corresponding principal points in the first and second images, but the composite 3D image is also subject to color and texture information, eg, by interpolation between the principal points. In addition, it may contain additional 3D information.

  Step 1635 is a step of storing the composite 3D image.

  In some embodiments, a processor inside the tracker is used to build and store a composite 3D image and combine it with the measured 3D coordinates. Such a processor may include one or more of a master processor 1520, a 6-DOF processor 1531, and a camera processor 1532. In addition, the computer 1590 may also be used to build and store a composite 3D image and combine it with 3D coordinates obtained from the master processor, or just for that purpose. In other embodiments, the cloud processor may be used to send raw data to a computer network, possibly a remote computer network, to form a composite 3D image. In other embodiments, different combinations of processors may be used. Such a combination of processors may be called a processor system.

  Although not included in the procedure 1600 of FIG. 16, it will be apparent that the method described above can be extended to an optional number of photoreceptor array images, whereby a set of principal points may be obtained from multiple images. In this case, each principal point may correspond to a principal point in some of the images obtained with the photoreceptor array 2504 with the AR camera 2508 in a different posture. For a principal point, the intersection of multiple lines projected from the photoreceptor array 2504 through the perspective projection center of the camera lens 2502 can be determined using, for example, least squares, using a best-fit method by optimization methods well known in the art. It may be determined using a minimization scheme. The alignment of the multiple images may be further optimized by providing a target on or near the test object as desired. The additional target may be, for example, a reflective target or a light emitting diode.

  If AR camera 2508 is a color camera, the reconstructed 3D surface may be represented in color, or other texture attributes may be derived. In other embodiments, the method 1600 may provide various features of the light pattern in addition to the 3D surface profile. For example, “X” marked on the surface of an object may be extracted in addition to the general coordinates corresponding to the position of “X”.

  In some cases, it may be known in advance that a particular portion of the surface being photographed is relatively smooth. In other words, these parts do not have distinct discontinuities or fine features. In such cases, it may be possible to construct an unmeasured portion of the surface in three dimensions using established principal points. For example, the principal points may be smoothly fitted into a cylindrical shape over a portion of the surface, and therefore the software may automatically provide a cylindrical shape.

  If the overall shape of a portion of the surface is known, it may be possible to project the captured image onto the surface. For example, suppose there is a color pattern on the surface, which can be projected onto an assumed surface, which in certain cases may be a flat surface. In this case, this pattern is on the assumed surface from each of the images acquired for the AR camera 2508 in different poses (where “pose” is a combination of a 3 degree of freedom position and a 3 degree of freedom orientation). It may be projected. In this example, the image would be expected to overlay the surface. If this is not the case, it means that the assumed shape is not correct and the shape should be changed. In this example, it may be good practice to obtain additional principal points based on images taken by differently positioned AR cameras 2508. These additional key points can then be used to more accurately determine the surface profile.

  The AR camera 2508 may be used, for example, to capture a background image of a relatively far background object with a relatively large field of view, which may also be used to capture a foreground image, This may be, for example, an image of an object scanned by the 6-DOF triangulation scanner 2500.

  As can be seen from the above-described “dynamic triangulation” method 1600 shown in FIG. 16, images taken by the AR camera 2508 at various camera positions and orientations are partly free of six degrees provided by a laser tracker. Aligned to each other based on degree knowledge. Therefore, it is possible to align the 2D image from the AR camera 2508 with appropriate dimensional expansion and reduction and fewer camera images than possible by another method.

  As 3D images are created according to embodiments of the present invention, data may be overlaid or overlaid on these images. For example, if the 3D image is being constructed or that of an already constructed object, the data superimposed on the 3D image may include CAD design data for that object. CAD data may be stored in a memory associated with the laser tracker 5 (FIG. 1). Other types of data may be superimposed on the camera image, for example, marks that indicate where various assembly operations (drilling, mounting, etc.) are to be performed.

  The AR camera 2508 in the 6-DOF triangulation scanner 2500 may be used to measure the surroundings instead of (or in addition to) the parts measured by the scanner 2500. For example, AR camera 2508 may have a relatively long focal length, thereby providing a higher resolution image that captures its surroundings in greater detail than triangulation camera 2530. In other embodiments, the AR camera 2508 has a relatively short focal length, thereby allowing a wider field of view than the triangulation camera 2530. The reconstruction method described above may be used to obtain a surrounding 3D representation based on AR camera images. For example, one or more parts measured by scanner 2500 with an accuracy of a few micrometers to tens of micrometers may be placed in a perimeter measured with an accuracy of a few millimeters based on AR camera images. .

  Software may be used to observe the object and surroundings from different viewpoints and different distances, and the parallax between the object and the surroundings is properly represented. In some cases, background information may be important. For example, a project may include attaching the structure to the object to be measured while ensuring that there is a suitable space in the 3D perimeter with the 3D image obtained with the AR camera 2508. Such a structure may be available as a CAD model, as a scanned image of a part or assembly, or as a scaled 3D representation obtained by using multiple camera images.

  In some cases, AR camera 2508 may typically be used to obtain a representation of an invisible area. For example, an AR camera may be used to view all surfaces of an object and obtain a 3D image of an area that is not easily measured by the scanner 2500. Such full coverage from all directions is useful when images are displayed, for example, in presentations, websites, or brochures. The addition of color (texture) from the AR camera 2508 is also valuable in this example. The 3D representation obtained from the AR camera 2508 may be supplemented by other 3D representations. Parts, assemblies, furniture, and other models may optionally be downloaded from a file or website and incorporated into a composite 3D representation.

  Another important application of AR camera 2030 and 6-DOF triangulation scanner 2500 is to properly scale the surroundings. For example, the wall may have a left surface, a right surface, an upper surface, and a lower surface. Although the method of matching principal points described above provides a scaled 3D image, the dimensional accuracy is much better when 3D coordinates are measured with a 6-DOF scanner 2500 than with a camera image alone. By combining the composite 3D image obtained from the 2D AR camera image with several measurements by the 6-DOF triangulation scanner 2500, the scaling accuracy of the composite 3D image can be greatly improved in many cases. For example, building scaling can be improved by measuring one or more positions of each of the left, right, top, and bottom surfaces with the scanner 2500.

  The AR camera 2508 may be used to measure ambient only, only objects, or both ambient and objects. As used herein, the term “object” means an item for which accurate dimensional information is desired. The object is typically measured by a 6-DOF triangulation scanner 2500 having an accuracy on the order of tens of micrometers. Measurement with the AR camera 2508 allows an image to be superimposed on a drawing (eg, CAD). In addition, by obtaining a 2D image of an object from multiple directions, it is possible to provide an overlay to an object from any direction.

  An object may be placed within its perimeter, and its 3D coordinates are obtained through the use of AR camera 2508. The information provided by the AR camera and the 6-DOF triangulation scanner 2500 allows the object to be viewed from various viewpoints with respect to the surroundings and the object or surroundings to be viewed from all directions.

  In some embodiments, a purely graphical element (which can be, for example, a photographic element, a drawing element, or a rendered element) is placed in a composite image. A first example of such a graphic element is the addition to a machine tool on the factory floor. Such addition may be superimposed on the CAD model, on which the composite color image is superimposed. The addition may be a newly machined part. Such additional sets may be installed with respect to the factory environment to ensure that all elements fit properly. A second example of such a graphic element is a new item of machinery or equipment placed in the same factory environment. The question may be whether such factors fit into the new plan. In some cases, a website may be utilized that allows such 3D images to be downloaded from a cloud of a network normally found on the Internet through a service provider. With some user interfaces, such 3D components may be moved to the proper position with a computer mouse and then viewed from different positions and orientations.

  Although the present invention has been described in terms of exemplary embodiments, those skilled in the art will recognize that various changes can be made and equivalent elements may be substituted without departing from the scope of the invention. I will. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from its basic scope. Accordingly, the invention is not limited to the specific embodiments disclosed as the best mode contemplated for carrying out the invention, and the invention is intended to be embraced by all embodiments that fall within the scope of the appended claims. It is intended to include. Furthermore, the use of terms such as first, second, etc. does not indicate any order or importance; rather, terms such as first, second, etc. Used to distinguish from. Further, the use of articles (a, an, etc.) is not a quantity limitation but indicates that there is at least one item mentioned.

Claims (9)

In a three-dimensional (3D) coordinate measurement system,
A six degree of freedom (6-DOF) unit having a unit reference frame, including a structure, a retroreflector, a triangulation scanner, and a virtual reality (AR) camera, the retroreflector, the triangulation scanner, and The AR camera is attached to the structure, the triangulation scanner includes a first camera and a projector, the projector is configured to project pattern light onto the object, and the first camera A first image is configured to form on the object, and the projector and the first camera are configured to cooperatively determine a first 3D coordinate of a point on the object in the unit reference frame. The determination is made based at least in part on the projected pattern light and the first image, and the AR camera acquires a color image within the unit reference frame A unit such that a color camera configured to so that,
A coordinate measuring device having a device reference frame, configured to measure the attitude of a retroreflector in the device reference frame, wherein the measured posture includes a six-degree-of-freedom measurement of the retroreflector. ,
A three-dimensional (3D) coordinate measurement system. The 3D coordinate measurement system according to claim 1,
The coordinate measuring device includes a light source and a distance meter, wherein the light source is configured to send a light beam to the retro-reflector and receive the reflected light beam in response, the distance meter to the retro-reflector A 3D coordinate measurement system configured to measure distance based at least in part on a reflected light beam. The 3D coordinate measurement system according to claim 2,
A 3D coordinate measuring system, wherein the AR camera has a wider field of view than the first camera. The 3D coordinate measurement system according to claim 1,
A 3D coordinate measurement system, further comprising one or more processors. The 3D coordinate measurement system according to claim 4,
The one or more processors are configured to receive, in the first case, a first AR image from the AR camera and a first measured pose from the device, the one or more processors further comprising: In a second case, the second AR image is received from the AR camera and configured to receive a second measured pose from the device, the one or more processors further comprising the first AR image and the second AR image. Configured to determine a first principal point common to a plurality of AR images, and the one or more processors are further configured to determine a second 3D coordinate of the first principal point in the device reference frame. 3D coordinate measuring system, characterized in that The 3D coordinate measurement system according to claim 5,
The 3D coordinate measurement system, wherein the one or more processors are further configured to combine the first AR image and the second AR image into a composite 3D image. The 3D coordinate measurement system according to claim 5,
The 3D coordinate measurement system, wherein the one or more processors are further configured to combine the composite 3D image with the first 3D coordinates of the points measured by the triangulation scanner. In a three-dimensional (3D) measurement method,
A first beam of first light from the coordinate measuring device is sent to a retro-reflector in a 6-degree-of-freedom (6-DOF) unit and reflected in response to the reflected first first light. Receiving a beam, wherein the coordinate measuring device has a device reference frame, the 6-DOF unit has a unit reference frame, the 6-DOF unit includes a structure, a retroreflector, a triangulation scanner, An AR camera, wherein the retroreflector, the triangulation scanner, and the AR camera are attached to the structure, the triangulation scanner includes a first camera and a projector, the AR camera within the unit reference frame Steps such as being a color camera configured to acquire a color image;
Determining a first attitude of the retro-reflector by the coordinate measuring device based on at least a part of the first beam reflected from the retro-reflector; A step that is a measurement of 6 degrees of freedom of the reflector;
Projecting the first pattern light from the projector onto the object;
Forming, with a first camera, a first image of reflection of a first pattern light from an object;
In one or more processors, the first 3D coordinates of the first point on the object in the device reference frame, at least in part, the first pattern light, the first image, and the determined retro-reflector Determining based on the first posture of
Photographing a first color image in a unit reference frame with an AR camera;
Combining, with one or more processors, the first 3D coordinates and the first color image in a common reference frame, at least in part, based on the determined first attitude of the retroreflector; A method characterized by comprising. The 3D measurement method according to claim 8,
Sending a second beam of first light from the coordinate measuring device to the retro-reflector and receiving a second reflected beam in response;
Determining, by a coordinate measurement device, a second attitude of the retroreflector based at least in part on the reflected second beam;
Projecting the second pattern light from the projector onto the object;
Forming with a first camera a second image of the reflection of the second pattern light from the object;
In one or more processors, the second 3D coordinates of the second point on the object in the device reference frame, at least in part, the second pattern light, the second image, and the determined retro-reflector Determining based on the second posture of
Photographing a second color image in the unit reference frame with an AR camera;
In one or more processors, a retroreflector determined at least in part from a first 3D coordinate, a second 3D coordinate, a first color image, and a second color image in a common reference frame Combining based on the second posture of the retroreflector determined to be the first posture of
A method comprising the steps of: