EP1076221B1 - Un robot avec système de jauge pour déterminer des mesures de données à 3 dimensions - Google Patents

Un robot avec système de jauge pour déterminer des mesures de données à 3 dimensions Download PDF

Info

Publication number
EP1076221B1
EP1076221B1 EP00117126A EP00117126A EP1076221B1 EP 1076221 B1 EP1076221 B1 EP 1076221B1 EP 00117126 A EP00117126 A EP 00117126A EP 00117126 A EP00117126 A EP 00117126A EP 1076221 B1 EP1076221 B1 EP 1076221B1
Authority
EP
European Patent Office
Prior art keywords
robot
contact sensor
position data
image data
sensor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP00117126A
Other languages
German (de)
English (en)
Other versions
EP1076221A2 (fr
EP1076221A3 (fr
Inventor
William D. Long
Charles C. Kingston
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Perceptron Inc
Original Assignee
Perceptron Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Perceptron Inc filed Critical Perceptron Inc
Publication of EP1076221A2 publication Critical patent/EP1076221A2/fr
Publication of EP1076221A3 publication Critical patent/EP1076221A3/fr
Application granted granted Critical
Publication of EP1076221B1 publication Critical patent/EP1076221B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/42—Simultaneous measurement of distance and other co-ordinates
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging

Definitions

  • the present invention relates generally to non-contact gauging systems and, more particularly, a robot-based gauging system and method for determining three-dimensional measurement data of an object.
  • non-contact sensors such as optical sensors
  • the sensor may be moved in relation to the workpiece. Therefore, it is important to know the exact location and orientation of the moving item (either the sensor or the workpiece) each time the sensor acquires image data of the workpiece. This tends to be fairly straightforward for accurate motion devices (e.g., a CMM), since the reported position reflects the actual position of the device.
  • the motion controller of an accurate motion device may include various compensation factors that get applied to the motion control to ensure the commanded position matches the actual position of the device.
  • the high cost associated with these types of motion devices is prohibitive to them being used in a typical manufacturing environment.
  • a conventional industrial robot as the motion device in the gauging system.
  • the robot can be used to move the sensor in relation to the workpiece.
  • Industrial robots are well suited to perform complex motion sequences which are customized for the workpiece.
  • many standard industrial tools are available which allow the programming of the motion sequences to be performed off-line and subsequently downloaded to the robot controller.
  • Robot controllers generally utilize ideal kinematic models when instructing robot motion.
  • the physical characteristics of the robot arm vary slightly from the ideal model, and thus the actual movement of the robot arm varies slightly from the commanded motion sequence. Furthermore, these physical characteristics are likely to change as the robot is subject to wear and thermal changes.
  • Most robot controllers are not equipped with any additional means for translating and/or correcting the reported position into an actual position of the robot arm.
  • conventional industrial robots have not heretofore been used in non-contact gauging systems and other highly accurate manufacturing workstation applications.
  • WO 97/05449 discloses a scanning apparatus which comprises a multiply jointed arm with an arm control unit and a probe.
  • the control unit includes a processing unit which is coupled to a computer.
  • the object is scanned to capture data from a plurality of points on the surface of the object so that the scanner may capture data from two or more points simultaneously.
  • the signal generated is structured light in the form of stripe or area from illumination sources such as a laser diode or bulbs which enable data for the position and color of the surface to be determined.
  • a robot system for recognizing three-dimensional shapes which comprises a slit-light detector which is constructed of a light projector for projecting light so as to form a slit-light on an object to be handled and an image detector for detecting an image of the light-section wave form.
  • the image detector is mounted on an operating member of the robot. The light is scanned, and the slit-light detector detects a range data, on the basis of which a position, posture, inclination etc. of the object in three-dimensional shape are detected, so that the robot can handle the object.
  • a structured light scanner for 3-D robot vision is known.
  • the system uses a 4 x 3 transformation matrix that converts points in the image plane into their corresponding 3D world coordinates.
  • Calibration of the scanner includes computing the coefficient of the matrix by showing to the system a set of lines generated by suitable object edges. A flat trapezoidal object that is shaped in such a manner that no two edges of the top-surface are parallel to each other, is scanned to obtain a calibration using different calibration steps.
  • an object of the invention to disclose a robot-based gauging system for determining three-dimensional measurement data for the surface of an object that allows for a very precise three-dimensional measurement of an object.
  • the gauging system includes a robot having at least one movable member and a robot controller for controlling the movement of the movable member in relation to the surface of an object.
  • a non-contact sensor is coupled to the movable member of the robot for capturing image data representative of the surface of the object.
  • a position reporting device configured as a photogrammetric measurement system is used to report position data representative of the position of the non-contact sensor.
  • a synch signal generator is used to generate and transmit a synch signal to each of the non-contact sensor and the position reporting device, wherein the non-contact sensor reports image data in response to the synch signal and the position reporting device reports position data in response to the synch signal.
  • the gauging system further includes a vision system adapted to retrieve image data from the non-contact sensor and position data from the position reporting device.
  • a vision system adapted to retrieve image data from the non-contact sensor and position data from the position reporting device.
  • the vision system must be able to synchronize the image data with the position data.
  • a memory storage space is used to store position data so that the vision system can synchronize the image data with the position data. In this way, the robot-based gauging system of the present invention is able to quickly and accurately determine three-dimensional measurement data for the object.
  • FIG. 1 An exemplary robot-based gauging system of the type commonly employed in vehicle assembly lines or other automotive applications is shown in Figure 1 .
  • a robot-based gauging system 10 is often used for ensuring that each vehicle body component is assembled within predefined tolerances.
  • the robot-based gauging system 10 may measure the door of a vehicle body 12.
  • the invention is not limited to automotive applications, an exemplary use for the robot-based gauging system 10 would be in an automotive assembly plant.
  • a single non-contact sensor 20 is mounted on a robot arm 22 of a robot 24.
  • the sensor 20 in the present embodiment is a contour sensor which is only capable of two dimensional (2D) measurements.
  • the sensor 20 emits a planar structured light pattern.
  • a characteristic reflected light pattern is achieved which can be detected by the sensor 20.
  • the contour sensor can measure (through triangulation) in two dimensions a plane on the reflecting workpiece.
  • Communication cables 28 connect the sensor 20 and the controller of the robot 24 to a machine vision computer 30 which includes a CRT (cathode ray tube) display 32.
  • a printer 34 is optionally provided with a typical machine vision computer.
  • the robot arm 22 is movable to a plurality of positions for measuring the periphery of a workpiece (e.g., a door).
  • the sensor 20 can continuously acquire data as the robot arm 22 traverses the surface of a workpiece.
  • three-dimensional measurement data for the workpiece may be acquired by "scanning" or continuously moving the two-dimensional measurement sensor in relation to the workpiece.
  • the gauging system 10 is also able to report the three-dimensional measurement data with respect to a predetermined reference frame or coordinate system associated, for example, with the vehicle body 12 to be measured, or with respect to an external reference frame associated with the gauging station.
  • Figure 2 illustrates the basic components associated with the robot-based gauging system 10.
  • a non-contact sensor 20 is affixed to the end of a robot arm 22 which in turn is connected to a robot controller 38.
  • the robot controller 38 is operative to control the movement of the robot arm 22 in relation to the surface of an object to be gauged by the system, and the non-contact sensor 20 is operative to capture image data representative of a portion of the surface of the object.
  • a position reporting device 26 is used to report position data representative of the position of the non-contact sensor 20.
  • the non-contact sensor 20 and the position reporting device 26 are each in turn electrically connected to a vision system 40 residing on the machine vision computer 30.
  • the position of the sensor 20 can be reported either directly or indirectly by the position reporting device 26.
  • the actual position of the sensor 20 is reported by a photogrammetry system to the vision system 40
  • the position of the sensor 20 is deduced from the position of the robot arm 22 which is reported (e.g., by the robot controller) to the vision system 40.
  • the vision system 40 synchronizes the image data with the position data, thereby determining three-dimensional measurement data for the workpiece.
  • the vision system 40 includes a synchronization module 42 and a timing signal generator 44.
  • the vision system 40 uses a timing signal or a hardware latch signal which is generated by the timing signal generator 44.
  • the timing signal is then simultaneously transmitted to the sensor 20 and the position reporting device 26.
  • the sensor 20 records image data for the workpiece and the position reporting device 26 records current position data for the non-contact sensor 20.
  • Image data and position data can then be requested by and delivered to the vision system 40.
  • the vision system 40 acquires the measurement data needed to construct a contour line representative of the surface of the workpiece. It should be noted that the vision system does not rely on the sensor motion being at a constant velocity in relation to the workpiece. On the contrary, it is envisioned that the velocity of the sensor may vary as it scans the workpiece.
  • FIG. 3 A gauging system not covered by the invention which utilizes the indirect approach to acquire position data is shown in Figure 3 .
  • the indirect approach deduces the position of the sensor from the position of the robot arm 22. To do so, the position data is reported by the robot controller 38 to the vision system 40. However, as previously described, the actual position of the robot arm varies from the commanded position data as reported by the robot controller 38.
  • the vision system 40 further incorporates a kinematic correction module 50 which applies a real time kinematic correction factor to the reported position data received from the robot controller 38.
  • the DynaCal Robot Cell Calibration System developed by Dynalog, Inc. of Bloomfield Hills, Michigan may be adapted to serve as the kinematic correction module 50.
  • the kinematic correction module 50 incorporates a procedure that determines physical deviations between the actual robot and its corresponding ideal model. These deviations are stored as robot-specific parameters.
  • the kinematic correction module 50 receives the reported position data from the robot controller 38. Since the robot controller 38 is unaware of the robot's physical deviations, the reported position data is based on the ideal model.
  • the kinematic correction module 50 translates the reported position data into actual position data by using the robot-specific parameters.
  • the actual position data of the robot arm is transformed to position data for the sensor.
  • the sensor position data is then provided to the synchronization module 42.
  • the non-contact sensor 20 is preferably a Tricam non-contact sensor which is manufactured by Perceptron, Inc. of Madison, Michigan. While the following description is provided with reference to the Tricam non-contact sensor, it is readily understood that the explanation is applicable to other non-contact sensors.
  • the timing signal generator 44 is a software module that resides on a circuit board which can easily be incorporated into the machine vision computer 30. Moreover, the timing signal generator 44 is associated with the sensor 20, such that the hardware latch signal is generated internal to the sensor 20. In operation, the sensor generates a hardware latch signal each time it acquires image data. The sensor 20 is further configured to simultaneously transmit the hardware latch signal to the robot controller 38.
  • the Tricam non-contact sensor may support an external input that allows for image acquisition to be initiated by an hardware latch signal which is generated external to the sensor 20.
  • the timing signal generator 44 may be associated with some external computing device (e.g., the robot controller).
  • the Tricam non-contact sensor In order to "scan" the sensor in relation to the workpiece, image data must be captured at a rate which allows for the continuous motion of the sensor.
  • the Tricam non-contact sensor In a continuous operation mode, the Tricam non-contact sensor is limited by the frame rate of its camera which operates at a frequency of 30 Hz.
  • the image data reported by the sensor 20 lags in time behind the position data reported by the robot controller 38 by two frame times (e.g., 2/30 second). Accordingly, the image data received from the sensor 20 requires synchronization with the position data received from the robot controller 38.
  • the present invention synchronizes the image data with the position data as shown in Figure 4 .
  • a memory space internal to the vision system is used to store and process position data received from the robot controller 38.
  • the memory space is further defined into an image memory space for storing image data and at least three positional memory spaces for storing position data.
  • a first timing signal causes the vision system to retrieve position data(t 0 ) from the robot controller which is in turn stored in a first memory space 62.
  • the vision system copies the position data(t 0 ) in the first memory space 62 to a second memory space 64 and then retrieves the next available position data(t 1 ) into the first memory space 62.
  • a third timing signal causes the position data(t 0 ) in the second memory space 64 to be copied to a third memory space 66, the position data(t 1 ) in the first memory space 62 to be copied to the second memory space 64, and new position data(t 2 ) to be retrieved into the first memory space 62.
  • the vision system 40 is able to store the three most recent sets of position data received from the robot controller 38
  • image data(t 0 ) corresponding to the initial position data(t 0 ) is available from the sensor to the vision system.
  • this image data is read into the image memory space 68 of the vision system.
  • the vision system is able to accurately construct measurement data for the workpiece. For each additional timing signal, this synchronization process is repeated by the vision system.
  • a timing signal tells the vision system to retrieve the current image data from the sensor and to retrieve the current position data from the robot controller.
  • the first instance of this timing signal does not occur until after the first set of image data has been processed and is available to the vision system. Since there are two frame times between the laser strobe and this first timing signal, the available position data no longer matches the available image data. In other words, the position data which is retreived in response to the first timing signal corresponds to the image data retrieved in response to the third timing signal. As a result, the first two sets of image data are discarded, and synchronization begins with the third set of image data.
  • the position of the sensor is directly reported to the vision system 40 by an independent measurement system as shown in Figure 5 .
  • one or more photogrammetric cameras are positioned at convenient vantage points in the gauging station.
  • Each photogrammetric camera operates under conventional photogrammetry techniques. That is, the photogrammetric camera emits a burst of quasi-monochromatic light and its camera detects the reflection of this light.
  • retroreflective markers may be placed on the surface of the non-contact sensor 20. When the markers are illuminated by infrared light from a camera, light is reflected by the markers, as bright light spots, back to a motion capture unit within the camera. The light spots are analyzed, such that their position and sizes are calculated by the camera, thereby determining the position of the markers. It is envisioned that the photogrammtric camera may operate using other conventional operational techniques, including but not limited to tracking of luminous targets.
  • the photogrammetry system will provide an accurate determination of the position of the sensor. In this way, the actual position of the sensor is reported to the vision system 40. It should be noted that since the photogrammetry system 80 reports the actual position of the sensor, according to the invention there is no need for the kinematic correction module.
  • the robot-based gauging system 10 can otherwise operate as previously discussed in relation to Figures 1-4 .
  • the present invention provides a significant advance in the art of robot-based gauging systems.
  • the invention accurately determines three-dimensional measurement data for a workpiece through the use of a synchronized scanning process and without the need for expensive sensor positioning devices, such as CMMs.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Length Measuring Devices By Optical Means (AREA)
  • Image Processing (AREA)
  • Length Measuring Devices With Unspecified Measuring Means (AREA)
  • Manipulator (AREA)

Claims (9)

  1. Système robotisé d'évaluation pour déterminer des données de mesure tridimensionnelles pour la surface d'un objet (12), comprenant :
    un robot (24) ayant un bras de robot (22) ;
    un dispositif de commande (38) pour déplacer ledit bras de robot (22) par rapport à la surface de l'objet (12) ;
    un capteur sans contact (20) disposé sur ledit bras de robot (22) pour recueillir des données d'image représentatives d'une partie de la surface de l'objet (12) ;
    un dispositif de rapport de position (80) pour rapporter des données de position représentatives de la position dudit capteur sans contact (20) ;
    un générateur de signal de synchronisation (44) pour générer et transmettre périodiquement un signal de synchronisation, ledit dispositif de rapport de position (80) étant électriquement connecté audit générateur de signal de synchronisation (44) et étant configuré pour rapporter des données de position en réponse à chacun desdits signaux de synchronisation, et ledit capteur sans contact (20) étant électriquement connecté audit générateur de signal de synchronisation (44) et configuré pour recueillir des données d'image en réponse à chacun desdits signaux de synchronisation ;
    un dispositif de traitement de données (40) adapté pour récupérer des données d'image à partir dudit capteur sans contact (20) et des données de position à partir dudit dispositif de rapport de position (80), et pouvant être utilisé pour synchroniser les données d'image avec les données de position, générant ainsi une représentation tridimensionnelle de l'objet (12) ;
    ledit dispositif de rapport de position (80) étant configuré comme un système de mesure photogrammétrique qui émet un faisceau de lumière quasi-monochrome, détecte des points lumineux résultant de la réflexion du faisceau de lumière depuis des marqueurs rétroréfléchissants placés sur la surface du capteur sans contact (20), et détecte la position des marqueurs.
  2. Système robotisé d'évaluation selon la revendication 1, dans lequel ledit bras de robot (22) balaye la surface de l'objet (12), de telle sorte que les données d'image rapportées par ledit capteur sans contact (20) sont ultérieures aux données de position rapportées par ledit dispositif de rapport de position (80).
  3. Système robotisé d'évaluation selon la revendication 1 ou 2, comprenant en outre un espace de stockage de mémoire (62, 64, 66) pour stocker les données de position dudit dispositif de rapport de position (80), ledit dispositif de traitement de données (40) étant connecté audit espace de stockage de mémoire (62, 64, 66) pour synchroniser les données d'image avec les données de position correspondantes.
  4. Système robotisé d'évaluation selon l'une quelconque des revendications 1 à 3, dans lequel ledit capteur sans contact (20) est un capteur de lumière bidimensionnel structurel (20).
  5. Système robotisé d'évaluation selon l'une quelconque des revendications 1 à 4, dans lequel ledit capteur sans contact (20) émet une lumière structurée dans une configuration planaire prédéfinie et ayant un récepteur optique pour recevoir la lumière réfléchie émise par ledit capteur sans contact (20), dans lequel la lumière réfléchie est indicative de la surface de l'objet (12).
  6. Système robotisé d'évaluation selon l'une quelconque des revendications 1 à 5, dans lequel
    ledit dispositif de commande est configuré comme un dispositif de commande de robot (38) pour commander les mouvements dudit bras de robot (22) ;
    ledit générateur de signal de synchronisation (44) est électriquement connecté audit dispositif de commande de robot (38) pour générer et transmettre périodiquement ledit signal de synchronisation, ledit dispositif de commande de robot (38) rapportant un ensemble desdites données de position en réponse à chacun desdits signaux de synchronisation, générant ainsi une pluralité d'ensembles de données de position ;
    ledit capteur sans contact (20) répond à chacun desdits signaux de synchronisation pour capturer un ensemble de données d'image représentatives d'une partie dudit objet (12), générant ainsi une pluralité correspondante d'ensembles de données d'image représentatives d'une pluralité de parties de l'objet (12) ; et
    ledit dispositif de traitement de données est configuré comme un système de vision (40) adapté pour récupérer ladite pluralité d'ensembles de données d'image à partir dudit capteur sans contact (20) et pour synchroniser chaque ensemble de données d'image avec un ensemble de données de position correspondant, générant ainsi une représentation tridimensionnelle de l'objet (12).
  7. Système robotisé d'évaluation selon la revendication 6, comprenant en outre un système de transformation de coordonnée pour transformer chaque ensemble de données de position en une trame de référence de capteur, dans lequel la trame de référence de capteur est associée audit capteur sans contact (20).
  8. Système robotisé d'évaluation selon la revendication 6 ou 7, dans lequel ledit dispositif de commande de robot (38) est configuré pour rapporter des données de position représentatives de la position dudit bras de robot (22).
  9. Système robotisé d'évaluation selon la revendication 6 ou 7, dans lequel ledit système de mesure photogrammétrique (80) a un champ d'observation d'étalonnage et peut être positionné à un point d'observation de telle sorte que ledit capteur sans contact (20) soit dans le champ d'étalonnage, et dans lequel ledit système de mesure photogrammétrique (80) se connecte électriquement audit générateur de signal de synchronisation (44) et rapporte un ensemble de données de position en réponse à chacun desdits signaux de synchronisation, générant ainsi une pluralité d'ensembles de données de position.
EP00117126A 1999-08-12 2000-08-10 Un robot avec système de jauge pour déterminer des mesures de données à 3 dimensions Expired - Lifetime EP1076221B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US09/372,871 US6166811A (en) 1999-08-12 1999-08-12 Robot-based gauging system for determining three-dimensional measurement data
US372871 1999-08-12

Publications (3)

Publication Number Publication Date
EP1076221A2 EP1076221A2 (fr) 2001-02-14
EP1076221A3 EP1076221A3 (fr) 2003-03-26
EP1076221B1 true EP1076221B1 (fr) 2010-04-14

Family

ID=23469960

Family Applications (1)

Application Number Title Priority Date Filing Date
EP00117126A Expired - Lifetime EP1076221B1 (fr) 1999-08-12 2000-08-10 Un robot avec système de jauge pour déterminer des mesures de données à 3 dimensions

Country Status (5)

Country Link
US (1) US6166811A (fr)
EP (1) EP1076221B1 (fr)
CA (1) CA2315508C (fr)
DE (1) DE60044170D1 (fr)
ES (1) ES2340462T3 (fr)

Families Citing this family (101)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6705526B1 (en) 1995-12-18 2004-03-16 Metrologic Instruments, Inc. Automated method of and system for dimensioning objects transported through a work environment using contour tracing, vertice detection, corner point detection, and corner point reduction methods on two-dimensional range data maps captured by an amplitude modulated laser scanning beam
US20020014533A1 (en) 1995-12-18 2002-02-07 Xiaxun Zhu Automated object dimensioning system employing contour tracing, vertice detection, and forner point detection and reduction methods on 2-d range data maps
US6662083B2 (en) * 2000-10-31 2003-12-09 Progressive Tool & Industries Co. Multiple robotic workstation with multiple fixtures
US6567162B2 (en) 2001-05-31 2003-05-20 The Regents Of The University Of Michigan Reconfigurable apparatus and method for inspection during a manufacturing process
DE10159574B9 (de) * 2001-10-15 2009-04-30 Tropf, Hermann, Dr.-Ing. Vorrichtung und Verfahren zur Korrektur der Bewegung von Greif- und Bearbeitungswerkzeugen
US7344082B2 (en) * 2002-01-02 2008-03-18 Metrologic Instruments, Inc. Automated method of and system for dimensioning objects over a conveyor belt structure by applying contouring tracing, vertice detection, corner point detection, and corner point reduction methods to two-dimensional range data maps of the space above the conveyor belt captured by an amplitude modulated laser scanning beam
FR2846413B1 (fr) * 2002-10-24 2005-03-11 Luc Vergnaud Dispositif de numerisation sans contact d'un objet en 3 dimensions
US7024032B2 (en) * 2002-10-31 2006-04-04 Perceptron, Inc. Method for assessing fit and alignment of a manufactured part
US20030137673A1 (en) * 2002-12-13 2003-07-24 Cox Cary B. Systems, and methods of use, employing distorted patterns to ascertain the shape of a surface, for road or runway profiling, or as input to control pro-active suspension systems
DE10311247B8 (de) * 2003-03-14 2008-05-08 Inos Automationssoftware Gmbh Portable Einrichtung zum Erfassen einer Lage und von Abmessungen eines Objekts
KR20060015557A (ko) * 2003-04-28 2006-02-17 스티븐 제임스 크램톤 외골격을 구비한 ｃｍｍ 암
DE10335472A1 (de) * 2003-08-02 2005-02-17 E. Zoller GmbH & Co. KG Einstell- und Messgeräte Mess- und/oder Einstellgerät
EP1524494A1 (fr) * 2003-10-17 2005-04-20 inos Automationssoftware GmbH Procédé d'étalonnage d'une unité caméra-laser par rapport à un objet de calibration
DE10354078B4 (de) * 2003-11-19 2008-09-04 Daimler Ag Spannvorrichtung für Werkstücke zur dreidimensionalen optischen Oberflächenmessung
US7693325B2 (en) 2004-01-14 2010-04-06 Hexagon Metrology, Inc. Transprojection of geometry data
DE102004024378B4 (de) * 2004-05-17 2009-05-20 Kuka Roboter Gmbh Verfahren zur robotergestützten Vermessung von Objekten
FR2873205B1 (fr) * 2004-07-19 2007-05-18 Plastic Omnium Cie Procede et poste de controle de peinture de piece de carrosserie de vehicules automobiles
US7289216B2 (en) * 2004-07-19 2007-10-30 Compagnie Plastic Omnium Station for inspecting the painting of motor vehicle parts
FR2873204B1 (fr) * 2004-07-19 2007-04-27 Plastic Omnium Cie Appareil de mesure pour le controle de pieces de carrosserie peintes, muni d'un dispositif anti-deterioration
DE102004046752B4 (de) * 2004-09-24 2010-10-21 GOM, Gesellschaft für optische Meßtechnik mit beschränkter Haftung Verfahren zur dreidimensionalen Erfassung von Messobjekten
AU2006225115B2 (en) * 2005-03-16 2011-10-06 Lucasfilm Entertainment Company Ltd. Three- dimensional motion capture
US8111904B2 (en) 2005-10-07 2012-02-07 Cognex Technology And Investment Corp. Methods and apparatus for practical 3D vision system
US7743524B2 (en) 2006-11-20 2010-06-29 Hexagon Metrology Ab Coordinate measurement machine with improved joint
US8542236B2 (en) * 2007-01-16 2013-09-24 Lucasfilm Entertainment Company Ltd. Generating animation libraries
US8130225B2 (en) 2007-01-16 2012-03-06 Lucasfilm Entertainment Company Ltd. Using animation libraries for object identification
US8199152B2 (en) * 2007-01-16 2012-06-12 Lucasfilm Entertainment Company Ltd. Combining multiple session content for animation libraries
US8126260B2 (en) * 2007-05-29 2012-02-28 Cognex Corporation System and method for locating a three-dimensional object using machine vision
US8224121B2 (en) * 2007-06-15 2012-07-17 The Boeing Company System and method for assembling substantially distortion-free images
US7743660B2 (en) * 2007-06-15 2010-06-29 The Boeing Company System and method for automated inspection of large-scale part
US8144153B1 (en) 2007-11-20 2012-03-27 Lucasfilm Entertainment Company Ltd. Model production for animation libraries
US7779548B2 (en) 2008-03-28 2010-08-24 Hexagon Metrology, Inc. Coordinate measuring machine with rotatable grip
US8122610B2 (en) * 2008-03-28 2012-02-28 Hexagon Metrology, Inc. Systems and methods for improved coordination acquisition member comprising calibration information
US20090326712A1 (en) * 2008-06-26 2009-12-31 Utica Enterprises, Inc. Calibration for vehicle body assembly
US8923602B2 (en) * 2008-07-22 2014-12-30 Comau, Inc. Automated guidance and recognition system and method of the same
DE102008063680A1 (de) * 2008-10-10 2010-04-15 Abb Ag Verfahren zum Einlernen (Teachen) eines Industrieroboters sowie ein entsprechend ausgestatteter Industrieroboter
US7908757B2 (en) 2008-10-16 2011-03-22 Hexagon Metrology, Inc. Articulating measuring arm with laser scanner
US9734419B1 (en) 2008-12-30 2017-08-15 Cognex Corporation System and method for validating camera calibration in a vision system
US9142024B2 (en) * 2008-12-31 2015-09-22 Lucasfilm Entertainment Company Ltd. Visual and physical motion sensing for three-dimensional motion capture
DE102009015920B4 (de) 2009-03-25 2014-11-20 Faro Technologies, Inc. Vorrichtung zum optischen Abtasten und Vermessen einer Umgebung
US9551575B2 (en) 2009-03-25 2017-01-24 Faro Technologies, Inc. Laser scanner having a multi-color light source and real-time color receiver
US9533418B2 (en) 2009-05-29 2017-01-03 Cognex Corporation Methods and apparatus for practical 3D vision system
AU2010273749B2 (en) 2009-06-30 2015-04-02 Hexagon Metrology Ab Coordinate measurement machine with vibration detection
US8151477B2 (en) 2009-11-06 2012-04-10 Hexagon Metrology Ab CMM with modular functionality
US9210288B2 (en) 2009-11-20 2015-12-08 Faro Technologies, Inc. Three-dimensional scanner with dichroic beam splitters to capture a variety of signals
US9529083B2 (en) 2009-11-20 2016-12-27 Faro Technologies, Inc. Three-dimensional scanner with enhanced spectroscopic energy detector
US9113023B2 (en) 2009-11-20 2015-08-18 Faro Technologies, Inc. Three-dimensional scanner with spectroscopic energy detector
DE102009057101A1 (de) 2009-11-20 2011-05-26 Faro Technologies, Inc., Lake Mary Vorrichtung zum optischen Abtasten und Vermessen einer Umgebung
US20110213247A1 (en) * 2010-01-08 2011-09-01 Hexagon Metrology, Inc. Articulated arm with imaging device
US8630314B2 (en) * 2010-01-11 2014-01-14 Faro Technologies, Inc. Method and apparatus for synchronizing measurements taken by multiple metrology devices
US9628775B2 (en) 2010-01-20 2017-04-18 Faro Technologies, Inc. Articulated arm coordinate measurement machine having a 2D camera and method of obtaining 3D representations
US8875409B2 (en) 2010-01-20 2014-11-04 Faro Technologies, Inc. Coordinate measurement machines with removable accessories
US8832954B2 (en) 2010-01-20 2014-09-16 Faro Technologies, Inc. Coordinate measurement machines with removable accessories
US8677643B2 (en) 2010-01-20 2014-03-25 Faro Technologies, Inc. Coordinate measurement machines with removable accessories
US8898919B2 (en) 2010-01-20 2014-12-02 Faro Technologies, Inc. Coordinate measurement machine with distance meter used to establish frame of reference
GB2489837A (en) * 2010-01-20 2012-10-10 Faro Tech Inc Portable articulated arm coordinate measuring machine and integrated environmental recorder
CN102713776B (zh) * 2010-01-20 2015-04-22 法罗技术股份有限公司 利用了多总线臂技术的便携式铰接臂坐标测量机
US9607239B2 (en) 2010-01-20 2017-03-28 Faro Technologies, Inc. Articulated arm coordinate measurement machine having a 2D camera and method of obtaining 3D representations
US8615893B2 (en) 2010-01-20 2013-12-31 Faro Technologies, Inc. Portable articulated arm coordinate measuring machine having integrated software controls
US9163922B2 (en) 2010-01-20 2015-10-20 Faro Technologies, Inc. Coordinate measurement machine with distance meter and camera to determine dimensions within camera images
US9879976B2 (en) 2010-01-20 2018-01-30 Faro Technologies, Inc. Articulated arm coordinate measurement machine that uses a 2D camera to determine 3D coordinates of smoothly continuous edge features
GB2489370B (en) 2010-01-20 2014-05-14 Faro Tech Inc Coordinate measuring machine having an illuminated probe end and method of operation
USD643319S1 (en) 2010-03-29 2011-08-16 Hexagon Metrology Ab Portable coordinate measurement machine
DE102010018979A1 (de) * 2010-05-03 2011-11-03 Steinbichler Optotechnik Gmbh Verfahren und Vorrichtung zum Bestimmen der 3D-Koordinaten eines Objekts
DE102010020925B4 (de) 2010-05-10 2014-02-27 Faro Technologies, Inc. Verfahren zum optischen Abtasten und Vermessen einer Umgebung
US9393694B2 (en) 2010-05-14 2016-07-19 Cognex Corporation System and method for robust calibration between a machine vision system and a robot
US8127458B1 (en) 2010-08-31 2012-03-06 Hexagon Metrology, Inc. Mounting apparatus for articulated arm laser scanner
DE112011102995B4 (de) 2010-09-08 2016-05-19 Faro Technologies Inc. Laserscanner oder Lasernachführungsgerät mit einem Projektor
US9168654B2 (en) 2010-11-16 2015-10-27 Faro Technologies, Inc. Coordinate measuring machines with dual layer arm
US9124873B2 (en) 2010-12-08 2015-09-01 Cognex Corporation System and method for finding correspondence between cameras in a three-dimensional vision system
USD668656S1 (en) * 2011-01-24 2012-10-09 Datalogic ADC, Inc. Tunnel scanner
US8948447B2 (en) 2011-07-12 2015-02-03 Lucasfilm Entertainment Companyy, Ltd. Scale independent tracking pattern
WO2013074926A1 (fr) 2011-11-18 2013-05-23 Lucasfilm Entertainment Company Ltd. Commande de personnage basée sur un trajet et une vitesse
US8763267B2 (en) 2012-01-20 2014-07-01 Hexagon Technology Center Gmbh Locking counterbalance for a CMM
DE102012100609A1 (de) 2012-01-25 2013-07-25 Faro Technologies, Inc. Vorrichtung zum optischen Abtasten und Vermessen einer Umgebung
US9069355B2 (en) 2012-06-08 2015-06-30 Hexagon Technology Center Gmbh System and method for a wireless feature pack
US8997362B2 (en) 2012-07-17 2015-04-07 Faro Technologies, Inc. Portable articulated arm coordinate measuring machine with optical communications bus
EP2705935A1 (fr) 2012-09-11 2014-03-12 Hexagon Technology Center GmbH Machine de mesure de coordonnées
US9513107B2 (en) 2012-10-05 2016-12-06 Faro Technologies, Inc. Registration calculation between three-dimensional (3D) scans based on two-dimensional (2D) scan data from a 3D scanner
DE102012109481A1 (de) 2012-10-05 2014-04-10 Faro Technologies, Inc. Vorrichtung zum optischen Abtasten und Vermessen einer Umgebung
US10067231B2 (en) 2012-10-05 2018-09-04 Faro Technologies, Inc. Registration calculation of three-dimensional scanner data performed between scans based on measurements by two-dimensional scanner
US9250214B2 (en) 2013-03-12 2016-02-02 Hexagon Metrology, Inc. CMM with flaw detection system
US20140327746A1 (en) * 2013-05-06 2014-11-06 Iphoton Solutions, Llc Volume reconstruction of an object using a 3d sensor and robotic coordinates
JP6217343B2 (ja) * 2013-11-26 2017-10-25 セントラル硝子株式会社 湾曲板形状検査装置
US9594250B2 (en) 2013-12-18 2017-03-14 Hexagon Metrology, Inc. Ultra-portable coordinate measurement machine
US9163921B2 (en) 2013-12-18 2015-10-20 Hexagon Metrology, Inc. Ultra-portable articulated arm coordinate measurement machine
US9759540B2 (en) 2014-06-11 2017-09-12 Hexagon Metrology, Inc. Articulating CMM probe
US10656617B2 (en) 2014-07-16 2020-05-19 Faro Technologies, Inc. Measurement device for machining center
WO2016044658A1 (fr) 2014-09-19 2016-03-24 Hexagon Metrology, Inc. Machine de mesure de coordonnées portative multi-mode
SI3012695T1 (en) 2014-10-23 2018-01-31 Comau S.P.A. System for monitoring and control of an industrial plant
CN105043315A (zh) * 2015-04-30 2015-11-11 天津菲利科电子技术有限公司 非机械式斗轮机斗臂角度及斗轮相对位置检测系统
DE102015122844A1 (de) 2015-12-27 2017-06-29 Faro Technologies, Inc. 3D-Messvorrichtung mit Batteriepack
US10059478B2 (en) 2016-06-02 2018-08-28 Becton Dickinson Rowa Germany Gmbh Method and device for dividing a blister strip
WO2018009980A1 (fr) 2016-07-15 2018-01-18 Fastbrick Ip Pty Ltd Bras pour le transport de matériaux
EP3485112B1 (fr) 2016-07-15 2021-08-25 Fastbrick IP Pty Ltd Véhicule qui incorpore une machine à poser les briques
US11441899B2 (en) 2017-07-05 2022-09-13 Fastbrick Ip Pty Ltd Real time position and orientation tracker
WO2019033165A1 (fr) 2017-08-17 2019-02-21 Fastbrick Ip Pty Ltd Configuration de système d'interaction
CN111226090B (zh) 2017-08-17 2023-05-23 快砖知识产权私人有限公司 具有改进的横滚角测量的激光跟踪器
WO2019071313A1 (fr) 2017-10-11 2019-04-18 Fastbrick Ip Pty Ltd Machine destinée à transporter des objets et carrousel à plusieurs compartiments destiné à être utilisé avec cette dernière
JP7073785B2 (ja) * 2018-03-05 2022-05-24 オムロン株式会社 画像検査装置、画像検査方法及び画像検査プログラム
IT201800005091A1 (it) 2018-05-04 2019-11-04 "Procedimento per monitorare lo stato di funzionamento di una stazione di lavorazione, relativo sistema di monitoraggio e prodotto informatico"
CN114851246B (zh) * 2022-04-19 2024-06-14 深圳市大族机器人有限公司 机器人性能测试系统与方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4575304A (en) * 1982-04-07 1986-03-11 Hitachi, Ltd. Robot system for recognizing three dimensional shapes
WO1997005449A1 (fr) * 1995-07-26 1997-02-13 Crampton Stephen J Appareil de balayage et procede

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB1591922A (en) * 1976-10-19 1981-07-01 Matsushita Electric Ind Co Ltd Magnetic recording and reproducing system
US4254433A (en) * 1979-06-25 1981-03-03 General Motors Corporation Visual motion tracking system
US4841460A (en) * 1987-09-08 1989-06-20 Perceptron, Inc. Method and apparatus for calibrating a non-contact gauging sensor with respect to an external coordinate system
US5142160A (en) * 1990-08-31 1992-08-25 Frank Storbeck Method and device for determining the volume of an object on a flat background
AT396181B (de) * 1990-12-10 1993-06-25 Sprecher Energie Oesterreich Einrichtung zum erfassen der masse eines gegebenenfalls bewegten gegenstandes
IS1666B (is) * 1991-02-19 1997-11-14 Marel Hf Aðferð og búnaður til ákvörðunar rúmmáls, forms og þyngdar fisks eða annarra hluta
US5319443A (en) * 1991-03-07 1994-06-07 Fanuc Ltd Detected position correcting method
US5400638A (en) * 1992-01-14 1995-03-28 Korea Institute Of Science And Technology Calibration system for compensation of arm length variation of an industrial robot due to peripheral temperature change
US5517311A (en) * 1993-05-25 1996-05-14 Toyota Jidosha Kabushiki Kaisha Optical 3D measuring apparatus used for measuring chamber volume of a cylinder head and chamber volume correcting method for a cylinder head of an engine
US5561526A (en) * 1994-05-26 1996-10-01 Lockheed Missiles & Space Company, Inc. Three-dimensional measurement device and system
US5719678A (en) * 1994-07-26 1998-02-17 Intermec Corporation Volumetric measurement of a parcel using a CCD line scanner and height sensor
CN1101056C (zh) * 1995-06-22 2003-02-05 3Dv系统有限公司 生成距景物距离的图象的方法和装置
US5699161A (en) * 1995-07-26 1997-12-16 Psc, Inc. Method and apparatus for measuring dimensions of objects on a conveyor
US5822486A (en) * 1995-11-02 1998-10-13 General Scanning, Inc. Scanned remote imaging method and system and method of determining optimum design characteristics of a filter for use therein
US5793900A (en) * 1995-12-29 1998-08-11 Stanford University Generating categorical depth maps using passive defocus sensing
US6044170A (en) * 1996-03-21 2000-03-28 Real-Time Geometry Corporation System and method for rapid shape digitizing and adaptive mesh generation
EP0851207B1 (fr) * 1996-12-31 2004-05-12 DATALOGIC S.p.A. Procédé pour la mesure du volume d'un objet par un balayeur laser et un détecteur CCD
JP2001515236A (ja) * 1997-09-04 2001-09-18 ダイナログ インコーポレイテッド ロボット検査システムを較正するための方法
US5910845A (en) * 1997-12-02 1999-06-08 Brown; Thomas Mattingly Peripheral viewing optical scanner for three dimensional surface measurement

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4575304A (en) * 1982-04-07 1986-03-11 Hitachi, Ltd. Robot system for recognizing three dimensional shapes
WO1997005449A1 (fr) * 1995-07-26 1997-02-13 Crampton Stephen J Appareil de balayage et procede

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHEN A.C.; KAK C.H.: "MODELING AND CALIBRATION OF A STRUCTURED LIGHT SCANNER FOR 3-D ROBOT VISION", PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION. RALEIGH, MAR. 30 - APR. 3, 1987, WASHINGTON, IEEE COMP. SOC. PRESS, US, no. 2, 31 March 1987 (1987-03-31), WASHINGTON, IEEE COMP. SOC. PRESS, US, pages 807 - 815, XP002079443 *

Also Published As

Publication number Publication date
CA2315508C (fr) 2008-10-14
CA2315508A1 (fr) 2001-02-12
DE60044170D1 (de) 2010-05-27
EP1076221A2 (fr) 2001-02-14
ES2340462T3 (es) 2010-06-04
US6166811A (en) 2000-12-26
EP1076221A3 (fr) 2003-03-26

Similar Documents

Publication Publication Date Title
EP1076221B1 (fr) Un robot avec système de jauge pour déterminer des mesures de données à 3 dimensions
JP2602812B2 (ja) 三次元物体の位置と姿勢との決定方法と装置
US10267619B2 (en) Three-dimensional coordinate scanner and method of operation
US10089415B2 (en) Three-dimensional coordinate scanner and method of operation
US6460004B2 (en) Method and apparatus for calibrating a non-contact gauging sensor with respect to an external coordinate system
US6064759A (en) Computer aided inspection machine
US6285959B1 (en) Method and apparatus for calibrating a non-contact gauging sensor with respect to an external coordinate system
US20030048459A1 (en) Measurement system and method
US20130293684A1 (en) Three-dimensional coordinate scanner and method of operation
US20140043622A1 (en) System for measuring the position and movement of an object
WO2014149702A1 (fr) Scanneur de coordonnées tridimensionnelles et procédé de mise en œuvre
JP2003505682A (ja) 大きな物体の幾何学的形状を走査するシステム
EP1091186A2 (fr) Méthode et appareil pour étalonner une jauge de mesure sans contact par rapport à un système de coordonées externes
JP2001515236A (ja) ロボット検査システムを較正するための方法
US5363185A (en) Method and apparatus for identifying three-dimensional coordinates and orientation to a robot
US11276198B2 (en) Apparatus for determining dimensional and geometric properties of a measurement object
WO2006114216A1 (fr) Procede et dispositif de balayage d'un objet avec un moyen de balayage sans contact manipule par un robot et un moyen separe de detection de position et d'orientation
Kyle et al. Robot calibration by optical methods
Agin Mounted on the Hand of a Robot

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

RIC1 Information provided on ipc code assigned before grant

Ipc: 7G 01B 7/00 A

Ipc: 7G 01B 11/25 B

17P Request for examination filed

Effective date: 20030823

AKX Designation fees paid

Designated state(s): BE DE ES FR GB IT

17Q First examination report despatched

Effective date: 20080311

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): BE DE ES FR GB IT

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 60044170

Country of ref document: DE

Date of ref document: 20100527

Kind code of ref document: P

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2340462

Country of ref document: ES

Kind code of ref document: T3

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20100826

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20100825

Year of fee payment: 11

Ref country code: FR

Payment date: 20100831

Year of fee payment: 11

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20100825

Year of fee payment: 11

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20100825

Year of fee payment: 11

26N No opposition filed

Effective date: 20110117

BERE Be: lapsed

Owner name: PERCEPTRON, INC.

Effective date: 20110831

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20110810

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20120430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110810

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110810

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110831

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20130605

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110811

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20190828

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 60044170

Country of ref document: DE