US10458784B2 - Method and apparatus for determining at least one of dimensional or geometric characteristics of a measurement object - Google Patents
Method and apparatus for determining at least one of dimensional or geometric characteristics of a measurement object Download PDFInfo
- Publication number
- US10458784B2 US10458784B2 US15/999,119 US201815999119A US10458784B2 US 10458784 B2 US10458784 B2 US 10458784B2 US 201815999119 A US201815999119 A US 201815999119A US 10458784 B2 US10458784 B2 US 10458784B2
- Authority
- US
- United States
- Prior art keywords
- measurement
- arrangement
- determining
- camera
- measurement results
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 238000005259 measurement Methods 0.000 title claims abstract description 416
- 238000000034 method Methods 0.000 title claims description 79
- 230000005855 radiation Effects 0.000 claims abstract description 21
- 230000001419 dependent effect Effects 0.000 claims abstract description 9
- 238000011156 evaluation Methods 0.000 claims description 24
- 239000013598 vector Substances 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 3
- 238000003860 storage Methods 0.000 claims description 2
- 238000012937 correction Methods 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 4
- 238000013213 extrapolation Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000002457 bidirectional effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000005315 distribution function Methods 0.000 description 2
- 238000005286 illumination Methods 0.000 description 2
- 238000012805 post-processing Methods 0.000 description 2
- 239000000523 sample Substances 0.000 description 2
- 238000013459 approach Methods 0.000 description 1
- 238000004422 calculation algorithm Methods 0.000 description 1
- 239000000919 ceramic Substances 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000011065 in-situ storage Methods 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2504—Calibration devices
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/002—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates
- G01B11/005—Measuring arrangements characterised by the use of optical techniques for measuring two or more coordinates coordinate measuring machines
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2513—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with several lines being projected in more than one direction, e.g. grids, patterns
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
- G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
- G01B11/2518—Projection by scanning of the object
Definitions
- the present invention relates to method and an apparatus for determining dimensional and/or geometric characteristics of a measurement object.
- Determining dimensional and/or geometric characteristics of an object is a typical task in industrial metrology.
- geometric and/or dimensional characteristics of an individual workpiece are determined for quality control in industrial production processes in order to check whether the workpiece observes specified tolerances.
- the dimensional and/or geometric characteristics can include distances between individual features of the workpiece, such as the distance between two edges or the diameter of a hole, and also the complex geometric shape of the workpiece.
- 3D scan of a workpiece or a workpiece part is a 3D scan of a workpiece or a workpiece part.
- the 3D scan provides a plurality of spatial coordinates (3D coordinates), which define the position of numerous measurement points on the workpiece relative to a reference coordinate system and which therefore describe the complex shape of the workpiece.
- the dimensional and/or geometric characteristics can be determined on the basis of the measured spatial coordinates.
- a probe element can be used to physically touch the desired measurement points, wherein the spatial coordinates of the measurement points are determined from the respective position of the calibrated probe element relative to the reference coordinate system.
- contactless methods for determining the position of measurement points relative to a reference coordinate system. Some methods are based on triangulation, in which the measurement object is recorded with one or more cameras and the evaluation of the camera images is based on trigonometric relationships. In some of these methods, the measurement object is illuminated with a defined pattern, for example in the case of what is known as fringe projection method.
- fringe projection methods are sensitive with respect to image noise and are not suitable for high-gloss surfaces.
- the method by Woodham is known by the term photometric stereo and is based on the assumption of point-like light sources, situated at infinity, with in each case the same light intensity. This assumption cannot be met in practice, which results in measurement errors when determining the surface normals.
- ACM Transactions on Graphics proposes a computational combination of the measurement results from a fringe projection method and from a photometric stereo method.
- the surface normals of the measurement object can be calculated using the 3D coordinates from the fringe projection method.
- Fringe projection methods result in rather short-wave/high-frequency measurement errors (with respect to the extent of the surface) in the form of apparently random, high-frequency noise.
- Nehab et al. propose to combine by calculation the “good” measurement results of the fringe projection method in the long-wave range with the “good” measurement results of the photometric stereo method in the short-wave/high-frequency range.
- the object surface is alleged to be reconstructed in a more detailed and more accurate manner in this way than by each individual method.
- DE 10 2010 007 396 A1 discloses a method and an apparatus for optically inspecting a measurement object having an at least partially reflective surface in order to determine the local scattering characteristics of the object surface.
- the object surface is successively illuminated with light sources at different positions, similar to the photometric stereo method.
- a camera takes a series of images with the different illuminations.
- an individual light origin region is determined for at least one camera pixel.
- the light origin region represents the spatial distribution of the individual light contributions produced by the individual light sources over the surface of the measurement object on the at least one pixel.
- the scattering characteristics of the surface point recorded by the camera pixel is determined on the basis of the individual light origin region. 3D coordinate measurement and determination of dimensional and/or geometric characteristics of the measurement object which is based thereon is not envisaged in this method.
- an object of the present invention to provide an alternative method and an alternative apparatus that allow for high measurement accuracy in the determination of dimensional and/or geometric characteristics of a measurement object. It is another object of the present invention to provide a method and an arrangement that allow to determine dimensional and/or geometric characteristics, such as shape characteristics, in an efficient manner.
- a method for determining at least one of dimensional or geometric characteristics of a measurement object comprising the steps of: providing a first measurement arrangement for determining 3D coordinates at first selected measurement points of the measurement object, providing a second measurement arrangement for determining surface normals at second selected measurement points of the measurement object, wherein the second measurement arrangement includes a number of light sources and a first camera which is directed to the measurement object, providing a calibration data set representing direction dependent individual radiation patterns of the light sources, determining 3D coordinates at the first selected measurement points using the first measurement arrangement, the 3D coordinates defining first measurement results, determining surface normals at the second selected measurement points using the second measurement arrangement and using the calibration data set and the first measurement results, the surface normals thus determined defining second measurement results, and determining the at least one of dimensional or geometric characteristics of the measurement object using the first and the second measurement results.
- an apparatus for determining at least one of dimensional or geometric characteristics of a measurement object comprising a first measurement arrangement for determining 3D coordinates at first selected measurement points of the measurement object, comprising a second measurement arrangement for determining surface normals at second selected measurement points of the measurement object, wherein the second measurement arrangement includes a number of light sources and a first camera which is directed to the measurement object, and comprising an evaluation and control unit comprising a calibration data set that represents individual direction dependent radiation patterns of the light sources and configured for determining the 3D coordinates at the first selected measurement points using the first measurement arrangement, the 3D coordinates defining first measurement results, for determining the surface normals at the second selected measurement points using the second measurement arrangement and using the calibration data set and the first measurement results, the surface normals thus determined defining second measurement results, for determining the at least one of dimensional or geometric characteristics of the measurement object using the first and second measurement results.
- a computer program product comprising program code stored on a non-transitory storage medium, the program code configured to execute a method for determining at least one of dimensional or geometric characteristics of a measurement object using a first measurement arrangement for determining 3D coordinates at first selected measurement points of the measurement object and using a second measurement arrangement for determining surface normals at second selected measurement points of the measurement object, wherein the second measurement arrangement includes a number of light sources and a first camera which is directed to the measurement object, the method comprising the steps of: providing a calibration data set representing individual radiation patterns of the light sources, determining 3D coordinates at the first selected measurement points using the first measurement arrangement, the 3D coordinates defining first measurement results, determining surface normals at the second selected measurement points using the second measurement arrangement and using the calibration data set and the first measurement results, the surface normals thus determined defining second measurement results, and determining the at least one of dimensional or geometric characteristics of the measurement object using the first and the second measurement results.
- the novel method and apparatus are based on the idea of using the 3D coordinates of the first measurement results already in the step of determining the surface normals in order to avoid, from the beginning, conceptional measurement errors in classical photometric stereo.
- the second measurement results are thus not determined entirely separately from the first measurement results and are subsequently, i.e. after the determination of the surface normals, combined with the first measurement results. Rather, information obtained by way of the first measurement results with respect to the individual measurement object are already incorporated in the determination of the surface normals from the outset.
- the novel method and the novel apparatus use a calibration data set that represents the individual direction-dependent radiation patterns of the light sources in the second measurement arrangement.
- This calibration data set provides a priori knowledge that can advantageously be used for the determination of the surface normals in order to avoid conceptional measurement errors in photometric stereo from the very beginning. Consequently, the second measurement arrangement provides second measurement results with a high measurement accuracy. In particular, this reduces from the outset the long-wave measurement errors that Nehab et al. only correct by a subsequent combination with the 3D coordinates from the first measurement arrangement.
- the higher accuracy of the second measurement arrangement advantageously translates to the measurement accuracy in the determination of the dimensional and/or geometric characteristics.
- the more accurate second measurement results can advantageously be used to correct measurement errors of the first measurement arrangement over the entire frequency range and thus reduce them even further overall.
- the novel method and the novel apparatus make possible a higher detail accuracy over the entire error frequency range due to an efficient use of the first measurement results. The abovementioned object is completely achieved.
- the first measurement results are corrected using the second measurement results in order to obtain third measurement results, wherein the dimensional and/or geometric characteristics of the measurement object are determined using the second and/or third measurement results.
- the characteristics of the measurement object could be determined alone on the basis of the second measurement results which already allows for a high measurement accuracy in the entire frequency range due to the inclusion of the a priori knowledge from the first measurement results.
- the first measurement results i.e. the 3D coordinates at the first selected measurement points, are verified using the now available knowledge from the second measurement results and corrected, if necessary, in order to achieve a measurement accuracy which is further increased.
- the first measurement results can be corrected over the entire frequency range because, in particular, the measurement errors of the second measurement arrangement in the long-wave range are reduced.
- 3D coordinates are determined for the second selected measurement points on the basis of the second measurement results, and the 3D coordinates from the first measurement results and the 3D coordinates from the second measurement results are combined, for example using the least squares error method or another matching algorithm.
- the first selected measurement points and the second selected measurement points can differ from one another.
- the first selected measurement points and the second selected measurement points can in each case be identical measurement points on the measurement object.
- a mixture is possible, such that for some selected measurement points both first measurement results and second measurement results are available, while for other selected measurement points, only a first measurement result or a second measurement result is available. Regardless thereof, the dimensional and/or geometric characteristics of the measurement object can be determined with a very high measurement accuracy and great detail and very efficiently in the preferred refinement.
- continuous regions of the measurement object are determined using the second measurement results, and the first measurement results are corrected only in the continuous regions using the second measurement results.
- Determining a continuous region can be done simply using the second measurement results by defining a tolerance interval for the alignment of the normal vectors of a continuous region. Adjacent measurement points, the surface normals of which are within the tolerance interval, are ascribed to the continuous region. Measurement points, the surface normals of which are outside the tolerance interval, are no longer ascribed to the continuous region. In this refinement, only second measurement results from surface regions which largely have similar surface normals are used for the correction of the first measurement results. The refinement allows for a high measurement accuracy at edges, corners and other abrupt object contours at which a continuity condition assumed for post processing is not fulfilled. Limiting the corrections of the first measurement values to continuous surface regions prevents non-continuous regions, such as workpiece edges, to be “rounded” during post processing.
- the first measurement results are interpolated and/or extrapolated using the second measurement results and the dimensional and/or geometric characteristics of the measurement object are determined using the interpolated and/or extrapolated measurement results.
- this refinement includes a subpixel-accurate reconstruction of the measurement surface using the first and second measurement results.
- the first and the second measurement arrangement each operate at a resolution that is determined by the method used and the components used.
- By interpolation and/or extrapolation of the measurement results obtained with the first measurement arrangement it is possible to computationally attain a higher resolution.
- surface regions of the measurement object at which the first measurement arrangement provides no evaluable 3D coordinates can be “filled” by interpolation and/or extrapolation.
- This refinement here has the advantage that, owing to the second measurement results, additional support points are available which can be used to verify and optimize the computational interpolation and/or extrapolation.
- the refinement advantageously contributes to a high measurement resolution and in particular to data completeness.
- BRDF bidirectional reflectance distribution function
- the second measurement arrangement has a first and at least one second light source, wherein the respective individual light vectors are determined separately for the first and for the at least one second light source, and wherein the individual direction-dependent radiation patterns of the first and of the at least one second light source are normalized using the individual light vectors.
- the second measurement arrangement has only one light source which is positioned at different positions relative to the measurement object in the course of the measurement. This can be done automatically in some exemplary embodiments, one of which will be described further below.
- the second measurement arrangement in contrast has a plurality of light sources which are positioned at different positions relative to the measurement object.
- the individual radiation patterns of the different light sources can vary, in particular as regards the light intensity and/or the focusing of the light cone.
- the radiation patterns of the different light sources are normalized or standardized by calculation. Hereby, measurement errors due to differences in the radiation patterns of the light sources are eliminated or at least reduced.
- the second measurement arrangement has a plurality of differently colored light sources.
- This refinement allows for a faster measurement process by using differently colored light sources to illuminate the measurement object at the same time.
- the evaluation and control unit can separate the light components of the individual light sources from one another in the image of the measurement object on the basis of the different light colors (light wavelengths).
- the method of the present refinement consequently requires fewer image recordings while the measurement accuracy remains the same.
- a first and a second image of a calibration object are taken with the camera, wherein the calibration object in the first image is illuminated with the differently colored light sources, and wherein the calibration object in the second image is illuminated with a white light source.
- the calibration object can be a diffusely reflective body which is positioned at one or more positions within the measurement volume. It is advantageous if the bidirectional reflectance distribution function BRDF of the calibration object is known and/or the calibration object reflects uniformly diffusely over its surface.
- the calibration object can be a ceramic plate or another molded body having a diffusely reflective surface.
- the second measurement arrangement has a housing accommodating the first camera, wherein the light sources are arranged at the housing.
- the refinement has the advantage that the position of these light sources relative to the camera is substantially fixed and therefore independent of any individual placement specified by the user.
- the second measurement arrangement can therefore be put into operation more simply and faster, and a calibration data set can be used over rather long operating periods, at least with respect to the light sources arranged at the housing.
- the second measurement arrangement can have (further) light sources that are positioned separately from the housing. Such a refinement is particularly advantageous for measuring large measurement objects using the novel method and the novel apparatus.
- the first measurement arrangement has a pattern projector and a second camera, which are arranged at a defined position and orientation with respect to one another.
- the first measurement arrangement advantageously operates in accordance with a fringe projection method or a similar method, in which suitable patterns are projected onto the measurement object surface and in which the 3D coordinates are determined on the basis of the trigonometric relationship between the pattern projector and the second camera.
- Such methods allow for fast measurement value recording at a plurality of measurement points. They provide good measurement results even without the correction of the first measurement values explained above, which is why such a first measurement arrangement can be employed highly advantageously in the novel apparatus and with the novel method.
- the first and the second camera are identical.
- the first measurement arrangement and the second measurement arrangement each use their own cameras.
- at least one camera forms part of both measurement arrangements.
- the novel apparatus has an individual camera which is both a part of the first measurement arrangement and a part of the second measurement arrangement.
- other exemplary embodiments of the novel apparatus can include two cameras, one of which is a part of the first and of the second measurement arrangement, while the other camera is merely a part of the first measurement arrangement.
- the use of two cameras as a part of the first measurement arrangement makes possible an (additional) trigonometric evaluation and may therefore be advantageous for some applications.
- this refinement has the advantage that the novel method and the novel apparatus can be realized cost-effectively.
- the second camera has a camera resolution and the pattern projector has a projector resolution that is lower than the camera resolution, wherein the first measurement arrangement limits the camera resolution in dependence on the second measurement results.
- This refinement helps to measure a measurement object within a measurement volume with optimum resolution.
- the measurement resolution of the first measurement arrangement is typically limited by the maximum resolution of the pattern projector, because the technical development in cameras is significantly more advanced and has resulted in higher resolutions than in the case of projectors.
- the resolution of a projected pattern on the measurement object does not depend only on the pattern projector itself, but also on the distance between the pattern projector and the object surface.
- the maximally achievable measurement resolution can vary in dependence on the positioning of the measurement object in the measurement volume.
- an optimum measurement resolution is advantageously determined and used for an individual measurement task. In some exemplary embodiments, this can be done adaptively, with the result that an optimum measurement resolution is used for each measurement object surface and preferably even for each region of the measurement object surface.
- a projection pattern with which the pattern projector illuminates the measurement object is determined in dependence on measurement values of the second measurement arrangement.
- the projection pattern is determined depending on information that is obtained by way of the second measurement arrangement.
- the novel method and the novel apparatus of this refinement can advantageously avoid unfavorable multiple reflections of the projection pattern at inner edges of the measurement object. Multiple reflections of the projection pattern can unfavorably influence in particular the measurement accuracy of the first measurement arrangement, which is efficiently avoided in this refinement.
- the measurement object is first measured with a “standard projection pattern” with the first measurement arrangement.
- the second measurement results are determined with the second measurement arrangement using the first measurement results and the calibration data set, which for the reasons explained above already provides an increased measurement accuracy.
- the object surface may once again be measured with the first measurement arrangement, wherein this second measurement process is performed with an optimized projection pattern that was selected and/or determined in dependence on the second measurement results.
- the measurement object is first measured with the second measurement arrangement, wherein in this case a conventional photometric stereo method is used to obtain preliminary surface normals.
- the object surface is measured with the first measurement arrangement, wherein an optimized projection pattern is selected and/or determined in dependence on the preliminary surface normals.
- the 3D coordinates from the first measurement arrangement may then advantageously be used in order to determine optimized surface normals with the second measurement arrangement.
- the second measurement arrangement can advantageously use the camera images of the first measurement process here, that is to say the camera images from the conventional photometric stereo method, because the new determination of the surface normals may take place by way of calculation. With this refinement, a particularly high measurement accuracy is attained.
- the last-mentioned exemplary embodiments additionally permit a measurement time which is shorter as compared to the first-mentioned exemplary embodiments.
- FIG. 1 shows a schematic illustration of a first exemplary embodiment of the novel apparatus
- FIG. 2 shows a schematic illustration for explaining the mode of function of the second measurement arrangement in the apparatus of FIG. 1 ,
- FIG. 3 shows a simplified illustration of a camera and projector arrangement with additional light sources for the exemplary embodiment in accordance with FIG. 1 ,
- FIG. 4 shows a further exemplary embodiment of the novel apparatus in a schematic illustration
- FIG. 5 shows a flowchart for explaining an exemplary embodiment of the novel method.
- FIG. 1 an exemplary embodiment of the novel apparatus is designated by the reference numeral 10 .
- the apparatus 10 has a first measurement arrangement 12 , which here includes a projector 14 , a camera 16 and an evaluation and control unit 18 .
- the projector 14 and the camera 16 are arranged in a defined and known position and orientation relative to one another.
- the projector 14 and the camera 16 can be arranged in a common device housing to permanently ensure the relative position and orientation.
- the projector 14 and the camera 16 may be arranged separately, wherein the relative position and orientation in that case has to be determined in situ.
- the evaluation and control unit 18 includes a processor 20 , which controls the projector 14 and the image recording of the camera 16 .
- the processor 20 furthermore evaluates the images recorded with the camera 16 .
- the evaluation and control unit 18 is implemented on a commercially available personal computer.
- the evaluation and control unit 18 with processor 20 can be a special computer which is optimized in particular for fast processing of large image data.
- the evaluation and control unit 18 further includes a memory in which a calibration data set 22 is stored.
- the memory may store further data (not illustrated here), such as in particular an executable computer program which is used to implement exemplary embodiments of the novel method.
- the apparatus 10 further includes a second measurement arrangement 24 having a camera and a plurality of light sources 26 .
- camera 16 is also part of the second measurement arrangement 24 .
- Camera 16 in this exemplary embodiment consequently serves for recording first images that are evaluated in the evaluation and control unit 18 as part of the first measurement arrangement 12 .
- the camera 16 also serves for recording second images that are evaluated in the evaluation and control unit 18 as part of the second measurement arrangement 24 .
- the evaluation and control unit 18 may thus be part of the first measurement arrangement 12 and part of the second measurement arrangement 24 .
- the light sources 26 are here illustrated by way of example as LED light sources arranged on an LED panel 28 .
- the LED panel 28 has a central opening through which the camera 16 can take images of a measurement object 30 , while the latter is illuminated by one or more light sources 26 .
- the second measurement arrangement 24 operates according to the photometric stereo method, as is known in principle from the publications by R. J. Woodham and Nehab et al., mentioned at the outset. These publications are therefore incorporated herein reference.
- the second measurement arrangement 24 may include further LED panels (not illustrated here) having further light sources, which are arranged laterally of, above and/or below the measurement object 30 .
- the evaluation and control unit 18 is configured to switch the various light sources 26 on and off in a defined sequence (individually or in defined groups) so as to illuminate the measurement object 30 from different directions.
- the evaluation and control unit 18 is further configured to record images of the measurement object 30 at the different illuminations using camera 16 .
- the evaluation and control unit 18 is further configured to determine surface normals 32 at a plurality of measurement points 34 at the surface of the measurement object 30 using said images.
- the surface normals 32 are (imaginary) vectors which are each orthogonal to the measurement object surface at the respective measurement points 34 . A plurality of such surface normals 32 consequently represents the profile of the surface and consequently the shape of the measurement object 30 .
- the first measurement arrangement 12 in this preferred exemplary embodiment operates in accordance with the principle of fringe projection. In principle, however, the first measurement arrangement could operate in accordance with another method that provides 3D coordinates of the object surface.
- the projector 14 is used here to project a defined pattern, in particular a stripe pattern, onto the surface of the measurement object 30 .
- the evaluation and control unit 18 is configured to record one or more images of the measurement object 30 including the projected pattern using the camera 16 . Since the projector 14 and the camera 16 have a defined and known position and orientation with respect to one another, the evaluation and control unit 18 can determine 3D coordinates of measurement points 36 at the surface of the measurement object 30 on the basis of trigonometric relationships. Instead of a stripe pattern, it is also possible to project other patterns onto the surface of the measurement object 30 using the projector 14 , such as squares, dots, individual lines or combinations thereof.
- the measurement object 30 is arranged in this exemplary embodiment on a rotary table 38 , which can be rotated about an axis 42 by way of a drive 40 . This makes possible the automated measurement of the measurement object 30 from several sides.
- the rotary table is actuated by the evaluation and control unit 18 .
- a calibration body is illustrated at reference numeral 44 .
- Calibration body 44 is used here in order to determine the calibration data set 22 .
- the calibration data set 22 represents the individual direction-dependent radiation patterns 45 of all light sources 26 of the measurement arrangement 24 , as will be explained in more detail below with reference to FIG. 2 .
- calibration data set 22 may include further calibration data in the preferred exemplary embodiments, such as calibration data that represent the internal calibration (imaging errors such as distortion etc.) and the external calibration of the camera 16 relative to the projector 14 .
- the calibration data set 22 may further include calibration data that represent individual characteristics of the projector 14 and/or of the rotary table 38 with the drive 40 .
- the processor 20 of the evaluation and control unit 18 advantageously uses the data from the calibration data set 22 in order to determine dimensional and/or geometric characteristics of the measurement object 30 in accordance with the novel method.
- FIG. 2 illustrates a section from the surface 46 of the measurement object 30 .
- the section could be from the surface of the calibration object 44 , specifically while the apparatus 10 is being calibrated, to determine the calibration data set 22 .
- the image sensor of the camera 16 is illustrated at reference numeral 48 .
- the image sensor 48 has a plurality of camera pixels 50 .
- the camera optical unit of the camera 16 is illustrated in simplified fashion at the reference numeral 52 .
- the camera 16 records light rays that are incident through the camera optical unit 52 and illuminate one or more camera pixels 50 of the image sensor 48 .
- an LED panel 28 which includes a plurality of LEDs 26 a , 26 b , 26 c .
- the LEDs 26 a , 26 b , 26 c may be differently colored LEDs.
- the LED 26 a may emit red light
- the LED 26 b may emit green light
- the LED 26 c may emit blue light.
- LED panels having a plurality of LEDs 26 a - 26 c are advantageous, other exemplary embodiments may include other types of light sources, such as fluorescent tubes.
- Illustrated at reference numeral 54 is here an optional white light source 54 , which may advantageously be used during calibration of the apparatus 10 in order to normalize the differently colored light sources.
- the individual LEDs 26 a - 26 c each generate one or more light ray(s) 58 a , 58 b , 58 c , which is reflected at surface 46 to be incident on the camera pixel 50 .
- Which light rays of the light sources 26 reach which camera pixels 50 by way of a reflection at the surface 46 depends, among other things, on the reflective characteristics of the surface 64 , which is illustrated here by way of a radiation lobe 56 at the measurement point 34 .
- the radiation lobe 56 represents the BRDF of the object surface at the measuring point 34 . Illustrated by way of example is a further light ray 60 a from the LED 26 a , which illuminates a different camera pixel 62 by a further reflection at a different measurement point 34 ′ of the surface 46 .
- Which light sources 26 illuminate which camera pixels 50 , 62 and how strongly over the surface 46 depends on the radiation pattern 45 of the respective light source 26 and in addition on the local inclination of the surface 46 at the measurement points.
- different light rays or light vectors 58 a , 58 b , 58 c (and consequently 66 , 68 ) are produced, which illuminate the camera pixels 50 , 62 of the camera 16 .
- the novel method and the novel apparatus use a calibration data set 22 which represents the individual radiation patterns 45 of the light sources 26 and preferably the individual light vectors 58 a , 58 b , 58 c of the individual light sources 26 for a known surface 46 .
- FIG. 3 shows camera 16 in a preferred exemplary embodiment in a view from the front.
- camera 16 has a camera housing 72 accommodating both the projector 14 and the camera 16 with the camera optical unit 52 .
- a plurality of LEDs 26 a , 26 b are arranged at the camera housing 72 .
- the LEDs 26 a , 26 b are arranged here in the region of the camera optical unit 52 .
- a plurality of LEDs 26 a , 26 b may surround the camera optical unit 52 .
- LEDs 26 and/or other light sources can be arranged in another way at the camera housing 72 or at a retainer that is mechanically coupled to the camera housing 72 .
- the projector 14 produces a stripe pattern 74 , which is reflected by surface 46 of the measurement object 30 to camera 16 .
- Camera 16 consequently records an image of the surface 46 of measurement object 30 with projected stripe pattern 74 .
- the evaluation and control unit 18 can determine 3D coordinates to measurement points 36 at the surface 46 .
- FIG. 4 shows a further exemplary embodiment of the apparatus, designated here with the reference numeral 10 ′. Identical reference signs designate the same elements as before.
- light sources 26 are arranged on two arms 78 , 80 .
- the arm 78 is rotatably mounted at a distance d 1 from the central axis of the workpiece table 38 , which can be rotatable or fixed.
- the arm 80 is arranged at a distance d 2 and is likewise rotatable. Consequently, the arms 78 , 80 can be rotated about the axis 42 , which is indicated by the arrow 82 .
- the light sources 26 can be moved to different positions around the measurement object 30 using the arms 78 , 80 and illuminate the measurement object.
- the respectively current position of the arms 78 , 80 is set by the evaluation and control unit 18 using a suitable drive (not illustrated here).
- the apparatus 10 ′ may have a single arm 78 with light sources 26 . As for the rest, the apparatus 10 ′ may operate in the same way as the apparatus 10 .
- a first step 86 provisional surface normals 32 at selected measurement points 34 of the measurement object 30 are determined using the second measurement arrangement 24 (in short MA) by way of a conventional photometric stereo method.
- step 88 an optimized projection pattern for a measurement with the first measurement arrangement 12 is selected such that multiple reflections at inner edges of the measurement object 30 are avoided as far as possible. Problematic inner edges that produce multiple reflection of the projection pattern may advantageously be identified on the basis of the provisional surface normals 32 . Accordingly, data or information determined in step 86 are/is advantageously used in step 88 , which is indicated by the arrows.
- the projection pattern can be divided into partial patterns that each illuminate only individual surface regions at which no multiple reflection or weakened multiple reflection occurs. Parts of the projection pattern may be masked and/or weakened in dependence on the information from step 86 . In some exemplary embodiments, a series of partial patterns is determined in step 88 , which are projected successively onto the surface of the measurement object 30 .
- step 90 an optimum camera resolution for the first measurement arrangement 12 with respect to the concrete measurement object 30 is determined using the information/data from step 86 .
- the steps 86 to 90 are optional, which is indicated here by way of dashed boxes.
- step 92 optimized 3D coordinates at selected measurement points 36 of the measurement object 30 are determined using the first measurement arrangement 12 .
- the first measurement arrangement 12 here advantageously operates in accordance with the fringe projection method which is known per se.
- the optimized projection patterns from step 88 and an optimum camera resolution in accordance with step 90 are preferably used, which is indicated by arrows.
- step 94 optimized surface normals are then determined with the second measurement arrangement 24 .
- the second measurement arrangement 24 operates in step 94 exactly as in step 86 .
- the optimized 3D coordinates from step 92 and the data from the calibration data set 22 are used to first determine individual light vectors 58 a , 58 b , 58 c and to determine, on that basis, the optimized surface normals 32 .
- the second measurement arrangement 24 in step 94 uses the original camera images from step 86 , i.e. a new image recording can be dispensed with.
- step 96 continuous regions of the measurement object surface are determined on the basis of a threshold value criterion for the optimized surface normals. Sharp edges and corners of the measurement object 30 are identified in this way and “excluded” from the continuous regions.
- step 98 third measurement values are determined by correcting the 3D coordinates from step 92 on the basis of the surface normals from step 94 .
- the correction in step 98 can include minimizing the error squares or another correction criterion for reconciling the 3D coordinates that were determined with the first measurement arrangement 12 and the optimized surface normals that were determined with the second measurement arrangement 24 .
- step 100 the third measurement values from step 98 are supplemented by interpolation and/or extrapolation, wherein the optimized surface normals 32 from step 94 are also advantageously used in this step.
- step 102 characteristics of the measurement object are determined in a known manner, specifically on the basis of the third measurement values from step 100 and/or on the basis of the optimized measurement values from step 94 .
- the novel apparatus and the novel method consequently use a calibration data set 22 that represents the direction-dependent radiation patterns 45 of the light sources 26 used in the second measurement arrangement 24 .
- the surface normals 32 are determined.
- the resulting higher accuracy of the second measurement arrangement 24 is advantageously used to obtain overall a higher measurement accuracy during the determination of dimensional and/or geometric characteristics of the measurement object.
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
Description
Claims (17)
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102017118767.7A DE102017118767B4 (en) | 2017-08-17 | 2017-08-17 | Method and device for determining dimensional and / or geometric properties of a measurement object |
DE102017118767.7 | 2017-08-17 | ||
DE102017118767 | 2017-08-17 |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190056218A1 US20190056218A1 (en) | 2019-02-21 |
US10458784B2 true US10458784B2 (en) | 2019-10-29 |
Family
ID=65235443
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/999,119 Expired - Fee Related US10458784B2 (en) | 2017-08-17 | 2018-08-16 | Method and apparatus for determining at least one of dimensional or geometric characteristics of a measurement object |
Country Status (2)
Country | Link |
---|---|
US (1) | US10458784B2 (en) |
DE (1) | DE102017118767B4 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10679367B2 (en) * | 2018-08-13 | 2020-06-09 | Hand Held Products, Inc. | Methods, systems, and apparatuses for computing dimensions of an object using angular estimates |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102018114479B3 (en) * | 2018-06-15 | 2019-11-14 | Polytec Gmbh | Method for determining the beam path of a measuring beam of an interferometric measuring device and measuring device for interferometric measurement of a measuring object |
US10871366B2 (en) * | 2018-08-16 | 2020-12-22 | Mitutoyo Corporation | Supplementary metrology position coordinates determination system for use with a robot |
US11002529B2 (en) | 2018-08-16 | 2021-05-11 | Mitutoyo Corporation | Robot system with supplementary metrology position determination system |
US11745354B2 (en) | 2018-08-16 | 2023-09-05 | Mitutoyo Corporation | Supplementary metrology position coordinates determination system including an alignment sensor for use with a robot |
US10913156B2 (en) | 2018-09-24 | 2021-02-09 | Mitutoyo Corporation | Robot system with end tool metrology position coordinates determination system |
DE102019103519B4 (en) | 2019-02-12 | 2021-09-16 | Carl Zeiss Industrielle Messtechnik Gmbh | Device for determining dimensional and / or geometric properties of a measurement object |
DE102019208474A1 (en) * | 2019-06-11 | 2020-12-17 | Micro-Epsilon Messtechnik Gmbh & Co. Kg | Method and system for optically measuring an object with a reflective and / or partially reflective surface and a corresponding measuring arrangement |
DE102020127894B4 (en) | 2020-10-22 | 2022-09-22 | Smart Optics Sensortechnik Gmbh | Method and device for the optical three-dimensional measurement of objects |
Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100235253B1 (en) | 1996-01-09 | 1999-12-15 | 최종수 | Error analysis method of common lighting source using photometric stereo method |
KR20030034274A (en) | 2001-09-14 | 2003-05-09 | 박보건 | Generalized photometric stereo for the hybrid reflectance model using an image sequences |
US6587183B1 (en) * | 1998-05-25 | 2003-07-01 | Matsushita Electric Industrial Co., Ltd. | Range finder and camera |
US6590669B1 (en) | 1999-04-30 | 2003-07-08 | Christoph Wagner | Method for optically detecting the shape of objects |
US20060001543A1 (en) * | 2004-07-01 | 2006-01-05 | Ramesh Raskar | Interactive wireless tag location and identification system |
US20080137101A1 (en) | 2004-11-03 | 2008-06-12 | Heriot-Watt University | Apparatus and Method for Obtaining Surface Texture Information |
US20080232679A1 (en) | 2005-08-17 | 2008-09-25 | Hahn Daniel V | Apparatus and Method for 3-Dimensional Scanning of an Object |
US7532333B2 (en) * | 2004-04-23 | 2009-05-12 | 3D-Shape Gmbh | Method and apparatus for determining the shape and the local surface normals of specular surfaces |
US20090297020A1 (en) * | 2008-05-29 | 2009-12-03 | Beardsley Paul A | Method and system for determining poses of semi-specular objects |
US7876455B2 (en) * | 2004-08-03 | 2011-01-25 | TechnoDream21 Co., Ltd. | Three-dimensional shape measuring method and apparatus for the same |
DE102010007396A1 (en) | 2010-02-03 | 2011-08-04 | Carl Zeiss OIM GmbH, 73117 | Method and device for optically inspecting a device under test with an at least partially reflecting surface |
US20120320157A1 (en) | 2011-06-14 | 2012-12-20 | Microsoft Corporation | Combined lighting, projection, and image capture without video feedback |
US20130135450A1 (en) * | 2010-06-23 | 2013-05-30 | The Trustees Of Dartmouth College | 3d Scanning Laser Systems And Methods For Determining Surface Geometry Of An Immersed Object In A Transparent Cylindrical Glass Tank |
US20140028801A1 (en) * | 2012-07-30 | 2014-01-30 | Canon Kabushiki Kaisha | Multispectral Binary Coded Projection |
US20140168379A1 (en) * | 2012-12-14 | 2014-06-19 | Faro Technologies, Inc. | Device for optically scanning and measuring an environment |
US20150077764A1 (en) * | 2008-07-08 | 2015-03-19 | Chiaro Technologies LLC | Multiple channel locating |
US20150253428A1 (en) * | 2013-03-15 | 2015-09-10 | Leap Motion, Inc. | Determining positional information for an object in space |
US20160156899A1 (en) * | 2014-12-01 | 2016-06-02 | Canon Kabushiki Kaisha | Three-dimensional measurement apparatus and control method for the same |
WO2016127173A1 (en) | 2015-02-06 | 2016-08-11 | The University Of Akron | Optical imaging system and methods thereof |
DE102015203396A1 (en) | 2015-02-25 | 2016-08-25 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method and device for determining the topography of a surface |
US20160261844A1 (en) | 2015-03-06 | 2016-09-08 | Massachusetts Institute Of Technology | Methods and Apparatus for Enhancing Depth Maps with Polarization Cues |
-
2017
- 2017-08-17 DE DE102017118767.7A patent/DE102017118767B4/en active Active
-
2018
- 2018-08-16 US US15/999,119 patent/US10458784B2/en not_active Expired - Fee Related
Patent Citations (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR100235253B1 (en) | 1996-01-09 | 1999-12-15 | 최종수 | Error analysis method of common lighting source using photometric stereo method |
US6587183B1 (en) * | 1998-05-25 | 2003-07-01 | Matsushita Electric Industrial Co., Ltd. | Range finder and camera |
US6590669B1 (en) | 1999-04-30 | 2003-07-08 | Christoph Wagner | Method for optically detecting the shape of objects |
KR20030034274A (en) | 2001-09-14 | 2003-05-09 | 박보건 | Generalized photometric stereo for the hybrid reflectance model using an image sequences |
US7532333B2 (en) * | 2004-04-23 | 2009-05-12 | 3D-Shape Gmbh | Method and apparatus for determining the shape and the local surface normals of specular surfaces |
US20060001543A1 (en) * | 2004-07-01 | 2006-01-05 | Ramesh Raskar | Interactive wireless tag location and identification system |
US7876455B2 (en) * | 2004-08-03 | 2011-01-25 | TechnoDream21 Co., Ltd. | Three-dimensional shape measuring method and apparatus for the same |
US20080137101A1 (en) | 2004-11-03 | 2008-06-12 | Heriot-Watt University | Apparatus and Method for Obtaining Surface Texture Information |
US20080232679A1 (en) | 2005-08-17 | 2008-09-25 | Hahn Daniel V | Apparatus and Method for 3-Dimensional Scanning of an Object |
US20090297020A1 (en) * | 2008-05-29 | 2009-12-03 | Beardsley Paul A | Method and system for determining poses of semi-specular objects |
US20150077764A1 (en) * | 2008-07-08 | 2015-03-19 | Chiaro Technologies LLC | Multiple channel locating |
DE102010007396A1 (en) | 2010-02-03 | 2011-08-04 | Carl Zeiss OIM GmbH, 73117 | Method and device for optically inspecting a device under test with an at least partially reflecting surface |
US20120327295A1 (en) * | 2010-02-03 | 2012-12-27 | Rolf Beck | Method and apparatus for optically inspecting a test specimen having an at least partly reflective surface |
US20130135450A1 (en) * | 2010-06-23 | 2013-05-30 | The Trustees Of Dartmouth College | 3d Scanning Laser Systems And Methods For Determining Surface Geometry Of An Immersed Object In A Transparent Cylindrical Glass Tank |
US20120320157A1 (en) | 2011-06-14 | 2012-12-20 | Microsoft Corporation | Combined lighting, projection, and image capture without video feedback |
US20140028801A1 (en) * | 2012-07-30 | 2014-01-30 | Canon Kabushiki Kaisha | Multispectral Binary Coded Projection |
US20140168379A1 (en) * | 2012-12-14 | 2014-06-19 | Faro Technologies, Inc. | Device for optically scanning and measuring an environment |
US20150253428A1 (en) * | 2013-03-15 | 2015-09-10 | Leap Motion, Inc. | Determining positional information for an object in space |
US20160156899A1 (en) * | 2014-12-01 | 2016-06-02 | Canon Kabushiki Kaisha | Three-dimensional measurement apparatus and control method for the same |
WO2016127173A1 (en) | 2015-02-06 | 2016-08-11 | The University Of Akron | Optical imaging system and methods thereof |
DE102015203396A1 (en) | 2015-02-25 | 2016-08-25 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Method and device for determining the topography of a surface |
US20160261844A1 (en) | 2015-03-06 | 2016-09-08 | Massachusetts Institute Of Technology | Methods and Apparatus for Enhancing Depth Maps with Polarization Cues |
Non-Patent Citations (7)
Title |
---|
Diego Nehab et al.; Efficiently Combining Positions and Normals for Precise 3D Geometry; 2005; 8 pp. |
Jahanzeb Ahmad et al.; Improving photometric stereo through per-pixel light vector calculation; 2013; 12 pp. |
Mohit Gupta et al.; A Practical Approach to 3D Scanning in the Presence of Interreflections, Subsurface Scattering and Defocus; 2013; 24 pp. |
Robert J. Woodham; Photometric method for determining surface orientation from multiple images; 1980; 6 pp. |
Steffen Herbort et al.; 3D range scan enhancement using image-based methods; ISPRS Journal of Photogrammetry and Remote Sensing 84; 2013; pp. 69-84. |
Thomas Papadhimitri et al., Uncalibrated Near-Light Photometric Stereo; 2014; 12 pp. |
Wuyuan Xie et al.; Photometric Stereo with Near Point Lighting: A Solution by Mesh Deformation; 9 pp. |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10679367B2 (en) * | 2018-08-13 | 2020-06-09 | Hand Held Products, Inc. | Methods, systems, and apparatuses for computing dimensions of an object using angular estimates |
Also Published As
Publication number | Publication date |
---|---|
US20190056218A1 (en) | 2019-02-21 |
DE102017118767A1 (en) | 2019-02-21 |
DE102017118767B4 (en) | 2020-10-08 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10458784B2 (en) | Method and apparatus for determining at least one of dimensional or geometric characteristics of a measurement object | |
US11105617B2 (en) | Hybrid light measurement method for measuring three-dimensional profile | |
JP5570126B2 (en) | Method and apparatus for determining the posture of an object | |
US8094322B2 (en) | Method and apparatus for the determination of the 3D coordinates of an object | |
JP5891280B2 (en) | Method and device for optically scanning and measuring the environment | |
US7711182B2 (en) | Method and system for sensing 3D shapes of objects with specular and hybrid specular-diffuse surfaces | |
US20220221270A1 (en) | A calibration method for fringe projection systems based on plane mirrors | |
US8482743B2 (en) | Method and system for optical edge measurement | |
US9544577B2 (en) | Method of capturing three-dimensional (3D) information on a structure | |
JP5133626B2 (en) | Surface reflection characteristic measuring device | |
US20140002610A1 (en) | Real-time 3d shape measurement system | |
Aliaga et al. | A self-calibrating method for photogeometric acquisition of 3D objects | |
JP7413372B2 (en) | Three-dimensional sensor with opposing channels | |
JP2014115109A (en) | Device and method for measuring distance | |
WO2009120073A2 (en) | A dynamically calibrated self referenced three dimensional structured light scanner | |
US20180240241A1 (en) | Three-dimensional imager | |
US10803623B2 (en) | Image processing apparatus | |
JP6351201B2 (en) | Distance measuring device and method | |
WO2022050279A1 (en) | Three-dimensional measurement device | |
US11276198B2 (en) | Apparatus for determining dimensional and geometric properties of a measurement object | |
JP4379626B2 (en) | Three-dimensional shape measuring method and apparatus | |
JP5633719B2 (en) | 3D information measuring apparatus and 3D information measuring method | |
JP5441752B2 (en) | Method and apparatus for estimating a 3D pose of a 3D object in an environment | |
Langmann | Wide area 2D/3D imaging: development, analysis and applications | |
CN107835361B (en) | Imaging method and device based on structured light and mobile terminal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: CARL ZEISS OPTOTECHNIK GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ULMER, FRANZ-GEORG;HOERR, CHRISTIAN;SIGNING DATES FROM 20181104 TO 20181107;REEL/FRAME:047600/0532 Owner name: CARL ZEISS INDUSTRIELLE MESSTECHNIK GMBH, GERMANY Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:CARL ZEISS OPTOTECHNIK GMBH;REEL/FRAME:047600/0788 Effective date: 20181031 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
FEPP | Fee payment procedure |
Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
LAPS | Lapse for failure to pay maintenance fees |
Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STCH | Information on status: patent discontinuation |
Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362 |
|
FP | Lapsed due to failure to pay maintenance fee |
Effective date: 20231029 |