CN112747671A - Three-dimensional detection system and three-dimensional detection method - Google Patents

Three-dimensional detection system and three-dimensional detection method Download PDF

Info

Publication number
CN112747671A
CN112747671A CN202011457266.0A CN202011457266A CN112747671A CN 112747671 A CN112747671 A CN 112747671A CN 202011457266 A CN202011457266 A CN 202011457266A CN 112747671 A CN112747671 A CN 112747671A
Authority
CN
China
Prior art keywords
hole
dimensional
calculator
characteristic parameters
feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202011457266.0A
Other languages
Chinese (zh)
Other versions
CN112747671B (en
Inventor
杜华
董伟超
徐玉凯
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Xianlin Tianyuan 3d Detection Technology Co ltd
Original Assignee
Tenyoun 3d Tianjin Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tenyoun 3d Tianjin Technology Co ltd filed Critical Tenyoun 3d Tianjin Technology Co ltd
Priority to CN202011457266.0A priority Critical patent/CN112747671B/en
Publication of CN112747671A publication Critical patent/CN112747671A/en
Application granted granted Critical
Publication of CN112747671B publication Critical patent/CN112747671B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/141—Control of illumination
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/64—Three-dimensional objects

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The present disclosure relates to a three-dimensional detection system and a three-dimensional detection method, the three-dimensional detection system including: the image acquisition sensor is used for acquiring an object surface image; a two-dimensional hole feature calculator for determining hole two-dimensional feature parameters based on the object surface image; the three-dimensional hole characteristic calculator is used for carrying out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and determining hole three-dimensional characteristic parameters; and the auxiliary function calculator is used for optimizing the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters. According to the technical scheme provided by the embodiment of the disclosure, the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters are determined by identifying the surface image of the object, and the hole three-dimensional characteristic parameters are optimized by using the auxiliary function calculator, so that the hole position can be effectively measured.

Description

Three-dimensional detection system and three-dimensional detection method
Technical Field
The present disclosure relates to the field of three-dimensional scanning technologies, and in particular, to a three-dimensional detection method and a three-dimensional detection system.
Background
An optical three-dimensional detection system is a commonly used rapid three-dimensional measurement system, and optical three-dimensional scanning is gradually becoming a mainstream detection technical means in the field of industrial detection. Industrial optical three-dimensional scanning technologies are mainly classified into two main categories according to scanning modes: fixed three-dimensional scanning and hand-held three-dimensional scanning. The fixed three-dimensional scanning has the advantages of area array scanning, high single-frame scanning efficiency and high measurement precision, and can obtain three-dimensional data of the whole visible area; the disadvantages of the fixed three-dimensional detection system are that the fixed three-dimensional detection system is usually heavy, the use is not convenient and fast, and the manual operation experience is not good. The handheld three-dimensional scanning system has the advantages of flexible and convenient operation, is more suitable for measuring workpieces with various shapes and sizes, and particularly has good adaptability to complex illumination, materials, colors and the like by adopting the handheld three-dimensional detection system of the laser light source.
With the increasing industrial manufacturing level and the increasing demand of quality control, the rapid and accurate measurement technology of various hole sites on a workpiece is gradually called as the focus of attention in the detection field. At present, the method for detecting hole sites by using a three-dimensional scanning device (or system) is generally as follows: and collecting three-dimensional point cloud of the inner surface of the upper hole wall of the workpiece, and fitting the characteristics by using the point cloud and measuring the characteristic dimension. However, when the surface area of the inner wall is small or the diameter of the inner wall is small, it is difficult to obtain sufficient or complete inner wall data through scanning, and the point cloud fitting characteristics fail, so that the hole position cannot be accurately and effectively measured.
Disclosure of Invention
To solve the technical problem or at least partially solve the technical problem, the present disclosure provides a three-dimensional detection system and a three-dimensional detection method.
The present disclosure provides a three-dimensional inspection system, comprising:
the two image acquisition sensors are used for acquiring surface images of the object;
a two-dimensional hole feature calculator for determining hole two-dimensional feature parameters based on the object surface image;
the three-dimensional hole characteristic calculator is used for carrying out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and determining hole three-dimensional characteristic parameters;
and the auxiliary function calculator is used for optimizing the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters.
In some embodiments, the image capture sensors are provided in at least two, the image capture sensors being provided in the same handheld scanning device.
In some embodiments, the handheld scanning device further comprises a fill light device and a pattern projector;
the pattern projector is used for projecting a structured light pattern to the surface of an object;
the light supplementing device is used for illuminating the surface of an object.
In some embodiments, the pattern projector employs a laser light source, an LED light source, or a halogen light source;
the light supplementing device is a global LED lamp.
In some embodiments, the two-dimensional hole feature calculator comprises:
the hole feature classifier is used for identifying the outline of the hole based on the object surface image and dividing the hole into different categories according to the shape of the outline;
an aperture feature extractor for extracting the aperture two-dimensional feature parameters based on a category of an outline of the aperture.
In some embodiments, the hole feature classifier comprises:
a circular hole identifier for fitting an ellipse based on the outline shape of the hole and classifying the hole as an elliptical hole when the fitting error is less than a first threshold;
the square hole recognizer is used for detecting the angular points of the closed contour which is not classified into the round hole, and classifying the hole which has only four angular points in the contour shape of the hole and has the contour between every two adjacent angular points on a straight line as the square hole;
the long round hole recognizer is used for calculating the minimum circumscribed rectangle of the closed contour which is not classified into the round hole and the square hole, determining the length, the width and the circular arc diameter of the reference long round hole, determining the reference circumscribed rectangle, aligning the reference circumscribed rectangle with the minimum circumscribed rectangle, and classifying the hole into the long round hole when the alignment error of the reference long round hole and the corresponding closed contour is smaller than a second threshold value.
In some embodiments, the hole feature extractor comprises:
the circular hole feature extractor is used for extracting hole two-dimensional feature parameters of the elliptical hole, and the hole two-dimensional feature parameters of the elliptical hole comprise an ellipse equation, a long axis direction, a long half axis and a short half axis;
the square hole feature extractor is used for extracting hole two-dimensional feature parameters of the square hole, and the hole two-dimensional feature parameters of the square hole comprise a center point coordinate and coordinates of four vertexes;
the long circular hole feature extractor is used for extracting hole two-dimensional feature parameters of the long circular hole, and the hole two-dimensional feature parameters of the long circular hole comprise a center point coordinate and four vertex coordinates of a minimum circumscribed rectangle.
In some embodiments, the three-dimensional hole feature calculator comprises:
the hole three-dimensional reconstruction calculator is used for performing three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and the binocular vision principle to obtain hole three-dimensional characteristic parameters;
and the verification calculator is used for verifying the hole three-dimensional characteristic parameters by utilizing the space geometric shape characteristics and eliminating wrong hole three-dimensional characteristic parameters.
In some embodiments, the bore three-dimensional reconstruction calculator is specifically configured to:
and carrying out epipolar matching on the surface image of the object acquired by the image acquisition sensor based on an epipolar geometry principle, and obtaining the hole three-dimensional characteristic parameters of the square hole, the round hole or the long round hole by utilizing triangular calculation when the matching relation is met.
In some embodiments, the auxiliary function calculator comprises:
a parameter optimization calculator for optimizing the hole three-dimensional characteristic parameters by using the hole two-dimensional characteristic parameters, the hole three-dimensional characteristic parameters and the weight coefficients based on a minimized reprojection error optimization method;
wherein the weight coefficient represents the weight corresponding to different object surface images.
In some embodiments, the auxiliary function calculator comprises:
the spatial positioning calculator is used for determining an included angle between a vector of origin points of the coordinate systems of the at least two image acquisition sensors pointing to the center of the hole and a normal direction of the hole, and registering spatial parameters of the hole obtained at different moments to the same coordinate system; the spatial parameters comprise the hole two-dimensional characteristic parameters and/or the hole three-dimensional characteristic parameters;
the data optimization calculator is used for determining the weight coefficient of the object surface image at the moment when the object surface image participates in calculation based on the included angle;
wherein the larger the included angle is, the smaller the weight coefficient is.
In some embodiments, the parameter optimization calculator is specifically configured to:
based on the hole two-dimensional characteristic parameter, the hole three-dimensional characteristic parameter, the weight coefficient and the objective function E ═ Sigma Wi(Pi(S)-Fi)2Optimizing the three-dimensional characteristic parameters of the holes;
determining a three-dimensional contour of the hole based on the three-dimensional characteristic parameters of the hole, projecting the three-dimensional contour onto each object surface image at each moment to obtain a contour reference value set { P of the three-dimensional contour on the object surface imagei(S) }, in which PiFor the projected contour reference, P, of the hole on the ith image of the object surfaceiIs a function of S, S is a hole three-dimensional characteristic parameter of the spatial hole; fiRepresenting the two-dimensional feature parameters of the hole determined by the two-dimensional feature calculator; wiRepresents the weight coefficient; and performing optimization calculation on the target function by adopting a nonlinear least square method to obtain the optimized hole three-dimensional characteristic parameters, wherein i is a positive integer.
The present disclosure also provides a three-dimensional detection method implemented by any one of the above three-dimensional detection systems, the three-dimensional detection method including:
the image acquisition sensor acquires an object surface image;
a two-dimensional hole feature calculator determines hole two-dimensional feature parameters based on the object surface image;
the three-dimensional hole characteristic calculator carries out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters to determine the hole three-dimensional characteristic parameters;
and the auxiliary function calculator optimizes the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters.
In some embodiments, the two-dimensional hole feature calculator comprises a hole feature classifier and a hole feature extractor; the two-dimensional hole feature calculator determines hole two-dimensional feature parameters based on the object surface image, including:
the hole feature classifier identifies the outline of the hole based on the object surface image and divides the hole into different categories according to the shape of the outline;
the hole feature extractor extracts the hole two-dimensional feature parameter based on a category of a contour of a hole.
In some embodiments, the three-dimensional bore feature calculator comprises a bore three-dimensional reconstruction calculator and a verification calculator; the three-dimensional hole feature calculator performs three-dimensional reconstruction on the hole based on the hole two-dimensional feature parameters, and determines hole three-dimensional feature parameters, including:
the hole three-dimensional reconstruction calculator carries out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and the binocular vision principle to obtain hole three-dimensional characteristic parameters;
and the verification calculator verifies the hole three-dimensional characteristic parameters by utilizing the space geometric shape characteristics to eliminate wrong hole three-dimensional characteristic parameters.
In some embodiments, the auxiliary function calculator spatial location calculator, data preference calculator, spatial location calculator, and parameter optimization calculator; the auxiliary function calculator optimizes the hole three-dimensional characteristic parameter based on the hole two-dimensional characteristic parameter and the hole three-dimensional characteristic parameter, and includes:
the space positioning calculator determines an included angle between a vector of the origin points of the coordinate systems of the at least two image acquisition sensors pointing to the center of the hole and the normal direction of the hole, and registers space parameters of the hole obtained at different moments to the same coordinate system; the spatial parameters comprise the hole two-dimensional characteristic parameters and/or the hole three-dimensional characteristic parameters;
the data optimization calculator determines a weight coefficient of the object surface image at the moment when the object surface image participates in calculation based on the included angle; the larger the included angle is, the smaller the weight coefficient is;
the parameter optimization calculator optimizes the hole three-dimensional characteristic parameters by adopting the hole two-dimensional characteristic parameters, the hole three-dimensional characteristic parameters and the weight coefficients based on a minimized reprojection error optimization method;
wherein the weight coefficient represents the weight corresponding to different object surface images.
Specifically, the parameter optimization calculator is based on the hole two-dimensional feature parameter, the hole three-dimensional feature parameter, the weight coefficient, and an objective function E ═ Σ Wi(Pi(S)-Fi)2Optimizing the three-dimensional characteristic parameters of the holes;
determining a three-dimensional contour of the hole based on the three-dimensional characteristic parameters of the hole, projecting the three-dimensional contour onto each object surface image at each moment to obtain a contour reference value set { P of the three-dimensional contour on the object surface imagei(S) }, in which PiFor the projected contour reference, P, of the hole on the ith image of the object surfaceiIs a function of S, S is a hole three-dimensional characteristic parameter of the spatial hole; fiRepresenting the two-dimensional feature parameters of the hole determined by the two-dimensional feature calculator; wiRepresents the weight coefficient; and performing optimization calculation on the target function by adopting a nonlinear least square method to obtain the optimized hole three-dimensional characteristic parameters, wherein i is a positive integer.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:
the embodiment of the present disclosure provides a three-dimensional detection system, wherein: the image acquisition sensor is used for acquiring an object surface image; a two-dimensional hole feature calculator for determining hole two-dimensional feature parameters based on the object surface image; the three-dimensional hole characteristic calculator is used for carrying out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and determining hole three-dimensional characteristic parameters; and the auxiliary function calculator is used for optimizing the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters. Therefore, the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters are determined by identifying the surface image of the object, and the hole three-dimensional characteristic parameters are optimized by combining the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters by using the auxiliary function calculator, so that the effective measurement of hole positions can be realized.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments or technical solutions in the prior art of the present disclosure, the drawings used in the description of the embodiments or prior art will be briefly described below, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
Fig. 1 is a schematic structural diagram of a three-dimensional inspection system according to an embodiment of the present disclosure;
FIG. 2 is a schematic structural diagram of another three-dimensional inspection system according to an embodiment of the present disclosure;
FIG. 3 is a schematic structural diagram of another three-dimensional inspection system according to an embodiment of the disclosure;
fig. 4 is a schematic flow chart of a three-dimensional detection method according to an embodiment of the present disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, aspects of the present disclosure will be further described below. It should be noted that the embodiments and features of the embodiments of the present disclosure may be combined with each other without conflict.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in other ways than those described herein; it is to be understood that the embodiments disclosed in the specification are only a few embodiments of the present disclosure, and not all embodiments.
In the related art, the method of measuring the hole site may include a conventional measuring method, a three-dimensional measuring method, and an auxiliary three-dimensional scanning measuring method.
The traditional measuring method adopts traditional tools, such as a ruler, a pen and the like, to carry out manual measurement, and has low measuring efficiency and poor precision.
In the three-coordinate measuring method, auxiliary measurement is performed by combining three-coordinate equipment, wherein the three-coordinate equipment can comprise a traditional three-coordinate machine and a three-coordinate measuring instrument, and can also comprise an optical (such as a light pen) three-coordinate measuring instrument; the principle is to perform dotting measurement on the surface of the feature, and then to use the sampling point to fit the feature to perform the required size detection. Thus, the equipment cost is high; the method belongs to a contact type measurement mode, and is poor in universality; and it is single point type measurement, and measurement efficiency is lower.
In the auxiliary three-dimensional scanning measurement method, an auxiliary such as a standard ball is placed on a hole, and information such as the size and the center coordinate of the hole is obtained through methods such as fitting and intersection operation of a scanned spherical point cloud and a local surface point cloud of a workpiece (hereinafter also referred to as an "object"), so that a measurement result required by hole position measurement is realized. However, various different standard auxiliary parts need to be prepared for hole sites with different specifications and sizes, and the preparation work is complicated; the method is usually used for measuring round holes, and holes with other shapes are difficult to apply; the auxiliary part is influenced by the action of gravity, the use mode of the auxiliary part is limited, the auxiliary part can only be arranged on the upper surface of a workpiece, other angles are difficult to use, or the auxiliary structure is very complex for avoiding the problems of looseness, falling and the like; the method is an indirect measurement method, and the uncertainty of system measurement is increased; and the process is complicated, the efficiency is low and the operator is required to have higher use experience.
In order to solve the above problems, embodiments of the present disclosure provide a three-dimensional detection system and a three-dimensional detection method, so as to implement high-efficiency hole site detection; the three-dimensional detection system solves the problems that a traditional optical three-dimensional detection system is complex in scheme, high in hardware and use cost and low in efficiency when measuring hole sites, enables a real-time high-precision hole site measurement technology based on a multi-view stereoscopic vision principle to be possible, enables traditional three-dimensional scanning equipment to have the capacity of efficiently detecting hole sites, and greatly improves the practicability of the system.
The three-dimensional inspection system and method provided by the embodiments of the present disclosure are exemplified below with reference to fig. 1 to 4.
Fig. 1 is a schematic structural diagram of a three-dimensional inspection system according to an embodiment of the present disclosure. Referring to fig. 1, the three-dimensional inspection system may include: an image acquisition sensor 12 for acquiring an image of a surface of an object; a two-dimensional hole feature calculator 13 for determining a hole two-dimensional feature parameter based on the object surface image; a three-dimensional hole feature calculator 14, configured to perform three-dimensional reconstruction on the hole based on the hole two-dimensional feature parameter, and determine a hole three-dimensional feature parameter; and the auxiliary function calculator 15 is used for optimizing the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters.
Optionally, the three-dimensional inspection system may further comprise at least one pattern projector 11 for projecting a structured light pattern onto the surface of the object; and the number of the image pickup sensors 12 may be one, two, or more.
The structured light pattern is a projection pattern with a geometric figure and is used for carrying out feature identification on the surface of the object to be detected so as to realize feature matching of the surface of the object to be detected. Illustratively, the shape of the projected pattern may be a fringe and/or speckle, for example: the stripes may include sinusoidal stripes or binary stripes, the speckles may include black and white snowflake patterns or patterns similar to the light and dark gaps of the two-dimensional code, and may also include other optical patterns with specific structures, which is not limited by the embodiments of the present disclosure.
Based on this, the image projector 11 can project the structured light pattern to the surface of the object, and the image capturing sensor 12 can implement three-dimensional measurement of the shape of the object by using the structured light three-dimensional scanning principle.
The number of the image capturing sensors 12 may be at least two, and specifically may be two, three or more, so as to capture the surface image of the object at different angles of the object, and form data that can be used for performing three-dimensional reconstruction and two-dimensional feature extraction. Illustratively, when the number of the image capturing sensors 12 is more than two, the object surface image can be captured using any at least two thereof.
Illustratively, rigid connection is adopted between the image acquisition sensors 12, so that the image acquisition sensors are ensured to be not easy to deform and have high data reliability.
The two-dimensional hole feature calculator 13 may extract two-dimensional hole feature parameters based on the surface image of the object acquired by the image acquisition sensor 12, and is an important basis for three-dimensional hole measurement.
The three-dimensional hole feature calculator 14 can perform three-dimensional reconstruction on the hole based on the two-dimensional hole feature calculator 13 to determine the three-dimensional hole feature parameters, and is favorable for realizing real-time high-precision hole position measurement.
The auxiliary function calculator 15 can optimize the hole three-dimensional characteristic parameters based on the data of dynamic multi-frame sampling by combining the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters, thereby improving the precision and robustness of hole position measurement.
The three-dimensional detection system provided by the embodiment of the disclosure can be used for hole site measurement, and in the three-dimensional detection system, the image acquisition sensor 12 can acquire an object surface image; the two-dimensional hole feature calculator 13 may determine a hole two-dimensional feature parameter based on the object surface image; the three-dimensional hole feature calculator 14 may perform three-dimensional reconstruction of the hole based on the hole two-dimensional feature parameters to determine hole three-dimensional feature parameters; the auxiliary function calculator 15 may optimize the hole three-dimensional characteristic parameter based on the hole two-dimensional characteristic parameter and the hole three-dimensional characteristic parameter; because the dotting measurement is not needed, and the auxiliary objects such as standard balls are not needed, when the system is used for hole position measurement, the universality is higher, the measurement efficiency is higher, the scheme is simple, the hardware and the use cost are lower, the measurement precision is higher, and the robustness is better.
Alternatively, when the three-dimensional detection system comprises the pattern projector 11, a three-dimensional reconstruction may be achieved based on the structured light pattern.
In some embodiments, fig. 2 is a schematic structural diagram of another three-dimensional inspection system according to an embodiment of the disclosure. On the basis of fig. 1, with reference to fig. 2, at least two image acquisition sensors 12 and at least one pattern projector 11 are provided in the same handheld scanning device 01.
Wherein, the basic hardware that realizes hole site measurement function is two image acquisition sensors 12, and it can adopt hand-held type scanning equipment 01 to realize, realizes promptly and hand-held type three-dimensional scanning equipment sharing basic hardware, so can need not additionally to increase the hardware architecture, has promoted hand-held type scanning equipment 01 simultaneously by a wide margin and has been used for the hole site measuring practicality.
In addition, in this embodiment, the spatial hole position measurement can be realized only by using at least two image acquisition sensors 12, and the optical scanning system has a simple structure, high portability and low cost.
In some embodiments, with continued reference to fig. 2, the handheld scanning device 01 further comprises at least one light supplementing apparatus 011; the light supplement device 011 is used for illuminating the surface of an object.
The light supplement device 011 is used for illuminating the surface of an object, and can improve the contrast of patterns on two sides of the outline of the hole or assist in identifying other features on the patterns, such as mark points.
It should be noted that fig. 1 and fig. 2 only show 2 image capture sensors 12 and 1 pattern projector 11, and fig. 2 only shows 1 fill light device 011, but neither of them constitutes a limitation on the three-dimensional detection system provided in the embodiment of the present disclosure. In other embodiments, the number of the image capturing sensor 12, the number of the pattern projector 11, and the number of the light supplement device 011 can be set according to the requirement of the three-dimensional detection system, which is not limited in the embodiments of the present disclosure.
In some embodiments, the pattern projector 11 employs a laser light source, an LED light source, or a halogen lamp light source; the light supplement device 011 is a global LED lamp.
The light source of the pattern projector 11 can be flexibly set according to the three-dimensional scanning requirement, and can also be structured light source of other types; similarly, the light supplement device 011 can also be other types of light sources, which is not limited in the embodiments of the present disclosure.
In some embodiments, fig. 3 is a schematic structural diagram of another three-dimensional inspection system according to an embodiment of the present disclosure. On the basis of fig. 1, referring to fig. 3, the two-dimensional hole feature calculator 13 includes: a hole feature classifier 131 for recognizing the outline of the hole based on the object surface image and classifying the hole into different categories according to the shape of the outline; an aperture feature extractor 132 for extracting aperture two-dimensional feature parameters based on the class of the outline of the aperture.
The hole feature classifier 131 may perform image processing on the object surface image, identify or extract a hole contour (herein, a closed contour) in the object surface image, and classify the holes into different categories according to the contour shape, such as round holes, square holes, oblong holes, or other categories of holes.
In some embodiments, with continued reference to fig. 3, the pore feature classifier 131 includes: a circular hole identifier 1311 for fitting an ellipse based on the outline shape of the hole, and classifying the hole as an elliptical hole when a fitting error is smaller than a first threshold; a square hole identifier 1312 for detecting the corner points of the closed contour not classified into the circular hole, and classifying the hole having only four corner points in the contour shape of the hole and having a contour on a straight line between every two adjacent corner points as a square hole; the long round hole identifier 1313 is configured to calculate a minimum circumscribed rectangle that is not classified into the closed contours of the round hole and the square hole, determine the length, the width, and the arc diameter of the reference long round hole, determine the reference circumscribed rectangle, align the reference circumscribed rectangle with the minimum circumscribed rectangle, and classify the hole as a long round hole when an alignment error between the reference long round hole and the corresponding closed contour is smaller than a second threshold.
First, the circular hole identifier 1311 performs ellipse fitting, and if the fitting error is smaller than the first threshold, the corresponding hole is classified as a circular hole.
The rear hole identifier 1312 performs corner detection on the closed contour that is not classified by the circular hole identifier 1311, and if the closed contour has only four corners and the contour between two adjacent corners is on a straight line, the corresponding hole is classified as a square hole.
Finally, the oblong hole identifier 1313 is used to calculate holes that are not identified by the square hole identifier 1312. The method specifically comprises the following steps: calculating a minimum circumscribed rectangle of the unclassified closed contour Pa, wherein the length of the minimum circumscribed rectangle corresponds to the length L of the long round hole, if only two sections of parallel straight edges with approximate equal length coincide with a group of opposite edges of the minimum circumscribed rectangle, counting the length of the straight edges of the long round hole, and marking the length as L0, the width of the long round hole, namely the diameter of the circular arc is L-L0 (a subsequent data optimization calculator ensures that the plane of the hole participating in calculation is approximately vertical to the direction of the optical axis of the camera, namely approximately orthographic projection is carried out on the imaging plane, and at the moment, the shearing distortion in the length/width direction can be ignored). Based on the length L and the arc diameter D, generating an oblong template P0 with the length L, the width and the arc diameters at two sides D, namely generating a reference oblong P0; and (3) aligning the minimum circumscribed rectangle (namely the reference circumscribed rectangle) of the long circular hole template P0 with the minimum circumscribed rectangle of the contour to be classified through affine transformation, and calculating the alignment error of the aligned long circular hole template P0 and the corresponding closed contour Pa. For example, n sampling points may be generated on the long circular hole template P0, n closest points on the corresponding closed contour Pa are found, n sets of corresponding point pairs are generated, an average value of distances between the corresponding points may be used as an alignment error of the long circular hole template P0 and the corresponding closed contour Pa, and n is a positive integer. When the alignment error of the reference oblong hole P0 with its corresponding closed profile Pa is less than a second threshold, then the closed profile Pa is classified as an oblong hole.
The first threshold and the second threshold may be set according to requirements of the three-dimensional detection system, and may be related to measurement accuracy requirements and a specific fitting algorithm adopted, which is not limited in the embodiment of the present disclosure.
Of course, the order of the holes is not limited.
In other embodiments, the hole feature classifier 131 may further include other types of hole identifiers, that is, multiple different types of identifiers (classifiers) may be set according to the hole location detection requirement, which is neither described nor limited in this embodiment of the present disclosure.
The hole feature extractor 132 may extract features of the classified holes, that is, extract feature shape parameters corresponding to the classified holes according to the shape categories to which the holes belong, thereby determining two-dimensional feature parameters of the holes.
In some embodiments, with continued reference to fig. 3, the aperture feature extractor 132 includes: the circular hole feature extractor 1321 is configured to extract hole two-dimensional feature parameters of the elliptical hole, where the hole two-dimensional feature parameters of the elliptical hole include an ellipse equation, a major axis direction, a major half axis, and a minor half axis; the square hole feature extractor 1322 is used for extracting hole two-dimensional feature parameters of the square hole, and the hole two-dimensional feature parameters of the square hole comprise a center point coordinate and coordinates of four vertexes; and the slotted hole feature extractor 1323 is used for extracting hole two-dimensional feature parameters of the slotted hole, wherein the hole two-dimensional feature parameters of the slotted hole comprise a center point coordinate and four vertex coordinates of a minimum circumscribed rectangle.
When the hole is a circular hole, the circular hole feature extraction 1321 extracts elliptical shape parameters, for example, parameters including center coordinates, major/minor semiaxes, major axis direction, an elliptical equation, and the like. When the hole is a quadrangle, that is, the projection shape of the square hole in the image of the object surface is a square, the square hole feature extractor 1322 extracts the shape parameters thereof, which may include parameters such as coordinates of a center point, coordinates of four vertices, and the like. When the hole is a long circular hole, the long circular hole feature extractor 1323 extracts shape parameters of the hole, which may include parameters such as coordinates of a central point, coordinates of four vertexes of a minimum circumscribed rectangle, and the like.
Therefore, the extraction of the two-dimensional characteristic parameters of the holes is realized based on the hole types.
In other embodiments, when the holes are holes of other types, the corresponding characteristic parameters may also be extracted, which is not limited in the embodiments of the present disclosure.
In some embodiments, with continued reference to fig. 3, the three-dimensional hole feature calculator 14 may include: a hole three-dimensional reconstruction calculator 141 which performs three-dimensional reconstruction of holes based on a binocular vision (multi-eye stereoscopic vision) principle to obtain hole three-dimensional characteristic parameters; and the verification calculator 142 is used for verifying the hole three-dimensional characteristic parameters by utilizing the space geometric shape characteristics to eliminate the wrong hole three-dimensional characteristic parameters.
Thus, the three-dimensional hole feature calculator 14 can realize three-dimensional reconstruction and verification of the hole, and can improve the accuracy and robustness of hole location measurement.
In some embodiments, the bore three-dimensional reconstruction calculator 141 is specifically configured to: based on an epipolar geometry principle, the object surface image acquired by the image acquisition sensor is subjected to epipolar matching, and when a matching relation is met, hole three-dimensional characteristic parameters of a square hole, a round hole or a long round hole are obtained through triangular calculation.
For a circular hole, performing epipolar matching on ellipse center coordinates in object surface images (hereinafter, may be simply referred to as "images") acquired by two image acquisition sensors (hereinafter, may be simply referred to as "cameras") by using an epipolar geometry principle, setting an ellipse A and an ellipse B as a set of matched hole pairs in two images obtained by the epipolar matching, sampling m points on the outline of the ellipse A, wherein m is a positive integer, calculating an epipolar equation for each sampling point a0, wherein the intersection point of the epipolar line and the outline of the ellipse B is a candidate matching point a0, and the maximum number of the intersection points is 2. And the condition that the local contour section is superposed with the epipolar line is not calculated, so that the subsequent overall shape fitting is not influenced. Because the imaging planes of the two image acquisition sensors are approximately consistent in the space direction, the relative orientation relation (up-down, left-right) between the corresponding points and the located outline in the two images has invariance; based on the above, the unique corresponding point B0 on the outline of the ellipse B can be obtained by screening according to the orientation of the a0 on the outline of the ellipse A, and the three-dimensional space point p0 is obtained by utilizing trigonometric calculation, namely, one point on the space outline of the round hole is obtained. And similarly, obtaining the spatial coordinates of all m sampling points, fitting a circular equation, namely a three-dimensional profile equation, and obtaining the three-dimensional characteristic parameters of the holes of the round holes to obtain the initial values of the spatial shape parameters of the round holes.
Aiming at the square hole, the center coordinates of the square hole in the images collected by the two cameras are subjected to antipodal matching by using an antipodal geometric principle, then four vertexes of the matching hole pair are subjected to antipodal matching respectively, if the four vertexes can meet the one-to-one corresponding matching relation, the space coordinates of the four vertexes of the square hole are reconstructed by using triangular calculation, and initial values of space shape parameters (center coordinates, side length, direction and the like) of the square hole can be obtained, namely the hole three-dimensional characteristic parameters of the square hole are obtained.
Aiming at the oblong holes, similar to the round holes, matched oblong hole pairs are obtained through center point-to-pole matching, then contour three-dimensional reconstruction is carried out, a minimum circumscribed rectangle of the contour is calculated, the length and the width of the circumscribed rectangle are the length and the width (the diameter of an arc) of the oblong hole, and it can be seen that the shape parameters of the oblong hole are equivalent to the space shape parameters of the minimum circumscribed rectangle, including parameters such as a center coordinate, four-vertex coordinates, side length and direction. Therefore, the hole three-dimensional characteristic parameters of the oblong hole can be obtained.
In other embodiments, when the holes are holes of other types, the hollow three-dimensional characteristic parameters may also be obtained by using the epipolar geometry principle, corresponding to the characteristic shape parameters, which is not limited in the embodiments of the present disclosure.
On the basis, the verification calculator 142 may verify the three-dimensional reconstruction result of the hole by using the spatial geometry feature (the spatial geometry feature of the hole is obtained based on the category of the outline of the hole), and exclude an error result.
Illustratively, for a round hole, when the fitting error of the circle is greater than a third threshold, the round hole is excluded from the result; for the square holes, when four sides are not equal (the maximum deviation of the side length is greater than a fourth threshold), the corresponding square holes are excluded from the result; and fitting straight lines to the straight edges and fitting circles to the circular edges for the long circular holes, and when the fitting error is larger than a fifth threshold value, excluding the corresponding long circular holes from the result. The values of the third threshold, the fourth threshold, and the fifth threshold may be set according to the requirements of the three-dimensional detection system, which is not limited in the embodiment of the present disclosure.
Therefore, the hole site measurement accuracy can be further improved by verifying the three-dimensional reconstruction result.
In some embodiments, with continued reference to fig. 3, the auxiliary function calculator 15 includes: a spatial orientation calculator 151, configured to determine an included angle between a vector in which the origin points of the coordinate systems of the at least two image capturing sensors point to the center of the hole and a normal direction of the hole (a spatial pose of the image capturing sensors can be obtained), and to register spatial parameters of the hole obtained at different times to the same coordinate system; the included angle can be determined by combining the result of three-dimensional reconstruction, and the space parameters comprise hole two-dimensional characteristic parameters and/or hole three-dimensional characteristic parameters; a data preference calculator 152, configured to determine, based on the included angle, a weight coefficient of the object surface image at the time point when the object surface image participates in the calculation; the larger the angle, the smaller the weight factor.
The spatial location calculator 151 may use any location method known to those skilled in the art and known in the three-dimensional scanning technology to register the spatial information (center coordinates, contour, etc.) of the hole in the device coordinate system obtained at different times into the same coordinate system.
For example, three-dimensional reconstruction calculation is performed on feature points (including mark points, geometric features, texture features and the like) on the surface of the workpiece at each moment, and a coordinate transformation matrix is calculated by using three or more feature points with the same name at different moments, that is, the transformation matrix can be used for registering spatial information of holes at different moments to the same coordinate system, which may be an object coordinate system for example. Therefore, splicing of data at different moments can be realized, and a relatively complete station surface data set is obtained.
The data preference calculator 152 is used for determining the weighting coefficients of the data at different time points in the splicing process. In the three-dimensional scanning process, when the included angle between the sight line direction from a camera to a hole (namely a vector of the origin of a camera coordinate system pointing to the center of the hole) and the normal direction of a spatial hole is large, the imaging of the inner wall of the hole may influence the detection precision of the hole outline during image processing, and the smaller the included angle is, the smaller the influence is, the more accurate the measurement result tends to; the larger the included angle, the larger the influence, and the inaccurate measurement result tends to be.
Based on this, the spatial pose (i.e. positioning information) of the image capturing sensor given by the spatial positioning calculator 151 is combined, the included angle between the normal direction of the hole and the sight line direction can be calculated in real time, the measurement data at this moment is evaluated by using the function of the included angle, and the estimated measurement data is used as the weight coefficient participating in the final optimization calculation, so that the larger the included angle, the smaller the weight of the sample data is. The function of the included angle may be an inverse distance weight function, a gaussian weight function, or other functions known to those skilled in the art, which is not limited in the embodiments of the present disclosure. It should be noted that, in this paragraph, the same weighting factor is given to the measurement data at a certain time, and individual measurement points in the measurement data are not distinguished, which is beneficial to simplifying the data processing process while ensuring higher measurement accuracy.
In some embodiments, with continued reference to fig. 3, the auxiliary function calculator 15 further comprises: a parameter optimization calculator 153, wherein the parameter optimization calculator 153 is configured to optimize the hole three-dimensional feature parameter by using the hole two-dimensional feature parameter, the hole three-dimensional feature parameter, and the weight coefficient based on a minimized reprojection error optimization method; wherein the weight coefficient represents the weight corresponding to different object surface images.
The method for optimizing the hole three-dimensional characteristic parameters based on the minimized reprojection error optimization method comprises the following steps: and constructing a minimized reprojection error optimization function based on the hole two-dimensional characteristic parameters, the hole three-dimensional characteristic parameters and the weight coefficients, and performing optimization calculation to obtain optimized hole three-dimensional characteristic parameters.
Wherein, the optimization function for minimizing the reprojection error is an objective function E ═ Sigma Wi(Pi(S)-Fi)2Therefore, the hole three-dimensional characteristic parameters can be optimized based on the hole two-dimensional characteristic parameters, the hole three-dimensional characteristic parameters, the weight coefficients and the objective function; determining a three-dimensional outline of the hole based on the three-dimensional characteristic parameters of the hole, namely determining an outline equation of the hole; combining the spatial pose of the image acquisition sensor, projecting the three-dimensional contour to each object surface image at each moment to acquire a contour reference value set { P } of the three-dimensional contour on the object surface imagei(S) }, in which PiFor the projected contour reference, P, of the hole on the ith object surface imageiIs a function of S, S is a hole three-dimensional characteristic parameter of the spatial hole; fiRepresenting a two-dimensional feature parameter of the hole determined by the two-dimensional feature calculator; wiRepresents a weight coefficient; and (4) performing optimization calculation on the target function by adopting a nonlinear least square method to obtain an optimized hole three-dimensional characteristic parameter, wherein i is a positive integer.
The hole site information (i.e., the hole three-dimensional characteristic parameters) obtained by the three-dimensional hole characteristic calculator 14 is affected by errors caused by single measurement data noise, measurement angles, and the like, and has a problem of insufficient precision and robustness.
In view of the above, by combining the dynamic scanning characteristic of the handheld scanning device and adopting a large number of sampling results of time sequences to optimize and calculate the hole position information, the measurement precision and the robustness can be remarkably improved.
Exemplarily, in combination with the above data stitching process, determining a three-dimensional profile of the hole based on the three-dimensional characteristic parameters of the hole at a plurality of different times; projecting the three-dimensional contour to each camera image at each moment to obtain an estimated value { P ] of the hole contour on the imagei(S) }, also called the set of contour reference values { Pi(S) }; wherein, PiAlso referred to as the estimate of the projected profile of the hole on the ith image, PiIs a function of S, S being a shape parameter (e.g. contour equation) of the spatial hole, the observed value of the hole contour on the image having been obtained by a two-dimensional feature calculator, i.e. Fi. Based on this, the objective function E ═ Σ W is seti(Pi(S)-Fi) 2; wherein, WiFor the weight coefficient determined by the data optimization calculator 152, a nonlinear least square method (e.g., L-M algorithm) is used to perform optimization calculation on the objective function M to obtain the final S, i.e., the shape parameter of the optimized spatial hole, so that more accurate measurement information such as three-dimensional coordinates, aperture, and the like of the spatial hole can be obtained by calculation.
The contour projection is to project a three-dimensional contour of the calculated hole in space onto an image, wherein the three-dimensional contour is a parameterized result fitted by the cooperative calculation of the calculators according to the acquired surface image of the object, and the projection of the three-dimensional contour is deviated from a two-dimensional contour actually detected on the image; on the other hand, the optimization calculation is to project the three-dimensional contour onto all effective images for optimization, and is not to project the reconstruction result of a certain frame back onto two images of the frame, and is not to simply perform the inverse calculation in the reconstruction process.
In the three-dimensional detection system provided by the embodiment of the disclosure, an image acquisition sensor in the handheld scanning device can acquire an image of the surface of an object, which is substantially a 2D image; a hole feature classifier in the two-dimensional hole feature calculator may determine a shape class to which the hole belongs based on the 2D image; a hollow feature extractor in the two-dimensional hole feature calculator can determine the feature parameters of the holes based on the shape types of the holes, namely determine the two-dimensional feature parameters of the holes; the hole three-dimensional reconstruction calculator in the three-dimensional hole feature calculator can perform three-dimensional reconstruction based on the object surface image and the shape feature parameters, determine the hole three-dimensional feature parameters, for example, can determine coordinates of a space central point, an angular point and the like of a hole, and determine a hole contour equation, shape parameters and the like; the checking calculator in the three-dimensional hole characteristic calculator can eliminate the special-shaped result, namely the wrong hole three-dimensional characteristic parameter, based on the shape class and the three-dimensional reconstruction result of the hole; the spatial positioning calculator in the auxiliary function calculator can determine the spatial pose of the handheld scanning equipment (the pose relation between the handheld scanning equipment and the hole plane in a certain image acquisition process) based on the three-dimensional reconstruction result and register the spatial parameters of the holes to an object coordinate system; the data optimization calculator in the auxiliary function calculator can determine a weight coefficient of the measurement data at the moment to participate in final calculation based on the spatial pose of the handheld scanning device; and the parameter optimization calculator in the auxiliary function calculator is used for optimizing the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters (namely characteristic shape parameters), the space pose of the handheld scanning equipment and the three-dimensional reconstruction result registered to the object coordinate system, and obtaining the optimized hole three-dimensional special diagnosis parameters.
The three-dimensional detection system provided by the embodiment of the disclosure can be used for hole site measurement, realizes a real-time high-precision hole site measurement method based on a multi-view stereoscopic vision principle, also enables the traditional handheld three-dimensional scanning equipment to have the capacity of efficiently detecting hole sites, greatly improves the practicability of the system, and solves the problems of complex scheme, high hardware and use cost, low efficiency and the like when the traditional three-dimensional detection system is used for hole site measurement.
The three-dimensional detection system provided by the embodiment of the disclosure has at least the following beneficial effects:
firstly, a hole site detection scheme based on a multi-eye stereoscopic vision principle is realized. The embodiment provides the space hole site detection system which can be realized by using at least two optical acquisition sensors, and the device has the advantages of simple structure, portability, low cost, capability of realizing real-time measurement, high measurement efficiency and high precision.
Second, the system can be integrated with three-dimensional scanning equipment, including handheld three-dimensional scanning equipment, so that the three-dimensional scanning equipment has the capability of rapidly measuring hole sites. Specifically, in the system provided in this embodiment, the basic hardware for implementing the hole site measurement function is at least two optical acquisition sensors (i.e., image acquisition sensors), which can be shared by the three-dimensional scanning device, so that the cost is not increased, but the practicability of the three-dimensional scanning device is greatly improved.
Thirdly, a two-dimensional hole characteristic calculator is arranged, the shape characteristics of the holes can be used for image recognition, the holes are classified, and characteristic property parameters are extracted, so that the two-dimensional hole characteristic calculator is an important basis for hole site three-dimensional measurement.
And fourthly, the three-dimensional hole characteristic calculator is arranged, so that three-dimensional reconstruction and verification of hole positions can be realized, and the accuracy and robustness of hole position measurement are improved.
And fifthly, an auxiliary function calculator is arranged, optimization and statistical optimization calculation can be carried out based on the data of dynamic multi-frame sampling, and the precision and the robustness of hole site measurement are improved.
On the basis of the above embodiments, the embodiments of the present disclosure further provide a three-dimensional detection method, which can be performed by any one of the three-dimensional detection systems in the above embodiments. Therefore, the three-dimensional detection method also has the beneficial effects of the three-dimensional detection system. The same can be understood by referring to the explanation of the three-dimensional detection system above, and the explanation is not repeated below.
In some embodiments, fig. 4 is a schematic flow chart of a three-dimensional detection method according to an embodiment of the disclosure. Referring to fig. 4, the three-dimensional detection method includes:
s201, an image acquisition sensor acquires an object surface image.
S202, the two-dimensional hole feature calculator determines hole two-dimensional feature parameters based on the object surface image.
S203, the three-dimensional hole feature calculator carries out three-dimensional reconstruction on the hole based on the hole two-dimensional feature parameters to determine the hole three-dimensional feature parameters.
And S204, optimizing the hole three-dimensional characteristic parameters by the aid of the auxiliary function calculator based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters.
The three-dimensional detection method provided by the embodiment of the disclosure can be used for hole site measurement, and in the three-dimensional detection method, an image acquisition sensor acquires an object surface image; a two-dimensional hole feature calculator determines hole two-dimensional feature parameters based on the object surface image; the three-dimensional hole characteristic calculator carries out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters to determine the hole three-dimensional characteristic parameters; the auxiliary function calculator optimizes the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters; because the dotting measurement is not needed, and the auxiliary objects such as standard balls are not needed, when the system is used for hole position measurement, the universality is higher, the measurement efficiency is higher, the scheme is simple, the hardware and the use cost are lower, the measurement precision is higher, and the robustness is better.
In some embodiments, which may be combined with FIG. 3, the two-dimensional hole feature calculator includes a hole feature classifier and a hole feature extractor. Based on this, S202 in fig. 4 may include:
the hole feature classifier identifies the outline of the hole based on the object surface image and divides the hole into different categories according to the shape of the outline;
the hole feature extractor extracts a hole two-dimensional feature parameter based on a category of a contour of a hole.
In this way, the two-dimensional hole feature calculator can identify the object surface image, extract the hole contour (here, closed contour) in the object surface image, and divide the holes into different categories according to the contour shape; and extracting the hole two-dimensional characteristic parameters of the classified holes, and providing data support for subsequent hole three-dimensional reconstruction.
In some embodiments, which may be combined with FIG. 3, the three-dimensional bore feature calculator includes a bore three-dimensional reconstruction calculator and a verification calculator. Based on this, S203 in fig. 4 may include:
the hole three-dimensional reconstruction calculator carries out three-dimensional reconstruction on the holes based on the hole two-dimensional characteristic parameters and the binocular vision principle to obtain the hole three-dimensional characteristic parameters;
the checking calculator checks the hole three-dimensional characteristic parameters by using the space geometric shape characteristics, and eliminates wrong hole three-dimensional characteristic parameters.
Therefore, the three-dimensional reconstruction and verification of the hole can be realized through the three-dimensional hole characteristic calculator, and the accuracy and the robustness of hole position measurement can be improved.
In some embodiments, an ancillary function calculator, a spatial location calculator, a data preference calculator, a spatial location calculator, and a parameter optimization calculator may be combined with FIG. 3. Based on this, S204 in fig. 4 may include:
the spatial positioning calculator determines an included angle between a vector of the origin points of the coordinate systems of the at least two image acquisition sensors pointing to the center of the hole and the normal direction of the hole, and registers spatial parameters of the hole obtained at different moments to the same coordinate system; the spatial parameters comprise hole two-dimensional characteristic parameters and/or hole three-dimensional characteristic parameters;
the data optimization calculator determines a weight coefficient of the object surface image at the moment when the object surface image participates in calculation based on the included angle; the larger the included angle is, the smaller the weight coefficient is;
the parameter optimization calculator is based on the hole two-dimensional characteristic parameter, the hole three-dimensional characteristic parameter, the weight coefficient and the objective function E ═ Sigma Wi(Pi(S)-Fi)2Optimizing the hole three-dimensional characteristic parameters;
determining a three-dimensional contour of the hole based on the three-dimensional characteristic parameters of the hole, projecting the three-dimensional contour onto each object surface image at each moment to obtain a contour reference value set { P ] of the three-dimensional contour on the object surface imagei(S) }, in which PiFor the projected contour reference, P, of the hole on the ith object surface imageiIs a function of S, S is a hole three-dimensional characteristic parameter of the spatial hole; fiRepresenting a two-dimensional feature parameter of the hole determined by the two-dimensional feature calculator; wiRepresents a weight coefficient; and (4) performing optimization calculation on the target function by adopting a nonlinear least square method to obtain an optimized hole three-dimensional characteristic parameter, wherein i is a positive integer.
Therefore, the auxiliary function calculator can be used for carrying out optimization and statistical optimization calculation based on the data of dynamic multi-frame sampling, and the precision and the robustness of hole site measurement are improved.
In the three-dimensional detection method provided by the embodiment of the disclosure, feature extraction is performed based on an object surface image, holes are classified according to shapes, and hole two-dimensional shape parameters are obtained, so that a 2D contour is obtained; then, performing three-dimensional reconstruction based on the sampling points on the 2D contour to obtain a single-frame (sheet) three-dimensional point cloud at the moment, and fitting and calculating space shape parameters by using the three-dimensional point cloud to obtain hole three-dimensional shape parameters including the three-dimensional contour of the hole; the fitted space shape parameters can be verified by utilizing the space geometric shapes, and error results in the space geometric shapes are eliminated; on the basis, splicing a plurality of pieces of three-dimensional point cloud; determining the weight coefficient of each frame of three-dimensional point cloud in the corresponding multiple frames of different three-dimensional point clouds based on the sight line direction from the camera to the hole and the normal included angle of the space hole during image acquisition; projecting the three-dimensional contour of the hole to each camera image at each moment to obtain an estimated value of the hole contour on the image, and establishing a function between the estimated value and a two-dimensional hole characteristic calculation value to perform optimization calculation to obtain an optimized three-dimensional shape parameter of the hole; thus, the measurement accuracy and robustness can be improved.
It is noted that, in this document, relational terms such as "first" and "second," and the like, may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or system that comprises the element.
The foregoing are merely exemplary embodiments of the present disclosure, which enable those skilled in the art to understand or practice the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (12)

1. A three-dimensional inspection system, comprising:
the image acquisition sensor is used for acquiring an object surface image;
a two-dimensional hole feature calculator for determining hole two-dimensional feature parameters based on the object surface image;
the three-dimensional hole characteristic calculator is used for carrying out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and determining hole three-dimensional characteristic parameters;
and the auxiliary function calculator is used for optimizing the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters.
2. The three-dimensional inspection system of claim 1, wherein there are at least two image capture sensors, the image capture sensors being disposed in the same hand-held scanning device.
3. The three-dimensional inspection system of claim 2, wherein the hand-held scanning device further comprises a fill-in light device and a pattern projector;
the pattern projector is used for projecting a structured light pattern to the surface of an object;
the light supplementing device is used for illuminating the surface of an object.
4. The three-dimensional inspection system of claim 2, wherein the pattern projector employs a laser light source, an LED light source, or a halogen light source;
the light supplementing device is a global LED lamp.
5. The three-dimensional inspection system of claim 1, wherein the two-dimensional hole feature calculator comprises:
the hole feature classifier is used for identifying the outline of the hole based on the object surface image and dividing the hole into different categories according to the shape of the outline;
an aperture feature extractor for extracting the aperture two-dimensional feature parameters based on a category of an outline of the aperture.
6. The three-dimensional inspection system of claim 1, wherein the three-dimensional hole feature calculator comprises:
the hole three-dimensional reconstruction calculator is used for performing three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and the binocular vision principle to obtain hole three-dimensional characteristic parameters;
and the verification calculator is used for verifying the hole three-dimensional characteristic parameters by utilizing the space geometric shape characteristics and eliminating wrong hole three-dimensional characteristic parameters.
7. The three-dimensional inspection system of claim 1, wherein the auxiliary function calculator comprises:
a parameter optimization calculator for optimizing the hole three-dimensional characteristic parameters by using the hole two-dimensional characteristic parameters, the hole three-dimensional characteristic parameters and the weight coefficients based on a minimized reprojection error optimization method;
wherein the weight coefficient represents the weight corresponding to different object surface images.
8. The three-dimensional inspection system of claim 7, wherein the auxiliary function calculator further comprises:
the spatial positioning calculator is used for determining an included angle between a vector of origin points of the coordinate systems of the at least two image acquisition sensors pointing to the center of the hole and a normal direction of the hole, and registering spatial parameters of the hole obtained at different moments to the same coordinate system; the spatial parameters comprise the hole two-dimensional characteristic parameters and/or the hole three-dimensional characteristic parameters;
the data optimization calculator is used for determining the weight coefficient of the object surface image at the moment when the object surface image participates in calculation based on the included angle;
wherein the larger the included angle is, the smaller the weight coefficient is.
9. A three-dimensional inspection method, characterized by being performed by applying the three-dimensional inspection system of any one of claims 1 to 8, the three-dimensional inspection method comprising:
two image acquisition sensors acquire the surface image of the object;
a two-dimensional hole feature calculator determines hole two-dimensional feature parameters based on the object surface image;
the three-dimensional hole characteristic calculator carries out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters to determine the hole three-dimensional characteristic parameters;
and the auxiliary function calculator optimizes the hole three-dimensional characteristic parameters based on the hole two-dimensional characteristic parameters and the hole three-dimensional characteristic parameters.
10. The three-dimensional inspection method of claim 9, wherein the two-dimensional hole feature calculator comprises a hole feature classifier and a hole feature extractor; the two-dimensional hole feature calculator determines hole two-dimensional feature parameters based on the object surface image, including:
the hole feature classifier identifies the outline of the hole based on the object surface image and divides the hole into different categories according to the shape of the outline;
the hole feature extractor extracts the hole two-dimensional feature parameter based on a category of a contour of a hole.
11. The three-dimensional inspection method of claim 9, wherein the three-dimensional hole feature calculator comprises a hole three-dimensional reconstruction calculator and a verification calculator; the three-dimensional hole feature calculator performs three-dimensional reconstruction on the hole based on the hole two-dimensional feature parameters, and determines hole three-dimensional feature parameters, including:
the hole three-dimensional reconstruction calculator carries out three-dimensional reconstruction on the hole based on the hole two-dimensional characteristic parameters and the binocular vision principle to obtain hole three-dimensional characteristic parameters;
and the verification calculator verifies the hole three-dimensional characteristic parameters by utilizing the space geometric shape characteristics to eliminate wrong hole three-dimensional characteristic parameters.
12. The three-dimensional inspection method of claim 9, wherein the auxiliary function calculator is selected from the group consisting of a spatial location calculator, a data preference calculator, a spatial location calculator, and a parameter optimization calculator; the auxiliary function calculator optimizes the hole three-dimensional characteristic parameter based on the hole two-dimensional characteristic parameter and the hole three-dimensional characteristic parameter, and includes:
the space positioning calculator determines an included angle between a vector of the origin points of the coordinate systems of the at least two image acquisition sensors pointing to the center of the hole and the normal direction of the hole, and registers space parameters of the hole obtained at different moments to the same coordinate system; the spatial parameters comprise the hole two-dimensional characteristic parameters and/or the hole three-dimensional characteristic parameters;
the data optimization calculator determines a weight coefficient of the object surface image at the moment when the object surface image participates in calculation based on the included angle; the larger the included angle is, the smaller the weight coefficient is;
the parameter optimization calculator optimizes the hole three-dimensional characteristic parameters by adopting the hole two-dimensional characteristic parameters, the hole three-dimensional characteristic parameters and the weight coefficients based on a minimized reprojection error optimization method;
wherein the weight coefficient represents the weight corresponding to different object surface images.
CN202011457266.0A 2020-12-10 2020-12-10 Three-dimensional detection system and three-dimensional detection method Active CN112747671B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202011457266.0A CN112747671B (en) 2020-12-10 2020-12-10 Three-dimensional detection system and three-dimensional detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202011457266.0A CN112747671B (en) 2020-12-10 2020-12-10 Three-dimensional detection system and three-dimensional detection method

Publications (2)

Publication Number Publication Date
CN112747671A true CN112747671A (en) 2021-05-04
CN112747671B CN112747671B (en) 2022-12-09

Family

ID=75648394

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011457266.0A Active CN112747671B (en) 2020-12-10 2020-12-10 Three-dimensional detection system and three-dimensional detection method

Country Status (1)

Country Link
CN (1) CN112747671B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116447978A (en) * 2023-06-16 2023-07-18 先临三维科技股份有限公司 Hole site information detection method, device, equipment and storage medium
CN117115144A (en) * 2023-10-18 2023-11-24 深圳市强达电路股份有限公司 Online detection system for hole site defects in PCB

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050088665A1 (en) * 2003-10-28 2005-04-28 Timbre Technologies, Inc. Azimuthal scanning of a structure formed on a semiconductor wafer
US20100245355A1 (en) * 2009-03-27 2010-09-30 Ju Yong Chang Method for Estimating 3D Pose of Specular Objects
CN102135417A (en) * 2010-12-26 2011-07-27 北京航空航天大学 Full-automatic three-dimension characteristic extracting method
CN103615980A (en) * 2013-12-13 2014-03-05 北京理工大学 Method and system for measuring parameters of round holes in plate
CN105066909A (en) * 2015-09-01 2015-11-18 华中科技大学 Hand-held multi-laser-stripe quick three-dimensional measuring method
CN106403845A (en) * 2016-09-14 2017-02-15 杭州思看科技有限公司 3D sensor system and 3D data acquisition method
CN108151671A (en) * 2016-12-05 2018-06-12 杭州先临三维科技股份有限公司 A kind of 3 D digital imaging sensor, 3 D scanning system and its scan method
CN109000582A (en) * 2018-03-15 2018-12-14 杭州思看科技有限公司 Scan method and system, storage medium, the equipment of tracking mode three-dimensional scanner
CN109341591A (en) * 2018-11-12 2019-02-15 杭州思看科技有限公司 A kind of edge detection method and system based on handheld three-dimensional scanner
CN109859206A (en) * 2019-02-28 2019-06-07 易思维（杭州）科技有限公司 A kind of extracting method of circular hole feature
CN109855566A (en) * 2019-02-28 2019-06-07 易思维（杭州）科技有限公司 A kind of extracting method of slot feature
CN110345877A (en) * 2019-07-03 2019-10-18 西安交通大学 A kind of tube sheet aperture and method for measuring pitch
CN110715618A (en) * 2019-09-29 2020-01-21 北京天远三维科技股份有限公司 Dynamic three-dimensional scanning method and device
CN111325723A (en) * 2020-02-17 2020-06-23 杭州鼎热科技有限公司 Hole site detection method, device and equipment
CN111811431A (en) * 2020-06-12 2020-10-23 北京天远三维科技股份有限公司 Three-dimensional scanner, three-dimensional scanning system and method

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050088665A1 (en) * 2003-10-28 2005-04-28 Timbre Technologies, Inc. Azimuthal scanning of a structure formed on a semiconductor wafer
US20100245355A1 (en) * 2009-03-27 2010-09-30 Ju Yong Chang Method for Estimating 3D Pose of Specular Objects
CN102135417A (en) * 2010-12-26 2011-07-27 北京航空航天大学 Full-automatic three-dimension characteristic extracting method
CN103615980A (en) * 2013-12-13 2014-03-05 北京理工大学 Method and system for measuring parameters of round holes in plate
CN105066909A (en) * 2015-09-01 2015-11-18 华中科技大学 Hand-held multi-laser-stripe quick three-dimensional measuring method
CN106403845A (en) * 2016-09-14 2017-02-15 杭州思看科技有限公司 3D sensor system and 3D data acquisition method
CN108151671A (en) * 2016-12-05 2018-06-12 杭州先临三维科技股份有限公司 A kind of 3 D digital imaging sensor, 3 D scanning system and its scan method
CN109000582A (en) * 2018-03-15 2018-12-14 杭州思看科技有限公司 Scan method and system, storage medium, the equipment of tracking mode three-dimensional scanner
CN109341591A (en) * 2018-11-12 2019-02-15 杭州思看科技有限公司 A kind of edge detection method and system based on handheld three-dimensional scanner
CN109859206A (en) * 2019-02-28 2019-06-07 易思维（杭州）科技有限公司 A kind of extracting method of circular hole feature
CN109855566A (en) * 2019-02-28 2019-06-07 易思维（杭州）科技有限公司 A kind of extracting method of slot feature
CN110345877A (en) * 2019-07-03 2019-10-18 西安交通大学 A kind of tube sheet aperture and method for measuring pitch
CN110715618A (en) * 2019-09-29 2020-01-21 北京天远三维科技股份有限公司 Dynamic three-dimensional scanning method and device
CN111325723A (en) * 2020-02-17 2020-06-23 杭州鼎热科技有限公司 Hole site detection method, device and equipment
CN111811431A (en) * 2020-06-12 2020-10-23 北京天远三维科技股份有限公司 Three-dimensional scanner, three-dimensional scanning system and method

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
万谦等: "铝合金压铸件微观孔洞三维特征及分布的研究", 《金属学报》 *
李江雄等: "基于结构光的微细管孔内表面三维重建", 《仪器仪表学报》 *
谭小群等: "基于线激光扫描和图像处理的基准孔检测技术研究", 《现代制造工程》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116447978A (en) * 2023-06-16 2023-07-18 先临三维科技股份有限公司 Hole site information detection method, device, equipment and storage medium
CN116447978B (en) * 2023-06-16 2023-10-31 先临三维科技股份有限公司 Hole site information detection method, device, equipment and storage medium
CN117115144A (en) * 2023-10-18 2023-11-24 深圳市强达电路股份有限公司 Online detection system for hole site defects in PCB
CN117115144B (en) * 2023-10-18 2024-05-24 深圳市强达电路股份有限公司 Online detection system for hole site defects in PCB

Also Published As

Publication number Publication date
CN112747671B (en) 2022-12-09

Similar Documents

Publication Publication Date Title
US10282855B2 (en) Determining object properties with respect to particular optical measurement
CN106595528B (en) A kind of micro- binocular stereo vision measurement method of telecentricity based on digital speckle
CN104266608B (en) Field calibration device for visual sensor and calibration method
KR102424135B1 (en) Structured light matching of a set of curves from two cameras
CN103615980B (en) Method and system for measuring parameters of round holes in plate
WO2007015059A1 (en) Method and system for three-dimensional data capture
CN110763204B (en) Planar coding target and pose measurement method thereof
CN110163918A (en) A kind of line-structured light scaling method based on projective geometry
CN112747671B (en) Three-dimensional detection system and three-dimensional detection method
CN114170284B (en) Multi-view point cloud registration method based on active landmark point projection assistance
CN111028280B (en) # -shaped structured light camera system and method for performing scaled three-dimensional reconstruction of target
JP2021193400A (en) Method for measuring artefact
CN113505626A (en) Rapid three-dimensional fingerprint acquisition method and system
CN116402792A (en) Space hole site butt joint method based on three-dimensional point cloud
CN112268525A (en) Three-dimensional scanning method and three-dimensional scanning device
KR20040010091A (en) Apparatus and Method for Registering Multiple Three Dimensional Scan Data by using Optical Marker
Jia et al. A field measurement method for large objects based on a multi-view stereo vision system
CN103884294A (en) Wide view field type infrared light three-dimensional morphology measurement method and device thereof
CN113012238B (en) Method for quick calibration and data fusion of multi-depth camera
Takatsuka et al. Low-cost interactive active monocular range finder
Yamauchi et al. Calibration of a structured light system by observing planar object from unknown viewpoints
CN108662980B (en) Welding stud position detection device and method based on silhouette technology
Li et al. Structured light based high precision 3D measurement and workpiece pose estimation
Percoco et al. 3D image based modelling for inspection of objects with micro-features, using inaccurate calibration patterns: an experimental contribution
Uyanik et al. A method for determining 3D surface points of objects by a single camera and rotary stage

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information

Address after: No. 517-519, Floor 5, Complex Building, No. A1, Qinghe Yongtai Garden, Haidian District, Beijing 100192

Applicant after: TENYOUN 3D (TIANJIN) TECHNOLOGY Co.,Ltd.

Address before: 100192 rooms 512-519, complex building a 1, Yongtai garden, Qinghe, Haidian District, Beijing

Applicant before: TENYOUN 3D (TIANJIN) TECHNOLOGY Co.,Ltd.

CB02 Change of applicant information
TA01 Transfer of patent application right

Effective date of registration: 20221115

Address after: 311258 No. 1398, Wenbin Road, Wenyan street, Xiaoshan District, Hangzhou, Zhejiang.

Applicant after: Hangzhou Xianlin Tianyuan 3D detection technology Co.,Ltd.

Address before: No. 517-519, Floor 5, Complex Building, No. A1, Qinghe Yongtai Garden, Haidian District, Beijing 100192

Applicant before: TENYOUN 3D (TIANJIN) TECHNOLOGY Co.,Ltd.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant