CN116772744B - 3D scanning device and method based on laser ranging and vision fusion - Google Patents
3D scanning device and method based on laser ranging and vision fusion Download PDFInfo
- Publication number
- CN116772744B CN116772744B CN202311069185.7A CN202311069185A CN116772744B CN 116772744 B CN116772744 B CN 116772744B CN 202311069185 A CN202311069185 A CN 202311069185A CN 116772744 B CN116772744 B CN 116772744B
- Authority
- CN
- China
- Prior art keywords
- measured
- laser ranging
- points
- point
- ranging module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 230000004927 fusion Effects 0.000 title claims abstract description 15
- 238000005259 measurement Methods 0.000 claims abstract description 30
- 238000004364 calculation method Methods 0.000 claims abstract description 4
- 230000033001 locomotion Effects 0.000 claims description 8
- 230000009466 transformation Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 claims description 5
- 238000009434 installation Methods 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 4
- 238000000605 extraction Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 230000006978 adaptation Effects 0.000 abstract description 3
- 230000007547 defect Effects 0.000 abstract description 3
- 238000005516 engineering process Methods 0.000 description 8
- 239000003016 pheromone Substances 0.000 description 6
- 238000013461 design Methods 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 239000003638 chemical reducing agent Substances 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/86—Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Electromagnetism (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention relates to the technical field of 3D scanning, in particular to a 3D scanning device based on laser ranging and vision fusion and a method thereof, wherein the 3D scanning device comprises a main control board, an auxiliary calibration plane bracket, a camera fixing bracket, a cradle head, a camera and a laser ranging module; the method comprises the steps of initializing and calibrating a cradle head and a laser ranging module; collecting an image of an object to be measured, and analyzing points to be measured; calculating a tracking sequence; tracking and measuring each point to be measured one by one according to the tracking sequence; and converting world coordinates of the points to be measured, which are tracked and measured, into pixel coordinates, calculating distance information, and generating a 3D image with the fused distance and vision. The invention solves the defects of low efficiency, low speed and less adaptation to the environment of the traditional contact measurement, reduces huge calculation amount and post pretreatment caused by a large amount of point clouds by a mode of combining vision and single-point laser ranging, saves the measurement time and the cost of operation equipment, and improves the scanning efficiency.
Description
Technical Field
The invention relates to the technical field of 3D scanning, in particular to a 3D scanning device and method based on laser ranging and vision fusion.
Background
At present, the traditional measurement means and method can not meet the personalized measurement demand, the product appearance and the acquisition of surface 3D contour data and the post-processing technology become new research directions in the field of geometric measurement, and particularly, the 3D contour measurement technology based on laser scanning is a new research point in the field of geometric measurement.
The traditional 3D scanning device mainly uses a contact type measuring method, and related parameters of a required measuring device are obtained by manually measuring a corresponding device through a measuring tool and judging daily experience. With the rapid development of automation technology, there is a corresponding device for automated measurement of devices. The three-Coordinate Measuring Machine (CMM) is the most classical contact type 3D contour measuring system, integrates various advanced front edge technologies such as precision machinery, photoelectric information technology, measurement and control technology and the like, is precision measuring equipment with higher integration level, and is widely applied to industrial product detection, quality control and advanced manufacturing technology. However, the method also adopts a contact type measurement method, and contacts are needed to contact the surface of the object to be measured. The method has the advantages of high accuracy, low measurement efficiency, long measurement time and high cost. The contact points are worn in daily use due to contact with the surface of the object, which leads to an increase in measurement errors and an increase in maintenance costs.
Therefore, the invention provides a 3D scanning device and a method thereof based on laser ranging and vision fusion, so as to at least solve the technical problems.
Disclosure of Invention
The invention aims to solve the technical problems that: A3D scanning device based on laser ranging and vision fusion and a method thereof are provided to at least solve the above part of technical problems.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the utility model provides a 3D scanning device based on laser rangefinder and vision fuses, includes the main control board, locates supplementary demarcation plane support on the main control board, locates supplementary demarcation camera fixed bolster and cloud platform on the plane support to install the camera on the camera fixed bolster to and install the laser rangefinder module on the cloud platform, be equipped with integrated chip in the main control board, camera, laser rangefinder module respectively with integrated chip electric connection.
Further, the cradle head comprises a rotating bearing arranged on the auxiliary calibration plane support and a vertical shaft adapter arranged on the rotating bearing, and the laser ranging module is arranged in the vertical shaft adapter.
Further, the rotary bearing is connected with a horizontal shaft stepping motor for driving the rotary bearing to horizontally rotate, the rotary bearing is provided with a horizontal potentiometer and a vertical shaft stepping motor, the vertical shaft stepping motor is connected with a first toothed belt pulley, the first toothed belt pulley is connected with a vertical potentiometer and a second toothed belt pulley, the second toothed belt pulley is connected with a vertical shaft adapter, and the horizontal shaft stepping motor, the horizontal potentiometer, the vertical shaft stepping motor and the vertical potentiometer are respectively and electrically connected with the integrated chip.
The invention also provides a method of the 3D scanning device based on laser ranging and vision fusion, which comprises the following steps:
step 1, carrying out initial calibration on a cradle head and a laser ranging module;
step 2, a camera collects images of an object to be measured, and points to be measured of the object to be measured are analyzed from the collected images;
step 3, calculating the tracking sequence of the points to be measured;
step 4, according to the tracking sequence, the laser ranging module tracks and measures each point to be measured one by one to obtain the world coordinates of the point to be measured, and the cradle head is controlled by adopting a PID algorithm to carry out negative feedback tracking;
and 5, converting world coordinates of the points to be measured into pixel coordinates, calculating distance information of all the points to be measured of the object to be measured, and combining the acquired images with the distance information to generate a 3D image with the distance and vision combined.
Further, step 1 includes: step 11, starting the 3D scanning device, and respectively reading voltage values of a horizontal potentiometer and a vertical potentiometer by an integrated chip to obtain the current position of the cradle head; step 12, driving a cradle head and a laser ranging module to assist in calibrating a plane bracket to serve as a calibration surface to move by a horizontal axis stepping motor and a vertical axis stepping motor; fitting and deriving the motion data of the horizontal axis stepping motor under the horizontal drive and the measurement data of the laser ranging module to obtain a horizontal point of a minimum value, taking the horizontal point of the minimum value as a horizontal axis initial point, fitting and deriving the motion data of the vertical axis stepping motor under the vertical drive and the measurement data of the laser ranging module to obtain a vertical point of the minimum value, taking the vertical point of the minimum value as a vertical axis initial point, and integrating the horizontal axis initial point and the vertical axis initial point as calibration initial points; and 13, driving the cradle head to return to the calibration initial point by the horizontal axis stepping motor and the vertical axis stepping motor, and recording the deviation angles of the motors before and after the return to the calibration initial point by the horizontal potentiometer and the vertical potentiometer.
Further, step 1 further includes: and 14, measuring the actual distance between the laser ranging module and the auxiliary calibration plane support, correcting the actual distance by adopting the factory installation distance between the laser ranging module and the auxiliary calibration plane support, and repairing the offset of the laser ranging module.
Further, step 2 includes: step 21, a camera collects an image of an object to be detected, image enhancement and contour extraction are carried out on the collected image, and the image of the object to be detected is obtained and converted into a gray level image; processing the gray image by adopting a Harris corner detection algorithm to obtain all corners of the current view angle of the object to be detected and extracting pixel coordinates of each corner; detecting all straight lines in the gray level image by adopting an LSD algorithm; 22, performing line connection on all the obtained angular points, and transforming all the straight lines in the gray image and the straight lines connected with the angular point lines by adopting Hough transformation of polar coordinates to respectively obtain a first coordinate point and a second coordinate point; matching the first coordinate point and the second coordinate point through a nearest neighbor algorithm to obtain respective line segments, and obtaining pixel information of the edge contour of the object to be measured under the current view angle based on the matched respective line segments; and step 23, analyzing all plane information according to the pixel information of the edge contour of the object to be measured, randomly generating a plurality of points to be measured on each plane, recording the pixel coordinates of the points to be measured under the plane, and feeding back all the points to be measured to the integrated chip as target points tracked by the laser ranging module.
Further, step 3 includes: and the integrated chip receives pixel coordinates of the points to be measured, and obtains the tracking sequence of the points to be measured by adopting an ant colony algorithm.
Further, step 4 includes: step 41, a laser ranging module emits laser to irradiate on an object to be measured to form a laser spot, the laser spot is captured by a camera to obtain the position of the current laser spot under the pixel coordinate, and the position information of the current laser spot under the pixel coordinate is returned to the integrated chip in real time; step 42, the integrated chip obtains world coordinates of the current point to be measured according to the returned position information and the deviation angle of the motor; and 43, the integrated chip performs difference solving according to the returned pixel coordinates and the pixel coordinates of the corresponding point to be measured, and then controls the cradle head to work according to the PID algorithm to perform negative feedback tracking measurement of the laser ranging module.
Further, step 5 includes: converting world coordinates of points to be measured into pixel coordinates, calculating distance information of all the points to be measured by combining pixel information of edge contours of the objects to be measured and all plane information, generating an RGB map by the obtained distance information, and overlapping the RGB map with the acquired image to obtain a 3D image with fused distance and vision.
Compared with the prior art, the invention has the following beneficial effects:
the invention solves the defects of low efficiency, low speed and less adaptation to the environment of the traditional contact measurement, reduces huge calculation amount and post pretreatment caused by a large amount of point clouds by a mode of combining vision and single-point laser ranging, saves the measurement time and the cost of operation equipment, and improves the scanning efficiency.
Drawings
Fig. 1 is a schematic structural view of the present invention.
Fig. 2 is a flow chart of the method of the present invention.
Wherein, the names corresponding to the reference numerals are:
the device comprises a main control board, a 2-auxiliary calibration plane support, a 3-camera fixing support, a 4-camera, a 5-laser ranging module, a 6-rotating bearing, a 7-vertical shaft adapter, an 8-horizontal shaft stepping motor, a 9-horizontal potentiometer, a 10-vertical shaft stepping motor, a 11-first toothed belt pulley, a 12-vertical potentiometer and a 13-second toothed belt pulley.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings, in order to make the objects, technical solutions and advantages of the present invention more apparent. It will be apparent that the described embodiments are only some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
In the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. are based on the directions or positional relationships shown in the drawings, are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a specific orientation or be constructed and operated in a specific orientation, and thus they should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and the like, are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In the description of the present invention, it should be noted that, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; of course, it may be mechanically or electrically connected; in addition, the connection may be direct, indirect via an intermediate medium, or communication between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art according to the specific circumstances.
As shown in fig. 1, the 3D scanning device based on laser ranging and vision fusion provided by the invention comprises a main control board 1, an auxiliary calibration plane support 2 arranged on the main control board 1, a camera fixing support 3 and a cradle head arranged on the auxiliary calibration plane support 2, a camera 4 arranged on the camera fixing support 3, and a laser ranging module 5 arranged on the cradle head, wherein an integrated chip is arranged in the main control board 1, and the camera 4 and the laser ranging module 5 are respectively and electrically connected with the integrated chip. The invention solves the defects of low efficiency, low speed and less adaptation to the environment of the traditional contact measurement, reduces huge calculation amount and post pretreatment caused by a large amount of point clouds by a mode of combining vision and single-point laser ranging, saves the measurement time and the cost of operation equipment, and improves the scanning efficiency.
In some embodiments, the cradle head includes a rotating bearing 6 disposed on the auxiliary calibration plane support 2, and a vertical axis adapter 7 disposed on the rotating bearing 6, and the laser ranging module 5 is installed in the vertical axis adapter 7. Preferably, the vertical shaft adapter 7 includes a connecting seat provided on the swivel bearing 6 and a vertical seat movably and vertically connected to the connecting seat, the vertical shaft adapter 7 has a vertical movement function, the swivel bearing 6 has a horizontal rotation function, so that the cradle head and the laser ranging module 5 on the cradle head move horizontally or vertically, and the emitted laser line moves horizontally or vertically.
In some embodiments, the rotary bearing 6 is connected with a horizontal shaft stepper motor 8 for driving the rotary bearing 6 to rotate horizontally, the rotary bearing 6 is provided with a horizontal potentiometer 9 and a vertical shaft stepper motor 10, the vertical shaft stepper motor 10 is connected with a first toothed belt pulley 11, the first toothed belt pulley 11 is connected with a vertical potentiometer 12 and a second toothed belt pulley 13, the second toothed belt pulley 13 is connected with the vertical shaft adapter 7, and the horizontal shaft stepper motor 8, the horizontal potentiometer 9, the vertical shaft stepper motor 10 and the vertical potentiometer 12 are respectively and electrically connected with the integrated chip. The horizontal axis stepping motor 8 drives the rotary bearing 6 and the laser ranging module 5 to horizontally rotate, and the vertical axis stepping motor 10 is driven by the first toothed belt pulley 11 and the second toothed belt pulley 13. Wherein the horizontal axis stepper motor 8 is provided with 14:1, which can increase the torque, and can directly drive the whole rotary bearing 6 to horizontally rotate. However, the vertical axis stepper motor 10 is a normal 42 stepper motor, and a speed reducer cannot be installed due to horizontal placement, so that on one hand, the speed reduction ratio is improved by the first toothed belt pulley 11 and the second toothed belt pulley 13, and on the other hand, the vertical axis stepper motor 10 and the laser ranging module 5 are not in the same level, and on the other hand, the power transmission is realized by the first toothed belt pulley 11 and the second toothed belt pulley 13 in consideration of reasonable distribution of gravity. Preferably, the first toothed belt pulley 11 is a 15-toothed belt pulley and the second toothed belt pulley 13 is a 30-toothed belt pulley.
As shown in fig. 2, the invention further provides a method for a 3D scanning device based on laser ranging and vision fusion, which comprises the following steps:
step 1, carrying out initial calibration on a cradle head and a laser ranging module;
step 2, a camera collects images of an object to be measured, and points to be measured of the object to be measured are analyzed from the collected images;
step 3, calculating the tracking sequence of the points to be measured;
step 4, according to the tracking sequence, the laser ranging module tracks and measures each point to be measured one by one to obtain the world coordinates of the point to be measured, and the cradle head is controlled by adopting a PID algorithm to carry out negative feedback tracking;
and 5, converting world coordinates of the points to be measured into pixel coordinates, calculating distance information of all the points to be measured of the object to be measured, and combining the acquired images with the distance information to generate a 3D image with the distance and vision combined.
Because the stepping motor cannot retrieve the previous position after power failure, the motor position is initialized after the equipment is started each time, the precision of the potentiometer feedback position does not meet the required requirement although the potentiometer feedback position exists, the auxiliary calibration plane support is matched with the stepping motor and the laser ranging module, the origin position can be calculated through fitting data, and the original position is returned to the origin position for subsequent operation.
In some embodiments, step 1 is an initial calibration of the cradle head and the laser ranging module, including: step 11, starting the 3D scanning device, and respectively reading voltage values of a horizontal potentiometer and a vertical potentiometer by an integrated chip to obtain the current position of the cradle head; step 12, driving a cradle head and a laser ranging module to assist in calibrating a plane bracket to serve as a calibration surface to move by a horizontal axis stepping motor and a vertical axis stepping motor; fitting and deriving the motion data of the horizontal axis stepping motor under horizontal driving and the measurement data of the laser ranging module to obtain a horizontal point of a minimum value, and taking the horizontal point of the minimum value as a horizontal axis initial point; fitting and deriving motion data of a vertical axis stepping motor under vertical driving and measurement data of a laser ranging module to obtain a vertical point of a minimum value, taking the vertical point of the minimum value as a vertical axis initial point, and integrating the horizontal axis initial point and the vertical axis initial point as calibration initial points; and 13, driving the cradle head to return to the calibration initial point by the horizontal axis stepping motor and the vertical axis stepping motor, and recording the deviation angles of the motors before and after the return to the calibration initial point by the horizontal potentiometer and the vertical potentiometer. The initial calibration point is limited, and the horizontal potentiometer 9 is used for detection, and whether the rotation is in a reasonable range or not is judged.
In the step 12, when only horizontal rotation is performed by the horizontal axis stepper motor, the laser ranging module is always in a continuous measurement state during the period, and measurement data (distance data) of the laser ranging module and movement data (rotation angle value of rotation) of the horizontal axis stepper motor obtained during the continuous rotation are recorded; forming a rectangular coordinate system by the obtained series of distance data and rotation angle values, and converting each group of data into corresponding coordinate values so as to obtain a coordinate set; using polynomial fitting, first define a polynomial form: y=f (x), where empirically 2-degree polynomials are employed to approximate the data, y=ax 2 +bx+c, where a, b, c are coefficients that need to be found; subsequently, a coefficient matrix is constructed, for a given data point (x i ，y i ) Constructing a coefficient matrix A and a value vector b, A being an n x 3 matrix, where n is the number of data points and the first column is x i 2 The second column is x i The third column is 1 and the value vector b is an n x 1 column vector containing y i Is a value of (2); then, solving an equation, and solving the equation ax=b through a least square method to obtain a coefficient vector x of a fitting quadratic polynomial, wherein the coefficient vector comprises coefficients of each term in the quadratic polynomial, namely a, b and c; finally obtaining a fitted curve, deriving the curve function to obtain an extremum, wherein the rotation angle value is a horizontal initial value, namely the initial value of a horizontal axisA starting point. When the stepping motor only rotates vertically, the same is done, so the description is omitted.
Preferably, step 1 further comprises: and 14, measuring the actual distance between the laser ranging module and the auxiliary calibration plane support, correcting the actual distance by adopting the factory installation distance between the laser ranging module and the auxiliary calibration plane support, and repairing the offset of the laser ranging module. The auxiliary calibration plane support is fixed with mechanical dimensions at the beginning of design, and the distance between installation points of each part can be known through earlier mechanical design, wherein the distance from the laser ranging module to the auxiliary calibration plane support is in mm. Because the laser ranging module can generate fine deviation on the measured distance along with the change of the measured environment such as temperature, humidity and the like, the actual distance is corrected through the factory installation distance between the laser ranging module and the auxiliary calibration plane support, so that the deviation caused by the current measured environment is compensated.
In some embodiments, step 2 is a visual pre-process comprising: step 21, a camera collects an image of an object to be detected, image enhancement and contour extraction are carried out on the collected image, and the image of the object to be detected is obtained and converted into a gray level image; processing the gray image by adopting a Harris corner detection algorithm to obtain all corners of the current view angle of the object to be detected and extracting pixel coordinates of each corner; detecting all straight lines in the gray image by adopting an LSD algorithm, wherein the LSD algorithm can extract the straight lines in the gray image and comprises a plurality of irrelevant line segments; the method comprises the steps of screening contour line segments, carrying out line connection on all obtained corner points, wherein connection among the corner points is artificial custom connection, the connection comprises contour line segments and non-line segments in an image, then carrying out Hough transformation of polar coordinates, and transforming all straight lines in a gray level image and straight lines after the corner points are connected to obtain a first coordinate point and a second coordinate point respectively; matching the first coordinate point and the second coordinate point through a nearest neighbor algorithm to obtain respective line segments, and obtaining pixel information of the edge contour of the object to be measured under the current view angle based on the matched respective line segments; and step 23, analyzing all plane information according to the pixel information of the edge contour of the object to be measured, randomly generating a plurality of points to be measured on each plane, recording the pixel coordinates of the points to be measured under the plane, and feeding back all the points to be measured to the integrated chip as target points tracked by the laser ranging module.
The traditional Hough transformation is to transform a straight line into a coordinate parameter space related to b and k, wherein one point in the coordinate space corresponds to a straight line on a rectangular coordinate system, the traditional Hough transformation has the condition that k and b have infinity, and the condition is more in a pixel coordinate system, so that the traditional Hough transformation is correspondingly transformed, the invention adopts the thinking of polar coordinates to replace b and k with the distance rho from an origin to the straight line and the included angle theta between the straight line and an x axis respectively to obtain the following steps ofThe value of ρ is the diagonal length range of the pixel coordinate system, and θ is-90 DEG to 90 deg.
In some embodiments, step 3 comprises: and the integrated chip receives pixel coordinates of the points to be measured, and obtains the tracking sequence of the points to be measured by adopting an ant colony algorithm. Because the integrated chip tracks and measures each point to be measured one by one after receiving the pixel coordinates of the point to be measured, the sequence of measurement has a decisive effect on the length of measurement time, the tracking of each coordinate one by one is similar to the problem of a traveller, the problem of NP-Hard is all solved, the operation amount of calculating and tracking the optimal sequence can be exponentially increased along with the increase of the points, and the traditional computer operation method is not applicable. The embedded device is not friendly to the operation, so that the embedded device adopts a random circuit and a counter of an integrated chip to realize the optimization of the ant colony algorithm in an embedded environment.
The ant colony algorithm belongs to one of intelligent optimization algorithms, but is different from other intelligent optimization algorithms in some aspects, so that the whole process is difficult to realize through mathematical modeling, and is split into a plurality of steps, each step is realized through an approximate mathematical model, and the core of the ant colony algorithm is the updating of pheromones and the selection of paths. Based on the principle of ant colony algorithm, the pheromone can be realized by a counter and a timer, the counter of the corresponding path is increased along with the passing of the ant colony to the corresponding path, the update of the path pheromone is realized, and the pheromone on each path is volatilized along with the update of time. The invention is realized by adopting the timer, and the counters on all paths are correspondingly reduced along with the increment of the timer, so that the function of volatilizing the pheromone is realized, and the value of the counter on each path represents the content of the pheromone on the corresponding path; the path selection is realized by adopting a method of a common traditional computer, and the generation of roulette randomness is realized by adopting a random circuit of an integrated chip.
In some embodiments, step 4 comprises: step 41, a laser ranging module emits laser to irradiate on an object to be measured to form a laser spot, the laser spot is captured by a camera to obtain the position of the current laser spot under the pixel coordinate, and the position information of the current laser spot under the pixel coordinate is returned to the integrated chip in real time; step 42, the integrated chip obtains world coordinates of the current point to be measured according to the returned position information and the deviation angle of the motor; and 43, the integrated chip performs difference solving according to the returned pixel coordinates and the pixel coordinates of the corresponding point to be measured, and then controls the cradle head to work according to the PID algorithm to perform negative feedback tracking measurement of the laser ranging module. The traditional 3D scanning needs to measure the whole space as much as possible, but the measurement of most points is not needed, so the invention obtains the key points (the positions of the points to be measured) to be measured through visual processing, and then the laser is controlled by the holder to track the points to be measured, and the negative feedback tracking is used for enabling the laser to move to the positions of the points to be measured more quickly and stably.
In some embodiments, step 5 comprises: converting world coordinates of points to be measured into pixel coordinates, combining pixel information of edge contours of objects to be measured and all plane information, calculating distance information of all the points to be measured, generating an RGB map which is the same as the size of an image acquired by a camera by using the obtained distance information, encoding the distance information on the RGB map of corresponding pixels, and superposing the RGB map and the acquired image to obtain a 3D image with the fused distance and vision.
The integrated chip used in the present invention is preferably an FPGA, and the camera 4, the laser ranging module 5, the horizontal axis stepper motor 8, the horizontal potentiometer 9, the vertical axis stepper motor 10 and the vertical potentiometer 12 used in the present invention are all known electrical devices and can be directly purchased in the market, and the structure, the circuit and the control principle thereof are all known technologies, so the structure, the circuit and the control principle of the camera 4, the laser ranging module 5, the horizontal axis stepper motor 8, the horizontal potentiometer 9, the vertical axis stepper motor 10 and the vertical potentiometer 12 are not described herein.
Finally, it should be noted that: the above embodiments are merely preferred embodiments of the present invention for illustrating the technical solution of the present invention, but not limiting the scope of the present invention; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the corresponding technical solutions; that is, even though the main design concept and spirit of the present invention is modified or finished in an insubstantial manner, the technical problem solved by the present invention is still consistent with the present invention, and all the technical problems are included in the protection scope of the present invention; in addition, the technical scheme of the invention is directly or indirectly applied to other related technical fields, and the technical scheme is included in the scope of the invention.
Claims (6)
1. The 3D scanning method based on laser ranging and vision fusion is characterized by comprising the following steps of:
step 1, carrying out initial calibration on a cradle head and a laser ranging module;
step 2, a camera collects images of an object to be measured, and points to be measured of the object to be measured are analyzed from the collected images;
step 3, calculating the tracking sequence of the points to be measured;
step 4, according to the tracking sequence, the laser ranging module tracks and measures each point to be measured one by one to obtain the world coordinates of the point to be measured, and the cradle head is controlled by adopting a PID algorithm to carry out negative feedback tracking;
step 5, converting world coordinates of points to be measured into pixel coordinates, calculating distance information of all the points to be measured of the object to be measured, combining the acquired images with the distance information, and generating a 3D image with the distance and vision combined;
step 4 comprises: step 41, a laser ranging module emits laser to irradiate on an object to be measured to form a laser spot, the laser spot is captured by a camera to obtain the position of the current laser spot under the pixel coordinate, and the position information of the current laser spot under the pixel coordinate is returned to the integrated chip in real time; step 42, the integrated chip obtains world coordinates of the current point to be measured according to the returned position information and the deviation angle of the motor; step 43, the integrated chip performs difference calculation according to the returned pixel coordinates and the pixel coordinates of the corresponding points to be measured, and then controls the cradle head to work according to the PID algorithm to perform negative feedback tracking measurement of the laser ranging module;
the 3D scanning method is based on a 3D scanning device, the 3D scanning device comprises a main control board (1), an auxiliary calibration plane support (2) arranged on the main control board (1), a camera fixing support (3) and a cradle head arranged on the auxiliary calibration plane support (2), a camera (4) arranged on the camera fixing support (3) and a laser ranging module (5) arranged on the cradle head, an integrated chip is arranged in the main control board (1), and the camera (4) and the laser ranging module (5) are respectively and electrically connected with the integrated chip;
the cradle head comprises a rotary bearing (6) arranged on the auxiliary calibration plane support (2) and a vertical shaft adapter (7) arranged on the rotary bearing (6), and the laser ranging module (5) is arranged in the vertical shaft adapter (7);
the rotary bearing (6) is connected with a horizontal shaft stepping motor (8) for driving the rotary bearing (6) to horizontally rotate, the rotary bearing (6) is provided with a horizontal potentiometer (9) and a vertical shaft stepping motor (10), the vertical shaft stepping motor (10) is connected with a first toothed belt pulley (11), the first toothed belt pulley (11) is connected with a vertical potentiometer (12) and a second toothed belt pulley (13), the second toothed belt pulley (13) is connected with a vertical shaft adapter (7), and the horizontal shaft stepping motor (8), the horizontal potentiometer (9), the vertical shaft stepping motor (10) and the vertical potentiometer (12) are respectively and electrically connected with an integrated chip.
2. The 3D scanning method based on laser ranging and vision fusion according to claim 1, wherein step 1 comprises: step 11, starting the 3D scanning device, and respectively reading voltage values of a horizontal potentiometer and a vertical potentiometer by an integrated chip to obtain the current position of the cradle head; step 12, driving a cradle head and a laser ranging module to assist in calibrating a plane bracket to serve as a calibration surface to move by a horizontal axis stepping motor and a vertical axis stepping motor; fitting and deriving the motion data of the horizontal axis stepping motor under the horizontal drive and the measurement data of the laser ranging module to obtain a horizontal point of a minimum value, taking the horizontal point of the minimum value as a horizontal axis initial point, fitting and deriving the motion data of the vertical axis stepping motor under the vertical drive and the measurement data of the laser ranging module to obtain a vertical point of the minimum value, taking the vertical point of the minimum value as a vertical axis initial point, and integrating the horizontal axis initial point and the vertical axis initial point as calibration initial points; and 13, driving the cradle head to return to the calibration initial point by the horizontal axis stepping motor and the vertical axis stepping motor, and recording the deviation angles of the motors before and after the return to the calibration initial point by the horizontal potentiometer and the vertical potentiometer.
3. The 3D scanning method based on laser ranging and vision fusion according to claim 2, wherein step 1 further comprises: and 14, measuring the actual distance between the laser ranging module and the auxiliary calibration plane support, correcting the actual distance by adopting the factory installation distance between the laser ranging module and the auxiliary calibration plane support, and repairing the offset of the laser ranging module.
4. The 3D scanning method based on laser ranging and vision fusion according to claim 2, wherein step 2 comprises: step 21, a camera collects an image of an object to be detected, image enhancement and contour extraction are carried out on the collected image, and the image of the object to be detected is obtained and converted into a gray level image; processing the gray image by adopting a Harris corner detection algorithm to obtain all corners of the current view angle of the object to be detected and extracting pixel coordinates of each corner; detecting all straight lines in the gray level image by adopting an LSD algorithm; 22, performing line connection on all the obtained angular points, and transforming all the straight lines in the gray image and the straight lines connected with the angular point lines by adopting Hough transformation of polar coordinates to respectively obtain a first coordinate point and a second coordinate point; matching the first coordinate point and the second coordinate point through a nearest neighbor algorithm to obtain respective line segments, and obtaining pixel information of the edge contour of the object to be measured under the current view angle based on the matched respective line segments; and step 23, analyzing all plane information according to the pixel information of the edge contour of the object to be measured, randomly generating a plurality of points to be measured on each plane, recording the pixel coordinates of the points to be measured under the plane, and feeding back all the points to be measured to the integrated chip as target points tracked by the laser ranging module.
5. The 3D scanning method based on laser ranging and vision fusion as claimed in claim 4, wherein the step 3 comprises: and the integrated chip receives pixel coordinates of the points to be measured, and obtains the tracking sequence of the points to be measured by adopting an ant colony algorithm.
6. The 3D scanning method based on laser ranging and vision fusion according to claim 5, wherein step 5 comprises: converting world coordinates of points to be measured into pixel coordinates, calculating distance information of all the points to be measured by combining pixel information of edge contours of the objects to be measured and all plane information, generating an RGB map by the obtained distance information, and overlapping the RGB map with the acquired image to obtain a 3D image with fused distance and vision.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311069185.7A CN116772744B (en) | 2023-08-24 | 2023-08-24 | 3D scanning device and method based on laser ranging and vision fusion |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311069185.7A CN116772744B (en) | 2023-08-24 | 2023-08-24 | 3D scanning device and method based on laser ranging and vision fusion |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116772744A CN116772744A (en) | 2023-09-19 |
| CN116772744B true CN116772744B (en) | 2023-10-24 |
Family
ID=88008495
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311069185.7A Active CN116772744B (en) | 2023-08-24 | 2023-08-24 | 3D scanning device and method based on laser ranging and vision fusion |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116772744B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117351045A (en) * | 2023-12-06 | 2024-01-05 | 深圳市宏源建设科技有限公司 | Support and hanger assembly position regulation and control method and system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104501740A (en) * | 2014-12-18 | 2015-04-08 | 杭州鼎热科技有限公司 | Handheld laser three-dimension scanning method and handheld laser three-dimension scanning equipment based on mark point trajectory tracking |
| JP2015228992A (en) * | 2014-06-05 | 2015-12-21 | 大日本印刷株式会社 | Visual axis analysis system and visual axis analysis device |
| WO2020056874A1 (en) * | 2018-09-17 | 2020-03-26 | 魔门塔(苏州)科技有限公司 | Automatic parking system and method based on visual recognition |
| CN111811395A (en) * | 2020-06-09 | 2020-10-23 | 天津大学 | Monocular vision-based dynamic plane pose measurement method |
| CN113465536A (en) * | 2021-06-30 | 2021-10-01 | 皖江工学院 | Laser holder based on camera guide and working method thereof |
| CN113963416A (en) * | 2021-11-05 | 2022-01-21 | 北京航空航天大学 | Eye movement interaction method and system based on laser visual feedback |
| CN113984026A (en) * | 2021-10-28 | 2022-01-28 | 江苏徐工工程机械研究院有限公司 | Total station scanning all-in-one instrument based on three-dimensional laser mapping technology and measuring method |
| WO2022170633A1 (en) * | 2021-02-15 | 2022-08-18 | 苏州优它科技有限公司 | Rail transit vehicle collision avoidance detection method based on vision and laser ranging |
| CN115540750A (en) * | 2022-09-19 | 2022-12-30 | 中国矿业大学 | Monocular vision based object size detection method and system and storable medium |
| CN115597551A (en) * | 2022-12-14 | 2023-01-13 | 成都量芯集成科技有限公司（Cn） | Handheld laser-assisted binocular scanning device and method |
| CN116184654A (en) * | 2022-12-27 | 2023-05-30 | 同济大学 | Coordinate system construction method, prism scanner multi-target detection scanning method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170261993A1 (en) * | 2016-03-10 | 2017-09-14 | Xerox Corporation | Systems and methods for robot motion control and improved positional accuracy |
| US20170309187A1 (en) * | 2016-04-22 | 2017-10-26 | Hsin-Nan Lin | Vehicle platoon system and method thereof |
| US10484599B2 (en) * | 2016-10-25 | 2019-11-19 | Microsoft Technology Licensing, Llc | Simulating depth of field |
-
2023
- 2023-08-24 CN CN202311069185.7A patent/CN116772744B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2015228992A (en) * | 2014-06-05 | 2015-12-21 | 大日本印刷株式会社 | Visual axis analysis system and visual axis analysis device |
| CN104501740A (en) * | 2014-12-18 | 2015-04-08 | 杭州鼎热科技有限公司 | Handheld laser three-dimension scanning method and handheld laser three-dimension scanning equipment based on mark point trajectory tracking |
| WO2020056874A1 (en) * | 2018-09-17 | 2020-03-26 | 魔门塔(苏州)科技有限公司 | Automatic parking system and method based on visual recognition |
| CN111811395A (en) * | 2020-06-09 | 2020-10-23 | 天津大学 | Monocular vision-based dynamic plane pose measurement method |
| WO2022170633A1 (en) * | 2021-02-15 | 2022-08-18 | 苏州优它科技有限公司 | Rail transit vehicle collision avoidance detection method based on vision and laser ranging |
| CN113465536A (en) * | 2021-06-30 | 2021-10-01 | 皖江工学院 | Laser holder based on camera guide and working method thereof |
| CN113984026A (en) * | 2021-10-28 | 2022-01-28 | 江苏徐工工程机械研究院有限公司 | Total station scanning all-in-one instrument based on three-dimensional laser mapping technology and measuring method |
| CN113963416A (en) * | 2021-11-05 | 2022-01-21 | 北京航空航天大学 | Eye movement interaction method and system based on laser visual feedback |
| CN115540750A (en) * | 2022-09-19 | 2022-12-30 | 中国矿业大学 | Monocular vision based object size detection method and system and storable medium |
| CN115597551A (en) * | 2022-12-14 | 2023-01-13 | 成都量芯集成科技有限公司（Cn） | Handheld laser-assisted binocular scanning device and method |
| CN116184654A (en) * | 2022-12-27 | 2023-05-30 | 同济大学 | Coordinate system construction method, prism scanner multi-target detection scanning method and system |
Non-Patent Citations (4)
| Title |
|---|
| 一种改进的视觉传感器与激光测距雷达特征匹配点提取算法;邓志红;刘明阳;付梦印;;光学技术(第01期);全文 * |
| 基于OpenMV的目标跟踪系统设计;刘义亭;董梦超;黄家才;范子霄;宗文锦;郭婧;;南京工程学院学报(自然科学版)(第01期);全文 * |
| 基于点云数据的树木骨架线提取研究;李杨;李秀峰;;科技创新与生产力(第06期);全文 * |
| 激光雷达和摄像机联合标定识别作物;许莎莎;李文彬;康峰;;广东农业科学(第24期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116772744A (en) | 2023-09-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12001191B2 (en) | Automated 360-degree dense point object inspection | |
| US11087458B2 (en) | Automated in-line object inspection | |
| CN116772744B (en) | 3D scanning device and method based on laser ranging and vision fusion | |
| CN102062588B (en) | Computer binocular vision denture scanning device and three-dimensional reconstruction method thereof | |
| CN210155545U (en) | Be used for automatic on-line measuring equipment of switch manufacturing process | |
| WO2020223594A2 (en) | Automated 360-degree dense point object inspection | |
| CN201522266U (en) | Computer-based binocular vision false-tooth scanning device | |
| CN105783723B (en) | Precision die surface processing accuracy detection device and method based on machine vision | |
| CN111028340B (en) | Three-dimensional reconstruction method, device, equipment and system in precise assembly | |
| CN106842216B (en) | A kind of workpiece pose online test method cooperateed with based on Kinect with three-dimensional laser | |
| CN112581451A (en) | Warehouse coil position detection system and method based on laser radar | |
| CN112762899B (en) | Fusion method of laser point cloud and BIM model with video information in visual transformer substation | |
| CN113155047B (en) | Long-distance hole distance measuring device and method, storage medium, equipment and rail vehicle | |
| CN112697058A (en) | Machine vision-based large-size plate assembly gap on-line measurement system and method | |
| CN113884002B (en) | Pantograph slide plate upper surface detection system and method based on two-dimensional and three-dimensional information fusion | |
| CN115972093B (en) | Workpiece surface measuring method and device and wing wallboard soft mold polishing method | |
| CN113188442B (en) | Multi-angle point cloud measuring tool for seat furniture and splicing method thereof | |
| CN115616528A (en) | Laser radar-based bridge prefabricated part appearance size detection device and method | |
| CN114841944A (en) | Tailing dam surface deformation inspection method based on rail-mounted robot | |
| CN109002008A (en) | A kind of cross slid platform automated calibration system based on monocular vision | |
| CN111985398A (en) | Infrared-assisted binocular distance measurement method and distance compensation heat value acquisition method | |
| Yan et al. | Multi-line laser structured light fast visual positioning system with assist of TOF and CAD | |
| CN116945126A (en) | Binocular initiative visual monitoring device suitable for precision machining | |
| CN111435075A (en) | Computer vision measurement system | |
| CN115222893A (en) | Three-dimensional reconstruction splicing method for large-size components based on structured light measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |