CN111476847B - Virtual instrument device for calibrating structured light parameters and method thereof - Google Patents
Virtual instrument device for calibrating structured light parameters and method thereof Download PDFInfo
- Publication number
- CN111476847B CN111476847B CN202010257728.8A CN202010257728A CN111476847B CN 111476847 B CN111476847 B CN 111476847B CN 202010257728 A CN202010257728 A CN 202010257728A CN 111476847 B CN111476847 B CN 111476847B
- Authority
- CN
- China
- Prior art keywords
- structured light
- virtual instrument
- parameters
- light system
- portable virtual
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 25
- 238000003384 imaging method Methods 0.000 claims abstract description 42
- 238000007500 overflow downdraw method Methods 0.000 claims abstract description 14
- 230000002452 interceptive effect Effects 0.000 claims abstract description 8
- 238000007689 inspection Methods 0.000 claims abstract description 7
- 230000000007 visual effect Effects 0.000 claims abstract description 6
- 230000006854 communication Effects 0.000 claims description 4
- 230000004927 fusion Effects 0.000 claims description 4
- 230000007175 bidirectional communication Effects 0.000 claims description 3
- 230000002457 bidirectional effect Effects 0.000 abstract description 2
- 238000012706 support-vector machine Methods 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a virtual instrument device for calibrating a structured light parameter, wherein a portable virtual instrument is connected with a structured light system through a high-speed data bus; the structured light system comprises a laser emitter and an imaging receiver; the portable virtual instrument is a special purpose system for calibrating the structured light visual parameters. The method for calibrating the structured light parameters comprises (1) starting a parameter inspection mode of a virtual instrument device, negotiating initial structured light parameters by a portable virtual instrument and a structured light system, and enabling the structured light system to work; (2) The virtual instrument device uses an interactive method to calibrate the structured light parameters of the structured light system; (3) The virtual instrument device calibrates the structured light parameters of the structured light system using a fusion method. The invention provides a method for calibrating parameters of a structured light system by using an interactive method, and the bidirectional operation parameter configuration is efficient and flexible; the parameters of the structured light system are calibrated by a fusion method, so that errors are obviously reduced, and the parameters are accurate.
Description
Technical Field
The invention belongs to the technical field of optics, and particularly relates to a virtual instrument device for calibrating a structured light parameter and a method thereof.
Background
A typical structured light system consists of a laser transmitter and an imaging receiver. The photoelectric imaging system of the structured light system has 3D characteristics, and the parameter calibration process is very complex. The laser transmitter is capable of projecting optical image information having encoded information onto a target object, the laser light being reflected off the target object, and then capturing an image of the target object through the imaging receiver. Parameters of the imaging receiver are calculated by analyzing the modes of the characteristic images, and parameters of the laser transmitter are calculated by transformation relations. The laser transmitter and the imaging receiver have deviation between actual imaging points and theoretical imaging points; imaging receivers typically use a CCD imaging assembly; the CCD imaging assembly and its optics have radial and tangential errors, which in turn cause imaging aberrations. The parameter calibration process of the structured light system is mainly embodied in the parameter calibration of the laser transmitter and the imaging receiver.
Disclosure of Invention
In order to solve the problems, the invention provides a method and a virtual instrument device for calibrating a structured light visual parameter. The virtual instrument device establishes high-speed two-way communication with the structured light system, and the virtual instrument device can finish parameter inspection, interactive parameter calibration and fusion type parameter calibration of the structured light system.
The specific technical scheme is as follows:
a virtual instrument device for structured light parameter calibration, comprising: a portable virtual instrument and a structured light system; the portable virtual instrument is connected with the structured light system through a high-speed data bus; the structured light system comprises a laser emitter and an imaging receiver; the portable virtual instrument is a special purpose system for calibrating the structured light visual parameters.
Virtual instrument device for structured light parameter calibration to operate a method for structured light parameter calibration, comprising the steps of:
(1) The virtual instrument device starts a parameter inspection mode, and the portable virtual instrument and the structured light system negotiate initial structured light parameters and enable the structured light system to work;
either of the following two steps is then employed:
(2) The virtual instrument device uses an interactive method to calibrate the structured light parameters of the structured light system;
(3) The virtual instrument device calibrates the structured light parameters of the structured light system using a fusion method.
The step (1) specifically comprises the following steps:
(1.1) the portable virtual instrument and the structured light system establish a communication connection through a high-speed bi-directional communication interface;
(1.2) the portable virtual instrument distributes default structured light parameters to initialize the structured light system;
(1.3) the structured light system emitting a cloud of laser points having an image of the feature according to the initial configuration;
(1.4) the imaging receiver of the structured light system receives the characteristic point cloud and sends the characteristic point cloud to the portable virtual instrument;
(1.5) the structured light system enters a patrol mode;
(1.6) the portable virtual instrument outputs the structured light parameters of the structured light system.
The step (2) specifically comprises the following steps:
(2.1) the portable virtual instrument adjusts the feature model type or parameters;
(2.2) the portable virtual instrument recalculates the optimal structured light parameters;
(2.3) the portable virtual instrument re-distributes the recalculated optimal structured light parameters to the structured light systems;
(2.4) the structured light system updates and validates the new structured light parameters;
(2.5) the imaging receiver of the structured light system receives the characteristic point cloud and sends the characteristic point cloud to the portable virtual instrument;
(2.6) the portable virtual instrument evaluates the parameter quality; if the imaging receiver parameters are already optimal, go to step (2.7); otherwise, repeating the step (2.2) -the step (2.6);
(2.7) updating parameters of the laser transmitter by parameters of the imaging receiver.
The step (3) specifically comprises the following steps:
(3.1) the structured light system loads a typical pattern image database;
(3.2) the structured light system enters a fusion type parameter calibration working mode;
(3.3) the structured light system loads the typical pattern image and emits a laser lattice cloud with a typical pattern;
(3.4) the imaging receiver of the structured light system receives the characteristic point cloud and sends the characteristic point cloud to the display device of the portable virtual instrument;
(3.5) the portable virtual instrument receives and renders the lattice cloud data; and recalculate the optimal structured light parameters;
(3.6) the portable virtual instrument storing the set of structured light parameters; if the portable virtual instrument is traversed, performing a step (3.7); otherwise, repeating the step (3.3) -the step (3.6);
and (3.7) calculating the optimal structured light parameters by adopting a fusion method based on a weight model.
The step (3) and the step (2) are in parallel relation, and can be completed to enable the virtual instrument to automatically find the optimal structured light parameters.
The fusion method based on the weight model is preferably SVM (support vector machine) and CNN (convolutional neural network).
The structured light parameters include: camera focal length, principal point, and distortion coefficient of the imaging receiver; the equation of the plane of light for the laser transmitter and the positional relationship of the plane of light with respect to the imaging receiver.
The virtual instrument device for calibrating the structured light parameters and the method thereof provided by the invention have the following technical effects:
(1) The virtual instrument device uses an interactive method to calibrate parameters of the structured light system, and bidirectional operation parameter configuration is efficient and flexible.
(2) The virtual instrument device uses a fusion method to calibrate parameters of the structured light system, so that errors are obviously reduced, and the parameters are accurate.
Drawings
FIG. 1 is a schematic diagram of the structure of the present invention;
FIG. 2 is a flow chart of a parameter inspection mode of the virtual instrument device of the present invention;
FIG. 3 is a flow chart of the parameters of the virtual instrument device of the present invention calibrating a structured light system using an interactive method;
FIG. 4 is a flow chart of calibrating parameters of a structured light system using a fusion method for a virtual instrument device according to the present invention;
fig. 5 is a typical feature image of a structured light system of the present invention.
Detailed Description
The specific technical scheme of the invention is described by combining the embodiments.
As shown in fig. 1, a virtual instrument device for structured light parameter calibration, comprising: a portable virtual instrument and a structured light system; the portable virtual instrument is connected with the structured light system through a high-speed data bus; the structured light system includes a laser transmitter and an imaging receiver.
The high speed data bus includes MIPI, USB, etc.
The portable virtual instrument is a special purpose software and hardware system for the structured light visual parameter calibration, and the hardware entity can be an industrial personal computer, a PC, a mobile phone, VR or AR and the like, and is internally provided with a software entity for the structured light visual parameter calibration.
Virtual instrument means for structured light parameter calibration to run a method for structured light parameter calibration; the method comprises the following steps:
1. as shown in fig. 2, the virtual instrument device starts a parameter inspection mode, and the parameter inspection mode works: the portable virtual instrument and the structured light system negotiate initial structured light parameters and operate the structured light system. The method specifically comprises the following steps: (1) The portable virtual instrument and the structured light system establish a communication connection through a high-speed bi-directional communication interface. (2) The portable virtual instrument distributes default structured light parameters to initialize the structured light system. (3) The structured light system emits a laser point cloud having an image of the feature according to an initial configuration. (4) An imaging receiver of the structured light system receives the characteristic point cloud and transmits the characteristic point cloud to the portable virtual instrument. (5) the structured light system enters a patrol mode. (6) The portable virtual instrument outputs structured light parameters of the structured light system. The structured light parameters include: camera focal length, principal point, and distortion coefficient of the imaging receiver; the equation of the plane of light for the laser transmitter and the positional relationship of the plane of light with respect to the imaging receiver.
2. As shown in fig. 3, the virtual instrument device calibrates parameters of the structured light system using an interactive method. The working mode works as follows: the user operates the feature model in the virtual instrument, such as a feature cube, adjusts the feature model type or parameters, such as scaling/rotation/translation, and the virtual instrument recalculates and assigns the optimal structured light parameters; the structured light system receives and validates the structured light parameters; the virtual instrument receives the new image and evaluates the image quality; the above process is iterated repeatedly until the effect is optimal. The detailed steps are as follows: (1) The portable virtual instrument adjusts the feature model type or parameters (zoom/rotate/translate). (2) The portable virtual instrument recalculates the optimal structured light parameters. (3) The portable virtual instrument will recalculate the optimal structured light parameters to be redistributed to the structured light system. (4) The structured light system updates and validates the new structured light parameters. (5) An imaging receiver of the structured light system receives the characteristic point cloud and transmits the characteristic point cloud to the portable virtual instrument. (6) the portable virtual instrument evaluates the quality of the parameter. If the imaging receiver parameters are already optimal, then turning to (7) updating the parameters of the laser transmitter with the imaging receiver parameters; otherwise, repeating the steps 2-6. The structured light basic parameters include: camera focal length, principal point, and distortion coefficient of the imaging receiver; the equation of the plane of light for the laser transmitter and the positional relationship of the plane of light with respect to the imaging receiver.
3. As shown in fig. 4, the virtual instrument device calibrates the structured light parameters of the structured light system using a fusion method. (1) structured light systems load a typical pattern image database. (2) And the structured light system enters a fusion type parameter calibration working mode. (3) The structured light system loads the typical pattern image and emits a laser lattice cloud with the typical pattern. (4) An imaging receiver of the structured light system receives the feature point cloud and transmits it to a portable computing display device. (5) The portable virtual instrument receives and renders the lattice cloud data. And recalculate the optimal structured light parameters. (6) the portable virtual instrument stores the set of structured light parameters. If the portable virtual instrument is traversed, carrying out (7) a fusion method based on a weight model, and calculating the optimal structured light parameters; otherwise, repeating the steps 3-6.
The fusion method based on the weight model is preferably SVM (support vector machine), CNN (convolutional neural network) and the like. Based on a fusion method of weight models, the input of the models is a parameter of an imaging receiver; the output of the model is also a parameter of the imaging receiver. The SVM (support vector machine) has better generalization capability, and can convert nonlinear optimization problems into linear optimization problems. The support vector machine fully utilizes the VC dimension theory and the structure risk minimum theory, can make better prediction according to limited sample information, and has strong generalization capability. The basic principle of convolutional neural networks is: a network is formed by connecting a plurality of convolution layers in series; the multi-layer convolution may extract features, i.e., low-level features, medium-level features, and high-level features, layer-by-layer. The network comprises a pooling layer and a full connection layer besides the convolution layer, and is used for data classification tasks.
As shown in fig. 5, characteristic images, spots, bars or facets, etc., commonly used in structured light systems. The structured light system may be configured to emit a specified feature image. The laser transmitter is capable of projecting optical image information having encoded information onto a target object, the laser light being reflected off the target object, and then capturing an image of the target object through the imaging receiver.
Claims (3)
1. The method for calibrating the structured light parameters uses a virtual instrument device for calibrating the structured light parameters and is characterized by comprising a portable virtual instrument and a structured light system; the portable virtual instrument is connected with the structured light system through a high-speed data bus; the structured light system comprises a laser emitter and an imaging receiver;
the portable virtual instrument is a special purpose system for calibrating the structured light visual parameters;
the method is characterized by comprising the following steps of:
(1) The virtual instrument device starts a parameter inspection mode, and the portable virtual instrument and the structured light system negotiate initial structured light parameters and enable the structured light system to work;
one of the following two steps is then taken:
the step (1) specifically comprises the following steps:
(1.1) the portable virtual instrument and the structured light system establish a communication connection through a high-speed bi-directional communication interface;
(1.2) the portable virtual instrument distributes default structured light parameters to initialize the structured light system;
(1.3) the structured light system emitting a cloud of laser points having an image of the feature according to the initial configuration;
(1.4) the imaging receiver of the structured light system receives the characteristic point cloud and sends the characteristic point cloud to the portable virtual instrument;
(1.5) the structured light system enters a patrol mode;
(1.6) outputting the structured light parameters of the structured light system by the portable virtual instrument;
(2) The virtual instrument device calibrates the structure light parameters of the structure light system by using an interactive method, and automatically calculates and evaluates the optimal structure light parameters;
the step (2) specifically comprises the following steps:
(2.1) the portable virtual instrument adjusts the feature model type or parameters;
(2.2) the portable virtual instrument recalculates the optimal structured light parameters;
(2.3) the portable virtual instrument re-distributes the recalculated optimal structured light parameters to the structured light systems;
(2.4) the structured light system updates and validates the new structured light parameters;
(2.5) the imaging receiver of the structured light system receives the characteristic point cloud and sends the characteristic point cloud to the portable virtual instrument;
(2.6) the portable virtual instrument evaluates the parameter quality; if the imaging receiver parameters are already optimal, go to step (2.7); otherwise, repeating the step (2.2) -the step (2.6);
(2.7) updating parameters of the laser transmitter by parameters of the imaging receiver;
(3) The virtual instrument device uses a fusion method to calibrate the structural light parameters of the structural light system, and automatically calculates and evaluates the optimal structural light parameters;
the step (3) specifically comprises the following steps:
(3.1) the structured light system loads a typical pattern image database;
(3.2) the structured light system enters a fusion type parameter calibration working mode;
(3.3) the structured light system loads the typical pattern image and emits a laser lattice cloud with a typical pattern;
(3.4) the imaging receiver of the structured light system receives the characteristic point cloud and sends the characteristic point cloud to the display device of the portable virtual instrument;
(3.5) the portable virtual instrument receives and renders the lattice cloud data; and recalculate the optimal structured light parameters;
(3.6) the portable virtual instrument storing the set of structured light parameters; if the portable virtual instrument is traversed, performing a step (3.7); otherwise, repeating the step (3.3) -the step (3.6);
and (3.7) calculating the optimal structured light parameters by adopting a fusion method based on a weight model.
2. The method for calibrating structured light parameters according to claim 1, wherein the fusion method based on weight model uses SVM and CNN.
3. A method of calibrating a structured light parameter according to claim 1 or 2, wherein said structured light parameter comprises: camera focal length, principal point, and distortion coefficient of the imaging receiver; the equation of the plane of light for the laser transmitter and the positional relationship of the plane of light with respect to the imaging receiver.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010257728.8A CN111476847B (en) | 2020-04-03 | 2020-04-03 | Virtual instrument device for calibrating structured light parameters and method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010257728.8A CN111476847B (en) | 2020-04-03 | 2020-04-03 | Virtual instrument device for calibrating structured light parameters and method thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111476847A CN111476847A (en) | 2020-07-31 |
| CN111476847B true CN111476847B (en) | 2024-01-30 |
Family
ID=71750604
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010257728.8A Active CN111476847B (en) | 2020-04-03 | 2020-04-03 | Virtual instrument device for calibrating structured light parameters and method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111476847B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998004186A1 (en) * | 1996-07-31 | 1998-02-05 | Virtual-Eye.Com | Visual field testing method and apparatus |
| CN105783786A (en) * | 2016-04-26 | 2016-07-20 | 北方工业大学 | Part chamfering measuring method and device based on structured light vision |
| CN108961899A (en) * | 2018-07-10 | 2018-12-07 | 上海健康医学院 | A kind of X-ray machine Experiment of Electrical Circuits analogue system based on virtual instrument technology |
| CN110689581A (en) * | 2018-07-06 | 2020-01-14 | Oppo广东移动通信有限公司 | Structured light module calibration method, electronic device and computer readable storage medium |
| CN110927115A (en) * | 2019-12-09 | 2020-03-27 | 杭州电子科技大学 | Lens-free dual-type fusion target detection device and method based on deep learning |
-
2020
- 2020-04-03 CN CN202010257728.8A patent/CN111476847B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO1998004186A1 (en) * | 1996-07-31 | 1998-02-05 | Virtual-Eye.Com | Visual field testing method and apparatus |
| CN105783786A (en) * | 2016-04-26 | 2016-07-20 | 北方工业大学 | Part chamfering measuring method and device based on structured light vision |
| CN110689581A (en) * | 2018-07-06 | 2020-01-14 | Oppo广东移动通信有限公司 | Structured light module calibration method, electronic device and computer readable storage medium |
| CN108961899A (en) * | 2018-07-10 | 2018-12-07 | 上海健康医学院 | A kind of X-ray machine Experiment of Electrical Circuits analogue system based on virtual instrument technology |
| CN110927115A (en) * | 2019-12-09 | 2020-03-27 | 杭州电子科技大学 | Lens-free dual-type fusion target detection device and method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111476847A (en) | 2020-07-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107194962B (en) | Point cloud and plane image fusion method and device | |
| WO2021063127A1 (en) | Pose positioning method and related equipment of active rigid body in multi-camera environment | |
| CN107516326B (en) | Robot positioning method and system fusing monocular vision and encoder information | |
| CN112859859A (en) | Dynamic grid map updating method based on three-dimensional obstacle object pixel object mapping | |
| CN111899301A (en) | Workpiece 6D pose estimation method based on deep learning | |
| CN104778688A (en) | Method and device for registering point cloud data | |
| CN111413970A (en) | Ultra-wideband and vision integrated indoor robot positioning and autonomous navigation method | |
| JPH10124658A (en) | Method for correcting image distortion of camera by utilizing neural network | |
| CN111652113B (en) | Obstacle detection method, device, equipment and storage medium | |
| CN113610172B (en) | Neural network model training method and device and sensing data fusion method and device | |
| CN110838122A (en) | Point cloud segmentation method and device and computer storage medium | |
| CN111192364A (en) | Low-cost mobile multi-robot vision simultaneous positioning and map creating method | |
| KR102229861B1 (en) | Depth estimation apparatus and method using low-channel lidar and stereo camera | |
| CN109752724A (en) | A kind of image laser integral type navigation positioning system | |
| CN111476847B (en) | Virtual instrument device for calibrating structured light parameters and method thereof | |
| CN101052121A (en) | Dynamic calibrating method and system for video frequency system parameter | |
| Wohlfeil et al. | Automated high resolution 3D reconstruction of cultural heritage using multi-scale sensor systems and semi-global matching | |
| Xu et al. | Dual surface structured light vision system based on multidimensional parameter coding | |
| CN112927304B (en) | Fish-eye lens calibration method based on convolutional neural network | |
| CN115147497A (en) | Calibration method and device and electronic equipment | |
| Rashidizad et al. | Efficient point cloud lossless data compression method based on an embedded Gray code structured light pattern sequence | |
| Wang et al. | Three-dimensional reconstruction of mobile binocular stereo vision based on push-broom line structured light for a workpiece surface | |
| Zimmer et al. | PointCompress3D--A Point Cloud Compression Framework for Roadside LiDARs in Intelligent Transportation Systems | |
| CN115953477B (en) | Coordinate data processing method and device | |
| CN117523021B (en) | AGV control system-based rapid map scanning method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |