WO2022036477A1

WO2022036477A1 - Augmented reality three-dimensional registration method using fused visual features and model point cloud

Info

Publication number: WO2022036477A1
Application number: PCT/CN2020/109425
Authority: WO
Inventors: 王丹丁; 金星鉴; 徐天成; 袁铮
Original assignee: 江苏瑞科科技有限公司
Priority date: 2020-08-17
Filing date: 2020-08-17
Publication date: 2022-02-24

Abstract

An augmented reality three-dimensional registration method using fused visual features and model point cloud, comprising a three-dimensional tracking registration technique (1). The three-dimensional tracking registration technique (1) comprises a computer vision-based tracking registration technique (2), a hardware sensor-based tracking registration technique (3) and a hybrid tracking registration technique (4). The computer vision-based tracking registration technique (2) comprises a marker-based tracking registration technique (5) and a marker-free tracking registration technique (6). The marker-free tracking registration technique (6) comprises a natural feature-based tracking registration technique (7), a model-based tracking registration technique (8) and a parallel reconstruction-based (9) tracking registration technique. Said method facilitates the combined usage between complementary fusion (14), competitive fusion (15) and cooperative fusion (16) by means of the setting of the hybrid tracking registration (4). The provision of a hardware sensor-based tracking registration technique (3) facilitates the combined usage of a mechanical sensor-based (10) tracking registration technique, a global positioning system-based tracking registration technique (11), an inertial sensor-based tracking registration technique (12) and an electromagnetic sensor-based tracking registration technique (13), thereby better improving the working efficiency and facilitating the use.

Description

An Augmented Reality 3D Registration Method Integrating Visual Features and Model Point Clouds

technical field

The invention relates to the technical field of three-dimensional registration, in particular to an augmented reality three-dimensional registration method integrating visual features and model point clouds.

Background technique

With the development of science and technology, the technology of 3D registration is getting better and better. An augmented reality 3D registration method that integrates visual features and model point clouds is a kind of connection between fusion visual features and model point clouds. A technology for a 3D registration method in augmented reality; an existing augmented reality 3D registration method that fuses visual features and model point clouds is inconvenient to use in a hybrid tracking registration module, and there is no convenient method that is easy to use. , In the process of using the 3D registration method, the hardware sensor system is relatively simple, and there are no mechanical, inertial and electromagnetic sensors, which are inconvenient to use and connect. Therefore, we propose an augmented reality 3D system that combines visual features and model point clouds. registration method.

technical problem

The main purpose of the present invention is to provide an augmented reality three-dimensional registration method integrating visual features and model point clouds, which can effectively solve the problems in the background art.

technical solutions

In order to achieve the above purpose, the technical solution adopted by the present invention is as follows.

An augmented reality three-dimensional registration method that integrates visual features and model point clouds, including three-dimensional tracking registration technology, the three-dimensional tracking registration technology includes computer vision-based, hardware sensor-based and hybrid tracking registration, and the computer vision-based includes marker-based registration. and marker-free, including natural feature-based, model-based, and parallel reconstruction-based hardware-based sensors including mechanical-based sensors, global positioning system-based, inertial sensors, and electromagnetic-based sensors, the hybrid Track registration includes complementary fusion, competitive fusion and collaborative fusion.

Preferably, the three-dimensional tracking registration technology and the computer vision-based, hardware sensor-based and hybrid tracking registrations are all interconnected, and the computer vision-based, hardware sensor-based and hybrid tracking registrations are all signal connections.

By adopting the above technical solutions, the following technical effects can be achieved: facilitating the connection between three-dimensional tracking and registration technology, based on computer vision, based on hardware sensors, and hybrid tracking and registration.

Preferably, the computer vision-based and marker-based and marker-free are interconnected, the marker-based and marker-free are signal connections, and the marker-free is connected to natural feature-based, model-based and Interconnections are based on parallel reconstructions.

By adopting the above technical solutions, the following technical effects can be achieved: facilitating the connection between computer vision-based and marker-based and marker-free.

Preferably, the hardware-based sensor and the mechanical-based sensor, global positioning system-based, inertial sensor-based and electromagnetic-based sensor are all interconnected, and the mechanical-based sensor, global positioning system-based, inertial sensor-based and The electromagnetic sensors are all signal connections.

By adopting the above technical solutions, the following technical effects can be achieved: the effect of facilitating the connection between hardware-based sensors and mechanical-based sensors, global positioning system-based, inertial sensors and electromagnetic-based sensors.

Preferably, the hybrid tracking registration and the complementary fusion, the competitive fusion and the cooperative fusion are interconnected, and the complementary fusion, the competitive fusion and the cooperative fusion are all signal connections.

By adopting the above technical solution, the following technical effects can be achieved: facilitating the signal connection between the hybrid tracking registration and the complementary fusion, the competitive fusion and the cooperative fusion.

Preferably, the mechanical sensors, the global positioning system, the inertial sensors, and the electromagnetic-based sensors are all conductive connections, and the mechanical sensors, the inertial sensors, and the electromagnetic-based sensors are all hardware devices. Described based on the global positioning system as a software device.

By adopting the above technical solutions, the following technical effects can be achieved: facilitating the connection and use of mechanical sensors, global positioning systems, inertial sensors and electromagnetic sensors.

beneficial effect

Compared with the prior art, the present invention has the following beneficial effects: in the augmented reality three-dimensional registration method integrating visual features and model point clouds, the user connects the computer terminal with the hand-eye voice interaction module, and transmits the signal to the analog signal for sending. The module transmits the signal to the analog signal receiving module through the analog signal sending module, the analog signal receiving module transmits the signal to the analog signal exchange module, and the analog signal exchange module transmits the signal to the cooperative assembly module, which is conducive to the transmission of the signal and is easy to use. In the working project of the collaborative assembly module, the system monitoring module will monitor the signal transmission of the computer terminal, and after monitoring the information, it will be transmitted to the user receiving module and the administrator receiving module at the same time, so that the user receiving module and the administrator receiving module can talk about the correctness at the same time. The computer terminal intercepts and limits its transmission, which has a strong practical effect and a better use effect.

Description of drawings

FIG. 1 is a schematic structural diagram of an augmented reality three-dimensional registration method integrating visual features and model point clouds according to the present invention.

In the figure: 1. 3D tracking registration technology; 2. Based on computer vision; 3. Based on hardware sensors; 4. Hybrid tracking and registration; 5. Based on markers; 6. Without markers; 7. Based on natural features; 8. Based on models 9. Based on parallel reconstruction; 10. Based on mechanical sensors; 11. Based on GPS; 12. Based on inertial sensors; 13. Based on electromagnetic sensors; 14. Complementary fusion; 15. Competitive fusion; 16. Collaborative fusion.

Embodiments of the present invention

In order to make the technical means, creative features, achievement goals and effects realized by the present invention easy to understand, the present invention will be further described below with reference to the specific embodiments.

As shown in Figure 1, an augmented reality 3D registration method that integrates visual features and model point clouds, including 3D tracking registration technology 1, 3D tracking registration technology 1 includes computer vision-based 2, based on hardware sensors 3 and hybrid tracking registration 4, Computer vision-based 2 includes marker-based 5 and marker-free 6, marker-free 6 includes natural feature-based 7, model-based 8 and parallel reconstruction 9, hardware-based sensor 3 includes mechanical sensor-based 10, GPS-based 11 , based on inertial sensors 12 and based on electromagnetic sensors 13 , the hybrid tracking registration 4 includes complementary fusion 14 , competitive fusion 15 and cooperative fusion 16 .

3D tracking technology 1 and computer vision-based 2, hardware-based sensor 3, and hybrid tracking system 4 are all connected to each other, and computer vision-based 2, hardware-based sensor 3, and hybrid tracking system 4 are all signal connections. It is convenient for the 3D tracking registration technology 1 to be used in connection with signals based on computer vision 2 and based on hardware sensors 3 .

Based on computer vision 2 and based on markers 5 and without markers 6 are interconnected, based on markers 5 and without markers 6 are signal connections, without markers 6 and based on natural features 7, based on models 8 and based on The parallel reconstructions 9 are interconnected, which is beneficial to the use of computer vision-based 2 and marker-based 5 and marker-free 6 to achieve better results.

The hardware-based sensor 3 and the mechanical-based sensor 10, the global positioning system-based 11, the inertial sensor-based 12, and the electromagnetic-based sensor 13 are all interconnected. The signal connection between the hardware-based sensor 3 and the mechanical-based sensor 10 , the GPS-based system 11 , the inertial sensor 12 and the electromagnetic-based sensor 13 are all signal connections.

Hybrid tracking registration 4 is interconnected with complementary fusion 14, competitive fusion 15 and cooperative fusion 16. Complementary fusion 14, competitive fusion 15 and cooperative fusion 16 are all signal connections, which is beneficial to hybrid tracking. Signal transmission between registration 4 and complementary fusion 14 , competitive fusion 15 and cooperative fusion 16 works well.

The mechanical-based sensor 10 , the GPS-based sensor 11 , the inertial sensor-based 12 , and the electromagnetic-based sensor 13 are all conductive connections, and the mechanical-based sensor 10 , the inertial sensor-based 12 , and the electromagnetic-based sensor 13 are all hardware devices. , based on the global positioning system 11 is a software device, which is beneficial to the use of the mechanical based sensor 10 , the global positioning system based 11 , the inertial sensor based 12 and the electromagnetic based sensor 13 .

It should be noted that the present invention is an augmented reality three-dimensional registration method that integrates visual features and model point clouds. When in use, the user combines the three-dimensional tracking registration technology 1 with computer vision-based 2, hardware sensor-based 3 and hybrid tracking registration. 4 is connected so that the command based on the computer vision 2 is sent between the marker 5 and the marker-free 6, and the signal is sent to the natural feature-based when the command is sent based on the computer vision 2 and the marker-free 6 receives the signal. 7. Based on the model 8 and based on the parallel reconstruction 9, based on the natural features 7, based on the model 8 and based on the parallel reconstruction 9, the interconnection and detection are used, so that the hardware based sensor 3 sends the signal to the mechanical based sensor 10, based on the global The positioning system 11, based on the inertial sensor 12 and based on the electromagnetic sensor 13, makes the power-on detection based on the mechanical sensor 10, based on the inertial sensor 12 and based on the electromagnetic sensor 13, and transmits the signal to the location based on the global positioning system 11, based on the global positioning system 11. The positioning system 11 collects and sends signals to the hybrid tracking register 4, which is convenient for use and improves work efficiency. When the GPS 11 sends the signal to the hybrid tracking register 4, the hybrid tracking register 4 branches and sends the signal to the complementary fusion 14, the competitive fusion 15 and the cooperative fusion 16 respectively, so that the three-dimensional register can be used for real-time tracking and practical. Sexual effect is strong.

The basic principles and main features of the present invention and the advantages of the present invention have been shown and described above. Those skilled in the art should understand that the present invention is not limited by the above-mentioned embodiments, and the descriptions in the above-mentioned embodiments and the description are only to illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have Various changes and modifications fall within the scope of the claimed invention. The claimed scope of the present invention is defined by the appended claims and their equivalents.

Claims

An augmented reality three-dimensional registration method integrating visual features and model point clouds, comprising a three-dimensional tracking and registration technology (1), characterized in that: the three-dimensional tracking and registration technology (1) includes computer vision-based (2), hardware sensor-based ( 3) and hybrid tracking registration (4), the computer vision-based (2) includes marker-based (5) and marker-free (6), the marker-free (6) includes natural feature-based (7), Model (8) and based on parallel reconstruction (9), the hardware-based sensors (3) include mechanical-based sensors (10), global positioning system-based (11), inertial sensors (12) and electromagnetic-based sensors (13) ), the hybrid tracking registration (4) includes complementary fusion (14), competitive fusion (15) and cooperative fusion (16).
The augmented reality three-dimensional registration method integrating visual features and model point clouds according to claim 1, characterized in that: the three-dimensional tracking registration technology (1) is combined with computer vision-based (2) and hardware sensor-based (3) and the hybrid tracking register (4) are all interconnected, and the computer vision-based (2), hardware sensor-based (3) and the hybrid tracking register (4) are all signal connections.
The augmented reality three-dimensional registration method integrating visual features and model point clouds according to claim 1, characterized in that: the computer vision-based (2), the marker-based (5) and the marker-free (6) are are interconnected, the marker-based (5) and the marker-free (6) are signal connections, the marker-free (6) is connected to the natural feature-based (7), model-based (8) and parallel-based Reconstruction (9) is interconnected.
The augmented reality three-dimensional registration method integrating visual features and model point clouds according to claim 1, characterized in that: the hardware sensor (3), mechanical sensor (10), global positioning system (11)-based ), inertial-based sensors (12) and electromagnetic-based sensors (13) are interconnected, the mechanical-based sensors (10), global positioning system-based (11), inertial-based sensors (12) and electromagnetic-based All the sensors (13) are connected by signals.
The augmented reality three-dimensional registration method for fusing visual features and model point clouds according to claim 1, characterized in that: the hybrid tracking registration (4) is combined with complementary fusion (14), competitive fusion (15) and The cooperative fusions (16) are interconnected, and the complementary fusions (14), the competitive fusions (15) and the cooperative fusions (16) are all signal connections.
The augmented reality three-dimensional registration method integrating visual features and model point clouds according to claim 1, characterized in that: said mechanical sensor (10), global positioning system (11), inertial sensor (12) ) and the electromagnetic-based sensor (13) are all conductive connections, and the mechanical-based sensor (10), the inertial sensor (12) and the electromagnetic-based sensor (13) are all hardware devices, and the GPS-based (11) is a software device.