CN111915745A - Underground pipeline display device based on augmented reality - Google Patents

Underground pipeline display device based on augmented reality Download PDF

Info

Publication number
CN111915745A
CN111915745A CN202010960670.3A CN202010960670A CN111915745A CN 111915745 A CN111915745 A CN 111915745A CN 202010960670 A CN202010960670 A CN 202010960670A CN 111915745 A CN111915745 A CN 111915745A
Authority
CN
China
Prior art keywords
module
augmented reality
information
display
pipeline
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010960670.3A
Other languages
Chinese (zh)
Inventor
王光东
施勇
潘晓波
施卫锋
吴�荣
邹斌
朱武
季新风
周宇峰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Shanghai Electric Power Co Ltd
Original Assignee
State Grid Shanghai Electric Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Shanghai Electric Power Co Ltd filed Critical State Grid Shanghai Electric Power Co Ltd
Priority to CN202010960670.3A priority Critical patent/CN111915745A/en
Publication of CN111915745A publication Critical patent/CN111915745A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/003—Navigation within 3D models or images
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
    • G01S19/01—Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
    • G01S19/13—Receivers
    • G01S19/33—Multimode operation in different systems which transmit time stamped messages, e.g. GPS/GLONASS
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics
    • G06T19/006—Mixed reality

Abstract

The invention relates to an underground pipeline display device based on augmented reality. The information acquisition unit, the storage unit and the correction unit take the acquired data as input information of the augmented reality processing unit. The augmented reality processing unit processes the input information to obtain a mixed image, and the mixed image is output to the AR display unit, and the AR display unit realizes the display of the mixed image and the display of the system state information. This pipeline display system based on augmented reality has combined camera technology and augmented reality technology, has realized the visual real-time display of pipeline, effectively avoids the unnecessary pipeline destruction and the bodily injury that blind worker produced, provides convenience for the pipeline overhauls simultaneously.

Description

Underground pipeline display device based on augmented reality
Technical Field
The invention relates to an underground pipeline display system based on augmented reality, which is used in the field of underground pipeline engineering.
Background
Augmented reality technology is a generic term for technology that combines virtual graphics with real scenes. Augmented reality technology generally has three characteristics of virtual-real combination, real-time interaction and three-dimensional registration. The virtual-real combination refers to the combination of virtual reality information and real world scenes, and the realization path comprises three types, namely optical perspective, video perspective and light field projection. The three-dimensional registration refers to positioning and tracking of the augmented reality equipment on a real scene, and the equipment is made to know the position change of a key object on the real scene through a sensor and track the object. Tomcoaudell of boeing corporation and his company have proposed the concept of augmented reality as early as the 90's of the 20 th century. Later 1994, paul milgram and FumioKishino suggested that there was "mixed reality" in the middle of the virtual environment and the real world, where what is closer to the real world is augmented reality, which is a technology for augmenting the real environment by generating various virtual objects by a computer.
In recent years, with the rapid improvement of computer computing capability, the rapid development of computer graphics and the wide application of machine vision, the augmented reality technology has gradually become a research hotspot of scholars. The application field of augmented reality is more and more extensive, and the augmented reality has better performance in the fields of medical treatment, education, tourism, entertainment and the like. The development and application of augmented reality technology are inevitable trends in technology development.
Currently, no augmented reality-based underground pipeline display device exists in the market. Before construction, the underground pipeline needs to be checked in the field, the position and the depth of the underground pipeline are determined, and then the construction can be planned. This results in a prolonged period and waste of social resources. The cable boundary plate is required to be searched firstly in the process of overhauling and maintaining the underground pipeline, the position of the pipeline is determined, and then the position of the ground well cover is searched, so that the time waste and the labor cost increase are caused.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides an augmented reality-based underground pipeline display device, which can realize real-time display of underground pipelines and enables users to see the distribution of the underground pipelines nearby by holding equipment with hands.
One technical scheme for achieving the above purpose is as follows:
the utility model provides an underground pipeline display device based on augmented reality, includes information acquisition unit, augmented reality processing unit, memory cell, correction unit, AR display element, its characterized in that: the output ends of the information acquisition unit, the storage unit and the correction unit are all connected with the input end of the augmented reality processing unit, and the output end of the augmented reality processing unit is connected with the input end of the AR display unit; the augmented reality processing unit comprises a pipeline information processing module, a system configuration module, a coordinate transformation module and an image synthesis module; the information acquisition unit comprises a camera, a GPS module and a gyroscope module and is used for acquiring a real scene, space positioning information and attitude information of the mobile terminal; the output end of the camera is connected with the input end of the image synthesis module, the output end of the GPS module is connected with the input end of the pipeline information processing module, the output end of the GPS module is connected with the input end of the system configuration module, and information fusion is carried out on the real scene, the space positioning information and the posture information of the mobile terminal acquired by the camera in the image synthesis module to generate a pipeline visual image; the output end of the gyroscope module is connected with the input end of the system configuration module; the storage unit stores a pipeline map file; the correction unit comprises a correction module for user input;
the information acquisition unit, the storage unit and the correction unit take the acquired data as input information of the augmented reality processing unit; the augmented reality processing unit processes the input information to obtain a mixed image, and the mixed image is output to the AR display unit, and the AR display unit realizes the display of the mixed image and the display of the system state information.
Further, the camera is mounted on the back of the AR display unit, so that an image obtained by shooting is the same as an image obtained by observing a visual angle of a person; the GPS module is used for providing position information of the camera and is arranged beside the camera; the gyroscope module is used for providing direction and elevation angle information of the camera and is positioned on the same plane with the camera.
Furthermore, the display unit comprises a mixed image display module and a state information display module, the mixed image display module is connected with the augmented reality processing unit and used for displaying the final composite image, and the state information display module is connected with the augmented reality processing unit and used for displaying the system configuration information and the system state information.
Compared with the prior art, the invention provides an underground pipeline display device based on augmented reality, which has the following beneficial effects:
1. this underground pipeline display system based on augmented reality, the realization mode is nimble. The method can be used for safely and efficiently realizing the visual display of the underground pipeline by using the ARM chip and peripheral equipment, and can also be used for reducing the total cost of system realization by only developing a software system without hardware cost by using a mobile phone platform.
2. According to the underground pipeline display system based on augmented reality, the underground pipeline map is combined with the real scene through the fusion of the underground pipeline map and the augmented reality technology, the real-time display of the underground pipeline is realized, and a user can see and obtain the underground pipeline, so that the risk of damage to the pipeline is reduced, and the construction cost is reduced; the maintenance of the underground pipeline is facilitated, and the maintenance cost is reduced.
Drawings
FIG. 1 is a schematic view of an augmented reality-based underground utility display apparatus according to the present invention;
fig. 2 is a schematic diagram illustrating information interaction among modules of an augmented reality-based underground pipeline display device according to the present invention.
Detailed Description
In order to better understand the technical solution of the present invention, the following detailed description is made by specific examples:
please refer to fig. 1 and fig. 2. The invention discloses an underground pipeline display device based on augmented reality. The input of augmented reality processing unit is connected to information acquisition unit's output electricity, augmented reality processing unit's input is connected to memory cell's output electricity, the input of augmented reality processing unit is connected to correction unit's output electricity, augmented reality processing unit's output electricity connect AR display element's input.
The information acquisition unit comprises a camera, a GPS module and a gyroscope module, the camera adopts a model 2072, the output resolution is 1280 x 720, 30 frames are output in each second, the focal length is 6mm, the visual angle is 55 degrees, the visiting distance is 0 to 10 meters, meanwhile, a distortion-free lens is configured and hung on a CPU bus, and the camera is installed on the back of the AR display unit, so that the image obtained by shooting is the same as the image obtained by observing the visual angle by people. The GPS module adopts an HLK-GS475 module, the hot start capturing time is 1 second, the positioning precision is 0.1m, positioning signals of a GPS, a GLONASS, a BDS, a GAL and an SBAS satellite system can be received at the same time, the positioning signals are mounted on a CPU bus through a serial SPI protocol, the GPS module is mounted beside a camera, and the positioning effect is more accurate when the distance between the GPS module and the camera is closer. The gyroscope module is JY901S type, can provide information such as acceleration, angular velocity, angle, magnetic field and the like, is mounted on the CPU bus through a UART interface, and is arranged on the same plane with the camera in order to enable the gyroscope module to accurately provide direction and elevation angle information of the camera.
The output end of the camera is electrically connected with the input end of the image synthesis module, the output end of the GPS module is electrically connected with the input end of the pipeline information processing module, the output end of the GPS module is electrically connected with the input end of the system configuration module, and the output end of the gyroscope module is electrically connected with the input end of the system configuration module.
The storage unit primarily includes pipeline coordinate information. The storage unit adopts a MicroSD card module which provides an interface connection function between the SD card and the CPU and mounts the SD card on a CPU peripheral bus through an SPI protocol. The SD card adopts 4Gclass10, the highest reading speed is 95MB/s, and the highest writing speed is 10 MB/s. The pipeline node information is stored in the SD card in a structure.
The augmented reality processing unit comprises a pipeline information processing module, a system configuration module, a coordinate transformation module and an image synthesis module, the main functions of the augmented reality processing unit are realized by a CPU, and an STM32F407ZGT6 is selected as a main CPU according to actual requirements. The CPU has a main frequency as high as 168MHz, an MPU can be as high as 210DMIPS/MHz, 1MB Flash, three modes of sleep, shutdown and standby, 4 UART interfaces, three SPI interfaces and 8-to-14-bit parallel camera interfaces, so that the requirements are met, and a function expansion space exists.
The pipeline information processing module provides coordinates (x) through a GPS modulec,yc) A camera position is determined. Reading the coordinate information of the pipeline 50 meters around the coordinate namely
Figure BDA0002680428280000041
L<50 m. Wherein L represents the scanning radius (x)i,yi) Representing the coordinates of the ith pipeline node. And reading the pipeline nodes meeting the conditions, connecting the corresponding nodes according to the sequence information of the nodes, and drawing a pipeline map.
The system configuration module reads configuration information of related hardware during program initialization, including but not limited to resolution of a camera, focal length of the camera, position information of the camera, direction of the camera, included angle between the camera and a horizontal line and the like. And acquiring necessary hardware information through a system configuration module to prepare for coordinate transformation.
The coordinate transformation module mainly realizes the transformation of a point pixel coordinate system in a world coordinate system. The coordinate transformation formula is as follows:
Figure BDA0002680428280000051
in the formula ZcFor a real world point to optical center distance, when the object distance is greater than 300mm, it can be expressed as:
Figure BDA0002680428280000052
in the formula Xw,Yw,ZwCoordinates of a point in the real world, h2Is the origin height difference of the camera and real world coordinates. CinIs a camera internal reference matrix, which can be expressed as:
Figure BDA0002680428280000053
in the formula fx,fyIs the component of the focal length in two axes of the two-dimensional image when projected. u. of0,v0Is the origin of coordinates of the imaging plane. CoutAs a camera external parameter, it can be expressed as:
Figure BDA0002680428280000054
where R represents 3 rotation parameters when the real coordinate system is converted into the camera coordinate system, and T represents 3 translation parameters when the real coordinate system is converted into the camera coordinate system. In summary, 10 parameters are required to be provided in the coordinate transformation formula, and the 10 parameters can be provided by the system configuration module.
The augmented reality processing unit is connected with the storage unit, the information acquisition unit and the correction unit and is respectively used for determining the distribution condition of underground pipelines according to a pipeline map, determining the scene in front of a user according to the camera, determining the position of the camera according to the GPS module, determining the direction and the horizontal direction included angle of the camera according to the gyroscope module, determining the height and partial system information of the camera according to the correction module, and the augmented reality processing unit is connected with the AR display unit and inputs the synthesized image into the AR display unit.
The reality processing unit comprises a pipeline information processing module, a user configuration information module, a coordinate transformation module and an image synthesis module; the pipeline information processing module acquires the current coordinate from the GPS module, reads the information of nearby pipeline sampling points from the storage unit and constructs a two-dimensional pipeline network map; the user configuration module acquires equipment height information, a camera wide angle, camera magnification information, screen resolution information and screen size information provided by a user from the correction module; the coordinate transformation module generates a pipeline map under the view angle of the camera according to system information provided by the system configuration module, a pipeline map provided by the pipeline information processing module, a direction and an elevation angle provided by the gyroscope module and coordinate information provided by the GPS module; and the image synthesis module superposes the pipeline map of the camera visual angle provided by the coordinate transformation module and the real-time image provided by the camera to form a real image.
The image synthesis module is used for mutually superposing the pipeline map subjected to coordinate transformation and a real image acquired by the camera to synthesize an augmented reality image.
The correction unit mainly comprises a correction module used for acquiring equipment height information provided by a user and providing a user interface, and when the automatic reading of the equipment information fails or has deviation, the user can input correct configuration information through the correction module so as to improve the accuracy of the pipeline map.
The AR display unit comprises two parts of mixed image display and state information display, wherein the specific hardware is an AML-FRD500A3901-CTP display screen, the size of the display screen is 5.0 inches, the resolution is 1280 x 720, the main control chip is an ILI9881C-ODA, the color width is 262K, and the display screen is mounted on the bus through an FSMC interface. And the CPU sends the processed composite image and the system state to an LCD display screen through an FSMC interface to realize an image display function. The display unit comprises a mixed image display module and a state information display module, the mixed image display module is electrically connected with the image synthesis module, the enhanced image is displayed, interaction with a user is completed, and visual display of the underground pipeline is achieved. The state information display module is electrically connected with the system configuration module and displays the system information corrected by the user, so that the user can know various information of the equipment conveniently and the running state of the system can be monitored.
In conclusion, the underground pipeline display system based on augmented reality can be manufactured into a special instrument by using an ARM chip and peripheral equipment, so that the visual display of the underground pipeline is safely and efficiently realized. According to the underground pipeline display system based on augmented reality, the underground pipeline map is combined with the real scene through the fusion of the underground pipeline map and the augmented reality technology, the real-time display of the underground pipeline is realized, and a user can see and obtain the underground pipeline, so that the risk of damage to the pipeline is reduced, and the construction cost is reduced; the maintenance of the underground pipeline is facilitated, and the maintenance cost is reduced.
It should be understood by those skilled in the art that the above embodiments are only for illustrating the present invention and are not to be used as a limitation of the present invention, and that changes and modifications to the above described embodiments are within the scope of the claims of the present invention as long as they are within the spirit and scope of the present invention.

Claims (3)

1. The utility model provides an underground pipeline display device based on augmented reality, includes information acquisition unit, augmented reality processing unit, memory cell, correction unit, AR display element, its characterized in that: the output ends of the information acquisition unit, the storage unit and the correction unit are all connected with the input end of the augmented reality processing unit, and the output end of the augmented reality processing unit is connected with the input end of the AR display unit; the augmented reality processing unit comprises a pipeline information processing module, a system configuration module, a coordinate transformation module and an image synthesis module; the information acquisition unit comprises a camera, a GPS module and a gyroscope module and is used for acquiring a real scene, space positioning information and attitude information of the mobile terminal; the output end of the camera is connected with the input end of the image synthesis module, the output end of the GPS module is connected with the input end of the pipeline information processing module, the output end of the GPS module is connected with the input end of the system configuration module, and information fusion is carried out on the real scene, the space positioning information and the posture information of the mobile terminal acquired by the camera in the image synthesis module to generate a pipeline visual image; the output end of the gyroscope module is connected with the input end of the system configuration module; the storage unit stores a pipeline map file; the correction unit comprises a correction module for user input;
the information acquisition unit, the storage unit and the correction unit take the acquired data as input information of the augmented reality processing unit; the augmented reality processing unit processes the input information to obtain a mixed image, and the mixed image is output to the AR display unit, and the AR display unit realizes the display of the mixed image and the display of the system state information.
2. The underground pipeline display device based on the augmented reality as claimed in claim 1, wherein: the camera is arranged on the back of the AR display unit, so that an image obtained by shooting is the same as an image obtained by observing a visual angle of a person; the GPS module is used for providing position information of the camera and is arranged beside the camera; the gyroscope module is used for providing direction and elevation angle information of the camera and is positioned on the same plane with the camera.
3. The underground pipeline display device based on the augmented reality as claimed in claim 1, wherein: the display unit comprises a mixed image display module and a state information display module, the mixed image display module is connected with the augmented reality processing unit and used for displaying the final composite image, and the state information display module is connected with the augmented reality processing unit and used for displaying system configuration information and system state information.
CN202010960670.3A 2020-09-14 2020-09-14 Underground pipeline display device based on augmented reality Pending CN111915745A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010960670.3A CN111915745A (en) 2020-09-14 2020-09-14 Underground pipeline display device based on augmented reality

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010960670.3A CN111915745A (en) 2020-09-14 2020-09-14 Underground pipeline display device based on augmented reality

Publications (1)

Publication Number Publication Date
CN111915745A true CN111915745A (en) 2020-11-10

Family

ID=73267910

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010960670.3A Pending CN111915745A (en) 2020-09-14 2020-09-14 Underground pipeline display device based on augmented reality

Country Status (1)

Country Link
CN (1) CN111915745A (en)

Similar Documents

Publication Publication Date Title
CN112894832B (en) Three-dimensional modeling method, three-dimensional modeling device, electronic equipment and storage medium
CN107223269B (en) Three-dimensional scene positioning method and device
US11557083B2 (en) Photography-based 3D modeling system and method, and automatic 3D modeling apparatus and method
US7301536B2 (en) Electro-optic vision systems
US6064398A (en) Electro-optic vision systems
US11748906B2 (en) Gaze point calculation method, apparatus and device
US20230037922A1 (en) Image display method and apparatus, computer device, and storage medium
CN112288853B (en) Three-dimensional reconstruction method, three-dimensional reconstruction device, and storage medium
CN106705964A (en) Panoramic camera fused IMU, laser scanner positioning and navigating system and method
Gomez-Jauregui et al. Quantitative evaluation of overlaying discrepancies in mobile augmented reality applications for AEC/FM
CN111337037B (en) Mobile laser radar slam drawing device and data processing method
US10672191B1 (en) Technologies for anchoring computer generated objects within augmented reality
CN111811462A (en) Large-component portable visual ranging system and method in extreme environment
CN112270702A (en) Volume measurement method and device, computer readable medium and electronic equipment
CN108401551B (en) Twin-lens low-light stereoscopic full views imaging device and its ultra-large vision field distance measuring method
US8451346B2 (en) Optically projected mosaic rendering
US8509522B2 (en) Camera translation using rotation from device
CN114283243A (en) Data processing method and device, computer equipment and storage medium
JP2002230586A (en) Image system for supporting three-dimensional survey using composite reality
CN114298946A (en) Deep learning point cloud completion method for enhancing frame details
JP4809134B2 (en) Image processing apparatus and processing method thereof
CN113496503B (en) Point cloud data generation and real-time display method, device, equipment and medium
CN212379884U (en) Underground pipeline display device based on augmented reality
CN113077500A (en) Panoramic viewpoint positioning and attitude determining method, system, equipment and medium based on plane graph
Zheng et al. Scanning scene tunnel for city traversing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination