CN112562092A - Electrical test intelligent interaction system based on AR technology - Google Patents
Electrical test intelligent interaction system based on AR technology Download PDFInfo
- Publication number
- CN112562092A CN112562092A CN202011536401.0A CN202011536401A CN112562092A CN 112562092 A CN112562092 A CN 112562092A CN 202011536401 A CN202011536401 A CN 202011536401A CN 112562092 A CN112562092 A CN 112562092A
- Authority
- CN
- China
- Prior art keywords
- unit
- mobile terminal
- technology
- terminal
- system based
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012360 testing method Methods 0.000 title claims abstract description 53
- 238000005516 engineering process Methods 0.000 title claims abstract description 37
- 230000003993 interaction Effects 0.000 title claims abstract description 33
- 230000000007 visual effect Effects 0.000 claims abstract description 24
- 238000004364 calculation method Methods 0.000 claims abstract description 18
- 230000002452 interceptive effect Effects 0.000 claims abstract description 18
- 230000001960 triggered effect Effects 0.000 claims abstract description 8
- 230000001360 synchronised effect Effects 0.000 claims abstract description 7
- 238000012545 processing Methods 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims description 25
- 238000004891 communication Methods 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 16
- 238000002474 experimental method Methods 0.000 claims description 9
- 239000004984 smart glass Substances 0.000 claims description 6
- 230000007613 environmental effect Effects 0.000 claims description 4
- 238000002372 labelling Methods 0.000 claims description 2
- -1 mobile phones Substances 0.000 claims 1
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 238000011161 development Methods 0.000 description 11
- 238000004422 calculation algorithm Methods 0.000 description 9
- 239000011521 glass Substances 0.000 description 8
- 238000013473 artificial intelligence Methods 0.000 description 7
- 238000000605 extraction Methods 0.000 description 5
- 230000003190 augmentative effect Effects 0.000 description 4
- 230000019771 cognition Effects 0.000 description 3
- 230000006378 damage Effects 0.000 description 3
- 238000012423 maintenance Methods 0.000 description 3
- 238000013024 troubleshooting Methods 0.000 description 3
- 208000027418 Wounds and injury Diseases 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000002708 enhancing effect Effects 0.000 description 2
- 230000004927 fusion Effects 0.000 description 2
- 208000014674 injury Diseases 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 206010063385 Intellectualisation Diseases 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013079 data visualisation Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000008030 elimination Effects 0.000 description 1
- 238000003379 elimination reaction Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 239000002360 explosive Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 210000001503 joint Anatomy 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
- G06T17/205—Re-meshing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1097—Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/02—Services making use of location information
- H04W4/024—Guidance services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04W—WIRELESS COMMUNICATION NETWORKS
- H04W4/00—Services specially adapted for wireless communication networks; Facilities therefor
- H04W4/30—Services specially adapted for particular environments, situations or purposes
- H04W4/33—Services specially adapted for particular environments, situations or purposes for indoor environments, e.g. buildings
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Computer Graphics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention relates to an electrical test intelligent interaction system based on AR technology, which comprises: 3D scanning modeling all-in-one machine (1): carrying out rapid scanning modeling on a laboratory space scene and electrical test equipment; mobile end (2): the system is used for acquiring field images and performing fast matching calculation processing, so that loading of a virtual scene is triggered and visual display is performed; expert terminal (3): the synchronous sharing mobile terminal (2) visually displays the picture and supports remote interactive operation on the shared picture; cloud (4): and establishing data connection with the mobile terminal (2) and the expert terminal (3) respectively. The invention is beneficial to the standard operation of the electrical test equipment and the accuracy and timeliness of fault disposal, and improves the efficiency; the visual and efficient help is provided for emergency evacuation of personnel in dangerous scenes, and the safety of equipment and personnel is guaranteed.
Description
Technical Field
The invention relates to the field of electrical test intellectualization, in particular to an electrical test interaction system based on AR technology.
Background
Augmented Reality (AR) is a new technology for seamlessly integrating real world information and virtual world information, and is to superimpose entity information originally in a certain time space range of the real world after simulation by scientific technologies such as computers and the like, apply virtual information to the real world, superimpose the real environment and virtual objects to the same picture or space in real time, and complement and superimpose the real environment and the virtual objects with each other. The augmented reality technology comprises new technologies and new means such as multimedia, three-dimensional modeling, real-time video display and control, multi-sensor fusion, cognitive reality, real scene recognition, real-time tracking and registration, scene fusion and the like.
The development trend of the new generation artificial intelligence (i.e. artificial intelligence 2.0) technology is the organic combination of AI (artificial intelligence) with AR, VR (Virtual Reality), life system, and electromechanical system. In the face of complex test equipment and strict operation rules, the data stream can arrive quickly and the information is consistent up and down based on the AI + AR technology, so that the working efficiency is improved; the real-time feedback and recording of the operation process and the result are realized through human-computer interaction; the rapid troubleshooting of the electrical equipment faults is realized through remote assistance and information sharing; emergency evacuation of workers is realized through virtual navigation; the controllable and traceable test operation and the related management process are realized through the continuous accumulation of the experience knowledge base.
In the electrical equipment test, the operation is strictly carried out according to the rules and the steps, a large amount of test phenomena and test data need to be recorded, and the high-voltage test needs supervision and guidance of professional technicians. The test operation has a direct influence on data recording, phenomenon observation and personnel safety. Meanwhile, the faults of the test equipment also need to be processed in time, so that the test process can be recovered as soon as possible. Accidents or disasters can be caused by improper test operation, and property and personal safety are harmed. In an emergency, a tester can be nervous and confused in action, an artificial intelligence technology needs to be introduced, and troubleshooting guidance and emergency treatment guidance are timely carried out through an intelligent interaction method in the electrical test process; without proper and timely guidance from emergency treatment, secondary equipment or personal damage may result. Therefore, AR and artificial intelligence technologies need to be introduced to realize intelligent interaction and state judgment, feedback and prompting in the electrical test process.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provide an electrical test intelligent interaction system based on the AR technology.
The invention is realized by the following technical scheme:
an electrical test intelligent interaction system based on AR technology, the system comprising:
3D scanning modeling all-in-one machine: carrying out rapid scanning modeling on a laboratory space scene and electrical test equipment;
a mobile terminal: the system is used for acquiring field images and performing fast matching calculation processing, so that loading of a virtual scene is triggered and visual display is performed;
an expert end: synchronously sharing the visual display picture of the mobile terminal and supporting the remote interactive operation on the shared picture;
cloud: and establishing data connection with the mobile terminal and the expert terminal respectively.
Preferably, the specific way of performing the fast scan modeling by the 3D scan modeling all-in-one machine is as follows: scanning a spatial scene of a laboratory and electrical test equipment, collecting environmental data, establishing a 3D model of each point location through a point cloud technology, and then splicing and optimizing the 3D models of the plurality of point locations to form a three-dimensional model of the spatial scene of the laboratory and a model of the electrical test equipment.
Preferably, the mobile terminal comprises an image acquisition unit, an identification calculation unit, a first communication unit and a mobile display unit;
the image acquisition unit acquires a field image;
the identification calculation unit identifies the field image acquired by the image acquisition unit and identifies related characteristic points and point cloud image objects preset in advance;
the method comprises the steps that a first communication unit requests and receives virtual information content related to an identification object from a cloud end;
and the mobile display unit receives the interactive instruction, acquires corresponding virtual information content according to the interactive instruction and superposes the virtual information content on the site image for visual display.
Preferably, the virtual information content comprises an operation guidance video; pictures, characters and voice of system data and equipment operation data visual display; animation of the operation procedures; 3D part models of internal structures can be clicked, exploded and disassembled and perspective; the operation results show a graph.
Preferably, the interactive instruction includes viewing and browsing the virtually superimposed information; operating the virtually overlapped 3D component model through a gesture instruction; and playing, pausing and closing the video and voice files which are virtually overlapped through a voice instruction.
Preferably, the mobile terminal further comprises a remote sharing unit, the remote sharing unit calls the expert terminal through the first communication unit, shares a visual display picture of the mobile terminal, performs information interaction with the expert terminal, and displays a remote interactive operation guide from the expert terminal through the mobile display unit.
Preferably, the mobile terminal performs synchronous recording in the remote sharing process and uploads the recorded data to the cloud for storage through the first communication unit.
Preferably, the mobile terminal further comprises an indoor virtual navigation unit, the image acquisition unit is aligned to the space to acquire space image information, the identification and calculation unit is used for rapidly calculating, comparing and determining position information of the mobile terminal, and the virtual navigation unit is triggered to call and display the escape sign and the escape route.
Preferably, the expert end comprises an expert display unit, a marking guidance unit and a second communication unit;
the expert display unit synchronously shares the visual display picture of the mobile terminal from the cloud terminal through the second communication unit;
the annotation guidance unit conducts remote operation guidance through the second communication unit.
Preferably, the mobile terminal and the expert terminal are configured to include any one or a combination of more of AR smart glasses, a mobile phone, a tablet and a microcomputer.
Compared with the prior art, the invention has the following advantages:
(1) according to the invention, an AR technology is adopted, an electrical test intelligent interaction system is built, the man-machine interaction of electrical test operation rules is realized, the virtual and real are combined, the intelligent degree is high, and the working efficiency is improved;
(2) according to the invention, by arranging the remote sharing unit, the intelligent judgment and remote expert assistance on the equipment state and the operation process are realized, the equipment fault is rapidly checked, the operation process and the result are fed back and recorded in real time, and the controllability and traceability of the test operation and the related management process are realized;
(3) the indoor virtual navigation unit is arranged, and workers are evacuated in an emergency manner through indoor virtual navigation, so that the injury of the workers is reduced, and the safety of test equipment and the safety of the workers are guaranteed.
Drawings
FIG. 1 is a block diagram of an electrical test intelligent interaction system based on AR technology according to the present invention;
in the figure, 1 is a 3D scanning and modeling all-in-one machine, 2 is a mobile terminal, 3 is an expert terminal, 4 is a cloud terminal, 21 is an image acquisition unit, 22 is an identification calculation unit, 23 is a first communication unit, 24 is a mobile display unit, 25 is a remote sharing unit, 26 is a virtual navigation unit, 31 is an expert display unit, 32 is an annotation guidance unit, and 33 is a second communication unit.
Detailed Description
The invention is described in detail below with reference to the figures and specific embodiments. Note that the following description of the embodiments is merely a substantial example, and the present invention is not intended to be limited to the application or the use thereof, and is not limited to the following embodiments.
As shown in fig. 1, an electrical experiment intelligent interaction system based on AR technology includes:
an electrical test intelligent interaction system based on AR technology, the system comprising:
3D scanning modeling all-in-one machine 1: carrying out rapid scanning modeling on a laboratory space scene and electrical test equipment;
and the mobile terminal 2: the system is used for acquiring field images and performing fast matching calculation processing, so that loading of a virtual scene is triggered and visual display is performed;
the expert terminal 3: the synchronous sharing mobile terminal 2 visually displays the display picture and supports remote interactive operation on the sharing picture;
cloud 4: and establishing data connection with the mobile terminal 2 and the expert terminal 3 respectively.
The mobile terminal 2 and the expert terminal 3 are configured to include any one or a combination of multiple of AR smart glasses, a mobile phone, a tablet, and a microcomputer, in this embodiment, the mobile terminal 2 is configured as AR smart glasses, and the expert terminal 3 is configured as a tablet (PAD).
The 3D scanning and modeling all-in-one machine 1 comprises two parts, namely hardware and software, and a tablet personal computer is used for carrying out operation control on the 3D scanning and modeling all-in-one machine 1. The 3D scanning and modeling all-in-one machine 1 scans field space and equipment, collects environmental data and rapidly establishes a 3D model of each point location through a point cloud technology. And then splicing and optimizing the 3D models of the multiple point positions through a program and an algorithm to form a complete and accurate laboratory space scene three-dimensional model and an electrical test equipment model, and presetting the model into the system program so as to quickly match the space information and the image information acquired by the AR SDK development kit and the AR intelligent glasses at the mobile terminal and establish a subsequent virtual-real combination and human-computer interaction augmented reality application scene triggering mechanism.
The mobile terminal 2 comprises an image acquisition unit 21, an identification calculation unit 22, a first communication unit 23 and a mobile display unit 24;
the image acquisition unit 21 acquires a live image;
the identification and calculation unit 22 identifies the field image acquired by the image acquisition unit 21 based on the AR SDK development kit, and identifies the related feature points and point cloud image objects preset in advance;
the first communication unit 23 requests and receives the virtual information content related to the recognition object from the cloud 4;
the mobile display unit 24 receives the interactive instruction, acquires corresponding virtual information content according to the interactive instruction, and superimposes the virtual information content on the live image for visual display.
The mobile terminal 2 is aligned with the electrical test space and the electrical test equipment, the images of the electrical test space and the electrical test equipment are collected to the mobile display unit 24 through the image collection unit 21, virtual information superposed on the real electrical test equipment is triggered, and reading is carried out through the mobile display unit 24 or interaction is carried out with a superposed virtual scene and expanded contents.
The virtual information content comprises an operation guide video; pictures, characters and voice of system data and equipment operation data visual display; animation of the operation procedures; 3D part models of internal structures can be clicked, exploded and disassembled and perspective; the operation results show a graph.
The interactive instruction comprises the operation of viewing and browsing the virtually superposed information; operating the virtually overlapped 3D component model through a gesture instruction, wherein the operation comprises previewing, amplifying, shrinking, rotating, moving and explosive disassembling; and playing, pausing and closing the video and voice files which are virtually overlapped through a voice instruction.
The mobile terminal 2 further comprises a remote sharing unit 25, the remote sharing unit 25 calls the expert terminal 3 through the first communication unit 23, shares a visual display picture of the mobile terminal 2, performs information interaction with the expert terminal 3, and displays a remote interactive operation guide from the expert terminal 3 through the mobile display unit 24.
The mobile terminal 2 performs synchronous recording of the remote sharing process and also performs synchronous recording of the sharing process, and uploads the recorded data to the cloud terminal 4 through the first communication unit 23 for storage.
The mobile terminal 2 further comprises an indoor virtual navigation unit 26, the image acquisition unit 21 is aligned to the space to acquire space image information, the identification calculation unit 22 is used for rapidly calculating, comparing and determining position information of the mobile terminal, and the virtual navigation unit 26 is triggered to call and display the escape sign and the escape route.
The expert end 3 comprises an expert display unit 31, a marking guidance unit 32 and a second communication unit 33;
the expert display unit 31 synchronously shares the visual display picture of the mobile terminal 2 from the cloud terminal 4 through the second communication unit 33;
the annotation guidance unit 32 performs remote operation guidance through the second communication unit 33.
The system workflow example is as follows:
1) and (3) carrying out rapid scanning modeling on a spatial scene and electrical test equipment of the laboratory by using the 3D scanning modeling all-in-one machine 1. The 3D scanning and modeling all-in-one machine 1 comprises two parts, namely hardware and software, and a tablet personal computer is used for operating and controlling the 3D scanning and modeling all-in-one machine 1. The 3D scanning and modeling all-in-one machine 1 scans field space and equipment, collects environmental data, and rapidly establishes a 3D model of each point location through an optical radar point cloud technology. And then, point cloud 3D models of a plurality of point locations are spliced and optimized into a triangular patch mesh model through a program and an algorithm to form a complete and accurate laboratory space scene three-dimensional model and an electrical test equipment model, and the complete and accurate laboratory space scene three-dimensional model and the electrical test equipment model are preset in the system program, so that space information and image information acquired by AR intelligent glasses of the mobile terminal 2 are quickly matched through an AR SDK development kit, and a follow-up virtual-real combination and man-machine interaction augmented reality application scene triggering mechanism is established.
An AR algorithm engine of the AR SDK development kit adopts a space-time cognition technology based on object video stream, does not use mark points and two-dimensional code auxiliary marks which play a supporting role in the traditional AR, but uniformly processes targets with different dimensions such as points, lines, surfaces, bodies, scenes and the like in space to form integral cognition on an object shot by a camera; after the time dimension is added, the space-time two coordinate systems are further fused to form the omnibearing three-dimensional AR supporting capability of characteristic quantity extraction, object identification, target tracking, space positioning, movement track extraction and the like, and a software development bottom layer foundation can be provided for typical AR applications such as sign positioning display, multi-display, SLAM, path planning, shooting tracking, three-dimensional coverage and the like.
An algorithm engine of the AR SDK development kit adopts a heterogeneous computing mode, supports various embedded computing platforms such as an ASIC (application specific integrated circuit), a CPU (central processing unit), a DSP (digital signal processor), an FPGA (field programmable gate array), a GPU (graphic processing unit), an MCU (micro control unit) and the like, realizes cross-platform deployment and multi-platform cooperation, and supports the installation and operation of an android mobile phone, an IOS mobile phone/tablet, a microcomputer and AR intelligent glasses; the performance and power consumption ratio is higher, and the development of AR application by various hardware platform users can be well supported.
2) And enhancing virtual information contents such as operation guide videos, operation guide animations or models, data information labels, equipment state labels, fault information labels, navigation virtual icons and the like which are generated by decomposing each equipment component and position according to operation steps and are added into the scanned and created 3D model through a content editor, and distributing the virtual information contents to the mobile terminal 2.
And loading the regional target model scanned and built by the 3D scanning and modeling all-in-one machine 1 into a content editor, accurately positioning by using an AR SDK development kit and an AR identification algorithm based on a light field, and enhancing the virtual information content and adding the virtual information content into the 3D model of the space and the electrical test equipment built by scanning. When the camera of the mobile terminal 2 acquires the spatial information and the image information, the AR SDK development kit rapidly performs recognition calculation with the previously established 3D model details, the position information and the posture information, and corresponding virtual information is called and triggered if the comparison is successful. Due to the adoption of the positioning and tracking technology, the superposition speed of virtual information such as a 3D digital model loaded into a virtual 3D space and the spatial position of physical equipment in a real 3D space is higher, the tracking is more stable and reliable, and the defects of AR superposition information drift and jitter caused by a conventional AR identification algorithm are avoided.
3) And based on the AR SDK development kit and the positioning and tracking algorithm, the identification triggering of the visualization of the operation rules of the electrical test equipment, the OCR, the man-machine interaction, the indoor navigation evacuation and the remote assistance is completed.
The AR algorithm of the AR SDK development kit integrates two space-time coordinate systems on the premise of adding time dimension to the whole cognition of an object shot by an AR intelligent glasses camera, performs feature extraction, space identification, target tracking, space positioning and moving track extraction, compares the space-time coordinate systems with the position and image information of a 3D model which is scanned and created, and immediately triggers the comparison success and performs enhanced display on a moving end 2(AR intelligent glasses), and the method comprises the following steps: covering the 3D transparent model of the electrical equipment on the real equipment in a three-dimensional manner; displaying virtual arrows, icons and pictures and texts at the operation part and the spatial position of the transparent model; displaying multiple virtual labels; SLAM tracking of virtual models, labels, annotations, and navigation icons; 3D model launching, explosion disassembling, internal structure perspective and the like.
4) Operation procedure visual guidance and equipment troubleshooting of mobile terminal 2
The paper specification of the electrical test equipment is digitized and decomposed to manufacture a plurality of operation instruction files, including operation instruction videos, operation prompt voices, equipment information labels, system data labels, 3D models of local parts or components, instruction animations of operation and fault maintenance and the like. A plurality of operation instruction files are pre-arranged in an AR system, during work, AR intelligent glasses recognize space and equipment image information through a camera, are in butt joint with a monitoring system, acquire system and equipment data information in real time, trigger and display a visual guidance interface of virtual operation and a visual interface of fault prompt, and carry out fault removal and repair through visual virtual guidance.
5) And communicating a remote expert to carry out a remote assistance request when the field fault cannot be solved through visual virtual guidance.
When the field fault can not be eliminated or repaired through visual virtual guidance, a field operator can dial through a remote assistance function module of the AR intelligent glasses to a remote expert seat. The scene operator wears the pictures seen by the AR intelligent glasses and synchronously displays on the PAD terminal of the expert, the expert can communicate with the scene operator through the system, can mark the device operation parts, methods, steps and the like in the scene by fingers, and can send maintenance cases or knowledge documents to the scene operator, and the scene operator can continue to remove and repair the faults according to the communication guidance, the scene marking and the knowledge documents. The whole process of both sides' conversation and operation can all be recorded and reach high in the clouds 4 and keep, and the conversation record can lead to high in the clouds 4 database, makes things convenient for going on problem trace back, case sharing and training etc. in the future.
The expert adopts a direct drawing and labeling mode on equipment in a real-time field picture. The callout will anchor to the real electrical device object surface in the environment, just as it would be drawn on the surface of the object itself. Even if the AR smart glasses camera of the field operator has been transferred to other positions and angles, the previously drawn annotations still exist in the original position to ensure the accuracy of the guidance.
6) The mobile terminal 2 aligns the electrical equipment button or indicator light which is finished, the image acquisition unit 21 of the mobile terminal 2 acquires the image information of the electrical equipment button (handle, valve, etc.) or indicator light to the mobile display unit 24 of the mobile terminal 2, the mobile terminal 2 image identification calculation unit 22 identifies the image object information through the AR identification algorithm based on the light field, and rapidly compares the image object information with the image template preset in the system in advance, and feeds back and displays the comparison result in a visual or voice and information push mode, and the user sees the equipment state judgment result, the button (handle, valve, etc.) in-place state judgment result, and the indicator light displays the equipment state or the operation is abnormal on the mobile display unit 24.
7) Indoor virtual navigation of mobile terminal 2
The 3D scanning modeling all-in-one machine 1 has 3D modeled the space. The mobile terminal 2(AR smart glasses) aligns with the space to obtain the space image information, and the recognition and calculation unit 22 performs fast calculation and comparison to determine the position information of the mobile terminal, and triggers retrieval and display of an escape sign and an escape route. When an emergency situation occurs, the functional module quickly identifies scene information, and selects evacuation and evacuation routes to minimize personal injury as much as possible.
In summary, the system construction process of the present invention is simple and summarized as follows: based on a 3D camera laser scanning point cloud modeling technology and an AR enhancement technology, feature point extraction and identification judgment are carried out on a test space and electrical test equipment, and regulation guidance of test operation, data visualization, fault elimination, operation judgment, remote assistance and emergency navigation evacuation are realized through real-time physical position information, image information, multiple superposition of a computer technology and a digital model, coverage of a transparent 3D model on real equipment, virtual content expansion, interactive operation and real-time data display.
The above embodiments only illustrate the interactive system of the present invention by using an electrical testing apparatus as an example, however, it should be understood by those skilled in the art that the application scope of the present invention is not limited to the electrical testing apparatus. The interactive system and the interactive method of the invention can be applied to more applications, such as inspection tour, equipment maintenance and the like. Nothing in this context is meant to limit the scope of the invention. These embodiments may be implemented in other various manners, and various omissions, substitutions, and changes may be made without departing from the technical spirit of the present invention.
Claims (10)
1. An electric test intelligent interaction system based on AR technology is characterized in that the system comprises:
3D scanning modeling all-in-one machine (1): carrying out rapid scanning modeling on a laboratory space scene and electrical test equipment;
mobile end (2): the system is used for acquiring field images and performing fast matching calculation processing, so that loading of a virtual scene is triggered and visual display is performed;
expert terminal (3): the synchronous sharing mobile terminal (2) visually displays the picture and supports remote interactive operation on the shared picture;
cloud (4): and establishing data connection with the mobile terminal (2) and the expert terminal (3) respectively.
2. The electrical test intelligent interaction system based on the AR technology as claimed in claim 1, wherein the 3D scanning modeling all-in-one machine (1) performs the fast scanning modeling in a specific manner that: scanning a spatial scene of a laboratory and electrical test equipment, collecting environmental data, establishing a 3D model of each point location through a point cloud technology, and then splicing and optimizing the 3D models of the plurality of point locations to form a three-dimensional model of the spatial scene of the laboratory and a model of the electrical test equipment.
3. The electrical experiment intelligent interaction system based on the AR technology as claimed in claim 1, wherein the mobile terminal (2) comprises an image acquisition unit (21), an identification calculation unit (22), a first communication unit (23) and a mobile display unit (24);
an image acquisition unit (21) acquires a live image;
the identification calculation unit (22) identifies the field image acquired by the image acquisition unit (21) and identifies related characteristic points and point cloud image objects preset in advance;
the first communication unit (23) requests and receives virtual information content related to the identification object from the cloud (4);
and the mobile display unit (24) receives the interactive instruction, acquires corresponding virtual information content according to the interactive instruction and superposes the virtual information content on the site image for visual display.
4. The electrical experiment intelligent interaction system based on AR technology as claimed in claim 3, wherein the virtual information content comprises operation guidance video; pictures, characters and voice of system data and equipment operation data visual display; animation of the operation procedures; 3D part models of internal structures can be clicked, exploded and disassembled and perspective; the operation results show a graph.
5. The electrical experiment intelligent interaction system based on AR technology as claimed in claim 4, wherein the interaction instruction includes viewing and browsing the virtually superimposed information; operating the virtually overlapped 3D component model through a gesture instruction; and playing, pausing and closing the video and voice files which are virtually overlapped through a voice instruction.
6. The electrical experiment intelligent interaction system based on AR technology as claimed in claim 3, wherein the mobile terminal (2) further comprises a remote sharing unit (25), the remote sharing unit (25) calls the expert terminal (3) through the first communication unit (23), the mobile terminal (2) is shared to visually display the picture, perform information interaction with the expert terminal (3), and display the remote interaction operation guidance from the expert terminal (3) through the mobile display unit (24).
7. The electrical experiment intelligent interaction system based on the AR technology as claimed in claim 6, wherein the mobile terminal (2) performs synchronous recording of the sharing process in the remote sharing process, and uploads the recorded information to the cloud terminal (4) for storage through the first communication unit (23).
8. The electrical experiment intelligent interaction system based on the AR technology as claimed in claim 3, wherein the mobile terminal (2) further comprises an indoor virtual navigation unit (26), the image acquisition unit (21) is aligned with the space to acquire space image information, the identification and calculation unit (22) is used for rapidly calculating, comparing and determining position information of the mobile terminal, and triggering the virtual navigation unit (26) to call and display the escape sign and the escape route.
9. The electrical experiment intelligent interaction system based on AR technology as claimed in claim 1, wherein the expert terminal (3) comprises an expert display unit (31), a labeling guidance unit (32) and a second communication unit (33);
the expert display unit (31) synchronously shares the visual display picture of the mobile terminal (2) from the cloud terminal (4) through the second communication unit (33);
the annotation guidance unit (32) performs remote operation guidance through the second communication unit (33).
10. The electrical experiment intelligent interaction system based on AR technology as claimed in claim 1, wherein the mobile terminal (2) and the expert terminal (3) are configured to include any one or more of AR smart glasses, mobile phones, tablets and microcomputers.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536401.0A CN112562092A (en) | 2020-12-22 | 2020-12-22 | Electrical test intelligent interaction system based on AR technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011536401.0A CN112562092A (en) | 2020-12-22 | 2020-12-22 | Electrical test intelligent interaction system based on AR technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112562092A true CN112562092A (en) | 2021-03-26 |
Family
ID=75031509
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011536401.0A Pending CN112562092A (en) | 2020-12-22 | 2020-12-22 | Electrical test intelligent interaction system based on AR technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112562092A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065524A (en) * | 2021-04-26 | 2021-07-02 | 深圳市商汤科技有限公司 | Intelligent workbench management system, intelligent workbench, method, device and medium |
| CN113379943A (en) * | 2021-06-16 | 2021-09-10 | 国网山西省电力公司 | AR system of patrolling and examining based on 5G communication |
| CN113411298A (en) * | 2021-05-07 | 2021-09-17 | 上海纽盾科技股份有限公司 | Safety testing method and device combined with augmented reality |
| CN114363375A (en) * | 2021-12-31 | 2022-04-15 | 北京科技大学 | Electrical cabinet AR intelligent inspection system and method based on 5G communication |
| CN115268722A (en) * | 2022-07-07 | 2022-11-01 | 国网上海市电力公司 | Intelligent interaction method and system for partial discharge test of insulating part |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108038910A (en) * | 2017-11-10 | 2018-05-15 | 广东电网有限责任公司教育培训评价中心 | The implementation method of substation's main equipment Virtual Reality Demonstration |
| CN109462637A (en) * | 2018-10-17 | 2019-03-12 | 广东电网有限责任公司 | MR expert's remote collaboration method and system based on power equipment 3D model library |
| CN110047150A (en) * | 2019-04-24 | 2019-07-23 | 大唐环境产业集团股份有限公司 | It is a kind of based on augmented reality complex device operation operate in bit emulator system |
| CN110322564A (en) * | 2018-03-29 | 2019-10-11 | 广东电网有限责任公司江门供电局 | A kind of 3 D model construction method of the substation operation environment suitable for VR/AR |
| CN111553974A (en) * | 2020-04-21 | 2020-08-18 | 北京金恒博远科技股份有限公司 | Data visualization remote assistance method and system based on mixed reality |
| CN111679740A (en) * | 2020-06-06 | 2020-09-18 | 西安热工研究院有限公司 | Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology |
| CN111738459A (en) * | 2020-05-30 | 2020-10-02 | 国网河北省电力有限公司石家庄供电分公司 | Holographic expert system is examined in communication dispatch operation based on AR technique |
-
2020
- 2020-12-22 CN CN202011536401.0A patent/CN112562092A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108038910A (en) * | 2017-11-10 | 2018-05-15 | 广东电网有限责任公司教育培训评价中心 | The implementation method of substation's main equipment Virtual Reality Demonstration |
| CN110322564A (en) * | 2018-03-29 | 2019-10-11 | 广东电网有限责任公司江门供电局 | A kind of 3 D model construction method of the substation operation environment suitable for VR/AR |
| CN109462637A (en) * | 2018-10-17 | 2019-03-12 | 广东电网有限责任公司 | MR expert's remote collaboration method and system based on power equipment 3D model library |
| CN110047150A (en) * | 2019-04-24 | 2019-07-23 | 大唐环境产业集团股份有限公司 | It is a kind of based on augmented reality complex device operation operate in bit emulator system |
| CN111553974A (en) * | 2020-04-21 | 2020-08-18 | 北京金恒博远科技股份有限公司 | Data visualization remote assistance method and system based on mixed reality |
| CN111738459A (en) * | 2020-05-30 | 2020-10-02 | 国网河北省电力有限公司石家庄供电分公司 | Holographic expert system is examined in communication dispatch operation based on AR technique |
| CN111679740A (en) * | 2020-06-06 | 2020-09-18 | 西安热工研究院有限公司 | Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology |
Non-Patent Citations (1)
| Title |
|---|
| 杜勇;陈钊;刘锋;王先兵;: "变电站现场作业与风险管控的三维实景推演仿真", 华侨大学学报(自然科学版), no. 02, 20 March 2017 (2017-03-20), pages 237 - 240 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113065524A (en) * | 2021-04-26 | 2021-07-02 | 深圳市商汤科技有限公司 | Intelligent workbench management system, intelligent workbench, method, device and medium |
| CN113411298A (en) * | 2021-05-07 | 2021-09-17 | 上海纽盾科技股份有限公司 | Safety testing method and device combined with augmented reality |
| CN113379943A (en) * | 2021-06-16 | 2021-09-10 | 国网山西省电力公司 | AR system of patrolling and examining based on 5G communication |
| CN114363375A (en) * | 2021-12-31 | 2022-04-15 | 北京科技大学 | Electrical cabinet AR intelligent inspection system and method based on 5G communication |
| CN115268722A (en) * | 2022-07-07 | 2022-11-01 | 国网上海市电力公司 | Intelligent interaction method and system for partial discharge test of insulating part |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112562092A (en) | Electrical test intelligent interaction system based on AR technology | |
| CN109246195B (en) | Intelligent management and control method and system for pipe network integrating augmented reality and virtual reality | |
| CN107610217B (en) | Nuclear power equipment virtual identification display system and method | |
| US20180253900A1 (en) | System and method for authoring and sharing content in augmented reality | |
| CN106920291B (en) | Body structure surface inspection and analysis system based on augmented reality | |
| JP2020087440A (en) | Ar compatible labeling using aligned cad models | |
| CN110728670A (en) | Low-voltage equipment operation and maintenance method based on AR technology | |
| CN109559381B (en) | Transformer substation acceptance method based on AR space measurement technology | |
| CN111222190B (en) | Ancient building management system | |
| WO2012091326A2 (en) | Three-dimensional real-time street view system using distinct identification information | |
| CN111127974A (en) | Virtual reality and augmented reality three-dimensional application service platform for transformer substation operation | |
| WO2023132555A1 (en) | Augmented reality-based construction site management method and server | |
| CN111047685A (en) | Method for realizing power plant BIM model panoramic roaming | |
| CN113657307A (en) | Data labeling method and device, computer equipment and storage medium | |
| CN112306233A (en) | Inspection method, inspection system and inspection management platform | |
| CN111598524A (en) | VR (virtual reality) technology-based multi-person cooperative investigation method and investigation system | |
| CN115469745A (en) | Machine room equipment fault processing method and system based on AR technology | |
| Hameed et al. | Development of Augmented Reality-based object recognition mobile application with Vuforia | |
| CN111858799A (en) | Dynamic positioning method, system and equipment for panoramic image of nuclear power plant | |
| CN113838193A (en) | Data processing method and device, computer equipment and storage medium | |
| Ren et al. | Architecture in an age of augmented reality: applications and practices for mobile intelligence BIM-based AR in the entire lifecycle | |
| CN113570732A (en) | Shield maintenance auxiliary method and system based on AR technology | |
| CN112562091A (en) | AR technology-based intelligent interaction method for electrical test | |
| CN109863746A (en) | Interactive data visualization environment | |
| CN116645247A (en) | Panoramic view-based augmented reality industrial operation training system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |