CN113407035A - Communication facility maintenance operation system based on MR mixed reality technology - Google Patents
Communication facility maintenance operation system based on MR mixed reality technology Download PDFInfo
- Publication number
- CN113407035A CN113407035A CN202110797210.8A CN202110797210A CN113407035A CN 113407035 A CN113407035 A CN 113407035A CN 202110797210 A CN202110797210 A CN 202110797210A CN 113407035 A CN113407035 A CN 113407035A
- Authority
- CN
- China
- Prior art keywords
- data
- glasses
- module
- maintenance
- communication facility
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/20—Administration of product repair or maintenance
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/141—Systems for two-way working between two video terminals, e.g. videophone
Abstract
The invention discloses a communication facility maintenance operation system based on an MR mixed reality technology, which comprises a PC (personal computer) end, a mobile end and an MR (magnetic resonance) terminal, wherein the PC end can display a first visual angle picture of a first-line maintenance maintainer on a PC (personal computer) by logging in a remote cooperative system, and transmits the picture of a commander to MR (magnetic resonance) glasses of the first-line maintenance maintainer in an online video mode, so that the first-line maintenance maintainer is guided to solve a troublesome technical problem through remote cooperation. According to the invention, the PC terminal, the mobile terminal and the MR terminal are communicated through the remote cooperative system, and a line of maintainers is guided to solve a troublesome technical problem through remote cooperation, and historical video data of a first visual angle of the line of maintainers can be called to analyze and investigate accident reasons, so that equipment loss caused by misoperation is reduced to the maximum extent, unnecessary economic loss is reduced, hidden dangers in actual work are eliminated, and safety accidents are avoided.
Description
Technical Field
The invention belongs to the field of communication facility maintenance, and particularly relates to a communication facility maintenance operation system based on an MR mixed reality technology.
Background
Mixed reality, which is a combination of virtual reality VR and augmented reality AR, in the maintenance work of daily communication infrastructure, maintenance personnel need to perform ascending operation, when the maintenance personnel have problems which cannot be solved, the online answer and guidance of a command center or experts need to be sought through modes such as video, voice and the like, and the traditional communication mode can generate two adverse factors, namely, the risk of safety accidents is increased, such as high-altitude falling tools and the like; secondly, because both hands can not be completely liberated, the working efficiency is reduced, and therefore, a communication facility overhauling operation system based on the MR mixed reality technology is provided.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides a communication facility overhaul operating system based on an MR mixed reality technology.
In order to achieve the purpose, the invention provides the following technical scheme:
a communication facility maintenance operation system based on MR mixed reality technology comprises a PC (personal computer) end, a mobile end and an MR (magnetic resonance) terminal, wherein the PC end can display a first visual angle picture of a first-line maintenance maintainer on a PC (personal computer) by logging in a remote cooperative system, and transmits the picture of a commander to MR (magnetic resonance) glasses of the first-line maintenance maintainer in an online video mode, so that the first-line maintenance maintainer is guided to solve a troublesome technical problem in a remote cooperative mode;
the mobile terminal is connected with the PC terminal through a login remote cooperative system, technical experts or commanders can log in the remote cooperative system through the mobile terminal no matter where the technical experts or the commanders are located, a first visual angle picture of a first-line maintenance maintainer is displayed, the technical experts or the commanders carry out real-time video communication with the first-line maintenance maintainer in an online video mode, the picture of the technical experts is transmitted to MR glasses of the first-line maintenance maintainer, and the first-line maintenance maintainer is guided to solve a troublesome technical problem through remote cooperation;
the MR terminal is set as MR glasses, a line of basic maintenance and repair personnel can realize the functions of three-dimensional guide standardization operation, remote cooperative guidance and standard operation flow data calling by wearing the MR glasses and complete the field maintenance and repair work, IR cameras, IR lamps and control processors are arranged in the left side and the right side of a front glasses cover of the MR glasses, and the control processors are connected with the IR cameras and used for receiving the pupil position information of the human eyes collected by the IR cameras; the control processor calculates the viewpoint direction of the eyeballs and the movement direction of the eyeballs according to the pupil position information of the eyeballs so as to control the man-machine interaction operation of the mouse points on the MR glasses screen;
a first RGB-IR image sensor used for collecting visible light images and infrared images is arranged in the left area of the outer right front part of the front cover of the MR glasses, and the first RGB-IR image sensor is connected with the control processor and transmits collected information to the control processor; a second RGB-IR image sensor used for collecting visible light images and infrared images is arranged in the right side area of the right front part of the outer side of the MR glasses front cover and is connected with the control processor and transmits collected information to the control processor;
an IR projection lamp for projecting an image formed by infrared stripes or points to an opposite object is arranged in the middle area of the outer right front part of the MR glasses front cover; the control processor calculates the three-dimensional space position of the opposite object according to the information transmitted by the first RGB-IR image sensor and the second RGB-IR image sensor, then draws the virtual scenery according to the three-dimensional space position, and superposes the virtual scenery on the RGB images obtained by the first RGB-IR image sensor and the second RGB-IR image sensor, and then displays the virtual scenery on the display screen.
Preferably, the MR glasses comprise a stereo shooting module, the stereo shooting module is mounted at the head position of a ship-end operator to acquire three-dimensional scene depth information from the fault equipment to the camera, and stereo shooting parameters of the three-dimensional camera are adjusted according to the three-dimensional scene depth information; the fusion compression module is used for receiving the three-dimensional scene data information transmitted by the stereo shooting module, performing fusion compression processing on the received information, and converting the information into a data format for transmission; and the communication module transmits the data information transmitted by the fusion compression module back to the mobile terminal by adopting a 5G technology.
Preferably, the remote cooperative system is used for synchronously transmitting the first visual angle picture shot by the MR glasses camera to the command center, and the technical expert video picture of the command center is synchronously transmitted to the MR glasses through the remote cooperative system, so that the maintenance personnel is guided to carry out field operation in real time, the problem is rapidly solved, the synchronous completion of the guidance and the maintenance is realized, and the working efficiency is improved; the remote cooperative system can simultaneously display at the PC end, the mobile end and the MR glasses end, cross-equipment seamless butt joint is achieved, the problem that professional technicians or experts are not in the field or in a command center is solved, and remote cooperative work can be achieved through a mobile phone.
Preferably, the PC computer end comprises a data entry module and a data storage module, wherein the data entry module is used for entering standard maintenance and repair work data, text data, video data, audio data and picture data at any time through a PC end login platform by a worker, and various data are classified according to types.
Preferably, the MR glasses comprise a three-dimensional guide module, when communication facilities are overhauled, the overhauling work content is modularized, programmed and standardized through the three-dimensional guide module, visual operation steps are realized, maintainers complete overhauling of the communication facilities item by item according to program guidance, and the accuracy and effectiveness of the maintenance work are improved; the MR glasses further comprise a data transferring module, the data transferring module is used for transferring standardized data at any time through the gesture recognition, eyeball tracking and voice recognition functions of the MR glasses when a line of maintenance workers maintain the equipment, maintenance work is carried out according to the data, and maintenance problems are solved on site.
Preferably, the system further comprises a mixed reality display module of the mobile terminal for receiving the data information transmitted by the communication module, wherein the mixed reality display module displays the received stereoscopic scene information in real time, the position of the human eye is taken as a viewpoint, the three-dimensional scene depth measured by the stereoscopic shooting module is taken as the stereoscopic scene display depth of the mixed reality display module, the three-dimensional stereoscopic scene of the fault communication facility is displayed, the synchronization of the three-dimensional stereoscopic scene of the mobile terminal is realized, and the fault judgment of the fault communication facility is performed in the three-dimensional stereoscopic scene.
Preferably, the PC computer end further comprises an audio and video calling module, and the audio and video calling module is used for allowing a worker to log in the platform through the PC end to call historical audio and video data when a safety accident or serious equipment damage condition occurs in the base station; the audio and video data comprises: the first visual angle video data, the remote cooperative video call data and the remote cooperative voice call data of the MR glasses of the first-line maintenance worker are retrieved, the accident occurrence reason is found back through retrieving the audio and video data, the accident occurrence caused by human misoperation or the quality reason of the equipment is determined, and the responsibility confirmation is determined.
Preferably, the data storage module is used for storing standardized operation tutorials, equipment model data, equipment pictures, characters, operation guide voice, first visual angle video data, video call data, voice call data and personnel information data contents.
Preferably, in the system, the MR glasses end can log in or update standard maintenance operation data including video, audio, text and picture data contents by logging in a communication facility maintenance and repair operation system, and carries out real-time voice communication with a front-line maintenance worker through a remote cooperative system; logging in a communication facility maintenance and overhaul operation system at a PC (personal computer) end, and calling historical video data of a first visual angle of a first-line maintainer to analyze and investigate accident reasons; and calling historical video call and voice call records, and determining accident responsibility confirmation.
Preferably, the control processor collects all infrared positioning lamps through the first RGB-IR image sensor and the second RGB-IR image sensor, calculates the position of the mobile terminal screen in a three-dimensional space according to the positions of the infrared positioning lamps, draws a virtual scenery according to the three-dimensional space position, overlays the virtual scenery on the three-dimensional space position of the mobile phone screen, and then displays the virtual scenery on the display screen of the MR glasses.
The invention has the technical effects and advantages that: compared with the traditional maintenance public head mode, the communication facility maintenance operation system based on the MR mixed reality technology provided by the invention has the advantages that the PC computer end, the mobile end and the MR terminal are communicated through the remote cooperative system, the picture of a commander can be transmitted to MR glasses of a first-line maintenance maintainer in an online video mode, the first-line maintainer is guided to solve a troublesome technical problem through remote cooperation, and historical video data of a first visual angle of the first-line maintainer can be called to analyze and investigate accident reasons; the historical video call and voice call records are called, the accident responsibility is confirmed, the communication infrastructure maintenance efficiency is improved, the equipment loss caused by misoperation is reduced to the maximum extent, unnecessary economic loss is reduced, the hidden danger in actual work is eliminated, and safety accidents are avoided.
Drawings
FIG. 1 is a functional block diagram of a communications facility service operating system according to the present invention;
fig. 2 is a diagram of a communications facility service operating system architecture of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
The invention provides a communication facility maintenance operation system based on an MR mixed reality technology as shown in figures 1-2, which comprises a PC computer end, a mobile end and an MR terminal, wherein the PC computer end can display a first visual angle picture of a first-line maintenance and repair person on the PC computer by logging in a remote cooperative system, and transmits the picture of a commander to MR glasses of the first-line maintenance and repair person in an online video mode, and guides the first-line maintenance and repair person to solve a troublesome technical problem through remote cooperative guidance;
the mobile terminal is connected with the PC terminal through a login remote cooperative system, technical experts or commanders can log in the remote cooperative system through the mobile terminal no matter where the technical experts or commanders are located, a first visual angle picture of a first-line maintenance and repair staff is displayed, and real-time screen conversation is carried out with the first-line maintenance and repair staff through an online video mode, the picture of the technical expert is transmitted into MR glasses of the first-line maintenance and repair staff, and the first-line maintenance and repair staff is guided to solve the troublesome technical problem through remote cooperation, the PC terminal comprises a data entry module and a data storage module, the data entry module is used for a worker to enter standardized maintenance and repair work data, text data, video data, audio data and picture data at any time through a PC terminal login platform, and various data are divided according to classes, the PC terminal also comprises an audio and video calling module, and the audio and video calling module is used for calling when a base station has a safety accident or equipment is seriously damaged, a worker logs in the platform through the PC terminal and calls historical audio and video data;
wherein, the audio and video data include: the method comprises the steps that a first-line maintenance worker MR glasses receives first visual angle video data, remote cooperative video call data and remote cooperative voice call data, and the reason of an accident is found by looking up audio and video data, so that the accident caused by human misoperation or the quality reason of equipment is determined, and the responsibility is determined;
the data storage module is used for storing standardized operation tutorials, equipment model data, equipment pictures, characters, operation guide voice, first visual angle video data, video call data, voice call data and personnel information data;
the MR terminal is set as MR glasses, a line of basic maintenance and repair personnel can realize the functions of three-dimensional guide standardization operation, remote cooperative guidance and standard operation flow data calling by wearing the MR glasses and complete the field maintenance and repair work, IR cameras, IR lamps and control processors are arranged in the left side and the right side of a front glasses cover of the MR glasses, and the control processors are connected with the IR cameras and used for receiving the pupil position information of the human eyes collected by the IR cameras; the control processor calculates the viewpoint direction of the eyeballs and the movement direction of the eyeballs according to the pupil position information of the eyeballs so as to control the man-machine interaction operation of the mouse point on the MR glasses screen;
a first RGB-IR image sensor used for collecting visible light images and infrared images is arranged in the left area of the outer right front part of the front cover of the MR glasses, and the first RGB-IR image sensor is connected with the control processor and transmits collected information to the control processor; a second RGB-IR image sensor used for collecting visible light images and infrared images is arranged in the right area of the right front part of the outer side of the MR glasses front cover and is connected with the control processor and transmits collected information to the control processor;
an IR projection lamp for projecting an image formed by infrared stripes or points to an opposite object is arranged in the middle area of the outer right front part of the MR glasses front cover; the control processor calculates the three-dimensional space position of the opposite object according to the information transmitted by the first RGB-IR image sensor and the second RGB-IR image sensor, then draws a virtual scenery according to the three-dimensional space position, and superposes the virtual scenery on the RGB images obtained by the first RGB-IR image sensor and the second RGB-IR image sensor, and then displays the RGB images on a display screen, the MR glasses further comprise a data calling module, and the data calling module is used for a line of maintenance worker to call and read standardized data at any time through the functions of gesture recognition, eyeball tracking and voice recognition of the MR glasses in the process of maintaining equipment, so as to carry out maintenance work on the basis of difficult data and solve maintenance work on site;
the control processor collects all infrared positioning lamps through the first RGB-IR image sensor and the second RGB-IR image sensor, calculates the position of the mobile terminal screen in a three-dimensional space according to the positions of the infrared positioning lamps, draws a virtual scenery according to the three-dimensional space position, superimposes the virtual scenery on the three-dimensional space position of the mobile phone screen, displays the virtual scenery on a display screen of the MR glasses, and does fine processing on 3D textures of a direct-view area of an eyeball without fine processing on a non-direct-view area after calculating the viewpoint direction of the eyeball and the motion direction of the eyeball; after the control processor calculates the viewpoint direction of the eyeballs and the movement direction of the eyeballs, fine lens correction is carried out on areas which are directly viewed by the eyeballs, and simplified lens correction is carried out on areas which are not directly viewed;
according to the design, the PC terminal, the mobile terminal and the MR terminal are communicated through the remote cooperative system, so that not only can the picture of a commander be transmitted to MR glasses of a first-line maintenance maintainer in an online video mode, but also the first-line maintainer can be guided to solve a troublesome technical problem through remote cooperation, and historical video data of a first visual angle of the first-line maintainer can be called to analyze and investigate accident reasons; calling historical video call and voice call records, determining accident responsibility confirmation, improving the maintenance efficiency of communication infrastructure, reducing equipment loss caused by misoperation to the maximum extent, reducing unnecessary economic loss, eliminating hidden dangers in actual work and avoiding safety accidents;
the remote cooperative system is used for synchronously transmitting the first visual angle picture shot by the MR glasses camera to the command center, and the technical expert video picture of the command center is synchronously transmitted to the MR glasses through the remote cooperative system, so that maintenance personnel can be guided to carry out field operation in real time, problems are rapidly solved, the guidance and maintenance are synchronously finished, and the working efficiency is improved; the remote cooperative system can simultaneously display at the PC end, the mobile end and the MR glasses end, thereby achieving cross-equipment seamless butt joint, solving the problem that professional technicians or experts are not in the field or a command center, realizing remote cooperative work through a mobile phone, matching the MR glasses with the mobile phone, a tablet and a common PC, and completing accurate control on a virtual scene through various positioning algorithms;
the MR glasses comprise a stereo shooting module, the stereo shooting module is arranged at the head position of a ship-end operator to acquire three-dimensional scene depth information from the fault equipment to the camera, and stereo shooting parameters of the three-dimensional camera are adjusted according to the three-dimensional scene depth information; the fusion compression module is used for receiving the three-dimensional scene data information transmitted by the stereo shooting module, performing fusion compression processing on the received information, and converting the information into a data format for transmission; the communication module transmits data information transmitted by the fusion compression module back to the mobile end by adopting a 5G technology, an accessory model in an accessory model library is called in a man-machine interaction device of the MR glasses, the accessory model in the model library adopts a three-dimensional modeling tool to create a three-dimensional model, the accessory model is moved through the data gloves to be assembled and displayed, a three-dimensional mouse is used for marking fault points, the data gloves and the position tracker are matched to detect hand shapes and bending angles of joints of hands in real time during working, and the position and motion state information of the handles are input into the MR glasses to grab, rotate and move the accessory model;
as an optional technical scheme, the remote collaboration system comprises an expert end module arranged at a PC end and an employee end module arranged on MR glasses;
the expert end module comprises: the label identification module is used for identifying the information label of the video carried out by the expert on the terminal equipment; if the information label is identified to be a prefabricated label, calling a model stored in a database through a stroke identification algorithm, and using the position, size and orientation information of a labeled screen drawn on the screen as prefabricated label information; if the information label is identified as a selection label drawn by an expert, taking the coordinate sequence of each point on the selection polygon as selection label information; if the information mark is identified as a character mark, the character mark information code and the screen position are used as character mark information;
the terminal equipment is arranged at an expert end and used for enabling the expert to interactively input corresponding operations through gestures to assist a user in completing an assembly and maintenance task; the terminal equipment comprises one or more of a PC (personal computer), a smart phone and a tablet personal computer, and is used for marking on a display screen of the PC end through a mouse or drawing a graph on a touch screen of the smart phone and the tablet personal computer;
wherein, prefabricated mark includes: a plugging instruction and a rotating instruction in gesture recognition; the rotation instruction comprises a rotation instruction in the horizontal direction and a rotation instruction in the vertical direction;
the employee end module includes: the system comprises an information acquisition module, a label generation module and a result display module, wherein the information acquisition module is used for acquiring image information, audio information and depth information of a field environment through augmented reality equipment;
and the label generation module is used for generating a three-dimensional model from the label information received by the augmented reality equipment and placing the three-dimensional model at a corresponding spatial position: if the augmented reality equipment receives the prefabricated marking information, unpacking the received prefabricated marking information and the position, size and orientation information of the screen; placing the three-dimensional model of the prefabricated annotation information at a corresponding space position according to the screen position, and adjusting the size and the orientation of the prefabricated annotation;
if the augmented reality equipment receives the circle marking information, firstly, generating a corresponding circle marking three-dimensional model for the received coordinate sequence through a classical constraint Delaunay triangulation dynamic algorithm, and acquiring the center point coordinates of a circle polygon through the coordinate sequence; placing the circled and marked three-dimensional model at a corresponding space position through the coordinate position of the central point;
if the augmented reality equipment receives the character marking information, the character information marking is rendered on the note model, and the note model is placed on the corresponding space position; the result display module is used for displaying the three-dimensional information label on a display screen of the augmented reality equipment;
in the system, the MR glasses end can log in or update standard maintenance operation data including video, audio, text and picture data contents in real time by logging in a communication facility maintenance and repair operation system, and carries out real-time voice communication with a front-line maintenance worker through a remote cooperative system; logging in a communication facility maintenance and overhaul operation system at a PC (personal computer) end, and calling historical video data of a first visual angle of a first-line maintainer to analyze and investigate accident reasons; calling historical video call and voice call records, and determining accident responsibility confirmation;
optionally, the method for constructing the three-dimensional space in the physical world by the MR spectacle side specifically includes:
1) three-dimensional information perception: the MR glasses sense the three-dimensional environment information through the vision sensor, and any currently known three-dimensional environment scanning algorithm for space sensing can be used;
the sensor adopts an IR camera, and can also be a TOF or structured light depth camera, but the complexity of the following algorithm is also determined by selecting different cameras;
2) three-dimensional scene reconstruction, namely performing three-dimensional reconstruction on a scene in real time through environment information obtained by a visual sensor;
the system also comprises a mixed reality display module of the mobile terminal for receiving the data information transmitted by the communication module, wherein the mixed reality display module displays the received three-dimensional scene information in real time, takes the position of human eyes as a viewpoint, takes the three-dimensional scene depth measured by the stereo shooting module as the three-dimensional scene display depth of the mixed reality display module, displays the three-dimensional scene of the fault communication facility, realizes the synchronization of the three-dimensional scene of the mobile terminal, and judges the fault of the fault communication facility in the three-dimensional scene; the auxiliary information adding module is used for receiving data information transmitted by the mixed reality display module, marking fault points in a three-dimensional scene displayed on the shore by using a three-dimensional interactive tool, calling accessory models from an accessory model library and displaying a maintenance and assembly method of communication infrastructure fault equipment; the auxiliary information adding module transmits the assembling method information to a fault communication facility through the communication module; and a graphic rendering module of the fault communication facility receives the data information transmitted by the auxiliary information adding module, carries out three-dimensional graphic rendering processing on the expert maintenance operation information and the model library accessory information, and renders the data information into maintenance auxiliary enhancement information according to the corresponding field angle of the mixed reality glasses and the three-dimensional scene depth matched with the scene.
Finally, it should be noted that: although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments or portions thereof without departing from the spirit and scope of the invention.
Claims (10)
1. The utility model provides a communication facilities overhauls operating system based on MR mixes reality technique, includes PC computer end, removes end and MR terminal, its characterized in that:
the PC computer end can display a first visual angle picture of a first-line maintenance and repair personnel on the PC computer by logging in the remote cooperative system, and transmits the picture of a commander to MR glasses of the first-line maintenance and repair personnel in an online video mode, so that the first-line maintenance and repair personnel are guided to solve a troublesome technical problem through remote cooperation;
the mobile terminal is connected with the PC terminal through a login remote cooperative system, technical experts or commanders can log in the remote cooperative system through the mobile terminal no matter where the technical experts or the commanders are located, a first visual angle picture of a first-line maintenance maintainer is displayed, the technical experts or the commanders carry out real-time video communication with the first-line maintenance maintainer in an online video mode, the picture of the technical experts is transmitted to MR glasses of the first-line maintenance maintainer, and the first-line maintenance maintainer is guided to solve a troublesome technical problem through remote cooperation;
the MR terminal is set as MR glasses, a line of basic maintenance and repair personnel can realize the functions of three-dimensional guide standardization operation, remote cooperative guidance and standard operation flow data calling by wearing the MR glasses and complete the field maintenance and repair work, IR cameras, IR lamps and control processors are arranged in the left side and the right side of a front glasses cover of the MR glasses, and the control processors are connected with the IR cameras and used for receiving the pupil position information of the human eyes collected by the IR cameras; the control processor calculates the viewpoint direction of the eyeballs and the movement direction of the eyeballs according to the pupil position information of the eyeballs so as to control the man-machine interaction operation of the mouse points on the MR glasses screen;
a first RGB-IR image sensor used for collecting visible light images and infrared images is arranged in the left area of the outer right front part of the front cover of the MR glasses, and the first RGB-IR image sensor is connected with the control processor and transmits collected information to the control processor; a second RGB-IR image sensor used for collecting visible light images and infrared images is arranged in the right side area of the right front part of the outer side of the MR glasses front cover and is connected with the control processor and transmits collected information to the control processor;
an IR projection lamp for projecting an image formed by infrared stripes or points to an opposite object is arranged in the middle area of the outer right front part of the MR glasses front cover; the control processor calculates the three-dimensional space position of the opposite object according to the information transmitted by the first RGB-IR image sensor and the second RGB-IR image sensor, then draws the virtual scenery according to the three-dimensional space position, and superposes the virtual scenery on the RGB images obtained by the first RGB-IR image sensor and the second RGB-IR image sensor, and then displays the virtual scenery on the display screen.
2. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: the MR glasses further comprise a stereo shooting module, the stereo shooting module is installed at the head position of a ship-end operator to acquire three-dimensional scene depth information from the fault equipment to the camera, and stereo shooting parameters of the three-dimensional camera are adjusted according to the three-dimensional scene depth information; the fusion compression module is used for receiving the three-dimensional scene data information transmitted by the stereo shooting module, performing fusion compression processing on the received information, and converting the information into a data format for transmission; and the communication module transmits the data information transmitted by the fusion compression module back to the mobile terminal by adopting a 5G technology.
3. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: the remote cooperative system is used for synchronously transmitting a first visual angle picture shot by the MR glasses camera to the command center, and the technical expert video picture of the command center is synchronously transmitted to the MR glasses through the remote cooperative system, so that maintenance personnel can be guided to carry out field operation in real time, problems are rapidly solved, the guidance and maintenance are synchronously finished, and the working efficiency is improved; the remote cooperative system can simultaneously display at the PC end, the mobile end and the MR glasses end, cross-equipment seamless butt joint is achieved, the problem that professional technicians or experts are not in the field or in a command center is solved, and remote cooperative work can be achieved through a mobile phone.
4. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: the PC computer end comprises a data entry module and a data storage module, wherein the data entry module is used for enabling workers to enter standardized maintenance and overhaul work data, character data, video data, audio data and picture data at any time through a PC end login platform, and various data are divided according to classes.
5. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: the MR glasses comprise a three-dimensional guide module, when communication facilities are overhauled, the overhauling work content is modularized, programmed and standardized through the three-dimensional guide module, visual operation steps are realized, maintainers complete overhauling of the communication facilities item by item according to program guidance, and the accuracy and effectiveness of maintenance work are improved; the MR glasses further comprise a data transferring module, the data transferring module is used for transferring standardized data at any time through the gesture recognition, eyeball tracking and voice recognition functions of the MR glasses when a line of maintenance workers maintain the equipment, maintenance work is carried out according to the data, and maintenance problems are solved on site.
6. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: the system also comprises a mixed reality display module of the mobile terminal for receiving the data information transmitted by the communication module, wherein the mixed reality display module displays the received three-dimensional scene information in real time, the position of the human eye is taken as a viewpoint, the three-dimensional scene depth measured by the stereo shooting module is taken as the three-dimensional scene display depth of the mixed reality display module, the three-dimensional scene of the fault communication facility is displayed, the synchronization of the three-dimensional scene of the mobile terminal is realized, and the fault judgment of the fault communication facility is carried out in the three-dimensional scene.
7. The MR mixed reality technology-based communication facility overhaul operating system according to claim 6, wherein: the PC computer end also comprises an audio and video calling module, and the audio and video calling module is used for allowing a worker to log in the platform through the PC end to call historical audio and video data when a safety accident or serious equipment damage condition occurs in the base station; the audio and video data comprises: the first visual angle video data, the remote cooperative video call data and the remote cooperative voice call data of the MR glasses of the first-line maintenance worker are retrieved, the accident occurrence reason is found back through retrieving the audio and video data, the accident occurrence caused by human misoperation or the quality reason of the equipment is determined, and the responsibility confirmation is determined.
8. The MR mixed reality technology-based communication facility overhaul operating system according to claim 4, wherein: the data storage module is used for storing standardized operation tutorials, equipment model data, equipment pictures, characters, operation guide voice, first visual angle video data, video call data, voice call data and personnel information data contents.
9. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: in the system, the MR glasses end can log in or update standard maintenance operation data including video, audio, text and picture data contents in real time by logging in a communication facility maintenance and repair operation system, and carries out real-time voice communication with a front-line maintenance worker through a remote cooperative system; logging in a communication facility maintenance and overhaul operation system at a PC (personal computer) end, and calling historical video data of a first visual angle of a first-line maintainer to analyze and investigate accident reasons; and calling historical video call and voice call records, and determining accident responsibility confirmation.
10. The MR mixed reality technology-based communication facility service operating system according to claim 1, wherein: the control processor collects all the infrared positioning lamps through the first RGB-IR image sensor and the second RGB-IR image sensor, calculates the position of the mobile terminal screen in a three-dimensional space according to the positions of the infrared positioning lamps, draws a virtual scenery according to the three-dimensional space position, superimposes the virtual scenery on the three-dimensional space position of the mobile phone screen, and displays the virtual scenery on the display screen of the MR glasses.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110797210.8A CN113407035A (en) | 2021-07-14 | 2021-07-14 | Communication facility maintenance operation system based on MR mixed reality technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110797210.8A CN113407035A (en) | 2021-07-14 | 2021-07-14 | Communication facility maintenance operation system based on MR mixed reality technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113407035A true CN113407035A (en) | 2021-09-17 |
Family
ID=77686542
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110797210.8A Pending CN113407035A (en) | 2021-07-14 | 2021-07-14 | Communication facility maintenance operation system based on MR mixed reality technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113407035A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114859556A (en) * | 2022-04-19 | 2022-08-05 | 国网江苏省电力有限公司南通供电分公司 | Circuit breaker overhauls VR equipment based on digit twin |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106168855A (en) * | 2016-07-01 | 2016-11-30 | 北京麦哲科技有限公司 | A kind of portable MR glasses, mobile phone and MR glasses system |
| CN106707810A (en) * | 2016-12-02 | 2017-05-24 | 大连海事大学 | Auxiliary system and method for ship remote fault diagnosis and maintenance based on mixed reality glasses |
| CN110266992A (en) * | 2019-06-24 | 2019-09-20 | 苏芯物联技术(南京)有限公司 | A kind of long-distance video interactive system and method based on augmented reality |
| CN110865917A (en) * | 2018-08-27 | 2020-03-06 | 国网天津市电力公司 | AR technology-based electric power machine room inspection operation method, system and application |
| CN111260084A (en) * | 2020-01-09 | 2020-06-09 | 长安大学 | Remote system and method based on augmented reality collaborative assembly maintenance |
| CN112507799A (en) * | 2020-11-13 | 2021-03-16 | 幻蝎科技(武汉)有限公司 | Image identification method based on eye movement fixation point guidance, MR glasses and medium |
-
2021
- 2021-07-14 CN CN202110797210.8A patent/CN113407035A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106168855A (en) * | 2016-07-01 | 2016-11-30 | 北京麦哲科技有限公司 | A kind of portable MR glasses, mobile phone and MR glasses system |
| CN106707810A (en) * | 2016-12-02 | 2017-05-24 | 大连海事大学 | Auxiliary system and method for ship remote fault diagnosis and maintenance based on mixed reality glasses |
| CN110865917A (en) * | 2018-08-27 | 2020-03-06 | 国网天津市电力公司 | AR technology-based electric power machine room inspection operation method, system and application |
| CN110266992A (en) * | 2019-06-24 | 2019-09-20 | 苏芯物联技术(南京)有限公司 | A kind of long-distance video interactive system and method based on augmented reality |
| CN111260084A (en) * | 2020-01-09 | 2020-06-09 | 长安大学 | Remote system and method based on augmented reality collaborative assembly maintenance |
| CN112507799A (en) * | 2020-11-13 | 2021-03-16 | 幻蝎科技(武汉)有限公司 | Image identification method based on eye movement fixation point guidance, MR glasses and medium |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114859556A (en) * | 2022-04-19 | 2022-08-05 | 国网江苏省电力有限公司南通供电分公司 | Circuit breaker overhauls VR equipment based on digit twin |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111526118B (en) | Remote operation guiding system and method based on mixed reality | |
| CN105182535B (en) | The method that automobile maintenance is carried out using intelligent glasses | |
| CN106340217B (en) | Manufacturing equipment intelligence system and its implementation based on augmented reality | |
| CN107168537B (en) | Cooperative augmented reality wearable operation guidance method and system | |
| CN111260084B (en) | Remote system and method based on augmented reality cooperative assembly maintenance | |
| CN109268010B (en) | Remote inspection intervention method for virtual reality mine fully-mechanized coal mining face | |
| CN111679740A (en) | Method for carrying out remote intelligent diagnosis on power station equipment by using Augmented Reality (AR) technology | |
| CN107004279A (en) | Natural user interface camera calibrated | |
| CN110728670A (en) | Low-voltage equipment operation and maintenance method based on AR technology | |
| CN106707810A (en) | Auxiliary system and method for ship remote fault diagnosis and maintenance based on mixed reality glasses | |
| CN109531566A (en) | A kind of robot livewire work control method based on virtual reality system | |
| CN113222184A (en) | Equipment inspection system and method based on augmented reality AR | |
| CN116911823A (en) | AR (augmented reality) glasses-based substation inspection interaction method and system | |
| KR20200109271A (en) | System for supporting parts maintenace based on mixed reality and method thereof | |
| CN112085232A (en) | Operation inspection system and method based on augmented reality technology | |
| CN111966068A (en) | Augmented reality monitoring method and device for motor production line, electronic equipment and storage medium | |
| CN113407035A (en) | Communication facility maintenance operation system based on MR mixed reality technology | |
| CN115469744A (en) | Machine room equipment inspection operation method based on AR technology | |
| CN112330753B (en) | Target detection method of augmented reality system | |
| CN114169546A (en) | MR remote cooperative assembly system and method based on deep learning | |
| CN116824102A (en) | AR equipment-based intelligent equipment inspection method | |
| CN115469745A (en) | Machine room equipment fault processing method and system based on AR technology | |
| CN101165719A (en) | Enhancement implement industrial maintenance method and system based on mark point | |
| CN113674426A (en) | Equipment maintenance method and device using AR technology | |
| CN115514885A (en) | Monocular and binocular fusion-based remote augmented reality follow-up perception system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210917 |