CN115314474B - Communication emergency equipment assists maintenance system based on AR - Google Patents

Communication emergency equipment assists maintenance system based on AR Download PDF

Info

Publication number
CN115314474B
CN115314474B CN202211245639.7A CN202211245639A CN115314474B CN 115314474 B CN115314474 B CN 115314474B CN 202211245639 A CN202211245639 A CN 202211245639A CN 115314474 B CN115314474 B CN 115314474B
Authority
CN
China
Prior art keywords
module
display module
assistance data
browser
server
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202211245639.7A
Other languages
Chinese (zh)
Other versions
CN115314474A (en
Inventor
邹远辉
黄宏业
罗少康
杨柳扬
李晓辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
China ComService Construction Co Ltd
Original Assignee
China ComService Construction Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by China ComService Construction Co Ltd filed Critical China ComService Construction Co Ltd
Priority to CN202211245639.7A priority Critical patent/CN115314474B/en
Publication of CN115314474A publication Critical patent/CN115314474A/en
Application granted granted Critical
Publication of CN115314474B publication Critical patent/CN115314474B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/40Support for services or applications
    • H04L65/401Support for services or applications wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference
    • H04L65/4015Support for services or applications wherein the services involve a main real-time session and one or more additional parallel real-time or time sensitive sessions, e.g. white board sharing or spawning of a subconference where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/544Buffers; Shared memory; Pipes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/20Administration of product repair or maintenance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/40Business processes related to the transportation industry
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/003Navigation within 3D models or images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/1066Session management
    • H04L65/1069Session establishment or de-establishment
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/14Systems for two-way working
    • H04N7/141Systems for two-way working between two video terminals, e.g. videophone
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2219/00Indexing scheme for manipulating 3D models or images for computer graphics
    • G06T2219/004Annotating, labelling

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Business, Economics & Management (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Resources & Organizations (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Marketing (AREA)
  • Computer Hardware Design (AREA)
  • Tourism & Hospitality (AREA)
  • Strategic Management (AREA)
  • Economics (AREA)
  • Computer Graphics (AREA)
  • Quality & Reliability (AREA)
  • Operations Research (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The invention discloses an AR-based communication emergency equipment auxiliary maintenance system, and relates to the technical field of AR auxiliary maintenance. The first browser and the second browser carry out audio and video communication through a direct communication path established by the WebRTC server; the background display module synchronously displays a video picture during audio and video communication and a background video picture acquired by the acquisition module; the second shared display module collects and displays the assistance data, and sends the assistance data to the first shared display module through the WebSocket server; the first sharing display module is used for determining target display coordinates and target display sizes of the assistance data; and the projection module is used for projecting a virtual screen at a preset distance in front of the AR terminal and displaying the assistance data on the virtual screen in a target display size. The system improves the maintenance efficiency of emergency maintenance personnel and avoids the delay condition.

Description

Communication emergency equipment assists maintenance system based on AR
Technical Field
The invention relates to the technical field of AR (augmented reality) assisted maintenance, in particular to an AR-based communication emergency equipment assisted maintenance system.
Background
Communication emergency equipment has the following problems in daily use and maintenance:
firstly, emergent personnel open the inefficiency problem when carrying out emergency operation in the open air, normally require personnel's equipment to accomplish after targetting in place that multiple equipment opens fast, the functional parameter configuration to various emergency equipment, use the operation flow must be accurate, in the face of some complicated equipment operation flows, its operation process can rely on the manifest, the tradition is based on paper, electronic document, record video show mode, whole operating efficiency is not high, misoperation appears more easily under the condition that does not have the manifest, lead to appearing opening the operation delay condition.
And secondly, an unknown sudden problem occurs in the emergency opening process, and the problem of low efficiency caused by seeking external assistance is solved. If the equipment is not successfully opened, some equipment is in hardware failure, some equipment is in manual operation error, the equipment can be communicated in modes of video or telephone and the like by seeking help background technical support or qualified experts, the assistance process needs to provide field conditions, meanwhile, according to prompt operation, the communication and the operation are difficult to be simultaneously and effectively carried out, the communication is slightly higher in a video mode than a telephone mode, but the accurate operation equipment mark is lacked, and the efficiency is low.
Disclosure of Invention
The present invention is directed to solving the above problems of the background art, and provides an AR-based communication emergency device assisted repair system.
The purpose of the invention can be realized by the following technical scheme:
the embodiment of the invention provides an AR-based communication emergency equipment assisting and overhauling system, which comprises an AR terminal, an assisting terminal, a WebSocket server and a WebRTC server; the AR terminal comprises a first browser, an acquisition module, a projection module and a first shared display module; the assistance terminal comprises a second browser, a background display module and a second shared display module;
the WebRTC server is configured to perform NAT traversal between the first browser and the second browser, and establish a direct communication path between the first browser and the second browser;
the first browser and the second browser are used for audio and video communication through the direct communication path;
the background display module is used for synchronously displaying a video picture during audio and video communication and a background video picture acquired by the acquisition module;
the second shared display module is used for acquiring and displaying assistance data and sending the assistance data to the first shared display module through the WebSocket server;
the first shared display module is configured to determine a target display coordinate and a target display size of the assistance data;
the projection module is used for projecting a virtual screen at a preset distance in front of the AR terminal and displaying the assistance data on the virtual screen in the target display size.
Optionally, the first shared display module includes a coordinate conversion sub-module and a scaling sub-module;
the coordinate conversion sub-module is configured to convert the coordinates of the assistance data in the second shared display module into coordinates on the virtual screen, where the coordinates are used as the target display coordinates;
the scaling submodule is configured to calculate, according to the size of the assistance data in the second shared display module and the size parameters of the second shared display module, the virtual screen, and the acquisition module, the size of the assistance data on the virtual screen as the target display size.
Optionally, the target display size comprises a target length and a target width;
the scaling submodule is specifically configured to:
calculating the target length of the assistance data on the virtual screen:
Figure 514969DEST_PATH_IMAGE001
Figure 341105DEST_PATH_IMAGE002
for the purpose of the target length, the length of the target,
Figure 537731DEST_PATH_IMAGE003
for the length of the assistance data in the second shared display module,
Figure 275749DEST_PATH_IMAGE004
is the length of the virtual screen or screens,
Figure 684865DEST_PATH_IMAGE005
is the length of the second shared display module,
Figure 887876DEST_PATH_IMAGE006
is the distance between the acquisition module and the virtual screen,
Figure 255404DEST_PATH_IMAGE007
is the focal length of the acquisition module;
calculating the target width of the assistance data on the virtual screen:
Figure 965871DEST_PATH_IMAGE008
Figure 427945DEST_PATH_IMAGE009
in order to be the target width of the image,
Figure 9099DEST_PATH_IMAGE010
for the width of the assistance data in the second shared display module,
Figure 829419DEST_PATH_IMAGE011
is the width of the virtual screen in question,
Figure 496023DEST_PATH_IMAGE012
is the width of the second shared display module,
Figure 496209DEST_PATH_IMAGE006
is the distance between the acquisition module and the virtual screen,
Figure 931870DEST_PATH_IMAGE007
is the focal length of the acquisition module.
Optionally, the AR terminal further comprises a positioning and tracking module and an inertial sensor; the positioning and tracking module comprises an instant positioning and mapping SLAM sub-module and an inertial navigation sub-module;
the SLAM submodule is used for carrying out initial positioning on an initial picture when the assistance data is collected to create an initial coordinate system;
the inertial navigation sub-module is used for acquiring data of the inertial sensor and determining the offset of the current picture relative to the initial coordinate system;
the first shared display module is further configured to adjust a real-time position of the assistance data according to the offset.
Optionally, the positioning and tracking module further comprises an offset correction module;
the SLAM submodule is also used for positioning the current picture to obtain correction information when the offset is detected to be smaller than a preset threshold;
and the deviation correction module is used for determining the static deviation of the inertial sensor according to the correction information and adjusting the real-time position of the assistance data according to the static deviation.
Optionally, the deviation rectification module is an extended kalman filter.
Optionally, the WebRTC server includes a signaling server and a traversal server; the traversal server is any one of a STUN server, a TURN server and an ICE server;
the signaling server is used for signaling interaction between the first browser and the second browser;
the traversal server is configured to perform NAT traversal between the first browser and the second browser, and establish a direct communication path between the first browser and the second browser;
optionally, the background display module and the second shared display module have the same size, and the second shared display module is covered on the background display module.
Optionally, the assistance data includes at least one of a text and an image.
Optionally, the AR terminal further includes a video display module;
and the video display module is used for synchronously displaying video pictures during audio and video communication.
The embodiment of the invention provides an AR-based communication emergency equipment assisting and overhauling system, which comprises an AR terminal, an assisting terminal, a WebSocket server and a WebRTC server; the AR terminal comprises a first browser, an acquisition module, a projection module and a first shared display module; the assistance terminal comprises a second browser, a background display module and a second shared display module; the WebRTC server is used for performing NAT traversal between the first browser and the second browser and establishing a direct communication path between the first browser and the second browser; the first browser and the second browser are used for audio and video communication through a direct communication path; the background display module is used for synchronously displaying a video picture during audio and video communication and a background video picture acquired by the acquisition module; the second shared display module is used for acquiring and displaying the assistance data and sending the assistance data to the first shared display module through the WebSocket server; the first sharing display module is used for determining target display coordinates and target display sizes of the assistance data; and the projection module is used for projecting a virtual screen at a preset distance in front of the AR terminal and displaying the assistance data on the virtual screen in a target display size. The AR terminal and the assistance terminal are in audio and video communication through the WebRTC technology, a backstage technical support or a qualified expert can know the maintenance field situation through the assistance terminal, remote assistance can be carried out on field emergency maintenance personnel through voice and/or assistance data, the assistance data is displayed through the AR technology, the maintenance efficiency of the emergency maintenance personnel is improved, and the delay situation is avoided.
Drawings
The invention is further described below with reference to the accompanying drawings.
Fig. 1 is a system block diagram of an AR-based communication emergency device assisted overhaul system according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The embodiment of the invention provides an AR-based communication emergency equipment auxiliary overhaul system. Referring to fig. 1, fig. 1 is a system block diagram of an AR-based communication emergency device assisted overhaul system according to an embodiment of the present invention. The system comprises an AR terminal, an assistance terminal, a WebSocket server and a WebRTC server; the AR terminal comprises a first browser, an acquisition module, a projection module and a first shared display module; the assistance terminal comprises a second browser, a background display module and a second shared display module;
the WebRTC server is configured to perform NAT traversal between the first browser and the second browser, and establish a direct communication path between the first browser and the second browser;
the first browser and the second browser are used for audio and video communication through the direct communication path;
the background display module is used for synchronously displaying a video picture during audio and video communication and a background video picture acquired by the acquisition module;
the second shared display module is used for acquiring and displaying assistance data and sending the assistance data to the first shared display module through the WebSocket server;
the first shared display module is configured to determine a target display coordinate and a target display size of the assistance data;
the projection module is used for projecting a virtual screen at a preset distance in front of the AR terminal and displaying the assistance data on the virtual screen in the target display size.
Based on the AR-based communication emergency equipment auxiliary maintenance system provided by the embodiment of the invention, the audio and video communication between the AR terminal and the auxiliary terminal is realized through the WebRTC technology, a backstage technical support or a senior specialist can know the maintenance field condition through the auxiliary terminal, can remotely assist field emergency maintenance personnel through voice and/or auxiliary data, and can display the auxiliary data through the AR technology, thereby improving the maintenance efficiency of the emergency maintenance personnel and avoiding the delay condition.
In one implementation, a WebRTC server establishes a direct communication path between a first browser and a second browser, so that transmission delay of audio/video communication between an AR terminal and an assistant terminal can be reduced.
In one implementation, the first shared display module and the second shared display module may be HTML 5-based virtual shared whiteboards.
In one implementation, the acquisition module may be a camera for acquiring video data of a maintenance site.
In one embodiment, the AR terminal includes a binocular light-guiding transparent holographic lens.
In one embodiment, the first shared display module includes a coordinate conversion sub-module and a zoom sub-module;
the coordinate conversion sub-module is used for converting the coordinates of the assistance data on the second sharing display module into coordinates on the virtual screen, and the coordinates are used as the target display coordinates;
the scaling submodule is configured to calculate, according to the size of the assistance data in the second shared display module and the size parameters of the second shared display module, the virtual screen, and the acquisition module, the size of the assistance data on the virtual screen as the target display size.
In one embodiment, the target display size comprises a target length and a target width;
the scaling submodule is specifically configured to:
calculating the target length of the assistance data on the virtual screen:
Figure 172358DEST_PATH_IMAGE013
(1)
Figure 74062DEST_PATH_IMAGE002
for the purpose of the target length, the length of the target,
Figure 894250DEST_PATH_IMAGE003
for the length of the assistance data in the second shared display module,
Figure 433685DEST_PATH_IMAGE004
is the length of the virtual screen or screens,
Figure 48337DEST_PATH_IMAGE005
for the length of the second shared display module,
Figure 174687DEST_PATH_IMAGE006
is the distance between the acquisition module and the virtual screen,
Figure 532987DEST_PATH_IMAGE007
is the focal length of the acquisition module;
calculating the target width of the assistance data on the virtual screen:
Figure 989245DEST_PATH_IMAGE014
(2)
Figure 509220DEST_PATH_IMAGE009
in order to be the target width of the image,
Figure 125795DEST_PATH_IMAGE010
width of the assistance data in the second shared display module,
Figure 287786DEST_PATH_IMAGE011
Is the width of the virtual screen in question,
Figure 349283DEST_PATH_IMAGE012
being the width of the second shared display module,
Figure 820585DEST_PATH_IMAGE006
is the distance between the acquisition module and the virtual screen,
Figure 905216DEST_PATH_IMAGE007
is the focal length of the acquisition module.
In one implementation, the length width of the virtual screen, the length width of the second shared display module, and the distance between the acquisition module and the virtual screen are constants set by a technician.
Figure 356051DEST_PATH_IMAGE015
The current focal length of the acquisition module can be obtained from the acquisition module.
In one embodiment, the AR terminal further comprises a positioning tracking module and an inertial sensor; the positioning and tracking module comprises an instant positioning and mapping (SLAM) sub-module and an inertial navigation sub-module;
the SLAM sub-module is used for initially positioning an initial picture when the assistance data is collected to create an initial coordinate system;
the inertial navigation sub-module is used for acquiring data of the inertial sensor and determining the offset of the current picture relative to the initial coordinate system;
the first shared display module is further configured to adjust a real-time position of the assistance data according to the offset.
In one implementation, the inertial sensors may be high-precision gyroscopes, acceleration sensors, distance sensors, and the like.
In one implementation, on the basis of positioning of the SLAM submodule, if a person wearing the AR terminal performs operations such as moving and rotating, the real-time position of the assistance data can be adjusted through the inertial navigation submodule, and seamless fusion of virtual display content and a real scene is guaranteed.
In one embodiment, the location tracking module further comprises an offset rectification module;
the SLAM submodule is further used for positioning the current picture to obtain correction information when the detected offset is smaller than a preset threshold;
and the deviation correction module is used for determining the static deviation of the inertial sensor according to the correction information and adjusting the real-time position of the assistance data according to the static deviation.
In one implementation, when the detected offset is smaller than the preset threshold, the AR terminal may be regarded as a stationary state, and at this time, the integral error accumulated over time in the inertial navigation may be corrected by the SLAM sub-module.
In one embodiment, the bias remediation module is an extended kalman filter.
In one embodiment, the WebRTC server comprises a signaling server and a traversal server; the traversal server is any one of a STUN server, a TURN server and an ICE server;
the signaling server is used for signaling interaction between the first browser and the second browser;
the traversal server is configured to perform NAT traversal between the first browser and the second browser, and establish a direct communication path between the first browser and the second browser;
in one embodiment, the background display module and the second shared display module are the same size, and the second shared display module is overlaid on the background display module.
In one implementation, the background technical support or the senior expert may use a mouse or a finger to draw or input text on the second shared display module according to the video content of the background display module, and may add text, send a file, and erase content on the second shared display module.
In one embodiment, the assistance data includes at least one of text and images.
In one implementation, objects or required maintenance steps in the video are marked through characters and images, so that on-site emergency maintenance personnel can more clearly understand the provided maintenance suggestions of the background technical support or the qualification experts, and the communication efficiency between the emergency maintenance personnel and the background technical support or the qualification experts is improved.
In one embodiment, the AR terminal further comprises a video display module;
and the video display module is used for synchronously displaying video pictures during audio and video communication.
Although one embodiment of the present invention has been described in detail, the description is only for the purpose of illustrating the preferred embodiments of the present invention and should not be taken as limiting the scope of the invention. All equivalent changes and modifications made within the scope of the present invention shall fall within the scope of the present invention.

Claims (9)

1. The communication emergency equipment assisting maintenance system based on the AR is characterized by comprising an AR terminal, an assisting terminal, a WebSocket server and a WebRTC server; the AR terminal comprises a first browser, an acquisition module, a projection module and a first shared display module; the assistance terminal comprises a second browser, a background display module and a second shared display module; the first sharing display module and the second sharing display module are virtual sharing white boards based on HTML 5;
the WebRTC server is configured to perform NAT traversal between the first browser and the second browser, and establish a direct communication path between the first browser and the second browser;
the first browser and the second browser are used for audio and video communication through the direct communication path;
the background display module is used for synchronously displaying a video picture during audio and video communication and a background video picture acquired by the acquisition module;
the second shared display module is used for acquiring and displaying assistance data and sending the assistance data to the first shared display module through the WebSocket server;
the first shared display module is configured to determine a target display coordinate and a target display size of the assistance data;
the projection module is used for projecting a virtual screen at a preset distance in front of the AR terminal and displaying the assistance data on the virtual screen in the target display size;
the AR terminal also comprises a video display module;
and the video display module is used for synchronously displaying video pictures during audio and video communication.
2. The AR-based communication emergency device assisted repair system of claim 1, wherein the first shared display module comprises a coordinate transformation sub-module and a scaling sub-module;
the coordinate conversion sub-module is configured to convert the coordinates of the assistance data in the second shared display module into coordinates on the virtual screen, where the coordinates are used as the target display coordinates;
the scaling sub-module is configured to calculate, according to the size of the assistance data in the second shared display module and the size parameters of the second shared display module, the virtual screen, and the acquisition module, the size of the assistance data on the virtual screen as the target display size.
3. The AR-based communication emergency equipment assisted services system of claim 2, wherein the target display size comprises a target length and a target width;
the scaling submodule is specifically configured to:
calculating the target length of the assistance data on the virtual screen:
Figure DEST_PATH_IMAGE002
Figure DEST_PATH_IMAGE004
in order to be said target length, the length of the target,
Figure DEST_PATH_IMAGE006
for the length of the assistance data in the second shared display module,
Figure DEST_PATH_IMAGE008
is the length of the virtual screen or screens,
Figure DEST_PATH_IMAGE010
for the length of the second shared display module,
Figure DEST_PATH_IMAGE012
is the distance between the acquisition module and the virtual screen,
Figure DEST_PATH_IMAGE014
is the focal length of the acquisition module;
calculating the target width of the assistance data on the virtual screen:
Figure DEST_PATH_IMAGE016
Figure DEST_PATH_IMAGE018
in order to be able to obtain the target width,
Figure DEST_PATH_IMAGE020
for the width of the assistance data in the second shared display module,
Figure DEST_PATH_IMAGE022
is the width of the virtual screen in question,
Figure DEST_PATH_IMAGE024
being the width of the second shared display module,
Figure 722204DEST_PATH_IMAGE012
is the distance between the acquisition module and the virtual screen,
Figure 710888DEST_PATH_IMAGE014
is the focal length of the acquisition module.
4. The AR-based communication emergency device assisted repair system of claim 1, wherein the AR terminal further comprises a location tracking module and an inertial sensor; the positioning and tracking module comprises an instant positioning and mapping SLAM sub-module and an inertial navigation sub-module;
the SLAM submodule is used for carrying out initial positioning on an initial picture when the assistance data is collected to create an initial coordinate system;
the inertial navigation sub-module is used for acquiring data of the inertial sensor and determining the offset of the current picture relative to an initial coordinate system;
the first shared display module is further configured to adjust a real-time position of the assistance data according to the offset.
5. The AR based communication emergency device assisted repair system of claim 4, wherein the position tracking module further comprises an offset rectification module;
the SLAM submodule is further used for positioning the current picture to obtain correction information when the detected offset is smaller than a preset threshold;
and the deviation correction module is used for determining the static deviation of the inertial sensor according to the correction information and adjusting the real-time position of the assistance data according to the static deviation.
6. The AR-based communication emergency device assisted overhaul system of claim 5, wherein the deviation rectification module is an extended Kalman filter.
7. The AR-based communication emergency device assisted overhaul system of claim 1, wherein the WebRTC server comprises a signaling server and a traversal server; the traversal server is any one of a STUN server, a TURN server and an ICE server;
the signaling server is used for signaling interaction between the first browser and the second browser;
and the traversing server is used for performing NAT traversal between the first browser and the second browser and establishing a direct communication path between the first browser and the second browser.
8. The AR based communication emergency device assisted overhaul system of claim 1, wherein the background display module and the second shared display module are the same size and the second shared display module overlays the background display module.
9. The AR-based communication emergency device assisted overhaul system of claim 1, wherein the assistance data comprises at least one of text and images.
CN202211245639.7A 2022-10-12 2022-10-12 Communication emergency equipment assists maintenance system based on AR Active CN115314474B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211245639.7A CN115314474B (en) 2022-10-12 2022-10-12 Communication emergency equipment assists maintenance system based on AR

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211245639.7A CN115314474B (en) 2022-10-12 2022-10-12 Communication emergency equipment assists maintenance system based on AR

Publications (2)

Publication Number Publication Date
CN115314474A CN115314474A (en) 2022-11-08
CN115314474B true CN115314474B (en) 2023-01-17

Family

ID=83868088

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211245639.7A Active CN115314474B (en) 2022-10-12 2022-10-12 Communication emergency equipment assists maintenance system based on AR

Country Status (1)

Country Link
CN (1) CN115314474B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111260084A (en) * 2020-01-09 2020-06-09 长安大学 Remote system and method based on augmented reality collaborative assembly maintenance
CN114662714A (en) * 2022-02-25 2022-06-24 南京邮电大学 Machine room operation and maintenance management system and method based on AR equipment

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110189416B (en) * 2019-05-29 2023-05-05 国家电网有限公司 Remote guiding method and system for overhaul site
CN110381111A (en) * 2019-06-03 2019-10-25 华为技术有限公司 A kind of display methods, location determining method and device
US11475661B2 (en) * 2020-02-10 2022-10-18 Fujifilm Business Innovation Corp. Systems and methods for augmented reality application for annotations and adding interfaces to control panels and screens

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111260084A (en) * 2020-01-09 2020-06-09 长安大学 Remote system and method based on augmented reality collaborative assembly maintenance
CN114662714A (en) * 2022-02-25 2022-06-24 南京邮电大学 Machine room operation and maintenance management system and method based on AR equipment

Also Published As

Publication number Publication date
CN115314474A (en) 2022-11-08

Similar Documents

Publication Publication Date Title
CN108932051B (en) Augmented reality image processing method, apparatus and storage medium
CN108307183A (en) Virtual scene method for visualizing and system
US7377650B2 (en) Projection of synthetic information
JPH06284330A (en) Monitor camera controller linked with map information
AU2007249585B2 (en) Real-time capture and transformation of hemispherical video images to images in rectilinear coordinates
CN112291478B (en) Method, device and equipment for monitoring high-altitude falling object and storage medium
CN115314474B (en) Communication emergency equipment assists maintenance system based on AR
JPH10136345A (en) Information service system
JP6696925B2 (en) Operation support device
CN117135146A (en) Remote guidance system based on AR technology
JP2000253391A (en) Panorama video image generating system
JPH11298872A (en) Video communication system
TWI793584B (en) Mapping and localization system for automated valet parking and method thereof
JP2005142765A (en) Apparatus and method for imaging
CN114742977A (en) Video perspective method based on AR technology
US20210168292A1 (en) Operation assistance device, operation assistance method, and recording medium
WO2021149509A1 (en) Imaging device, imaging method, and program
CN115268658A (en) Multi-party remote space delineation marking method based on augmented reality
JP4568121B2 (en) Remote work support system, remote work support method and program
CN113965550A (en) Intelligent interactive remote auxiliary video system
CN117640914B (en) Remote equipment maintenance method and system based on AR video and multiparty real-time cooperation
CN110602456A (en) Display method and system of aerial photography focus
JP2002152722A (en) Image projector and its method
TW201033586A (en) Navigation device with real-time image incorporating navigating information and method thereof
JP2005123750A (en) Video-map interlocking system and interlocking method thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant