CN105759960A - Augmented reality remote guidance method and system in combination with 3D camera - Google Patents
Augmented reality remote guidance method and system in combination with 3D camera Download PDFInfo
- Publication number
- CN105759960A CN105759960A CN201610074722.0A CN201610074722A CN105759960A CN 105759960 A CN105759960 A CN 105759960A CN 201610074722 A CN201610074722 A CN 201610074722A CN 105759960 A CN105759960 A CN 105759960A
- Authority
- CN
- China
- Prior art keywords
- information
- augmented reality
- video flowing
- target
- limb
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses an augmented reality remote guidance method and system in combination with a 3D camera. The method includes the steps that a video stream and corresponding augmented reality information sent by an operation guidance subsystem are received, wherein the video stream is obtained by carrying out optical collection on a target, and the augmented reality information is obtained by registering a virtual vidicon; a 3D model of the target is established according to the augmented reality information; the video stream is displayed through a 3D viewport, and the 3D model of the target is loaded; limb motion information of a guider is collected through the 3D camera; a limb 3D model and 3D motion information are generated according to the limb motion information and are loaded and displayed in the 3D viewport; 3D guidance information is generated according to the limb 3D model and the 3D motion information; the 3D guidance information is sent to the operation guidance subsystem so that the operation guidance subsystem can generate and display a guidance scene according to the video stream, the augmented reality information and the 3D guidance information. By means of the method, accurate and visible limb operation guidance is provided for field operation staff.
Description
Technical field
The application relates to long-range bootstrap technique field, is specifically related to a kind of augmented reality remote guide method and system in conjunction with 3D photographic head.
Background technology
The technical scheme of existing long-range guiding execute-in-place mainly includes two parts, a part is on-the-spot end, a part is remote port, the operator of on-the-spot end wear or hold the intelligent terminal with optical capturing equipment, the video acquisition of site environment and send remote port to, the personnel that instruct of remote port can make mark in the on-the-spot outdoor scene video received, and returns in real time on the display device of operator of on-the-spot end, and the operation of operator is comparatively instructed intuitively.
For relating to the directiveness mark relevant with limbs, traditional method be remote port instruct personnel, the model (such as hands) preset by mouse-keyboard operation carries out some simulated operations, it is usually present the problem that operation is very loaded down with trivial details, and the simulated operation inputted by mouse-keyboard is often difficult to the operation that accurate expression actually wants to realize, thus being difficult for site operation personnel to provide accurate and limbs operation guide intuitively.
Summary of the invention
In view of drawbacks described above of the prior art or deficiency, it is desirable to provide a kind of and can provide accurate and limbs operation guide intuitively the augmented reality remote guide method and system in conjunction with 3D photographic head for site operation personnel.
First aspect, the present invention provides a kind of augmented reality remote guide method in conjunction with 3D photographic head of operation guide subsystem for on-the-spot end, and described method includes:
Target is carried out optically detecting to obtain video flowing；
Registration virtual video camera, generates the augmented reality information corresponding to described video flowing；
Described video flowing and described augmented reality information is sent to long-range diagnostic subsystem；
Receive the 3D guidance information that described long-range diagnostic subsystem returns；Wherein, described 3D guidance information generates according to limb three-dimensional model and three-dimensional motion information, and the limb action information instructing personnel that described limb three-dimensional model and three-dimensional motion information gather according to 3D photographic head generates；
Generate according to described video flowing, described augmented reality information and 3D guidance information and guide scene；
Show described guiding scene.
Second aspect, the present invention provides a kind of augmented reality remote guide method in conjunction with 3D photographic head of long-range diagnostic subsystem for remote port, and described method includes:
Receive the video flowing that operation guide subsystem sends and the augmented reality information corresponding to described video flowing；Wherein, described video flowing obtains by target is carried out optically detecting, and described augmented reality information obtains by registering virtual video camera；
The threedimensional model of target according to described augmented reality information creating；
Show described video flowing by 3D viewport and load the threedimensional model of described target；
The limb action information of personnel is instructed by 3D camera collection；
Generate limb three-dimensional model and three-dimensional motion information according to described limb action information, and in described 3D viewport loaded and displayed；
3D guidance information is generated according to described limb three-dimensional model and three-dimensional motion information；
Send described 3D guidance information to described operation guide subsystem, generate for described operation guide subsystem according to described video flowing, described augmented reality information and described 3D guidance information and show guiding scene.
The third aspect, the present invention provides a kind of augmented reality remote guide system in conjunction with 3D photographic head, and described system includes operation guide subsystem and long-range diagnostic subsystem.
Described operation guide subsystem includes:
Collecting unit, for carrying out optically detecting to obtain video flowing to target；
First processing unit, is used for registering virtual video camera, generates the augmented reality information corresponding to described video flowing, generates according to described video flowing, described augmented reality information and 3D guidance information and guides scene；
First display unit, is used for showing described guiding scene；
First communication unit, for sending described video flowing and described augmented reality information to described long-range diagnostic subsystem, receives the 3D guidance information that described long-range diagnostic subsystem returns；
Described long-range diagnostic subsystem includes:
3D photographic head, for gathering the limb action information instructing personnel；
Second communication unit, is used for receiving described video flowing and described augmented reality information, sends 3D guidance information to described operation guide subsystem；
Second processing unit, for the threedimensional model of target according to described augmented reality information creating, generate limb three-dimensional model and three-dimensional motion information according to described limb action information, and generate 3D guidance information according to described limb three-dimensional model and three-dimensional motion information；
Second display unit, for showing described video flowing by 3D viewport and loading the threedimensional model of described target, and in described 3D viewport limb three-dimensional model and three-dimensional motion information described in loaded and displayed.
The augmented reality remote guide method and system in conjunction with 3D photographic head that the many embodiments of the present invention provide is operated by guiding subsystem gather the video flowing of target and register virtual video camera, generate the augmented reality information corresponding to video flowing and send to remote port, thus building at long-range diagnostic subsystem and showing the threedimensional model of target；The limb action information of personnel is instructed by the 3D camera collection of long-range diagnostic subsystem, generate corresponding limb three-dimensional model and three-dimensional motion information and by 3D viewport loaded and displayed, achieve and accurately and intuitively input limbs operation, and the 3D guidance information generating correspondence sends to on-the-spot operation guide subsystem；On-the-spot operation guide subsystem generates according to described video flowing, described augmented reality information and 3D guidance information and guides scene, finally achieves and provides accurate and limbs operation guide intuitively for site operation personnel；
The augmented reality remote guide method and system in conjunction with 3D photographic head that some embodiments of the invention provide marks in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target and eventually through guiding scene to show by inputting, it is further provided more accurate limbs operation guide；
The threedimensional model that the augmented reality remote guide method and system in conjunction with 3D photographic head that some embodiments of the invention provide realizes target by augmented reality builds, and by the sequential that Work-flow control shows in the guiding scene generated according to the 3D enhancing guidance information that the limb action information of remote camera collection generates, guide intuitively eventually through guiding scene accurately to achieve；
The augmented reality remote guide method and system in conjunction with 3D photographic head that some embodiments of the invention provide is by visual tracking registration, sensor Tracing Registration or the accuracy mixing the threedimensional model that the multiple virtual video camera logon mode such as registration ensure that the video flowing according to optically detecting generates.
Accompanying drawing explanation
By reading the detailed description that non-limiting example is made made with reference to the following drawings, other features, purpose and advantage will become more apparent upon:
The flow chart of the augmented reality remote guide method in conjunction with 3D photographic head of the operation guide subsystem for on-the-spot end that Fig. 1 provides for one embodiment of the invention.
Fig. 2 be augmented reality remote guide method shown in Fig. 1 an embodiment in the flow chart of step S18.
Fig. 3 be augmented reality remote guide method shown in Fig. 1 a preferred embodiment in the flow chart of step S18.
Fig. 4 is the flow chart of a preferred embodiment of augmented reality remote guide method shown in Fig. 1.
The flow chart of the augmented reality remote guide method in conjunction with 3D photographic head of the long-range diagnostic subsystem for remote port that Fig. 5 provides for one embodiment of the invention.
Fig. 6 is the flow chart of a preferred embodiment of augmented reality remote guide method shown in Fig. 5.
The structural representation of the augmented reality remote guide system in conjunction with 3D photographic head that Fig. 7 provides for one embodiment of the invention.
Fig. 8 is the structural representation of a preferred embodiment of augmented reality remote guide system shown in Fig. 7.
Detailed description of the invention
Below in conjunction with drawings and Examples, the application is described in further detail.It is understood that specific embodiment described herein is used only for explaining related invention, but not the restriction to this invention.It also should be noted that, for the ease of describing, accompanying drawing illustrate only and invent relevant part.
It should be noted that when not conflicting, the embodiment in the application and the feature in embodiment can be mutually combined.Describe the application below with reference to the accompanying drawings and in conjunction with the embodiments in detail.
The flow chart of the augmented reality remote guide method in conjunction with 3D photographic head of the operation guide subsystem for on-the-spot end that Fig. 1 provides for one embodiment of the invention.
As it is shown in figure 1, in the present embodiment, the augmented reality remote guide method in conjunction with 3D photographic head provided by the invention includes:
S11: target is carried out optically detecting to obtain video flowing；
S13: registration virtual video camera, generates the augmented reality information corresponding to described video flowing；
S15: send described video flowing and described augmented reality information to long-range diagnostic subsystem；
S17: receive the 3D guidance information that described long-range diagnostic subsystem returns；Wherein, described 3D guidance information generates according to limb three-dimensional model and three-dimensional motion information, and the limb action information instructing personnel that described limb three-dimensional model and three-dimensional motion information gather according to 3D photographic head generates；
S18: generate according to described video flowing, described augmented reality information and described 3D guidance information and guide scene；
S19: show described guiding scene.
Specifically, by collecting unit, target is carried out optically detecting, after obtaining video flowing, determine position and the visual angle of collecting unit by registering virtual video camera, thus the position that should present in the real scene of described video flowing of the threedimensional model calculating target and attitude.
Described augmented reality information comprises the three-dimensional modeling data of described target, the threedimensional model position in the real scene of described video flowing of described target and attitude information (abbreviation posture information).
Described three-dimensional modeling data can comprise complete three-dimensional modeling data in certain embodiments, can only comprise the coding that described threedimensional model is corresponding in a certain presetting or three-dimensional modeling data storehouse of preassignment in further embodiments.
After long-range diagnostic subsystem receives described video flowing and described augmented reality information, the threedimensional model of target according to described augmented reality information creating, and the limb action of personnel is instructed by the scanning of 3D photographic head, to gather the limb action information instructing personnel, generate corresponding limb three-dimensional model and three-dimensional motion information according to the limb action information gathered, generate 3D guidance information further according to described limb three-dimensional model and three-dimensional motion information.
Described 3D guidance information can comprise complete limb three-dimensional model and three-dimensional motion information in certain embodiments, and the information prestored in preassignment or presetting technical data data base in described limb three-dimensional model and three-dimensional motion information replaces with the coding of correspondence in more embodiment.After the operation guide subsystem of on-the-spot end receives described 3D guidance information, in preassignment or presetting technical data data base, obtain the information of correspondence to reduce described limb three-dimensional model and three-dimensional motion information by the coding of described correspondence.
Finally by the real scene of described video flowing, through virtual video camera registration target three-dimensional and described limb three-dimensional model and three-dimensional motion information synthesis, generate guide scene, and finally show guide scene.
Above-described embodiment is operated by guiding subsystem gather the video flowing of target and register virtual video camera, generates the augmented reality information corresponding to video flowing and sends to remote port, thus building at long-range diagnostic subsystem and showing the threedimensional model of target；The limb action information of personnel is instructed by the 3D camera collection of long-range diagnostic subsystem, generate corresponding limb three-dimensional model and three-dimensional motion information and by 3D viewport loaded and displayed, achieve and accurately and intuitively input limbs operation, and the 3D guidance information generating correspondence sends to on-the-spot operation guide subsystem；On-the-spot operation guide subsystem generates according to described video flowing, described augmented reality information and 3D guidance information and guides scene, finally achieves and provides accurate and limbs operation guide intuitively for site operation personnel.
In a preferred embodiment, described three-dimensional motion information also includes the interactive information of described limb three-dimensional model and target three-dimensional.
Specifically, long-range diagnostic subsystem calculates, according to the technical data relevant to target of storage in the data base called, the interactive information with target three-dimensional generation that the action of described limb three-dimensional model causes, and shown by 3D viewport in real time, make to instruct personnel the pattern of " What You See Is What You Get " precise manipulation to target three-dimensional can be realized, guide scene again through generating the 3D guidance information transmission operation guide subsystem generation to on-the-spot end, finally realize accurate limbs operation guide.
In a preferred embodiment, described 3D guidance information also includes marking in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target.Wherein, the threedimensional model of described target is according to described augmented reality information creating.
Specifically, long-range diagnostic subsystem is showing on the basis of target three-dimensional, limb three-dimensional model and both interactive operations by 3D viewport, markup information can also be marked by input block at any part of limb three-dimensional model or target three-dimensional further, described markup information include word, image, voice, to all kinds of different information such as the pseudo operations of threedimensional model.
Above-described embodiment marks in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target and eventually through guiding scene to show by inputting, it is further provided more accurate limbs operation guide.
Fig. 2 be augmented reality remote guide method shown in Fig. 1 an embodiment in the flow chart of step S18.
As in figure 2 it is shown, in one embodiment, step S18 includes:
S181: according to described 3D guidance information retrieval technique document data base, generates 3D and strengthens guidance information；
S183: strengthen guidance information according to described video flowing, described augmented reality information and 3D and generate guiding scene.
Wherein, described technical data data base is for storing the technical data relevant to described target and threedimensional model.
Specifically, by retrieval technique document data base reduction partial 3-D model and/or markup information, make 3D guidance information need not comprise complete threedimensional model and/or markup information, the data that part prestores technical data data base can significantly be compressed, thus reducing remote port and the communication pressure of on-the-spot end.
Fig. 3 be augmented reality remote guide method shown in Fig. 1 a preferred embodiment in the flow chart of step S18.
As it is shown on figure 3, in a preferred embodiment, step S18 includes:
S181: according to described 3D guidance information retrieval technique document data base, generates 3D and strengthens guidance information；
S182: control described 3D and strengthen the sequential that guidance information shows in the guiding scene generated；
S183: strengthen guidance information according to described video flowing, described augmented reality information and 3D and generate guiding scene.
The threedimensional model that above-described embodiment realizes target by augmented reality builds, and by the sequential that Work-flow control shows in the guiding scene generated according to the 3D enhancing guidance information that the limb action information of remote camera collection generates, guide intuitively eventually through guiding scene accurately to achieve.
In a preferred embodiment, described registration virtual video camera includes visual tracking registration, sensor Tracing Registration or mixing registration.
The registration of described visual tracking includes: identify the image of target described in described video flowing, and the image of described target is processed, and obtains visual tracking log-on message, generates the augmented reality information corresponding to described video flowing.
Described sensor Tracing Registration includes: obtain spatial transformation information by sensor, according to described spatial transformation information, described video flowing is processed, and obtains sensor Tracing Registration information, generates the augmented reality information corresponding to described video flowing.
Described mixing registration includes: carries out described visual tracking registration and described sensor Tracing Registration, comprehensive described visual tracking log-on message and described sensor Tracing Registration information respectively and generates the augmented reality information corresponding to described video flowing.
Wherein, described sensor and the collecting unit that target carries out optically detecting are synchronized with the movement.
Specifically, mixing registration mode is adopted in the present embodiment.Reference frame corresponding to the measurement data that the registration of described visual tracking and described sensor Tracing Registration respectively obtain is each different, need the constant-coordinate conversion first demarcated between the two, then measurement data is transformed in a unified reference frame, then measure mixing or the fusion of data.When visual tracking weight reduces, using the measurement data of sensor tracking as compensation, by default weighted fusion algorithm, it is ensured that certainty of measurement.
In more embodiments, it is possible to individually adopt visual tracking registration mode or sensor Tracing Registration pattern according to real needs.
In more embodiments, described registration virtual video camera also includes optical head location tracking registration law.
In more embodiments, described mixing registration also includes optical head location tracking and registers and visual tracking registration, or, the registration of optical head location tracking is registered with sensor.
Above-described embodiment is registered by visual tracking, sensor Tracing Registration or the mixing multiple virtual video camera logon mode such as registration ensure that the accuracy of threedimensional model that the video flowing according to optically detecting generates.
In a preferred embodiment, described visual tracking registers through the three-dimensional modeling data storehouse having the three-dimensional modeling data relevant to described target and identifies the image of target described in described video flowing, and generates described augmented reality information.
Fig. 4 is the flow chart of a preferred embodiment of augmented reality remote guide method shown in Fig. 1.
As shown in Figure 4, in a preferred embodiment, also include before step S15:
S14: generate according to described video flowing and described augmented reality information and show guiding scene.
Specifically, after virtual video camera has been registered, namely can be displayed in real scene the threedimensional model merging target.
The flow chart of the augmented reality remote guide method in conjunction with 3D photographic head of the long-range diagnostic subsystem for remote port that Fig. 5 provides for one embodiment of the invention.
As it is shown in figure 5, in the present embodiment, the augmented reality remote guide method in conjunction with 3D photographic head provided by the invention includes:
S21: receive the video flowing that operation guide subsystem sends and the augmented reality information corresponding to described video flowing；Wherein, described video flowing obtains by target is carried out optically detecting, and described augmented reality information obtains by registering virtual video camera；
S22: the threedimensional model of target according to described augmented reality information creating；
S23: show described video flowing by 3D viewport and load the threedimensional model of described target；
S25: instructed the limb action information of personnel by 3D camera collection；
S26: generate limb three-dimensional model and three-dimensional motion information according to described limb action information, and in described 3D viewport loaded and displayed；
S28: generate 3D guidance information according to described limb three-dimensional model and three-dimensional motion information；
S29: send described 3D guidance information to described operation guide subsystem, generate for described operation guide subsystem according to described video flowing, described augmented reality information and described 3D guidance information and show guiding scene.
In a preferred embodiment, described three-dimensional motion information also includes the interactive information of described limb three-dimensional model and target three-dimensional.
Fig. 6 is the flow chart of a preferred embodiment of augmented reality remote guide method shown in Fig. 5.
As shown in Figure 6, in a preferred embodiment, also include after step S26:
S27 input mark in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target, and in described 3D viewport markup information described in loaded and displayed.
In step S28, described 3D guidance information generates according to described limb three-dimensional model, described three-dimensional motion information and described markup information.
The structural representation of the augmented reality remote guide system in conjunction with 3D photographic head that Fig. 7 provides for one embodiment of the invention.
As it is shown in fig. 7, in the present embodiment, the augmented reality remote guide system in conjunction with 3D photographic head provided by the invention includes operation guide subsystem 10 and long-range diagnostic subsystem 20.
Operation guide subsystem 10 includes:
Collecting unit 101, for carrying out optically detecting to obtain video flowing to target；
First processing unit 103, is used for registering virtual video camera, generates the augmented reality information corresponding to described video flowing, generates according to described video flowing, described augmented reality information and 3D guidance information and guides scene；
First display unit 105, is used for showing described guiding scene；
First communication unit 107, for sending described video flowing and described augmented reality information to long-range diagnostic subsystem 20, receives the 3D guidance information that long-range diagnostic subsystem 20 returns.
Long-range diagnostic subsystem 20 includes:
3D photographic head 207, for gathering the limb action information instructing personnel；
Second communication unit 201, is used for receiving described video flowing and described augmented reality information, sends 3D guidance information to operation guide subsystem 10；
Second processing unit 203, for the threedimensional model of target according to described augmented reality information creating, generate limb three-dimensional model and three-dimensional motion information according to described limb action information, and generate 3D guidance information according to described limb three-dimensional model and three-dimensional motion information；
Second display unit 205, for showing described video flowing by 3D viewport and loading the threedimensional model of described target, and in described 3D viewport limb three-dimensional model and three-dimensional motion information described in loaded and displayed.
In a preferred embodiment, the first processing unit 103 includes:
Registration subelement, is used for registering virtual video camera, generates the augmented reality information corresponding to described video flowing, strengthens guidance information according to described video flowing, augmented reality information and 3D and generates guiding scene；
Intelligent retrieval subelement, strengthens guidance information for generating 3D according to described 3D guidance information, and controls the sequential that described 3D enhancing guidance information shows in the guiding scene generated.
In a preferred embodiment, described registration subelement includes:
Virtual video camera Registering modules, for described video flowing having been processed registration virtual video camera by image procossing or the spatial transformation information that obtained by sensor, generation is corresponding to the augmented reality information of described video flowing；
Guide scenario generating module, guide scene for generating according to described video flowing and described augmented reality information, or, generate according to described video flowing, described augmented reality information and 3D enhancing guidance information and guide scene.
Wherein, described sensor and described collecting unit are synchronized with the movement.
In a preferred embodiment, described virtual video camera Registering modules includes:
Visual tracking registration submodule, for identifying the image of target described in described video flowing, and processes the image of described target, obtains visual tracking log-on message；
Sensor Tracing Registration submodule, for obtaining spatial transformation information by described sensor, processes described video flowing according to described spatial transformation information, obtains sensor Tracing Registration information；
Augmented reality information generating module, for generating the augmented reality information corresponding to described video flowing according to described visual tracking log-on message and/or described sensor Tracing Registration information.
In a preferred embodiment, the first processing unit 103 also includes:
Three-dimensional modeling data storehouse, for storing the three-dimensional modeling data relevant to described target, registers the image of target described in video flowing described in submodule identification for described visual tracking, and generates described augmented reality information.
In a preferred embodiment, described intelligent retrieval subelement includes:
Guide retrieval module, for according to described 3D guidance information retrieval technique document data base, generating 3D and strengthen guidance information；
Work-flow control module, strengthens, for controlling described 3D, the sequential that guidance information shows in the guiding scene generated；
Technical data data base, for storing the technical data relevant to described target and threedimensional model.
In a preferred embodiment, described three-dimensional motion information also includes the interactive information of described limb three-dimensional model and target three-dimensional.
In a preferred embodiment, long-range diagnostic subsystem 20 also includes:
Input block 209, marks in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target for inputting.
Second processing unit 203 is additionally operable to generate 3D guidance information according to described limb three-dimensional model, described three-dimensional motion information and described markup information.
Specifically, input block 209 include following at least one: keyboard, mouse, touch screen, voice-input device and body-sensing input equipment.
Above description is only the preferred embodiment of the application and the explanation to institute's application technology principle.Skilled artisan would appreciate that, invention scope involved in the application, it is not limited to the technical scheme of the particular combination of above-mentioned technical characteristic, when also should be encompassed in without departing from described inventive concept simultaneously, other technical scheme being carried out combination in any by above-mentioned technical characteristic or its equivalent feature and being formed.Such as features described above and (but not limited to) disclosed herein have the technical characteristic of similar functions and replace mutually and the technical scheme that formed.
Claims (19)
1. the augmented reality remote guide method in conjunction with 3D photographic head, it is characterised in that described method includes:
Target is carried out optically detecting to obtain video flowing；
Registration virtual video camera, generates the augmented reality information corresponding to described video flowing；
Described video flowing and described augmented reality information is sent to long-range diagnostic subsystem；
Receive the 3D guidance information that described long-range diagnostic subsystem returns；Wherein, described 3D guidance information generates according to limb three-dimensional model and three-dimensional motion information, and the limb action information instructing personnel that described limb three-dimensional model and three-dimensional motion information gather according to 3D photographic head generates；
Generate according to described video flowing, described augmented reality information and described 3D guidance information and guide scene；
Show described guiding scene.
2. augmented reality remote guide method according to claim 1, it is characterised in that described three-dimensional motion information also includes the interactive information of described limb three-dimensional model and target three-dimensional.
3. augmented reality remote guide method according to claim 2, it is characterised in that described 3D guidance information also includes marking in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target；Wherein, the threedimensional model of described target is according to described augmented reality information creating.
4. the augmented reality remote guide method according to any one of claim 1-3, it is characterised in that described generation according to described video flowing, described augmented reality information and 3D guidance information guides scene to include:
According to described 3D guidance information retrieval technique document data base, generate 3D and strengthen guidance information；
Strengthen guidance information according to described video flowing, described augmented reality information and 3D and generate guiding scene；
Wherein, described technical data data base is for storing the technical data relevant to described target and threedimensional model.
5. augmented reality remote guide method according to claim 4, it is characterised in that described enhancing before guidance information generates guiding scene according to described video flowing, described augmented reality information and 3D also includes:
Control described 3D and strengthen the sequential that guidance information shows in the guiding scene generated.
6. the augmented reality remote guide method according to any one of claim 1-3, it is characterised in that described registration virtual video camera includes visual tracking registration, sensor Tracing Registration or mixing registration；
The registration of described visual tracking includes: identify the image of target described in described video flowing, and the image of described target is processed, and obtains visual tracking log-on message, generates the augmented reality information corresponding to described video flowing；
Described sensor Tracing Registration includes: obtain spatial transformation information by sensor, according to described spatial transformation information, described video flowing is processed, and obtains sensor Tracing Registration information, generates the augmented reality information corresponding to described video flowing；
Described mixing registration includes: carries out described visual tracking registration and described sensor Tracing Registration, comprehensive described visual tracking log-on message and described sensor Tracing Registration information respectively and generates the augmented reality information corresponding to described video flowing；
Wherein, described sensor and the collecting unit that target carries out optically detecting are synchronized with the movement.
7. augmented reality remote guide method according to claim 6, it is characterized in that, described visual tracking registers through the three-dimensional modeling data storehouse having the three-dimensional modeling data relevant to described target and identifies the image of target described in described video flowing, and generates described augmented reality information.
8. the augmented reality remote guide method according to any one of claim 1-3, it is characterised in that described send described video flowing and described augmented reality information to long-range diagnostic subsystem before also include:
Generate according to described video flowing and described augmented reality information and show guiding scene.
9. the augmented reality remote guide method in conjunction with 3D photographic head, it is characterised in that described method includes:
Receive the video flowing that operation guide subsystem sends and the augmented reality information corresponding to described video flowing；Wherein, described video flowing obtains by target is carried out optically detecting, and described augmented reality information obtains by registering virtual video camera；
The threedimensional model of target according to described augmented reality information creating；
Show described video flowing by 3D viewport and load the threedimensional model of described target；
The limb action information of personnel is instructed by 3D camera collection；
Generate limb three-dimensional model and three-dimensional motion information according to described limb action information, and in described 3D viewport loaded and displayed；
3D guidance information is generated according to described limb three-dimensional model and three-dimensional motion information；
Send described 3D guidance information to described operation guide subsystem, generate for described operation guide subsystem according to described video flowing, described augmented reality information and described 3D guidance information and show guiding scene.
10. augmented reality remote guide method according to claim 9, it is characterised in that described three-dimensional motion information also includes the interactive information of described limb three-dimensional model and target three-dimensional.
11. the augmented reality remote guide method according to claim 9 or 10, it is characterised in that described according to described limb action information generation limb three-dimensional model and three-dimensional motion information, and also include after loaded and displayed in described 3D viewport:
Input mark in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target, and in described 3D viewport markup information described in loaded and displayed；
Described 3D guidance information generates according to described limb three-dimensional model, described three-dimensional motion information and described markup information.
12. the augmented reality remote guide system in conjunction with 3D photographic head, it is characterised in that described system includes operation guide subsystem and long-range diagnostic subsystem；
Described operation guide subsystem includes:
Collecting unit, for carrying out optically detecting to obtain video flowing to target；
First processing unit, is used for registering virtual video camera, generates the augmented reality information corresponding to described video flowing, generates according to described video flowing, described augmented reality information and 3D guidance information and guides scene；
First display unit, is used for showing described guiding scene；
First communication unit, for sending described video flowing and described augmented reality information to described long-range diagnostic subsystem, receives the 3D guidance information that described long-range diagnostic subsystem returns；
Described long-range diagnostic subsystem includes:
3D photographic head, for gathering the limb action information instructing personnel；
Second communication unit, is used for receiving described video flowing and described augmented reality information, sends 3D guidance information to described operation guide subsystem；
Second processing unit, for the threedimensional model of target according to described augmented reality information creating, generate limb three-dimensional model and three-dimensional motion information according to described limb action information, and generate 3D guidance information according to described limb three-dimensional model and three-dimensional motion information；
Second display unit, for showing described video flowing by 3D viewport and loading the threedimensional model of described target, and in described 3D viewport limb three-dimensional model and three-dimensional motion information described in loaded and displayed.
13. augmented reality remote guide system according to claim 12, it is characterised in that described first processing unit includes:
Registration subelement, is used for registering virtual video camera, generates the augmented reality information corresponding to described video flowing, strengthens guidance information according to described video flowing, augmented reality information and 3D and generates guiding scene；
Intelligent retrieval subelement, strengthens guidance information for generating 3D according to described 3D guidance information, and controls the sequential that described 3D enhancing guidance information shows in the guiding scene generated.
14. augmented reality remote guide system according to claim 13, it is characterised in that described registration subelement includes:
Virtual video camera Registering modules, for described video flowing having been processed registration virtual video camera by image procossing or the spatial transformation information that obtained by sensor, generation is corresponding to the augmented reality information of described video flowing；
Guide scenario generating module, guide scene for generating according to described video flowing and described augmented reality information, or, generate according to described video flowing, described augmented reality information and 3D enhancing guidance information and guide scene；
Wherein, described sensor and described collecting unit are synchronized with the movement.
15. augmented reality remote guide system according to claim 14, it is characterised in that described virtual video camera Registering modules includes:
Visual tracking registration submodule, for identifying the image of target described in described video flowing, and processes the image of described target, obtains visual tracking log-on message；
Sensor Tracing Registration submodule, for obtaining spatial transformation information by described sensor, processes described video flowing according to described spatial transformation information, obtains sensor Tracing Registration information；
Augmented reality information generating module, for generating the augmented reality information corresponding to described video flowing according to described visual tracking log-on message and/or described sensor Tracing Registration information.
16. augmented reality remote guide system according to claim 15, it is characterised in that described first processing unit also includes:
Three-dimensional modeling data storehouse, for storing the three-dimensional modeling data relevant to described target, registers the image of target described in video flowing described in submodule identification for described visual tracking, and generates described augmented reality information.
17. augmented reality remote guide system according to claim 13, it is characterised in that described intelligent retrieval subelement includes:
Guide retrieval module, for according to described 3D guidance information retrieval technique document data base, generating 3D and strengthen guidance information；
Work-flow control module, strengthens, for controlling described 3D, the sequential that guidance information shows in the guiding scene generated；
Technical data data base, for storing the technical data relevant to described target and threedimensional model.
18. the augmented reality remote guide system according to any one of claim 12-17, it is characterised in that described three-dimensional motion information also includes the interactive information of described limb three-dimensional model and target three-dimensional.
19. the augmented reality remote guide system according to any one of claim 12-17, it is characterised in that described long-range diagnostic subsystem also includes:
Input block, marks in the markup information of described limb three-dimensional model or any part of the threedimensional model of described target for inputting；
Described second processing unit is additionally operable to generate 3D guidance information according to described limb three-dimensional model, described three-dimensional motion information and described markup information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610074722.0A CN105759960A (en) | 2016-02-02 | 2016-02-02 | Augmented reality remote guidance method and system in combination with 3D camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610074722.0A CN105759960A (en) | 2016-02-02 | 2016-02-02 | Augmented reality remote guidance method and system in combination with 3D camera |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105759960A true CN105759960A (en) | 2016-07-13 |
Family
ID=56329652
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610074722.0A Pending CN105759960A (en) | 2016-02-02 | 2016-02-02 | Augmented reality remote guidance method and system in combination with 3D camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105759960A (en) |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106297472A (en) * | 2016-10-25 | 2017-01-04 | 深圳市科创数字显示技术有限公司 | The cornea intelligent operation training system that AR with VR combines |
| CN106446569A (en) * | 2016-09-29 | 2017-02-22 | 宇龙计算机通信科技(深圳)有限公司 | Movement guidance method and terminal |
| CN107277172A (en) * | 2017-07-28 | 2017-10-20 | 深圳增强现实技术有限公司 | Machine operates monitor system and its ustomer premises access equipment |
| CN107277451A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure |
| CN107610168A (en) * | 2017-09-14 | 2018-01-19 | 触景无限科技（北京）有限公司 | A kind of remote guide method, apparatus and system |
| CN107645651A (en) * | 2017-10-12 | 2018-01-30 | 北京临近空间飞艇技术开发有限公司 | A kind of remote guide method and system of augmented reality |
| CN107656991A (en) * | 2017-09-14 | 2018-02-02 | 触景无限科技（北京）有限公司 | A kind of remote guide method, apparatus and system |
| CN108093244A (en) * | 2017-12-01 | 2018-05-29 | 电子科技大学 | A kind of remotely servo-actuated stereo visual system |
| CN108090968A (en) * | 2017-12-29 | 2018-05-29 | 光锐恒宇（北京）科技有限公司 | Implementation method, device and the computer readable storage medium of augmented reality AR |
| CN109195002A (en) * | 2018-08-06 | 2019-01-11 | 光锐恒宇（北京）科技有限公司 | Generation method, device, intelligent terminal and the computer readable storage medium of video |
| CN109814709A (en) * | 2018-12-27 | 2019-05-28 | 北京航天智造科技发展有限公司 | Operating Guideline method and device based on AR system |
| CN110072089A (en) * | 2019-05-29 | 2019-07-30 | 润电能源科学技术有限公司 | A kind of method and relevant device of remote transmission image data |
| CN110111428A (en) * | 2019-05-28 | 2019-08-09 | 艾瑞迈迪科技石家庄有限公司 | A kind of virtual target scaling method and device applied to augmented reality |
| CN110390705A (en) * | 2018-04-16 | 2019-10-29 | 北京搜狗科技发展有限公司 | A kind of method and device generating virtual image |
| CN110728756A (en) * | 2019-09-30 | 2020-01-24 | 亮风台（上海）信息科技有限公司 | Remote guidance method and device based on augmented reality |
| CN111526118A (en) * | 2019-10-29 | 2020-08-11 | 南京翱翔信息物理融合创新研究院有限公司 | Remote operation guiding system and method based on mixed reality |
| CN111915819A (en) * | 2020-08-14 | 2020-11-10 | 中国工商银行股份有限公司 | Remote virtual interaction method, device and system |
| CN112509148A (en) * | 2020-12-04 | 2021-03-16 | 全球能源互联网研究院有限公司 | Interaction method and device based on multi-feature recognition and computer equipment |
| CN113223353A (en) * | 2021-04-14 | 2021-08-06 | 佛山科学技术学院 | Remote real-time guiding method based on virtual experiment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1815409A (en) * | 2005-01-31 | 2006-08-09 | 佳能株式会社 | System, image processing apparatus, and information processing method |
| CN1845064A (en) * | 2005-04-08 | 2006-10-11 | 佳能株式会社 | Information processing method and apparatus |
| US20120249586A1 (en) * | 2011-03-31 | 2012-10-04 | Nokia Corporation | Method and apparatus for providing collaboration between remote and on-site users of indirect augmented reality |
| US9088787B1 (en) * | 2012-08-13 | 2015-07-21 | Lockheed Martin Corporation | System, method and computer software product for providing visual remote assistance through computing systems |
| US20150247723A1 (en) * | 2013-07-12 | 2015-09-03 | Magic Leap, Inc. | Method and system for obtaining texture data of a space |
-
2016
- 2016-02-02 CN CN201610074722.0A patent/CN105759960A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1815409A (en) * | 2005-01-31 | 2006-08-09 | 佳能株式会社 | System, image processing apparatus, and information processing method |
| CN1845064A (en) * | 2005-04-08 | 2006-10-11 | 佳能株式会社 | Information processing method and apparatus |
| US20120249586A1 (en) * | 2011-03-31 | 2012-10-04 | Nokia Corporation | Method and apparatus for providing collaboration between remote and on-site users of indirect augmented reality |
| US9088787B1 (en) * | 2012-08-13 | 2015-07-21 | Lockheed Martin Corporation | System, method and computer software product for providing visual remote assistance through computing systems |
| US20150247723A1 (en) * | 2013-07-12 | 2015-09-03 | Magic Leap, Inc. | Method and system for obtaining texture data of a space |
Non-Patent Citations (1)
| Title |
|---|
| 麻兴东: "增强现实的系统结构与关键技术研究", 《无线互联科技》 * |
Cited By (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446569A (en) * | 2016-09-29 | 2017-02-22 | 宇龙计算机通信科技(深圳)有限公司 | Movement guidance method and terminal |
| CN106297472A (en) * | 2016-10-25 | 2017-01-04 | 深圳市科创数字显示技术有限公司 | The cornea intelligent operation training system that AR with VR combines |
| CN107277451A (en) * | 2017-07-14 | 2017-10-20 | 福建铁工机智能机器人有限公司 | A kind of utilization AR realizes the method and apparatus of remote guide scene investigation failure |
| CN107277172A (en) * | 2017-07-28 | 2017-10-20 | 深圳增强现实技术有限公司 | Machine operates monitor system and its ustomer premises access equipment |
| CN107656991A (en) * | 2017-09-14 | 2018-02-02 | 触景无限科技（北京）有限公司 | A kind of remote guide method, apparatus and system |
| CN107610168A (en) * | 2017-09-14 | 2018-01-19 | 触景无限科技（北京）有限公司 | A kind of remote guide method, apparatus and system |
| CN107645651A (en) * | 2017-10-12 | 2018-01-30 | 北京临近空间飞艇技术开发有限公司 | A kind of remote guide method and system of augmented reality |
| CN108093244B (en) * | 2017-12-01 | 2021-02-09 | 电子科技大学 | Remote follow-up stereoscopic vision system |
| CN108093244A (en) * | 2017-12-01 | 2018-05-29 | 电子科技大学 | A kind of remotely servo-actuated stereo visual system |
| CN108090968A (en) * | 2017-12-29 | 2018-05-29 | 光锐恒宇（北京）科技有限公司 | Implementation method, device and the computer readable storage medium of augmented reality AR |
| CN108090968B (en) * | 2017-12-29 | 2022-01-25 | 光锐恒宇（北京）科技有限公司 | Method and device for realizing augmented reality AR and computer readable storage medium |
| CN110390705B (en) * | 2018-04-16 | 2023-11-10 | 北京搜狗科技发展有限公司 | Method and device for generating virtual image |
| CN110390705A (en) * | 2018-04-16 | 2019-10-29 | 北京搜狗科技发展有限公司 | A kind of method and device generating virtual image |
| CN109195002A (en) * | 2018-08-06 | 2019-01-11 | 光锐恒宇（北京）科技有限公司 | Generation method, device, intelligent terminal and the computer readable storage medium of video |
| CN109814709A (en) * | 2018-12-27 | 2019-05-28 | 北京航天智造科技发展有限公司 | Operating Guideline method and device based on AR system |
| CN110111428A (en) * | 2019-05-28 | 2019-08-09 | 艾瑞迈迪科技石家庄有限公司 | A kind of virtual target scaling method and device applied to augmented reality |
| CN110111428B (en) * | 2019-05-28 | 2023-06-20 | 艾瑞迈迪科技石家庄有限公司 | Virtual target calibration method and device applied to augmented reality |
| CN110072089A (en) * | 2019-05-29 | 2019-07-30 | 润电能源科学技术有限公司 | A kind of method and relevant device of remote transmission image data |
| CN110728756A (en) * | 2019-09-30 | 2020-01-24 | 亮风台（上海）信息科技有限公司 | Remote guidance method and device based on augmented reality |
| CN110728756B (en) * | 2019-09-30 | 2024-02-09 | 亮风台（上海）信息科技有限公司 | Remote guidance method and device based on augmented reality |
| CN111526118A (en) * | 2019-10-29 | 2020-08-11 | 南京翱翔信息物理融合创新研究院有限公司 | Remote operation guiding system and method based on mixed reality |
| CN111915819A (en) * | 2020-08-14 | 2020-11-10 | 中国工商银行股份有限公司 | Remote virtual interaction method, device and system |
| CN112509148A (en) * | 2020-12-04 | 2021-03-16 | 全球能源互联网研究院有限公司 | Interaction method and device based on multi-feature recognition and computer equipment |
| WO2022116545A1 (en) * | 2020-12-04 | 2022-06-09 | 全球能源互联网研究院有限公司 | Interaction method and apparatus based on multi-feature recognition, and computer device |
| CN113223353A (en) * | 2021-04-14 | 2021-08-06 | 佛山科学技术学院 | Remote real-time guiding method based on virtual experiment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105759960A (en) | Augmented reality remote guidance method and system in combination with 3D camera | |
| CN105739704A (en) | Remote guidance method and system based on augmented reality | |
| Zollmann et al. | Augmented reality for construction site monitoring and documentation | |
| US7139685B2 (en) | Video-supported planning of equipment installation and/or room design | |
| US7061485B2 (en) | Method and system for producing a model from optical images | |
| US7714895B2 (en) | Interactive and shared augmented reality system and method having local and remote access | |
| US10699165B2 (en) | System and method using augmented reality for efficient collection of training data for machine learning | |
| JP2020535509A (en) | Methods, devices and systems for automatically annotating target objects in images | |
| CN108509026B (en) | Remote maintenance support system and method based on enhanced interaction mode | |
| CN105760106A (en) | Interaction method and interaction device of intelligent household equipment | |
| US20100134601A1 (en) | Method and device for determining the pose of video capture means in the digitization frame of reference of at least one three-dimensional virtual object modelling at least one real object | |
| JP2003526842A (en) | Dynamic visual alignment of 3D objects using graphical models | |
| WO2001088681A1 (en) | Apparatus and method for indicating a target by image processing without three-dimensional modeling | |
| CN108828996A (en) | A kind of the mechanical arm remote control system and method for view-based access control model information | |
| KR20220085142A (en) | Intelligent construction site management supporting system and method based extended reality | |
| CN106814963A (en) | A kind of human-computer interaction system and method based on 3D sensor location technologies | |
| CN104899361A (en) | Remote control method and apparatus | |
| US11395102B2 (en) | Field cooperation system and management device | |
| CN107229887A (en) | Multi-code scanning device and multi-code scan method | |
| KR20220085150A (en) | Intelligent construction site management supporting system server and method based extended reality | |
| CN113660469A (en) | Data labeling method and device, computer equipment and storage medium | |
| CN101165719A (en) | Enhancement implement industrial maintenance method and system based on mark point | |
| CN114565849B (en) | Asset AI (artificial intelligence) identification and positioning method and system based on digital twin | |
| CN109732606A (en) | Long-range control method, device, system and the storage medium of mechanical arm | |
| CN115731349A (en) | Method and device for displaying house type graph, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20160713 |