CN109859538A - A kind of key equipment training system and method based on mixed reality - Google Patents
A kind of key equipment training system and method based on mixed reality Download PDFInfo
- Publication number
- CN109859538A CN109859538A CN201910243741.5A CN201910243741A CN109859538A CN 109859538 A CN109859538 A CN 109859538A CN 201910243741 A CN201910243741 A CN 201910243741A CN 109859538 A CN109859538 A CN 109859538A
- Authority
- CN
- China
- Prior art keywords
- key equipment
- module
- hologram image
- dismounting
- mixed reality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The key equipment training system based on mixed reality that the present invention provides a kind of, comprising: room data obtains module, for being scanned to real space, forms room data, and establish space coordinates；Hologram image generation module, for the internal structure of actual key equipment and operation file to be generated key equipment hologram image in advance and stored；Mixed reality scene module for transferring an at least key equipment hologram image, and is placed in the space coordinates established, hologram image is superimposed the key equipment in the real space to according to the actual situation；Interactive module, for issuing dismounting order to the superimposed key equipment hologram image of the mixed reality scene module by interactive command according to demand.The threedimensional model of nuclear power station key equipment is placed into true environment by the present invention, and user can watch the superimposed image of real equipment and virtual unit in real time and interact, and provides teaching, training and the new mode of on-the-spot guidance for the nuclear power people that works.
Description
Technical field
The invention belongs to computer virtual reality technology field more particularly to a kind of key equipment trainings based on mixed reality
Exercising system and method.
Background technique
VR (Virtual Reality) technology has been gradually applied in Nuclear power project, passes through simulation reconstruction three dimensional field
Scape and insertion simulation operations, provide advantageous help and service for engineering design, construction, maintenance and operator training.
AR, by real scene additional virtual information help people to more fully understand real world.MR (mixed reality) technology
Then contain AR and VR technology.The technology between virtual world, real world and user by setting up an interaction feedback
Information circuits, to enhance the sense of reality of user.Existing design is with depth defense and is related to reliability principle for guidance, uses energy
It moves and introduces many new technologies with the passive advanced design theory combined.Currently, culture a batch is familiar with nuclear power work people
Member seems especially urgent.
In current existing training method and technology, all there is many deficiencies.First, traditional two-dimensional drawing
And video teaching, can not intuitively it experience to student.Second, using VR virtual reality technology, it can provide and immerse to student
The collaborative virtual learning environment of formula, but due to its opacity, student learns in virtual world completely, can not be true in conjunction with scene
Environment and equipment is not high with live conjugation.Third, using AR technology, AR technology can be by virtual image superposition to really
In equipment, realize actual situation combine, solve the problems, such as that VR is opaque, but AR technology do not solve virtual object as with true environment and
The interaction problems of equipment have interactivity deficiency.
Related training existing at present and teaching are mainly carried out by modes such as video, documents, VR Virtual Simulation
It is applied in some contents of courses.Student is learnt by way of voluntarily browsing video and document, for complicated machinery
Device structure, can not split layer by layer, deeply understand the internal structure of key equipment.The virtual emulation teaching of VR is able to solve not straight
The problem of sight, but the higher VR helmet of current performance needs that computer is cooperated to use, teaching position is fixed, and building site can not be carried out
On-the-spot teaching.And VR virtual emulation provides the academic environment of complete immersion, can not combine with live bad border.Current VR head
The 3D dizziness problem serious curtailment of helmet its use time and usage comfort.And pass through video, document and VR technology at present
Carry out the modes such as teaching, training, has the following problems: 1) the not intuitive of video and document.Document and video can not be by nuclear powers
The stereochemical structure of key equipment is intuitively presented in face of student, is not easy to student's understanding.2) opacity of VR technology.VR technology
The virtual environment of complete immersion is provided, can not be combined with true environment and equipment, VR technology can only do site environment
Simulation.3) stationarity imparted knowledge to students.Whether through video.Document and VR technology, common teaching can all be arranged in training
Classroom or Administrative Area can not directly impart knowledge to students inside nuclear power workshop, and technological guidance can not be carried out inside workshop.
So a kind of key equipment training system based on mixed reality need to be sought, solves key equipment in the prior art and teach
The problems such as virtual unit and true environment can not being combined in.
Summary of the invention
The present invention in view of the above technical problems, provides a kind of key equipment training system based on mixed reality and side
Method solves the not intuitive of video in the teaching of key equipment in the prior art, document, the opacity of VR teaching, teaching
Fixed technical problem of environment.
The present invention is a kind of key equipment training based on mixed reality for solving the technical solution of the above technical problem
System, comprising: room data obtains module, for being scanned to real space, forms room data, and establish space coordinate
System；Hologram image generation module, for the internal structure of actual key equipment and operation file to be generated key equipment in advance
Hologram image simultaneously stores；Mixed reality scene module, connects the hologram image generation module and the room data obtains mould
Block for transferring an at least key equipment hologram image, and is placed in the space coordinates established, realizes the key
Equipment hologram image is superimposed with the real space；Interactive module connects the mixed reality scene module, for according to need
It asks and dismounting order is issued to the superimposed key equipment hologram image of the mixed reality scene module by interactive command.
Further, the training system further include: more people's sharing modules connect the interactive module and the mixing is existing
Real scene module is shared for receiving multiusers interaction order, and by multiusers interaction order, according to multi-purpose after sharing
Family order issues dismounting order to the key equipment hologram image.
Further, the mixed reality scene module is also used to receive that interactive module and more people's sharing modules issue tears open
Dress order, and key equipment hologram image is dismounted according to dismounting order, the dismounting image after obtaining dismounting.
Further, the training system further include: third party visual angle module connects the mixed reality scene module,
For providing third party visual angle, the dismounting image after dismounting is transformed into two dimensional image by 3-D image, and two dimensional image is thrown
Shadow to large screen is shown.
Further, it includes: surface viewer module that the room data, which obtains module, for sweeping to real space
It retouches, identifies all spatial surfaces in actual room, and create spatial surface information；Poller module connects the surface observation
Device module is used for continual search space surface information, monitors the variation of spatial surface information；Grid cache module, connection
The poller module is handled and is cached for the variation to spatial surface information；Properties Control module connects the grid
Cache module is optimized for the spatial surface information according to actual needs to caching；Grid rendering module connects the property
Energy control module forms room data, and establish for rendering according to the spatial surface information after optimization to spatial surface
Space coordinates.
Further, the interactive command includes staring, and gesture and voice command are described to stare, gesture and voice command
Dismounting order is issued by staring module, gesture module and voice module respectively.
On the other hand, the present invention also provides for example a kind of key equipment Training Methodology based on mixed reality, including it is as follows
Step: S100, being scanned real space, forms room data, and establish space coordinates；S200, in advance will be actual
The internal structure and operation file of key equipment generate key equipment hologram image and store；S300, it transfers an at least key and sets
Standby hologram image, and be placed in the space coordinates established, realize the key equipment hologram image and the reality
The superposition in space；S400, according to demand superimposed key equipment hologram image is issued to dismount by interactive command and is ordered.
Further, the method also includes: S500, receive multiusers interaction order, and by multiusers interaction order into
Row is shared, is ordered according to the multi-user after shared to key equipment hologram image sending dismounting order.
Further, the method also includes: S600, receive the dismounting order that interactive command issues, and ordered according to dismounting
Order dismounts key equipment hologram image, the dismounting image after obtaining dismounting.
Further, the method also includes: S700, third party visual angle is provided, by user by interactive command to key
3-D image after equipment hologram image is dismounted is transformed into two dimensional image, and two dimensional image is projected to large screen and is shown.
Further, the step S100 further comprises: S110, being scanned to real space, identifies actual room
In all spatial surfaces, and create spatial surface information；The spatial surface information that S120, continual inquiry create, monitoring
The variation of spatial surface information.S130, the variation of spatial surface information is handled and is cached；S140, according to actual needs
The spatial surface information of caching is optimized；S150, spatial surface is rendered according to the spatial surface information after optimization,
Room data is formed, and establishes space coordinates.
Further, the interactive command in step S400 includes staring, gesture and voice command, described to stare, hand
Gesture and voice command issue dismounting order.
Technical solution provided in an embodiment of the present invention has the benefit that a kind of based on mixed reality by establishing
Key equipment training system and method solve the not intuitive of video in the teaching of key equipment in the prior art, document, VR
The opacity of teaching, fixed technical problem of teaching environment, the present invention sufficiently combine mixed reality technology, innovatively
The dummy model of key equipment is combined with live physical surroundings and equipment, creates a kind of new working method, virtual mould
Type can occur directly in true environment, can be interacted with surrounding enviroment, and it is existing to have broken conventional teaching disengaging nuclear power construction
Mode, teaching is directly placed on nuclear power scene, can real-time instruction equipment maintenance work, create a kind of new teaching side
Formula and work on the spot guidance.
Detailed description of the invention
Present invention will be further explained below with reference to the attached drawings and examples, in attached drawing:
Fig. 1 is the structural schematic diagram for the key equipment training system based on mixed reality that the embodiment of the present invention one provides；
Fig. 2 is the specific device structure for the key equipment training system based on mixed reality that the embodiment of the present invention one provides
Schematic diagram；
Fig. 3 is the workshop arrangement effect for the key equipment training system based on mixed reality that the embodiment of the present invention one provides
Figure；
Fig. 4 is the primary Ioops effect for the key equipment training system based on mixed reality that the embodiment of the present invention one provides
Figure；
Fig. 5 is that the key equipment training system room data based on mixed reality that the embodiment of the present invention one provides obtains mould
The structural schematic diagram of block；
Fig. 6 is the structure for the key equipment training system interactive module based on mixed reality that the embodiment of the present invention one provides
Schematic diagram；
Fig. 7 is the structure that the key equipment training system based on mixed reality that the embodiment of the present invention one provides stares module
Schematic diagram；
Fig. 8 is the structure for the key equipment training system gesture module based on mixed reality that the embodiment of the present invention one provides
Schematic diagram；
Fig. 9 is the structure for the key equipment training system voice module based on mixed reality that the embodiment of the present invention one provides
Schematic diagram；
Figure 10 is that the key equipment training system interactive command based on mixed reality that the embodiment of the present invention one provides is synchronous
Flow diagram；
Figure 11 is the key equipment training system anchor point command synchronization based on mixed reality that the embodiment of the present invention one provides
Flow diagram；
Figure 12 is the key equipment training system third party visual angle mould based on mixed reality that the embodiment of the present invention one provides
Block development process schematic diagram；
Figure 13 is the key equipment Training Methodology flow chart provided by Embodiment 2 of the present invention based on mixed reality；
Figure 14 is the stream of the key equipment Training Methodology step S100 provided by Embodiment 2 of the present invention based on mixed reality
Cheng Tu.
Specific embodiment
It is proposed by the present invention by establishing a kind of key equipment training system and method based on mixed reality, solve existing
There are the not intuitive of video in the teaching of key equipment in technology, document, the opacity of VR teaching, the stationarity of teaching environment
The technical issues of, the present invention is innovative to be introduced into mixed reality technology in the teaching, training of key equipment.By will be crucial
The threedimensional model of equipment is placed into true environment, and user can watch the superimposed image of real equipment and virtual unit in real time
And interacted, provide the virtual browse of equipment threedimensional model, the operation such as fractionation, assembling, parameter are checked.The present invention can allow existing
Field operator is when nuclear power workshop works, and by training system, operator can check virtual unit simultaneously and really set
It is standby.Operator is allowed to get information about equipment construction, theory of constitution in real time.It can be with the relevant drawing of real time review and test procedure
With relevant operation handbook.It can be with the debugging of real time inspection equipment, maintenance step etc..Teaching, training is provided for nuclear power work people and is showed
Instruct new mode in field.
In order to enable those skilled in the art that the present invention is more clearly understood, below in conjunction with attached drawing and specific implementation
Example does further detailed description to the present invention.
Embodiment one
Fig. 1 is a kind of key equipment training system structural representation based on mixed reality that the embodiment of the present invention one provides
Scheme, as shown in Figure 1: a kind of key equipment training system based on mixed reality, comprising: room data obtains module 100, is used for
Real space is scanned, forms room data, and establish space coordinates；Wherein, the room data of formation includes user
Walkable region in real space and to real space gridded data etc.；Hologram image generation module 200, for pre-
The internal structure of actual key equipment and operation file are first generated into key equipment hologram image and stored, tune when being easy to use
With；Mixed reality scene module 300, connects hologram image generation module 100 and room data obtains module 200, for transferring
At least internal structure of a key equipment and operation file hologram image stored out, and be placed on the space established and sit
In mark system, being superimposed for the key equipment hologram image and the real space is realized；Interactive module 400 connects mixed reality
Scene module 300, for complete to the superimposed key equipment of mixed reality scene module by interactive command according to demand
It ceases image and issues dismounting order, user can watch real equipment and key equipment hologram image in real space simultaneously, allow behaviour
Make personnel and get information about equipment construction in real time, it should be noted that operator can also pass through interactive command call operation text
Part, for the relevant drawing of user's real time review and test procedure and relevant operation handbook.
Further, training system further include: more people's sharing modules 500 connect interactive module 400 and mixed reality scene
Module 300 is shared for receiving multiusers interaction order, and by multiusers interaction order, according to the multi-user after sharing
It orders to the key equipment hologram image and issues dismounting order, more people's sharing modules 500 provide shared characteristic for training system,
Operation is interacted to same key equipment hologram image for multiple users；Wherein: mixed reality scene module 300 is also used to connect
The dismounting order issued according to interactive module 400 and more people's sharing modules 500 is received, and according to dismounting order to key equipment holography
Image is dismounted, the dismounting image after obtaining dismounting.Third party visual angle module 600 connects mixed reality scene module 300,
For providing third party visual angle, dismounting image of the user after interactive command dismounts is transformed into X-Y scheme by 3-D image
Picture, and two dimensional image is projected to large screen and is shown, the holographic interaction for teaching demonstration and large-scale speech is shown.
It is further described in conjunction with the key equipment in practice: as shown in Fig. 2, in practice, firstly,
Space environment is scanned, room data is established, to establish space coordinates；Secondly, user is by pre-stored basis
The internal structure and operation file of actual key equipment generate key equipment hologram image and recall, and are deployed to space environment
In, and the key equipment hologram image being deployed in space environment is handled, realize the key equipment hologram image with
The superposition of the real space enhances user experience；Then, user can issue dismounting order, mixing by interactive module 400
Reality scene module 300 receives the dismounting order that interactive module 400 issues, and according to dismounting order to being deployed in space environment
Key equipment hologram image real-time interactive and it is dismounted, until training terminates, thus: user can be intuitive in real time
Pass through interactive mode dragging and split key equipment hologram image, understand key equipment construction, theory of constitution etc..
Wherein, the key equipment in practice mainly includes that workshop arrangement, primary Ioops system and key equipment, equipment change
Income mesh and innovation item and master system.Workshop arrangement includes three dimensional arrangement information and workshop internal structure；Primary Ioops
System and key equipment include: primary Ioops system principle and key equipment information, such as: pressure vessel and main pump；Great innovation is set
Meter includes Core Design and design of pressure vessels.
Wherein, the key step key equipment hologram image being deployed in environment handled are as follows: (1) according to reality
Situation judges the positional relationship of key equipment hologram image and room data, and places it on the surface of true environment；(2)
The hiding relation of key equipment hologram image and room data is judged according to the actual situation, and keeps hologram image shield portions true
Environment creates a kind of perception really lived in same physical space with user；(3) reinforce key in user environment space to set
The presence of standby hologram, such as the Working fluid flow in analog primary Ioops system, (4) authorize holographic role mode identical with user
It navigates in true environment, specifically: the region and barrier that user can walk can be provided；(5) side of data grids is utilized
Formula visualizes environmental data, carries out gridding to surface, facilitates acquisition environmental data, interact with hologram image.
Being superimposed for the key equipment hologram image and the real space is realized by above-mentioned five steps, so that it is existing to establish mixing
Real field scape, and realize the purpose of the experience sense of enhancing user.
Fig. 3 is the effect picture arranged by workshop after mixed reality technology；Fig. 4 is to pass through one time after mixed reality technology
Road effect picture.By Fig. 3 and Fig. 4 it can be seen that by the way that the threedimensional model of nuclear power station key equipment is placed into true environment, use
Family can watch the superimposed image of real equipment and virtual unit in real time, specifically, as can be seen from Figure, be placed in true
Key equipment hologram image in room has blocked the true room in part, and can see key equipment hologram image in figure
The parameters such as pressure and temp；Therefore, user is it is seen that the mixing for combining true room data and key equipment hologram image is existing
Real field scape allows operator to get information about equipment construction, theory of constitution in real time.
Further, in conjunction with attached drawing 5, it will thus be seen that it includes: surface viewer module that room data, which obtains module 100,
110, for being scanned to real space, identify all spatial surfaces in actual room, and create spatial surface information；Tool
Body, surface viewer module 110 creates surface viewer first, and sets acquired spatial data range, set
In the range of identify fixed region and moveable region respectively；After having obtained region, space is created to acquired region
Surface information；Poller module 120, connection surface viewer module 110 are used for continual search space surface information, in real time
Spatial surface is monitored, the variation of spatial surface information is monitored；Grid cache module 130 connects poller module 120, for sky
Between the variation of surface information handled and cached, specifically, the content of grid caching, which specifically includes that, to be got after poll
The data of spatial surface variation (such as by surface that key equipment hologram image blocks) increase spatial surface, updated and are deleted
The processing that removes and asynchronous grid request is handled；Properties Control module 140 connects grid cache module 130, is used for root
It is optimized according to spatial surface information of the actual demand to caching, specifically includes the density to triangular mesh, view frustums are cut
Mode and the sequence of rendering optimize, and under the premise of meet demand, improve processing speed as far as possible；Grid renders mould
Block 150, switching performance control module 140 are formed for being rendered according to the spatial surface information after optimization to spatial surface
Room data, and establish space coordinates；Increase the authenticity of virtual scene and the experience sense of user by rendering.
Further, as shown in Figure 6: interactive command includes staring, and gesture and voice command are described to stare, gesture and language
Sound order issues dismounting order by staring module 410, gesture module 420 and voice module 430 respectively, specific: to stare
Module 410 connects mixed reality scene module 300, for according to staring ray localization key equipment hologram image, and to key
Equipment hologram image issues dismounting order；In conjunction with attached drawing 7, the detailed process that module 410 issues dismounting order is stared are as follows: start first
Camera is first created, the eyes of user are replaced using camera, this is because indicating the number of people with camera in the exploitation of unity
The position and orientation in portion；Secondly, by camera divergent-ray, judgement stares the crosspoint of ray and key equipment hologram image to accuse
Tell the current focus of user, i.e. visualization point；Then, whether detection visualization point with room data has collision, there is collision table
Show visualization point on the surface of actual environment, if there is the point of impingement, obtains colliding object and the point of impingement；Based on the point of impingement and stare
Ray issues dismounting order to key equipment hologram image, and later, the hair of module 410 is stared in the reception of mixed reality scene module 300
Dismounting order out, and according to dismounting order key equipment hologram image is chosen, activates and pulled to realize to key equipment
The dismounting of hologram image；If collisionless point, ray is stared in movement, continually looks for the point of impingement；Gesture module 420 is used for basis
The gesture being previously set issues dismounting order to the key equipment hologram image in room data, in conjunction with attached drawing 8, gesture mould
Block 420 issues the detailed process of dismounting order are as follows: gesture identification example is created at the beginning first, secondly, registration gesture-type；It connects
Subscription gesture event；After completing these work, gesture identification, the specific steps of gesture identification are as follows: basis first can be started
User instruction judges whether to identify, if so, carry out gesture identification, if it is not, then stop gesture identification, when gesture identification can
Matching gesture identification example is gone according to the gesture of user made, example gesture event is called by the gesture identification being matched to,
Key equipment hologram image is issued such as according to the example gesture event of calling: one hologram image of mobile, rotation or scaling
Dismounting order, later, mixed reality scene module 300 receive gesture module 420 issue dismounting order, and according to dismounting order
Order dismounts key equipment hologram image；Such as: if established gesture identification example in advance are as follows: user's long-pressing gesture table
Show for picking up a hologram image component；When user makes above-mentioned gesture, gesture module 420 is to mixed reality scene module
300 issue the dismounting order for picking up a hologram image component, and mixed reality scene module 300 receives this dismounting order, and does
The movement of a hologram image component is picked up out.Voice module 430, for passing through voice to the key equipment in room data
Hologram image issues dismounting order.In conjunction with attached drawing 9, voice module 430 issues the detailed process of dismounting order are as follows: starts to infuse first
Volume keyword, secondly initializes speech recognition example, then registered events, after completing these preparations, can start voice
Identification, judges whether to speech recognition according to user instructions first, if it is not, then stopping speech recognition, if so, carrying out voice
Identification, the detailed process of speech recognition are as follows: user says keyword first, and voice module 400 is known according to Keywords matching voice
Other example, and dismounting order, mixed reality scene are issued to key equipment hologram image according to the speech recognition example being matched to
Module 300 receives dismounting order, and is dismounted according to the speech recognition example being matched to key equipment hologram image, example
Such as: after opening voice module, user says " reactor core is moved 10 centimetres along the x axis ", then user is crucial by above-mentioned voice
Equipment hologram image issues the dismounting order of " reactor core is moved 10 centimetres along the x axis ", and mixed reality scene module 300 receives
The dismounting order of " reactor core is moved 10 centimetres along the x axis ", and make the dynamic of " reactor core is moved 10 centimetres along the x axis "
Make, voice input allows user directly to issue order by voice, and does not have to by gesture.It should be noted that voice command
Or control command as " returning to last action " is led in this way, voice input can be good at improving complicated interaction
Complicated UI (interface) menu can be much less by crossing a voice command.
It should be understood that staring is the first input form in holographic applications, it be used to position object, stare ray
Function is similar with the cursor pointer of PC；Gesture input allows user to interact using hand with hologram image, that is, passes through hand to hologram
As issuing dismounting order, the core interactive mode of gesture command is that user makes click, unclamps and burst forth gesture.It chooses and activates
The primary manner of holographic object is Air-tap gesture, it contains one and clicks and unclamp gesture.Gesture is burst forth to be used for from holography
Using start menu is aroused, similar to return push-button.More complicated gesture can be used by mobile hand.Pass through above-mentioned three kinds of interactions
Mode can issue quickly and effectively dismounting to the key equipment hologram image being placed in room data and order.
Further, more people's sharing modules 500 are for receiving multiusers interaction order, and multiusers interaction order is carried out
It is shared, it is ordered according to the multi-user after shared to key equipment hologram image sending dismounting order, it should be noted that be
Guarantee that multi-user is to issue interactive command to same key equipment hologram image, is shared carrying out multi-user's order
Before, it is also necessary to room data is shared.That is: more people's sharing modules 500 include that the synchronization of interactive command is synchronous with room.
Specifically, the synchronization of interactive command sends mutually message, realizes synchronous function by establishing socket communication mechanism
Energy.In conjunction with attached drawing 10, the synchronization detailed process of interactive command are as follows: initially set up server, and connect server, secondly in local
Execute interactive command, and the interactive command that user is executed is sent, and judges whether server distributes information, if so, it is local its
His user receives message, the synchronization of interactive command is completed, if it is not, then only local executing interactive command.
Synchronizing for room is mainly realized by synchronizing for space anchor point, and after the system scanning space environment, user can be with
Selecting to establish some space anchors by manual or programming, the information of space anchor can serialize and pass to other equipment,
Each equipment can navigate in space the position of this space anchor according to information with these information of unserializing.In conjunction with attached
Figure 11, the detailed process of the synchronization in room are as follows: server is initially set up and connect, secondly, local initialization anchor point, and by anchor point
It uploads；Then room is identified by server, and judges whether it is same room, if so, synchronous room information, and download anchor
Ground data；If it is not, then locally initializing anchor point.
Further, there is required hardware when third party visual angle module 600 is specifically developed: at least two respectively
Hololens, a computer, a slr camera output (supporting HDMI), the fixed bracket of Hololens and video frequency collection card.Knot
Close attached drawing 12, the specifically development process of third party visual angle module 600 are as follows: connection camera, computer and Hololens first, secondly,
The image of camera and Hololens are verified；Projects are loaded into after the completion of verification in Unity；Combined camera with
Hololens image；Secondly, synchronizing to anchor point, realize that scene is shared；Then, to tearing open after hologram image and interactive operation
Dress image synchronizes, and 3-D image is converted into two dimensional image, demonstrates finally, projecting two dimensional image to large screen, supplies
The 3-D image that MR equipment user is seen is transformed into two dimensional image and is presented on large screen by third party visual angle makes third party
Solve the content that MR is shown.This technology can be used for teaching demonstration and the holographic interaction of large-scale speech is shown, solves to work as and use
When the progress mixed reality experience of the mixed reality helmet is dressed at family, bystander can not know that the technology of the experience content of wearer is asked
Topic.
Embodiment two
The embodiment of the invention provides a kind of key equipment Training Methodology based on mixed reality is suitable for one institute of embodiment
A kind of key equipment training system based on mixed reality shown, in order to realize the above-mentioned key equipment training based on mixed reality
The structure of system, Figure 13 are a kind of key equipment Training Methodologies based on mixed reality provided by Embodiment 2 of the present invention, mainly
Include the following steps:
S100, real space is scanned, forms room data, and establish space coordinates；
S200, the internal structure of actual key equipment and operation file are generated into key equipment hologram image in advance and deposited
Storage；Convenient for calling, the training time is saved.
S300, an at least key equipment hologram image is transferred, and be placed in the space coordinates established, realize institute
State being superimposed for key equipment hologram image and the real space；Specifically, according to the actual situation using room data to key
Equipment hologram image is handled, and user experience is enhanced；And by key equipment hologram image combination user, room data arrangement
In place and orientation.
S400, according to demand superimposed key equipment hologram image is issued to dismount by interactive command and is ordered.
Further, method further include: S500, receive multiusers interaction order, and multiusers interaction order is total to
It enjoys, is ordered according to the multi-user after shared to key equipment hologram image sending dismounting order；Realize that multiple users can be with
See same mixed reality scene, while multiple users are giveed training.
Further, method further include: S600, receive the dismounting order that interactive command issues, and according to dismounting order pair
Key equipment hologram image is dismounted, the dismounting image after obtaining dismounting；
S700, third party visual angle is provided, by user after interactive command dismounts key equipment hologram image
Dismounting image is transformed into two dimensional image by 3-D image, and two dimensional image is projected to large screen and is shown.
Further, in conjunction with attached drawing 14, institute step S100 further comprises:
S110, real space is scanned, identifies all spatial surfaces in actual room, and create spatial surface letter
Breath；Specifically, surface viewer is created, and sets acquired spatial data range, is identified respectively in set range
Fixed region and moveable region；After having obtained region, spatial surface information is created to acquired region；
The spatial surface information that S120, continual inquiry create, real-time monitoring spatial surface monitor spatial surface information
Variation；
S130, the variation of spatial surface information is handled and is cached；Specifically, the content of grid caching is mainly wrapped
It includes: by the data of the spatial surface got after poll variation (surface such as blocked by key equipment hologram image) to spatial table
The processing and asynchronous grid request is handled that face is increased, updated and deleted；
S140, the spatial surface information of caching is optimized according to actual needs；It specifically includes to triangular mesh
The sequence of density, view frustums cutting method and rendering optimizes, and under the premise of meet demand, raising is handled as far as possible
Speed；
S150, spatial surface is rendered according to the spatial surface information after optimization, forms room data, and establish sky
Between coordinate system.Increase the authenticity of virtual scene and the experience sense of user by rendering.
Further, interactive command include stare, gesture and voice command, can be to being placed on room by three kinds of interactive modes
Between key equipment hologram image in data issue quickly and effectively dismounting order.
It is proposed by the present invention to have following skill by establishing a kind of key equipment training system based on mixed reality and method
Art effect: mixed reality technology being introduced into the teaching, training of nuclear power station key equipment, by by the three-dimensional mould of key equipment
Type is placed into true environment, and user can watch the superimposed image of real equipment and virtual unit in real time and interact,
There is provided the virtual browse of equipment threedimensional model, the operation such as fractionation, assembling, parameter are checked；Meanwhile the present invention can allow execute-in-place
Personnel can check virtual unit and real equipment when nuclear power workshop works simultaneously；Operator is allowed to get information about in real time
Equipment construction, theory of constitution；It can be with the relevant drawing of real time review and test procedure and relevant operation handbook.It can be with real time inspection
Debugging, maintenance step of equipment etc.；Teaching, training and the new mode of on-the-spot guidance are provided for the nuclear power people that works；By virtual key
Three peacekeeping 2-D datas of equipment are placed into true environment, and shelter from virtual image using spatial surface, create one kind
The perception of the same space is lived in user；User of service can arbitrarily walk about simultaneously, watch virtual image from multiple angles,
Secondary, by staring, gesture and voice command are realized to issue key equipment hologram image and quickly and effectively dismount order, and raising is torn open
Fill efficiency；And the present invention can allow multiple users that the service is added by establishing server, establish Network Synchronization function.More people
The facility information of key equipment can be checked simultaneously, and carries out real time interactive operation；The present invention is by establishing third party visual angle function
Can, using the fusion of mixed reality contextual data and camera data, the spectators for not seeing the side of mixed reality scene can be allowed,
Also the hologram image that the user being immersed in mixed reality scene is operating can be understood in real time.
It should be understood that above-described embodiment provides system when Training Methodology is realized, only with above-mentioned each functional module
It divides and carries out for example, can according to need by above-mentioned function distribution by different functional modules in practical application, i.e., will
The internal structure of equipment is divided into different functional modules, to complete all or part of the functions described above.In addition, above-mentioned
The system and method embodiment that embodiment provides belongs to same design, and specific implementation process is detailed in the description of embodiment of the method,
Which is not described herein again.
Those of ordinary skill in the art will appreciate that realizing that all or part of the steps of above-described embodiment can pass through hardware
It completes, relevant hardware can also be instructed to complete by program, the program can store in a kind of computer-readable
In storage medium.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all in spirit of the invention and
Within principle, any modification, equivalent replacement, improvement and so on be should all be included in the protection scope of the present invention.
Claims (12)
1. a kind of key equipment training system based on mixed reality, which is characterized in that the system comprises:
Room data obtains module, for being scanned to real space, forms room data, and establish space coordinates；
Hologram image generation module, for the internal structure of actual key equipment and operation file to be generated key equipment in advance
Hologram image simultaneously stores；
Mixed reality scene module, connects the hologram image generation module and the room data obtains module, for transferring
An at least key equipment hologram image, and be placed in the space coordinates established, realize the key equipment hologram
Picture is superimposed with the real space；
Interactive module connects the mixed reality scene module, for passing through interactive command according to demand to the mixed reality
The superimposed key equipment hologram image of scene module issues dismounting order.
2. training system according to claim 1, which is characterized in that the system also includes:
More people's sharing modules connect the interactive module and the mixed reality scene module, for receiving multiusers interaction life
It enables, and multiusers interaction order is shared, ordered according to the multi-user after shared to key equipment hologram image hair
Dismounting order out.
3. training system according to claim 2, which is characterized in that the mixed reality scene module is also used to receive institute
The dismounting order that interactive module and more people's sharing modules issue is stated, and key equipment hologram image is torn open according to dismounting order
Dress, the dismounting image after obtaining dismounting.
4. training system according to claim 3, which is characterized in that the system also includes:
Third party visual angle module connects the mixed reality scene module, for providing third party visual angle, by the dismounting after dismounting
Image is transformed into two dimensional image by 3-D image, and two dimensional image is projected to large screen and is shown.
5. training system according to claim 4, which is characterized in that the room data obtains module and includes:
Surface viewer module identifies all spatial surfaces in actual room, and create for being scanned to real space
Spatial surface information；
Poller module connects the surface viewer module, is used for continual search space surface information, monitors spatial surface
The variation of information；
Grid cache module connects the poller module, is handled and is cached for the variation to spatial surface information；
Properties Control module connects the grid cache module, for the spatial surface information according to actual needs to caching into
Row optimization；
Grid rendering module connects the Properties Control module, for according to the spatial surface information after optimization to spatial surface
It is rendered, forms room data, and establish space coordinates.
6. training system according to claim 5, which is characterized in that the interactive command includes staring, gesture and voice
Order, described to stare, gesture and voice command issue dismounting life by staring module, gesture module and voice module respectively
It enables.
7. a kind of key equipment Training Methodology based on mixed reality, which is characterized in that using such as any institute of claim 1 to 6
System is stated, described method includes following steps:
Real space is scanned, forms room data, and establish space coordinates；
The internal structure of actual key equipment and operation file are generated into key equipment hologram image in advance and stored；
An at least key equipment hologram image is transferred, and is placed in the space coordinates established, realizes that the key is set
Standby hologram image is superimposed with the real space；
Dismounting order is issued to superimposed key equipment hologram image by interactive command according to demand.
8. method according to claim 7, which is characterized in that the method also includes:
Multiusers interaction order is received, and multiusers interaction order is shared, according to the multiusers interaction order after sharing
Dismounting order is issued to the key equipment hologram image.
9. method according to claim 8, which is characterized in that the method also includes:
The dismounting order that interactive command issues is received, and key equipment hologram image is dismounted according to dismounting order, is obtained
Dismounting image after dismounting.
10. method according to claim 9, which is characterized in that the method also includes:
Third party visual angle is provided, 3-D image of the user after interactive command dismounts key equipment hologram image is turned
Become two dimensional image, and two dimensional image is projected to large screen and is shown.
11. method according to claim 10, which is characterized in that it is described that real space is scanned, room data is formed,
And it establishes space coordinates and further comprises:
Real space is scanned, identifies all spatial surfaces in actual room, and create spatial surface information；
The spatial surface information of continual inquiry creation, monitors the variation of spatial surface information.
The variation of spatial surface information is handled and cached；
The spatial surface information of caching is optimized according to actual needs；
Spatial surface is rendered according to the spatial surface information after optimization, forms room data, and establish space coordinates.
12. according to the method described in claim 10, gesture and voice are ordered it is characterized in that, the interactive command includes staring
It enables, described to stare, gesture and voice command issue dismounting to key equipment hologram image and order.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910243741.5A CN109859538B (en) | 2019-03-28 | 2019-03-28 | Key equipment training system and method based on mixed reality |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910243741.5A CN109859538B (en) | 2019-03-28 | 2019-03-28 | Key equipment training system and method based on mixed reality |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109859538A true CN109859538A (en) | 2019-06-07 |
| CN109859538B CN109859538B (en) | 2021-06-25 |
Family
ID=66902358
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910243741.5A Active CN109859538B (en) | 2019-03-28 | 2019-03-28 | Key equipment training system and method based on mixed reality |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109859538B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349467A (en) * | 2019-07-12 | 2019-10-18 | 中国民用航空飞行学院 | A kind of navigation training on maintenance system and method based on augmented reality |
| CN110554142A (en) * | 2019-08-02 | 2019-12-10 | 珠海格力电器股份有限公司 | Air quality monitoring method, device and system, storage medium and electronic equipment |
| CN110688005A (en) * | 2019-09-11 | 2020-01-14 | 塔普翊海（上海）智能科技有限公司 | Mixed reality teaching environment, teacher and teaching aid interaction system and interaction method |
| CN111367221A (en) * | 2020-03-23 | 2020-07-03 | 国网江苏省电力有限公司镇江供电分公司 | Transformer substation intelligent operation inspection auxiliary system man-machine interaction method based on mixed reality technology |
| CN111427456A (en) * | 2020-06-09 | 2020-07-17 | 杭州翔毅科技有限公司 | Real-time interaction method, device and equipment based on holographic imaging and storage medium |
| CN111467789A (en) * | 2020-04-30 | 2020-07-31 | 厦门潭宏信息科技有限公司 | Mixed reality interaction system based on Holo L ens |
| CN111724486A (en) * | 2020-06-16 | 2020-09-29 | 重庆鑫泺源智能科技有限公司 | Electric power meter installation and power connection practical training system based on mixed reality technology |
| CN111899590A (en) * | 2020-08-25 | 2020-11-06 | 成都合纵连横数字科技有限公司 | Mixed reality observation method for simulation operation training process |
| CN112183223A (en) * | 2020-09-07 | 2021-01-05 | 北京劳动保障职业学院 | Underground pipe network teaching auxiliary system and method based on MR glasses |
| CN112230772A (en) * | 2020-10-14 | 2021-01-15 | 华中师范大学 | Virtual-actual fused teaching aid automatic generation method |
| CN112235552A (en) * | 2020-10-12 | 2021-01-15 | 山东得知科技发展有限公司 | Holographic projection remote control system and method |
| CN113610985A (en) * | 2021-06-22 | 2021-11-05 | 富泰华工业（深圳）有限公司 | Virtual-real interaction method, electronic equipment and storage medium |
| CN114550525A (en) * | 2020-11-24 | 2022-05-27 | 郑州畅想高科股份有限公司 | Locomotive component overhauls real standard system based on mix reality technique |
| CN115346413A (en) * | 2022-08-19 | 2022-11-15 | 南京邮电大学 | Assembly guidance method and system based on virtual-real fusion |
| TWI798514B (en) * | 2019-12-25 | 2023-04-11 | 亞達科技股份有限公司 | Artificial intelligence and augmented reality system and method and computer program product |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106843150A (en) * | 2017-02-28 | 2017-06-13 | 清华大学 | A kind of industry spot simulation method and device |
| CN106874570A (en) * | 2017-01-20 | 2017-06-20 | 清华大学 | Virtual reality cockpit engineering simulator |
| CN107291879A (en) * | 2017-06-19 | 2017-10-24 | 中国人民解放军国防科学技术大学 | The method for visualizing of three-dimensional environment map in a kind of virtual reality system |
| CN107340870A (en) * | 2017-07-13 | 2017-11-10 | 深圳市未来感知科技有限公司 | A kind of fusion VR and AR virtual reality display system and its implementation |
| CN107590862A (en) * | 2017-09-14 | 2018-01-16 | 深圳市彬讯科技有限公司 | A kind of system for orienting Fast rendering threedimensional model |
| CN107705636A (en) * | 2017-07-28 | 2018-02-16 | 武汉理工大学 | A kind of ship experiment teaching system based on augmented reality |
| CN107765843A (en) * | 2017-04-10 | 2018-03-06 | 青岛陶知电子科技有限公司 | A kind of system that intelligent interactive teaching is carried out using virtual reality technology |
| CN109448126A (en) * | 2018-09-06 | 2019-03-08 | 国营芜湖机械厂 | A kind of aircraft equipment repairing auxiliary system and its application method based on mixed reality |
-
2019
- 2019-03-28 CN CN201910243741.5A patent/CN109859538B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106874570A (en) * | 2017-01-20 | 2017-06-20 | 清华大学 | Virtual reality cockpit engineering simulator |
| CN106843150A (en) * | 2017-02-28 | 2017-06-13 | 清华大学 | A kind of industry spot simulation method and device |
| CN107765843A (en) * | 2017-04-10 | 2018-03-06 | 青岛陶知电子科技有限公司 | A kind of system that intelligent interactive teaching is carried out using virtual reality technology |
| CN107291879A (en) * | 2017-06-19 | 2017-10-24 | 中国人民解放军国防科学技术大学 | The method for visualizing of three-dimensional environment map in a kind of virtual reality system |
| CN107340870A (en) * | 2017-07-13 | 2017-11-10 | 深圳市未来感知科技有限公司 | A kind of fusion VR and AR virtual reality display system and its implementation |
| CN107705636A (en) * | 2017-07-28 | 2018-02-16 | 武汉理工大学 | A kind of ship experiment teaching system based on augmented reality |
| CN107590862A (en) * | 2017-09-14 | 2018-01-16 | 深圳市彬讯科技有限公司 | A kind of system for orienting Fast rendering threedimensional model |
| CN109448126A (en) * | 2018-09-06 | 2019-03-08 | 国营芜湖机械厂 | A kind of aircraft equipment repairing auxiliary system and its application method based on mixed reality |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110349467A (en) * | 2019-07-12 | 2019-10-18 | 中国民用航空飞行学院 | A kind of navigation training on maintenance system and method based on augmented reality |
| CN110349467B (en) * | 2019-07-12 | 2020-10-20 | 中国民用航空飞行学院 | Navigation maintenance training system and method based on augmented reality |
| CN110554142A (en) * | 2019-08-02 | 2019-12-10 | 珠海格力电器股份有限公司 | Air quality monitoring method, device and system, storage medium and electronic equipment |
| CN110688005A (en) * | 2019-09-11 | 2020-01-14 | 塔普翊海（上海）智能科技有限公司 | Mixed reality teaching environment, teacher and teaching aid interaction system and interaction method |
| TWI798514B (en) * | 2019-12-25 | 2023-04-11 | 亞達科技股份有限公司 | Artificial intelligence and augmented reality system and method and computer program product |
| CN111367221A (en) * | 2020-03-23 | 2020-07-03 | 国网江苏省电力有限公司镇江供电分公司 | Transformer substation intelligent operation inspection auxiliary system man-machine interaction method based on mixed reality technology |
| CN111467789A (en) * | 2020-04-30 | 2020-07-31 | 厦门潭宏信息科技有限公司 | Mixed reality interaction system based on Holo L ens |
| CN111427456A (en) * | 2020-06-09 | 2020-07-17 | 杭州翔毅科技有限公司 | Real-time interaction method, device and equipment based on holographic imaging and storage medium |
| CN111724486A (en) * | 2020-06-16 | 2020-09-29 | 重庆鑫泺源智能科技有限公司 | Electric power meter installation and power connection practical training system based on mixed reality technology |
| CN111899590A (en) * | 2020-08-25 | 2020-11-06 | 成都合纵连横数字科技有限公司 | Mixed reality observation method for simulation operation training process |
| CN111899590B (en) * | 2020-08-25 | 2022-03-11 | 成都合纵连横数字科技有限公司 | Mixed reality observation method for simulation operation training process |
| CN112183223A (en) * | 2020-09-07 | 2021-01-05 | 北京劳动保障职业学院 | Underground pipe network teaching auxiliary system and method based on MR glasses |
| CN112235552A (en) * | 2020-10-12 | 2021-01-15 | 山东得知科技发展有限公司 | Holographic projection remote control system and method |
| CN112230772A (en) * | 2020-10-14 | 2021-01-15 | 华中师范大学 | Virtual-actual fused teaching aid automatic generation method |
| CN112230772B (en) * | 2020-10-14 | 2021-05-28 | 华中师范大学 | Virtual-actual fused teaching aid automatic generation method |
| CN114550525A (en) * | 2020-11-24 | 2022-05-27 | 郑州畅想高科股份有限公司 | Locomotive component overhauls real standard system based on mix reality technique |
| CN114550525B (en) * | 2020-11-24 | 2023-09-29 | 郑州畅想高科股份有限公司 | Locomotive component overhauling practical training system based on mixed reality technology |
| CN113610985A (en) * | 2021-06-22 | 2021-11-05 | 富泰华工业（深圳）有限公司 | Virtual-real interaction method, electronic equipment and storage medium |
| CN113610985B (en) * | 2021-06-22 | 2024-05-17 | 富泰华工业（深圳）有限公司 | Virtual-real interaction method, electronic equipment and storage medium |
| CN115346413A (en) * | 2022-08-19 | 2022-11-15 | 南京邮电大学 | Assembly guidance method and system based on virtual-real fusion |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109859538B (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109859538A (en) | A kind of key equipment training system and method based on mixed reality | |
| Pinho et al. | Cooperative object manipulation in immersive virtual environments: framework and techniques | |
| EP3457253B1 (en) | Collaboration methods to improve use of 3d models in mixed reality environments | |
| Hedley et al. | Explorations in the use of augmented reality for geographic visualization | |
| CN108762482B (en) | Data interaction method and system between large screen and augmented reality glasses | |
| CN105912110A (en) | Method, device and system for performing target selection in virtual reality space | |
| CN107479705A (en) | A kind of command post's work compound goods electronic sand map system based on HoloLens | |
| CN104916182A (en) | Immersion type virtual reality maintenance and training simulation system | |
| Piekarski et al. | Augmented reality working planes: A foundation for action and construction at a distance | |
| CN111047708B (en) | Complex equipment high-risk project training system based on mixed reality | |
| CN103488292B (en) | The control method of a kind of three-dimensional application icon and device | |
| CN111986334A (en) | Hololens and CAVE combined virtual experience system and method | |
| CN108765576A (en) | VIVE virtual earths based on OsgEarth roam browsing method | |
| Zhang et al. | Application of mixed reality based on Hololens in nuclear power engineering | |
| Rosendo et al. | Towards the development of a 3D serious game for training in power network maintenance | |
| CN103680248A (en) | Ship cabin virtual reality simulation system | |
| WO2022047768A1 (en) | Virtual experience system and method combining hololens and cave | |
| CN115619990A (en) | Three-dimensional situation display method and system based on virtual reality technology | |
| CN116450092A (en) | Multi-display terminal simulation data synchronization and interaction method and system | |
| Duval et al. | Embedding the features of the users' physical environments to improve the feeling of presence in collaborative Virtual Environments | |
| CN115185368A (en) | Mobile robot interactive operation system based on Hololens | |
| CN109147034B (en) | Three-dimensional visual model building method and system for pumped storage power station | |
| Schmidt et al. | A layer-based 3d virtual environment for architectural collaboration | |
| Buriol et al. | A virtual reality training platform for live line maintenance of power distribution networks | |
| Ha et al. | Automatic control of virtual mirrors for precise 3D manipulation in VR |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |