CN109782907A - A kind of virtual filling coorinated training system based on polyhybird real world devices - Google Patents

A kind of virtual filling coorinated training system based on polyhybird real world devices Download PDF

Info

Publication number
CN109782907A
CN109782907A CN201811627407.1A CN201811627407A CN109782907A CN 109782907 A CN109782907 A CN 109782907A CN 201811627407 A CN201811627407 A CN 201811627407A CN 109782907 A CN109782907 A CN 109782907A
Authority
CN
China
Prior art keywords
mixed reality
model
server
polyhybird
intelligent glasses
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811627407.1A
Other languages
Chinese (zh)
Inventor
郑帅
刘倩
赵嗣瑜
李宝童
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Jiaotong University
Original Assignee
Xian Jiaotong University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Jiaotong University filed Critical Xian Jiaotong University
Priority to CN201811627407.1A priority Critical patent/CN109782907A/en
Publication of CN109782907A publication Critical patent/CN109782907A/en
Pending legal-status Critical Current

Links

Landscapes

  • Processing Or Creating Images (AREA)

Abstract

The virtual filling coorinated training system based on polyhybird real world devices that the invention discloses a kind of, hardware facility mainly include a server, multiple stage computers, more mixed reality intelligent glasses, a manipulation and control handle, Ji Yitai wireless router;The present invention by building there is the virtual filling compared with high emulation to equip threedimensional model, establishes filling operation three dimensional physical simulating sports engine, is projected using core computer and Wireless Communication Equipment into each mixed reality intelligent glasses;When model carries out simulating sports, system utilizes coordinate synchronized algorithm and interior SLAM calibration technique, the unified world coordinate system using threedimensional model as reference has been determined, and accurately calculating for mixed reality equipment local coordinate system is carried out, realize the effect from all mixed reality equipment carry out threedimensional model from orientation where itself and angle;Each user can carry out virtual collaborative training by modes such as control crank, gesture, voice operatings.

Description

A kind of virtual filling coorinated training system based on polyhybird real world devices
Technical field
This method is related to mixed reality technology, human-computer interaction and virtual emulation, and in particular to a kind of existing based on polyhybird The virtual filling coorinated training system of real equipment.
Technical background
Mixed reality technology (MR) is the further development of virtual reality technology, and the technology is empty by presenting in reality scene Quasi- scene information, sets up the information circuits of an interaction feedback, between real world, virtual world and user to enhance user The sense of reality of experience;
It is a series of that traditional loading training needs master operator to be executed with control crank to model assembly: translate, rotate, The operations such as flexible, pick-and-place, hard and soft conversion, auxiliary operator carry out solution to model lock and the behaviour that unlocks manually at operating console Make.This traditional loading training must be in outdoor progress, interference of the pole by outside environmental elements because occupying larger physical space.? The subjectivity that evaluation is trained during loading training is too strong, leads to the evaluation method for training no a set of science.Meanwhile because of filling Equipment cost is high, certain training can only single single machine carry out, cause training effectiveness low.The interaction of operator and filling equipment is complete By manipulation and control platform, cause operator's interactive mode single and malfunction.Being likely to occur in the training process can not The contingency of prediction in this way can damage filling equipment, heavy then cause operator injured.
The present invention is built into virtual three-dimensional simulation model by will load model and is projected on mixed reality intelligent glasses, Improve the operation sense and feeling of immersion when training operator;Training when operator can by control crank, gesture, voice side Formula operates dummy model, improves the flexibility of operation, makes interactive mode more diversification;System can be given and grasp in operating process Work person's intelligent prompt simultaneously will be prompted to information and pass through Impactor calibration, collision detection, the shape of holographic character and Intelligent voice broadcasting Formula shows, so that operation is simpler.The present invention supports multi-person synergy training, improves training effect and operating efficiency; System can refer to the operation in training process
Summary of the invention
In order to solve the problems, such as that traditional loading training exists, the purpose of the present invention is to provide one kind based on polyhybird reality The virtual filling coorinated training system of equipment;For operator provide simple to operation, interactive mode diversification, high-immersion it is more The experience of user collaborative loading training.
In order to achieve the above object, the technical solution that this method uses are as follows:
A kind of hardware facility of the virtual filling coorinated training system based on polyhybird real world devices specifically includes that a clothes Business device, multiple stage computers, more mixed reality intelligent glasses, a manipulation and control handle and a wireless router.
Server, multiple stage computers and more mixed reality intelligent glasses are to carry out network company by wireless router It connects.
Server is existing to described more mixing by the virtual filling equipment threedimensional model image projecting of the high emulation of building Real equipment.
Operator is interacted by way of operating control handle, gesture, voice with three-dimensional filling model.
Manipulation and control handle passes through handle drives for the digital data transmission of operation to server.
Mixed reality intelligent glasses include: processor, memory, camera, headset, and mixed reality intelligent glasses can lead to The gesture and voice messaging of camera, headset acquisition operator are crossed, intelligent glasses are by collected audio data and camera The gesture image data real-time synchronization captured is to server.
Further, server has processor and memory, and the processor can be the interactive information solution received The corresponding command information of model sport is analysed into, three-dimensional loads model and makes movement, phantom images according to specific instruction Synchronizing information is into more mixed reality smart machines.The memory can record the movement state information of model when training And it saves.
The beneficial effects of the present invention are embodied in:
The present invention is built into virtual three-dimensional simulation model by will load model and is projected on mixed reality intelligent glasses, Improve the operation sense and feeling of immersion when training operator;Training when operator can by control crank, gesture, voice side Formula operates dummy model, improves the flexibility of operation, makes interactive mode more diversification;System can be given and grasp in operating process Work person's intelligent prompt simultaneously will be prompted to information and pass through Impactor calibration, collision detection, the shape of holographic character and Intelligent voice broadcasting Formula shows, so that operation is simpler.The present invention supports multi-person synergy training, improves training effect and operating efficiency; System can store the operational order in training process in the database, reappear convenient for operation in the future.
Detailed description of the invention
Fig. 1 is a kind of structure chart of the virtual filling coorinated training system based on polyhybird real world devices of the present invention.
Fig. 2 is a kind of network connection diagram of the virtual filling coorinated training system based on polyhybird real world devices of the present invention.
Fig. 3 is a kind of command information transmission of virtual filling coorinated training system based on polyhybird real world devices of the present invention Figure.
Fig. 4 is that a kind of equipment room message of the virtual filling coorinated training system based on polyhybird real world devices of the present invention is same Walk mechanism choice.
Fig. 5 is a kind of model motion information of the virtual filling coorinated training system based on polyhybird real world devices of the present invention Save figure.
Fig. 6 is a kind of model sport reproduction of virtual filling coorinated training system based on polyhybird real world devices of the present invention Figure.
Specific embodiment
This method is described in detail with reference to the accompanying drawings and examples.
The present invention devises a kind of virtual filling coorinated training system based on polyhybird real world devices.Wherein software includes A set of virtual filling coorinated training system based on Unity 3D display engine of independent development, hardware include: a server, Multiple stage computers, more mixed reality intelligent glasses, a manipulation and control handle, Ji Yitai wireless router;
Such as Fig. 1, a kind of realization of the virtual filling coorinated training system based on polyhybird real world devices includes the following contents: 1. constructing the virtual filling equipment threedimensional model of high emulation;2. the network of server, polyhybird real world devices and computer connects It connects;3. three-dimensional loads the absolutization of model spatial position and the multi-angle of view observation of model pose;4. multioperation person collaboration with Model interacts;5. interactive information is transmitted to server and is parsed into the instruction of controllable model sport;6. polyhybird is real The consistency of device model state;7. the preservation of model motion information and the reproduction of training process;
The present invention has good operability and feeling of immersion is high, and equipment is simple and convenient, is easy to carry about with one, not dry by external environment It disturbs, wearing mixed reality equipment in space can carry out virtual filling coorinated training indoors;By voice, gesture identification and mix Real world devices interaction, liberates both hands;Multi-person synergy training, improves experience sense and training effect;Operation in training process refers to Order is saved in a computer in a manner of text, is reappeared convenient for operation in the future.
A kind of virtual filling coorinated training system based on polyhybird real world devices is accomplished by
First, construct the virtual filling equipment threedimensional model of high emulation;The present invention utilizes 3DS Max D modeling tool The building for carrying out virtual filling model, wherein the modeling method selected has: mesh modeling, dough sheet modeling and NURBS modeling;Its In, mesh modeling method, which can be used, in model hard and that details is less is seemed for appearance;And for complexity, surface light Sliding, organized model selection dough sheet modeling and NURBS modeling method are more suitable;The diversified combination of these three modeling methods Use available most perfect modeling effect.
After the completion of moving component modeling, model is directed into Unity 3D rendering engine, using in Unity 3D Shader tinter renders model surface texture.For the fidelity for guaranteeing virtual render scenes, Unity 3D wash with watercolours is utilized It contaminates engine and provides the physical simulation characteristic true to nature such as gravity, collision, flexibility, inertia, weather for model assembly.Pass through analysis model Father and son's nest relation between movement dependence setting model assembly between all parts, is finally completed entire three-dimensional Load the building of model moving scene tree;
Second, the network connection of server, polyhybird real world devices and computer;
Server, polyhybird reality intelligent glasses and PC are attached by wireless router, such as Fig. 2;Network connection In follow ICP/IP protocol, system runtime server automatically opens network monitoring, monitors network in real time, waits mixed reality intelligence Energy glasses client carries out matching connection with it;Mixed reality intelligent glasses enter virtual training scene in operator's wearing, right The operation information of model is by wireless network incoming service device, while each mixed reality intelligent glasses have been correspondingly connected with one The phantom images that intelligent glasses are observed can be displayed on the screen by PC, each end PC, meet the model views body of multi-angle of view It tests.
Third, three-dimensional load the absolutization of model spatial position and the multi-angle of view observation of model pose;Benefit of the invention The spatial coordinate location of three-dimensional filling model is thought in absolute terms with indoor SLAM calibration technique, and has determined and has been with threedimensional model The unified world coordinate system of reference, several operators wear mixed reality equipment Shi Junke simultaneously and roam in three-dimensional scenic, The phantom images observed in real time are sent to multiple stage computers by observing and nursing in all directions, mixed reality equipment, and computer will Image is displayed on the screen;Operator comprehensive observing and nursing, other staff can also pass through in virtual three-dimensional scene Computer display observes the state of model multi-angle of view.
4th, multioperation person's collaboration is interacted with model;The present invention supports multiple operators to wear more mixing simultaneously Real intelligent glasses collaborative training;Loading training task is subdivided into multiple subtasks, and the completion of entire training mission relies on Completion in each subtask;Before each subtask carries out, the model sport portion will first be operated in subtask is needed Part is unlocked, and could control the movement of the component with manipulation and control handle later;After subtask terminates, it is also desirable to this A moving component is locked.As Fig. 3 has a master operator and several auxiliary operators, main operation in the training process Member wears intelligent glasses and enters virtual training scene, passes through the motion state of manipulation and control handle Controlling model;Auxiliary operator Main auxiliary master operator, is unlocked target component before each subtask carries out, to target after task terminates Component is locked.Auxiliary operator can stare target component, viewpoint is placed on when a certain component is unlocked and is locked On target component, then makes fixed gesture or say " unlock "/" locking ", it will be able to target component is unlocked/ Locking.
5th, interactive information is transmitted to server and is parsed into the instruction of controllable model sport;Such as Fig. 3, operator is led to Operation handle, gesture, the mode of voice and three-dimensional filling model is crossed to interact.Handle operation driving matched with handle It is connected on the USB interface of server, handle operation driving refers to the operation that the operation control stream that handle inputs resolves to model Order, which is spread, gives server;There is corresponding program in machine code in server, for setting the specific response of every demand model;Operation When member carries out gesture, voice operating, viewpoint can be concentrated on the three dimensional virtual models portion to be operated by mixed reality intelligent glasses On part, when operator provides specific gesture motion or voice, just by the model in viewpoint in mixed reality intelligent glasses Component information and gesture, voice messaging are transmitted on server;There have a set of specific program on server to be current for parsing Gesture operation and the corresponding command information of voice operating, and command information is acted on three-dimensional filling model and controls mould The movement of type;
6th, the consistency of polyhybird real world devices model state;For the present invention relates to the more equipment of more people it is real-time The characteristics of operation collaboration, message consistency detection in the domain host is introduced in the one-to-one communication mode of client-server, is being passed It unites on the basis of socket is connected to the network and equipment room real-time messages synchronization mechanism is added.Such as Fig. 4, have in server and client side Message data is shared when being modified in the domain host for being arbitrarily marked as [SyncVar] label, and data are shared to by wireless network All devices in the domain host, and before the data manipulation that response receives, equipment is by the data information received in host Domain carries out publicity, and whether the data information for checking that all devices receive is consistent, if unanimously, carried out according to message data into one Step rendering, otherwise, all devices all abandon the response to this message, until collaborative share message data becomes consistent.
7th, the preservation of model motion information and the reproduction of training process;In training process the motion information of model with The form of instruction set saves in the database, a kind of corresponding specific instruction of each operation of model, and time shaft is sequentially special The connection for determining instruction set just constitutes complete training process.Such as Fig. 5, when training starts, first by the movement instruction information of model It saves in memory, if instruction is interrupted in the training process, just empties the instruction saved in memory, only when entire training At the end of process, just instruction all in memory is saved into database;Such as Fig. 6, when to look back certain training process, first Training process is selected in systems, and system is loaded onto the information of the Training scene later, is loaded into scene information and mainly first locks Instruction set in database, then instruction set is stored in the memory of server;Play video i.e. by the instruction in memory one by one It reads, reappears the movement of model.

Claims (3)

1. a kind of virtual filling coorinated training system based on polyhybird real world devices, which is characterized in that a server, more Computer, more mixed reality intelligent glasses, a manipulation and control handle and a wireless router;Server, more calculating Machine and more mixed reality intelligent glasses are connected to the network by wireless router;Server is by the high emulation of building Virtual filling equipment threedimensional model image projecting to the more mixed reality equipment;Operator by operating control handle, Gesture, the mode of voice interact to load model with three-dimensional;Manipulation and control handle passes through handle drives for operation Digital data transmission is to server.
2. a kind of virtual filling coorinated training system based on polyhybird real world devices according to claim 1, feature It is, mixed reality intelligent glasses include: processor, memory, camera, headset, and mixed reality intelligent glasses can pass through Camera, headset acquire the gesture and voice messaging of operator, and intelligent glasses catch collected audio data and camera The gesture image data real-time synchronization grasped is to server.
3. a kind of virtual filling coorinated training system based on polyhybird real world devices according to claim 1, feature It is, server has processor and memory, and the interactive information received can be parsed into model sport by the processor Corresponding command information, three-dimensional filling model make movement according to specifically instructing, and phantom images synchronizing information is to more In platform mixed reality smart machine, the memory records and saves the movement state information of model when training.
CN201811627407.1A 2018-12-28 2018-12-28 A kind of virtual filling coorinated training system based on polyhybird real world devices Pending CN109782907A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811627407.1A CN109782907A (en) 2018-12-28 2018-12-28 A kind of virtual filling coorinated training system based on polyhybird real world devices

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811627407.1A CN109782907A (en) 2018-12-28 2018-12-28 A kind of virtual filling coorinated training system based on polyhybird real world devices

Publications (1)

Publication Number Publication Date
CN109782907A true CN109782907A (en) 2019-05-21

Family

ID=66498879

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811627407.1A Pending CN109782907A (en) 2018-12-28 2018-12-28 A kind of virtual filling coorinated training system based on polyhybird real world devices

Country Status (1)

Country Link
CN (1) CN109782907A (en)

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111028595A (en) * 2019-11-30 2020-04-17 国网河南省电力公司孟州市供电公司 Method for realizing virtual scene manufacturing of cable head by using VR technology
CN111047708A (en) * 2019-11-26 2020-04-21 中国人民解放军空军工程大学 Complex equipment high-risk project training system based on mixed reality
CN111258473A (en) * 2020-01-15 2020-06-09 西安理工大学 Training method for improving generalization ability of autistic children
CN111341168A (en) * 2020-02-28 2020-06-26 西安交通大学 Collaborative loading training teaching system and method
CN111899590A (en) * 2020-08-25 2020-11-06 成都合纵连横数字科技有限公司 Mixed reality observation method for simulation operation training process
CN112230772A (en) * 2020-10-14 2021-01-15 华中师范大学 Virtual-actual fused teaching aid automatic generation method
CN112419827A (en) * 2020-12-16 2021-02-26 江苏一鼎堂软件科技有限公司 Intelligent manufacturing practical training system based on MR mixed reality technology
CN113593347A (en) * 2021-08-10 2021-11-02 中国人民解放军63919部队 Many people training system in coordination based on virtual reality
CN115470710A (en) * 2022-09-26 2022-12-13 北京鼎成智造科技有限公司 Air game simulation method and device

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103941861A (en) * 2014-04-02 2014-07-23 北京理工大学 Multi-user cooperation training system adopting mixed reality technology
CN105788390A (en) * 2016-04-29 2016-07-20 吉林医药学院 Medical anatomy auxiliary teaching system based on augmented reality
KR20160142648A (en) * 2015-06-03 2016-12-13 주식회사 지엔아이씨티 Presentation training system through the virtual reality
CN106340217A (en) * 2016-10-31 2017-01-18 华中科技大学 Augmented reality technology based manufacturing equipment intelligent system and its implementation method
CN106980385A (en) * 2017-04-07 2017-07-25 吉林大学 A kind of Virtual assemble device, system and method
CN107111894A (en) * 2014-09-08 2017-08-29 西姆克斯有限责任公司 For specialty and the augmented reality simulator of education and training
CN107122045A (en) * 2017-04-17 2017-09-01 华南理工大学 A kind of virtual man-machine teaching system and method based on mixed reality technology
CN107145223A (en) * 2017-04-06 2017-09-08 北京讯腾智慧科技股份有限公司 Multi-point interaction control system and method based on Unity d engines and the VR helmets
CN108334199A (en) * 2018-02-12 2018-07-27 华南理工大学 The multi-modal exchange method of movable type based on augmented reality and device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103941861A (en) * 2014-04-02 2014-07-23 北京理工大学 Multi-user cooperation training system adopting mixed reality technology
CN107111894A (en) * 2014-09-08 2017-08-29 西姆克斯有限责任公司 For specialty and the augmented reality simulator of education and training
KR20160142648A (en) * 2015-06-03 2016-12-13 주식회사 지엔아이씨티 Presentation training system through the virtual reality
CN105788390A (en) * 2016-04-29 2016-07-20 吉林医药学院 Medical anatomy auxiliary teaching system based on augmented reality
CN106340217A (en) * 2016-10-31 2017-01-18 华中科技大学 Augmented reality technology based manufacturing equipment intelligent system and its implementation method
CN107145223A (en) * 2017-04-06 2017-09-08 北京讯腾智慧科技股份有限公司 Multi-point interaction control system and method based on Unity d engines and the VR helmets
CN106980385A (en) * 2017-04-07 2017-07-25 吉林大学 A kind of Virtual assemble device, system and method
CN107122045A (en) * 2017-04-17 2017-09-01 华南理工大学 A kind of virtual man-machine teaching system and method based on mixed reality technology
CN108334199A (en) * 2018-02-12 2018-07-27 华南理工大学 The multi-modal exchange method of movable type based on augmented reality and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
张茂于: "《产业专利分析报告 第51册 虚拟现实与增强现实》", 30 June 2017 *

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111047708A (en) * 2019-11-26 2020-04-21 中国人民解放军空军工程大学 Complex equipment high-risk project training system based on mixed reality
CN111047708B (en) * 2019-11-26 2022-12-13 中国人民解放军空军工程大学 Complex equipment high-risk project training system based on mixed reality
CN111028595A (en) * 2019-11-30 2020-04-17 国网河南省电力公司孟州市供电公司 Method for realizing virtual scene manufacturing of cable head by using VR technology
CN111258473A (en) * 2020-01-15 2020-06-09 西安理工大学 Training method for improving generalization ability of autistic children
CN111258473B (en) * 2020-01-15 2021-08-06 西安理工大学 Training method for improving generalization ability of autistic children
CN111341168A (en) * 2020-02-28 2020-06-26 西安交通大学 Collaborative loading training teaching system and method
CN111899590B (en) * 2020-08-25 2022-03-11 成都合纵连横数字科技有限公司 Mixed reality observation method for simulation operation training process
CN111899590A (en) * 2020-08-25 2020-11-06 成都合纵连横数字科技有限公司 Mixed reality observation method for simulation operation training process
CN112230772A (en) * 2020-10-14 2021-01-15 华中师范大学 Virtual-actual fused teaching aid automatic generation method
CN112230772B (en) * 2020-10-14 2021-05-28 华中师范大学 Virtual-actual fused teaching aid automatic generation method
CN112419827A (en) * 2020-12-16 2021-02-26 江苏一鼎堂软件科技有限公司 Intelligent manufacturing practical training system based on MR mixed reality technology
CN113593347A (en) * 2021-08-10 2021-11-02 中国人民解放军63919部队 Many people training system in coordination based on virtual reality
CN115470710A (en) * 2022-09-26 2022-12-13 北京鼎成智造科技有限公司 Air game simulation method and device

Similar Documents

Publication Publication Date Title
CN109782907A (en) A kind of virtual filling coorinated training system based on polyhybird real world devices
CN108986189B (en) Method and system for capturing and live broadcasting of real-time multi-person actions based on three-dimensional animation
CN102413885B (en) Systems and methods for applying model tracking to motion capture
CN103941861B (en) Multi-user cooperation training system adopting mixed reality technology
US20200005538A1 (en) Remote Collaboration Methods and Systems
CN102596340B (en) Systems and methods for applying animations or motions to a character
CN102947777B (en) Usertracking feeds back
CN102741885B (en) Decoration display environment
CN104778872B (en) The emulation experiment training platform studied for the man-machine interaction of mechanical arm remote operating
CN103854300B (en) The implementation method of three-dimensional scenic cooperation carrying under multi-client networking control
CN102622774B (en) Living room film creates
CN111047708B (en) Complex equipment high-risk project training system based on mixed reality
CN104881115B (en) A kind of method and system that personalized interaction is realized using equipment operation control terminal
Barakonyi et al. Agents that talk and hit back: Animated agents in augmented reality
CN110515467A (en) A kind of multi-person synergy interactive system in virtual reality training field
CN106327589A (en) Kinect-based 3D virtual dressing mirror realization method and system
CN104637088B (en) It is a kind of that 3D graphical virtuals managing and control system and its implementation are realized to intelligent appliance
CN103246765A (en) Developing method for equipping virtual training platform
CN106780769B (en) Three-dimensional model drawing system and method for reducing shielding of close-distance object
CN110515455A (en) It is a kind of based on the dummy assembly method cooperateed in Leap Motion and local area network
CN110083235A (en) Interactive system and data processing method
CN115220578A (en) Interactive VR system and method based on optical motion capture
Sénécal et al. Modelling life through time: cultural heritage case studies
Fu et al. Real-time multimodal human–avatar interaction
CN109815557A (en) A kind of methods of exhibiting of robot model, device and intelligent terminal

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190521