TWM606337U - Cloud multi-person and remote joint training mixed reality simulation system - Google Patents

Abstract

A cloud multi-person and remote joint training mixed reality simulation system includes a server-side host and a user-side host connected through a communication network. The training simulation execution module of the server-side host is connected with a simulation situation database to set the execution virtual environment. The required simulation situation information, and the virtual character database connected to the training staff execution module is used to set the training character information required to execute the virtual environment, so that the server-side host can create the virtual environment and provide the situation as the simulation execution of joint training ; Each user-end host can participate in the construction of virtual environment training character information through the connected simulation device and the joint training performed in conjunction with the simulation situation information.

Description

Cloud multi-person and remote joint training mixed reality simulation system

This creation provides a virtual simulation system about multi-person connection, especially a joint training system that allows multiple people to connect at the same time in different places for mixed reality simulation.

In the current training courses for different purposes, the common forms are to allow participants to gather in a specific place for actual operation exercises, or through online courses to explain the teaching by watching videos. The former is limited by geographical location and transportation. , It is not conducive to the participation of participants who are relatively far away, and the latter is limited by the fact that the movie can only be used for viewing and cannot be operated in practice, and lacks the sense of presence required for training.

In view of the fact that the purpose of training is to actually encounter the situation presented by the training, the trainees who have participated in the training can better respond to the current situation, the training process needs to be more realistic simulation situations, and to make people want to participate The trainees can join the training more conveniently to facilitate the achievement of the training course purpose.

The main purpose of this creation is to make the simulation of training more in line with the purpose of training and the actual situation, and it is also more convenient for participants to join.

In order to achieve the above purpose, the authoring cloud multi-person remote joint training mixed reality simulation system includes: the system includes a server-side host, including a processing module, and a training module connected to the processing module. A training simulation execution module and a training personnel execution module connected to the processing module. The processing module has a transmission unit. The training simulation execution module is connected to a simulation situation database for setting and executing a virtual environment. A simulated situational information. The training staff execution module is connected to a virtual character database for setting at least one participant information required to execute the virtual environment, so that the server-side host can create the virtual environment and provide contextual actions Simulated execution of joint training; a communication network connected to the transmission unit for the server host to upload and download data to the outside; plural user hosts, which are connected to the server host through the communication network, and Each is connected to a simulation device, through which the training character information of the virtual environment can be constructed and the joint training performed in accordance with the simulation situation information can be participated.

When the training staff executes the module, it is linked to a user-side setting interface, and the user-side setting interface includes a plurality of types for the network connection of each simulation device to select a group to which it belongs to, and cooperate with the virtual character database The simulated information of the corresponding trainees is displayed in the virtual environment of other simulation devices for joint training.

According to requirements, two different groups can be selected, and the corresponding training character information can be simulated in conjunction with the virtual character database and displayed in the virtual environment of the simulation device of the unselected group for joint training.

The training simulation execution module is linked to a server-side setting interface when the training simulation execution module is executed, and the required simulation situation information is selected from the simulation situation database through the server-side setting interface, and the simulation situation information is transmitted through the network connection To display in the virtual environment of each simulation device.

The simulated situation database includes physical characteristic data such as facial expressions, voices, blood pressure, heartbeat, respiration, and electrocardiogram of the virtual character for identifying the current situation of the virtual character.

The server-side host is connected to a display, and through the server-side host, the trainee information and the simulated situation information constructed by each user-side host can be simultaneously displayed on the display. The display is a head-mounted display device for VR, AR, MR, XR and other contextual devices.

The simulation device includes at least one display, at least one controller, at least one audio device, and at least one communication device connected to the user-end host, or is connected to the user-end host's VR, AR, MR, XR and other contextual devices Head-mounted equipment, handles, gloves and other accessory accessories.

The simulation system of this creation can also be connected only to a single client host, and the simulation device can also participate in the construction of the virtual environment's training character information and the training performed in conjunction with the simulation situation information, which is on the client configuration interface Select all groups through the network connection, and cooperate with the virtual character database to simulate the information of the participants in the corresponding group to be displayed in the virtual environment for individual training.

In order to achieve the above purpose, another technical means for creating a cloud multi-person remote joint training mixed reality simulation system includes: a server-side host including a processing module, a training simulation execution module connected to the processing module, and A trainer execution module connected to the processing module, the training simulation execution module is connected to a simulation situation database for setting up a piece of simulation situation information required to execute a virtual environment, and the trainer execution module is connected to a The virtual character database is used to set the training character information required to execute the virtual environment, so that the server-side host can create the virtual environment and provide the context as the simulation execution of joint training; a communication network connected to the processing module Group for the server host to upload and download external data; a client host connects to the server host through the communication network, and is connected to an analog device through which the virtual environment can be constructed The information of the participants in the training and the training performed in conjunction with the simulated situation information.

When the training staff executes the module, it is linked to a user-side setting interface. The user-side setting interface includes multiple groups. The simulation device selects all groups through the network connection and simulates the corresponding with the virtual character database. The information of the grouped participants is displayed together in the virtual environment for separate training.

100: analog system

10: Server host

11: Processing module

12: Training simulation execution module

13: Trainer execution module

14: Simulation situation database

15: Trainees database

16: transmission unit

20: Communication network

30: Using end host

40: Simulator

41: display

42: Controller

43: Audio source equipment

44: call equipment

45: Somatosensory equipment

46: Head-mounted display device

47: Accessories

Figure 1 is a schematic diagram showing the overall structure of the best embodiment of this creation; Figure 2 is a schematic diagram showing the simulation device in this creation; Figure 2A is another schematic diagram showing the simulation device in this creation; Figure 3 shows this A schematic diagram of the virtual environment created during multi-person training in different places; Figure 4 is a schematic diagram of the user-side configuration interface in the creation; Figure 5 is a schematic diagram of the server-side configuration interface in the creation.

Please refer to Figures 1-3, which is a schematic diagram showing the overall architecture of the best embodiment of the creative cloud multi-person remote joint training mixed reality simulation system. The following is the application of the simulation system 100 in the first aid of advanced cardiac life-saving surgery (ACLS) Take simulation training as an example.

The simulation system 100 includes a server host 10, which is connected to a user host 30 set in a different place through a communication network 20, and the user hosts 30 in different places are each connected to an analog device 40, which is connected at the same time. The local user-end host 30 conducts multi-person remote joint emergency operation simulation training.

The server host 10 includes a processing module 11, a training simulation execution module 12 connected to the processing module 11, and a training personnel execution module 13 connected to the processing module 11.

The training simulation execution module 12 is connected to a simulation situation database 14 for setting the simulation situation information required to execute the virtual environment 50. During emergency simulation training, the default virtual character of the simulation situation information is the patient at the emergency scene. The current situation is set based on the patient’s facial expression, voice, heart rhythm and electrocardiogram and other physiological characteristics data in the simulated situation database 14, so that the trainee can identify the current situation of the virtual person; and the training staff execution module 13 is connected to a virtual The character database 15 is used to set the information of the trainees required to execute the virtual environment. During the first aid simulation training, since the actual trainees may be located in different places, the information of the trainees in the virtual environment 50 needs to be created correspondingly. Cooperate with other co-trainees to train together, and enable the server host 10 to create a virtual environment 50 and provide the required scenarios as a simulation execution of the joint training.

The processing module 11 has a transmission unit 16 connected to the communication network 20, so that the server host 10 can upload and download data externally, and the server host 10 can be connected to the communication network 20 as shown in Fig. 3 A plurality of end-user hosts 30 in different places, each of the end-user hosts 30 is connected to a simulation device 40. The processing module 11 provides trainees in different places to participate in the construction of the virtual environment 50 through the simulation device 40 and cooperate with the trainees. Joint training performed by simulated situational information.

As shown in Figure 2, the simulation device 40 includes a display 41 connected to the simulation host 30, a controller 42, a sound source device 43, a communication device 44, and a motion sensing device 45 to simulate the trainee’s actions with other trainees in different locations. interactive.

The simulation device 40 may include a display 41, a controller 42, a sound source device 43, a communication device 44, and a motion sensing device 45 connected to the user-end host 30, or may be connected to the user-end host 30 The head-mounted display device 46 of VR, AR, MR, XR and other situational devices, as well as accessory accessories 47 such as handles and gloves, can be used to interact with other trainees.

In actual training, refer to Figure 3 as shown. For example, in four different locations, there is a corresponding trainee A, trainee B, trainee C, and trainee D in each of the first place to the fourth place. Take the trainee B from the two places as an example, he wears a simulation device 40 and is connected to the server host 10 through the client host 30.

When the trainer executes the module 13, it is preset to link a user-side setting interface 60. As shown in Figure 4, the user-side setting interface 60 is preset with multiple groups for trainees to use the simulation device 40. The route connection selects the group to which it belongs, and cooperates with the virtual character database 15 to simulate the corresponding training character information, such as virtual characters A1, C1, D1, for displaying in the virtual environment 50 of the simulation device 40 of other groups. Joint training.

The user-side setting interface 60 in Figure 4 is preset with four different groups, which are used in conjunction with the first aid simulation training of advanced cardiac life-saving (ACLS). They are the first group responsible for airway treatment (Airway team) 61, responsible The second group (Circulation team) 62 for chest compression quality and electric shock, the third group (Leader team) 63 responsible for command, and the fourth group (Support team) 64 responsible for injection, drug delivery, recording, timing and other support. If trainee B chooses the third group 63, click his own code in the drop-down menu to select trainee B as the participant of the third group 63, and then display the information of trainee B's participants in other groups for training In the virtual environment 50 of the simulation device 40 of A, C, and D, joint training is performed together.

In the virtual environment 50 of the simulation device 40 of the trainee B in the second place, as shown in Figure 3, the trainees A, C, and D of the other groups in the first, third, and fourth places are also trained The personnel execution module 13 cooperates with the virtual character database 15 to simulate the training character information corresponding to the three respectively, and then The virtual environment 50 of the simulation device 40 displayed on the trainee B is used for joint training with other trainees A, C, and D in different places.

Trainee B can also select two different groups on the user-side setting interface 60, such as the first group 61 and the third group 63. The corresponding trainee information can also be simulated from the simulation situation database 14 for display in other groups. Joint training is carried out in the virtual environment 50 of the simulation device 40 of the unselected group.

Each emergency simulation training can be preset through the server host 10 for different emergency situations. When the training simulation execution module 12 is executed, the default is linked to a server setting interface 70, as shown in Figure 5, through the server The setting interface 70 selects the required simulation situation information from the simulation situation database 14 and simultaneously transmits the simulation situation information to the virtual environment 50 of each simulation device 40 for display through the network connection.

The server-side setting interface 70 in Figure 5 has preset multiple heart-related abnormalities options, such as atrial fibrillation, flutter, bradycardia, and atrial tachycardia , Atrioventricular block, asystole, ventricular fibrillation, and heart-related abnormalities that can be customized by the user as required, except for the currently known Set simulated situational information for heart-related abnormalities. You can also set up physiological characteristics such as blood pressure, heartbeat, and respiration in the interface to allow trainees to identify the virtual person’s current emergency situation. The server-side setting interface 70 may also include other ECG-type physiological characteristic data not shown in the figure, as well as situational data such as the expression and voice of the virtual person to be rescued.

Depending on the needs of the training situation, the server-side host 10 can also be connected to a display (not shown in the figure). Through the server-side host 10, the virtual character information and simulation situation constructed by each user-end host 30 can be Information is displayed on the display at the same time. The display may be a head-mounted display device such as VR, AR, MR, XR, etc., which can interact with trainees in the virtual environment 50.

Furthermore, the server host 10 can also cooperate with a single person for training. In the case of a single client host 30, the trainee can still use the simulation device 40 to select all groups in the client setting interface 60 through a network connection. In the same way, the virtual character database 15 can be used to simulate the information of the corresponding group of trained characters to be displayed in the virtual environment 50 for separate training.

The simulation system 100 in this creation allows each group to perform exercises with single or multiple players when personnel participate in grouping, and supports simultaneous exercises of multiple groups, and can further record the results of each training in the corresponding database (Figure Not shown), the results of the exercise can be scored and reviewed later.

The above-disclosed embodiments are just examples to illustrate the preferred implementation of the creation, and to illustrate the possible implementation modes and implementation interfaces of the creation with diagrams, and are not intended to limit the scope of patent application of the creation.

100: analog system

10: Server host

11: Processing module

12: Training simulation execution module

13: Trainer execution module

14: Simulation situation database

15: Trainees database

16: transmission unit

20: Communication network

30: Using end host

40: Simulator

Claims (11)

A cloud multi-person and remote joint training mixed reality simulation system. The system includes: a server-side host including a processing module, a training simulation execution module connected to the processing module, and a training simulation execution module connected to the processing module Training personnel execution module, the processing module has a transmission unit, the training simulation execution module is connected to a simulation situation database for setting a piece of simulation situation information required to execute a virtual environment, and the training personnel execution module is connected to a simulation situation database. The virtual character database is used to set at least one participant information required to execute the virtual environment, so that the server-side host can create the virtual environment and provide context as a simulation execution of joint training; a communication network connected to the virtual environment Transmission unit for the server host to upload and download data to the outside; multiple client hosts are connected to the server host through the communication network, and each is connected to an analog device, through which the analog device can participate in the construction of the Participants information in the virtual environment and joint training performed in conjunction with the simulated situation information. For example, the cloud multi-person remote joint training mixed reality simulation system of request 1, wherein the trainer executes the module when executing a user-side configuration interface, and the user-side configuration interface includes multiple groups for each simulation The network connection of the device selects the group to which it belongs, and the corresponding training character information is simulated with the virtual character database and displayed in the virtual environment of the simulation device of other groups, and joint training is carried out together. For example, the cloud multi-person remote joint training mixed reality simulation system of request 1, wherein the trainer executes the module when executing a user-side configuration interface, and the user-side configuration interface includes multiple groups for each simulation Select two different groups of device network connection to match the avatar The database simulates the information of the corresponding trainees and displays it in the virtual environment of the simulation device of the unselected group, and conducts joint training together. For example, the cloud multi-person remote joint training mixed reality simulation system of request 1, in which the training simulation execution module is linked to a server setting interface when the training simulation execution module is executed, and the required setting is selected from the simulation situation database through the server setting interface The simulated situation information is transmitted to the virtual environment of each simulation device through a network connection for display. For example, the cloud multi-person remote joint training mixed reality simulation system of request item 1 or 4, wherein the simulation situation database includes the virtual character's facial expression, voice, blood pressure, heartbeat, respiration and electrocardiogram and other physiological characteristics data for identification of virtual characters The current situation of the character. For example, the cloud multi-person remote joint training mixed reality simulation system of claim 1, wherein the server-side host is connected to a display, through the server-side host, the trainee information and the simulation situation constructed by each user-end host can be obtained Information is also displayed on the display. For example, the cloud multi-person remote joint training mixed reality simulation system of claim 6, wherein the display is a head-mounted display device of VR, AR, MR, XR and other contextual devices. For example, the cloud multi-person remote joint training mixed reality simulation system of claim 1, wherein the simulation device includes at least one display, at least one controller, at least one audio source device, and at least one communication device connected to the user-end host. For example, the cloud multi-person remote joint training mixed reality simulation system of claim 1, wherein the simulation device includes a head-mounted display device connected to the user-end host's VR, AR, MR, XR and other contextual devices and a handle for use in conjunction , Gloves and other accessories. A cloud multi-person and remote joint training mixed reality simulation system, which includes: A server-side host includes a processing module, a training simulation execution module connected to the processing module, and a trainer execution module connected to the processing module. The training simulation execution module is connected to a simulation situation data The library is used to set up a piece of simulated situation information required to execute a virtual environment. The training staff execution module is connected to a virtual character database for setting up the training character information required to execute the virtual environment so that the server host By creating the virtual environment and providing scenarios as the simulation execution of joint training; a communication network connected to the processing module for the server-side host to upload and download external data; a client-side host through the The communication network is connected to the server-side host, and is connected to a simulation device, through which the training character information of the virtual environment can be constructed and the training performed in accordance with the simulation situation information. For example, the cloud multi-person remote joint training mixed reality simulation system of claim 10, wherein the trainer executes the module in conjunction with a user-side setting interface when the module is executed. The user-side setting interface includes multiple groups, and the simulation device is connected through the Internet The road connection selects all groups, and cooperates with the virtual character database to simulate the information of the participants in the corresponding group and display it in the virtual environment for individual training.

Images