KR20240098181A - ITREATMENT TRAINING PLATFORM OF XR-Based TRAUMA INJURY TREATMENT TRAINING SYSTEM - Google Patents

Abstract

The present invention relates to a virtual reality data integrated management method of an XR-based severe trauma treatment training system, and a virtual reality data integrated management method of a virtual reality data integrated management system in an XR-based severe trauma treatment training system. From the administrator's perspective, the system login process (S2), AR/VR content creation process (S4), basic information input process (content title, date and time) (S6), content classification selection process (AR/VR classification) (S8) ), user preparation confirmation process (S10), content start process (S12), monitoring process (S14), and content termination process (S16). From the user (member) side, AR/VR wearing (S20), identity verification (S22) (The user information registered with the content being prepared and the identity verification check process are performed. At this time, the administrator confirms the user preparation (S10), waits for the content (S24), and starts the content (S26) (AR/VR The content starts, and the administrator also checks it (S12)), the content progresses (S28) (content progress information is displayed, and is monitored by the administrator (S14)), so that severe trauma treatment training is performed and monitored. It is characterized by

Description

Virtual reality data integrated management method for XR-based severe trauma treatment training system {ITREATMENT TRAINING PLATFORM OF XR-Based TRAUMA INJURY TREATMENT TRAINING SYSTEM}

The present invention relates to a virtual reality data integrated management method for an XR-based severe trauma treatment training system, and more specifically, VR (Virtual Reality) technology, AR (Augmented Reality) technology, and MR (Mixed) technology. An XR-based severe trauma treatment training system that provides content related to severe trauma and helps nurses, doctors, and medical workers train in severe trauma treatment in medical settings to effectively treat seriously ill patients. The treatment training platform provides the ability to integrate and manage various virtual reality contents, and allows simultaneous execution and control of various virtual reality contents, allowing integrated management of AR and VR contents. This is about a virtual reality data integrated management method for a new type of XR-based severe trauma treatment training system that can manage VR hardware control integratedly and allows individuals and groups to select and use AR and VR content simultaneously. .

VR (Virtual Reality) technology provides objects and backgrounds in the real world only as CG (Computer Graphics) images, while AR (Augmented Reality) technology provides virtually created CG images on top of images of real objects, and MR (Mixed Reality) technology provides ) technology is a computer graphics technology that mixes and combines virtual objects in the real world. The aforementioned VR, AR, MR, etc. are all simply referred to as XR (extended reality) technologies.

XR technology can be applied to HMD (Head-Mount Display), HUD (Head-Up Display), glasses-type glass, mobile phones, tablet PCs, laptops, desktops, TVs, digital signage, etc. Devices to which XR technology is applied are called XR. It can be called a device (XR Device).

The XR device described above is a display that displays information and may be equipped with a transparent display.

The user can see objects in the real world located across from the transparent display through the transparent display, and can also view information provided by the XR device through the transparent display.

Various application technologies based on the above XR technology are being implemented.

Meanwhile, the limitations of existing medical practice and simulator practice are explained.

Time is limited for both the teaching staff and the learners, and the skills to be learned are many and varied. Patients' resistance to medical visits is increasing. Practice in the field may result in medical accidents. (Virtual reality surgical simulation in training, Patel et al, Expert Rev. Anticancer Ther, 2012).

Clinical practice is essential for nursing students, but sometimes clinical practice is conducted only to a limited extent for reasons of patient safety and privacy (Nursing students' virtual simulation practice experience, Yunjeong Kim et al., JKASNE, 2020).

The group that received surgical training using virtual reality showed learning skills 29% faster than the group that did not, had a success rate of more than 9 times, and reduced the rate of complications by 1/5 (Virtual Reality Training Improves Operating Room Performance, Seymour et al , ANNALS OF SURGERY, 2002).

Knowledge and confidence improved by using a virtual reality simulation program in nursing education for prostate cancer patients, and in the case of a study on the development and effectiveness of children's nursing skills using virtual games, nursing students' satisfaction with learning child nursing skills through virtual games increased. reported to be high. (Effects of virtual reality simulation and blended simulation education on nursing children with asthma, Mi-gang Kim et al., CHNR, 2019).

Meanwhile, various countries at home and abroad are introducing XR-based training systems in the evaluation of national qualification examinations.

In the United States, Israel, Germany, Denmark, and Australia, national qualification examinations using simulators are being implemented.

Texas Governor Abbott announced on March 21, 2020 that those who have trained more than 50% through simulation to secure nursing staff due to the coronavirus will be recognized as having achieved the clinical goal.

Thirty U.S. states, including 50 states and Washington, D.C., have written regulations for the use of medical simulation in nursing programs.

A new practical test subject for the national exam for dentists will be established in 2021, and the Korea National Institute of Health and Medical Services is reviewing whether to introduce virtual reality (VR) technology in the practical test. In other words, by conducting part of the test process in VR, it is possible to save equipment and standardized patient training costs, permanently reuse production models, and save quantitative scoring and scoring labor costs by collecting behavioral data.

Meanwhile, in March 2018, the government announced the pan-government 『Severe Trauma Treatment System Improvement Measures』 and is promoting 10 major tasks in 3 major areas, including trauma center treatment capacity and training center expansion.

The Ministry of National Defense and the Ministry of Health and Welfare plan to dispatch military surgeons to regional trauma centers and expand trauma specialist training centers for specialized training of trauma center personnel (promoting the dispatch and training of military medical residents to regional trauma centers with excellent training conditions and trauma care capabilities). We are promoting regional trauma center training (required for training in specialized treatment courses at trauma centers, such as multidisciplinary treatment through teams and trauma resuscitation areas).

Additionally, technology is currently being developed to create a virtual human body based on actual clinical data and perform analysis through various simulations.

Through the e-Spine project, the Korea Institute of Science and Information (KISTI) developed a simulation system for constructing a skeletal shape model with degenerative spinal diseases and developing spinal implants for the human spine.

Additionally, KIST developed software that implements MRI and CT images in 3D to diagnose diseases and plan surgery using artificial intelligence.

Electronics and Telecommunications Research Institute of Korea developed a medical twin to assist in the diagnosis of musculoskeletal diseases based on clinical data, and developed 3D simulation-based diagnostic assistance and disease prediction technology for knee arthritis, posture abnormalities, sarcopenia, and shoulder diseases.

In this way, devices are being developed to present various physical haptic sensations in virtual reality to users.

In addition, domestic and international patents in the medical virtual 3D human body field are expected to continue to increase due to attempts to incorporate ICT into the medical field.

In addition, there are no direct standardization activities in the medical simulation field, and most standardization has been carried out related to medical information.

Meanwhile, with regard to medical information, the National Institute of Technology and Standards is working to apply the international standards developed by ISO/TC215 as KS standards and is working on national standardization by considering the conditions for revitalizing domestic industries for international standards such as HL7, IEEE, and DICOM. .

HL7 (Health Level 7) is a standard that refers to the communication protocol for medical information and stipulates application protocols for the exchange of data in the medical field. The HL7 Korean branch is being established and operated at the Korea Health Industry Development Institute in accordance with the agreement.

Additionally, various services that perform training based on virtual medical simulation are being developed.

At Cedars-Sinai Hospital in Los Angeles, more than 300 doctors use VR to learn skills such as how to assess a patient's symptoms or perform CPR while wearing protective gear.

The 2019 School of Nursing (APN) students will integrate technology throughout their curriculum and integrate not only the previously mentioned training in early nursing, but also the Objective Structured Clinical Examination (OSCE), skills training, high-fidelity manikin training, and standardized patient care. and hybrid training with mannequins.

The Emergency Medicine and Medical Simulation Training Center at the Johanniter Clinic-Waldkrankenhaus has engaged more than 250,000 participants from more than 80 countries around the world to reduce disability, disability and death from cardiovascular and neurological problems.

Canada's CAE, a company specializing in civil aircraft simulators, expanded its long experience in simulator development into medical services to develop medical simulators and health care services.

Oxford Medical Simulation (OMS) in the UK has partnered with the University of Oxford to allow students to use VR headsets to practice patient care and receive feedback on steps taken to improve performance.

Meanwhile, efforts to commercialize a system that reproduces realistic medical situations using XR for training are continuing.

In addition, due to the expansion of virtual reality devices and the special characteristics of the surgical environment, technologies related to anatomy and surgery have been developed and are being commercialized.

[Prior art literature]

Republic of Korea Patent Publication No. 10-2019-0098925 (published on August 23, 2019) (Title of invention: XR device and control method thereof)

The present invention was proposed in consideration of the above-described conventional situation, and provides content related to severe trauma using VR (Virtual Reality) technology, AR (Augmented Reality) technology, and MR (Mixed) technology to provide nurses, nurses, and nurses in medical settings. On the treatment training platform of the XR-based severe trauma treatment training system, which helps doctors and medical workers to effectively treat seriously ill patients by training them in severe trauma treatment, various virtual reality contents can be integrated and managed. It provides functions that enable simultaneous execution and control of various virtual reality contents, allowing integrated management of AR content and VR content, integrated management of AR and VR hardware control, and AR content for individuals and groups. The purpose is to provide a virtual reality data integrated management method for a new type of XR-based severe trauma treatment training system that allows selection and simultaneous use of VR content.

Another purpose of the present invention is to provide content related to severe trauma using VR (Virtual Reality) technology, AR (Augmented Reality) technology, and MR (Mixed) technology to treat severe trauma for nurses, doctors, and medical workers in medical settings. From the treatment training platform of the XR-based severe trauma treatment training system, which helps to effectively treat seriously ill patients by providing training, AR, VR content and peripheral device control management, both individual and group use of the content. The purpose is to provide a system that can accept and control and manage related functions.

Another purpose of the present invention is to provide content related to severe trauma using VR (Virtual Reality) technology, AR (Augmented Reality) technology, and MR (Mixed) technology to prevent severe trauma from nurses, doctors, and medical workers in medical settings. On the treatment training platform of the XR-based severe trauma treatment training system, which helps to effectively treat seriously ill patients by providing treatment training, AR, VR, and other hardware and software can be integrated and managed on the system. The goal is to provide a system that can be used simultaneously by individuals and groups when using functional features and content.

Another purpose of the present invention is to provide content related to severe trauma using VR (Virtual Reality) technology, AR (Augmented Reality) technology, and MR (Mixed) technology to prevent severe trauma from nurses, doctors, and medical workers in medical settings. The purpose is to provide a treatment training platform for the XR-based severe trauma treatment training system that helps patients effectively treat seriously ill patients through treatment training.

Another purpose of the present invention is to develop a medical twin based on actual severe trauma data, produce realistic MR content, and apply it to the severe trauma treatment training system platform, providing military nursing for severe trauma treatment through the military nursing training simulator center. The purpose is to conduct training demonstrations and provide a treatment training platform for the XR-based severe trauma treatment training system to disseminate and expand the training system for training trauma center personnel in each region.

According to a preferred embodiment of the present invention for achieving the above-described object, training scenario information processing means, electrocardiogram, heart rate, oxygen Mannequin patient biometric information processing means that performs the function of processing saturation, respiratory rate per minute, body temperature, and blood pressure information, pulse band, haptic glove, HoloLens, stethoscope, and training device information processing means that performs the function of processing CPR information , a severe trauma processing training platform that performs the function of collecting processing results from the training scenario information processing means, the mannequin patient biometric information processing means, and the training device information processing means and analyzing the training data, the severe trauma treatment training platform, and A cloud infrastructure that is connected and performs the functions of managing, storing, and providing data received through a communication network from a remote location, operates by being connected to the severe trauma treatment training platform 100, and includes a training information registration UI, real-time training monitoring UI, and training evaluation. Severe trauma treatment training/evaluation UI/UX including management UI (User Interface), training that is connected to the above severe trauma treatment training platform to manage, store, and provide personal evaluation information, evaluation information by teamwork, and comprehensive training evaluation information. It includes an evaluation information processing unit and an open platform that is connected to the severe trauma treatment training platform and provides military hospital information sharing, training education material sharing, and data security management functions, and provides training scenarios and trainee information from the training scenario information processing means. XR (eXtended Reality), which provides a severe trauma treatment training platform that receives input and monitors training (mannequin information, trainee motion tracking information) and provides the ability to analyze comprehensive training evaluation and evaluation information. ) An XR-based severe trauma treatment training system is provided to provide severe trauma treatment training for medical workers by providing based content.

also,

The severe trauma treatment training platform is

Treatment training platform management system,

A doctor helicopter that is connected to the treatment training platform management system and a wireless communication network to transmit and receive patient monitoring data, and

It includes a severe trauma treatment training center that is connected to the treatment training platform management system and a wireless communication network to transmit and receive patient monitoring data,

The treatment training platform management system is

Training Evaluation Server (TES) that evaluates training, Training Control Server (TCS: Training Control Server) that controls training, DB Server (DBS: DB Sever) that stores and manages data transmitted and received during system operation, Includes an operation & management server (O&M) that controls the system operation and management process,

The training control server includes a training control S/W module (TCM: Training Control S/W Module) that controls training, and a Farcom Mannequin Tracking S/W module (MTM: Mannequin Tracking S/W) that tracks the position of the mannequin and the position of the trainee. Module), test monitoring S/W module (TESm:TES Monitoring S/W Module), which monitors the training test process, TCS monitoring S/W module (TCSm:TCS Monitoring S/W Module), which controls the training monitoring process, training It includes a DBS monitoring S/W module (DBSm:DBS Monitoring S/W Module) that stores and manages the data monitoring process, and a storage monitoring S/W module (STRm:Storage Monitoring S/W Module) that stores training data. It is characterized by

also,

Doctor helicopter

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes images transmitted from the one or more HoloLens and transmits them wirelessly through a treatment training platform management system;

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

It stores and transmits data received through wired or wireless communication with the smart terminal, and includes a patient monitor control tablet that monitors and stores or transmits patient data related to the mannequin.

In addition, the severe trauma treatment training site is

A marketing tracking device that measures trainee location or mannequin location and posture values and transmits them through a wireless AP device,

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes images transmitted from the one or more HoloLens and transmits them wirelessly through a treatment training platform management system;

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits mannequin-related patient data,

It is characterized by including a VR device that provides VR data for training in treating seriously ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device.

In addition, it performs the function of collecting training-related information logs at the training site, and collects and analyzes mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure) from the mannequin patient biometric information processing means,

It is characterized by performing a process of collecting and analyzing trainee motion tracking information through information collected from the training device information processing means.

In addition, in the severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system to provide severe trauma treatment training for medical workers by providing XR (eXtended Reality)-based content,

The severe trauma treatment training platform cloud is

Treatment training platform management system from A to N points,

It includes an evaluation server that is connected to the treatment training platform management system at points A to N through the Internet and stores and evaluates patient monitoring data transmitted through the treatment training platform management system,

The severe trauma treatment training site at point A above is

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes and transmits images transmitted from the one or more HoloLens,

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits mannequin-related patient data,

A VR device that provides VR data for training in treating critically ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device, and

It is connected to the HoloLens, smart terminal, patient monitor control tablet, and VR device through a wired and wireless communication network, and is connected to the evaluation server through the Internet to transmit and receive data. Patient monitoring data through the mannequin is measured and transmitted through the Internet network. It is characterized by including a training control server that does.

In addition, it performs the function of collecting training-related information logs at the training site, and collects and analyzes mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure) from the mannequin patient biometric information processing means,

It is characterized by performing a process of collecting and analyzing trainee motion tracking information through information collected from the training device information processing means.

In addition, in the severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system to provide severe trauma treatment training for medical workers by providing XR (eXtended Reality)-based content,

The severe trauma treatment training platform cloud is

A~N customer treatment training platform management system,

It includes an evaluation server that is connected to the treatment training platform management system of the customers A to N through the Internet and stores and evaluates patient monitoring data transmitted through the treatment training platform management system,

The evaluation server includes a training control server that controls severe trauma treatment training by receiving patient monitoring data through a mannequin through an Internet network through a HoloLens, a smart terminal, a patient monitor control tablet, a VR device, and a wired or wireless communication network,

The severe trauma treatment training center for customer A above is

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes and transmits images transmitted from the one or more HoloLens,

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits patient data related to the mannequin, and

It is characterized by including a VR device that provides VR data for training in treating seriously ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device.

In addition, it performs the function of collecting training-related information logs at the training site, and collects and analyzes mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure) from the mannequin patient biometric information processing means,

It is characterized by performing a process of collecting and analyzing trainee motion tracking information through information collected from the training device information processing means.

According to another aspect of the present invention, user devices such as MR devices, VR devices, haptic devices, and practice mock-ups (mannequins) are devices (500-1 to 500-3) that allow users to directly write and view content, and the content A device that transmits data occurring within,

The AV/VR integrated management system is 1300, which is connected to the user device through a wired or wireless communication network to transmit and receive data and collect and manage data for severe trauma treatment training.

An integrated management web (1330) including an integrated dashboard, MR training monitoring WEB UI, MR training management WEB UI, MR scenario evaluation WEB UI, and operation management (DevOps) (module) is provided, and is provided for managers, training professors, The web used by the trainer is given menus and functions according to each authority,

API GW and OPEN API (module) represent the linking API between the integrated management web (1330), the integrated data collection server (1310), and the integrated data operation management server (1320),

The sum data collection server 1310 includes a data collection system (API, MQ, Adapter, AAA) that collects data, a scenario management system (CMS, MR Contents, Ctrl, Ctrl) that manages scenarios, and a scenario evaluation that evaluates scenarios. It serves as a server for content data including system (Analysis, Tracking, Test, Monitoring), user data, and content operation result data.

The integrated data operation management server 1320 represents a data server for operation, acts as an operation management manager, and performs alarms, failure management, configuration management, and statistics management.

There is also a cloud-type XR server and a cloud/deployment-type infrastructure as a stand-alone XR server.

The data storage includes databases including clinical data DB, photography/auscultation/palpation data DB, medical twin DB, MR content DB, manikin DB, and trainer DB.

Docker Registry for services is distributed to the integrated data collection server (1310),

In the XR-based severe trauma treatment training system, the administrator server is equipped with a system control PC and is connected to the AR/VR integrated management system (1300) through a network to control the overall system. A virtual reality data integrated management system is provided.

In addition, the integrated data collection server (13010) receives the data uploaded from the user device information from the scenario control server, determines whether it fits the scenario, delivers the result to the scenario evaluation server through open API, and displays it in the user UI (web) and monitoring. However,

The scenario control server (1200) is

Network communication is performed with the integrated data collection server (1301) of the AR/VR integrated management system, and various sensor equipment transmitted through HW devices (500-1 to 500-3), that is, various equipment linkages, AR devices, and VR devices Collects information (TCP/UDP), transmits device sensor information, collects and secures user information,

Through API G/W and OPEN API

The integrated data collection server, including the data collection server, scenario management server (or training control server), and scenario evaluation server, transmits scenario numerical data/result data to the integrated data web server, receives scenario sensor information, and trains Data is received from the control server and transmitted to the evaluation server, and the evaluation server determines the time limit for severe trauma treatment training and whether or not it has been performed.

Data acquired during scenario execution is stored and managed in a data storage.

According to another aspect of the present invention, in the virtual reality data integrated management method of the virtual reality data integrated management system in the XR-based severe trauma treatment training system,

From the administrator's perspective, system login process (S2), AR/VR content creation process (S4), basic information input process (content title, date and time) (S6), content classification selection process (AR/VR classification) (S8), user Performs the preparation confirmation process (S10), content start process (S12), monitoring process (S14), and content end process (S16).

From the user (member) side, AR/VR wearing (S20), identity verification (S22) (linking user information registered with content being prepared and identity verification check process) are performed, and at this time, the administrator performs user preparation verification (S10). content standby (S24), content start (S26) (AR/VR content starts, and the administrator also checks it (S12)), content progress (S28) (content progress information is displayed, and the content is monitored by the administrator) (S14) A virtual reality data integrated management method is provided in the XR-based severe trauma treatment training system, which is characterized by performing and monitoring severe trauma treatment training.

According to the XR-based severe trauma treatment training system according to the present invention described above, effective XR-based medical training is possible, and it is possible to improve the patient's survival rate through rapid and accurate response during the severe trauma treatment process. XR virtual training for severe trauma treatment has the effect of enabling mastery of physical assessment and treatment skills for severe trauma treatment.

1 is an explanatory diagram explaining a future medical platform.
Figure 2 is a schematic system configuration diagram of an open healthcare platform.
Figure 3 is a configuration diagram schematically showing an example of an XR-based severe trauma treatment training platform according to the present invention.
Figure 4 is a configuration diagram schematically showing an example of an XR-based severe trauma treatment training system according to the present invention.
Figure 5 is a diagram showing an example of the medical standard HL7 communication structure in the system of Figure 4.
Figure 6 is a diagram showing the development process of a training device data linkage adapter in the XR-based severe trauma treatment training platform according to the present invention.
Figure 7 is a diagram showing the development process of a training device data linking adapter in the XR-based severe trauma treatment training platform according to the present invention.
Figure 8 is a diagram showing the development process of a training device data linking adapter in the XR-based severe trauma treatment training platform according to the present invention.
Figure 9 is a diagram showing the data security process in the XR-based severe trauma treatment training platform according to the present invention.
Figure 10 is a schematic diagram of the configuration of the XR-based severe trauma treatment training platform according to the present invention.
Figures 11 and 12 are schematic diagrams of the configuration of the XR-based severe trauma treatment training platform cloud according to the present invention.
Figure 13 is a conceptual diagram illustrating a virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.
Figure 14 is a configuration diagram showing an example of a virtual reality data integrated management system in the XR-based severe trauma treatment training system according to the present invention.
Figure 15 is a diagram showing an example of a linked service data flow chart for integrated management of virtual reality data in the XR-based severe trauma treatment training system of Figure 14.
Figure 16 is a flowchart schematically showing an example of the virtual reality data integrated management process in the XR-based severe trauma treatment training system according to the present invention.
Figure 17 is a diagram showing an example of an interface screen of the real-time monitoring status of education and training in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.
Figure 18 is a diagram showing an example of an interface screen of an education and training evaluation table in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.
Figure 19 is a diagram showing an example of an education and training management interface screen in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.
Figure 20 is a diagram showing an example of an interface screen at the end of training in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail.

However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

The terms used in this application are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as generally understood by a person of ordinary skill in the technical field to which the present invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted in an ideal or excessively formal sense unless explicitly defined in the present application. No.

Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. In order to facilitate overall understanding when describing the present invention, the same reference numerals are used for the same components in the drawings, and duplicate descriptions for the same components are omitted.

Hereinafter, the XR-based severe trauma treatment training system according to the present invention will be described in detail with reference to the accompanying drawings, focusing on preferred embodiments.

Figure 1 is an explanatory diagram explaining a future medical platform, Figure 2 is a schematic system configuration diagram of an open healthcare platform, and Figure 3 is a schematic diagram of an example of an XR-based severe trauma treatment training platform according to the present invention. , and Figure 4 is a diagram schematically showing an example of an XR-based severe trauma treatment training system according to the present invention, and Figure 5 is an example of the medical standard HL7 communication structure in the system of Figure 4. , and Figure 6 is a diagram showing the development process of a training device data linkage adapter in the XR-based severe trauma treatment training platform according to the present invention, and Figure 7 is a diagram showing the XR-based severe trauma treatment training platform according to the present invention. It is a diagram showing the development process of a training device data linking adapter in a severe trauma treatment training platform, and Figure 8 is a diagram showing the development process of a training device data linking adapter in an XR-based severe trauma treatment training platform according to the present invention. , Figure 9 is a diagram showing the data security process in the XR-based severe trauma treatment training platform according to the present invention.

Referring to Figure 1, the future medical platform technology performs personalized health management using patient PHR data, application of medical platform technology based on the HL7 FHIR standard, and provision of health care services through 3D human body modeling.

The patient's (P) life log (1), treatment data (2), and health checkup data (3) are collected from the medical platform (4).

After performing human body digital twin processing, the medical platform (4) stores and manages data in the cloud infrastructure, a remote information storage, through the health examination system, AI decision support system, PHR health management system, and hospital information system.

Personal information, health checkup information, bio information, EMR information, public medical information, cohort information, clinical information, and PHR information are stored and managed in the cloud infrastructure.

The human body digital twin processing results processed on the medical platform (4) can be viewed through a device capable of 3D/AR monitoring of the patient's condition.

Additionally, the disease prediction simulation (6) is linked with the medical platform (4) to predict the disease.

Referring to Figure 2, looking at an example of an open healthcare platform, it supports international standards based on HL7 FHIR, provides scalability with a cloud-based Micro Service Architecture structure, and predicts five types of chronic diseases using medical big data. It is equipped with a blockchain-based PHR data sharing function and provides a precision medicine service development environment using data lake and data sandbox. Obtained HL7 FHIR compatibility certification.

The open healthcare platform (aka Redwood Platform) (10) is

It is connected to the PHR-based healthcare service (server) 20 and performs data inquiry/control functions.

The healthcare mobile app 30 on the user terminal transmits and receives healthcare-related data with the PHR-based healthcare service server (server) 20.

The PHR supply organization (server) 40 is connected to the open healthcare platform 10 and supplies PHR through a standard interface.

The PHR device 50 is connected to the open healthcare platform 10 and supplies PHR through a standard interface.

The small and medium-sized venture business support portal 70 is linked to the open healthcare platform 10 and supports developers (terminals) (terminals of small and medium-sized businesses, individuals, and startups) 60.

The PHR user organization (server) 80 is connected to the open healthcare platform 10 through a standard interface.

Referring to Figure 3, it is an overall conceptual diagram related to the construction of an XR-based severe trauma treatment training system through the XR-based severe trauma treatment platform according to the present invention. A medical twin based on actual severe trauma data is developed, and an immersive training system is developed. By producing MR content and applying it to the severe trauma treatment training system platform, we aim to demonstrate military nursing training for severe trauma treatment through the military nursing training simulation center and to spread the training system for training trauma center personnel in each region. The goal is to make the concept a reality.

Severe trauma data (110) collected from severe trauma centers in each local area is collected through the medical twin production system (120), managed/stored through the severe trauma treatment training platform (130), and training evaluation is performed.

The remote severe trauma treatment training center 140 and the severe trauma treatment training center 150 are connected to the severe trauma treatment training platform 130 through a communication network and are a system capable of transmitting and receiving data.

Meanwhile, to briefly explain Medical Twin, it is one of the effects that can be expected when applying virtual convergence (XR) technology to existing industries, and some of it, like AR glasses, is already being used in practice. Reserve medical staff can receive training virtually to provide treatment in trauma situations, and they can also predict and manage future health conditions using a ‘medical twin (virtual body)’ that is digitally replicated by reflecting personal medical information.

Meanwhile, looking at the main process of physical assessment training, the physical assessment is conducted on the patient's condition through activities such as questioning (questions/answers), inspection (observation), auscultation (listening), palpation (touching), and percussion (tapping). It is a process of understanding. In XR severe trauma training, it is necessary to present patients with random symptoms based on MR devices and realistic content so that appropriate treatment can be judged and body assessment skills for various symptoms that are easy to misjudge can be acquired. It will provide a technical background for realistic training based on XT for triage of large-scale emergency patients that may occur in wartime or mass disaster situations.

Looking at the technical analysis of XR physical assessment training, it is analyzed through questionnaire (voice recognition-based interaction), visual examination (presentation of real-virtual matching rendering), auscultation (reproduction of disease-related sounds), palpation and percussion (haptic rendering, feedback). do.

Looking at the main process of trauma treatment training, it is the process of establishing a treatment plan and performing surgical treatment based on the results of physical assessment diagnosis, and XR treatment training is a real-virtual fusion system that combines MR content with a mannequin that provides physical sensation. It is a process of training appropriate treatment procedures through interaction.

The development of an XR severe trauma training course according to the present invention is briefly described.

For XR-based severe trauma training education, a random patient designated by a professor is presented to the simulator as a virtual severe patient, and training is conducted to improve rapid patient diagnosis and treatment skills and perform evaluation.

Presentation of severe trauma cases: Realistic training content is created based on actual severe trauma cases and provided to enable repeated training.

Diagnosis and Treatment: Through simulation training, students acquire initial assessment and treatment competencies in the A-B-C-D-E sequence and experience for each trauma case. in other words,

A. Airway fixes the cervical spine so that it does not move, and maintains the airway to ensure the patient's breathing.

B. Breathing is provided by determining whether breathing is secured, whether the person is resting comfortably, whether the sound is okay, and whether oxygen is sufficient, and provides appropriate assistance.

C. Circulation evaluates whether there is a lack of blood in the body, stops bleeding where bleeding is seen, supplies fluid, and considers blood transfusion.

D. Disability is assessed by other neurological tests, including consciousness.

E. Exposure and Environment remove clothing and maintain body temperature

Evaluation: Students' treatment behaviors are stored and evaluated by developing objective evaluation indicators.

The nursing curriculum used at the Armed Forces Nursing Academy, which is the core consumer of the results, generally consists of nurse definition, nursing diagnosis, nursing planning, nursing intervention, and nursing evaluation.

According to the present invention, it is a system for breaking away from apprenticeship-style education, preserving and disseminating digital-based medical cases, and introducing an education system.

Medical Twin technology, which creates a virtual human body based on actual patient cases, has the significance of digitizing medical cases based on realistic content and digitizing skills and experiences that were preserved by human capabilities.

Meanwhile, there is a large gap in improving the capabilities of medical staff due to differences in the number of patients due to regional population imbalance, and in Europe, remote medical services are actively supported as a solution to the increase in medical desertification, where there are not enough doctors per 100,000 people. there is. The XR Transformation-based medical training system according to the present invention is expected to expand into an online platform for sharing patient experience cases, resolve regional medical staff competency imbalances, and lead a new medical education system used to strengthen the capabilities of overseas medical staff in the long term. do.

In addition, it will be possible to create a new market for medical content based on XR Transformation suitable for the post-corona era.

In addition to the rapid increase in demand for non-face-to-face remote medical training triggered by infectious diseases, the demand for XR-based medical training is expected to steadily increase due to the expansion of mandatory first aid training targets, strengthening of patient human rights, and shortage of cadavers.

In the present invention, the development of a medical twin model capable of reproducing arbitrary symptoms is performed. In other words, we develop medical twin-based training content that reproduces actual severe trauma cases and develop a differentiated training system that can reproduce mass disasters.

In addition, the present invention builds a medical twin-based severe trauma treatment training simulator. In other words, we develop a training platform for sharing patient cases and experiences through an online open platform, and build a training simulator environment that maximizes realism (helicopter, operating room, field presentation).

In addition, the present invention carries out a verification process to advance treatment training based on a military nursing simulation center. In other words, we will promote verification through the introduction of the Armed Forces Nursing Academy curriculum and the use of the Armed Forces Capital Hospital and Ajou University Hospital, and provide a continuing education program on severe trauma treatment for approximately 2,800 military doctors and nursing officers nationwide, including experts at the Armed Forces Capital Hospital Trauma Center*. Develop and implement.

Referring to Figure 4, the severe trauma treatment training management system is a system that provides training management and evaluation tools for verification linkage with military nursing training centers.

The severe trauma treatment training platform 100 receives training scenarios and trainee information from the training scenario information processing means 300, and links the monitoring (mannequin information, trainee motion tracking information) information for processing training to comprehensively evaluate and evaluate training. Provides the function to analyze information.

The severe trauma treatment training platform 100 performs a training-related information log collection function, that is, the mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, Blood pressure) collects and performs analysis. ( Electrocardiogram is a record of the heart's electrical activity - Heart rate is the number of heartbeats per unit of time - Respiration rate is the number of breaths per minute - Body temperature is the patient's body temperature - Blood pressure is the rate at which blood moves through the walls of blood vessels systolic pressure and diastolic pressure)

In addition, the severe trauma treatment training platform 100 performs a trainee motion tracking information collection and analysis process through information collected from the training device information processing means 500 (pulse band: pulse information of the mannequin - haptic glove: trainee's Tracking of tactile treatment information - Holo Lens: Tracking of trainee's visual treatment information - Stethoscope: Mannequin's respiratory rate information - CPR: Chest compression information).

The training scenario information processing means 300 performs the functions of registering and managing trainee information, registering and modifying training scenarios, and managing training evaluation standard information.

The mannequin patient biometric information processing means 400 performs the function of processing electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, and blood pressure information.

The training device information processing means 500 performs the function of processing pulse band, haptic glove, HoloLens, stethoscope, and CPR information.

The severe trauma treatment training platform 100 collects processing results from the training scenario information processing means 300, the mannequin patient biometric information processing means 400, and the training device information processing means 500 and analyzes the training data. performs its function.

In other words, the training evaluation system of the severe trauma treatment training platform 100 includes a training information management unit that manages trainee information management, training scenarios, and training evaluation information, a training treatment management unit that manages training monitoring, training motion tracking, and training mannequin treatment information; It includes a training evaluation management department that provides individual/teamwork management, training performance evaluation, and comprehensive analysis reports.

Additionally, the severe trauma treatment training platform 100 includes an OPEN API such as data collection, PHR information collection, and service data storage, and a training data collection system including technical support portals such as DATA LAKE and DARA SANDBOX.

The cloud infrastructure 900 performs the functions of managing, storing, and providing data received through a communication network from a remote location in connection with the severe trauma treatment training platform 100.

The severe trauma treatment training/evaluation UI/UX 200 operates in connection with the severe trauma treatment training platform 100 and includes a training information registration UI, a real-time training monitoring UI, and a training evaluation management UI (User Interface).

The training evaluation information management unit 600 is connected to the severe trauma treatment training platform 100 to manage, store, and provide personal evaluation information, evaluation information by teamwork, and comprehensive training evaluation information.

The open platform 700 is connected to the severe trauma treatment training platform 100 and provides military hospital information sharing, training education material sharing, and data security management functions.

Referring to Figures 5 and 6, the severe trauma handling training platform 100 designs and implements a severe clinical case medical institution data linkage system development training device (HL7-based) data linkage adapter and an external system (training MR server) linkage adapter. A medical data linkage system will be provided.

Referring to Figure 5, between transmission and reception, a data structure of 1 to N Bytes is transmitted and received. Data Content has an HL7 message.

Referring to FIG. 6, the severe trauma training platform 100 receives training device information through the training device 500-1 through the gateway (training system) 800.

The severe trauma training platform 100 is

Training device data collection (PHR-based) department;

PHR core (PHR-based),

Includes training common functions and data processing unit.

The training device data (PHR-based) part collects based on PHR and performs data processing to enable analysis of the collected data, and includes a training standard adapter, external training system interconnection (API), wearable IoT adapter, and training device HL7 FHIR adapter. Includes.

The PHR core includes Core API, Manager API, MASTER API, and collection database, and transmits data from the training device data collection unit to the training common function and data processing unit.

The training common function and data processing unit includes API GW, OPEN API, authentication security, training information management, and service database, and performs the function of processing data transmitted from the PHR core. In Figure 6, it is possible to develop a training device data linking adapter. That is,

● Provides data collection function by linking training devices (haptic rendering device, mannequin biometric information)

● Design of interconnection structure based on medical standard HL7 protocol

● Implementation of interlocking adapter based on medical standard HL7 protocol

The process is carried out.

Referring to FIG. 7, which is a diagram similar to FIG. 6, the training device 500-2 is an MR device, and the MR data collected through it is provided to a severe trauma training platform through a gateway (training system) (Hospital Gateway) 800. It is delivered to (100) and processed. In Figure 7, an external system interworking adapter is developed. in other words,

● Link training data that cannot be directly linked to training devices that do not support HL7 and severe trauma training platforms.

● Analysis and design of various interconnection specifications depending on data characteristics

The process is carried out.

In Figure 7, implementation and testing of external training data collection integration that is not supported by HL7 can be performed.

Referring to FIG. 8, which is a diagram similar to FIG. 7, the training device 500-3 is directly connected to the severe trauma training platform 100 without going through a gateway as mannequin biometric information, haptic rendering device, and MR training information to provide data. can also be transmitted. Training information history/log statistics, event logs, and device status information can be stored in the service database of the severe trauma training platform. in other words,

● Training device data processing

● Distributed processing of large amounts of data

● Data integration collection log analysis and processing

The process is carried out.

In addition, according to the present invention, the following functions can be performed. That is,

Training motion tracking system

Track training movements through motion tracking information to analyze the behavior of training subjects.

Biosignal processing that collects and stores various logs generated from MR-based severe trauma training (mannequin patient biometric information, etc.)

It provides an overall training management environment that allows classification of training categories according to location and situation and management of information on the current training scenario.

Management system such as training scenario updates, etc.

UI and functions that allow you to register training scenario information,

Trainer information and evaluation criteria registration, modification, and deletion functions;

Comprehensive monitoring UI and functions that allow you to monitor real-time training information,

The ability to manage comprehensive data and evaluation scores on training evaluations becomes possible.

Training status monitoring

Monitoring of the training status of individual and team trainers from the start of training to the end of training,

Monitoring of secondary training situations and indoor and confined space training situations

Information linking function according to each training scenario situation of the trainee

Video recording of training content and monitoring function that allows you to understand the training situation from various angles

The ability for evaluators to monitor by adding emergency emergency scenarios becomes possible.

Mannequin biometric information monitoring

Provides an interface that can link training mannequin information,

Ability to monitor data transmitted from the training mannequin for each biological function

Individual monitoring function for training information on individual mannequins

Comprehensive monitoring UI and functions can be implemented that allow the evaluator to view the trainee's first aid in real time.

Training data analysis

Can link evaluation criteria data for training scenarios,

After starting training, algorithms necessary for analyzing trainee data for various situations can be linked,

Comprehensive training data analysis of training mannequins, haptic rendering devices, and MR training information becomes possible.

Training Evaluation Tool

Training Evaluation Process

● The treatment training procedure consists of securing an airway, checking breathing, maintaining circulation, checking consciousness, and controlling the exposure environment.

● The target of training evaluation is divided into physical assessment and first aid items, a training scenario is created, and each evaluation method is applied.

● The training evaluation of physical assessment uses the function of MR, and the training evaluation of first aid uses a mannequin and MR simultaneously.

Training Evaluation Process

Development of severe trauma evaluation system and analysis of learning effects

● Development of a severe trauma evaluation system and analysis of learning effects based on international standard trauma guidelines and textbooks

● The scenario management and evaluation system for treatment training applies international standards such as ATLS, PHTLS, etc.

● Develop evaluation indicators linked to haptics, motion recognition technology, biosignal data, etc.

● Establish an automated training evaluation system based on objective training data

The process is carried out.

Referring to FIG. 9, data security in the present invention will be described.

Data and DB data security

Apply the security function of digitizing patient information by encrypting and decrypting data and DB data within the platform

● Data security

● Targets personal information and biometric information

● When collecting, the data is transmitted as encrypted data and stored in encrypted form in DB and storage.

Referring to Figure 9, the severe trauma processing platform 100 is

Collection data/video/file 116 collected through the device, API GW 118 performs an authorization/authentication check when requesting data and file encryption, and has an encryption, decryption, and decryption key management unit ( 112) encrypts and stores personal information/biometric information, and encrypts and stores medical record/image information files in the database 114.

The severe trauma processing platform 100 performs SSL-based data/file/video transmission and reception with the client 1000 terminal.

In the data security process, upon transmission, the requester's authority and authentication are checked, and then the decryption key and transmitted encrypted data are decrypted and displayed on the client 1000.

Data security is:

● DB security

● Investigate the status of personal information and biometric information to identify encryption targets

● Classify data subject to one-way and two-way encryption

● Encryption is performed on data stored in the DB to fundamentally prevent DB data leakage due to illegal access by unauthorized users.

According to the XR-based severe trauma treatment training system according to the present invention described above, effective XR-based medical training is possible, and it is possible to improve the patient's survival rate through rapid and accurate response during the severe trauma treatment process. Through virtual training for XR severe trauma treatment, it is possible to master physical assessment and treatment skills for severe trauma treatment.

Figure 10 is a schematic diagram of the configuration of the XR-based severe trauma treatment training platform according to the present invention, and Figures 11 and 12 are XR-based severe trauma treatment training platform cloud according to the present invention. This is a schematic drawing of the configuration.

Referring to Figure 10, in the severe trauma treatment training platform of the XR-based severe trauma treatment training system for severe trauma treatment training,

The severe trauma treatment training platform is

Treatment training platform management system (100-1),

A doctor helicopter (1000) connected to the treatment training platform management system (100-1) through a wireless communication network to transmit and receive patient monitoring data, and

It includes a severe trauma treatment training center (150-1) that is connected to the treatment training platform management system (100-1) and a wireless communication network to transmit and receive patient monitoring data,

The treatment training platform management system is

Training Evaluation Server (TES) (10-1) that evaluates training, Training Control Server (TCS: Training Control Server) (20-1) that controls training, and stores and manages data transmitted and received during system operation. It includes a DB server (DBS: DB Sever) (30-1) that controls the system operation and management process, and an operation & management server (O&M: Operation & Management server) (40-1) that controls the system operation and management process. represents the database),

The training control server (20-1) includes a training control S/W module (TCM: Training Control S/W Module) that controls severe trauma training, and a Farcom mannequin tracking S/W module ( MTM:Mannequin Tracking S/W Module), test monitoring S/W module that monitors the training test process (TESm:TES Monitoring S/W Module), TCS monitoring S/W module that controls the training monitoring process (TCSm:TCS Monitoring S/W Module), DBS monitoring S/W module (DBSm:DBS Monitoring S/W Module) that stores and manages the training data monitoring process, storage monitoring S/W module (STRm:Storage Monitoring S/W module) that stores training data W Module).

The doctor helicopter (1000) is

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes images transmitted from the one or more HoloLens and transmits them wirelessly through a treatment training platform management system;

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

It stores and transmits data received through wired or wireless communication with the smart terminal, and includes a patient monitor control tablet that monitors and stores or transmits patient data related to the mannequin.

The severe trauma treatment training center (150-1) is

A marketing tracking device (152) that measures the trainee's position or mannequin's position and posture and transmits it through a wireless AP device,

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes images transmitted from the one or more HoloLens and transmits them wirelessly through a treatment training platform management system;

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits mannequin-related patient data,

It includes a VR device 154 that provides VR data for training in treating critically ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device.

The severe trauma treatment training platform of the XR-based severe trauma treatment training system performs the function of collecting training-related information logs at the training site, and collects mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation) from the mannequin patient biometric information processing means. , respiratory rate per minute, body temperature, blood pressure) and perform analysis;

A trainee motion tracking information collection and analysis process is performed through the information collected from the training device information processing means.

The built-in configuration shown in FIG. 10 has the feature of being able to process data in real time and has the feature of minimizing processing delay time.

Referring to Figure 11, in the severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system for severe trauma treatment training of medical workers,

The severe trauma treatment training platform cloud is

Treatment training platform management system from A to N points,

It includes an evaluation server (2000) that is connected to the treatment training platform management system at points A to N through the Internet and stores and evaluates patient monitoring data transmitted through the treatment training platform management system, (unexplained reference numeral 2100 is represents the database).

The severe trauma treatment training center (1500-1) at point A above is

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes and transmits images transmitted from the one or more HoloLens,

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits mannequin-related patient data,

A VR device that provides VR data for training in treating critically ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device, and

It is connected to the HoloLens, smart terminal, patient monitor control tablet, and VR device through a wired and wireless communication network, and is connected to the evaluation server through the Internet to transmit and receive data. Patient monitoring data through the mannequin is measured and transmitted through the Internet network. It includes a training control server (1510-1).

In the severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system, the training-related information log collection function at the training site is performed, and the mannequin patient biometric information processing means (electrocardiogram, Perform collection and analysis (heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure),

It is characterized by performing a process of collecting and analyzing trainee motion tracking information through information collected from the training device information processing means.

Unexplained codes 1500-1~1500-N indicate the trauma treatment training areas for points A~N, and 1510-1~1510-N indicate the trauma treatment training areas for customers A~N.

The cloud configuration in Figure 11 is competitive in system price compared to the built-up type (Figure 10), enables reduction of TCO such as system installation cost and maintenance by dedicated personnel, and improves commercialization ability by accepting various customers.

Referring to Figure 12, in the severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system for severe trauma treatment training of medical workers,

The severe trauma treatment training platform cloud is

A~N customer treatment training platform management system,

It includes an evaluation server that is connected to the treatment training platform management system of the customers A to N through the Internet and stores and evaluates patient monitoring data transmitted through the treatment training platform management system,

The evaluation server is a training control server (2000-1) that controls severe trauma treatment training by receiving patient monitoring data through the mannequin through the Internet network through HoloLens, smart terminals, patient monitor control tablets, VR devices, and wired and wireless communication networks. Includes, (unexplained code 2100-1 indicates database),

The severe trauma treatment training center (1510-1) of customer A above is

One or more HoloLens that captures and transmits images through the mannequin,

A wireless AP device that wirelessly processes and transmits images transmitted from the one or more HoloLens,

A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,

A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits patient data related to the mannequin, and

It is characterized by including a VR device that provides VR data for training in treating seriously ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device.

In the severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system, the training-related information log collection function at the training site is performed, and the mannequin patient biometric information processing means (electrocardiogram, Perform collection and analysis (heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure),

It is characterized by performing a process of collecting and analyzing trainee motion tracking information through information collected from the training device information processing means.

The cloud configuration in FIG. 12 is different from the cloud configuration in FIG. 11, and the ARR (Azure Remoter Rendering) function is added. The cloud configuration in Figure 12 is more price-competitive than the cloud configuration in Figure 11, can reduce TCO such as system installation costs and dedicated personnel maintenance compared to the built-up type (Figure 10), and accommodates small customers (2 HoloLens). And the service can be provided at a low price.

Figure 13 is a conceptual diagram illustrating a virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.

Referring to Figure 13, individuals or groups can use AR HoloLens or VR devices individually or in combination to provide severe trauma treatment training. Administrators store and manage AR and VR content data transmitted through the network in the database.

Figure 14 is a configuration diagram showing an example of a virtual reality data integrated management system in the XR-based severe trauma treatment training system according to the present invention.

Referring to Figure 14,

User devices such as MR devices, VR devices, haptic devices, and practice mock-ups (mannequins) represent devices (500-1 to 500-3) that users can directly use and view content, and are devices that transmit data occurring in content. It is a device.

AV/VR integrated management system (1300)

An integrated management web (1330) including an integrated dashboard, MR training monitoring WEB UI, MR training management WEB UI, MR scenario evaluation WEB UI, and operation management (DevOps) (module) is provided, and is provided for managers, training professors, The web used by trainers is given menus and functions according to each authority.

API GW and OPEN API (module) represent the linking API between the integrated management web (1330), the integrated data collection server (1310), and the integrated data operation management server (1320).

In other words, the integrated data collection server 1310 includes a data collection system (API, MQ, Adapter, AAA) that collects data, a scenario management system (CMS, MR Contents, Ctrl, Ctrl) that manages scenarios, and a scenario evaluation system. It serves as a server for content data, user data, and content operation result data, including a scenario evaluation system (Analysis, Tracking, Test, Monitoring).

The integrated data operation management server 1320 represents a data server for operation, acts as an operation management manager, and performs alarms, failure management, configuration management, and statistics management.

There is also a cloud-type XR server and a cloud/deployment-type infrastructure as a stand-alone XR server.

In addition, the data storage is equipped with a database including clinical data DB, photography/auscultation/palpation data DB, medical twin DB, MR content DB, manikin DB, and trainer DB.

Docker Registry for services is distributed to the integrated data collection server (1310).

Reference numeral 12 is an administrator server equipped with a system control PC and is connected to the AR/VR integrated management system 1300 through a network to control the overall system.

Figure 15 is a diagram showing an example of a linked service data flow chart for integrated management of virtual reality data in the XR-based severe trauma treatment training system of Figure 14.

Referring to FIG. 15, the integrated data collection server (13010) receives the data uploaded from the user device information from the scenario control server, determines whether it fits the scenario, and transmits the resulting value to the scenario evaluation server through open API and user UI (web) , shows the flow chart displayed for monitoring.

That is, the scenario control server 1200

Perform network communication with the integrated data collection server (1301) of the AR/VR integrated management system,

Receives various sensor device information (TCP/UDP) transmitted through HW devices (500-1~500-3), that is, various equipment linkages, AR devices, and VR devices, transmits device sensor information, and collects and secures user information. can do.

Through API G/W and OPEN API

The integrated data collection server, including the data collection server, scenario management server (or training control server), and scenario evaluation server, can transmit scenario numerical data/result data to the integrated data web server and receive scenario sensor information. there is. In other words, data is received from the training control server and transmitted to the evaluation server, and the evaluation server makes a decision (time limit and whether or not to judge performance). (Please review *******).

Data acquired during scenario execution is stored and managed in a data storage.

Figure 16 is a flowchart schematically showing an example of the virtual reality data integrated management process in the XR-based severe trauma treatment training system according to the present invention.

Referring to Figure 16,

From the administrator's perspective, the system login process (S2),

AR/VR content creation process (S4),

Basic information input process (content title, date and time) (S6),

Content classification selection process (AR/VR classification) (S8),

User preparation confirmation process (S10),

Content starting process (S12),

Monitoring process (S14),

Includes a content termination process (S16),

From the user (member) perspective

Wear AR/VR (S20),

Identity verification (S22) (The user information registered with the content being prepared and the identity verification check process are performed. At this time, the administrator performs user preparation verification (S10)),

Wait for content (S24),

Content start (S26) (AR/VR content starts, and the administrator also checks (S12)),

Content progress (S28) (Content progress information is displayed and monitored by the administrator (S14),

The content termination process (S30) proceeds.

Figure 17 is a diagram showing an example of an interface screen of the real-time monitoring status of education and training in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.

Referring to Figure 17, this is an example of an interface screen where you can view the real-time status of education and training, and the real-time training status is displayed in detail. That is, SCENE (check uncontrolled external bleeding), mission (check severe uncontrolled external bleeding, photo of bleeding area), performance/limit time (20 seconds/30 seconds). Name, result, and score items are displayed. . As training information, training name, location, professor in charge, training method, trainees, and training/evaluation items are displayed, and buttons for selecting training, pausing training, and ending training are displayed. Patient monitoring information can be displayed as a graph at the bottom (heart rate, oxygen saturation, etc.)

Figure 18 is a diagram showing an example of an interface screen of an education and training evaluation table in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.

Referring to Figure 18, SCENE1 (oxygen administration), category, evaluation content, performance/limit time, name, points allocated, and result (success) are displayed.

SCENE2 (infusion solution connection), category, evaluation details, performance/limit time, name, points allocated, and result (success) are displayed.

Evaluation findings are displayed.

Close and Save buttons are displayed.

Figure 19 is a diagram showing an example of an education and training management interface screen in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.

Referring to Figure 19, an example of an education and training management interface screen is shown, which may indicate that education is in progress and basic information about the content being trained is displayed.

Trainee information can be viewed through the user login process.

You can also view information about the VR device you are using.

You can see that the training content is also classified through the training settings.

Figure 20 is a diagram showing an example of an interface screen at the end of training in the virtual reality data integrated management method in the XR-based severe trauma treatment training system according to the present invention.

Referring to FIG. 20, it is a screen similar to FIG. 19, which indicates that the training has been completed, and as basic information, the following items are displayed: category, location, training/evaluation, name of training, date and time, and evaluation method.

Trainee information is also displayed, and if you click on the trainee, a message is displayed indicating that the evaluation table can be checked.

Device and HoloLens are displayed in the device information section.

Classification, population characteristics, and training type may be displayed in the training settings items.

As described above, the optimal embodiment has been disclosed in the drawings and specifications. Although specific terms are used here, they are used only for the purpose of describing the present invention and are not used to limit the meaning or scope of the present invention described in the claims. Therefore, those skilled in the art will understand that various modifications and other equivalent embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the attached claims.

100: Severe trauma treatment platform
112: Dark ventilator module
114: database
116: Collected data/video/file
118:API GW
110: Severe trauma data
120: Medical twin production system
140: Severe trauma treatment training lecture hall
150: Severe trauma treatment training center
200: Severe trauma treatment training/evaluation UI/UX
300: Training scenario information processing means
400: Mannequin patient biometric information processing means
500: Training device information processing means
500-1,500-2,500-3: Training device
600: Training evaluation information processing department
700: Open platform
800: Gateway (training system)
100-1: Treatment training platform management system
1000: Doctor Helicopter
150-1: Severe trauma treatment training area
10-1: Training evaluation server
20-1: Training control server
30-1: DB server
40-1: Operation management server
50-1: Database
1510-1: Training control server
2000: Evaluation server
2100: database
1500-1~1500-N: Trauma treatment training center at points A~N
1510-1~1510-N: Trauma treatment training center for A~N customers

Claims (12)

Training scenario information processing means that performs trainee information registration and management, training scenario registration and modification, and training evaluation standard information management functions, and a mannequin that performs the function of processing electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, and blood pressure information. Patient biometric information processing means, pulse band, haptic glove, HoloLens, stethoscope, the training device information processing means that performs the function of processing CPR information, the training scenario information processing means, mannequin patient biometric information processing means, and training device. A severe trauma processing training platform that collects processing results from information processing means and performs the function of analyzing training data, and is connected to the severe trauma treatment training platform to manage, store, and provide data received through a communication network from a remote location. Cloud infrastructure that performs, operates in connection with the severe trauma treatment training platform 100, and includes a training information registration UI, real-time training monitoring UI, and training evaluation management UI (User Interface). Severe trauma treatment training/evaluation UI/UX , a training evaluation information processing unit that is connected to the severe trauma treatment training platform and manages, stores, and provides personal evaluation information, teamwork-specific evaluation information, and comprehensive training evaluation information; a military hospital information sharing unit that is connected to the severe trauma treatment training platform; It includes an open platform that provides training education material sharing and data security management functions, and receives training scenarios and trainee information from the training scenario information processing means and monitors the training (mannequin information, trainee motion tracking information) information. XR-based for severe trauma treatment training of medical workers by providing XR (eXtended Reality)-based content, which is characterized by providing a severe trauma treatment training platform that provides comprehensive training evaluation and the ability to analyze evaluation information by linking. Severe trauma treatment training system. According to claim 1,
The XR-based severe trauma treatment training system includes a severe trauma treatment training platform, and the severe trauma treatment training platform is
Treatment training platform management system,
A doctor helicopter that is connected to the treatment training platform management system and a wireless communication network to transmit and receive patient monitoring data, and
It includes a severe trauma treatment training center that is connected to the treatment training platform management system and a wireless communication network to transmit and receive patient monitoring data,
The treatment training platform management system is
Training Evaluation Server (TES) that evaluates training, Training Control Server (TCS: Training Control Server) that controls training, DB Server (DBS: DB Sever) that stores and manages data transmitted and received during system operation, Includes an operation & management server (O&M) that controls the system operation and management process,
The training control server includes a training control S/W module (TCM: Training Control S/W Module) that controls training, and a Farcom Mannequin Tracking S/W module (MTM: Mannequin Tracking S/W) that tracks the position of the mannequin and the position of the trainee. Module), test monitoring S/W module that monitors the training test process (TESm:TES Monitoring S/W Module), TCS monitoring S/W module (TCSm:TCS Monitoring S/W Module) that controls the training monitoring process, training It includes a DBS monitoring S/W module (DBSm: DBS Monitoring S/W Module) that stores and manages the data monitoring process, and a storage monitoring S/W module (STRm: Storage Monitoring S/W Module) that stores training data. Characterized by XR-based severe trauma treatment training system. According to claim 1,
Doctor helicopter
One or more HoloLens that captures and transmits images through the mannequin,
A wireless AP device that wirelessly processes images transmitted from the one or more HoloLens and transmits them wirelessly through a treatment training platform management system;
A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,
XR-based severe trauma, comprising a patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits patient data related to the mannequin. Severe trauma treatment training platform of the treatment training system. According to claim 1,
The severe trauma treatment training center is
A marketing tracking device that measures trainee location or mannequin location and posture values and transmits them through a wireless AP device,
One or more HoloLens that captures and transmits images through the mannequin,
A wireless AP device that wirelessly processes images transmitted from the one or more HoloLens and transmits them wirelessly through a treatment training platform management system;
A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,
A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits mannequin-related patient data,
XR-based, characterized in that it includes a VR device that provides VR data for training in treatment of critically ill patients using mannequins at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device. Severe trauma treatment training platform of the severe trauma treatment training system. The method of claim 1, wherein the training-related information log collection function is performed at the training site, and the mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure) is collected and analyzed from the mannequin patient biometric information processing means. perform,
A severe trauma treatment training platform of the XR-based severe trauma treatment training system, characterized in that the trainee motion tracking information collection and analysis process is performed through the information collected from the training device information processing means. According to claim 1,
The XR-based severe trauma treatment training system for providing severe trauma treatment training to medical workers by providing XR (eXtended Reality)-based content includes a severe trauma treatment training cloud platform,
The severe trauma treatment training cloud platform is
Treatment training platform management system from A to N points,
It includes an evaluation server that is connected to the treatment training platform management system at points A to N through the Internet and stores and evaluates patient monitoring data transmitted through the treatment training platform management system,
The severe trauma treatment training site at point A above is
One or more HoloLens that captures and transmits images through the mannequin,
A wireless AP device that wirelessly processes and transmits images transmitted from the one or more HoloLens,
A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,
A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits mannequin-related patient data,
A VR device that provides VR data for training in treating critically ill patients using a mannequin at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device, and
It is connected to the HoloLens, smart terminal, patient monitor control tablet, and VR device through a wired and wireless communication network, and is connected to the evaluation server through the Internet to transmit and receive data. Patient monitoring data through the mannequin is measured and transmitted through the Internet network. An XR-based severe trauma treatment training system characterized by including a training control server. According to claim 6, the function of collecting training-related information logs at the training site is performed, and the mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure) is collected and analyzed from the mannequin patient biometric information processing means. perform,
Severe trauma treatment training platform cloud of the XR-based severe trauma treatment training system, characterized in that it collects and analyzes trainee motion tracking information through the information collected from the training device information processing means. According to claim 1,
The XR-based severe trauma treatment training system for providing severe trauma treatment training to medical workers by providing XR (eXtended Reality)-based content includes a severe trauma treatment training cloud platform,
The severe trauma treatment training platform cloud is
A~N customer treatment training platform management system,
It includes an evaluation server that is connected to the treatment training platform management system of the customers A to N through the Internet and stores and evaluates patient monitoring data transmitted through the treatment training platform management system,
The evaluation server includes a training control server that controls severe trauma treatment training by receiving patient monitoring data through a mannequin through an Internet network through a HoloLens, a smart terminal, a patient monitor control tablet, a VR device, and a wired or wireless communication network,
The severe trauma treatment training center for customer A above is
One or more HoloLens that captures and transmits images through the mannequin,
A wireless AP device that wirelessly processes and transmits images transmitted from the one or more HoloLens,
A smart terminal equipped with a stethoscope/pulse band that diagnoses the pulse from the mannequin and measures and transmits mannequin diagnosis data,
A patient monitor control tablet that stores and transmits data received through wired or wireless communication with the smart terminal, and monitors and stores or transmits patient data related to the mannequin, and
XR-based, characterized in that it includes a VR device that provides VR data for training in treatment of critically ill patients using mannequins at a training site, collects response data signals for the VR data, and transmits them through the wireless AP device. Severe trauma treatment training system. According to claim 8, the function of collecting training-related information logs at the training site is performed, and the mannequin patient biometric information (electrocardiogram, heart rate, oxygen saturation, respiratory rate per minute, body temperature, blood pressure) is collected and analyzed from the mannequin patient biometric information processing means. perform,
XR-based severe trauma treatment training system, characterized in that the trainee motion tracking information collection and analysis process is performed through the information collected from the training device information processing means. These are user devices such as MR devices, VR devices, haptic devices, and practice mock-ups (mannequins), which are devices (500-1 to 500-3) that allow users to directly use and view content, and devices that transmit data generated within the content. ,
The AV/VR integrated management system is 1300, which is connected to the user device through a wired or wireless communication network to transmit and receive data and collect and manage data for severe trauma treatment training.
An integrated management web (1330) including an integrated dashboard, MR training monitoring WEB UI, MR training management WEB UI, MR scenario evaluation WEB UI, and operation management (DevOps) (module) is provided, and is provided for managers, training professors, The web used by the trainer is given menus and functions according to each authority,
API GW and OPEN API (module) represent the linking API between the integrated management web (1330), the integrated data collection server (1310), and the integrated data operation management server (1320),
The sum data collection server 1310 includes a data collection system (API, MQ, Adapter, AAA) that collects data, a scenario management system (CMS, MR Contents, Ctrl, Ctrl) that manages scenarios, and a scenario evaluation that evaluates scenarios. It serves as a server for content data including system (Analysis, Tracking, Test, Monitoring), user data, and content operation result data.
The integrated data operation management server 1320 represents a data server for operation, acts as an operation management manager, and performs alarms, failure management, configuration management, and statistics management.
There is also a cloud-type XR server and a cloud/deployment-type infrastructure as a stand-alone XR server.
The data storage includes databases including clinical data DB, photography/auscultation/palpation data DB, medical twin DB, MR content DB, manikin DB, and trainer DB.
Docker Registry for services is distributed to the integrated data collection server (1310),
In the XR-based severe trauma treatment training system, the administrator server is equipped with a system control PC and is connected to the AR/VR integrated management system (1300) through a network to control the overall system. Virtual reality data integrated management system. According to claim 10,
The integrated data collection server (13010) receives the data uploaded from the user device information from the scenario control server, determines whether it fits the scenario, transmits the resulting value to the scenario evaluation server through open API, and displays it on the user UI (web) and monitoring,
The scenario control server (1200) is
Network communication is performed with the integrated data collection server (1301) of the AR/VR integrated management system, and various sensor equipment transmitted through HW devices (500-1 to 500-3), that is, various equipment linkages, AR devices, and VR devices Collects information (TCP/UDP), transmits device sensor information, collects and secures user information,
Through API G/W and OPEN API
The integrated data collection server, including the data collection server, scenario management server (or training control server), and scenario evaluation server, transmits scenario numerical data/result data to the integrated data web server, receives scenario sensor information, and trains The data is received from the control server and transmitted to the evaluation server, and the evaluation server determines the time limit for severe trauma treatment training and whether or not it has been performed.
A virtual reality data integrated management system in the XR-based severe trauma treatment training system, where data acquired during scenario performance is stored and managed in a data storage. In the virtual reality data integrated management method of the virtual reality data integrated management system in the XR-based severe trauma treatment training system,
From the administrator's perspective, system login process (S2), AR/VR content creation process (S4), basic information input process (content title, date and time) (S6), content classification selection process (AR/VR classification) (S8), user Performs the preparation confirmation process (S10), content start process (S12), monitoring process (S14), and content end process (S16).
From the user (member) side, AR/VR wearing (S20), identity verification (S22) (linking user information registered with content being prepared and identity verification check process) are performed, and at this time, the administrator performs user preparation verification (S10). content standby (S24), content start (S26) (AR/VR content starts, and the administrator also checks it (S12)), content progress (S28) (content progress information is displayed, and the content is monitored by the administrator) (S14) A virtual reality data integrated management method in the XR-based severe trauma treatment training system, which is characterized by performing and monitoring severe trauma treatment training.