KR20220043562A - System for providing treatment and skill simulation using virtual reality - Google Patents

Abstract

The present invention relates to a system for providing medical treatment and surgical technique simulation using virtual reality (VR). According to the present invention, the system comprises: a VR device including a VR head mounted display (HMD) rendering data received from a participant's terminal into a 3D model in real-time to conduct an interview with the patient in a virtual space and perform a surgical technique by controlling a virtual tool and a hand controller controlling a virtual object or tracks the participant's hand motion; and a control device using role information received from the participant terminal to match a participant in the role of a patient and a participant in the role of a medical worker, and outputting a virtual patient character according to a situation scenario on the VR HMD of a first participant who plays the role of the medical worker and outputting a virtual medical worker character and outputting a list to an evaluation index on the VR HMD of the second participant who plays the role of the patient when the matching is completed, thereby performing a diagnosis or technique simulation. Accordingly, a virtual patient character according to a patient's age, gender, weight, race, and body type is generated and voice is modulated in real-time to match the generated virtual character, thereby maximizing realism of a virtual reality situation. An evaluation result is provided after performing the treatment and surgical technique, thereby increasing learning effect.

Description

System for providing treatment and skill simulation using virtual reality}

The present invention relates to a system for providing medical treatment and technique simulation using virtual reality, and more particularly, to perform treatment and technique simulation using virtual reality, and to evaluate and feedback on patients and medical personnel who perform treatment and technique simulation It relates to a system for providing medical treatment and skill simulation.

College students who want to become medical personnel (doctors, nurses, paramedics, etc.) must acquire an appropriate level of medical technology and patient interviewing skills in the course of learning or obtaining a license.

In the past, educators and students shared roles with each other and learned about medical technology and patient interviewing technology. However, in the conventional method, the sense of reality is very low in the appearance of the opponent irrelevant to the clinical situation, and it is difficult to seriously perform the role due to the formation of a familiar relationship. In addition, the conventional learning method has problems in that it is not easy to prepare for various clinical situations, consumables are continuously used, and above all, it is difficult for several people to learn at the same time.

In order to solve this problem, various kinds of clinical technique simulations using virtual reality have recently been developed and are receiving a lot of attention. Clinical skill simulation using virtual reality can be learned regardless of time and place, and various clinical situations can be implemented.

However, clinical skill simulation does not provide evaluation and feedback according to the performance of medical technology and patient interview technology, so it is merely at the level of experiential activity. In particular, it is more difficult to train and evaluate the interview or situational judgment part where natural language processing is essential.

The technology that is the background of the present invention is disclosed in Korean Patent Registration No. 10-1725682 (published on April 5, 2017).

As described above, according to the present invention, an object of the present invention is to provide a treatment and skill simulation providing system that performs treatment and skill simulation using virtual reality and provides evaluation and feedback to patients and medical personnel who perform treatment and skill simulation.

According to an embodiment of the present invention for achieving this technical task, in the system for providing medical treatment and skill simulation using virtual reality, the first and the Virtual reality (VR) including a virtual reality HMD (Head Mount Display) that renders the data received from the second participant terminal in real-time as a 3D model, and a hand controller that controls virtual objects or tracks participants' hand movements The device and the role information received from the first and second participant terminals are used to match the first participant performing the role of a medical practitioner with the second participant performing the role of the patient, and when the matching is completed, the first participant is worn A control that outputs a virtual patient character generated according to a situation scenario to the virtual reality HMD and outputs a list to the virtual medical person character and evaluation index to the virtual reality HMD worn by the second participant to perform diagnosis or skill simulation includes the device.

The control device includes an interface unit for receiving request information for a role of a medical practitioner or a role of a patient from the plurality of participants, and when receiving a matching request signal from the plurality of participants, the controller is configured to use the role information to serve as a medical practitioner A matching unit that matches the first participant with a second participant who will take on the role of a patient, a mode selector that receives a selection of a treatment mode or a technique mode from the first and second participants, and provides a scenario of a specific situation in response to the selected mode A scenario providing unit that generates a virtual patient character using patient information corresponding to the scenario of the specific situation and provides it to the virtual reality HMD worn by the first participant, and creates a virtual medical person character to be worn by the second participant A virtual character providing unit that provides a virtual reality HMD, and the voice data of the first and second participants received from the virtual reality HMD worn by the first or second participant according to the virtual medical person character or patient character It may include a voice conversion unit provided by modulation.

The control device provides an evaluation index including an automatic scoring index according to the presence or absence of a specific physical examination and a specific clinical technique and a manual scoring index according to the attitude and observation of the medical personnel toward the patient, and the evaluation input by the second participant An evaluation information acquisition unit that acquires results, a point provision unit that pays points to perform treatment and skill simulation, adds or subtracts points according to the selected role, and the tracking information received from the VR device and the generated A control unit that provides virtual reality by matching virtual characters, and provides virtual objects for physical examination according to the clinical technique mode request signal received from the terminal of the first participant who plays the role of a medical person during the simulation process may further include.

The scenario providing unit may provide at least one of basic information about the patient, physical examination information, and medical interview information to the second participant terminal.

The basic information about the patient includes at least one of information on name, age, gender, race, skin, occupation, weight and body type, and the physical examination information is a state that can be observed with the eyes from the face, limbs, and torso information and at least one of urine, defecation, and pain, and the medical interview information may include at least one information among symptom patterns, family history, power, drinking history, medication status, working conditions, and eating habits. can

The point providing unit may deduct some of the points held by the first participant when selecting the role of a medical professional, and additionally pay points to the second participant when selecting the role of the patient.

When the first participant and the second participant select a treatment mode, the control unit continuously transmits a tracking control signal to the hand controller of the first participant, and stops tracking to the hand controller of the second participant. and transmits a control signal to perform the role of a mouse, and a control signal for tracking tracking only when expressing the patient's behavior according to the request of the first participant.

When the hand controller of the second participant performs the role of a mouse, the control unit recognizes the movement of the hand controller as text, and the recognized text may be provided as evaluation information for the first participant who plays the role of a medical practitioner. there is.

When the first participant and the second participant select a technique mode, the controller may provide a tool necessary for a clinical technique in virtual reality.

The evaluation information obtaining unit may determine whether the tools are used in a predetermined order as the technique is performed, and may automatically score and score the determined results.

As described above, according to the present invention, a virtual patient character is created according to the patient's age, gender, weight, race and body type, and the voice is modulated in real time to match the created virtual character to maximize the sense of reality of the virtual reality situation. It is possible to increase the learning effect by providing evaluation results after performing treatment and techniques.

In addition, points are paid to participants participating in the simulation, but points are consumed when playing the role of a medical professional, and points are accumulated when playing the role of a patient. can increase the matching rate.

In addition, according to the present invention, a rating is given according to the role performance of medical personnel and patients, and when the rating is low, a disadvantage is provided to induce active participation in the role, and problems such as discrimination and harassment that may occur during learning Recording is performed in preparation for undesirable behavior to prevent undesirable behavior from occurring in advance.

In addition, according to the present invention, since medical treatment and technique simulation can be performed online, medical technique and patient interview technique can be acquired by performing dynamics of a medical person or patient regardless of time and place.

1 is a configuration diagram for explaining a system for providing medical treatment and skill simulation according to an embodiment of the present invention.
FIG. 2 is a configuration diagram for explaining the control device shown in FIG. 1 .
3 is a flowchart for explaining a method of learning treatment and skill using a simulation of treatment and skill according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a login screen for performing a medical treatment and a technique simulation.
5 is an exemplary diagram illustrating a screen for selecting a role by a participant in step S310 shown in FIG. 3 .
FIG. 6 is a flowchart for explaining step S350 shown in FIG. 3 .
7 is an exemplary diagram illustrating a screen output through the HMD of a participant performing a patient role in step S351 shown in FIG. 6 .
FIG. 8 is an exemplary diagram illustrating a screen output through the HMD of a participant performing the role of a medical person in step S351 shown in FIG. 6 .

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings. In this process, the thickness of the lines or the size of the components shown in the drawings may be exaggerated for clarity and convenience of explanation.

In addition, the terms to be described later are terms defined in consideration of functions in the present invention, which may vary according to intentions or customs of users and operators. Therefore, definitions of these terms should be made based on the content throughout this specification.

Hereinafter, a system for providing medical treatment and technique simulation according to an embodiment of the present invention will be described in more detail with reference to FIG. 1 .

1 is a configuration diagram for explaining a system for providing medical treatment and skill simulation according to an embodiment of the present invention.

As shown in FIG. 1 , the system for providing medical treatment and technique simulation according to an embodiment of the present invention includes first and second virtual reality (VR) devices 110 and 120 , and first and second participant terminals 210 and 220 . ) and a control device 300 .

According to an embodiment of the present invention, for convenience of explanation, a participant who will take the role of a medical professional is described as a first participant, and a participant who will take the role of a patient is described as a second participant. Therefore, in the embodiment of the present invention, a virtual reality (VR) device mounted to a first participant among a plurality of virtual reality (VR) devices 110 and 120 is called a first virtual reality (VR) device 110 , 2 A virtual reality (VR) device mounted to the participant is called a second virtual reality (VR) device 120 .

In addition, the virtual reality HMD constituting the first virtual reality (VR) device 110 is the first virtual reality HMD 111 , and the hand controller is called a first hand controller 112 . Similarly, the virtual reality HMD constituting the second virtual reality (VR) device 120 is the second virtual reality HMD 121 , and the hand controller is called a second hand controller 122 .

Next, the participant terminals 210 and 220 are terminals possessed by the first and second participants to perform medical treatment and technique simulation, and include a first virtual reality (VR) device 110 and a second virtual reality (VR) device. The 120 and each perform communication through a wireless or wired.

That is, the participant terminal that communicates with the first virtual reality (VR) device 110 is called the first participant terminal 210 , and the participant terminal that communicates with the second virtual reality (VR) device 120 is It is called the second participant terminal 220 .

First, the first and second virtual reality (VR) devices 110 and 120 conduct an interview with a patient in a virtual space and control the virtual tool to perform a technique by first and second participant terminals 210 and 220 . It is a wearable device that renders the data received from the 3D model in real time.

The first and second virtual reality (VR) devices 110 and 120 are virtual reality head mount displays (HMDs) 111 and 121 that output images and hand controllers that control virtual objects or track the hand motions of participants. (112,122).

The first and second virtual reality HMDs 111 and 121 output images received from the first and second participant terminals 210 and 220, which will be described later, through short-range communication such as Bluetooth or Zigbee through a display. In other words, the first and second virtual reality HMDs 111 and 121 are display devices worn on the participant's head, and show the virtual world from the participant's perspective while being blocked from the outside. The first and second virtual reality HMDs 111 and 121 include a microphone, a speaker, and a plurality of sensors. The plurality of sensors include a visual sensor, a magnetic sensor, and the like.

Accordingly, the first and second virtual reality HMDs 111 and 121 continuously track the positions of the participants' heads and eyes, and the location information generated by tracking is transmitted to the participant terminals 210 and 220 . Then, the participant terminals 210 and 220 match the tracked movement of the participant's head and torso to the output image and provide the first and second virtual reality HMDs 111 and 121 .

The first and second hand controllers 112 and 122 are worn on the participant's hand, and track the movement of the hand using a sensor included therein. The hand controllers 112 and 122 perform a role of a mouse or track a participant's hand position to perform screen control or information search. In addition, the tracked participant's hand motion is matched with the virtual character's hand and output through the first and second virtual reality HMDs 111 and 121 .

The first participant terminal 210 accesses the control device 300 and executes a medical treatment and technique simulation, but receives information necessary to perform the role of a medical person. Then, the first participant terminal 210 transmits the information received from the control device 300 to the first virtual reality (VR) device 110 . Then, the first participant performs treatment and surgery as a medical person in the three-dimensionally configured screen by applying the virtual reality technique through the first virtual reality (VR) device 110 .

In addition, the second participant terminal 220 accesses the control device 300 to execute a medical treatment and technique simulation, but receives information necessary to perform a patient role. Then, the second participant terminal 220 transmits the information received from the control device 300 to the second virtual reality (VR) device 120 . Then, the first participant performs treatment and surgery as a patient in the three-dimensionally configured screen by applying the virtual reality technique through the second virtual reality (VR) device 120 .

On the other hand, the first participant terminal 210 and the second participant terminal 220 may be various known means capable of accessing the network through a communication network, and may include a smart phone, a smart pad, various mobile devices, a notebook computer, and the like. there is.

Finally, the control device 300 matches the participant in the patient role with the participant in the medical care role by using the role information received from the first participant terminal 210 and the second participant terminal 220 , respectively. When the matching is completed, the control device 300 transmits information about the situation scenario and the virtual patient character corresponding thereto to the first participant terminal 210 serving as a medical person, and the second participant terminal serving as a patient In 220, a list of virtual patient characters and evaluation indicators is transmitted to perform diagnosis or skill simulation.

Hereinafter, the control device 300 will be described in more detail with reference to FIG. 2 .

FIG. 2 is a configuration diagram for explaining the control device shown in FIG. 1 .

As shown in FIG. 2 , the control device 300 according to the embodiment of the present invention includes an interface unit 310 , a matching unit 320 , a mode selection unit 330 , a scenario providing unit 340 , and a virtual character system. It includes a study 350 , a voice conversion unit 360 , an evaluation information obtaining unit 370 , a point providing unit 380 , and a control unit 390 .

First, the interface unit 310 receives information about a role selected from a plurality of participants.

The medical treatment and technique simulation providing system according to an embodiment of the present invention allows a plurality of participants to select one role from among a medical staff role and a patient role.

The interface unit 310 receives information on the role selected by the participant from the participant terminals 210 and 220 .

Next, the matching unit 320 uses the received role information to classify the participant who will take the medical care role and the patient who will take the patient role, and mutually associates the classified participant who will take the medical care role and the patient role. match

As described above, for convenience of explanation, a participant who selects the role of a medical professional from among a plurality of participants is described as a first participant, and a participant who selects the role of a patient is described as a second participant.

Upon receiving a matching request signal from the first terminal 210 used by the first participant or the second terminal 220 used by the second participant, the matching unit 320 sequentially performs the matching request signal according to the received matching request signal. The first participant and the second participant are matched with each other.

Then, the mode selection unit 330 receives information on the mode selected by the first participant or the second participant from among the medical treatment mode and the technique mode. The medical treatment mode is a mode for acquiring the skills of observing and interviewing the patient like a simple examination, and the skill mode is a mode for acquiring medical skills such as auscultation, palpation, and percussion.

When any one of the progress mode and the skill mode is selected by the first participant or the second participant, the scenario providing unit 340 displays a situation scenario generated according to a specific situation in response to the selected mode to the first participant terminal 210 ) and the second participant terminal 220, respectively.

The situation scenario includes information necessary for treatment, and includes basic information, physical examination information, and medical interview information.

The virtual character providing unit 350 provides a character matching the situation scenario. In this case, the virtual character providing unit 350 generates and provides a patient character and a medical person character. The patient character is created using basic patient information included in the situation scenario, and the medical person character provides a character in the shape of a general medical person or creates a character reflecting the request of the first participant in charge of the medical personnel role.

The voice conversion unit 360 modulates and provides the voice data of the first participant received from the first virtual reality HMD 111 according to the generated virtual medical person character. In addition, the voice conversion unit 360 modulates and provides the voice data of the second participant received from the second virtual reality HMD 121 according to the generated virtual patient character. That is, the voice converter provides a voice modulated according to the gender and age of the generated medical person character or patient character.

The evaluation information acquisition unit 370 provides an evaluation index including an automatic scoring index according to the presence or absence of a specific physical examination and a specific clinical technique, and a manual scoring index according to the attitude and observation of the medical personnel toward the patient. Then, the evaluation information acquisition unit 370 acquires the evaluation result written by the first participant or the second participant according to the evaluation index.

The point providing unit 380 provides points to perform treatment and skill simulation, and adds or subtracts points according to the selected role. In other words, a plurality of participants perform treatment and technique simulation using points. At this time, when the participant selects the role of a medical professional, the point providing unit 380 deducts some of the points paid to the participant. On the other hand, when the participant selects a patient role, the point providing unit 380 additionally provides points to the participant. Also, the point providing unit 380 deducts or adds or subtracts points according to the obtained evaluation result.

The system for providing medical treatment and skill simulation according to an embodiment of the present invention prevents the participants from being focused on only one of the roles of medical personnel and patients by adding or subtracting points, so that the first and second participants It has the effect of increasing the matching rate and inducing active participation.

Finally, the controller 390 controls the tracking information received from the plurality of sensing modules included in the first virtual reality (VR) device 110 and the second virtual reality (VR) device 120 and the generated virtual character. It provides virtual reality by matching. In addition, the control unit 390 virtually provides an object for execution capable of examining a patient's body. Executable objects are provided when in skill mode. That is, upon receiving the clinical technique mode request signal from the first participant terminal 210 , the controller 390 virtually provides an execution object capable of precise examination, auscultation, percussion, and palpation. Here, the execution object includes a stethoscope, a magnifying glass, and a percussion instrument.

Also, the controller 390 may set the patient's behavior according to a specific situation scenario, and implement the set patient's behavior like an animation. In other words, since the second hand controller 122 worn on the second participant does not track the hand movement of the second participant, the patient displayed through the first virtual reality HMD 111 performs only the same operation. Accordingly, the controller 390 presets the patient's behavior according to a specific situation scenario, and outputs the preset behavior through the first virtual reality HMD 111 . For example, the virtual patient may suddenly become angry or cry while receiving treatment, or walk forward at the doctor's request.

Hereinafter, a method of performing treatment and skill learning using the treatment and skill simulation according to an embodiment of the present invention will be described in more detail with reference to FIGS. 3 to 8 .

3 is a flowchart for explaining a method for learning treatment and skill using the treatment and skill simulation according to an embodiment of the present invention, and FIG. 4 is an exemplary view showing a login screen for performing treatment and skill simulation; 5 is an exemplary diagram illustrating a screen for selecting a role by a participant in step S310 shown in FIG. 3 .

First, a plurality of participants who wish to participate in the treatment and technique simulation according to an embodiment of the present invention access the control device 300 using the terminals 210 and 220 possessed. Then, the plurality of participants perform membership registration by inputting personal information through the terminals 210 and 220 . In this case, the personal information includes at least one of the participant's name, ID, and password.

When membership registration is completed, a plurality of participants receive a certain amount of points from the point providing unit 380 .

Next, as shown in FIG. 3 , the participant inputs information on a role to be performed and information on a type to be learned through the terminals 210 and 220 . Then, the participant terminals 210 and 220 transmit the role information and the type information to be learned to the control device 300 (S310).

In other words, as shown in FIG. 4 , the participant accesses the program provided by the control device 300 through the terminals 210 and 220 and logs in. Then, as shown in FIG. 5 , the participant selects one of the roles of the medical personnel and the patient through the terminals 210 and 220 , and selects type information to be learned. Then, the participant transmits a matching application signal through the terminals 210 and 220 .

Then, the matching unit 320 matches the participant who wants to perform the role of a medical person with the participant who wants to perform the role of the patient (S320).

In detail, the matching unit 320 uses the role information received from the participant terminals 210 and 220 to classify a participant who wants to perform the role of a medical professional as a first participant, and sets the participant who wants to perform the role of a patient as a second participant. classified as Then, the matching unit 320 matches the first participant with the second participant in consideration of the type information to be learned.

Here, the type information to be learned includes at least one of clinical expression, disease, and clinical field, and the matching unit 320 uses any one category among the clinical expression, disease, and clinical field categories to match the first participant and the second participant. 2 Matching of participants is performed, or matching is performed at random.

Meanwhile, the treatment and skill simulation according to an embodiment of the present invention may be used to perform practice between students, or may be used to perform a test or batch training.

When performing a practice between students, the matching unit 320 performs matching using information on roles selected by students, ie, participants, and type information to be learned. On the other hand, in the case of performing a test or batch training, the matching unit 320 provides the participants with fixed role and type information to learn, and matches the pre-classified participant to learn and the participant to be evaluated. do.

When step S320 is completed, the scenario providing unit 340 provides a scenario of a specific situation to the second participant terminal 220 used by the participant performing the patient role (S330).

Here, the scenario of a specific situation includes basic information, physical examination information, and medical interview information. First, the basic information includes at least one of the patient's name, age, gender, race, skin color, occupation, weight, and body type.

Further, the physical examination information includes state information observable from the face, limbs, and torso, and at least one of urine, defecation, and pain.

In addition, the medical interview information includes at least one of symptom pattern, family history, power, drinking history, medication status, working conditions, and eating habits.

Next, the virtual character providing unit 350 generates a virtual medical person character and a patient character. Then, the generated medical person character is output through the first virtual reality HMD 111, and the generated patient character is output through the second virtual reality HMD 121 (S340).

In other words, the virtual character providing unit 350 generates a virtual medical person character in the shape of a general medical person and transmits it to the second participant terminal 220 . Then, the second participant terminal 220 outputs the received specific situation scenario and the virtual medical person character through the second virtual reality HMD 121 .

In addition, the virtual character providing unit 350 generates a virtual patient character matching the scenario of a specific situation, and provides the created patient character to the first participant terminal 210 . Then, the first participant terminal 210 outputs the virtual patient character received from the control device 300 through the first virtual reality HMD 111 .

Next, the first participant and the second participant perform medical treatment and technique simulation in virtual reality output through the first virtual reality HMD 111 and the second virtual reality HMD 121 ( S350 ).

Hereinafter, a method of performing treatment and technique simulation using FIGS. 6 to 8 will be described in more detail.

FIG. 6 is a flowchart for explaining step S350 shown in FIG. 3 .

As shown in FIG. 6 , the participant terminals 210 and 220 output the character generated by the virtual character providing unit 350 through the virtual reality HMDs 111 and 121 ( S351 ).

In other words, the second participant terminal 220 outputs the virtual medical person character generated by the virtual character providing unit 350 through the second virtual reality HMD 121 .

7 is an exemplary diagram illustrating a screen output through the HMD of a participant performing a patient role in step S351 shown in FIG. 6 .

As shown in FIG. 7 , the second virtual reality HMD 121 of the second participant acting as a patient outputs a screen divided into a plurality of pieces.

The first screen outputs the environment that appears in front of the participant, that is, the virtual medical person when viewed from the patient's point of view. The second screen outputs scenario information of a specific situation. Accordingly, the participant provides an answer corresponding to the medical person's question according to the output scenario.

The third screen allows scoring according to the evaluation index provided by the evaluation information acquisition unit 370, and the fourth screen allows the participant to record a subjective evaluation of the doctor as he/she proceeds with treatment and surgery. .

In addition, the first participant terminal 210 outputs the virtual patient character generated by the virtual character providing unit 350 through the first virtual reality HMD 111 .

FIG. 8 is an exemplary diagram illustrating a screen output through the first virtual reality HMD of the first participant performing the role of a medical person in step S351 shown in FIG. 6 .

As shown in FIG. 8 , the first virtual reality HMD 111 of the first participant acting as a patient outputs a virtual patient character generated from the point of view of a medical practitioner.

Then, the first participant performs medical treatment on the virtual patient character output through the first virtual reality HMD 111 .

Then, the voice conversion unit 360 receives the voice of the first participant and the voice of the second participant, respectively, generated as a result of the treatment, and modulates the received voice according to the virtual character (S352).

The medical treatment and technique simulation according to an embodiment of the present invention provides a virtual medical person character and a patient character in order to protect the anonymity of the participant. Accordingly, the voice converter 260 modulates the voice uttered by the first participant and outputs it through the second virtual reality HMD 121 worn by the second participant, modulates the voice uttered by the second participant, It is output through the first virtual reality HMD 111 worn by the first participant.

While performing treatment, the second participant inputs evaluation information using the second hand controller 122 . Then, the evaluation information acquisition unit 370 collects the input evaluation information (S353).

In other words, the first hand controller 121 used by the first participant tracks the participant's hand, and a signal generated by tracking is output through the first virtual reality HMD 111 as a hand gesture of a virtual doctor character.

on the other side. The second hand controller 122 used by the second participant serves as a mouse for screen control or information search. In addition, the participant's hand is tracked only when the patient's hand gesture or a specific motion is performed according to the medical practitioner's question or request, and the signal generated by tracking is a virtual patient character's hand gesture through the second virtual reality HMD 121. is output

Accordingly, when performing medical treatment, the control unit 390 recognizes the movement of the second hand controller 122 of the second participant as text, and transmits the recognized result to the evaluation information obtaining unit 370 . Then, the evaluation information acquisition unit 370 collects the recognized text information as evaluation information about the medical personnel.

On the other hand, when performing a close examination, auscultation, percussion, and palpation, the first participant makes a request to proceed with the technique by voice. Then, the control unit 390 provides a virtual execution object so that the skill can be performed according to the skill progress request received through the first participant terminal 210 (S354).

For example, when the first participant utters "I will auscultate", the control unit 390 provides an execution object such as "stethoscope" in response to voice data according to the request for skill progress, and the provided execution object is the first It is output through the first virtual reality HMD 111 worn by the participant.

While the technique is being performed using the execution object provided in step S354, the evaluation information obtaining unit 370 automatically performs scoring according to the operation of the medical practitioner (S355).

In other words, the evaluation information obtaining unit 370 determines whether the technique is performed in a predetermined order as the technique is performed, and scores the determined result. For example, it is assumed that auscultation consists of cardiac auscultation, breathing auscultation, and abdominal palpation. The evaluation information acquisition unit 370 determines whether the heart auscultation, breathing auscultation, and stomach palpation proceed in the order or whether the omitted order of auscultation occurs, and scores the determined result.

When step S355 is completed, the evaluation information obtaining unit 370 transfers the final result obtained by adding the evaluation information obtained in step S353 and the result automatically evaluated in step S355 to the first participant terminal 210, and the first The participant terminal 210 outputs the final result through the first virtual reality HMD 111 (S356).

The first virtual reality HMD 111 may output the received final result as a modulated voice of the patient. At this time, the final result includes information about the evaluation results obtained manually and automatically, whether or not passed, and information about errors.

Meanwhile, the first participant or the second participant may verbally express the evaluation of the other party. In other words, the first participant and the second participant speak as if talking to each other about the evaluation result. For example, the first participant inquires about the parts that need to be supplemented, the wrong part, etc. in performing the role of the medical person, and the second participant answers the questions. When the first participant and the second participant provide evaluation results through conversation, the voices of the first and second participants remain modulated according to the virtual doctor character and the patient character. The output of the final result is Upon completion, the first participant may award additional points to the participant for performing the patient role. That is, the first participant uses the first hand controller 121 or the first participant terminal 210 to input points to be provided to the participant who has performed the patient role. Then, the point providing unit 280 subtracts the points to be provided to the participant who has performed the patient role from among the points granted to the first participant, and the subtracted points are provided to the second participant. As described above, the system for providing medical treatment and skill simulation according to the present invention creates a virtual patient character according to the patient's age, gender, weight, race, and body type, and modulates the voice in real time to match the created virtual character to create a virtual It is possible to maximize the sense of reality of the real situation, and to provide evaluation results after performing treatment and techniques to increase the learning effect.

In addition, the medical treatment and skill simulation providing system according to the present invention pays points to participants participating in the simulation, but consumes points in the role of a medical person and accumulates points in the role of a patient to participate in the patient role By increasing the degree, it is possible to increase the matching rate between the participants in the patient role and the health care provider role. In addition, the treatment and skill simulation providing system according to the present invention gives a rating according to the role performance of medical personnel and patients, and provides a disadvantage when the rating is low to induce active participation in the role, and may Recording is performed in preparation for problems such as , discrimination, and harassment, so that undesirable behavior can be prevented in advance.

In addition, since the medical treatment and technique simulation providing system according to the present invention can perform medical treatment and technique simulation online, medical technology and patient interview technique can be performed by performing the dynamics of a medical person or patient regardless of time and place can be acquired

Although the present invention has been described with reference to the embodiment shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and equivalent other embodiments are possible therefrom. will be. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the following claims.

110: first virtual reality (VR) device
111: first virtual reality HMD
112: first hand controller
120: second virtual reality (VR) device
121: second virtual reality HMD
122: second hand controller
210: first participant terminal
220: second participant terminal
300: control device
310: interface unit
320: matching unit
330: mode selection unit
340: scenario providing unit
350: virtual character providing unit
360: voice conversion unit
370: evaluation information acquisition unit
380: point providing unit
390: control unit

Claims (10)

In a system for providing medical treatment and skill simulation using virtual reality,
A virtual reality HMD (Head Mount Display) and virtual reality HMD (Head Mount Display) that renders the data received from the first and second participant terminals in a 3D model in real time to conduct an interview with the patient in a virtual space and control a virtual tool to perform a technique A virtual reality (VR) device including a hand controller that controls an object or tracks a participant's hand motion, and
Using the role information received from the first and second participant terminals, the first participant performing the medical care role and the second participant performing the patient role are matched, and when the matching is completed, the virtual reality HMD worn by the first participant Outputs a virtual patient character created according to a situation scenario, and a virtual reality HMD worn by a second participant includes a control device that outputs a list of virtual medical personnel characters and evaluation indicators to perform diagnosis or skill simulation. A system for providing medical care and skill simulation. According to claim 1,
The control device is
an interface unit for receiving request information for a role of a medical person or a role of a patient from the plurality of participants;
a matching unit that matches the first participant who will take the role of a medical professional with the second participant who will take the role of a patient by using the role information when receiving a matching request signal from the plurality of participants;
A mode selection unit receiving a selection of a treatment mode or a technique mode from the first and second participants;
A scenario providing unit that provides a scenario of a specific situation in response to the selected mode;
A virtual patient character is created using the patient information corresponding to the scenario of the specific situation and provided as a virtual reality HMD worn by the first participant, and a virtual medical person character is created and converted into a virtual reality HMD worn by the second participant A virtual character providing unit that provides, and
Treatment and technique simulation including a voice conversion unit that modulates the voice data of the first and second participants received from the virtual reality HMD worn by the first participant or the second participant according to the virtual medical person character or patient character and provides it delivery system. 3. The method of claim 2,
The control device is
An evaluation that provides an evaluation index including an automatic scoring index according to the presence or absence of a specific physical examination and a specific clinical technique and a manual scoring index according to the attitude and observation of medical personnel toward the patient, and obtains the evaluation result input by the second participant information acquisition department,
A point providing unit that pays points to perform treatment and skill simulation, and adds or subtracts points according to the selected role; and
It provides virtual reality by matching the tracking information received from the VR device and the generated virtual character, and examines the body according to the clinical technique mode request signal received from the terminal of the first participant who plays the role of a medical practitioner during the simulation process A medical treatment and technique simulation providing system further comprising a control unit that provides a virtual object for execution that can be performed. 3. The method of claim 2,
The scenario providing unit,
A medical treatment and technique simulation providing system for providing at least one of basic information about the patient, physical examination information, and medical interview information to a second participant terminal. 5. The method of claim 4,
Basic information about the patient,
including at least one of information about name, age, gender, race, skin, occupation, weight and body type;
The physical examination information is
It includes at least one information of state information observable from the face, limbs, and torso, urine, defecation, and pain,
The medical interview information is
A medical treatment and skill simulation providing system that includes at least one information from symptom pattern, family history, absorption power, drinking history, medication status, working conditions, and eating habits. 4. The method of claim 3,
The point providing unit,
A system for providing medical treatment and skill simulation that deducts some of the points held by the first participant when selecting the role of a medical person, and additionally pays points to the second participant when the role of the patient is selected. 3. The method of claim 2,
When the first participant and the second participant select a treatment mode,
The control unit is
Transmitting a control signal for continuously tracking tracking to the hand controller of the first participant,
The second participant's hand controller transmits a control signal to stop tracking and perform the role of a mouse, and transmits a control signal to track tracking only when expressing the patient's behavior according to the first participant's request and a skill simulation providing system. 8. The method of claim 7,
When the hand controller of the second participant acts as a mouse,
The control unit is
A system for providing medical treatment and skill simulation that recognizes the movement of the hand controller as text, and the recognized text is provided as evaluation information for a first participant who plays the role of a medical person. 3. The method of claim 2,
When the first participant and the second participant select a skill mode,
The control unit is
A medical treatment and skill simulation providing system that provides tools necessary for clinical skills in virtual reality. 10. The method of claim 9,
The evaluation information acquisition unit,
A system for providing medical care and skill simulation that determines whether the tools are used in a predetermined order as the skill progresses, and automatically scores and scores the judged results.

Images