KR20220098321A - a First aid training type medical system based on extended reality first aid guide information - Google Patents

Abstract

The present invention relates to a first aid man training medical system in accordance with an extended reality type first aid guide information, capable of recognizing each part of a human body, which comprises: a first aid man booth unit (100) including XR treatment equipment (110) and a treatment tool unit (130); an operation control server (200) including a data exchange module (210) and a database management module (240); and a remote medical center booth unit (300) including XR remote equipment (310) and remote treatment device unit (320).

Description

First aid training type medical system based on extended reality first aid guide information based on extended reality first aid guide information

The present invention relates to an emergency medical treatment system based on virtual reality, and more specifically, a marker for bio-object recognition for an eXtended Reality system capable of recognizing each part of the human body, and XR emergency medical care using the same The invention relates to a treatment system.

In general, Augmented Reality is a technology that allows computer graphics (CG) created by computers to coexist with the real world so that computer graphics can feel as if they exist in the real world. Augmented reality, a concept that complements the real world with the virtual world, uses a virtual environment created with computer graphics, but the main character is the real environment. That is, it serves to provide additional information necessary for the real environment by overlapping the 3D virtual image on the actual image (object) viewed by the user.

This augmented reality technology is an extension of virtual reality, and unlike virtual reality technology that consists only of a virtual environment, an image in which the real environment and virtual objects are mixed.

Therefore, the user can see the real environment, providing better realism and additional information. Therefore, augmented reality technology is used for remote medical diagnosis, broadcasting, architectural design, and manufacturing process management. In particular, with the recent widespread use of smartphones, they have entered the stage of full-scale commercialization, and are developing various products in the game and mobile solution industries and education fields.

For example, a guide service in which information such as the location and phone number of a nearby store is displayed in a three-dimensional image when illuminating the surroundings with a smartphone camera has already been operated as a commercial service.

This augmented reality technology is being combined with virtual reality to develop into a mixed reality technology.

In particular, these technologies are being actively applied, utilized, and developed in the field of medical technology. In particular, prompt treatment is very important for patients who need urgent relief measures (first aid) due to an accident. Or, the reality is that the same practitioner or mannequin is repeating the practice without realism.

On the other hand, as an important technical task in the implementation of augmented reality, it is important that an image in which the real environment and the virtual object are accurately matched temporally and spatially should be implemented. Various types of markers that can be used are needed.

In the prior art, such a marker is provided in a manner such as attaching or printing an artificially produced or generated marker to an object in a real environment, or pre-registering a feature point of an object capable of image recognition as a marker.

As a related technology, Registered Patent No. 10-2077607 (Augmented Reality Implementation Method for Patient Treatment Practice, hereinafter referred to as the prior art) is "executed in an augmented reality realization apparatus having a camera and a display, and receiving an image through the camera" and extracting and recognizing a marker included in the input image, synthesizing a virtual image corresponding to the marker with a real object image and outputting an augmented reality image to a display of an augmented reality realizing device; Matching each treatment practice virtual image with each corresponding marker in storing a plurality of virtual images for treatment practice to be used in reality in a database, and applying the treatment practice virtual image to the coordinates of the object to be expressed in the augmented reality implementation device In the augmented reality implementation method comprising the step of setting, in setting the position of the object in the coordinates, the marker output on the display of the augmented reality implementation device and the treatment practice virtual image are expressed to be spaced apart from each other at a predetermined interval during treatment practice. It is characterized in that the marker is prevented from being covered by the practitioner, and the type of wound or injury and the part of the body expressed are described in the marker so that learners can easily know what kind of education and treatment practice is being performed. Augmented reality implementation method for patient treatment practice" has been presented.

In the case of prior art and prior art, a marker used as an object recognition means in conventional virtual reality, augmented reality, mixed reality, or eXtended reality, etc. The present invention Difficulties arise in recognizing biometric objects in various situations, such as bleeding, shock (deformation of the face), cutting body parts, and wearing clothes in an emergency patient that needs to be recognized in the hospital.

In particular, although most of the conventional recognition fields use the object recognition method using feature points, even if the marker-based recognition technology is applied by the factors of the patient's body part and surrounding environment (military uniform pattern, patient-specific body type, etc.), it is difficult to find the feature point. It is a very difficult reality to accurately recognize the injured area of an emergency patient.

In addition, even if information on the body features (arms, legs, etc.) of the emergency patient is provided, the biometric object recognition module of the computer operation control server makes it difficult to guess the relevant part, as well as bleeding, shock (facial deformation), and body parts. In the case of emergency patients in various situations such as amputation and wearing clothes, the object recognition rate is significantly lower.

In addition, even if a technology such as deep learning is used, it is difficult to recognize emergency patients by learning the number of emergency patients.

That is, it is difficult to learn the emergency patient state according to various unpredictable conditions in advance, and accordingly, the patterning by the feature point is difficult, so that there is a lot of difficulty in recognizing the emergency patient.

Accordingly, the present invention intends to provide a recognition marking and tracking technology for recognizing a biometric object using an image from a first aid person's point of view in order to solve the object recognition problem.

In addition, the present invention provides an XR emergency medical treatment system in which first responders (such as dispatchers, dispatch doctors, dispatch medics or surgeons) can directly treat emergency patients using XR equipment in operational situations, disasters, and emergency patient situations. It is intended to provide a marker for bio-object recognition.

Another object of the present invention is to provide an XR emergency medical treatment system using a bio-object recognition marker for the XR emergency medical treatment system.

The present invention in order to solve the above needs and problems,

In the marker for bio-object recognition used in the XR system,

It provides a marker 400 for bio-object recognition for the XR system, which includes a taping body part 410 , a distance calculation unit 420 , and a tilt rotation angle calculation unit 430 .

In the present invention, the above-described taping body portion 410 is formed of a front portion 411 and a rear portion 412,

The distance calculating part 420 is formed of recognition distance parts 421 formed at a predetermined distance L on the taping body part 410,

The inclination rotation angle calculation unit 430 has a directional feature point 431, so that the inclination and rotation angle of the center of the marker can be calculated. ) is provided.

In the present invention, the above-described tilt rotation angle calculation unit 430 may have a polygonal shape or an irregular pattern, and may be formed in a color different from other constituent parts of the marker. (400) is provided.

In addition, the present invention includes a marker 400 for bio-object recognition for the XR system,

It provides an XR emergency medical treatment system configured to include a first responder boot unit 100 , an operation control server 200 , and a telemedicine office boot unit 300 .

In addition, the present invention is configured to include an XR treatment equipment 110, a treatment device unit 120, and a treatment tool unit 130.

The above-described telemedicine booth unit 300 provides an XR emergency medical treatment system, characterized in that it includes an XR remote device 310 and a remote treatment device unit 320 .

The marker for bio-object recognition for the XR system according to the present invention provides a three-dimensional bio-object recognition technology using a 2D image from the first aid person's point of view, and it is a method that quickly tracks the change of the first aid person's point of view and the movement of the patient. It has the effect of providing technology.

In addition, the marker for bio-object recognition for the XR system of the present invention performs a function of deriving the calculation of real-time bio-object size and rotation angle calculation using the marking information, thereby significantly increasing the registration rate of the three-dimensional modeled bio-object. .

In addition, since the marker for bio-object recognition for the XR system of the present invention is in the form of a taping, the effect of improving the convenience that first responders can easily use in the XR emergency medical treatment system appears.

In addition, the XR emergency medical treatment system using the marker for bio-object recognition for the XR system of the present invention uses virtual reality, augmented reality or/and mixed reality for emergency patients that are provided from the remote medical center booth through the operation control server. According to the XR (extended reality) first aid guide information, the action and effect of enabling first responders to treat emergency patients accurately will appear.

1 is an XR (extended reality, eXtended Reality) emergency medical treatment system configuration diagram according to the present invention.
Figure 2 is a configuration diagram of the first responder boots of the XR emergency medical treatment system according to the present invention.
Figure 2b is a configuration diagram of the treatment device of the XR emergency medical treatment system according to the present invention.
3 is a block diagram of a marker for bio-object recognition for the XR system according to the present invention.
3B is a rear view of the marker for bio-object recognition for the XR system according to the present invention.
3c is a block diagram of a marker for bio-object recognition for the bandage-type XR system according to the present invention.
3d is a block diagram of a bio-object recognition marker for the disposable band-aid type XR system according to the present invention.
Figure 3e is a block diagram of a marker for bio-object recognition for the roll-type band-aid type XR system according to the present invention.
Fig. 4 is a view of use when a plurality of markers according to the present invention are attached to a body part (injured part) of an emergency patient.
5 is a block diagram of a tilt rotation angle calculator of a marker according to the present invention.
5B is a conceptual diagram of measuring an intersection angle (θ2) for calculating the inclination of the inclination rotation angle calculating unit of the cross-shaped structure according to the present invention.
5C is a conceptual diagram of measuring an intersection angle (θ2) for calculating a slope of a tilt rotation angle calculating unit of a rectangular structure according to the present invention and an intersection angle (θ1) measurement by calculating the rotation angle;
5D and 5E are conceptual diagrams for measuring an intersection angle (θ2) and an intersection angle (θ1) by calculating the rotation angle for the slope calculation of the slope rotation angle calculation unit of the rhombus structure according to the present invention.
6 is a detailed configuration diagram and functional operation diagram of the XR emergency medical treatment system according to the present invention.
7 is a conceptual diagram of an operation of a 3D object transformation matrix calculator in an image of the XR emergency medical treatment system according to the present invention.
8 is a configuration diagram of a telemedicine booth unit of the XR emergency medical treatment system according to the present invention.
8B is a block diagram of the remote treatment device of the XR emergency medical treatment system according to the present invention.

Hereinafter, the present invention will be described in detail with reference to the drawings.

The present invention provides an XR (extended reality, eXtended Reality)-based emergency medical treatment system configured to include a first responder boot unit 100 , an operation control server 200 , and a telemedicine office boot unit 300 .

That is, in the present invention, when an emergency patient occurs, the emergency patient is placed in the emergency responder's boot unit 100 installed in an ambulance, etc., and the first responder performs first aid, from the remote medical center's boot unit 300 to the operation control server 200 First aid treatment according to first aid guide information implemented in virtual reality (VR), augmented reality (AR), mixed reality (MR) and/or XR (eXtended Reality) using these It provides an XR (extended reality, eXtended Reality)-based emergency medical treatment system that enables self-medication for emergency patients.

XR (eXtended Reality, extended reality) used in the present invention is a concept including all concepts of virtual reality (VR), augmented reality (AR), mixed reality (MR) and/or extended reality (XR) using them.

In addition, the present invention provides an XR-based emergency medical treatment system in which first responders (such as dispatchers, dispatch doctors, dispatch medics or surgeons) can directly treat emergency patients in operational situations, disasters, and emergency patient situations as described above. A marker 400 for bio-object recognition for the XR system used is provided.

As shown in FIG. 1 , the first aid unit 100 of the present invention allows first aid personnel (emergency personnel, dispatch doctors, dispatch medics or surgeons, etc.) directly to emergency patients in operational situations, disasters, and emergency patient situations. A device or means that provides equipment and space for treatment (or treatment).

As shown in FIG. 2 , the first aid unit 100 of the present invention is configured to include an XR treatment device 110 , a treatment device unit 120 , and a treatment tool unit 130 .

The XR treatment equipment 110 of the present invention is configured to include a display unit 111 , a headset unit 112 , a sensor sensing unit 113 , and a treatment gesture tracking unit 114 .

The XR treatment equipment 110 of the present invention is configured to include equipment or means used to implement virtual reality, augmented reality, mixed reality, extended reality, and the like.

The display unit 111 of the present invention refers to an interface that allows a user to easily view virtual information at a glance while working, and includes a device including a Head Mounted Display (HMD), a Face Mounted Display (FMD), or means means.

First responders wear such HMDs so that they can provide first aid to the patient according to the video guide that instructs the medical treatment from the telemedicine officer or/and the first aid method/precautions according to the patient's injury type and area. function is performed.

The above-described headset unit 112 of the present invention performs a function of allowing to listen to medical treatment instructions from a telemedicine officer or/and a first aid method/precaution guide according to the patient's injury type and site. means a device or means.

The headset unit 112 may include a microphone unit (not shown) and performs a function of transmitting the voice of a medical practitioner.

The sensor sensing unit 113 of the present invention senses a marker for recognizing the treatment site of the emergency patient and/or senses the first aid treatment process of the first aid responder and transmits it to the treatment device unit 120 . or means.

The treatment gesture tracking unit (Gesture tracking) 114 of the present invention performs a function to be worn on the hand of the first responder, senses the process of the first responder giving emergency treatment, and performs a function of acquiring the sensed information. means a device or means.

The sensing information about the emergency patient and/or the sensed information about the treatment process for the first responder acquired by the XR treatment device 110 of the present invention is transmitted to the operation control server 200 and is again remote from the operation control server 200 . It is transmitted to the medical office booth unit 300 .

The treatment device unit 120 of the present invention has a function of inputting information necessary for treatment of an emergency patient, a function of transmitting the inputted information to the operation control server 200, a first aid treatment method and caution for an emergency patient It refers to a device or means that performs a function of assisting a medical practitioner to provide first aid, such as a matter guide performing a function input from the operation control server 200 .

The treatment device unit 120 can be implemented as a computer, a server, etc., equipped with a typical central processing unit, a data input/output system, an application program, a control device, a memory, and the like.

The treatment device unit 120 of the present invention includes a tool setting module 121 , an information input module 122 , a damage site selection module 123 , a damage location information module 124 , and a first aid guide module 125 . is composed of

The above-described module of the present invention means a program, a function or device on software, etc. that executes what is intended to function in each module.

The tool setting module 121 means performing a function for setting a medical treatment tool that can be used by the first responder.

In the tool setting module, the first responder sets or inputs a medical action tool for treating an emergency patient.

The above-described information input module 122 performs a function of inputting personal information and patient evaluation information for an emergency patient.

The above-described damaged part selection module 123 performs a function in which the type of injury of the emergency patient is selected and the damaged body part is selected and set.

The damage occurrence location information module 124 performs a function of inputting or setting a geographic location where an emergency patient's damage occurred.

The first aid guide module 125 performs a function of presenting first aid methods and precautions according to the inputted emergency patient's damage type and damage site.

The first aid guide module 125 may be a video or voice guide, and may be preset in the treatment device 120 and may be provided or updated by a telemedicine officer.

Information input or set in the treatment device unit 120 of the present invention is transmitted to the operation control server 200 , and such information is again provided to the first responder boot unit 100 .

A monitor unit is added to the treatment device unit 120 of the present invention, so that an emergency responder can operate the treatment device unit 120 through the monitor unit.

The treatment tool unit 130 of the present invention means a device or means equipped with a practical surgical tool, medicine, equipment, etc. for treating an emergency patient.

The operation control server 200 of the present invention receives various information such as image information, patient information, damage information, etc. provided by the first aider boot unit 100 and recognizes a biological object, a 3D modeling function of a biological object , performs 3D modeling data matching function, 3D modeling and medical data matching function, first aid guide matching function according to the visual position of first responder, performs data exchange function, bio-object recognition function, database management functions, etc.

In addition, the operation control server 200 has a function of receiving and processing information provided or input from the telemedicine booth unit 300 and/or a function of transmitting the processed information to the first responder booth unit 100 . carry out

The operation control server 200 is equipped with a typical central processing unit, a data input/output system, an application program, a control device, a memory, and the like.

The operation control server 200 of the present invention is configured to include a data exchange module 210 , an operation control module 220 , a biometric object recognition module 230 , and a database management module 240 .

The above-described data exchange module 210 has a function of exchanging or receiving commands and information provided from the first aid unit 100 and/or commands and information provided from the telemedicine unit 300 with each other. carry out

The above-described operation control module 220 controls the configuration and operation of the operation control server 200 by the manager of the operation control server 200, the first aid unit 100 or/and the telemedicine unit 300. It performs the function that allows you to control the

The bio-object recognition module 230 recognizes the emergency patient and the damaged area through the recognition information on the emergency patient recognized and provided by the XR treatment equipment 110 of the first responder boot unit 100 in extended reality (XR). ) to perform the recognition function.

In particular, the bio-object recognition module 230 allows the XR treatment equipment 110 to recognize the damaged part of the emergency patient by recognizing the marker attached to the emergency patient, and the treatment gesture tracking unit 114 . Through this, it performs the function of recognizing the process of emergency treatment of first responders.

Marker used as an object recognition means in conventional virtual reality, augmented reality, mixed reality, etc. bleeding, shock (face deformation) of emergency patients to be recognized in the present invention , body parts cutting, clothing worn state, etc., difficulties arise in recognizing biometric objects in various situations.

In particular, although most of the conventional recognition fields use the object recognition method using feature points, even if the marker-based recognition technology is applied by the factors of the patient's body part and surrounding environment (military uniform pattern, patient-specific body type, etc.), it is difficult to find the feature point. It is a very difficult reality to accurately recognize the injured area of an emergency patient.

In addition, even if information on the parts of the human body of an emergency patient (arms, legs, etc.) is provided, the biometric object recognition module 230 of the operation control server 200, which is a computer, makes it difficult to guess the part, as well as bleeding, shock ( In the case of an emergency patient in various situations such as face deformation), body part cutting, clothing worn state, etc., the object recognition rate is significantly lower.

In addition, even if a technology such as deep learning is used, it is difficult to recognize emergency patients by learning the number of emergency patients.

That is, it is difficult to learn the emergency patient state according to various unpredictable conditions in advance, and accordingly, the patterning by the feature point is difficult, so that there is a lot of difficulty in recognizing the emergency patient.

Accordingly, the present invention provides a recognition marking and tracking technology for recognizing a biometric object using an image from a first aid person's point of view in order to solve the object recognition problem.

Therefore, the XR-based emergency medical treatment system of the present invention further includes a marker 400 for recognizing a biometric object for the XR system.

The present invention provides a marker 400 for bio-object recognition for an effective XR system for recognizing the direction and size of a three-dimensional bio-object.

Therefore, the present invention provides a bio-object recognition technology in a three-dimensional form using the 2D image of the first aider's point of view using the above-described marker 400, and a technology for rapidly tracking the change of the first aid person's point of view and the movement of the patient will provide

The marker 400 for bio-object recognition for the XR system of the present invention performs a function of deriving real-time bio-object size and rotation angle calculation using marking information.

In addition, the marker 400 for recognizing a biometric object for the XR system of the present invention is capable of calculating a distance, calculating an inclination and a rotation angle, and has the convenience of a taping type.

The marker 400 for bio-object recognition for the XR system of the present invention is configured to include a taping body part 410 , a distance calculation unit 420 , and a tilt rotation angle calculation unit 430 .

As shown in FIG. 3 , the taping body part 410 is formed of a front part 411 and a back part 412 .

The taping body 410 may be formed in various directions and is generally in the form of a band.

The front part 411 means a part in which the distance calculating part 420 and the inclination rotation angle calculating part 430 are formed.

The back part 412 performs a function of being attached to an emergency patient, and therefore, an adhesive part having an adhesive material formed thereon is provided on the back part to easily attach a marker to the emergency patient.

The distance calculating unit 420 of the present invention is a shape, structure, or means that performs a function of allowing the actual distance to be measured to the injured part of the emergency patient, which is the recognition target, when the marker attached to the actual emergency patient is recognized. it means.

Accordingly, as an embodiment of the distance calculating unit 420 , it may be formed of recognition distance units 421 formed at a predetermined distance L on the taping body 410 .

As shown in FIG. 3 , the recognition distance part 421 of the distance calculating part 420 may be formed of lines 421 formed at a predetermined distance L on the taping body part 410 .

In this case, the distance L2 between the lines 421 with the inclination rotation angle calculating unit 430 interposed therebetween may be preset with the above-described distance L between the lines and the lines in a predetermined formula relationship, The distance L2 between the lines 421 with each calculation unit 430 interposed therebetween may be derived by an equation when the distance L between the lines is known.

Accordingly, the distance between the lines 421 of the distance calculating unit is set to a predetermined constant distance L, and when these lines are recognized, the distance between the injured part of the emergency patient can be actually measured.

As shown in FIG. 4, when one or more markers 400 of the present invention are attached to a body part (injured part) of an emergency patient in need of treatment to recognize the markers, the markers are recognized and a distance calculating unit formed on the marker ( 420), the effect of accurately measuring and recognizing the distance between the body parts (damaged parts) of the emergency patient is exerted.

The above-described inclination rotation angle calculation unit 430 of the present invention is formed in the taping body portion 410, and refers to a shape, structure or means formed for calculating the inclination and rotation angle of the central portion of the marker.

The tilt rotation angle calculation unit 430 may be formed in various shapes to calculate the tilt and rotation angle of the central portion of the marker, may have a polygonal shape or an irregular pattern, and may be formed in a color different from other constituent parts of the marker. have.

The above-described tilt rotation angle calculating unit 430 has a direction feature point 431 is formed, and it is preferable that two or more of the direction feature point are formed.

As shown in FIG. 5, the direction feature point 431 of the present invention is set as an inflection point at which the inclination rotation angle calculation unit 430 rapidly changes a line or surface on the spatial coordinates distinguished from the taping body unit 410. it means.

Accordingly, the direction feature points 431 are preset as spatial coordinates of the marker.

5 shows an inclination rotation angle calculating unit 430 having a cross-shaped structure as an example of the inclination rotation angle calculating unit 430 .

The tilt rotation angle calculator 430 of the cross-shaped structure has a total of eight or more directional feature points 431 are formed, two each in the east, west, south, and north directions, and each of the directional feature points 431 is the spatial coordinate of the marker. is preset to

Therefore, when the marker is attached to the emergency patient, the bio-object recognition module 230 for the recognition information recognized and provided by the XR treatment equipment 110 of the first aid unit 100 is the above-described tilt rotation angle calculation unit ( Pre-set spatial coordinates of the direction feature points 431 of 430 are recognized.

As shown in FIG. 5 , in this case, the biometric object recognition module 230 includes the center of the direction line D1 connecting the direction feature point 431-1 and the direction feature point 431-2 and the direction feature point 431-5 and The upper and lower reference lines S1 connecting the centers of the direction lines D2 connecting the direction feature points 431-6 are recognized,

The center of the direction line D3 connecting the direction feature point 431-3 and the direction feature point 431-4 and the center of the direction line D4 connecting the direction feature point 431-7 and the direction feature point 431-8 The connecting left and right reference lines S2 are recognized.

As shown in Figure 5b, the present invention is based on the upper and lower baseline (S1) or / and the left and right baseline (S2) of the initially recognized marker MF1. The inclination of the marker is measured by measuring the intersection angle (θ2) with the upper and lower reference lines (S1-1) and/or left and right reference lines (S2-1) recognized from the recognized marker (MF2) in the case where the patient with the marker is moved. can be calculated, which will soon calculate the patient's movement gradient.

In addition, the bio-object recognition module 230 recognizes the rotation angle T1 connecting the direction feature point 431-2 and the direction feature point 431-6.

The rotation angle means an intersection angle θ1 formed by crossing the upper and lower reference lines S1 and/or the left and right reference lines S2.

Accordingly, the rotation angle line T1 is a line formed by connecting two selected from a plurality of direction feature points 431, and intersects the upper and lower reference lines S1 and/or the left and right reference lines S2 to form an intersection angle θ1. includes all lines.

As such, the direction feature point 431 forming the angle of rotation T1 is preset, and one or more angles of rotation may be set and formed.

By calculating the intersection angle θ1, the rotation angle of the spatially moved marker can be calculated, and the movement of the patient can be more accurately recognized through the rotation angle calculation.

The above-described inclination rotation angle calculation unit 430 of the present invention is formed in the taping body portion 410, and refers to a shape, structure or means formed for calculating the inclination and rotation angle of the central portion of the marker.

FIG. 5C shows the inclination rotation angle calculator 430 of a rectangular shape, in which four direction feature points 431 are set, and accordingly the upper and lower reference lines S1 and/or the left and right reference lines S2 are set and rotated Since the angle line T1 is also set, the rotation angle can be calculated by forming the intersection angle θ1.

In addition, based on the upper and lower baselines (S1) or/and the left and right baselines (S2) of the initially recognized marker (MF1), when the first responder sees the emergency patient and moves according to the movement (or when the patient with the marker moves) ), the intersection angle θ2 with the upper and lower reference lines S1-1 and/or the left and right reference lines S2-1 recognized by the recognized marker MF2 is measured to form a structure in which the slope of the marker can be calculated.

5D and 5E show a rhombus-shaped inclination rotation angle calculator 430, in which five directional feature points 431 are set, and accordingly, a directional feature point 431-1 and a directional feature point 431-3. and an upper and lower reference line S1 connecting the center of the direction line D3 connecting the direction feature point 431-4, or the center and the direction feature point of the line connecting the direction feature point 431-2 and the direction feature point 431-3 The left and right reference lines S2 are set by connecting the centers of the lines connecting 431-4 and the direction feature points 431-5.

Accordingly, such an upper and lower reference line (S1) and/or left and right reference lines (S2) are set, and a rotation angle line (T1) is also set by connecting the direction feature points (431-2) and the direction feature points (431-4). By forming θ1), the rotation angle can be calculated.

In addition, based on the upper and lower baselines (S1) or/and the left and right baselines (S2) of the initially recognized marker (MF1), when the first responder sees the emergency patient and moves according to the movement (or when the patient with the marker moves) ), the intersection angle θ2 with the upper and lower reference lines S1-1 or/and the left and right reference lines S2-1 recognized by the recognized marker MF2 is measured to form a structure that can calculate the slope of the marker.

Therefore, the tilt rotation angle calculation unit 430 of the present invention includes all the shapes, structures, or means of polygons or irregular patterns formed to calculate the tilt and rotation angles of the center of the marker.

As shown in FIG. 4 , the marker 400 of the present invention can be attached to a plurality of bars. In this way, the distance calculation, inclination and rotation angle for each of the markers 400 attached to the plurality of markers are calculated, and between the markers. If the distance is calculated, it becomes possible for the first responder to accurately recognize the injured part of the emergency patient according to the change of movement and point of view.

That is, the marker 400 of the present invention is a marker 400-1, a marker 400-2, a marker 400-3, a plurality of markers such as the marker 400-n, an emergency patient ( or the damaged part), so that the distance calculation between the respective markers is enabled by the distance value preset in the distance calculating unit 420 included in the individual markers.

3c to 3e, the marker of the present invention may be formed in various forms, such as a bandage type, a disposable band-aid type, a roll type band-aid type, and the like.

When the marker becomes a disposable band-aid type or a roll type band-aid type, it can be attached to all parts of the body, such as tourniquets, clothing, and the human body, so it is possible to cope with various types of situations.

The biometric object recognition module 230 of the present invention performs a function of generating 3D biometric object information and 3D object transformation matrix information for an emergency patient to be provided to the telemedicine booth unit 300 .

As shown in FIG. 6 , the biometric object recognition module 230 receives the image information (PI) recognized for the emergency patient provided by the first aid unit 100, and then the image information acquisition unit 231, Marking information acquisition processing unit 232, bio-object size gradient rotation angle calculation unit 233, 3D bio-object modeling unit 234, viewer and biological object relative coordinate calculation unit 235, 3D object transformation matrix in the image Through the calculation unit 236 , the coordinate transformation and registration error calculation unit 237 , 3D biological object information and 3D object transformation matrix information I for an emergency patient are generated and provided to the telemedicine booth 300 .

In addition, the 3D biological object information and the 3D object transformation matrix information (I) for the emergency patient generated as described above are transmitted to the first responder boot unit 100 and outputted to the emergency patient through the XR treatment device 110 and telemedicine According to the first aid guide information (GI) transmitted from the tube boot unit 300, the first aid person can perform medical treatment based on this.

The image information acquisition unit 231 of the present invention includes a module, a program, a device, etc., that performs a function of receiving and acquiring image information recognized for an emergency patient provided by the first responder boot unit 100. it means.

The above-described marking information acquisition processing unit 232 means a module, program, device, etc., which performs a function of acquiring marking information obtained from a marker attached to an emergency patient provided by the first aid unit 100 .

The biological object size gradient rotation angle calculation unit 233 performs a function of calculating the size, inclination and rotation angle of a biological object (emergency patient, etc.) through the marking information provided by the marking information acquisition processing unit 232 . means a module, program, device, etc.

The 3D inter-object modeling unit 234 is a module, program, device, etc. that performs a function of modeling in a normal 3D (three-dimensional) model using the image information recognized for the emergency patient obtained above. means

Such a 3D (three-dimensional) model is included in the 3D biological object information and the 3D object transformation matrix information (I).

The relative coordinate calculation unit 235 between the viewer and the living body object means a module, program, device, etc., that generates or calculates 3D coordinates for the emergency patient when the first responder sees the emergency patient.

The relative coordinate calculation unit 235 between the viewer and the living object is generated as 3D coordinates for the emergency patient in a preset space on the system through the image information about the emergency patient acquired by the image information acquisition unit 231, or or/and the function being calculated is performed.

The 3D object transformation matrix calculation unit 236 in the image is the 3D model obtained from the 3D inter-object modeling unit 234, and the 3D coordinates obtained from the relative coordinate calculation unit 235 between the viewer and the living object. , A module, program, and device that performs a function of converting into a transformed 3D human body model by applying a transformation matrix through the size, gradient, and rotation angle of the emergency patient obtained by the bio-object size gradient rotation angle calculation unit 233 means such as

As shown in Fig. 7, the 3D module (standard 3D human body module) (SM) is converted into 3D by applying the size (OS) of the object (emergency patient, damaged area, etc.) recognized by the marker through the transformation matrix (TR). It is converted into a human body model (TM).

Such a transformed 3D human body model (TM) is included in the 3D biological object information and the 3D object transformation matrix information (I).

As the above-described transformation matrix TR, a conventional transformation matrix may be used, and a transformation matrix such as the following [Equation 1] may be used as an embodiment.

[Equation 1]

In the above R _x (θ), X means the X coordinate in space with respect to the marker and the biological object, and θ means the inclination or/and the rotation angle.

In addition, in R _y (θ), y denotes a spatial y coordinate for a marker and a biological object, and θ denotes a tilt or/and a rotation angle.

In addition, in R _z (θ), z denotes a spatial z coordinate for a marker and a biological object, and θ denotes a tilt or/and a rotation angle.

As such, by applying the converted 3D human body model (TM) to be output on XR, the matching rate with an actual object (emergency patient) is significantly increased.

Accordingly, the converted 3D human body model (TM) converted in this way is transmitted to the first aid unit 100 and/or the telemedicine unit 300 to provide a transformed 3D human body model (TM) that is accurately matched to the emergency patient. As a result, the telemedicine officer can perform an accurate medical treatment guide, and the first responder is provided with an accurate medical treatment guide for the exact treatment area of the emergency patient, resulting in the effect of accurately treating the emergency patient.

The coordinate transformation and matching error calculation unit 237 calculates the 3D object for the emergency patient at the point of view of the emergency patient by the first responder obtained from the relative coordinate calculation unit 235 between the viewpoint and the living object. Means a module, program, device, etc., that performs calculations to match screen coordinates for projection on a dimensional plane.

Coordinate transformation and registration error calculation unit 237 may apply a registration rate calculation method such as a conventional Gaussian interpolation method used in screen coordinate transformation technology for applying a conventional three-dimensional object to a two-dimensional plane. can

The transformed 3D human body model (TM) reset by the action of further increasing the matching rate with the actual emergency patient with respect to the transformed 3D human body model (TM) of the coordinate transformation and matching error calculation unit 237 is first aid treatment It is transmitted to the self-boosting unit 100 and/or the telemedicine unit 300 and provided with a converted 3D human body model (TM) that is accurately matched to the emergency patient, so that the telemedicine unit can perform a more accurate medical treatment guide. The effect of treating emergency patients more accurately is shown by providing an accurate medical treatment guide for the exact treatment area of the emergency patient.

The database management module 240 of the present invention has a function to store and manage information generated or recognized by the operation control server 200 , the first aid unit 100 and/or the telemedicine unit 300 . carry out

The telemedicine booth unit 300 of the present invention provides various information such as image information, patient information, and damage information provided by the first responder boot unit 100 or/and the operation control server 200 using the same. The first aid operator boots 100 for the telemedicine to perform first aid using the XR image (determined reality image) formed by receiving 3D biological object information and 3D object transformation matrix information (I) for an emergency patient means a device means to perform a function provided by

The telemedicine in the present invention is a concept including a doctor, a telemedicine, a telemedicine, and the like who guide emergency treatment from a distance.

As shown in FIG. 8 , the telemedicine booth unit 300 of the present invention is configured to include an XR remote device 310 and a remote treatment device unit 320 .

The XR remote device 310 of the present invention is configured to include a remote display unit 311 , a remote headset unit 312 , a remote sensor sensing unit 313 , and a guide gesture tracking unit 314 .

The XR remote device 310 of the present invention is configured to include equipment or means used to implement virtual reality, augmented reality, mixed reality, extended reality, and the like.

The remote display unit 311 of the present invention refers to an interface that allows a user to easily view virtual information at a glance while working, and includes a head mounted display (HMD) and a face mounted display (FMD). or means.

Telemedicine refers to a device or means in which a function of generating an image guide instructing medical treatment by viewing an XR image (extended reality image) of an emergency patient by wearing an HMD is performed.

The remote headset unit 312 of the present invention allows the telemedicine officer to input or output information for instructing medical treatment or/and first aid method/precaution guide according to the patient's type and location of damage by voice. means a device or means by which a function is performed.

Therefore, the remote headset unit 312 may have a microphone unit added thereto, and performs a function of transmitting the voice of the telemedicine unit.

The remote sensor sensing unit 313 of the present invention senses an instructional operation for a telemedicine officer to view an XR image (extended reality image) of an emergency patient and perform medical treatment by wearing a treatment gesture tracking unit This means a device or means for performing a function of transmitting to the remote treatment device unit 320 .

The treatment gesture tracking unit (Gesture tracking) 314 of the present invention performs a function worn on the hands of the telemedicine, and information on the hand gestures and movements that the telemedicine performs emergency treatment through the XR image (extended reality image) means an apparatus or means such as a motion device that provides

The first aid guide of the telemedicine obtained from the XR remote device 310 of the present invention is transmitted to the operation control server 200 , and is again transmitted from the operation control server 200 to the first aid unit 100 .

The remote device unit 320 of the present invention performs a patient medical information input function, a function to check available medical treatment tools for first responders, a function to check patient status using patient personal information and evaluation information, and a function to provide emergency treatment guide information. means a device or means.

The remote device unit 320 may be implemented as a computer, a server, etc. equipped with a typical central processing unit, a data input/output system, an application program, a control device, a memory, and the like.

The remote device unit 320 of the present invention includes a patient medical information input module 321 , a medical treatment tool confirmation module 322 available for first responders, a patient status confirmation module 323 using patient personal information and evaluation information, and an emergency It is configured to include a condition guide information providing module 324 and an image management module 325 .

The above-mentioned module of the present invention means a program, software function or device, etc. that performs a function desired in each module.

The above-described patient medical information input module 321 performs a function of inputting personal information and patient evaluation information for an emergency patient by a telemedicine officer.

The module 322 for checking available medical treatment tools of the first responder means that the telemedicine officer performs a function of setting available medical treatment tools for emergency patients.

In the module for checking available medical treatment tools of the first responder, the telemedicine sets or inputs a medical action tool for treating an emergency patient.

The patient status confirmation module 323 using the patient personal information and evaluation information described above uses the personal information and patient evaluation information for the emergency patient input from the information input module 122 of the first aid unit 100 to determine the patient status. function to show

The first aid guide information providing module 324 and the first aid guide module 125 perform a function of providing first aid methods and precautions according to the inputted emergency patient's damage type and damage site.

The first aid guide information providing module 324 may provide a first aid guide through video or voice, and may be input or updated by a telemedicine officer.

In addition, the first aid guide information providing module 324 can directly implement the first aid guide through extended reality when the telemedicine directly treats (or treats) the emergency patient using the XR remote device 310, The same first aid guide information is transmitted to the first responder boot unit 100 to perform a function.

The image management module 325 performs a function of managing the image transmitted from the first aid unit 100 by a telemedicine officer.

The present invention provides a marker for bio-object recognition for an XR system having the above configuration and functions, and an XR-based emergency medical treatment system using the same.

The present invention is useful in industries that produce, manufacture, sell, distribute, and research markers used in virtual reality, augmented reality, mixed reality, or extended reality.

In particular, it is very useful for emergency medical treatment systems using virtual reality, augmented reality, mixed reality, or extended reality.

First Aid Booth (100),
XR treatment equipment 110, treatment device unit 120, treatment tool unit 130,
operation control server 200,
Tool setting module 121, information input module 122, damaged part selection module 123, damage location information module 124, first aid guide module 125,
Telemedicine Center Bootsbu (300),
XR remote equipment 310, remote treatment device unit 320,
Marker 400 for bio-object recognition for the XR system;
Taping body portion 410, distance calculation unit 420, tilt rotation angle calculation unit 430,

Claims (3)

XR treatment equipment 110, treatment device unit 120, first aid boot unit 100 comprising a treatment tool unit 130,
The operation control server 200 comprising a data exchange module 210 , an operation control module 220 , a biometric object recognition module 230 , and a database management module 240 ,
A first aid training type medical system according to the expanded reality first aid guide information configured including the remote medical center boot unit 300 including the XR remote equipment 310 and the remote treatment device unit 320 .
The method of claim 1,
The XR treatment equipment 110 is configured to include a display unit 111, a headset unit 112, a sensor sensing unit 113, and a treatment gesture tracking unit (Gesture tracking) 114,
The treatment device unit 120 includes a tool setting module 121, an information input module 122, a damaged site selection module 123, a damage location information module 124, and a first aid guide module 125. First responder education type medical system according to extended reality first aid guide information, characterized in that there is.
The method of claim 1,
A first aid training type medical system according to extended reality first aid guide information, characterized in that the marker 400 for biometric object recognition is further included.

Images