KR102673626B1 - System and method for support remote medical service - Google Patents

Abstract

A remote medical support system and a remote medical support method are disclosed. The remote medical support method according to the present invention includes the steps of acquiring an image of a patient with a marker attached to each of the joint areas using a camera of a visualization device worn by a field agent; Obtaining 3D coordinate values of the patient from the patient's image and acquiring 3D coordinate values of markers; Setting the three-dimensional coordinate values of each marker as first fixed coordinates; Receiving input from field personnel regarding first aid location and first aid method on the patient's image displayed through a visualization device; Setting the first aid location as a second fixed coordinate; Projecting a three-dimensional image of a first aid method onto second fixed coordinates and displaying it through a visualization device; and matching a three-dimensional image to the patient's image displayed through a visualization device based on the first fixed coordinates and the second fixed coordinates.

Description

Remote medical support system and remote medical support method {SYSTEM AND METHOD FOR SUPPORT REMOTE MEDICAL SERVICE}

The present invention relates to a remote medical support system and a remote medical support method.

An emergency patient is defined as a patient whose symptoms are critical and requires emergency treatment, and for these emergency patients, the initial response within 5 minutes from the time the initial emergency occurs has a huge impact on the survival rate. For example, there is a significant difference in survival rates between cases where initial response such as CPR is performed within 5 minutes after the heart stops and when initial response is performed after 5 minutes, and the preventable mortality rate among all trauma patients is 35.2% (2010 The initial response to emergency patients is directly related to the survival rate of emergency patients.

Typically, initial response to emergency patients is possible only when field personnel arrive at the scene, and even if field personnel arrive at the scene, rescuers without professional medical knowledge can perform basic initial response such as CPR, but professional medical knowledge is not required. In the event of an emergency, this on-site agent was only able to respond initially after being moved to a nearby hospital by the on-site agent, and as the initial response to emergency patients was delayed, preventable incidents in which emergency patients ended up dying frequently occurred. It is happening.

To solve this problem, if an emergency patient is in critical condition, initial emergency treatment can be performed through phone calls from field personnel and professional medical staff. However, there are limits to the doctor's ability to accurately recognize the patient's condition simply through voice. Also, in an urgent situation, a field agent calling a doctor using his/her mobile communication terminal and making the call has the limitation of wasting precious time and is not of practical help.

Therefore, as the incidence of emergency patients continues to increase due to improvements in people's lives and changes in the structure of diseases, there is an urgent need for research on medical support to accurately and quickly handle the initial response to emergency patients at disaster sites.

Republic of Korea Patent Publication No. 10-2011-0054382 Republic of Korea Patent No. 10-1063998

Therefore, the technical problem to be solved by the present invention is to provide a remote medical support system and remote medical support method that accurately and quickly handles the initial response to emergency patients and facilitates medical support by professional medical staff when necessary.

According to one aspect of the present invention, acquiring an image of a patient with a marker attached to each of the joint areas using a camera of a visualization device worn by a field agent; Obtaining 3D coordinate values of the patient from the patient's image and obtaining 3D coordinate values of the markers; Setting each of the three-dimensional coordinate values of the markers as first fixed coordinates; Receiving input from field personnel regarding first aid location and first aid method on the image of the patient displayed through the visualization device; Setting the first aid location as second fixed coordinates; Projecting a three-dimensional image related to the first aid method on the second fixed coordinates and displaying it through the visualization device; and matching the 3D image to the image of the patient displayed through the visualization device based on the first fixed coordinates and the second fixed coordinates.

The step of registering the 3D image may include setting a registration area in which the 3D image will be registered to the patient's image based on the first fixed coordinates and the second fixed coordinates; calculating a rotation angle, size change value, and position change value of the three-dimensional image with respect to the matching area; Transforming the three-dimensional image using the calculated rotation angle, size change value, and position change value of the three-dimensional image; and matching the deformed 3D image to the matching area.

In the case of an emergency situation in which the patient's arm or leg is fractured, the registration area is an area between the first fixed coordinates adjacent to both sides of the second fixed coordinate, and the three-dimensional image is a treatment method for fracture. This is a bandage method, and the three-dimensional image can be overlapped and registered in the registration area by calculating the volume surface defined by the body standard.

In the case of an emergency situation where the patient is in cardiac arrest, the registration area is the first central coordinate that is the center point of the first fixed coordinates corresponding to both shoulder joints of the patient, respectively, and the second coordinate area corresponds to both pelvic joints of the patient. 1 It is a registration position that is 1/4 of the distance to the second central coordinate, which is the center point of the fixed coordinates. The three-dimensional image is CPR, which is a treatment method for cardiac arrest, and the three-dimensional image corresponds to both shoulder joints of the patient. The first fixed coordinates may be aligned with the matching position in a direction perpendicular to a triangle having the matching position as a vertex.

Receiving the field agent input may include displaying an image of the patient through the visualization device worn by the field agent; and specifying the first aid location through a gesture of a field worker on the image of the patient displayed through the visualization device and selecting the first aid method from a manual.

In the step of acquiring an image of the patient, an image of the patient's entire body to which the markers are attached may be obtained from a distance when the patient is initially discovered.

The marker may be a QR code containing data information about the patient's joint area.

If the marker cannot be attached to the patient's joint area, the marker can be attached to the floor adjacent to the patient's corresponding joint area.

When the field agent requests remote medical support, the method may further include searching for and connecting to a connectable medical staff from a list of medical staff, and sharing an image of the patient through a visualization device worn by the medical staff.

Embodiments of the present invention can accurately and quickly handle the initial response to emergency patients.

Additionally, embodiments of the present invention can improve the patient's biometric object recognition and matching rate by using markers attached to the patient's joint areas.

In addition, embodiments of the present invention enable professional medical staff to more accurately recognize the condition of emergency patients through images, allowing accurate and prompt medical support services to be received from professional medical staff.

1 is a conceptual diagram showing a remote medical support system of the present invention.
Figure 2 is a configuration diagram showing a field operation device according to the present invention.
Figure 3 is a flowchart showing a remote medical support method according to the present invention.
4 to 14 are diagrams showing image registration according to the present invention.

In order to fully understand the present invention, its operational advantages, and the objectives achieved by practicing the present invention, reference should be made to the accompanying drawings illustrating preferred embodiments of the present invention and the contents described in the accompanying drawings.

Hereinafter, the present invention will be described in detail by explaining preferred embodiments of the present invention with reference to the accompanying drawings. The same reference numerals in each drawing indicate the same member.

Figure 1 is a conceptual diagram showing a remote medical support system of the present invention, and Figure 2 is a configuration diagram showing a field operation device according to the present invention.

Referring to Figures 1 and 2, the remote medical support system according to the present invention is a system that ensures quick and accurate initial first aid treatment for emergency patients.

The remote medical support system is carried by field personnel and is connected to a field operation device (100) that provides medical support services to field agents, and is connected to the field operation device (100) and provides medical support services to the field operation device (100). It includes a remote support server 200 and a remote medical support device 300 that is connected to the remote support server 200 and provides emergency scene information to professional medical staff and delivers medical support services from professional medical staff to field personnel.

As shown in FIG. 2, the field operation device 100 according to this embodiment is connected to a visualization device 110 worn on the head by a field agent (or rescuer), and is communicated with the visualization device 110 to display a visualization device ( It includes a portable controller 120 that controls 110).

The visualization device 110 may provide at least one virtual reality among VR (virtual reality), AR (Augmented Reality), or MR (Mixed Reality) to field personnel.

In particular, mixed reality (MR) is a technology that displays a virtual world with additional information in real time over the real world as a single image. Because mixed reality technology provides visible information, it can be usefully applied in the medical field. Mixed reality technology can provide image information needed for emergency situations, medical treatment, etc. In order for mixed reality technology to be practically used in medical settings, field personnel must be able to visually see necessary information while providing first aid to patients in emergency situations. Accordingly, in this embodiment, a visualization device 110 such as Google Glass or Microsoft HoloLens can be used.

The portable controller 120 is connected to the visualization device 110 and controls the visualization device 110. The portable controller 120 includes an image acquisition unit 121, an image coordinate acquisition unit 123, an image matching unit 125, and a display unit 127.

The image acquisition unit 121 acquires and stores real-time real-time images of the patient. The image acquisition unit 121 acquires an image of the patient using the camera of the visualization device 110. At this time, a marker (M) is attached to each of the patient's joint areas and an image of the patient is taken (see Figure 4). In the case of acquiring images of a patient to whom markers (M) are attached, field personnel acquire images of the patient's entire body to which markers (M) are attached from a distance when the patient is initially discovered.

In this embodiment, the marker (M) may be a QR code containing data information about the patient's joint area. The marker (M) has data information on the patient's fracture area, for example, shoulder joint, pelvic joint, wrist joint, etc. Attach the corresponding marker (M) to each joint area. Meanwhile, if the marker (M) cannot be attached to the patient's joint area, the marker (M) is attached to the floor adjacent to the joint area.

The image coordinate acquisition unit 123 acquires the 3D coordinate values of the patient from the patient's image and acquires the 3D coordinate values (J0 to J16) of the markers (M) attached to the patient (see FIG. 5). If a marker (M) cannot be attached to the patient's joint area, 3D coordinate values are obtained through a marker (M) attached to the floor adjacent to the joint area.

The image registration unit 125 matches the 3D image of the first aid method to the real-time image of the patient. The image registration unit 125 controls the display unit 127 to match the patient's real-time image and the 3D image with the same coordinates and output an image that overlays the 3D image on the patient's real-time image.

And the image registration unit 125 may set the 3D coordinate values (J0 to J16) of the markers (M) as first fixed coordinates (anchor index), respectively.

The three-dimensional image of the first aid method is generated by a visualization device ( This is the first aid method displayed in 110).

In addition, when a field agent specifies or selects a first aid position in an image of a patient, the image registration unit 125 sets the first aid position to the second fixed coordinate (T) and displays a three-dimensional image of the first aid method in the second fixed coordinate. The image is projected and displayed on the visualization device 110.

And the image registration unit 125 is configured to add a three-dimensional image of first aid to the image of the patient displayed through the visualization device 110 based on the first fixed coordinates (J0 to J16) and the second fixed coordinate (T). Match.

The display unit 127 outputs an image in which the real-time image of the patient and the 3D image of the first aid method are matched by the image registration unit 125 to the visualization device 110.

The remote support server 200 according to this embodiment is connected to the field operation device 100 and the remote medical support device 300, and provides medical support services to the field operation device 100. Additionally, the remote support server 200 delivers medical support services from professional medical staff provided by the remote medical support device 300 to field personnel.

The remote support server 200 communicates data with the portable controller 120 and the remote medical support device 300, allowing field personnel and professional medical staff to share images of emergency situations and patients.

The remote medical support device 300 according to this embodiment provides medical services to field personnel when the field personnel reports the patient's condition and requests remote medical support from professional medical staff. The remote medical support device 300 searches for and connects to connectable medical staff from the medical staff list. At this time, the remote medical support device 300 allows the professional medical staff to share the patient's image with the medical staff through the visualization device 310 worn by the medical staff. It also delivers medical services from medical staff to field personnel.

The remote medical support method according to the present invention configured as described above is as follows.

Figure 3 is a flowchart showing a remote medical support method according to the present invention, and Figures 4 to 14 are diagrams showing image registration according to the present invention.

Referring to FIG. 3, a field worker who arrives at the scene of an emergency situation wears the visualization device 110 and then acquires an image of the patient using the camera of the worn visualization device 110 (S100). At this time, as shown in FIG. 4, prior to acquiring an image of the patient, a marker (M), which is a QR code with data information about the patient's joint region, is attached to each of the patient's joint regions. For example, 17 markers (M) can be attached to each shoulder joint, pelvic joint, etc. Meanwhile, if the marker (M) cannot be attached to the patient's joint area, the marker (M) can be attached to the floor adjacent to the joint area.

The initial image of the patient is obtained from a distance of the patient's entire body with markers (M) attached. The image of the patient captured by the camera is stored in the image acquisition unit 121.

And as shown in FIG. 5, the image coordinate acquisition unit 123 acquires the 3D coordinate values of the patient from the patient's image, and obtains the 3D coordinate values (J0 to J16) of the markers (M) attached to the patient. Obtain (S200).

Then, the field personnel determines the patient's condition and determines whether the patient's condition corresponds to a situation in which emergency treatment is possible according to the manual (S300).

If emergency treatment according to the manual is possible, a 3D image of the emergency treatment method is matched to the real-time image of the patient through the image registration unit 125 (S400). The 3D coordinate values of the real-time image of the patient and the 3D image of the first aid method are matched with the same coordinates, and the 3D image overlaid on the real-time image of the patient is output to the display unit 127.

To match the 3D image of the patient and the first aid method, set the 3D coordinate values (J0 to J16) of the markers (M) attached to the patient as the first anchor index (see FIG. 6). ).

It then receives input from field personnel. That is, the field agent designates the first aid location (T) through a gesture on the real-time image of the patient displayed through the visualization device 110 and selects the first aid method from the manual.

When a field worker specifies or selects a first aid position in an image of a patient, the image registration unit 125 can set the first aid position (T) as the second fixed coordinate (see FIG. 7), and the second fixed coordinate (T ) A three-dimensional image related to first aid may be projected and displayed on the visualization device 110.

For example, if the patient is in cardiac arrest, a 3D image related to CPR is displayed on the second fixed coordinate (T) (see FIG. 8), or if the patient is in a state of leg fracture, it is displayed on the second fixed coordinate (T). A three-dimensional image of the bandaging method can be displayed (see Figure 9).

And the image registration unit 125 is configured to add a three-dimensional image of first aid to the image of the patient displayed through the visualization device 110 based on the first fixed coordinates (J0 to J16) and the second fixed coordinate (T). Match.

To match the 3D image of the first aid method to the patient's image, set the registration area where the 3D image will be registered to the patient's image based on the first fixed coordinates (J0~J16) and the second fixed coordinate (T). do. Then, the rotation angle, size change value, and position change value of the 3D image for the registration area are calculated. Then, the 3D image is transformed using the calculated rotation angle, size change value, and position change value of the 3D image. Then, the transformed 3D image is matched to the registration area. Field personnel can perform first aid on patients according to the registered 3D image.

In the case of an emergency situation where the patient is in cardiac arrest, the registration area is located at each pelvic joint on both sides of the patient at the first central coordinate (A), which is the center point of the first fixed coordinates (J5, J2) corresponding to each shoulder joint of the patient. The registration position (P) is 1/4 of the distance (d) to the second center coordinate (B), which is the center point of the corresponding first fixed coordinates (J13, J9) (see FIG. 10). And the 3D image is CPR, which is a treatment method for cardiac arrest, and the 3D image is a triangle with first fixed coordinates (J5, J2) and registration position (P) corresponding to both shoulder joints of the patient as vertices. It is aligned vertically at the matching position (P) (see FIGS. 11 and 12).

And in the case of an emergency situation where the patient's arm or leg is fractured, the registration area may be an area between the first fixed coordinates adjacent to both sides of the second fixed coordinate. The 3D image is a bandage treatment method for fractures, and the 3D image is overlapped and registered in the registration area by calculating the volume surface defined by the body standard. For example, when the patient's right arm is fractured, the registration area is an area between the first fixed coordinates (J6, J7) adjacent to both sides of the second fixed coordinate (T) (see FIG. 13). The 3D image is a bandage treatment method for fractures, and the 3D image is overlapped and registered in the registration area by calculating the volume surface defined by the body standard (see Figure 14).

Meanwhile, if the field agent determines that emergency treatment according to the manual is not possible due to the patient's condition and that medical support services from professional medical staff are necessary, the field agent requests remote medical support services (S500). When a request for remote medical support service is received from field personnel, the remote support server 200 searches for and connects to a medical staff that can be connected from the list of medical staff, and shares the patient's image through the visualization device 310 worn by the professional medical staff. And through communication with professional medical staff, field personnel can perform emergency treatment according to the patient's condition.

As such, the present invention is not limited to the described embodiments, and it is obvious to those skilled in the art that various modifications and changes can be made without departing from the spirit and scope of the present invention. Accordingly, such modifications or variations should be considered to fall within the scope of the claims of the present invention.

100: Field operation device 110: Visualization device
120: Portable controller 121: Image acquisition unit
123: Image coordinate acquisition unit 125: Image matching unit
127: Display unit 200: Remote support server
300: Remote medical support device

Claims (9)

(a) When an initial patient is discovered, an image of the patient's entire body with markers attached to each joint area is obtained from a distance using the camera of a visualization device worn by field personnel, and images of the patient are acquired in real time. steps;
(b) obtaining 3D coordinate values of the patient from the patient's image using an image coordinate acquisition unit and obtaining 3D coordinate values of the markers;
(c) When receiving input from field personnel that the patient's condition is capable of emergency treatment according to the manual, 3D mapping of the markers using an image registration unit so that emergency treatment is performed on the patient by field personnel. Each coordinate value is set as a first fixed coordinate, input from the field agent regarding the first aid location and first aid method is received on the real-time image of the patient displayed through a visualization device worn by the field agent, and the first aid location is displayed. A second fixed coordinate is set, a three-dimensional image related to the first aid method is projected on the second fixed coordinate and displayed through a visualization device worn by a field agent, and the real-time image of the patient is displayed as a three-dimensional image related to the first aid method. Registering the three-dimensional image; and
(d) If the patient's condition is such that emergency treatment according to the above manual is not possible and medical support services from professional medical staff are required, and medical support services from professional medical staff are requested from field personnel, a list of medical staff can be provided using a remote medical support device. Search for and connect to a medical staff that can be connected, and provide medical staff with information about the patient through a visualization device worn by the medical staff so that first aid according to the patient's condition is performed by field personnel who have received medical services from the medical staff through communication with the medical staff Including the step of sharing the video,
The three-dimensional image registration of the first aid method to the real-time image of the patient in step (c) is,
Setting a registration area in which the three-dimensional image of the first aid method will be matched to the real-time image of the patient based on the first fixed coordinates and the second fixed coordinates,
Calculate the rotation angle, size change value, and position change value of the three-dimensional image related to the first aid method for the registration area,
Transforming the three-dimensional image regarding the first aid method using the calculated rotation angle, size change value, and position change value of the three-dimensional image regarding the first aid method,
A remote medical support method for matching the three-dimensional image of the modified first aid method to the matching area.
According to paragraph 1,
In the case of an emergency situation in which the patient's arm or leg is fractured, the registration area is an area between the first fixed coordinates adjacent to both sides of the second fixed coordinate, and the three-dimensional image is a treatment method for fracture. It is a bandage method, and a remote medical support method in which the three-dimensional image is overlapped and registered in the registration area by calculating a volume surface defined by body standards.
According to paragraph 1,
In the case of an emergency situation where the patient is in cardiac arrest,
The registration area extends from the first central coordinate, which is the center point of the first fixed coordinates corresponding to both shoulder joints of the patient, to the second central coordinate, which is the center point of the first fixed coordinates, respectively, corresponding to both shoulder joints of the patient. The registration position is 1/4 of the distance, and the three-dimensional image is CPR, which is a treatment method for cardiac arrest, and the three-dimensional image is the first fixed coordinates and the registration position corresponding to both shoulder joints of the patient. A remote medical support method that is aligned to the matching position in a direction perpendicular to a triangle having as a vertex.
According to paragraph 1,
A remote medical support method where the marker is a QR code having data information about the joint area of the patient.
According to paragraph 1,
A remote medical support method in which, when the marker cannot be attached to the patient's joint area, the marker is attached to a floor adjacent to the patient's corresponding joint area. delete delete delete delete

Images